Mestric logo

Sharing is caring

Learn with us! We want to give you an easy-to-follow guide to manufacturing processes and show you the best optimization process.
Section dividerSection divider
Engineer checking sensor on production machine
October 6, 2026

Ship Production Anomaly Detection: MES, Edge Models F1 0.96–1.0

The most effective approach to odkrivanje anomalij v proizvodnji is real-time monitoring tightly integrated with a Manufacturing Execution System, using hybrid statistical and machine-learning models validated against ISO 22400 KPIs. Teams that report F1 and AUC scores alongside OEE and first-pass yield move faster from pilot to production, because an MES gives the data capture and dashboards that modelling alone cannot.


TL;DR:

  • Anomaly detection models should be chosen based on data volume, labeling resources, and latency needs, ranging from control charts to transfer learning approaches.
  • Proper sensor data collection requires accurate timestamping and product linkage, with noise filtering and feature extraction key to reliable models.
  • Deployment challenges include noise robustness, threshold calibration, scheduled retraining, and seamless MES integration for effective real-time actions.
  • Model effectiveness is best measured through KPIs like F1 score, AUC, OEE, and scrap ratios, with a focus on translating detection into operational improvements.
  • Starting with high-impact, high-frequency lines and involving operators early accelerates progress and enhances threshold tuning accuracy.

Mestric
mestric.com
Turn Production Data Into Action
Mestric connects with manufacturing equipment to track performance, quality, downtime, and costs in real time for informed decisions.

Explore Mestric MES

Table of Contents

1. What counts as an anomaly on your production line?

Not every irregular reading deserves an alarm, and not every alarm deserves a supervisor’s attention. Point anomalies are single out-of-range readings, a torque spike on one cycle, a temperature dip on a single part. Contextual anomalies only look wrong given the surrounding conditions: a vibration level that is normal at full speed but abnormal during a slow-start sequence. Collective or batch anomalies appear across a sequence of parts, often tool wear creeping in over hundreds of cycles or a material-batch drift that shows up gradually rather than as a single spike.

Which type matters most depends on your line:

  • Point anomalies demand real-time detection when they signal safety risks or scrap, such as a press overload.

  • Contextual anomalies need models that understand machine state, not just raw thresholds.

  • Batch anomalies are often better caught with periodic statistical review than instant alerts, since a single late part rarely proves a trend.

Recipe errors sit in their own category. A changeover that silently reverts to the wrong parameter set behaves like a sudden step change rather than drift, and it needs a different detection strategy again.

2. Data and sensors: building a foundation worth trusting

Anomaly models are only as good as the signals feeding them. Most production lines already generate usable data: vibration, current and torque, temperature, pressure, cycle time and, increasingly, images from inline cameras. The question is rarely whether a sensor exists, but whether its readings are timestamped, serialised to a specific product, and aligned with the machine state at the moment of capture.

ISO/TR 22400-10 ties KPI calculation directly to accurate event logging, so a missing timestamp or an unlinked serial number does not just hurt a model, it breaks the business reporting built on top of it.

Before modelling, expect to spend real effort on:

  • Filtering noisy raw signals without erasing the deviation you are trying to detect.
  • Windowing continuous signals into cycle-aligned segments that match production events.
  • Feature extraction, turning raw vibration or current traces into statistics a model can use.
  • Handling gaps, deciding whether a missing reading is interpolated, flagged or dropped.

Pro Tip: Log every sensor dropout as its own event rather than silently filling the gap. A pattern of dropouts is often an early warning in itself.

3. Choosing a modelling approach that matches your constraints

There is no single best model, only the model that fits your data volume, labelling budget and latency requirements.

  1. Statistical control charts remain the right starting point when data is limited and interpretability matters. They need no training data beyond a stable baseline and are easy to explain to operators.
  2. Normal-only models, such as autoencoders or one-class classifiers, learn what healthy production looks like and flag deviations without needing labelled failures, useful where anomalies are rare and poorly documented.
  3. Supervised classifiers earn their complexity only when you have enough labelled failure examples to train and validate them properly, which is rare on most shop floors.
  4. Transfer learning and recipe-based methods address the data-scarcity problem directly. Recipe-based and adaptable learning approaches in injection moulding detected many more defects than an integrated model across a multi-thousand-cavity dataset, because recipe-specific models capture process variation that a single generalised model misses. Knowledge-integrated transfer learning built on multi-scale CNNs and graph attention reports average F1 and AUC scores above 0.9 when transferring across working conditions, with performance gains varying by dataset.
  5. Edge-optimised models trade some accuracy for latency. VARADE++ demonstrates a balance of accuracy and low-latency inference on public benchmarks and a sensorised pilot line, showing that edge inference can be efficient and effective compared to cloud-based methods.

4. Implementation challenges and best practices for live deployment

Models that perform well on historical data often struggle once deployed, mainly because production conditions shift and noise behaves differently line to line.

  • Physically-informed features outperform purely data-driven ones in noisy settings; research on in-production anomaly detection found that integrating machine physics or state logic into feature design raises robustness where raw signal thresholds fail.
  • Threshold selection deserves as much attention as model choice. A fixed threshold tuned on one shift often misfires on another, so conformal calibration or rolling baselines reduce false positives better than a static cut-off.
  • Retraining cadence should be scheduled, not reactive. Logging every model version against the production conditions it was trained on makes drift easier to diagnose later.
  • Integration patterns matter as much as the model itself: detection events need a clear path into your MES or SCADA system so an alert becomes an operator action, not a dashboard nobody checks.

Physically-informed Bayesian feature optimisation shows that semi-supervised monitoring built on these principles stays robust to hyperparameter changes, which matters when a line rarely sits still long enough for perfect tuning.

5. Measuring success: metrics and KPIs that connect the two

A model’s performance only matters once it is translated into terms a plant manager already tracks. Precision tells you how many flagged anomalies were real; recall tells you how many real anomalies you caught; F1 balances the two when both false alarms and missed defects carry a cost; AUC summarises performance across thresholds.

Advanced process anomaly-detection models have reached F1 scores of 0.96 to 1.0 in controlled industrial experiments on repetitive processes, with transfer-learning frameworks reporting F1 increases of roughly 25 to 26% over baseline methods in multi-condition monitoring. Numbers like these only mean something when mapped to OEE, first-pass yield and scrap ratios, the elements ISO 22400 defines for Manufacturing Operations Management. A short pilot should report both sets of figures side by side: model metrics and business KPIs, so a decision-maker can judge impact without needing to interpret a confusion matrix.

5. Measuring success: metrics and KPIs that connect the two — overview diagram

6. How an MES turns detection into action on the shop floor

A model that flags an anomaly in isolation changes nothing. The value appears once detection connects to a system operators already use. A practical architecture runs edge capture into lightweight inference, then streams events into an MES for central analysis and dashboarding.

A capable MES for this purpose needs to offer:

  • Live KPI dashboards that show detected anomalies alongside OEE and quality figures in one view.
  • Direct equipment connectivity so readings and flags arrive without manual entry.
  • Automated, timestamped data capture that satisfies the traceability requirements behind ISO-aligned reporting.

Closing that loop shortens the gap between a detected deviation and a corrective action, and it removes a layer of manual transcription that tends to introduce its own errors.

7. Where to start and how to scale an anomaly detection project

Start on the highest-frequency, highest-impact line you have, with a control chart or a normal-only model rather than a complex classifier you cannot yet validate. Bring operators in from day one: their judgement on false alarms shapes thresholds faster than any amount of offline tuning. Treat the first deployment as one iteration among several, not a finished system.

— Andraž

How Mestric™ MES supports anomaly detection in practice

Once a detection approach proves itself, the harder problem is usually operational: getting flagged events in front of the right person, on the right machine, fast enough to matter. Mestric™ MES connects directly to shop-floor equipment and captures timestamped data automatically, so the traceability that anomaly detection and ISO-aligned KPI reporting both depend on is already in place rather than bolted on afterwards.

Mestric

Live KPI dashboards bring detection events into the same view as OEE, downtime and quality figures, which shortens the path from a flagged deviation to a corrective action on the floor. For teams comparing options, our piece on MES against traditional manufacturing approaches lays out where an integrated system earns its place over spreadsheets and standalone sensors.

  • We connect directly to existing equipment rather than requiring a separate sensor layer.
  • We display anomaly and quality data in the same dashboards production managers already check.
  • We reduce manual data entry, which cuts a common source of error in reporting.

If you are ready to see how this looks on a real production line, you can request a demonstration of Mestric™.

FAQ

What is the difference between real-time and batch anomaly detection?

Real-time detection flags deviations as they happen, which suits safety-critical or high-cost defects where delay is expensive. Batch or post-process inspection reviews data after a run completes, which works well for slower-forming issues like gradual material drift where instant alerts would add noise without adding value.

Which sensors matter most for production anomaly detection?

Vibration, current or torque, temperature, pressure and cycle time cover most mechanical and thermal failure modes, with inline images added where visual defects matter. The right mix depends on your failure history more than on any general rule.

How do F1 and AUC scores translate into shop-floor terms?

F1 balances false alarms against missed defects, while AUC shows how a model performs across different alert thresholds. Research on knowledge-integrated transfer learning reports an average F1 of 0.9226 and AUC of 0.9353 across working conditions, figures that should be read alongside OEE and scrap-rate changes, not instead of them.

Can anomaly detection work with very little labelled failure data?

Yes. Recipe-based and transfer-learning methods are built for exactly this problem. Recipe-based adaptable learning in injection moulding detected far more real defects than a generalised model trained without that structure, which makes it a practical route when labelled failures are scarce.

Does anomaly detection replace scheduled maintenance?

No, it complements it. Detected anomalies, particularly gradual ones like rising vibration or torque drift, help shift maintenance timing from a fixed schedule to one based on actual machine condition, which tends to catch wear before it causes a breakdown.

Sources

ISO 22400 defines the KPI framework behind OEE, first-pass yield and MTBF/MTTR calculations used throughout this guide. The VARADE++ paper covers edge-friendly real-time detection, the MDPI Sensors study covers recipe-based learning for injection moulding, and the Applied Sciences benchmark review covers transfer-learning performance across multi-condition factories. For operational controls that sit alongside these KPIs, see this bizSAFE Star implementation guide.


crossmenu