AI & Lean Six Sigma
15 September 2026

Control Charts for AI: Using SPC to Monitor Machine Learning Model Drift

Share on:

Machine learning models don’t fail the way traditional software fails. They don’t throw exceptions or crash. They keep producing predictions, and the predictions keep looking plausible, while the underlying accuracy quietly erodes. A McKinsey review of AI risk practice notes that model monitoring in production needs to track data drift, model metrics, and bias metrics in outcomes as a continuous discipline rather than a one-time validation step.
That is a control problem, and it is exactly the kind of problem Lean Six Sigma practitioners have been solving with control charts for decades.

Data Drift and Concept Drift Are Two New Sources of Special-Cause Variation

In a manufacturing process, variation comes from machines, materials, methods, and people. In a deployed model, variation comes from the data feeding it.
A quality manager doing root cause analysis on a degraded model needs to separate these input-side shifts (data drift) from cases where the relationship between inputs and the correct output itself has changed (concept drift). Both are process variation. Neither shows up in a code review.

The Mechanics: Center Line, Control Limits, and the Western Electric Rules

A control chart applied to a model metric works the same way it works on a production line.
Applied to a model’s daily accuracy or error rate, these rules catch gradual degradation that a simple accuracy threshold would miss.

A Clinical Case Worth Studying

A 2024 peer-reviewed framework built for radiological AI systems shows what this looks like in practice. Researchers combined SPC with machine learning to catch
That is an average-run-length problem, the same metric used to judge how quickly a traditional control chart flags a shifted process.

Machine-Learning-Enhanced Charts Are Starting to Replace Simple Thresholds

SPC for ML drift is not staying static either.
, judging performance by average run length. This matters for practitioners choosing between a standard process capability approach and something more adaptive, an exercise no different from the tool-selection decisions taught in any Lean Six Sigma tool selection module. A more recent 2025 paper on bootstrapped control limits for concept drift extends this further, building control charts specifically around model prediction scores rather than raw feature statistics, which avoids the false-alarm problem that plain sigma thresholds create when data isn’t normally distributed.

Folding Model Monitoring into a DMAIC Control Plan

None of this replaces the discipline of DMAIC. A model monitoring program earns its place in the Control phase the same way any stabilized process does: define the metric (accuracy, error rate, a feature’s mean or proportion), set limits from a stable baseline period, and assign ownership for investigating breaches. The Databricks engineering team, describing production ML pipelines, puts it plainly:
, which is a governance description of exactly what a Black Belt would call moving from measurement to control.

Who Owns the Chart Once the Model Ships

Someone has to own the breach investigation, and that ownership question is where most model monitoring programs quietly fail. A Black Belt already knows how to write an escalation path for an out-of-control signal on a machined part; the same skill transfers to a model accuracy chart, provided the data science team documents baseline behavior clearly enough for someone outside the model-building team to read it. Leadership sponsorship matters here specifically because model drift alerts are easy to dismiss as noise until a credit, fraud, or clinical decision goes wrong in production. Building that ownership into a project charter, with a named process owner and a documented response threshold, is standard DMAIC practice applied to a newer kind of process.

Sources

enENarARdeDEesESfrFRitITnlNLtrTR
No results