The twin that models everything and decides nothing

A digital twin of an MRO operation reproduces how the system behaves. It does not tell you how the system should act. Reinforcement learning is how that gap gets closed — and why the twin has to come first.

Most digital twins of an MRO operation are built to answer descriptive questions. What does demand for this material look like once you strip out the duplicate part numbers? How often does this pump actually fail, as opposed to how often someone raised a notification against it? If lead time on a vital spare stretches from 45 days to 96, where does the stockout land?

These are worth answering, and a twin answers them well. But notice what all of them have in common: they describe. The twin reproduces the behaviour of the system. The decisions — how much to hold, when to expedite, whether to repair — still come from somewhere else.

Where the logic actually lives

In almost every operation we look at, that somewhere else is a layer of hand-written rules. Reorder points set from a service-level table. An expedite threshold that fires when cover drops below a fortnight. A repair-versus-replace rule keyed to a percentage of replacement cost. Batching rules that hold work orders until a crew is scheduled. Criticality overrides that exempt vital spares from the ordinary policy.

Each of these is defensible on its own. Most were written by people who understood the operation, and most were correct at the time. Two things then happen. The rules are rarely revisited, because nobody owns them collectively. And they interact.

Interaction is the part that hurts

Take three rules that are individually sound. Safety stock is sized for a 95% service level. Expediting triggers when projected cover falls under fourteen days. Purchase orders are consolidated weekly to earn a freight discount.

Now run them together through a period when one supplier slips. Consolidation delays the replenishment by up to six days, which pushes projected cover under the expedite threshold, which triggers premium freight on a line that would have arrived in time anyway. The expedite consumes budget, so the next genuine emergency is questioned. The safety stock that was sized assuming stable lead times is now absorbing variance it was never sized for, so it depletes, so the reorder point fires early, so more cash goes into a material that is not short. Nobody wrote a bad rule. The behaviour is bad regardless.

This is the characteristic failure of policy-by-rule in a coupled system. The rules are local and the consequences are global. You cannot debug it by reading the rules, because the fault is not in any of them.

What reinforcement learning actually changes

Reinforcement learning does not produce a better forecast. It produces a policy: a mapping from the state the operation is in to the action it should take. It learns that mapping by acting, observing the consequence, and adjusting — millions of times, across conditions far more varied than any single year of operating history contains.

For an MRO operation, the three pieces look like this.

State

Stock on hand and on order by location, open work orders and their criticality, asset condition, supplier lead-time performance to date, repair-loop occupancy, budget consumed against period.

Action

Order or hold. Order how much, from whom, to which store. Expedite or wait. Repair, replace, or cannibalise. Move stock between plants. Release a work order now or batch it with next week.

Reward

What the operation is actually trying to buy: uptime on the assets that matter, less cash tied up in slow-moving stock, fewer emergency freight bills, less risk carried on a sole-sourced vital spare.

The important word is consequence. A rules engine cannot learn that consolidating this particular order class tends to trigger expensive expediting three weeks later, because a rule has no memory of what followed it. A learned policy is built out of exactly that: the delayed, indirect cost of an action that looked cheap at the time.

The reward is the hard part, and it is not a modelling problem

It is tempting to treat the reward function as a technical detail. It is the opposite. The reward is where the operation states, in numbers, what it is actually optimising — and most organisations have never had to write that down without ambiguity.

Uptime and working capital pull against each other. Risk on a sole-sourced vital spare is real but does not appear in any monthly report until it materialises. If the reward counts only inventory value, the policy will learn to run the operation dangerously lean and be congratulated for it. If it counts only service, it will learn to hoard. If it counts service and cash but not the criticality of the asset behind the demand, it will protect the cheap fast-movers and let a vital spare go short, because that trade improves the score.

Getting this right is a conversation between maintenance, supply and finance, and it usually surfaces disagreements that predate the project. That is a feature. A reward function nobody will sign is a decision nobody should be automating.

Why the twin has to come first

You cannot train a policy on the live operation. Learning requires exploration, and exploration in production means deliberately taking actions you expect to be wrong, on real assets, to find out how wrong. No plant manager will fund that, correctly.

So the twin stops being a reporting artefact and becomes the training environment. That raises the bar on what it has to get right. Point estimates are no longer enough: the policy will learn to exploit any part of the model that is smoother than reality. The twin needs the shape of the distributions — lead times with their real tails, failure processes fitted to the actual history rather than assumed exponential, repair loops with their true turnaround variance, and the operational rules that constrain what is physically possible.

This is also the honest limit of the method. A policy trained against a twin is only as good as the twin's fidelity, and a twin is only as good as the master data underneath it. Duplicate part numbers, misattributed failures and free-text histories that nobody has resolved will all be faithfully learned. This is why the sequencing matters — audit, then model, then optimise. Reversing it produces confident nonsense.

Where it pays first

Not everywhere at once. The decisions worth handing over early share a shape: they repeat often, their consequences are delayed enough that people judge them badly, and the cost of being wrong is measurable.

• Expediting. High frequency, immediate visible cost, delayed invisible benefit. Human judgement here is systematically biased toward acting.
• Repairable rotables. Repair, replace, or hold, with a loop whose occupancy affects every future decision. Rules handle the loop poorly.
• Multi-echelon positioning. Which store holds the unit, when a plant three hours away also needs cover. Rules tend to answer this per-site, which is the wrong scope.
• Order consolidation. The trade between freight discount and delay is exactly the kind of small, repeated, coupled decision that compounds.

Guardrails, before anyone asks

A learned policy is not a black box you point at the ERP. Three constraints make it governable, and we would not run it without them.

Constrain the action space. The policy proposes within bounds the operation sets — it cannot order beyond a budget envelope, cannot take a vital spare below a floor, cannot skip a compliance-driven replacement. Most of the catastrophic behaviours people fear are simply excluded by construction.

Evaluate offline before anything is live. Held-out periods, replayed against the historical record: what would this policy have done, and what would it have cost? A policy that cannot beat the incumbent rules on history it has never seen does not proceed.

Run it in shadow. For a period, the policy recommends and the existing process decides, and the two are compared in the open. This is slower than switching over, and it is the only version of the transition anyone should accept.

What this is not

It is not autonomy, and it does not remove planners. What it removes is the expectation that a planner should hold the interaction effects of forty coupled rules in their head while working a queue. The policy handles the repetitive, coupled, consequence-delayed decisions. People handle the exceptions, the supplier relationships and the judgement calls that no reward function should be trusted to encode.

The twin was always the expensive part to build. Using it only to describe the operation, when it is capable of teaching the operation how to act, is where most of the value is currently being left.