Industrial resilience: recovery has a capacity limit

The war on ordinary machines · Part 7 of 7

Backups, spare equipment and an on-call contract only preserve operating hours when people can put them to work. The final part of this series examines recovery capacity, shared dependencies and the economics of keeping a usable fallback ready.

On 26 March 2019, a week after the cyberattack on aluminium producer Norsk Hydro, a single company update described two very different industrial situations. For three business units within Extruded Solutions, Hydro reported an aggregate production rate of 70–80%, without detailing the denominator. A fourth, Building Systems, remained almost at a standstill. Elsewhere, production was running normally, sometimes with more manual work. Fallback capacity already had a geography, a workforce and a limit. 1

On the day of the attack, 19 March, Hydro had announced that it was isolating its operations and switching to manual procedures wherever possible. It also reported stoppages caused by an inability to connect to production systems. The contrast between operations raises a practical question behind the broad promise of resilience: which functions remain available when the usual computing environment disappears? 2

This final instalment examines the resources that allow a business to keep going. The earlier articles followed machine commands, remote access, evidence, losses and regulation. The remaining question is how to fund a fallback that can actually be used. Public documents explain the mechanisms. An entirely hypothetical numerical case then shows how the number of available recovery teams changes the cost of an outage. Its inputs make the arithmetic inspectable; they are neither market prices nor an attack forecast.

Method and scope

Company disclosures dated 26 and 19 March 2019. The aggregate 70–80% rate covers Extrusion Europe, Extrusion North America and Precision Tubing; its detailed denominator is not provided. “Normal” and “almost at a standstill” are company assessments; no 100% or 0% is imputed. A map of reported states, not an avoided-loss estimate or a reconstruction of internal architecture.

Start with the function that needs to survive

In its retrospective account, updated on 15 May 2024, Hydro puts the total cost of the attack at approximately NOK 800 million. It describes labour-intensive workarounds. This remains the affected company’s estimate for one event at its own scale. The figure conveys the size of the disruption; the separate contributions of architecture, staff and manual operation to avoided losses remain unknown. 3

The story could easily become an argument for going back to paper. A more useful investigation asks what makes temporary operation feasible. For a refrigerated warehouse, for example, that would mean examining temperature control, monitoring, decisions about stock and available staff together. This is an analytical example, not a reconstruction of an attacked installation or advice on permissible temperatures.

The UK’s National Cyber Security Centre (NCSC) describes the target as minimum viable operations: a level at which an organisation can operate safely, meet its obligations and retain the trust of partners. It is a definition built around the service being delivered. An application may be accessible again while the operation that depends on it remains too fragile to resume. 4

An operator’s first useful expenditure might therefore fund the work of specifying which function must continue, for how long, using which resources, and under what conditions it must be stopped. Those answers give the budget a purpose. They help distinguish the case for stronger protection, local autonomy or faster restoration.

The industrial security guide from the US National Institute of Standards and Technology (NIST) follows this logic by calling for recovery objectives to be set before selecting the means, including spare inventory where those objectives justify it. This article uses the final September 2023 edition. A notice dated 21 September 2026 points to an initial Revision 4 draft, with comments due by 30 November. Its draft status is kept separate. 5 6

Two devices can share one point of failure

Redundancy provides an additional resource capable of taking over a function. An isolated hardware failure and the compromise of a shared administrative route test that arrangement in different ways. NIST explicitly observes that redundancy in supervisory control systems may be insufficient against a malicious attack. 5

Consider a hypothetical arrangement in which two control devices are administered through the same account and environment. Duplicating the devices leaves that shared dependency in place. Assessing the fallback requires examining what would happen to both devices if that environment were lost or compromised, then identifying which functions would survive. The diagram below asks that question. It represents no particular product or victim.

Separating dependencies brings costs of its own. Distinct access routes need management, configurations need to remain consistent, and staff need to understand the recovery options. Technical diversity can make maintenance harder. The investment decision should identify which common causes of failure it addresses and how much additional work the design creates.

Guidance published by the NCSC on 1 October 2026 connects limiting the impact of compromise with network segmentation, protected backups and control of administrative privileges. These measures describe ways to contain damage. Their effectiveness in a particular installation still needs local evidence. 7

Method and scope

Teaching diagram based on NIST and NCSC mechanisms. It depicts no real network, product or operating protocol. The recovery branch identifies dependencies to demonstrate, not a certified design. Actual separation, surviving functions and safety conditions require local assessment.

Isolation poses a related test. A plant may rely on a remote service for a function whose loss has physical consequences. The NCSC recommends an isolation plan linked to service continuity, tested and designed for multi-site situations, with suppliers included in the planning. 8

The investment can buy an operational choice: suspend a compromised connection while retaining necessary functions, or arrange a controlled shutdown. The business case can specify the resources needed to preserve that choice and the conditions for using them. Its value depends on the extent of disruption it can contain and the operating conditions it preserves, rather than the number of devices installed.

Recovery has a capacity limit

Recommendation R76 in the November 2025 industrial security guide from ANSSI, France’s cybersecurity agency, calls for recovery priorities to be agreed with management. It recognises that several sites may be affected together and may have different levels of criticality. This class C1 recommendation exposes a central constraint: restoration needs can arrive simultaneously, while available resources have to be allocated. 9

To make that constraint visible, take six fictional sites, each losing the same function. The safety conditions required for intervention are assumed to have been met. Two teams become available two hours after the interruption begins. Each restoration job, including validation, takes four hours. Parts and configurations are ready, sites have equal priority, and travel between jobs takes no time in the model. These assumptions deliberately simplify the problem.

With two teams, two sites are restored at hour six, another two at hour ten, and the last pair at hour fourteen. Adding those durations gives 60 site-hours of unavailability. One hour without the function at one site is one site-hour. This unit measures cumulative interruption; the elapsed time until the last site is restored is fourteen hours.

A third team, available at the same time and equally productive, restores three sites at hour six and three at hour ten. The total falls to 48 site-hours. The improvement is twelve site-hours. The first restoration still occurs at hour six, while the final restoration moves from hour fourteen to hour ten. The extra team changes waiting time, with different benefits for sites at different positions in the queue.

Method and scope

Entirely fictional l0g calculation: six site functions unavailable from t = 0; two-hour mobilisation; four-hour complete jobs, including validation. Identical equal-priority sites and interchangeable teams; safety prerequisites met before work, parts, configurations and trusted tools ready; no travel, handover delays or shared bottleneck. Two-team finish times: 6, 6, 10, 10, 14, 14 h, totalling 60 site-hours. Three teams: 6, 6, 6, 10, 10, 10 h, totalling 48. Saving: 12 site-hours; first restoration unchanged at 6 h, final restoration advanced from 14 to 10 h. Blocks show work only; totals include mobilisation and waiting. ANSSI R76 motivates the multi-site question and supplies none of these durations. Safety and service criticality take priority over the fictional equal ordering.

The calculation exposes a straightforward contractual question. A business paying for on-call support needs to understand which resources would be available if several customers called at once. The supplier’s customer base, priority commitments and available skills become relevant evidence. The NCSC’s continuity guidance includes suppliers and asks organisations to consider an abrupt loss of their services. 10

The model becomes less favourable when a prerequisite is missing. A shared investigation may block every team; only one person may be authorised to validate restoration; a component or trusted recovery environment may be unavailable. Useful capacity is then constrained by that bottleneck. Funding another team should be compared with funding the prerequisite that would allow the teams already present to work.

For a real organisation, every duration would require a measurement or a documented assumption. Sites would need individual treatment, and supplier-unavailability scenarios would need examination. A priority driven by human safety or an essential service takes precedence over the convenient ordering used here.

Saved operating hours and the break-even threshold

Continue with this hypothetical case only. Suppose the arrangement providing access to the third team costs €30,000 to establish and €6,000 a year to maintain. Over five years, the incremental budget is €60,000. These are teaching inputs, unrelated to a quotation or pay agreement. They stand for an assumed available capability whose actual procurement and staffing would need investigation.

For the exercise, each site-hour of outage causes €2,500 of permanently lost contribution margin. This is the revenue that would have contributed to continuing fixed costs and profit after deducting the variable costs avoided. Sales merely delayed and later recovered would require a different calculation. The twelve site-hours saved therefore preserve €30,000 per incident of the specified kind.

Two identical incidents over five years arithmetically offset the €60,000 cost. This conditional break-even threshold describes two comparable events; their future frequency requires a separate estimate. Physical damage, insurance recoveries, tax and effects on other firms sit outside the model. Amounts are constant, with no inflation or discounting and zero assumed residual value. Incident-time expenditure is assumed identical under both arrangements, so it has no effect on the calculated difference.

Method and scope

All values are fictional. Cost = 30,000 + 5 × 6,000 = €60,000. Conditional benefit = 12 site-hours × permanently lost contribution per site-hour. Threshold = cost / benefit. Sensitivity: €1,000, €2,500 and €5,000 per site-hour yield 5, 2 and 1 comparable events in five years. Constant euros assumed; no discounting, tax, residual value or estimated event probability. Incident-time expenses assumed identical between options. Physical damage, insurance and third-party effects excluded. Safety is a prerequisite.

Sensitivity is immediate. At €1,000 of lost contribution per site-hour, the improvement is worth €12,000 per incident and the threshold rises to five events over the period. At €5,000, one event is sufficient in the model. These are three economic assumptions, not a confidence interval or industry estimate. They show why identical recovery equipment can have very different value depending on the function it supports.

Management can use the threshold to ask for the missing evidence: the value of output that would genuinely be irrecoverable, contractually available capacity, tested mobilisation time and the period over which the assumptions remain credible. Any probability introduced later needs its own basis, limitations and sensitivity analysis. A convenient round number would turn an incomplete understanding into an apparently precise return.

Financial comparison operates within the set of options compatible with safety and applicable obligations. Some spending serves protective or continuity requirements whose value extends beyond the firm’s own earnings. The model helps compare resources; qualified people still have to establish safe operating conditions.

Maintaining capacity while it waits

Initial expenditure is the easiest part to display. A replacement component occupies a place in inventory. Its future usefulness also depends on compatibility, configuration and the ability to commission it. NIST includes installation media, licence keys and configuration information among the items needed for usable backups. The conditions in which spare equipment is stored also matter to recovery. 5

Someone must own that continuity of knowledge. Every system change raises a question about which recovery resources and evidence need review. Reserving human capacity similarly requires retaining skills and access to useful information. ANSSI specifically addresses the transfer of knowledge when people leave or change roles. 9

Recurring expenditure can maintain a readiness that is largely invisible during normal operation: training time, checks on spares, exercise preparation or review of a supplier commitment. Assessing it means linking each item of spending to the capability it keeps available. A procurement list alone would provide little information about how that capability changes over time.

The NCSC recommends keeping recovery information secure and separate from primary systems, and documenting infrastructure dependencies. It calls for clarity about incident response and system reconstruction, which may be delivered under different service arrangements. 11

That distinction provides a useful way to read an invoice. Is the organisation buying available expertise, reconstruction work, a parts reserve or evidence that a working combination can be restored? Several may be needed, but they deliver different outcomes. Acceptance should connect the purchased service with the function being funded and the conditions under which it was tested.

A useful rehearsal produces bounded, dated evidence

The NCSC distinguishes tabletop exercises, which examine roles and decisions, from technical testing that reveals practical restoration difficulties. They answer complementary questions. A discussion can expose missing decision-making authority; a test can reveal that a particular step takes longer than expected. 10

For an industrial installation, such verification requires an authorised scope, a prior risk assessment and conditions for stopping the exercise. The approach proposed here is documentary: prepare questions with the operator, control engineers and safety specialists through the authorised process, preserving its safety and continuity requirements.

Useful evidence records the version tested, the function restored, the people and resources actually present, elapsed time and the limitations encountered. It also records favourable test conditions: parts already on site, a prepared test environment or advance notice to a supplier. These details help establish what the result can reasonably imply for a different crisis.

Method and scope

Editorial governance diagram based on ANSSI and NCSC recommendations. A tabletop exercise does not by itself establish a restoration time. A technical test needs process-specific planning, competent authorisation, safety conditions and a rollback route. The diagram prescribes no intervention or technical threshold. Changes may require impact assessment, followed by a targeted test where appropriate.

Human practice needs its own place. The NCSC’s staff-preparation guidance recommends exercises and alternative communications when familiar tools are unavailable. 12 The budget question then becomes specific: has time for that preparation been allocated, and which teams have actually received it?

The strongest argument against continually adding recovery resources is the complexity they create. An unfamiliar fallback may delay decisions; a security change may disturb the process; a second operating arrangement requires maintenance of its own. ANSSI asks organisations to ensure cyber controls preserve degraded operating modes. The NCSC also notes that containment decisions can themselves cause disruption. 9 13

A test must therefore be allowed to disappoint. If an additional intervention does little for the priority function, money can be redirected towards the delay, skill or dependency that matters more. That learning loop gives testing value even when the originally proposed solution becomes less attractive.

Fund the capability, then check that it still exists

Across an industry, the same question applies to specialists, spare parts and shared services. Pooled capacity can lower individual preparation costs; its sizing needs to be tested against simultaneous demand. This is an inference from the mechanism examined here, not a claim to have measured a national shortage of technicians or equipment.

A buyer can ask how priorities will be set, which resources are reserved and what remains available if the provider is affected too. A funder can ask which improvement has been observed and under what conditions. An operator can document the level of service it can sustain and how long its people can sustain it. These questions connect the contract to the physical function that needs protection.

Across these seven parts, a remote command connects equipment with the people, service providers and rules that shape its operation. An incident leaves evidence and losses whose consequences require their own assessment. A resilience budget funds a specific capability in a specific setting. Its quality rests on the service preserved and the recovery demonstrated, with a verification date and the resources needed to do it again.

The numerical case, spelled out

l0g teaching model: six site functions unavailable from hour zero; teams mobilised at hour two; four hours per complete job; independent work, interchangeable teams, material resources ready and zero travel time. Two teams finish at 6, 6, 10, 10, 14 and 14 hours; three finish at 6, 6, 6, 10, 10 and 10. Totals: 60 and 48 site-hours. Five-year incremental cost: €30,000 + 5 × €6,000 = €60,000. Benefit per event: 12 × €2,500 = €30,000. Threshold: €60,000 ÷ €30,000 = two comparable events. Attack frequency remains unknown, and the threshold preserves that uncertainty.

Research scope

Documentary research checked on 3 November 2026. The Hydro case retains the dates and scopes of the company’s 2019 statements and its account updated in 2024. The contribution of individual measures to avoided losses remains unknown. Agency recommendations retain their status and context. The model, its durations and every monetary input are wholly hypothetical; the assumptions and arithmetic appear above. Private contracts, site audits and live installations remain outside the research.

Further reading

• Part 1: supermarket freezers become a target.
• Part 2: who actually controls the machines?.
• Part 3: remote maintenance and contractor access.
• Part 4: industrial sabotage and attribution.
• Part 5: stock loss, insurance and cash.
• Part 6: European rules, support and installed equipment.

Sources

Documents checked on 3 November 2026. The notices identify relevant passages, dates and the scope of each source.

1. [1] · Norsk Hydro · 26 March 2019 · Checked on 3 November 2026Update on cyber attack March 26

   Reported output of 70–80% across Extrusion Europe, Extrusion North America and Precision Tubing; Building Systems almost at a standstill. Status of other operations and manual work.

   Passages and scope

   Operational status: three Extruded Solutions units; normal output elsewhere, sometimes with more manual work. Company disclosure. The detailed capacity denominator and site tonnages are not provided. The rate excludes Building Systems; no percentage is imputed to that unit. Full official page read directly on 3 November 2026.

2. [2] · Norsk Hydro · 19 March 2019 · Checked on 3 November 2026Update: Hydro subject to cyber attack

   Reported isolation of operations and manual work where possible; inability to connect to production systems.

   Passages and scope

   Opening statement and operational status, especially Extruded Solutions and Rolled Products. Day-of-incident disclosure. No internal architecture or causal estimate of a control’s effectiveness is supplied. Full official page read directly on 3 November 2026.

3. [3] · Norsk Hydro · Updated 15 May 2024; March 2019 incident · Checked on 3 November 2026Cyber-attack on Hydro

   Retrospective estimate of around NOK 800 million and labour-intensive manual workarounds.

   Passages and scope

   Cyber-attack on Hydro in brief; total-cost estimate and displayed update date. Company estimate, without avoided-loss measurement or a comparable prevention budget. No net-of-insurance calculation or euro conversion. Full official page read directly on 3 November 2026.

4. [4] · NCSC, United Kingdom · Published 28 July 2026; reviewed 1 October 2026 · Checked on 3 November 2026Recovering from a highly disruptive cyber attack: 2. Recovery and ongoing investigations

   Minimum viable operations, recovery organisation, and workarounds requiring review and reassessment.

   Passages and scope

   §2.1, MVO definition; §2.2.4, workarounds, security review, data reconciliation and review dates. General UK guidance. Minimum operations depend on the organisation; no universal throughput or avoided-loss rate is established. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.

5. [5] · National Institute of Standards and Technology (NIST) · Final publication, September 2023 · Checked on 3 November 2026Guide to Operational Technology Security, NIST SP 800-82 Revision 3

   Redundancy and common dependencies, usable backups, recovery objectives, reconstitution and component availability. Recovery objectives can justify spare inventory where other means are insufficient. Human and environmental safety take precedence over restarting operations.

   Passages and scope

   Page 12; §6.2.4.3, pp.112–113; §6.2.4.5, pp.114–115; Appendix F, CP-9 and CP-10, pp.250–251; SR-5(1), p.297. Corresponding PDF pages: 29, 129–132, 267–268 and 314. Guidance used voluntarily by nongovernmental organisations. Duplication does not establish independence against malicious compromise. Equipment, testing and restored state need process-specific assessment; no universal recovery duration, operating procedure or avoided-loss estimate is established. Revision 4 remains a separate draft. Official 316-page PDF downloaded and cited passages read. Printed page 114 rendered and inspected; version and status cross-checked against the current NIST publication page.

6. [6] · NIST, Computer Security Resource Center · September 2023 final-publication page; planning note dated 21 September 2026 · Checked on 3 November 2026Official publication status of NIST SP 800-82 Revision 3

   The page distinguishes final Revision 3 from the initial public draft of Revision 4, published on 21 September 2026 with comments due on 30 November 2026.

   Passages and scope

   Date Published, Planning Note (09/21/2026) and Document History; linked Revision 4 Initial Public Draft page, whose displayed dates were also read. This source establishes documentary status as reviewed. Publishing an initial draft does not make it final or legally binding. No future comment outcome or revision is assumed. Official page read directly, together with the linked initial Revision 4 draft page.

7. [7] · NCSC, United Kingdom · Published and reviewed 1 October 2026 · Checked on 3 November 2026Reducing the risk of highly disruptive cyber attacks: 3. Limiting the impact of a successful compromise

   Segmentation, backup protection and privilege controls to constrain attacker movement and damage.

   Passages and scope

   §§3.1–3.4, especially segmentation (§3.2) and backups and administrative access (§3.4). Recommended mechanisms whose effectiveness needs contextual validation. No risk-reduction percentage is assigned. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.

8. [8] · NCSC, United Kingdom · Displayed publication 14 January 2026; displayed review 18 March 2024; version 1.0 · Checked on 3 November 2026Secure connectivity principles for OT: Principle 8, Establish an isolation plan

   Isolation planning linked to continuity, testing, independent functions where possible, suppliers and multi-site scenarios.

   Passages and scope

   Opening paragraphs, multi-site dependencies and critical flows; the three isolation strategies. Guidance requiring assessment of physical consequences and applicable commitments. The displayed dates are chronologically inconsistent; no correction or new chronology is inferred. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.

9. [9] · ANSSI, French Republic · Version 2.0, 27 November 2025, ANSSI-PA-108 · Checked on 3 November 2026Industrial systems cybersecurity: detailed measures, version 2.0

   Backups and restoration tests, skills, continuity plans, multi-site reconstruction priorities and preservation of degraded modes. R8–R11, R15, R73–R74 and R76–R77 are labelled C1; R75 and R82 are labelled C2.

   Passages and scope

   Disclaimer, p.1; reading conventions and C1–C4 table, p.6; R8–R11, p.30; R15, p.32; R73–R75, p.49; R76–R77, p.50; R82, p.51. Unless otherwise specified by regulation, these are non-normative recommendations requiring prior adaptation and validation. C1 covers classes 1–4; C2 covers classes 2–4. The annual testing references in R75/R82 are not universal legal deadlines. Criticality and safety supersede the article’s illustrative equal-priority order. Current official 124-page PDF downloaded; cited passages extracted and read. Printed pages 49–50 rendered and inspected to confirm wording and C1/C2 labels.

10. [10] · NCSC, United Kingdom · Published and reviewed 1 October 2026 · Checked on 3 November 2026Preparing for highly disruptive cyber incidents: 1. Prepare your business continuity

    Priority services, partner dependencies, sudden supplier unavailability and continuity exercises.

    Passages and scope

    §§1.1–1.4: services and dependencies; partners and suppliers; rebuilding; tabletop versus technical recovery testing. General guidance without a guaranteed recovery time or supplier availability commitment. Tabletop exercises and technical tests address different objectives. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.

11. [11] · NCSC, United Kingdom · Published and reviewed 1 October 2026 · Checked on 3 November 2026Preparing for highly disruptive cyber incidents: 3. Prepare your technical recovery capability

    Documentation accessible outside primary systems, tested backups and restoration, dependencies, and the scope of incident-response and rebuilding suppliers.

    Passages and scope

    §§3.1–3.4; §3.6 for OT; §3.7 for supplier skills and responsibilities. The guide calls for clear responsibilities: incident response and rebuilding may have different service scopes. It mandates neither two separate teams nor any victim’s actual contracted capacity. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.

12. [12] · NCSC, United Kingdom · Published and reviewed 1 October 2026 · Checked on 3 November 2026Preparing for highly disruptive cyber incidents: 2. Prepare your people

    Exercises involving technical and business roles, alternative communications, delegated decisions and staff capacity.

    Passages and scope

    §§2.1–2.3; §§2.4–2.6 for welfare, skills and external assistance. Staff-preparation guidance; no universal headcount, mobilisation time or effectiveness gain is quantified. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.

13. [13] · NCSC, United Kingdom · Published and reviewed 1 October 2026 · Checked on 3 November 2026Preparing for highly disruptive cyber incidents: 4. Prepare to make critical decisions

    Decision authority, pre-agreed principles and assessment of the consequences of pre-emptive containment.

    Passages and scope

    Opening section and §4.1, containment decisions and disruption that containment itself can cause. General guidance subject to context, obligations and risk appetite. No operating disconnection command or site-specific safety threshold is supplied. Complete official HTML copy received with HTTP 200; cited passages and displayed dates read directly.