How do organizations measure success in resilience?

By Robert Hall

The well-quoted statement that ‘If you can’t measure it, you can’t manage it’ has sometimes been given a slightly different spin with the replacement of the word ‘manage’ by ‘improve’ (1). The amendment reflects the notion that there are many things that cannot be measured and still must be managed but there are many things that cannot be measured; and about which managers must still make decisions.

Resilience is one of those subjects that falls into the hard-to-measure category. This is because it has many diverse elements – structural, cultural, emotional, economic, etc – that sit in many different contexts. Resilience in the construction industry, for example, is very different to resilience in the health or defence sectors. Furthermore, as resilience is more akin to a protracted journey of learning and improvement through multiple events rather than an early destination when all becomes clear and crises avoided.

As a result, no definitive, standardised measure of resilience exists to date. Nevertheless, there are ways and means of getting a handle on some of the key components which reveal whether the journey is on the right track and affords the appropriate vistas (i.e. outcomes) that can generate improved resilience along the route. Success can be a measure of this improvement but it is a constantly moving target.

Those ways and means include indicators, benchmarks, standards, assurance methods, and regulatory policies to list some of the major techniques. Each has its own merit but can only deliver part of the complex picture. This does not mean that the effort isn’t worthwhile, as even a partial picture can help users and stakeholders get some measure of how resilient an organization is to be able to weather disruption and what can be done by way of preparation to avoid the worst of the damage.

Indicators

One way to approach the challenge is to look for indicators that can provide organizations with guidance on their resilience strengths and weaknesses (2). Indicators should reflect the desired objectives and outcomes that an organization defines ahead of a crisis. Combining both quantitative and qualitative indices gives a more complete picture of resilience as the former offers hard facts while the latter provides the softer context.

Indicators should be a measure of potential vulnerability (downside) as well as adaptive capacity (upside) in a crisis. They need to span the entire panoply of resilience measures while respecting the key components of resilience; namely preparation (before), recovery (during), and adaptation (after). As an alternative, distinct functional categories can be used.

One approach, advocated by FEMA in the US, looks at inputs, processes, outputs, and outcomes with each category having a different focus (3). Yet another approach would be to examine people, place, processes, performance and preparation – the so-called five Ps (4). The criteria for each would vary according to the size of an organization, its role and activities.

Benchmarks

By adopting the chosen indicators and their associated criteria, it is possible to design a simple scaling mechanism, say 0 for poor to 5 for excellent, that in turn allows scoring to occur. The quantitative readings are not important in their own right, but they can give an indication of the state of play, ranging from individual departments to corporate entities. The measurements can be conducted either internally (self-assessments) or externally (inspections). Depending on market-sector transparency, it should be possible to conduct organizational comparisons in resilience with the sharing of results, experiences and lessons, bearing in mind like-for-like compatibility. The exercise can be repeated at periodic intervals to avoid staleness and maintain relevance.

The question then arises as to how to use such benchmarking to generate success by being more resilient. Here, the idea of resilience maturity modelling comes to the fore. This is a technique that relies on a series of developmental stages to show progressive improvements. There are usually several stages in most models, beginning with the most basic and progressing to the most sophisticated. British Standard 65000:2022 (5) has six stages ranging from ‘Immature’ to ‘Optimising’ while the new FEMA guide on national resilience has four stages from ‘Ad Hoc’ to ‘Integrated’. The terminology indicates the increasing sophistication as the models progress.

Whatever approach is adopted, the aim should be to embed a road map for improvement. There is no final goal (ultimate success) but rather a gradual process of renewal and refinement. The description of the levels is of secondary importance to the notion of moving up the resilience ladder within a sound, co-ordinated, framework. The levels can of course be modified to suit an organization, with key performance indicators (KPIs) set out along the way. It is crucial, however, that a nominated individual takes ownership of the model and delivery of the KPIs. At board level, the maturity model provides a dashboard of the organization’s transition over time.

Standards

In resilience, as in many other fields, there is an increasing tendency to revert to standards to reflect common baselines against which to assess competence. There are industry, national, and international standards across a wide range of sectors. Standards can allow greater operability, encourage collaboration, and spur performance, while conferring reputation and improving safety. Again, they can be internally measured and externally assessed.

Without standards, there is room for error with very real and very serious consequences. Standards or good-practice guides can be – and should be – a framework from which to improve, providing a ladder through which goals can be achieved. They should standardise what one should be able to do, rather than standardise against a specific set of risks, in essence maintaining a risk-agnostic approach.

Standards can always be improved to enhance their applicability and utility for organizations of all sizes. They are not a resilience panacea as they have three potential weaknesses. The first is that they can be too complex; second, they may not be scalable for the various size of organizations that exist; and, third, there can be negative consequences such as short cuts in order to claim accreditation or avoidance due to their bureaucratic nature.

Despite these downsides, organizations remain attracted to the idea of achieving and promoting certain standards as it shows to customers and clients their conformity to good practice, as well as helping internally to keep staff focused on the expectations.

Assurance and regulation

According to a Gaelic proverb, assurance is two-thirds of success. Assurance covers a broad range of activities, including assessment, evaluation, testing, audit, exercising and validation. These enable judgements to be made about an organization’s level of resilience. Reflecting the differences between inputs, outputs, and outcomes, assurance focuses on what an organization has, what it does, and what it can do. The food-safety industry and the air-transport sector already use an extensive array of assurance methods to ensure compliance with desired practices, thereby enhancing resilience.

Assurance can be supported by regulation and even legislation. According to a survey by the BCI, regulation is a primary driver for operational resilience programmes (6). In the UK financial sector, for example, the regulatory authority’s policy on operational resilience is assessed in key areas, namely governance, operational risk management, business continuity planning, and the management of outsourced relationships. Regrettably, the introduction of legislation by way of a statutory Resilience Statement ‘for [large] companies to report on matters that they consider a material challenge to resilience over the short and medium term, together with an explanation of how they have arrived at this judgement of materiality’ was withdrawn by the last UK Conservative government; its future under a new government is uncertain (7). While the financial sector has been at the forefront of resilience development, largely because of the dangers of making costly mistakes in the market, it is expected that the sector will develop other solutions and other sectors may follow as part of their annual reporting requirements.

Single-domain regulatory frameworks have dominated the landscape to date but as the interconnectedness and interdependence of systems become ever more apparent and pertinent, particularly in the CNI sector, better multi-sector regulatory systems need to be designed for whole-system and whole-nation preparedness and resilience (8).

Measure up

There is no perfect method for quantifying quality. Resilience, whether in application or measurement, is an imprecise art form that has multiply facets and features. Yet, there are techniques that can help with getting a handle on the key components that range across people, place, processes, performance, and preparation. While all are important, the last of these is particularly significant for an organization if it is to be ready to respond to poly-crises at any time and from any direction.

It is important, nevertheless, not to over-compartmentalise or unduly fragment the measurement of resilience into too many discrete categories. Resilience itself depends on breaking down silos and working across departments. In the same way, we should see measurement as a holistic exercise that looks across an organization and tries to identify not only gaps but also ways to plug those gaps in a progressive and professional manner. That will help deliver success.

The author

Robert Hall is cofounder and former Executive Director of Resilience First. He is now an independent consultant, writer and speaker.

This is the last of four articles based on the book Building Resilient Futures (2023) (9). The author’s second book The Resilience Mindset: A Philosophical Journey will be released later this year (10).

Read the previous articles:

References

(1) The original quote is attributed to Dr W Edwards Deming. The variation is attributed to Peter Drucker.
(2) Hall, R. (2023) Indicators and Warnings, Crisis Response Journal, Vol 18, Issues 4.
(3) National Resilience Guidance, FEMA, August 2024.
(4) Hall, R. (2023) Ibid. See also Resilience First’s Self-Assessment Tool.
(5) BS 65000:2022. Organizational resilience. Code of Practice.
(6) BCI Operational Resilience Report 2022, June 2022.
(7) The UK government’s original proposals set out to ‘improve how organisations identify, manage and report on their resilience risks that are most material to their business’. The intention was that the Resilience Statement would apply to entities with 750 or more employees and £750 million or more in annual turnover.
(8) Judge, R., Elahi, S. (2024) Regulating for Resilience, National Preparedness Commission, 6 September 2024.
(9) Hall, R. (2023) Building Resilient Futures, Austin Macauley Publishers. ISBN: 9781035812622.
(10) Hall, R. (2024) The Resilience Mindset: A Philosophical Journey, Austin Macauley Publishers. ISBN: 9781035878284.