Beyond awareness: engineering resilience against Shadow AI

By Mark Kirby

AI is now part of our global infrastructure story. Across supply chains, healthcare, and education, artificial intelligence is being woven into core operational systems at unprecedented speed. From automating logistics and patient triage to powering adaptive learning tools, AI is transforming resilience itself.

But beneath that transformation, a silent risk has taken root: shadow AI. The problem isn’t new, but its scope, speed, and potential impact have grown significantly. According to IBM, shadow AI now accounts for roughly 20% of global breaches, adding an average of $670,000 to incident costs where unmanaged AI use is widespread. Despite that, 63% of organizations lack an AI governance framework, and most have never audited for unsanctioned AI activity.

These are not abstract numbers. They represent a widening exposure gap between rapid AI adoption and organizational control; one that organizations cannot afford to ignore.

The hidden risk

Shadow AI is simply the modern face of the long-standing challenge that shadow IT represents: employees using unapproved systems or software without formal oversight. What has changed is the scale of integration. Many AI tools now plug directly into email servers, document management systems, and sensitive data repositories. If you are not supplying an AI solution to your employees, there is a good chance that they are bringing in their own.

An engineer pasting proprietary code into a public chatbot to debug an issue, or a clinician using an AI writing assistant to summarise patient notes, may both think they’re saving time. In reality, they could be exposing commercially sensitive or personally identifiable information to external servers, sometimes in jurisdictions beyond regulatory reach.

In supply chain and healthcare networks where even milliseconds of downtime – or a single data compromise – can cascade across partners and systems, these risks become systemic.

Compounding this, today’s generative AI systems are not passive tools; they can ‘hallucinate’, producing information that is entirely fabricated yet delivered with absolute confidence. When such outputs are used to inform decisions, write reports, or populate datasets, the line between reliable automation and misinformation blurs dangerously. Recent cases, such as Deloitte’s $440,000 AI-assisted report containing false data, illustrate how even trusted organizations can be caught out.

Adding to this is the growing challenge of ‘trust decay’; as more unverified AI-generated material enters corporate workflows, the ability to distinguish genuine insight from confident fabrication erodes. Beyond business, this is a wider societal issue. A recent Reuters Institute article warns that ‘AI slop’ – low-quality, auto-generated content that sounds authoritative but lacks accuracy or depth – is rapidly flooding the Internet. Unlike deliberate misinformation, it’s often created for clicks or SEO, yet still erodes trust, clogs search results, and risks degrading future AI models trained on this polluted data. Some experts liken it to digital spam that platforms will learn to filter out, while others see a growing threat to journalism and information integrity if left unchecked.

From data leak to systemic exposure

The technical threat profile of shadow AI extends far beyond data leakage. In complex operational environments, unauthorised AI use introduces:

Integrity risk, where AI-generated outputs feed into operational systems or reports without verification.
Privilege escalation, as AI assistants integrated with enterprise platforms can access documents beyond user clearance levels.
Regulatory exposure, particularly where protected data, such as patient records or student information, is processed via consumer-grade AI platforms.
Supply chain propagation, as compromised data or code is shared between interconnected vendors, research partners, or service providers.

What begins as a localised breach can quickly escalate into a multi-party incident, with consequences that extend across entire critical infrastructure ecosystems.

Governance as a resilience mechanism

Governance is often framed as a compliance exercise, but in the context of AI, it is an operational defence mechanism. The EU AI Act and the UK’s AI Assurance Roadmap both assign responsibility for AI oversight to the organizations deploying it. For sectors already bound by GDPR, NIS2, and healthcare or education-specific regulations, AI governance is now an essential control within the cyber resilience stack.

Practically, that means combining policy, process, and technology:

Policy: define what constitutes authorised AI use, where AI may interact with sensitive data, and what the review process should be before deployment.
Process: embed AI audits into standard risk and compliance routines, ensuring every department’s use of AI tools is catalogued and periodically reviewed.
Technology: deploy monitoring solutions, such as Microsoft Defender or Cloud App Security, to detect unauthorised AI traffic and integrate with identity and access management systems to enforce usage boundaries.

This triad shifts governance from a paper exercise to a measurable resilience function. The sectors most at risk are often those least able to absorb disruption. Healthcare providers operate under resource constraints and regulatory pressure; educational institutions depend on trust and continuity; supply chain operators face both operational and reputational exposure.

For many critical organizations, the goal is not to suppress AI use but to standardise and supervise it. That starts with leadership. Assigning clear AI oversight to an accountable executive – whether a Chief Digital Officer, CTO, CISO, or dedicated AI lead – establishes ownership and accountability. From there, organizations should maintain a central AI register that lists all approved platforms, their integration points, and associated risk ratings.

Access control is equally crucial. AI-connected services must follow the principle of ‘least privilege’, ensuring that data segmentation and sensitivity boundaries are preserved. Finally, governance cannot succeed without people: mandatory AI awareness training helps employees recognise both the risks of unsanctioned tools and the social engineering tactics increasingly exploiting generative AI interfaces.

Together, these measures create a culture of disciplined experimentation, encouraging innovation within defined parameters rather than in the shadows.

The conversation about AI risk is maturing. What’s needed now is engineering discipline. Critical organizations must design resilience into their systems, treating AI governance as a form of digital hygiene – a living process combining auditability, automation, and education.

A final thought: handled well, AI strengthens the foundations of operational resilience; left unchecked, it can erode them from within.