By Andy Mills
Generative AI is now ramping up, with open source having been joined by proprietary as-a-service offerings. It’s now set on breaking new ground in the form of agentic AI. This means that not only does AI draw upon data to craft a response but it also acts autonomously by making recommendations to aid decision-making. It’s a leap forward that is expected to see the technology become firmly entrenched but these advances are now happening so fast that security and governance are struggling to keep pace.
GenAI security poses some unique challenges because we need to not only protect technology from being exploited by attackers but also from itself. There have been countless examples of AI systems hallucinating and returning incorrect results or leaking data, so preventing these applications from going rogue is a must. There needs to be guardrails for the AI as well as protection against the AI.
Testing during development and continuing to monitor these AI applications can help reduce the likelihood of these issues occurring but where should you begin?
Using APIs to secure AI
Generative AI applications source data and talk to one another via application programming interfaces (APIs) which facilitate these exchanges. APIs are also used to integrate these GenAI tools into the business, so AI and API security are inextricably linked. In fact, the number of API requests being made to service GenAI is increasing exponentially and, as that traffic starts to take over, we can expect the attack surface via APIs to grow.
For developers, core precepts such as the Secure Development Life Cycle process still apply but, now, they are having to think about threats specific to AI. The LLM won’t just be interacting with humans but also other apps and APIs, requiring security to understand those behaviours and to put controls around them. AI agents will perceive, reason, act, and learn and the business will need to secure data collection and transfer over APIs, protect access to proprietary sources, enforce action guardrails based on behaviour, and defend against attacks on the data.
From a governance point of view, this means laying some groundwork for agentic AI. The business will need to determine who is responsible for establishing the guardrails, who is using the system and what they are allowed to do, requiring authentication and authorisation controls. Intention is also a key consideration to ensure that the agent doesn’t abuse its function; and limiting reach is vital to ensure that the AI doesn’t overextend itself or blindly follow orders, necessitating prompt supervision.
A rulebook of risk
Security testing AI also requires a different approach. We need to assess the potential for these applications to succumb to some very specific vulnerabilities. Thankfully, OWASP has identified the top threats in its Top 10 Guide for LLM Applications which has just been newly updated for 2025.
In addition to recognised issues such as prompt injection (LLM01:2025), sensitive information disclosure (LLM02:2025), and data and model poisoning (LLM04:2025), the revised list now also incorporates excessive agency (LLM06:2025). This section has been expanded to accommodate agentic architectures and the use of autonomous AI, reflecting the fact such agents or plug-ins could be assigned excessive functionality, permissions, or autonomy. New to the list are vector and embedding weaknesses (LLM08:2025) which refer to the risks posed by Retrieval-Augmented Generation (RAG) and system prompt leakage (LLM07:2025) due to the fact prompts are not always secure.
Denial of Service has also been updated and renamed as unbounded consumption (LLM10:2025) as LLMs can max out resources. GenAI scraping bots do not always observe the limitations of robots.txt files or ‘allow to scrape’ lists when crawling websites, for instance, which can see them dive deep into the website and even harvest intellectual property. Misconfiguration, human error, application error, or malicious intent can also see the AI become greedy and over consume.
Testing AI
The OWASP list is invaluable as it provides us with the information needed to run a battery of tests on these applications. Using synthetic traffic it is possible to identify if the application is susceptible to any of these vulnerabilities and ensure they are secured before deployment. For instance, during recent testing of several popular GenAI applications it was found that several were vulnerable to indirect prompt injection. While standard prompts did not yield a response, malicious prompts were able to extract additional information from the AI. Following such testing, recommendations can be made to the developers who can then take corrective action before the application goes live.
If we look at some of the new categories in the OWASP list, we can see how API security can be applied not only when the application is in preproduction but also after going live. Monitoring the API requests to and from the GenAI can determine if sensitive data (LLM02:2025) is being returned in responses to and from the model, for instance. This can ensure that the LLM is checked for data leakage, with policies set to look for specific data and block requests. Similarly, API monitoring can track usage to see if it becomes excessive and rules and policies can be put in place to meter that usage according to set parameters to prevent unbounded consumption (LLM10:2025). And concerning the supply chain (LLM03:2025), anomalous behaviour in terms of how the system, interacts with third-party systems can be blocked. Behaviour analytics can also be used to observe user interactions to determine if a user is attempting to manipulate the system, such as through direct prompt injection (LLM01:2025).
These examples illustrate the symbiotic relationship between APIs and AI. With this in mind, businesses would do well to adopt the following five-point plan. Firstly, to discover and inventory all of the GenAI APIs they are using, preferably using a runtime system to ensure this is continually updated. Secondly, to create an approved list of GenAI tools to mitigate the risk of shadow AI being introduced to the network. The third step is to install sensitive data controls that can determine what’s being accessed or sent out by the AI to prevent intellectual property from leaving the environment. The development of AI tools comes fourth – if the business is building its own it needs to make guardrails a priority. And finally, the fifth step is to adopt a different mindset when it comes to identifying abuse of APIs and AI, by viewing this through the lens of behaviour and tracking and analysing this rather than speeds and feeds.
There’s no doubt we’re at a turning point with GenAI and agentic AI and governing and securing those applications will be crucial to protecting the business and its data. Putting these applications through their paces while in production and then monitoring them via their APIs has to be the way forward if we are to control autonomous AI.
The author
By Andy Mills, VP of EMEA, Cequence Security
