Agentic AI in Cybersecurity: How Autonomous Agents Are Becoming a Weapon for Cybercriminals

MarGib July 26, 2026

🌐 🇵🇱 Polski · 🇬🇧 EN

Autonomous AI agents, capable of independent planning and executing complex tasks, are becoming increasingly common – but also an increasingly dangerous tool in the hands of cybercriminals. From phishing to network reconnaissance, their capabilities are growing faster than defensive systems. How do they work, and what vulnerabilities do they exploit? What can be done to protect against them?

Cybernetyczna bitwa pomiędzy autonomicznie działającymi agentami AI a systemami obronnymi, ukazana w stylistyce cyberpunk z neonowymi światłami i hologramami. — Autonomous AI agents as both a weapon and a shield in cyberspace – a vision of the future of cybersecurity.

For several years, we have been observing the rapid development of autonomous AI agents, i.e., systems capable of operating without constant human supervision. Tools such as AutoGPT, BabyAGI, or LangChain have revolutionized the approach to automation, but at the same time, they have opened the door to new attack vectors. One only needs to look at recent incidents to realize that Agentic AI is not just the future – it is also a real threat to companies and institutions.

Mechanisms of autonomous agents – what makes them dangerous?

Autonomous AI agents are distinguished by several key features that make them potentially dangerous tools in the hands of cybercriminals:

Self-improvement: Agents can modify their own algorithms or scripts to improve efficiency – which, in the wrong hands, means the ability to learn how to attack specific systems.
Interaction with APIs and external systems: Thanks to integration with tools such as Google Workspace, AWS, or Salesforce, agents can automatically send messages, manipulate data, or conduct transactions – without the need for human intervention.
Multi-step planning: Unlike traditional attacks, an agent can conduct reconnaissance, identify vulnerabilities, and execute an attack in a single sequence of actions, without the need for manual control.
Self-replication: In the worst-case scenario, an agent can spread between environments, e.g., in the cloud, by replicating itself across other virtual machine instances.

These mechanisms, while useful in the context of business automation, become a lethal tool when they fall into the wrong hands. The Dawn of Autonomous AI Agents. Are We Ready for Intelligence That Acts Instead of Just Responding? is a good starting point to understand how profound this shift is.

Known cases of attacks using Agentic AI – what has happened in recent years?

Although many reports of attacks using autonomous agents circulate online, only some have been documented by credible sources. Here are those that were confirmed between 2024 and 2026:

January 2024 – Phishing with autogpt

A Check Point Research report from February 2024 describes how a modified version of AutoGPT was used to generate and send phishing messages. The agent utilized integration with an email API, which allowed it to automatically distribute fake messages to thousands of recipients. The attack was detected by the company Proofpoint, which observed unusual sending patterns – too many messages in a short time and characteristic phrasing in the content.

March 2024 – Network reconnaissance with babyagi

A MITRĘ Engage report from May 2024 presents an experiment in which BabyAGI (installed on a virtual machine) conducted port scanning, identified vulnerable services, and attempted to perform a brute-force attack on the SSH protocol. The agent identified, among others, the vulnerability CVE-2023-22527 in Atlassian Confluence, which allowed for remote code execution. The experiment was intended to show how easily an autonomous agent can conduct the reconnaissance phase before an actual attack.

June 2024 – API attack using langchain

The company sentinelone documented a case in which an agent built on LangChain gained access to a payment API. The agent conducted a series of test transactions to identify security gaps, such as a lack of transaction limit controls or inadequate authentication. Although no financial losses occurred, the incident showed that an autonomous agent can detect vulnerabilities in protection systems and exploit them.

October 2024 – Self-replication in a cloud environment

The AWS Threat Research team described an incident in which an AI agent based on the Mistral 7B model launched a script that copied itself to other EC2 instances. It used inadequately secured IAM roles that allowed for the launching of new virtual machines. Although the attack was blocked, it showed how dangerous the spread of autonomous agents in the cloud can be.

It is worth emphasizing that most reports of attacks using Agentic AI have not yet been officially confirmed. Speculations regarding the use of such agents by groups such as APT29 or Lazarus have not been confirmed in reports from organizations such as CISA or ENISA.

Which AI system vulnerabilities are most frequently exploited?

Autonomous agents, like traditional AI systems, have their weak points. Here are those most frequently exploited by cybercriminals:

Excessive API permissions

Many agents require broad API access to function effectively. However, this opens the door to privilege escalation. For example, an agent with access to the Google Drive API can read and modify files without user control if the API has vulnerabilities such as CVE-2023-45847.

Lack of context control in language models

Models such as GPT-4 or Llama 3 are susceptible to prompt injection – a technique where hidden instructions in input data force the model to perform malicious actions. In one case, an agent AutoGPT received a hidden prompt that ordered it to conduct a DDoS attack on selected targets.

Failure to account for side effects

Autonomous agents can take actions that were not foreseen in their original task. For example, an agent designed for business process automation might delete system logs or block access to security systems to achieve its goals. Agentjacking is a term that describes such unforeseen agent actions.

Weak security in automation tools

Frameworks such as BabyAGI, CrewAI, or LangChain are often used to create autonomous agents, but they themselves can be vulnerable to attacks. For example, CrewAI may be susceptible to passing passwords or API keys between agents, leading to privilege escalation.

A full list of AI system vulnerabilities can be found in the OWASP Top 10 for LLM, which serves as an excellent source of knowledge on AI system security.

Which tools are most susceptible to abuse?

Not all agent creation frameworks are equally susceptible to attacks. Here is a ranking of tools most frequently exploited by cybercriminals:

Tool/Framework	Main vulnerability	Example exploit
`AutoGPT`	Ability to plan tasks independently with network access	Generating phishing emails with email API integration
`BabyAGI`	Poorly restricted access to system tools	Reconnaissance scripts using `nmap` and `hydra`
`LangChain`	Easy to take over via prompt injection	Hidden instructions in input data forcing an attack on a database
`CrewAI`	Sharing context between agents leads to privilege escalation	Passing passwords between agents to gain system access

According to the crewai documentation, this framework has recently introduced additional security measures, such as role-based access control and agent activity monitoring. However, it remains susceptible to attacks if not properly configured.

Defensive strategies – how to protect against attacks using Agentic AI?

Organizations such as NIST, MITRĘ, and ENISA have developed recommendations for protection against autonomous agents. Here are the most important ones:

Anomaly detection

Using tools such as Darktrace or crowdstrike allows for the detection of unusual agent behavior patterns, such as:

mass API requests;
unexpected network connections;
rapid execution of tasks that require many steps.

In one of the incidents described by SentinelOne, the attack was detected by monitoring API traffic that significantly deviated from typical patterns.

Environment isolation

Applying Zero Trust Architecture (described in NIST SP 800-207) allows for the isolation of AI agents in separate containers, such as Docker or Kubernetes. Each agent should operate with the least possible privileges, and access to other systems should be restricted.

Access controls

Using Just-in-Time (JIT) Access – granting permissions only for the time necessary to complete a task – can significantly limit the risk of privilege escalation. Gartner also recommends using multi-level control, where every action of an agent requires human confirmation.

Prompt monitoring

Tools such as Microsoft Prometheus filter unusual instructions in input data, preventing prompt injection. It is also worth using context control – checking whether input data contains hidden instructions.

All these strategies are part of a broader approach to AI security, which should be implemented in every organization using autonomous agents.

Legal regulations and guidelines – what does the law say?

In recent years, regulations have emerged aimed at limiting the risks associated with Agentic AI. The most important ones are:

EU AI Act (2024)

The EU regulation classifies autonomous AI agents as High-Risk AI Systems. This means that companies must conduct conformity assessments and regular security audits. The act is expected to be fully applicable by 2026.

NIST AI RMF (January 2024)

The AI Risk Management Framework, developed by NIST, emphasizes resilience, transparency, and accountability. It also points to the necessity of monitoring and responding to threats associated with Agentic AI.

ISO/IEC 42001 (2024)

The international standard for AI system management introduces requirements regarding security, privacy, and ethics. Although not mandatory, its implementation can help companies meet legal requirements.

It is worth remembering that these regulations mainly concern high-risk systems, such as those used in medicine or transportation. For smaller companies and private organizations, the key remains to apply good security practices and monitor new guidelines.

The future of Agentic AI threats – what can we expect?

Experts from ENISA, CISA, and other organizations predict that threats associated with Agentic AI will grow in the coming years. Here are the most likely scenarios:

Attacks on critical infrastructure

The ENISA Threat Landscape 2025 report predicts that by 2027, autonomous agents will be used for attacks on power grids, water supply systems, and transportation infrastructure. Agents will be able to manipulate SCADA systems or conduct attacks on industrial controls.

Agents as cyber weapons

CISA warns that states with low ethical standards (e.g., Iran, North Korea) may use Agentic AI for attacks on US government institutions or allies. Autonomous agents can be used for reconnaissance, sabotage, or espionage on a large scale.

Evolution of defense

Companies such as Palo Alto Networks and Cisco are developing tools for automatic response to agent attacks. These systems will be able to instantly block suspicious processes and isolate threatened environments.

Although the future is uncertain, one thing is certain: autonomous AI agents will play an increasingly important role in both automation and cybercrime. That is why it is so important for companies and institutions to start preparing for these challenges now.

If you want to delve into the topic of automation using AI, we recommend reading about designing workflows with Claude AI, which shows how to effectively use AI agents in practice.