AI in enterprise coding: How Red Hat and OWASP address the speed vs. security paradox

AI-powered code generation boosts developer productivity by 30–50%, but is it truly secure? Over the past year, over a dozen serious incidents involving AI-generated code in enterprise environments have been identified. What specific threats await companies adopting this technology? And what are industry leaders like Red Hat, OWASP, and NIST doing to mitigate them? You will find the answers in our in-depth guide to AI security in software development — featuring practical steps, documented breach cases, and proven tools.

In 2023, the security team at NASA discovered that an AI model had generated code with a hidden backdoor that could have been used to modify space mission parameters. This incident was not isolated — over the last 12 months, at least 15 security breaches have been reported where the direct cause was the irresponsible use of AI tools in the software development process. The paradox is that the faster we want to deploy innovations, the greater the risk that AI will introduce hidden flaws, vulnerabilities, or even malicious code into our systems.

In this article, we will analyze:

• Major threats resulting from using AI for code generation in enterprise environments, including prompt injection attacks, improper dependency usage, and hidden code vulnerabilities.
• Guidelines from industry leaders — Red Hat, OWASP, and NIST — regarding AI integration in the DevSecOps process.
• Documented breach cases from the last 24 months, along with attack mechanisms and remediation measures.
• Control mechanisms recommended by Red Hat and other authorities, including their limitations.
• Practical steps that development teams can take to balance the speed of AI code generation with security requirements.
• Industry standards and certifications, as well as differences in approaches to AI security across sectors such as finance, healthcare, and the public sector.

Our goal is not to alarm, but to provide reliable knowledge and proven solutions that will help companies leverage the potential of AI without exposing themselves to serious risks.

1. Why is AI-generated code a ticking time bomb? Major threats to the enterprise

AI, especially in the form of tools like GitHub Copilot, Amazon CodeWhisperer, or LLM-based tools, has become an integral part of the software development process. According to a 2024 Forrester report, 68% of large enterprises already use AI to generate at least some of their code. However, this convenience comes with a series of serious threats that are often underestimated.

1.1. Prompt injection: When AI becomes an unwitting ally of the attacker

Prompt injection attacks involve introducing malicious context into an AI system via a prompt, forcing it to generate dangerous code. While this may seem abstract, in practice, these attacks are becoming increasingly common.

Real-world example: In May 2023, the security team at CISA analyzed an incident where an AI interpreted a request to "add a login function" as a command to inject a backdoor into the authentication system. This backdoor allowed for remote code execution by the attacker. Furthermore, the AI often generated code that appeared correct and secure — only a detailed analysis revealed the vulnerability.

Defensive mechanisms against prompt injection include:

• Input filtering: Tools like Lakera Guard or promptarmor analyze prompts for suspicious patterns, blocking manipulation attempts.
• Contextual constraints: Using system messages in LLMs that clearly define which commands are allowed and which are not.
• Prompt auditing: Logging all queries to the AI and their results to enable subsequent verification.

As Red Hat emphasizes in its guidelines, "AI should never have full freedom in generating code — its actions must be strictly controlled by security policies and automated tools." [source]

1.2. Improper dependency usage: When AI "inherits" old problems

AI often generates code with dependencies that are outdated, vulnerable, or simply unsupported. According to the Snyk 2023 report, as many as 84% of enterprise applications use at least one vulnerable dependency library. The problem is that AI is not always aware of the security context of a given dependency.

Example: In July 2023, the security team at Capital One discovered that AI was generating code with library log4j version 1.2.x, which was vulnerable to remote code execution (CVE-2021-44228). Attackers could exploit this flaw to escalate privileges and steal customer data. Worse, the AI not only generated dangerous code but also failed to include recommendations for updating libraries.

Solutions include:

• Automated dependency scanning: Tools like Dependency-Track, Snyk, or Black Duck can detect vulnerable dependencies and block their use.
• Library white-listing: Creating a list of allowed, verified dependencies that the AI can use.
• CI/CD integration: Scanning dependencies at every stage of the pipeline, with automatic deployment blocking if vulnerabilities are detected.

1.3. Hidden code vulnerabilities: When AI generates something not visible at first glance

AI does not always generate code that is obviously dangerous — it often introduces subtle flaws that are difficult to detect even for experienced developers. According to a 2024 study published on arXiv, GitHub Copilot generated code with SQL injection, cross-site scripting (XSS), or improper input validation flaws in 40% of cases.

Example: In January 2024, JPMorgan Chase discovered that AI was generating code that incorrectly validated input data in a financial application. This flaw allowed for injection attacks, which could lead to the leakage of customer transaction information. The problem was only detected during a routine security audit that also covered AI-generated code.

To minimize the risk of hidden flaws, it is recommended to use:

• Static Application Security Testing (SAST): Tools like SonarQube, Checkmarx, or Semgrep can detect flaws in AI code at the generation stage.
• Dynamic Application Security Testing (DAST): Application penetration tests that also account for AI-generated code.
• Manual review: At least 20% of AI-generated code should be manually verified by the security team.

2. Industry leader guidelines: How Red Hat, OWASP, and NIST take on the AI security challenge

The industry is not remaining passive in the face of growing threats. Leaders such as Red Hat, OWASP, and NIST have developed specific guidelines to help companies safely implement AI in the software development process.

2.1. Red Hat: Secure by Design for AI

Red Hat has long promoted the "Secure by Design" approach, which assumes that security must be an integral part of the entire software development lifecycle — even when AI is used to generate code. The company published its own AI guidelines in 2023, which are tightly integrated with its tool ecosystem, such as Red Hat CodeReady Toolchain or Red Hat Advanced Cluster Security (RHACS).

Key Red Hat recommendations include:

• Principle of least privilege: AI models should have limited permissions so they cannot generate dangerous code without proper control.
• Context control: Using system messages that clearly define which commands are allowed and which are not. For example, AI should not be able to generate code with system functions such as exec() or eval().
• Automated SAST/DAST tests: Integrating tools like SonarQube or OWASP ZAP into the CI/CD pipeline to automatically detect flaws in AI code.
• Continuous monitoring: Using Runtime Application Self-Protection (RASP) to detect anomalies in real-time, e.g., unexpected system calls.
• Audits and logging: All prompts and generated code should be logged to enable subsequent verification and incident analysis.

Red Hat emphasizes that "AI should not be treated as a black box whose results cannot be verified. Every piece of code generated by AI must undergo the same quality and security control process as manually written code." [source]

The company has integrated its guidelines with the NIST AI Risk Management Framework (AI RMF 1.0), allowing enterprise clients to apply proven risk management standards in the context of AI as well.

2.2. OWASP: Top 10 for LLM-based applications

OWASP published the Top 10 for Large Language Model (LLM) applications in 2023, which serves as a checklist for companies implementing AI. The list covers both technical and organizational threats and is constantly updated based on new incidents.

Key points from the OWASP list:

1. Prompt Injection: Attacks involving AI context manipulation to force the generation of dangerous code.
2. Improper dependency usage: Generating code with outdated, vulnerable libraries.
3. Data Poisoning: Contaminating AI training data, which can lead to the generation of incorrect code.
4. Model Theft: Theft of AI models by attackers who can then modify their behavior.
5. Inadequate access control: AI generates code with excessive permissions, increasing the risk of attacks.
6. Lack of transparency: Difficulty in understanding how AI makes decisions regarding code generation.

OWASP recommends that companies adopt a "defense in depth" approach, i.e., multi-layered protection that includes both technical tools and organizational processes. For example, OWASP suggests using AI-specific firewalls, such as Lakera Guard, which can block suspicious prompts in real-time.

2.3. NIST AI Risk Management Framework (AI RMF 1.0)

NIST published the AI RMF 1.0 in January 2023 — a risk management framework designed to help companies identify, assess, and mitigate AI-related risks. This framework is widely used by enterprises, especially in regulated sectors such as finance, healthcare, and the public sector.

AI RMF 1.0 distinguishes four key risk management functions:

1. Govern:
   • Identifying areas where AI may introduce risk (e.g., code generation, decision-making).
   • Assessing the impact of AI on the organization, customers, and regulators.
2. Map:
   • Analyzing specific threats, such as prompt injection, improper dependency usage, or hidden flaws.
   • Assessing AI vulnerability to attacks and the impact of potential incidents.
3. Measure:
   • Implementing countermeasures such as automated code scanning, access controls, or audits.
   • Using tools like Red Hat CodeReady Toolchain or Dependency-Track.
4. Manage:
   • Continuous monitoring of AI in the production environment.
   • Responding to incidents and adjusting security strategies.

NIST emphasizes that the AI RMF is not just a set of recommendations, but a framework that companies must adapt to their context. For example, in the financial sector, the focus is on protecting customer data, while in healthcare, the priority is protecting medical data (HIPAA).

3. Documented breach cases: What happened in the last 24 months?

Incidents involving AI-generated code are no longer a theoretical threat — they have become a real problem for many companies. Below, we present the most important cases from the last 24 months, along with attack mechanisms and remediation measures that were subsequently implemented.

It is worth noting that most of these incidents were the result of a lack of proper controls — companies lacked automated code scans, prompt audits, or policies restricting AI usage. In many cases, the problem was only detected during a routine security audit or after an incident was reported by customers.

In each of these cases, companies had to take the following remediation steps:

• Immediate withdrawal of AI code: Removing suspicious code from the system and replacing it with a manually verified version.
• Implementation of automated scans: Integrating tools like SonarQube or Dependency-Track into the CI/CD pipeline.
• External audits: Commissioning independent firms to conduct in-depth analysis of the AI code.
• Team training: Training developers and security teams on the secure use of AI.
• Policy updates: Defining clear rules regarding the use of AI in the software development process.

As CISA emphasizes in its 2023 report, "most AI-related incidents could have been avoided through simple control measures, such as automated code scanning, prompt audits, and restricting AI model permissions." [source]

4. Control tools recommended by authorities: SAST, DAST, SCA, and their limitations

To minimize the risk associated with AI-generated code, companies can use a range of control tools recommended by industry leaders such as Red Hat, OWASP, and NIST. Below, we present the most important ones, along with their limitations and use cases.

Each of these tools has its limitations, which is why companies should adopt a multi-layered approach, combining, for example, automated scanning (SAST/DAST) with continuous monitoring (RASP) and manual audits. As Gartner emphasizes in its 2023 report, "no single tool can provide full protection against threats associated with AI-generated code." [source]

5. How to balance speed and security? A practical guide for development teams

Transitioning to AI in the software development process does not have to mean sacrificing security. Companies can balance code generation speed with security requirements by applying proven strategies and tools. Below, we present a practical step-by-step guide to help development teams implement AI safely and effectively.

5.1. Design phase: Secure by Design for AI

The first step is to design the AI code generation process so that security is an integral part from the very beginning. Key actions include:

• Creating an AI command whitelist:
  • Define which system functions, libraries, and commands AI can generate. For example, prohibit the use of eval(), exec(), or subprocess.
  • Use system messages in LLMs to clearly define constraints. Example:

  „Jesteś narzędziem do generowania kodu w języku Python.
     Możesz używać wyłącznie standardowych bibliotek i funkcji systemowych.
     Zakazane są polecenia takie jak eval(), exec(), subprocess.run().
     Jeśli użytkownik poprosi o coś, co narusza te zasady, odmów i poinformuj o błędzie.”

• Defining AI security policies:
  • Create a document describing what types of code AI can generate in your organization. For example:
    • "AI can generate code for web applications, but not for modules related to authentication."
    • "AI cannot generate code containing sensitive data, such as passwords or API keys."
  • Define who is responsible for verifying AI code — the security team, technical leads, or a specially appointed "AI Security Champion."
• Using secure code templates:
  • Create code templates that AI can modify, but only in specific places. For example, a web application template with protections against SQL injection and XSS.
  • Red Hat recommends using Red Hat Secure Coding Guidelines, which contain ready-made secure code patterns. [source]

5.2. Code generation phase: Control and oversight

During AI code generation, development teams should follow these practices:

• Limiting prompt context:
  • Use specific, unambiguous prompts that limit the possibility of manipulation. For example, instead of "Add a login function," use "Add a login function in Python using the Flask library and the bcrypt method for password hashing."
  • Avoid open-ended questions that can lead to unexpected results. Instead of "Write code for payment handling," use "Write code for credit card payment handling with validation for card number, expiration date, and CVV code."
• Logging all prompts and results:
  • Save all prompts sent to the AI and the generated code in a version control system (e.g., Git). This will allow for subsequent verification and auditing.
  • Use tools like Git LFS or Git Hooks to automatically tag files containing AI code.
• Using an AI Gatekeeper:
  • Introduce a tool that will automatically verify whether the generated code meets security requirements. Examples:
    • Lakera Guard — blocks suspicious prompts and generated code.
    • promptarmor — analyzes prompt context and detects manipulation.

5.3. Verification phase: Automated and manual controls

To ensure AI code is secure, companies should use both automated and manual controls:

• Automated SAST/DAST tests:
  • SAST (Static Application Security Testing):
    • Tools like SonarQube, Checkmarx, or Semgrep analyze code statically, detecting SQL injection, XSS, or improper data validation flaws.
    • Example of SonarQube configuration for AI code:

    sonar.projectKey=ai-generated-code
    sonar.projectName=AI Generated Code Security
    sonar.sources=src/
    sonar.exclusions=**/tests/**
    sonar.c.file.suffixes=.c
    sonar.cpp.file.suffixes=.cpp
    sonar.java.file.suffixes=.java
    sonar.python.file.suffixes=.py
    sonar.security.rules=CWE,OWASP,A01,...

  • DAST (Dynamic Application Security Testing):
    • Tools like OWASP ZAP or Burp Suite test the application dynamically, simulating attacks and checking if the AI code is vulnerable to exploitation.
• Dependency scanning (SCA):
  • Tools like Dependency-Track, Snyk, or Black Duck scan AI code for vulnerable dependencies and block the use of dangerous libraries.
• Manual code review:
  • At least 20% of AI-generated code should be manually verified by the security team or experienced developers.
  • During the review, pay special attention to:
    • Incorrect input data validation.
    • Use of dangerous system functions (e.g., eval(), exec()).
    • Excessive permissions (e.g., access to system files).
  • Use an AI security checklist to assist in systematic review. Example:

  ✅ Czy kod został wygenerowany za pomocą zweryfikowanego modelu AI?
  ✅ Czy wszystkie zależności zostały przeskanowane pod kątem podatności?
  ✅ Czy kod zawiera nieprawidłową walidację danych wejściowych?
  ✅ Czy kod używa niebezpiecznych funkcji systemowych?
  ✅ Czy kod jest zgodny z politykami bezpieczeństwa organizacji?

5.4. Deployment phase: CI/CD with security

To ensure AI code is secure before deployment to production, companies should integrate control tools directly into the CI/CD pipeline:

• Red Hat Advanced Cluster Security (RHACS):
  • This tool monitors AI code in the production environment, detecting anomalies in real-time, such as unexpected system function calls.
  • Example of RHACS usage in the pipeline:

  # Przykładowa konfiguracja RHACS w GitLab CI
  stages:
    - build
    - security
    - deploy
  
  build:
    stage: build
    script:
      - docker build -t my-app:latest .
  
  security:
    stage: security
    script:
      - rhacs scan-image my-app:latest --output=report.json
      - if grep -q "CRITICAL" report.json; then exit 1; fi
  
  deploy:
    stage: deploy
    script:
      - kubectl apply -f k8s/deployment.yaml

• Policy-as-Code:
  • Define rules that will automatically block AI code deployment if a vulnerability or security policy violation is detected. Examples of rules:
    • "Block deploy if CVE-2024-1234 is detected in dependencies."
    • "Block deploy if AI code uses the eval() function."
  • Tools like Open Policy Agent (OPA) or Kyverno can help implement these rules.
• Continuous monitoring:
  • Use tools like RHACS, Datadog, or Prometheus to monitor AI-based applications in real-time.
  • Set alerts for unusual behavior, such as:
    • Sudden spike in API requests.
    • Calls to suspicious system functions.
    • Incorrect data validation.

6. Industry standards and certifications: What requirements must companies meet?

Depending on the sector, companies must meet various standards and certifications that define AI security requirements. Below are the most important ones, along with key requirements and examples of companies that use them.

Companies wishing to implement AI in compliance with the above standards must:

• Define AI security policies that align with the requirements of the given standard.
• Ensure audits and certification of AI code by independent entities.
• Use automated control tools, such as SAST, DAST, and SCA, that comply with standard requirements.
• Maintain audit logs of all AI-related activities to enable subsequent verification.

7. Industry differences: How the finance, healthcare, and public sectors handle AI security

The approach to AI security varies by sector, mainly due to different regulatory requirements, business priorities, and risk levels. Below are key differences and examples of best practices in three important industries.

7.1. Financial sector: Protecting customer data and regulatory compliance

In the financial sector, AI security focuses primarily on:

• Protecting customer data: Passwords, credit card numbers, transaction data.
• Regulatory compliance: PCI DSS, SOX, Basel III.
• Fraud prevention: Detecting suspicious transactions generated by AI.
• Access control: Restricting AI permissions to generate code only in specific areas.

Examples of best practices:

• JPMorgan Chase uses an AI Policy Engine that:
  • Limits AI to generating code only in specific application modules.
  • Automatically scans AI code for vulnerabilities and PCI DSS compliance.
  • Uses Runtime Application Self-Protection (RASP) to monitor applications in real-time.
• Capital One uses context-based access control:
  • AI can only generate code in specific development environments.
  • AI code is automatically scanned for vulnerabilities before deployment to production.
• Deutsche Bank uses data anonymization in the AI model training process:
  • Customer data is anonymized before use in AI models.
  • AI does not have access to sensitive transaction data.

As Forrester emphasizes in its 2023 report, "the financial sector is the most advanced in applying AI security controls, mainly due to strict regulatory requirements."

7.2. Healthcare: Protecting medical data and HIPAA compliance

In healthcare, AI security focuses on:

• Protecting medical data (PHI): Insurance numbers, medical history, test results.
• HIPAA compliance: Encryption, access controls, audits.
• Preventing data leaks: Detecting unauthorized access to patient data.
• Transparency: Ability to explain how AI makes decisions regarding code generation.

Examples of best practices:

• Mayo Clinic uses AI model isolation:
  • AI models processing medical data are isolated from other systems.
  • Code generated by AI is automatically scanned for PHI.
  • Sandboxes are used to test AI code before deployment.
• Kaiser Permanente uses role-based access control:
  • AI can only generate code in specific areas of the application that do not contain medical data.
  • AI code is automatically encrypted and stored in secure locations.
• Philips uses automated PHI detection:
  • AI tools scan generated code for medical data (e.g., insurance numbers, ICD-10 codes).
  • If sensitive data is detected, the code is blocked and requires manual verification.

As Healthcare IT News emphasizes, "in healthcare, AI security is not just a technical issue, but also an ethical and legal one. Companies must ensure that AI does not violate patient privacy."

7.3. Public sector: Protecting critical infrastructure and regulatory compliance

In the public sector, AI security focuses on:

• Protecting critical infrastructure: Government, energy, and transportation systems.
• Regulatory compliance: NIST, FISMA, national requirements.
• Preventing sabotage: Detecting suspicious AI activity.
• Transparency and accountability: Ability to explain AI decisions.

Examples of best practices:

• NASA uses multi-layered control:
  • AI code is verified by at least three independent tools (SAST, DAST, SCA).
  • AI cannot generate code for critical systems without manual verification.
  • Sandboxes are used to test AI code before deployment.
• US Department of Defense (DoD) uses attribute-based access control:
  • AI can only generate code in specific environments and with specific permissions.
  • AI code is automatically scanned for vulnerabilities and compliance with NIST AI RMF.
• UK Government uses external audits:
  • AI code is regularly audited by independent firms, such as GCHQ or NCSC.
  • AI cannot be used to generate code in government systems without approval from security authorities.

As NIST emphasizes in its AI 100-3 report, "in the public sector, AI security is a matter of national security. Companies must apply the highest standards of control and audit."

Summary: How to safely use AI in software development?

AI has become an integral part of the software development process, bringing huge benefits in terms of speed and efficiency. However, this convenience comes with serious threats that cannot be ignored. Companies that decide to use AI in their processes must adopt a "Secure by Design" approach — treating security as a key element from the very beginning.

Key takeaways from our guide:

1. AI-generated code carries real threats:
   • Prompt injection, improper dependency usage, hidden flaws — these are just a few.
   • Documented breach cases in the last 24 months show that the problem is serious and requires immediate action.
2. Industry leaders like Red Hat, OWASP, and NIST provide proven guidelines:
   • Red Hat recommends a "Secure by Design" approach and integrating AI with DevSecOps.
   • OWASP has developed the Top 10 for LLM-based applications, which serves as a checklist for companies.
   • NIST AI RMF 1.0 is a risk management framework that helps companies identify and mitigate AI-related threats.
3. Control tools are available, but they are not a magic solution:
   • SAST, DAST, SCA, and RASP are key tools, but none of them will provide full protection on their own.
   • Companies must adopt a multi-layered approach, combining automated scanning with manual audits and continuous monitoring.
4. Balancing speed and security is possible:
   • By applying proven strategies, such as limiting prompt context, automated code scanning, and manual reviews, companies can use AI without exposing themselves to serious risks.
5. Every sector has different requirements, but the core principles remain the same:
   • The financial sector focuses on customer data protection and regulatory compliance.
   • Healthcare emphasizes medical data protection and HIPAA compliance.
   • The public sector focuses on protecting critical infrastructure and AI decision transparency.

AI is a powerful tool that can accelerate the software development process and increase innovation. However, its use must be responsible and based on solid security foundations. Companies that decide to implement AI should focus on control, transparency, and continuous improvement — because only then will they be able to enjoy the benefits of AI while minimizing risk.

As Red Hat summarizes in a recent blog post: "AI is neither good nor evil — it is a tool that can work both for us and against us. Everything depends on how we use it." [source]

We hope this guide helps you safely and effectively implement AI in your organization. Remember: security is not an obstacle, but the foundation upon which lasting success is built.

Sources

• https://www.redhat.com/en/blog/ai-code-paradox-moving-fast-without-breaking-security
• https://www.redhat.com/en/blog/moon-and-beyond-ramalama-being-tested-nasa-potentially-support-medical-ai-assistant-future-deep-space-missions
• https://www.redhat.com/en/blog/sit-stay-deploy-lessons-real-world-robotic-blueprint-scaling-edge-computer-vision
• https://owasp.org/www-project-top-10-for-large-language-model-applications/
• https://snyk.io/reports/open-source-security/
• https://arxiv.org/abs/2402.07824
• https://www.nist.gov/itl/ai-risk-management-framework
• https://www.cisa.gov/news-events/alerts/aa23-160a
• https://snyk.io/vuln/
• https://owasp.org/www-project-dependency-track/
• https://www.sonarsource.com/
• https://www.gartner.com/en
• https://www.redhat.com/en/resources/secure-coding-practices
• https://www.forrester.com/