The security of language models is a critical topic for business today. When OpenAI decides to publish the results of external cybersecurity audits, the industry immediately asks: do the discovered vulnerabilities pose a threat to companies using AI? We examine what these analyses really show and what steps should be taken to safely implement artificial intelligence in your organization.
OpenAI Audits: Who and How Evaluates Model Security?
As part of external security evaluations, OpenAI regularly commissions independent entities to assess its language models. These audits involve specialized firms and research organizations, such as:
- NCC Group – a global consultancy known for advanced penetration testing and cloud infrastructure assessment.
- Trail of Bits – a US-based organization with extensive experience in security audits of AI systems and cryptography.
- Independent research teams – including experts focused on verifying models for vulnerabilities and compliance with security standards.
Criteria for selecting auditors typically include documented experience in evaluating large language models (LLMs) and the absence of conflicts of interest. These studies often align with recognized standards, such as NIST guidelines for secure software development lifecycles.
It is worth noting that while AI developers are increasing their cooperation with regulatory bodies – especially in the context of implementing the European AI Act – these technical audits are primarily commercial and expert-driven, aimed at hardening systems against real-world threats.
What Was Checked? Scope of Audits and Key Threats
Security analyses typically focus on several key areas:
- Model vulnerability to attacks – including prompt injection (injecting malicious instructions), data poisoning (attempts to contaminate training data), and jailbreaking (bypassing built-in security filters).
- Cloud infrastructure security – verification of the security of platforms where models are hosted and the data encryption protocols used.
- Privacy and user data protection – compliance with regulations such as GDPR, data retention policies, and the effectiveness of anonymization mechanisms.
- API security – control of authorization, authentication, and real-time abuse detection systems.
During testing, auditors combine documentation analysis with attack simulations (so-called red-teaming). This involves both proprietary scripts and open-source tools, such as garak (for automated vulnerability detection in LLMs) or LLMFuzzer.
One of the most difficult challenges remains protection against indirect prompt injection. This is an attack where the model retrieves malicious instructions not directly from the user, but from external sources, such as a scanned website or a PDF document.
Audit Results: What Was Discovered and How Do AI Developers Respond?
Audit reports rarely reveal critical vulnerabilities that allow for total control over models, but they regularly point out areas requiring improvement. The most common challenges include:
- High-priority vulnerabilities:
- Indirect prompt injection – models can still be susceptible to manipulation via external data. In response, AI providers are constantly developing additional filters at the API level.
- Model inversion and data extraction – the risk that with sufficiently persistent queries, a model might reveal fragments of the data it was trained on. Newer model versions are significantly better protected against this.
- Medium and low-priority vulnerabilities:
- Jailbreaking – creative users still find ways to bypass barriers (e.g., through role-playing or complex hypothetical scenarios). This requires continuous updates to filters and rule bases.
- API configuration issues – e.g., the need for stricter enforcement of multi-factor authentication (MFA) for organizational accounts.
In response to these challenges, OpenAI is developing its security framework (such as the Preparedness Framework) and running Bug Bounty programs, offering financial rewards for reporting newly discovered vulnerabilities.
Recommendations for Companies: How to Implement AI Safely?
Model security on the provider's side is only half the battle. Companies implementing AI should ensure their own procedures are in place:
- "Defense in depth" principle – do not rely solely on OpenAI's security. Use your own input and output filters, monitor API traffic, and encrypt sensitive data.
- Team education – employees must understand what prompt injection is and why confidential data or trade secrets should not be entered into public models.
- Monitoring and auditing queries – log interactions with models (ensuring data anonymization in accordance with GDPR). Log management systems and compliance-supporting platforms can be helpful.
- Access control (RBAC) – grant permissions to use API keys only to those employees and systems that truly require them.
- Regular testing of your own applications – applications integrated with LLMs should undergo periodic security testing, especially after deploying new features.
It is worth relying on proven standards, such as the NIST AI Risk Management Framework or the OWASP Top 10 for LLM Applications guidelines.
Is OpenAI's Approach Unique? A Market Overview
Transparency regarding security is slowly becoming a market standard, although individual players approach it differently:
- Google – invests heavily in its own research teams (e.g., Google DeepMind Safety Team) and collaborates with external risk assessment organizations (like METR), although detailed technical reports are not always made public.
- Anthropic – has positioned itself from the start as a safety-first company, working closely with national AI safety institutes (e.g., in the US and UK).
- Open-source models (e.g., Meta, Mistral AI) – give companies full control over the infrastructure but also shift the responsibility for security and content filtering directly to the implementer.
The industry is still evolving toward uniform standards. Although organizations like Partnership on AI create general recommendations, regulatory and market pressure will be the main drivers of change in the coming years.
Limitations of Audits: What Cannot Be Easily Detected?
Even the most rigorous technical audit has its limits:
- Emergent threats – large language models may exhibit unforeseen behaviors only under specific, rare parameter combinations or in new usage contexts.
- Social impact and disinformation – technical vulnerability tests do not solve problems related to the generation of convincing but false information (hallucinations) or the risk of abuse in disinformation campaigns.
- Client-side implementation security – auditors evaluate the model and the provider's infrastructure, but have no influence on how securely a company integrates the model with its internal databases.
Many security experts emphasize that a one-time audit is not enough. In the world of rapidly developing AI, risk assessment must be a continuous process.
Impact of Audits on Business Trust
Access to audit results and clear security declarations is a key factor for highly regulated industries:
- Financial sector and banking – financial institutions are increasingly adopting LLM-based solutions, but they require isolated cloud environments and rigorous prompt filtering.
- Healthcare – medical implementations require absolute compliance with regulations regarding medical data (e.g., HIPAA). Publishing independent audits makes it easier for hospitals and medical facilities to assess legal risk.
- Administration and public sector – public institutions increasingly require AI providers to have compliance certificates and clear declarations regarding data sovereignty.
Despite progress in transparency, the debate continues within the expert community. Some specialists point out that published reports are often stripped of the most sensitive technical details, which hinders independent verification. Conversely, research organizations like the AI Now Institute emphasize that technical audits should go hand-in-hand with deeper analysis of the long-term social impacts of artificial intelligence.
Summary: Is AI Safe Yet?
External security audits show that AI technology is maturing and its providers are taking cybersecurity more seriously. This does not mean, however, that these systems are completely free of flaws. Companies planning to implement artificial intelligence should keep a few principles in mind:
- Security is a continuous process – it requires constant monitoring and adaptation to new types of threats.
- AI implementation requires client-side responsibility – proper API configuration and access control are crucial.
- It is worth using proven frameworks, such as the NIST AI RMF or OWASP guidelines, to systematically manage risk.
Provider transparency and regular audits are a step in the right direction, but the ultimate security of an implementation depends on the maturity of the organization's own procedures.
Sources
- https://openai.com/index/third-party-cyber-evaluations-involving-openai-models
- https://openai.com/index/third-party-cyber-evaluations/
- https://atlas.mitre.org/
- https://openai.com/blog/bug-bounty-program
- https://www.reuters.com/business/finance/jpmorgan-expands-openai-use-after-security-audit-2026-04-15/
- https://www.mayoclinic.org/ai-initiative
- https://digital-strategy.ec.europa.eu/en/policies/ai-office
- https://www.nist.gov/itl/ai-risk-management-framework
Comments