AI security: how to protect the company’s AI journey
Risks, guardrails, agent protection, frameworks and a checklist for using generative AI securely, from inventory to monitoring in the SOC.
In short: AI security is the set of controls that protects a company’s artificial intelligence systems (models, generative AI applications, agents and the data they use) against attacks, leakage and misuse. It combines inventory, risk assessment, governance, guardrails, least privilege for agents and continuous monitoring, with references such as OWASP, MITRE ATLAS, ISO/IEC 42001 and the NIST AI RMF.
What is AI security?
AI security, or artificial intelligence security, protects a company’s AI systems against attacks and misuse. It covers four targets.
The risks are new because a language model does not separate instructions from data: a sentence hidden in a document can become a command (OWASP). An agent acts with the permissions it has been given. And data sent to an AI app with no contract leaves the reach of the company’s policies.
AI protection and secure AI are other names for the same goal: using AI with known, controlled risk.
AI security, AI governance and AI-powered security: what is the difference?
The three terms are often confused: AI security protects the company’s AI, governance decides how it is used and AI-powered security uses AI to protect the company, as a SOC does.
| Aspect | AI security | AI governance | AI-powered security |
|---|---|---|---|
| What it is | Protecting models, applications, agents and data against attacks and misuse | Deciding how the company uses AI: policies, roles, risk and compliance | Using AI to defend the company, as in alert triage |
| Examples | Guardrails, DLP for AI, least privilege for agents, red teaming | Committee, acceptable use policy, inventory, risk classification | AI triage, case summaries, natural-language investigation |
| At Vortex | AI Journey Assessment and the Protection pillar | AI Journey Assessment, CAIO as a Service and the governance platform | Cyber Defense Center, the Converged NG-SOC (NG-SOC Convergente) |
This guide covers AI security and governance. AI used by the SOC is covered in the AI in the SOC guide and on the Cyber Defense Center page. In medicine, the rules of the CFM (Brazil’s Federal Council of Medicine) for the use of AI are covered in the guide to Resolução CFM nº 2.454/2026 (PT).
What are the main AI security risks, and how do you control each one?
LLM and generative AI security risks follow the codes of the OWASP Top 10 for LLM Applications 2026. No single control solves them: according to OWASP, there is still no reliable way to prevent prompt injection, and the defense is to limit what an injection can do.
| Risk | How it happens | Controls | Guardrails |
|---|---|---|---|
| Direct prompt injection and jailbreaking (LLM01) | The user tries to change the model’s behavior or make it violate its safety rules | Role and limits in the system instructions; red teaming | Input filter against injection and jailbreaking |
| Indirect prompt injection (LLM01) | Instructions hidden in a document, email, web page or tool response that the model reads | External content treated as untrusted; least privilege | Source label on external content; human approval for sensitive actions |
| Sensitive information disclosure (LLM02) | Personal or strategic data goes into a prompt or upload, or comes back in a response | Data classification; only what is needed goes to the provider; search with the permissions of the person asking | DLP for AI on input and output |
| Excessive agency (LLM03) | The agent has more tools, permissions or autonomy than the task requires | Minimal tools; least privilege; the agent’s own identity | Policy that validates each action; human approval for anything irreversible |
| Supply chain (LLM04) | A third-party model, library, dataset or MCP server arrives tampered with or vulnerable | Component inventory (AI BOM); pinned and verified versions | Allowlist of models and tools |
| Data and model poisoning (LLM05) | Training, fine-tuning or knowledge base data is tampered with | Data provenance and integrity; control over who writes to the knowledge base and to memory | Approval before writing to the agent’s memory |
| Improper output handling (LLM10) | The response becomes a query, command or code without being checked | Output validation in code, outside the model | Format checking; sanitization of links and commands |
| Shadow AI | Use of AI apps without approval, outside the inventory and the controls | Usage discovery; acceptable use policy; an approved option | Blocking of, or alerts on, unapproved apps |
The OWASP list also includes unbounded consumption (LLM06), misinformation (LLM07), hidden context exposure (LLM08) and vector and embedding weaknesses (LLM09). Shadow AI, in the last row, is not on it.
What are AI guardrails, and where do they sit?
Guardrails are technical controls around the model that limit what goes in, what comes out and what the AI can do. They sit outside the model, in the application code, in an AI gateway or in the platform, and enforce the company’s policies on every request.
None is perfect, because language filters can be bypassed. OWASP recommends layers and a design in which a successful injection does not turn into an attack.
Input
What reaches the model?
Input guardrailsPrompt, file, web page, email or tool response.
Guardrails at this point
- Prompt injection and jailbreak detection
- DLP for AI: sensitive data masked or blocked
- Source label on external content
Model
What rules does it work under?
Policies and limitsThe system instructions define the model’s role.
Guardrails at this point
- Declared role and restrictions
- No secrets in the instructions
- Only approved models, with usage quotas
Output
What can the response contain?
Output guardrailsThe response is checked before it reaches a person or another system.
Guardrails at this point
- Format validation in code
- Blocking of sensitive data and prohibited content
- Sanitization of links, code and commands
Action
What can the agent execute?
Action limitsEvery call goes through permission and policy checks.
Guardrails at this point
- Minimal tools and least privilege
- Human approval for anything irreversible
- Value and volume limits, with logging
How do you protect AI agents?
When the model becomes an agent, with tools, memory and actions in other systems, OWASP recommends adding the Top 10 for Agentic Applications, with risks such as agent goal hijack (ASI01), tool misuse and exploitation (ASI02) and identity and privilege abuse (ASI03).
How do you protect the AI journey, step by step?
Protection follows the use of AI and works as a cycle: each round measures again and updates the inventory.
- Step 1 What AI is in use here?
Discover and inventory
List the models, applications, agents and AI apps in use, including unapproved ones, and the data each one accesses.
AI inventory - Step 2 What could go wrong?
Assess the risk
Classify each use case by the data, the decision it influences and what it executes, with a framework such as the NIST AI RMF.
Risk classification - Step 3 Who decides?
Govern
Define a committee, roles, an acceptable use policy and an approval workflow, with evidence for audits. ISO/IEC 42001 organizes this management system.
AI policy - Step 4 Which controls are missing?
Protect
Apply guardrails, DLP for AI, least privilege for agents and red teaming, starting with the highest-risk cases.
Guardrails - Step 5 Are the controls working?
Monitor and measure again
Send AI telemetry to the SOC, correlated with IT and OT, respond through the playbooks and repeat the assessment.
Telemetry in the SOC
Which frameworks and standards should you use, and what does each one cover?
ISO/IEC 42001 and the NIST AI RMF organize governance; OWASP and MITRE ATLAS describe attacks and controls; the EU AI Act and Brazil’s LGPD (General Data Protection Law) are binding; PL 2338/2023, Brazil’s AI bill, is still going through Congress. Status checked against the official sources on November 2, 2026.
| Reference | What it covers | Is it mandatory? | Version and status |
|---|---|---|---|
| ISO/IEC 42001 | Requirements for an AI management system | No; certification is voluntary | December 2023 edition |
| NIST AI RMF | AI risk in four functions: Govern, Map, Measure and Manage | No | Version 1.0, of January 26, 2023, being revised by NIST |
| NIST AI 600-1 | Generative AI profile: 12 risks, such as confabulation and data privacy, with suggested actions | No | Published on July 26, 2024 |
| EU AI Act (Regulation (EU) 2024/1689) | Obligations by risk level for those who provide or use AI in the European Union, including from outside it | Yes, for those in scope | In force since August 1, 2024; Regulation (EU) 2026/1744 postponed the high-risk obligations to December 2, 2027 (Annex III) and August 2, 2028 (Annex I) |
| PL 2338/2023 | Brazil’s AI legal framework: risk levels, algorithmic impact assessment for high risk and a national system coordinated by the ANPD (Brazil’s National Data Protection Authority) | Not yet law | Approved by the Federal Senate on December 10, 2024; in the Chamber of Deputies since March 17, 2025, awaiting the rapporteur’s report in the special committee |
| LGPD (Lei 13.709/2018) | Personal data in training, prompts and decisions: security (art. 46), review of automated decisions (art. 20) and incidents (art. 48) | Yes, with personal data | In force; incidents reported within three business days (Resolução CD/ANPD nº 15/2024, an ANPD regulation) |
| OWASP Top 10 for LLM Applications | The ten most critical risks of LLM applications, with prevention | No | 2026 version, August 2026 |
| OWASP Top 10 for Agentic Applications | Ten risks of autonomous agents, from ASI01 to ASI10 | No | 2026 edition, December 9, 2025 |
| MITRE ATLAS | Tactics and techniques for attacking AI systems, modeled on MITRE ATT&CK | No | Open knowledge base, updated by MITRE |
Sources: ISO, certification, NIST AI RMF, NIST AI 600-1, European Commission, AI Act, Regulation (EU) 2026/1744, Chamber of Deputies (PT), Federal Senate (PT), LGPD (PT), ANPD (PT), OWASP LLM, OWASP agents and MITRE ATLAS.
Where to start? AI security checklist
Use the list to check what is already in place and what is missing.
- Inventory the models, applications, agents and AI apps, approved and unapproved, and the data each one accesses.
- Appoint an owner and publish an acceptable use policy: approved tools and data that must not go into AI.
- Classify use cases by risk and start with the highest-impact ones.
- Turn on DLP for AI in prompts, uploads and responses, and offer an approved option to those using shadow AI.
- Put input and output guardrails in your own applications and validate output in code.
- Give each agent its own identity, least privilege and human approval for anything irreversible.
- Include models, datasets and MCP servers in the supplier inventory.
- Run red teaming before going live and periodically after that.
- Send prompt, response and action logs to the SOC, without storing sensitive data unnecessarily.
- Check the LGPD for every use involving personal data, and keep track of the EU AI Act and PL 2338/2023.
How does Vortex protect the AI journey?
The Vortex AI Journey is Vortex Security’s offer for AI governance and security, in three pillars. A company can start with any of them; the recommended starting point is the diagnostic.
Vortex is a Leader in the ISG Provider Lens 2026 for Next-Gen SOC/MDR Services (Midmarket, Brazil). For hospitals and healthcare networks, see also the page on cybersecurity in healthcare (PT).
Frequently asked questions
What are AI guardrails?
They are technical controls around the model that limit what goes in, what comes out and what the AI can do: input filters, output validation and action limits for agents, with human approval for anything irreversible. They sit outside the model, in code, in a gateway or in the platform.
What is the difference between AI security and AI governance?
AI security protects models, applications, agents and data against attacks and misuse. Governance decides how the company uses AI: policies, roles, inventory, risk and compliance. Governance says what to protect; security enforces it.
What is prompt injection?
It is when text that reaches the model changes its behavior. In direct injection, the user writes the instructions, and jailbreaking tries to make the model violate its safety rules. In indirect injection, the instructions come hidden in a document, email, web page or tool response. It is risk LLM01 in the OWASP Top 10 for LLM Applications.
What is shadow AI?
It is the use of AI tools without the company’s approval, outside the inventory and the controls. The main risk is data leakage. The way forward is to discover the usage, offer an approved option, publish a policy and apply DLP for AI.
How do you protect AI agents?
By treating each agent as a privileged user: its own identity, minimal tools and least privilege. A policy outside the model validates each action, anything irreversible waits for human approval, memory is protected and everything is logged and monitored by the SOC.
Is ISO/IEC 42001 mandatory?
No. It is an international standard for AI management systems, from 2023, with voluntary certification (ISO), which customers and contracts may require. Legal obligations come from laws such as the LGPD and, for those operating in Europe, the EU AI Act.
Is PL 2338/2023 already law?
No. Brazil’s AI legal framework bill was approved by the Federal Senate on December 10, 2024 and has been in the Chamber of Deputies since March 17, 2025, awaiting the rapporteur’s report in the special committee (Chamber of Deputies (PT), November 2, 2026). The LGPD already applies to every use of AI involving personal data.
What is AI red teaming?
It is testing that simulates attacks on an AI system, such as prompt injection, jailbreaking, data extraction and tool abuse, to find flaws before an attacker does. NIST AI 600-1 describes the practice, both before and after the system goes into use.
Where do you start with AI security?
With the inventory: which models, applications, agents and AI apps the company uses, approved or not, and what data they access. Then classify the risk, define a policy and owners, and protect the highest-impact cases first. At Vortex, the recommended starting point is the AI Journey Assessment.
Sources
Pages consulted on November 2, 2026.
- OWASP Top 10 for LLM Applications 2026, August 2026.
- OWASP Top 10 for Agentic Applications for 2026, December 9, 2025.
- MITRE ATLAS.
- NIST AI Risk Management Framework, AI RMF 1.0 (NIST AI 100-1), January 26, 2023.
- NIST AI 600-1: Generative Artificial Intelligence Profile, July 26, 2024.
- ISO/IEC 42001:2023 and ISO/IEC 42001 explained, ISO.
- AI Act: regulatory framework on AI, European Commission.
- Regulation (EU) 2024/1689 and Regulation (EU) 2026/1744, EUR-Lex.
- PL 2338/2023 in the Chamber of Deputies (PT), legislative tracking page.
- PL 2338/2023 in the Federal Senate (PT) and news of its approval (PT), Agência Senado, December 10, 2024.
- Lei nº 13.709/2018 (LGPD) (PT), Planalto, the Brazilian Presidency’s legislation portal.
- Security incident reporting (PT), ANPD.
Want to know where the AI risks are in your company?
Tell us where your company stands with AI and a Vortex expert will get in touch.
Updated on