Defend inputs, outputs, identities, data, models, tools, and decisions while operationalizing privacy, compliance, responsible AI, and human oversight.
Layer validation, prompt-injection defenses, content controls, output schemas, grounding, policy checks, and safe failure behavior.
Apply identity, encryption, private connectivity, tenancy boundaries, minimization, masking, retention, and traceable access.
Define ownership, inventory, approval, evidence, monitoring, change control, exception handling, and regulatory mapping.
Evaluate fairness, harmful behavior, transparency, explainability, human oversight, misuse, and affected-user consequences.
The model is not a security boundary. It can interpret instructions and produce useful classifications or drafts, but deterministic systems must enforce identity, authorization, data access, transaction rules, budgets, and prohibited actions. Safety is a layered system spanning the user interface, input processing, retrieval, prompt construction, model, tools, output validation, monitoring, governance, and incident response.
Threat-model the entire data and action flow. Identify trusted and untrusted inputs, tenant boundaries, privileged tools, external content, data stores, model providers, logs, reviewers, and downstream consumers. Then assign each control to the layer that can actually enforce it.
| Objective | Required judgment | Proof |
|---|---|---|
| 3.1 Implement input and output safety | Combine validation, injection defenses, content controls, grounding, schemas, policy, and safe failure | A representative attack set cannot trigger prohibited disclosure or action |
| 3.2 Protect data security and privacy | Apply identity, encryption, private paths, minimization, masking, tenancy, retention, and access evidence | Protected data is unavailable across unauthorized identities and tenants |
| 3.3 Implement AI governance and compliance | Establish inventory, owners, approvals, evidence, monitoring, change, incidents, exceptions, and mapping | Each production use case has accountable lifecycle records and current controls |
| 3.4 Apply responsible AI | Evaluate fairness, harmful behavior, transparency, explainability, oversight, misuse, and affected users | Known impact groups and failure modes have tests, thresholds, and response owners |
Classify inputs by origin and trust. User text, uploaded files, retrieved documents, websites, emails, tool results, and memory can all contain direct or indirect prompt injection. Delimit untrusted content and instruct the model to treat it as data, but do not rely on prompt wording alone. Limit what content is retrieved, which tools are available, what identities they use, and which actions require approval.
Validate file type, size, encoding, structure, malware where applicable, and extraction behavior. Remove or mask unnecessary sensitive data before model invocation. Use an allowlist for supported content and reject ambiguous parser states. Track provenance so output can be connected to its input and source version.
Amazon Bedrock Guardrails can apply supported content, denied-topic, sensitive-information, and grounding-related policies in designed flows. A guardrail does not replace IAM, tenant filtering, tool authorization, or application-specific correctness checks. Test guardrail versions on representative normal and adversarial traffic, measure both misses and false positives, and define behavior when the guardrail intervenes or is unavailable.
Validate structure before using generated output. Parse into an explicit schema; reject extra fields, invalid values, unsupported citations, and malformed tool requests. Apply business rules and authorization after parsing. For factual answers, require supported evidence or a clear uncertainty response. For code, configuration, or security guidance, use review and testing appropriate to impact.
Separate content safety from action safety. A polite response can still request an unauthorized refund or data export. A harmful-looking string may be legitimate evidence in a security workflow. Evaluate the task, authenticated user, target resource, policy, and consequence in deterministic code. Require human approval for irreversible or high-impact changes and show reviewers enough context to make an informed decision.
Design safe failure. Do not expose system prompts, hidden context, credentials, policy internals, or another tenant's data in an error. Do not silently proceed when output validation fails. Give users a bounded explanation and a next safe step. Preserve privacy-safe evidence for investigation.
Use least-privilege service and workload roles. Restrict model invocation, knowledge-base access, data sources, vector stores, prompt resources, agents, tools, keys, logs, and deployment actions separately. Use private connectivity where the requirement warrants it and verify DNS and endpoint policies. Encrypt data in transit and at rest, but also minimize collection and access.
Partition tenant data before context assembly. Test cross-tenant retrieval, cache, memory, log, and tool paths. Decide whether prompts and model invocations may be logged; invocation logging can be valuable but may capture sensitive data. Redact or omit sensitive values and scope telemetry access. Define retention and deletion for prompts, outputs, embeddings, feedback, evaluation records, and backups.
Maintain an inventory of production AI use cases, owners, models, data sources, purposes, impacted users, risk tier, controls, evaluations, approvals, Regions, dependencies, and review dates. Record provider and model changes. Separate experimental access from production approval. An exception must identify risk owner, rationale, compensating controls, expiry, and retest.
Responsible AI is context-specific. Assess harmful content, stereotyping, disparate quality, accessibility, transparency, explainability needs, overreliance, misuse, and consequences for people affected by the output. Use representative datasets and human review where judgment is required. Average quality can hide severe failure for a small group or critical scenario, so maintain hard thresholds for unacceptable outcomes.
Tell users when they interact with AI where appropriate, identify limitations, and provide correction or escalation paths. Human oversight must be meaningful: reviewers need time, authority, evidence, and a usable way to reject or amend output. A rubber-stamp approval does not reduce risk.
| Risk | Enforce primarily in | Evidence |
|---|---|---|
| Cross-tenant retrieval | Identity, data partition, metadata filters, resource policies | Isolation tests with adversarial queries |
| Prompt injection | Context boundaries plus authorization and tool constraints | Direct and indirect injection suite |
| Harmful content | Guardrails, application policy, human review where needed | Safety metrics and reviewed cases |
| Invalid structured output | Schema parser and business validation | Rejected malformed and extra-field cases |
| Excessive agency | Scoped tool roles, budgets, approvals, stop conditions | Denied mutation and bounded-loop tests |
| Privacy leakage through logs | Minimization, redaction, access, retention | Telemetry inspection and access review |
A recruiting assistant summarizes résumés, ranks candidates, and drafts interview recommendations.
