Draw the data flow first
Map user input, retrieved documents, model requests, tool arguments, logs, and caches. Mark the data class, retention period, and visible role at every node. Without this map it is hard to know what can reach an external provider.
Prompt injection, unauthorized retrieval, sensitive echo, poisoned tool arguments, and log leakage are common risks. Put controls at code boundaries: filter ACLs before retrieval, validate arguments and permissions before tool calls, and inspect output for sensitive data.
Keep secrets out of prompts
API keys, session tokens, and internal configuration cannot be secured by asking a model to keep them secret. Store secrets in restricted server configuration and mask or hash logs. Send only necessary fields to a provider and confirm its retention and training policy.
Regress against red-team samples
Keep samples for unauthorized questions, indirect injection, malicious links, context flooding, and data exfiltration. Run them after every model or retriever upgrade and record block rate, false positives, and human review. A security incident should be traceable to request, tool, and policy versions.
Sources
See the OWASP LLM Top 10, NIST AI RMF, and Google SAIF.
