100 Security for AI Apps resources for developers

An open-source multi-layered self-hosted prompt injection detector that uses heuristics, vector lookups, and LLM-based analysis.

NVIDIA's toolkit for defining programmable guardrails using Colang to ensure LLM interactions stay within predefined safety boundaries.

A framework for adding structural, type, and quality validation to LLM outputs using Pydantic-style schemas.

An enterprise-grade API for real-time protection against prompt injection, jailbreaking, and PII leakage.

A CLI tool and library for testing prompt quality and security by running test cases against multiple models simultaneously.

An LLM vulnerability scanner that probes models for known weaknesses including hallucination, jailbreaks, and data extraction.

Microsoft's Python Risk Identification Tool for red teaming generative AI systems to identify security and fairness risks.

A lightweight tool for scanning prompt templates for common injection patterns before deployment.

A security proxy layer that sits between your application and the LLM provider to filter malicious payloads.

A security scanner for LLM prompts that detects prompt injection and jailbreak attempts using a signature-based approach.

An open-source data protection SDK that provides PII identification and anonymization for text and images.

A comprehensive tool for both input and output sanitization, including PII detection, toxicity filtering, and secret detection.

A high-accuracy PII redaction API that supports over 50 languages and can be deployed on-premise to maintain data residency.

A data privacy vault that allows developers to isolate, protect, and govern sensitive customer data used in LLM workflows.

Built-in document transformers for LangChain to scrub sensitive data from documents before they are indexed in vector stores.

An API for encrypted LLM processing that ensures the model provider never sees the raw underlying data.

Transparent application-layer encryption for data stored in vector databases like Pinecone or Weaviate.

Managed service to filter toxic content and PII from inputs and outputs specifically for models hosted on AWS Bedrock.

A simple, fast, and safe file format for storing tensors that prevents arbitrary code execution during model loading.

Implementation pattern using hashing or tokenization for document metadata to prevent LLM reconstruction of user identities.

Manage and rotate LLM API keys dynamically to prevent long-lived credential exposure in environment variables.

Encrypting high-dimensional vectors in databases using unique keys to ensure data at rest is unreadable without specific IAM roles.

An egress proxy that provides visibility, rate limiting, and security filtering for all outbound LLM API requests.

Role-Based Access Control for vector namespaces to ensure users only query segments of the vector database they are authorized to access.

Detailed tracing and logging of all LLM inputs and outputs to provide an audit trail for compliance and security forensics.

Static analysis tool configured to detect insecure patterns in LLM orchestration code, such as unsanitized prompt templates.

Zero-trust networking to secure communications between application servers and private self-hosted GPU inference nodes.

Restricting LLM application pods to only communicate with verified model provider endpoints (e.g., api.openai.com).

Automated detection of leaked OpenAI, Anthropic, and Hugging Face tokens within private and public repositories.

Vulnerability scanning for container images containing AI runtimes and libraries like PyTorch or TensorFlow.