Patterns
Data Protection
GDPR · SOC 2 · Privacy by Design

Data Protection

Masking, Anonymization, Tokenization, Pseudonymization, PII Redaction, and Synthetic Data Generation — with pre/post LLM hooks and agent-native implementation patterns.

Data Protection & Privacy Techniques in Agentic AI Pipelines

Masking, Anonymization, Tokenization, Pseudonymization, PII Redaction, and Synthetic Data Generation — with pre/post LLM hooks and agent-native implementation patterns.


Why This Problem Exists in Agentic Systems

In traditional software, PII flows through systems you own, on infrastructure you audit. In agentic AI pipelines, something changes: data enters the context window of a third-party model. That means raw customer records, financial transactions, healthcare notes, or identity fields may be processed by an external LLM — potentially logged, retained, or used for fine-tuning depending on the provider’s data handling policies.

GDPR Article 5 requires data minimization — collecting and processing only what is strictly necessary. SOC 2 Type II requires demonstrating access controls, confidentiality, and audit trails over sensitive data. Neither regulation was written with LLM context windows in mind, but both apply.

The practical consequence: you cannot send raw PII to an LLM without data protection controls in place. The solution is a pipeline-level privacy layer — a set of agent skills or script hooks that sanitize data before the LLM sees it and restore or reconcile outputs after the fact.


The Agent Pipeline Privacy Architecture

The core pattern is a sanitize → infer → restore loop. It wraps every LLM call that touches sensitive data.

[Raw Input with PII]
        │
        ▼
┌──────────────────────┐
│  PRE-LLM HOOK        │  ← Agent Skill / Script
│  (Sanitization Layer)│
│  - Detect PII        │
│  - Apply technique   │
│  - Build vault/map   │
└──────────┬───────────┘
           │ [Sanitized Input — no raw PII]
           ▼
┌──────────────────────┐
│  LLM CALL            │
│  (Claude / GPT / etc)│
└──────────┬───────────┘
           │ [LLM Output with placeholders/tokens]
           ▼
┌──────────────────────┐
│  POST-LLM HOOK       │  ← Agent Skill / Script
│  (Restoration Layer) │
│  - Re-inject values  │
│  - Audit trail log   │
│  - Validate output   │
└──────────┬───────────┘
           │
           ▼
[Safe Output — PII re-introduced only where allowed]

Which technique you apply in the pre-hook determines whether the post-hook can restore anything at all — and that choice is a compliance decision, not just a technical one.


The Five Core Techniques


1. Masking ↩️ Reversible

What it is: Replace sensitive field values with a structurally consistent placeholder. The format is preserved, but the original value is hidden.

Mechanics:

  • Name: John Doe████ ███
  • Credit Card: 4111 1111 1111 1234**** **** **** 1234
  • Email: john.doe@corp.comj***.d**@corp.com
  • Date of birth: 1985-04-12****-04-** (partial masking for analytics)

Masking can be static (a fixed output per input, reproducible) or dynamic (different mask per call, non-reproducible). Dynamic masking is stronger from a privacy standpoint; static masking enables consistent downstream lookups.

As an Agent Pre-Hook:

import re

MASKING_RULES = {
    "email":   (r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}", "***@***.***"),
    "phone":   (r"\b\d{10}\b", "**********"),
    "ssn":     (r"\b\d{3}-\d{2}-\d{4}\b", "***-**-****"),
    "cc":      (r"\b\d{4}[\s-]\d{4}[\s-]\d{4}[\s-](\d{4})\b", r"****-****-****-\1"),
}

def mask_pii(text: str) -> str:
    for field, (pattern, replacement) in MASKING_RULES.items():
        text = re.sub(pattern, replacement, text)
    return text

# Usage in agent skill
sanitized_prompt = mask_pii(raw_user_input)
llm_response = call_llm(sanitized_prompt)
# No restoration needed — mask was intentional

Reversibility: Conditionally reversible (only if the mapping is stored; otherwise irreversible).

GDPR Compliance:

  • Qualifies as pseudonymization if a mapping is retained (Art. 4(5)).
  • Qualifies as anonymization if no mapping exists and re-identification is computationally infeasible.
  • Reduces GDPR risk without fully removing data from scope.

SOC 2 Implication: Supports confidentiality controls by limiting exposure of PII fields in non-production environments, logs, and LLM context windows.

Best for: Dev/test environments, non-production LLM evaluations, audit logs, analytics pipelines where format preservation matters.


2. Anonymization 🔒 Irreversible

What it is: The irreversible removal or transformation of personal data such that the individual cannot be re-identified — directly or indirectly — even with additional data sources. Under GDPR, fully anonymized data ceases to be personal data.

Mechanics:

  • Generalization: Age 34Age 30–39, City: MumbaiRegion: West India
  • Suppression: Remove the field entirely — no placeholder, no value
  • Data Swapping: Shuffle values across records so they’re real-looking but no longer tied to the individual
  • Noise Addition: Add statistical perturbation to numerical fields to prevent precise inference

The Re-identification Risk Test (GDPR Recital 26): Anonymization must account for “all means reasonably likely to be used” for re-identification. A dataset with age, zip code, and gender alone can re-identify ~87% of the US population (Sweeney, 2000). True anonymization requires accounting for quasi-identifiers, not just direct identifiers.

As an Agent Pre-Hook:

from typing import Any

def anonymize_record(record: dict) -> dict:
    """
    Removes direct identifiers and generalizes quasi-identifiers.
    This is a one-way transformation — no vault, no key.
    """
    DIRECT_IDS = {"name", "email", "ssn", "phone", "address", "ip_address"}
    
    clean = {k: v for k, v in record.items() if k not in DIRECT_IDS}
    
    # Generalize age to bucket
    if "age" in clean:
        age = int(clean["age"])
        clean["age_band"] = f"{(age // 10) * 10}s"
        del clean["age"]
    
    # Generalize zip to region prefix
    if "zip" in clean:
        clean["region_prefix"] = clean["zip"][:3] + "**"
        del clean["zip"]
    
    return clean

# Post-hook: nothing to restore — data is permanently anonymized

Reversibility: None. Irreversible by design.

GDPR Compliance:

  • Anonymized data falls outside GDPR scope (Recital 26).
  • Frees the organization from most GDPR obligations for that data.
  • Highest privacy protection achievable, but lowest data utility.

SOC 2 Implication: Effectively eliminates PII from the system, reducing scope for data classification, access controls, and audit requirements.

Best for: Training data preparation, public-facing analytics, research datasets, any case where re-use of specific individual data is never required.


3. Tokenization ↩️ Reversible

What it is: Replace a sensitive value with a randomly generated, meaningless token. The original value is stored in a secure token vault and can be retrieved by authorized systems only.

Mechanics:

  • 4111 1111 1111 1234tok_7f3a9c2e1b84
  • john.doe@corp.comtok_e2c8f1a97b30
  • PAN: ABCDE1234Ftok_3d91b7c4e058

Tokens are opaque — they carry no structural resemblance to the original and cannot be reverse-engineered. The vault is the only authoritative mapping. This is the dominant pattern in payment tokenization (PCI DSS) and is increasingly applied to general PII fields.

Vault Architecture:

import secrets
import hashlib

class TokenVault:
    """
    In-memory vault for demo. Production: HSM-backed KMS or Vault by HashiCorp.
    """
    def __init__(self):
        self._store: dict[str, str] = {}  # token → value
        self._index: dict[str, str] = {}  # hash(value) → token (for dedup)

    def tokenize(self, value: str) -> str:
        value_hash = hashlib.sha256(value.encode()).hexdigest()
        if value_hash in self._index:
            return self._index[value_hash]  # same value → same token (format-stable)
        token = f"tok_{secrets.token_hex(8)}"
        self._store[token] = value
        self._index[value_hash] = token
        return token

    def detokenize(self, token: str) -> str:
        if token not in self._store:
            raise ValueError(f"Unknown token: {token}")
        return self._store[token]

vault = TokenVault()

# Pre-LLM hook
def tokenize_prompt(text: str) -> tuple[str, list[str]]:
    tokens_used = []
    for pii in extract_pii(text):      # your PII detector here
        token = vault.tokenize(pii)
        text = text.replace(pii, token)
        tokens_used.append(token)
    return text, tokens_used

# Post-LLM hook
def detokenize_response(response: str, tokens: list[str]) -> str:
    for token in tokens:
        original = vault.detokenize(token)
        response = response.replace(token, original)
    return response

Reversibility: Fully reversible — by design, under controlled access.

GDPR Compliance:

  • Tokenized data is pseudonymized (Art. 4(5)) — the mapping (vault) holds the re-identification key.
  • Still considered personal data under GDPR; data subject rights (access, erasure) still apply.
  • Significantly reduces processing risk and supports Data Protection by Design (Art. 25).
  • Vault access logs support accountability (Art. 5(2)).

SOC 2 Implication: Enforces least-privilege data access — only vault-authorized processes can see real values. Supports confidentiality and availability principles.

Best for: Payment data (PCI DSS), API-level PII exchange, any case where the LLM needs to reference a value but not know what it is.


4. Pseudonymization ↩️ Reversible

What it is: Replace identifying attributes with pseudonyms — artificial identifiers — while maintaining a key or additional data that enables re-identification. The key is stored separately, with restricted access.

Mechanics:

  • Replace user_id: john_doe_2847 with pseudo_id: usr_9f2c
  • Replace medical record owner with a study code
  • Replace employee name with an internal reference number

Pseudonymization is a category, not a single technique. Tokenization is one form of pseudonymization. Deterministic encryption, hash-based replacement, and key-coded aliases are others.

Key Difference from Anonymization: The mapping exists. Under GDPR, if anyone with access to the key can re-identify the subject, the data remains personal data.

As an Agent Skill Pattern:

import hmac, hashlib, os

PSEUDO_SECRET = os.environ["PSEUDO_HMAC_KEY"]  # stored in secrets manager, never hardcoded

def pseudonymize(value: str, namespace: str = "default") -> str:
    """
    Deterministic pseudonymization via HMAC-SHA256.
    Same input always produces the same pseudonym — enables cross-record joins.
    Secret key must be protected — loss of key = permanent pseudonymization.
    """
    raw = f"{namespace}:{value}".encode()
    digest = hmac.new(PSEUDO_SECRET.encode(), raw, hashlib.sha256).hexdigest()
    return f"pseudo_{digest[:12]}"

# Determinism enables analytics:
# pseudonymize("john.doe@corp.com") → "pseudo_e3f1a7c92b40" (always)
# Two records for same email → same pseudonym → joinable without revealing email

# Pre-LLM hook
def pseudonymize_payload(payload: dict) -> dict:
    PII_FIELDS = ["email", "name", "phone", "user_id"]
    return {
        k: pseudonymize(str(v)) if k in PII_FIELDS else v
        for k, v in payload.items()
    }

# Post-LLM hook: re-identification requires separate key lookup table
# maintained by a privileged service, not the LLM layer

Reversibility: Conditionally reversible — only with possession of the secret key or mapping table.

GDPR Compliance:

  • Pseudonymized data remains personal data subject to GDPR (Recital 26).
  • Reduces risk and is explicitly incentivized by GDPR as a risk-reduction measure (Art. 89).
  • Supports lawful processing for research and statistics with reduced safeguards.
  • Key separation — keeping the key in a different system from the pseudonymized data — is a critical control.

SOC 2 Implication: Supports access control and confidentiality. Audit trails on key access satisfy the accountability requirements.

Best for: Analytics pipelines that need to correlate records across time, research datasets, audit trails where re-identification must remain possible under strict governance.


5. PII Redaction 🔒 Irreversible

What it is: The permanent, irreversible removal of sensitive personal information from a document or record. Unlike masking (which preserves format) or tokenization (which preserves reference), redaction eliminates the data entirely — often represented visually as a black bar in documents.

Mechanics:

  • Patient Name: Sarah ConnerPatient Name: [REDACTED]
  • Remove or blacken fields in PDFs before sharing externally
  • Strip metadata from files (EXIF, document properties, author fields)
  • Blank out fields in database export for public data sharing

As an Agent Pre-Hook (using a detection + redaction pipeline):

import re
from dataclasses import dataclass

@dataclass
class RedactionResult:
    redacted_text: str
    redaction_count: int
    redacted_fields: list[str]

PII_PATTERNS = {
    "EMAIL":   r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}",
    "PHONE":   r"\b(?:\+91[\s-]?)?\d{10}\b",
    "AADHAAR": r"\b\d{4}\s\d{4}\s\d{4}\b",
    "PAN":     r"\b[A-Z]{5}[0-9]{4}[A-Z]{1}\b",
    "SSN":     r"\b\d{3}-\d{2}-\d{4}\b",
    "IP":      r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
    "NAME":    r"\b[A-Z][a-z]+ [A-Z][a-z]+\b",  # simplistic — use NER in production
}

def redact_pii(text: str) -> RedactionResult:
    count = 0
    fields_hit = []
    for label, pattern in PII_PATTERNS.items():
        matches = re.findall(pattern, text)
        if matches:
            text = re.sub(pattern, f"[{label} REDACTED]", text)
            count += len(matches)
            fields_hit.append(label)
    return RedactionResult(text, count, fields_hit)

# Usage
result = redact_pii(raw_document_text)
llm_response = call_llm(result.redacted_text)
# Post-hook: NOTHING to restore — redaction is permanent by design

Production-grade redaction uses Named Entity Recognition (NER) models (spaCy, Amazon Comprehend, Google Cloud Sensitive Data Protection, Microsoft Presidio) rather than regex alone, because regex cannot reliably detect names, organizations, or context-sensitive PII.

Reversibility: None. Permanent and irreversible.

GDPR Compliance:

  • Redacted data may qualify as anonymized if no re-identification pathway exists (Recital 26).
  • Commonly used to fulfill Right to Erasure (Art. 17) — redacting a subject’s data from records on request.
  • Required for public disclosures, Freedom of Information responses, and legal document sharing.

SOC 2 Implication: Eliminates PII from the data scope entirely. Ideal for satisfying breach disclosure requirements — if redacted data leaks, no PII is exposed.

Best for: Legal document publishing, external data sharing, FOIA responses, public-sector records, fulfillment of erasure requests.


Synthetic Data Generation

The Cleanest Option: Don’t Use Real Data at All

If the purpose of sending data to an LLM is to test a pipeline, evaluate a prompt, fine-tune a model, or generate synthetic examples — don’t use real records. Generate statistically faithful synthetic data that has the same schema, distribution, and edge-case coverage as production data, with zero actual PII.

Synthetic data generation is not a replacement for the above techniques in production flows, but it is often the best option for:

  • Prompt engineering and evaluation
  • QA and load testing of LLM-powered features
  • Fine-tuning or training on sensitive domains
  • Sharing test data across teams without compliance risk

Generation Approaches

1. Rule-Based / Library Synthesis

Fast, deterministic, no model required.

from faker import Faker
import random

fake = Faker("en_IN")  # locale-aware

def generate_synthetic_customer(n: int = 10) -> list[dict]:
    return [
        {
            "name":         fake.name(),
            "email":        fake.email(),
            "phone":        fake.phone_number(),
            "city":         fake.city(),
            "state":        fake.state(),
            "account_id":   fake.uuid4(),
            "credit_score": random.randint(550, 850),
            "joined_date":  fake.date_between(start_date="-5y", end_date="today").isoformat(),
        }
        for _ in range(n)
    ]

records = generate_synthetic_customer(100)
# Send to LLM for testing — zero GDPR exposure

Tools: Faker (Python), Mimesis, DataFaker (Java), @faker-js/faker (Node).

2. Statistical Model-Based Synthesis (SDV)

Generate data that preserves real statistical distributions — column correlations, conditional probabilities, outlier ratios — without any real records.

from sdv.tabular import GaussianCopula  # or CTGAN for complex distributions

model = GaussianCopula()
model.fit(real_dataframe)              # train on real data (one-time, in secure env)

synthetic_df = model.sample(num_rows=1000)  # generate — safe to export

Tools: SDV (Synthetic Data Vault), CTGAN, TVAE, Gretel.ai.

Compliance note: Training the synthesis model itself involves processing real data — this must occur in a compliant environment. The output of the model is what becomes shareable.

3. LLM-Based Synthesis

Use the LLM to generate realistic-looking records from a schema description, with no real data ever entering the prompt.

SCHEMA_PROMPT = """
Generate 10 realistic but entirely fictional customer support tickets for a fintech SaaS.
Schema:
- ticket_id: string (format: TKT-XXXX)
- customer_name: fictional Indian name
- issue_type: one of [payment_failure, kyc_issue, transfer_delay, account_locked]
- description: 2-3 sentence summary of the issue
- created_at: ISO date in 2024
- severity: low | medium | high

Return JSON array only. No markdown. No preamble.
"""

response = call_llm(SCHEMA_PROMPT)
synthetic_tickets = json.loads(response)

This is especially useful for domain-specific edge cases — LLMs understand narrative context in ways statistical models don’t.

4. Differential Privacy (DP) — Adding Mathematical Noise

For scenarios where you must use real data as a statistical input, Differential Privacy adds mathematically calibrated noise to query results, providing a formal privacy guarantee — no individual’s data can be inferred from the output.

import diffprivlib as dp

ages = real_dataframe["age"].values

# DP mean with epsilon=1.0 (lower ε = more privacy, less accuracy)
dp_mean_age = dp.tools.mean(ages, epsilon=1.0, bounds=(18, 90))
dp_std_age  = dp.tools.std(ages, epsilon=1.0, bounds=(18, 90))

# Safe to publish — mathematically privacy-preserving
print(f"Mean age: {dp_mean_age:.1f}, Std: {dp_std_age:.1f}")

Tool: diffprivlib (IBM), OpenDP, Google’s DP library.

GDPR relevance: Differential privacy is recognized as a rigorous anonymization mechanism that satisfies the “all means reasonably likely” standard from Recital 26 — unlike simple aggregation, which does not.


Compliance Mapping

TechniqueGDPR Status of OutputReversibleData Still “Personal”?SOC 2 FitBest Agent Hook Position
MaskingPseudonymized or Anon.ConditionalDepends on mappingStrongPre-LLM
AnonymizationOutside GDPR scopeNoNoEliminates scopePre-LLM
TokenizationPseudonymizedYes (vault)YesVery strongPre-LLM + Post-LLM
PseudonymizationPersonal data (reduced)Yes (key)YesStrongPre-LLM + Post-LLM
PII RedactionLikely anonymizedNoNoEliminates scopePre-LLM
Synthetic DataNot personal dataN/ANoNo PII scopePre-LLM / Replace
Differential PrivacyAnonymized (formal)NoNoEliminates scopePre-LLM (aggregation)

Decision Framework: Which Technique to Use

Do you need to re-identify later?
    │
    ├── YES → Does downstream code need to reference the original value?
    │              ├── YES → Tokenization (vault-backed, reversible by design)
    │              └── NO  → Pseudonymization (HMAC/alias, key-separated)
    │
    └── NO  → Is data utility important (format, analytics, distributions)?
                   ├── YES → Masking (format-preserved) or Synthetic Data (analytics-safe)
                   └── NO  → Anonymization (full removal) or PII Redaction (document publishing)

Are you in a testing / evaluation / fine-tuning context?
    └── Always consider Synthetic Data Generation first — zero PII exposure by design.

Do you need a formal, mathematical privacy guarantee for statistical outputs?
    └── Differential Privacy — required for publishing aggregates from sensitive datasets.

Production Implementation Notes

Use a Battle-Tested PII Detection Layer

Regex alone will miss names, contextual PII, and indirect identifiers. Use:

  • Microsoft Presidio — open-source, extensible NER + regex, Python/Docker
  • AWS Comprehend — managed NLP with medical and general PII detection
  • Google Cloud DLP (now Sensitive Data Protection) — 150+ predefined infoTypes, de-identification pipelines
  • spaCy + custom NER — fully local, trainable on domain-specific entities

Vault Security for Tokenization

  • Never store the vault in the same DB as tokenized data
  • Use HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault for production key management
  • Log every detokenize operation — who accessed what, when, for what purpose

Schema-Level Protection

Apply protections at the schema level, not just at the field level:

PII_FIELD_POLICY = {
    "email":       "tokenize",
    "name":        "pseudonymize",
    "dob":         "mask_partial",        # keep year only
    "address":     "redact",
    "account_bal": "differential_noise",
    "user_id":     "passthrough",         # internal ref, not PII
}

def apply_policy(record: dict) -> dict:
    return {
        k: DISPATCH[PII_FIELD_POLICY.get(k, "passthrough")](v)
        for k, v in record.items()
    }

Audit Trail Requirements (SOC 2 + GDPR Art. 5(2))

Every sanitization event should produce a log entry:

  • Timestamp
  • Record or document ID (not the PII itself)
  • Technique applied
  • Fields affected
  • Agent or process that triggered the operation
  • Whether the operation was reversible

Summary

PriorityReach For
Maximum privacy, no utility loss neededRedaction or Anonymization
Need cross-record analyticsPseudonymization (HMAC)
LLM must reference original laterTokenization (vault)
Format preservation for testingMasking
No real data at allSynthetic Data Generation
Publishing statistical aggregatesDifferential Privacy

In agentic pipelines, privacy is not a post-processing concern — it is a pre-LLM gate. The sanitize → infer → restore pattern, combined with the right technique for the right context, is how you build systems that are both AI-capable and compliance-ready from day one.

Claude Code — Configuration Guide

How to configure masking, redaction, PII gating, and synthetic data techniques via Claude Code’s hooks, settings, CLAUDE.md, and permissions — with real caveats from production.


The Configuration Stack

Claude Code gives you five distinct surfaces to enforce privacy controls. Each operates at a different layer of the agent loop:

┌─────────────────────────────────────────────────────────────┐
│  MANAGED SETTINGS          → Enterprise policy enforcement  │
│  (~org-level, MDM/server-managed, not overridable)          │
├─────────────────────────────────────────────────────────────┤
│  CLAUDE.md                 → Behavioral privacy directives  │
│  (what Claude knows & is instructed to respect)             │
├─────────────────────────────────────────────────────────────┤
│  .claudeignore             → File-level exclusion           │
│  (keeps paths out of context entirely)                      │
├─────────────────────────────────────────────────────────────┤
│  permissions.deny / allow  → Tool-call gating               │
│  (settings.json — restricts which tools can touch what)     │
├─────────────────────────────────────────────────────────────┤
│  HOOKS                     → Deterministic runtime control  │
│  (PreToolUse / PostToolUse / UserPromptSubmit scripts)      │
└─────────────────────────────────────────────────────────────┘

Critical production caveat: permissions.deny rules have documented bugs where they fail silently — Claude Code has been observed reading and modifying denied files without any error or warning. There are open GitHub issues (filed August 2025 – February 2026) documenting this. Hooks are the only surface that reliably enforces data protection at runtime. Use permissions.deny as a declared policy layer, and hooks as the enforced gate.


Layer 1: CLAUDE.md — Behavioral Privacy Directives

CLAUDE.md is the system-prompt-level instruction file. It shapes how Claude interprets tasks. Place privacy policies here as explicit directives.

File location:

  • Project-level: CLAUDE.md or .claude/CLAUDE.md in your repo root (shared with team)
  • User-level: ~/.claude/CLAUDE.md (applies across all projects)
  • Local override: CLAUDE.local.md (gitignored, personal overrides)

Example .claude/CLAUDE.md with privacy directives:

# Privacy & Data Protection Policy

## PII Handling

- NEVER include real names, emails, phone numbers, or national IDs in any
  generated code, tests, configs, logs, or commit messages.
- When writing test fixtures or seed data, use Faker-generated synthetic values.
  Never copy real records from production files into test files.
- If you encounter PII in a file you are asked to analyze, summarize the
  structure only. Do not echo or reproduce personal data fields in your response.

## Sensitive Files — Do Not Read or Modify

- `.env`, `.env.*`, `.env.local`, `.env.production` — treat as off-limits.
  Ask before reading. Never modify.
- `secrets/`, `credentials/`, `vault/`, `*.pem`, `*.key` — do not read or write.
- `data/raw/`, `data/production/` — read-only analytics use only. Never edit.

## External Transmission

- Before configuring any service that transmits data externally (APIs, telemetry,
  webhooks, upload endpoints), ask explicitly whether to include identifying fields.
- Default to disabling telemetry, analytics, and external upload flags in
  any config you generate.

## Synthetic Data Default

- For tests, seeds, fixtures, and demos: always generate synthetic data.
  Preferred library: Faker (Python) or @faker-js/faker (Node).
- Never use real email addresses, even obfuscated ones, in test data.

Limitation: CLAUDE.md is a soft control — it’s instruction, not enforcement. The model can still violate it (especially in long sessions or complex agentic tasks). A real-world GitHub issue (April 2026) documented Claude writing a real station coordinate to a config file that uploaded to a public API, despite a “Never include PII” directive being active in CLAUDE.md. Always back CLAUDE.md with hook-level enforcement.


Layer 2: .claudeignore — File-Level Exclusion

.claudeignore uses the same syntax as .gitignore and tells Claude Code which files and directories to never index, read into context, or make available to the agent.

File location: Project root (.claudeignore)

Example .claudeignore for PII protection:

# Environment and secrets
.env
.env.*
.env.local
.env.production
.env.staging
*.secret
secrets/
credentials/
vault/
*.pem
*.key
*.p12
*.pfx

# Production data — never in context
data/raw/
data/production/
data/exports/
dumps/
*.sql.gz
*.sql.bak

# PII datasets
customers.csv
users.csv
patient_records/
kyc_documents/
pii_exports/

# Audit logs with PII
logs/audit/
logs/access/
*.access.log

Limitation noted in community research (March 2026): After testing, .claudeignore was found not to be reliably respected in all tool call paths. Claude acknowledged in-session that it “won’t respect .claudeignore” when using certain tools. Treat it as a signal and a best-effort exclusion, not a hard enforcement boundary. Pair it with hook-level blocks described below.


Layer 3: permissions.deny — Tool-Call Gating

permissions.deny rules are declared in settings.json and restrict which tool+argument combinations Claude Code can use. This is your declarative policy layer.

File locations:

  • ~/.claude/settings.json — user-level (all projects)
  • .claude/settings.json — project-level (committed to git, team-shared)
  • .claude/settings.local.json — local override (gitignored)
  • /Library/Application Support/ClaudeCode/managed-settings.json (macOS) — managed/enterprise

Example .claude/settings.json with DP-focused deny rules:

{
  "$schema": "https://json.schemastore.org/claude-code-settings.json",
  "permissions": {
    "allow": [
      "Read(./src/**)",
      "Read(./tests/**)",
      "Edit(./src/**)",
      "Edit(./tests/**)",
      "Bash(npm run test *)",
      "Bash(npm run lint)"
    ],
    "deny": [
      "Read(./.env)",
      "Read(./.env.*)",
      "Read(./secrets/**)",
      "Read(./credentials/**)",
      "Read(./data/production/**)",
      "Read(./data/raw/**)",
      "Read(./logs/audit/**)",
      "Edit(./.env)",
      "Edit(./.env.*)",
      "Edit(./secrets/**)",
      "Bash(curl *)",
      "Bash(wget *)",
      "WebFetch(*)"
    ]
  }
}

Read deny before allow — always. Deny rules take precedence over allow rules in Claude Code’s permission evaluation order. An explicit deny blocks the call even if a broader allow rule would otherwise match.

Enterprise managed settings (non-overridable):

For SOC 2 or GDPR compliance posture across a team, deploy these via managed-settings.json — users cannot override them:

// /Library/Application Support/ClaudeCode/managed-settings.json  (macOS)
// /etc/claude-code/managed-settings.json                          (Linux/WSL)
// C:\Program Files\ClaudeCode\managed-settings.json               (Windows)

{
  "permissions": {
    "deny": [
      "Read(./.env*)",
      "Read(**/secrets/**)",
      "Read(**/credentials/**)",
      "Read(**/pii/**)",
      "Bash(curl *)",
      "Bash(wget *)",
      "WebFetch(*)"
    ]
  },
  "allowManagedHooksOnly": true
}

Setting allowManagedHooksOnly: true ensures only centrally deployed hooks run — no rogue project-level hooks can bypass your org’s privacy controls.

Again: deny rules have known silent-failure bugs. Do not rely on them alone. Use hooks (Layer 5) as the enforced gate.


Layer 4: Hooks — Deterministic Runtime PII Enforcement

This is the most reliable enforcement layer. Hooks are shell scripts (or HTTP endpoints or LLM prompts) that execute at specific points in the agent loop. Unlike permissions.deny, hooks fire as actual subprocess calls — not as LLM-evaluated instructions.

Key hook events for data protection:

EventWhen it firesCan block?DP Use Case
UserPromptSubmitWhen you submit a prompt, before Claude reads itYes (exit 2)Scan/redact PII in the prompt itself
PreToolUseBefore any tool call (Read, Edit, Bash, WebFetch…)Yes (exit 2)Block reads of sensitive files, scrub tool input
PostToolUseAfter any tool call completesBlock onlyAudit what Claude saw/wrote, alert on PII
SessionStartOn session openNoInject privacy context, load vault state
ConfigChangeWhen settings files change mid-sessionYesAudit or block unauthorized config edits

Configure hooks in .claude/settings.json under the hooks key.


Hook 1: PII Scan on UserPromptSubmit

Intercept the prompt before Claude sees it. Scan for PII patterns and either redact them or block the prompt.

Hook script: .claude/hooks/pii-prompt-scan.sh

#!/usr/bin/env bash
# PII scan on user prompt — blocks if high-confidence PII detected

INPUT=$(cat)
PROMPT=$(echo "$INPUT" | python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('prompt',''))")

# Python inline scan using regex patterns
PII_HIT=$(python3 << 'PYEOF'
import sys, re, json

text = sys.stdin.read()

patterns = {
    "AADHAAR":  r"\b\d{4}\s\d{4}\s\d{4}\b",
    "PAN":      r"\b[A-Z]{5}[0-9]{4}[A-Z]\b",
    "EMAIL":    r"[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}",
    "PHONE_IN": r"\b(?:\+91[\s\-]?)?\d{10}\b",
    "SSN":      r"\b\d{3}-\d{2}-\d{4}\b",
    "CC":       r"\b(?:\d{4}[\s\-]){3}\d{4}\b",
    "IP":       r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
    "API_KEY":  r"\b[A-Za-z0-9_\-]{32,}\b",
}

hits = []
for label, pattern in patterns.items():
    matches = re.findall(pattern, text)
    if matches:
        hits.append({"type": label, "count": len(matches)})

print(json.dumps(hits))
PYEOF
)

PROMPT="$PROMPT" python3 -c "
import sys, os
hits = __import__('json').loads(os.environ.get('PII_HIT','[]') if False else open('/dev/stdin').read() or '[]')
" <<< "$PII_HIT"

HIT_COUNT=$(echo "$PII_HIT" | python3 -c "import json,sys; hits=json.load(sys.stdin); print(len(hits))")

if [ "$HIT_COUNT" -gt "0" ]; then
    echo "PII detected in prompt: $PII_HIT — blocked. Remove personal data before proceeding." >&2
    exit 2
fi

exit 0

Register in .claude/settings.json:

{
  "hooks": {
    "UserPromptSubmit": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/pii-prompt-scan.sh"
          }
        ]
      }
    ]
  }
}

Note on additionalContext: For UserPromptSubmit, you can inject sanitized context back into the prompt (instead of blocking) by outputting a JSON object with additionalContext to stdout before exiting 0. Use this to append a redacted version of the prompt.


Hook 2: Sensitive File Read Block (PreToolUse)

Block Claude from reading files that match sensitive path patterns. More reliable than permissions.deny because this runs as an actual subprocess with a hard exit 2.

Hook script: .claude/hooks/block-sensitive-reads.sh

#!/usr/bin/env bash
# Block Read/Edit tool calls on sensitive paths

INPUT=$(cat)
TOOL=$(echo "$INPUT" | jq -r '.tool_name // empty')
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // .tool_input.path // empty')

# Only gate on Read and Edit tools
if [[ "$TOOL" != "Read" && "$TOOL" != "Edit" && "$TOOL" != "Write" ]]; then
    exit 0
fi

# Sensitive path patterns
SENSITIVE_PATTERNS=(
    ".env"
    ".env."
    "secrets/"
    "credentials/"
    "vault/"
    ".pem"
    ".key"
    ".p12"
    "data/raw/"
    "data/production/"
    "logs/audit/"
    "logs/access/"
    "customers.csv"
    "users_export"
    "pii_"
    "kyc_"
    "patient_"
)

for pattern in "${SENSITIVE_PATTERNS[@]}"; do
    if [[ "$FILE_PATH" == *"$pattern"* ]]; then
        echo "BLOCKED: '$FILE_PATH' matches sensitive pattern '$pattern'. Access denied under data protection policy." >&2
        exit 2
    fi
done

exit 0
chmod +x .claude/hooks/block-sensitive-reads.sh
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Read|Edit|Write",
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/block-sensitive-reads.sh"
          }
        ]
      }
    ]
  }
}

Hook 3: Redact Tool Input Before It Reaches Claude (PreToolUse)

Rather than blocking, strip PII from the content Claude is about to process by using the updatedInput field in the hook’s JSON output. This is the masking/redaction pattern applied at the agent hook layer.

Hook script: .claude/hooks/redact-tool-input.py

#!/usr/bin/env python3
"""
PreToolUse hook: redacts PII from file content before Claude processes it.
Returns updatedInput to replace the tool call's input.
"""

import sys
import json
import re

PATTERNS = {
    "AADHAAR":  r"\b\d{4}\s\d{4}\s\d{4}\b",
    "PAN":      r"\b[A-Z]{5}[0-9]{4}[A-Z]\b",
    "EMAIL":    r"[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}",
    "PHONE":    r"\b(?:\+91[\s\-]?)?\d{10}\b",
    "SSN":      r"\b\d{3}-\d{2}-\d{4}\b",
    "CC":       r"\b(?:\d{4}[\s\-]){3}\d{4}\b",
    "IP":       r"\b(?:\d{1,3}\.){3}\d{1,3}\b",
}

def redact(text: str) -> tuple[str, int]:
    """Returns (redacted_text, count_of_redactions)"""
    total = 0
    for label, pattern in PATTERNS.items():
        matches = re.findall(pattern, text)
        if matches:
            text = re.sub(pattern, f"[{label} REDACTED]", text)
            total += len(matches)
    return text, total

hook_input = json.load(sys.stdin)
tool_name = hook_input.get("tool_name", "")
tool_input = hook_input.get("tool_input", {})

# Only operate on tools that return file content
if tool_name not in ("Read",):
    sys.exit(0)

content = tool_input.get("content", "")
if not content:
    sys.exit(0)

redacted_content, count = redact(content)

if count > 0:
    # Return updatedInput to replace what Claude sees
    output = {
        "hookSpecificOutput": {
            "hookEventName": "PreToolUse",
            "permissionDecision": "allow",
            "updatedInput": {
                **tool_input,
                "content": redacted_content
            },
            "additionalContext": f"Note: {count} PII instance(s) were automatically redacted from this file before you processed it."
        }
    }
    print(json.dumps(output))

sys.exit(0)
chmod +x .claude/hooks/redact-tool-input.py
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Read",
        "hooks": [
          {
            "type": "command",
            "command": "python3 \"$CLAUDE_PROJECT_DIR\"/.claude/hooks/redact-tool-input.py"
          }
        ]
      }
    ]
  }
}

Hook 4: PostToolUse PII Audit Log

After Claude reads or writes a file, scan the tool output for PII and log it. This is the audit trail layer — it doesn’t block, but it gives you a compliance record of what PII was processed.

Hook script: .claude/hooks/pii-audit-log.sh

#!/usr/bin/env bash
# PostToolUse: audit log for PII in tool outputs

INPUT=$(cat)
TOOL=$(echo "$INPUT" | jq -r '.tool_name // empty')
OUTPUT=$(echo "$INPUT" | jq -r '.tool_response // empty')
FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

# Python PII scan of tool output
PII_HITS=$(echo "$OUTPUT" | python3 << 'PYEOF'
import sys, re, json

text = sys.stdin.read()
patterns = {
    "AADHAAR":  r"\b\d{4}\s\d{4}\s\d{4}\b",
    "EMAIL":    r"[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}",
    "PHONE":    r"\b(?:\+91[\s\-]?)?\d{10}\b",
    "SSN":      r"\b\d{3}-\d{2}-\d{4}\b",
    "CC":       r"\b(?:\d{4}[\s\-]){3}\d{4}\b",
    "API_KEY":  r"\b[A-Za-z0-9_\-]{32,64}\b",
}
hits = {k: len(re.findall(p, text)) for k, p in patterns.items() if re.findall(p, text)}
print(json.dumps(hits))
PYEOF
)

# Log to audit file (never log the actual PII content — only counts and field types)
LOG_ENTRY=$(jq -n \
    --arg ts "$TIMESTAMP" \
    --arg tool "$TOOL" \
    --arg file "$FILE_PATH" \
    --argjson pii "$PII_HITS" \
    '{timestamp: $ts, tool: $tool, file: $file, pii_fields_detected: $pii}')

echo "$LOG_ENTRY" >> "${CLAUDE_PROJECT_DIR}/.claude/logs/pii-audit.jsonl"

exit 0
{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Read|Edit|Write|Bash",
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/pii-audit-log.sh"
          }
        ]
      }
    ]
  }
}
# Create the audit log directory (add the log file to .gitignore)
mkdir -p .claude/logs
echo ".claude/logs/" >> .gitignore

This produces a pii-audit.jsonl with entries like:

{"timestamp": "2026-06-19T10:32:04Z", "tool": "Read", "file": "./data/users.json", "pii_fields_detected": {"EMAIL": 47, "PHONE": 12}}

Hook 5: Block External Data Transmission (PreToolUse on Bash/WebFetch)

Prevent Claude from transmitting data to external endpoints — a critical control when working with codebases that contain PII or secrets.

Hook script: .claude/hooks/block-exfil.sh

#!/usr/bin/env bash
# Block data exfiltration via curl, wget, WebFetch to non-allowlisted domains

INPUT=$(cat)
TOOL=$(echo "$INPUT" | jq -r '.tool_name // empty')

ALLOWED_DOMAINS=(
    "api.github.com"
    "registry.npmjs.org"
    "pypi.org"
    "internal.your-company.com"
)

if [[ "$TOOL" == "WebFetch" ]]; then
    URL=$(echo "$INPUT" | jq -r '.tool_input.url // empty')
    DOMAIN=$(echo "$URL" | python3 -c "from urllib.parse import urlparse; import sys; print(urlparse(sys.stdin.read().strip()).netloc)")
    for allowed in "${ALLOWED_DOMAINS[@]}"; do
        if [[ "$DOMAIN" == "$allowed" || "$DOMAIN" == *".$allowed" ]]; then
            exit 0
        fi
    done
    echo "BLOCKED: WebFetch to '$DOMAIN' not in allowlist. Add to ALLOWED_DOMAINS in block-exfil.sh if legitimate." >&2
    exit 2
fi

if [[ "$TOOL" == "Bash" ]]; then
    CMD=$(echo "$INPUT" | jq -r '.tool_input.command // empty')
    EXFIL_PATTERNS=("curl " "wget " "nc " "netcat " "python.*requests" "http.post" "fetch(" "axios.post")
    for pattern in "${EXFIL_PATTERNS[@]}"; do
        if [[ "$CMD" == *"$pattern"* ]]; then
            echo "BLOCKED: Bash command matches exfiltration pattern '$pattern'. Review and approve manually." >&2
            exit 2
        fi
    done
fi

exit 0
{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash|WebFetch",
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/block-exfil.sh"
          }
        ]
      }
    ]
  }
}

Complete settings.json — Combined Configuration

{
  "$schema": "https://json.schemastore.org/claude-code-settings.json",
  "permissions": {
    "allow": [
      "Read(./src/**)",
      "Read(./tests/**)",
      "Read(./scripts/**)",
      "Edit(./src/**)",
      "Edit(./tests/**)",
      "Bash(npm run test *)",
      "Bash(npm run lint)",
      "Bash(pip install *)",
      "Bash(git *)"
    ],
    "deny": [
      "Read(./.env)",
      "Read(./.env.*)",
      "Read(./secrets/**)",
      "Read(./credentials/**)",
      "Read(./data/production/**)",
      "Read(./data/raw/**)",
      "Read(./logs/audit/**)",
      "Edit(./.env)",
      "Edit(./.env.*)",
      "Edit(./secrets/**)",
      "Bash(curl *)",
      "Bash(wget *)",
      "WebFetch(*)"
    ]
  },
  "hooks": {
    "UserPromptSubmit": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/pii-prompt-scan.sh"
          }
        ]
      }
    ],
    "PreToolUse": [
      {
        "matcher": "Read|Edit|Write",
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/block-sensitive-reads.sh"
          }
        ]
      },
      {
        "matcher": "Read",
        "hooks": [
          {
            "type": "command",
            "command": "python3 \"$CLAUDE_PROJECT_DIR\"/.claude/hooks/redact-tool-input.py"
          }
        ]
      },
      {
        "matcher": "Bash|WebFetch",
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/block-exfil.sh"
          }
        ]
      }
    ],
    "PostToolUse": [
      {
        "matcher": "Read|Edit|Write|Bash",
        "hooks": [
          {
            "type": "command",
            "command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/pii-audit-log.sh"
          }
        ]
      }
    ],
    "ConfigChange": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "jq -c '{timestamp: now | todate, event: \"ConfigChange\", file: .file_path, source: .source}' >> \"$CLAUDE_PROJECT_DIR\"/.claude/logs/config-audit.jsonl"
          }
        ]
      }
    ]
  }
}

Synthetic Data via a Custom Claude Code Skill

For test data generation, wire synthetic data as a slash command skill so Claude generates compliant fixtures on demand — without ever touching real records.

File: .claude/commands/synthetic-data.md

---
description: Generate synthetic PII-safe test data using Faker
allowed-tools: Bash, Write
---

Generate $ARGUMENTS synthetic records as JSON fixtures using Python's Faker library.

Rules:
- All names, emails, phones, and IDs must be Faker-generated (never real)
- Use locale `en_IN` for Indian PII fields (Aadhaar-style, PAN-style)
- Output to `tests/fixtures/synthetic_<entity>_<timestamp>.json`
- Include a header comment: `# SYNTHETIC DATA — NOT REAL PII`
- Never copy structure from `data/production/` or `data/raw/`

Example schema to generate (adapt to $ARGUMENTS if a schema is specified):
{
  "id": uuid4,
  "name": full_name,
  "email": email,
  "phone": phone_number,
  "city": city,
  "state": state,
  "account_id": uuid4,
  "created_at": iso_date
}

Usage from Claude Code CLI:

/synthetic-data 50 records for the User entity

Mapping DP Techniques to Claude Code Hooks

DP TechniqueWhere to configure in Claude CodeHook eventReversible?
Maskingredact-tool-input.py (PreToolUse, updatedInput)PreToolUseNo (in context)
AnonymizationCLAUDE.md directive + block-sensitive-reads.shPreToolUse (block)N/A — prevents access
TokenizationExternal vault script called from PreToolUse/PostToolUsePre + Post ToolUseYes (vault-side)
PseudonymizationHMAC script in PreToolUse, key held externallyPreToolUseYes (key-side)
PII Redactionredact-tool-input.py with [FIELD REDACTED] outputPreToolUseNo
Synthetic Data/synthetic-data slash command + CLAUDE.md directiveGeneration-timeN/A — no PII ever
Audit Trailpii-audit-log.sh (PostToolUse)PostToolUseN/A — append-only log
Exfil Blockblock-exfil.sh (PreToolUse on Bash/WebFetch)PreToolUseN/A — block

File Structure — What to Commit vs. Gitignore

your-project/
├── CLAUDE.md                          # ✅ commit — team privacy directives
├── .claudeignore                      # ✅ commit — shared file exclusions
├── .claude/
│   ├── settings.json                  # ✅ commit — shared permissions + hooks
│   ├── settings.local.json            # ❌ gitignore — personal overrides
│   ├── CLAUDE.local.md                # ❌ gitignore — personal notes
│   ├── hooks/
│   │   ├── pii-prompt-scan.sh         # ✅ commit
│   │   ├── block-sensitive-reads.sh   # ✅ commit
│   │   ├── redact-tool-input.py       # ✅ commit
│   │   ├── pii-audit-log.sh           # ✅ commit
│   │   └── block-exfil.sh             # ✅ commit
│   ├── commands/
│   │   └── synthetic-data.md          # ✅ commit
│   └── logs/
│       ├── pii-audit.jsonl            # ❌ gitignore — runtime audit trail
│       └── config-audit.jsonl         # ❌ gitignore — runtime audit trail

Key Caveats — What Doesn’t Work Reliably

ControlStatusWhat to do instead
permissions.denyKnown silent failures (bugs open)Back with PreToolUse hooks (exit 2)
CLAUDE.md directivesSoft instruction — model can ignoreEnforce with hook-level blocks
.claudeignoreInconsistent across tool pathsBack with block-sensitive-reads.sh
settings.local.json denyNot always inherited by subagentsUse managed settings for org enforcement
CLAUDE.md + memoryIgnored in some agentic sub-tasksHooks are the only deterministic gate

Bottom line: Hooks with exit 2 are the only hard enforcement boundary in Claude Code’s current architecture. Everything else — deny rules, CLAUDE.md, .claudeignore — is best-effort and should be treated as a documentation layer that informs hooks, not replaces them.


Compliance references: GDPR Regulation (EU) 2016/679 — Articles 4(5), 5(1)(c), 17, 25, 89; Recitals 26, 28. SOC 2 Trust Services Criteria — CC6 (Logical Access), C1 (Confidentiality).

Interactive Tools

Practical tools to help you think systematically, build better AI agents, and master prompt engineering.

Future References

Explore these resources for deeper learning on AI agent development, spec-driven development, and prompt engineering tools.

Spec-Driven Development

Comprehensive guide on Spec-Driven Development practices and methodologies.

Awesome Copilot

Curated list of GitHub Copilot resources, extensions, and best practices.

Promptfoo

Tool for testing, evaluating, and improving LLM prompts and applications.

Prompts.chat

Collection of prompt engineering resources and templates.

Agent Skills

Agent skills resources and documentation for building AI agent skills.

Awesome Skills

Curated list of awesome skill repositories and collections.

Agent Skills Topic

GitHub topic for discovering agent-related skills and repositories.

AI Agent Topic

Trendshift topic for discovering AI agents.

AI Skills Topic

Trendshift topic for discovering AI skills.

Agent Governance Toolkit

Agent governance toolkit.

Pattern Sources

Our patterns are curated from industry-leading sources with proper attribution and licensing compliance.

Refactoring.Guru

Classic GoF design patterns, code smells catalog, and refactoring techniques (https://refactoring.guru).

Enterprise Integration Patterns

65 messaging patterns for integrating enterprise applications by Gregor Hohpe and Bobby Woolf (CC BY 4.0).

Microservices.io

Comprehensive patterns for microservice architectures by Chris Richardson.

Agent Catalog Patterns

Patterns for agentic systems from agentpatternscatalog.org (CC BY 4.0).

OWASP Foundation

Security patterns from OWASP Top 10 for Web Applications, LLM Applications, and Agentic Applications (CC BY-SA 4.0).

Industry Research

ML/AI patterns from Microsoft, Google, Anthropic, and academic research.

AI Agent Patterns

Spec-driven development patterns from Claude, Gemini, OpenAI, and GitHub Copilot on github/spec-kit and OpenSpec.

Data Engineering Leaders

Data platform patterns from Martin Fowler (Data Mesh), Kimball Group (Dimensional Modeling), and cloud providers.

MLOps Best Practices

Data science patterns from MLflow, Great Expectations, and MLOps practitioners.

Streaming & Analytics

Real-time patterns from Confluent/Kafka, Apache projects, and serverless analytics platforms.

Academic Papers

Rigorous ML patterns from peer-reviewed research including data leakage prevention and active learning.

5-Day AI Agents Course

Intensive Vibe Coding Course With Google by Brenda Flynn et al. (2026) on Kaggle.

The Agent Loop

Foundational Agent Definition (Perceive + Act):
Russell, S. J., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Prentice Hall. (Current edition: 4th Ed., Pearson, 2020)

Modern Iterative LLM Agent Loop:
Yao, S., Zhao, J., Yu, D., et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.

Historical Context:
Incorporating AIMA's perceive/act model, Classical robotics' Sense-Plan-Act loop (Brooks, 1986), and ReAct's Thought→Action→Observation cycle.