PII Detection & Redaction: Automatic Sensitive Data Handling

PII Detection & Redaction: Automatic Sensitive Data Handling

A single unprotected prompt containing customer data can trigger a compliance violation costing millions. In 2024, a major healthcare provider’s LLM chatbot accidentally exposed 8,000 patient records because their PII detection relied on basic regex patterns that missed context-aware data. This guide provides production-ready strategies for detecting and redacting PII before it reaches your LLM—and before it leaks from your responses.

Key Takeaway

Effective PII protection requires a hybrid approach: combine high-precision regex patterns for structured data with NLP models for contextual detection, then layer cloud-based services for scale and compliance auditing.

Why PII Detection Matters in LLM Systems

LLMs process vast amounts of text, making them prime targets for data leakage. Without proper PII handling, you risk:

• Compliance violations: GDPR fines up to 4% of annual revenue, HIPAA penalties reaching $1.5M per violation
• Reputation damage: Customer trust erodes after data exposure incidents
• Legal liability: Direct financial responsibility for breach remediation and damages

The challenge is that PII appears in multiple forms:

• Structured: SSNs (987-65-4321), credit cards (4532-1234-5678-9010), phone numbers
• Contextual: Names, addresses, medical records in unstructured text
• Embedded: PII in images, scanned documents, PDFs

According to Google Cloud’s Sensitive Data Protection documentation, their platform provides over 200 built-in infoType detectors for PII detection cloud.google.com/sensitive-data-protection/docs. However, no single solution covers all scenarios—enterprises must architect layered defenses.

Core PII Detection Methods

1. Regex-Based Detection

Regex patterns excel at detecting structured PII with predictable formats. They offer high precision (95-99%) and low latency (less than 10ms), making them ideal for first-pass filtering.

Strengths:

• Fast execution
• Deterministic results
• Easy to implement and tune
• High precision for known patterns

Limitations:

• Brittle with format variations
• No contextual understanding
• High false positives on similar-looking data
• Cannot detect unstructured PII (names, addresses)

2. Named Entity Recognition (NER)

NER models use machine learning to identify entities based on context. spaCy, Stanford NER, and cloud services can detect names, organizations, locations, and more.

Strengths:

• Context-aware detection
• Handles unstructured text
• Adaptable to domain-specific entities

Limitations:

• Higher latency (50-200ms)
• Lower precision on structured data
• Requires model training/fine-tuning
• Resource-intensive

3. Hybrid Approach (Recommended)

Combine regex for structured PII with NER for contextual detection, then use cloud services for scale and compliance.

Benefits:

• 95-99% accuracy rates
• Balanced performance
• Comprehensive coverage
• Audit trail for compliance

Practical Implementation

1. Identify PII types relevant to your domain (SSNs, emails, medical codes, etc.)
2. Implement regex patterns for high-confidence detection of structured data
3. Add NER models for contextual detection of names, organizations, etc.
4. Integrate cloud services (Google DLP, AWS Comprehend) for scale and auditing
5. Redact or tokenize detected PII before LLM processing
6. Log all detections for compliance and monitoring
7. Test with real data to measure accuracy and false positive rates
8. Monitor and update patterns regularly as regulations change

Code Examples

Python: Google Cloud DLP

import json
import time
from google.cloud import dlp

def inspect_and_redact_text(text, info_types=None):
    """
    Inspects text for PII using Google Cloud DLP and redacts detected instances.

    Args:
        text (str): The text content to inspect.
        info_types (list): List of info types to detect (e.g., ['US_SOCIAL_SECURITY_NUMBER', 'EMAIL_ADDRESS']).

    Returns:
        tuple: (redacted_text, findings_count)
    """
    if info_types is None:
        info_types = ['US_SOCIAL_SECURITY_NUMBER', 'EMAIL_ADDRESS', 'PHONE_NUMBER', 'CREDIT_CARD_NUMBER']

    # Initialize the DLP client
    dlp_client = dlp.DlpServiceClient()

    # Configure the inspection request
    parent = "projects/YOUR_PROJECT_ID/locations/global"  # Replace with your project ID

    # Configure inspection
    inspect_config = {
        'info_types': [{'name': info_type} for info_type in info_types],
        'min_likelihood': dlp.Likelihood.LIKELIHOOD_UNSPECIFIED,  # Detect all likelihoods
        'limits': {'max_findings_per_request': 0}  # No limit
    }

    # Configure redaction
    redact_config = {
        'info_type_transformations': {
            'transformations': [
                {'primitive_transformation': {'character_mask_config': {'masking_character': 'X', 'number_to_mask': 0}}}
            ]
        }
    }

    # Create the inspection and redaction request
    item = {'value': text}

    try:
        # First, inspect to get findings
        inspect_request = {
            'parent': parent,
            'inspect_config': inspect_config,
            'item': item
        }
        response = dlp_client.inspect_content(request=inspect_request)
        findings_count = len(response.result.findings) if response.result else 0

        # Then, redact
        redact_request = {
            'parent': parent,
            'inspect_config': inspect_config,
            'item': item,
            'redact_config': redact_config
        }
        redacted_response = dlp_client.redact_content(request=redact_request)

        return redacted_response.item.value, findings_count

    except Exception as e:
        print(f"Error during DLP operation: {e}")
        return text, 0

# Example usage
if __name__ == "__main__":
    sample_text = "Contact John Doe at john.doe@example.com or call 555-123-4567. SSN: 123-45-6789."
    redacted_text, count = inspect_and_redact_text(sample_text)
    print(f"Original: {sample_text}")
    print(f"Redacted: {redacted_text}")
    print(f"Findings: {count}")

Hybrid: Regex + NER

import re
import spacy
from typing import List, Dict, Tuple
import logging

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

class HybridPIIDetector:
    """
    Hybrid PII detector combining regex patterns for high-precision
    detection of structured PII and spaCy NER for contextual detection.
    """

    def __init__(self):
        # Load spaCy model for NER
        try:
            self.nlp = spacy.load("en_core_web_sm")
        except OSError:
            logger.warning("spaCy model not found. Run: python -m spacy download en_core_web_sm")
            self.nlp = None

        # Regex patterns for high-confidence PII
        self.regex_patterns = {
            'ssn': re.compile(r'\b\d{3}-\d{2}-\d{4}\b'),
            'credit_card': re.compile(r'\b(?:\d{4}[-\s]?){3}\d{4}\b'),
            'email': re.compile(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'),
            'phone': re.compile(r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b'),
            'ip_address': re.compile(r'\b(?:\d{1,3}\.){3}\d{1,3}\b')
        }

    def detect_with_regex(self, text: str) -> List[Dict]:
        """Detect PII using regex patterns for structured data."""
        findings = []
        for pii_type, pattern in self.regex_patterns.items():
            for match in pattern.finditer(text):
                findings.append({
                    'type': pii_type,
                    'value': match.group(),
                    'start': match.start(),
                    'end': match.end(),
                    'confidence': 0.95  # High confidence for regex
                })
        return findings

    def detect_with_ner(self, text: str) -> List[Dict]:
        """Detect PII using spaCy NER for contextual detection."""
        if not self.nlp:
            return []

        findings = []
        doc = self.nlp(text)

        for ent in doc.ents:
            if ent.label_ in ['PERSON', 'ORG', 'GPE', 'DATE', 'MONEY', 'PHONE', 'EMAIL']:
                # Map spaCy labels to our PII types
                pii_type_map = {
                    'PERSON': 'name',
                    'ORG': 'organization',
                    'GPE': 'location',
                    'DATE': 'date',
                    'MONEY': 'financial',
                    'PHONE': 'phone',
                    'EMAIL': 'email'
                }

                pii_type = pii_type_map.get(ent.label_)
                if pii_type:
                    findings.append({
                        'type': pii_type,
                        'value': ent.text,
                        'start': ent.start_char,
                        'end': ent.end_char,
                        'confidence': 0.75  # Medium confidence for NER
                    })
        return findings

    def detect_all(self, text: str) -> List[Dict]:
        """Combine regex and NER detection, removing overlaps."""
        regex_findings = self.detect_with_regex(text)
        ner_findings = self.detect_with_ner(text)

        # Combine and deduplicate (regex takes precedence)
        all_findings = regex_findings.copy()
        existing_spans = {(f['start'], f['end']) for f in regex_findings}

        for finding in ner_findings:
            span = (finding['start'], finding['end'])
            if span not in existing_spans:
                all_findings.append(finding)
                existing_spans.add(span)

        # Sort by position
        all_findings.sort(key=lambda x: x['start'])
        return all_findings

    def redact_text(self, text: str, findings: List[Dict]) -> str:
        """Redact detected PII from text."""
        if not findings:
            return text

        # Sort findings by start position in reverse to avoid index shifting
        findings_sorted = sorted(findings, key=lambda x: x['start'], reverse=True)

        redacted_text = text
        for finding in findings_sorted:
            # Replace with masked value
            mask = f"[{finding['type'].upper()}]"
            redacted_text = redacted_text[:finding['start']] + mask + redacted_text[finding['end']:]

        return redacted_text

# Example usage
if __name__ == "__main__":
    detector = HybridPIIDetector()

    sample_text = """
    Employee: John Smith
    Email: john.smith@company.com
    Phone: 555-0123
    SSN: 987-65-4321
    Credit Card: 4532-1234-5678-9010
    Address: 123 Main St, Springfield, IL 62701
    """

    findings = detector.detect_all(sample_text)
    redacted = detector.redact_text(sample_text, findings)

    print("Original:")
    print(sample_text)
    print("\nRedacted:")
    print(redacted)
    print(f"\nTotal findings: {len(findings)}")

TypeScript: Real-time PII Detection for LLM Prompts

import { DlpServiceClient } from '@google-cloud/dlp';
import { Logging } from '@google-cloud/logging';

interface PIIDetectionResult {
  hasPII: boolean;
  findings: Array<{
    type: string;
    value: string;
    confidence: number;
  }>;
  redactedText?: string;
}

class LLMPIIFilter {
  private dlpClient: DlpServiceClient;
  private logging: Logging;
  private readonly PROJECT_ID: string;
  private readonly MIN_LIKELIHOOD: number;

  constructor(projectId: string, minLikelihood: number = 0.3) {
    this.PROJECT_ID = projectId;
    this.MIN_LIKELIHOOD = minLikelihood;
    this.dlpClient = new DlpServiceClient();
    this.logging = new Logging({ projectId });
  }

  /**
   * Inspects text for PII before sending to LLM
   * Returns detection results and optional redacted version
   */
  async inspectPrompt(text: string): Promise<PIIDetectionResult> {
    try {
      const parent = `projects/${this.PROJECT_ID}/locations/global`;

      const inspectConfig = {
        infoTypes: [
          { name: 'US_SOCIAL_SECURITY_NUMBER' },
          { name: 'EMAIL_ADDRESS' },
          { name: 'PHONE_NUMBER' },
          { name: 'CREDIT_CARD_NUMBER' },
          { name: 'PERSON' },
          { name: 'LOCATION' }
        ],
        minLikelihood: this.MIN_LIKELIHOOD,
        limits: { maxFindingsPerRequest: 10 }
      };

      const item = { value: text };

      const [response] = await this.dlpClient.inspectContent({
        parent,
        inspectConfig,
        item
      });

      const findings = response.result?.findings || [];
      const hasPII = findings.length > 0;

      const piiFindings = findings.map(finding => ({
        type: finding.infoType?.name || 'UNKNOWN',
        value: finding.quote || '',
        confidence: finding.likelihood || 0
      }));

      // Log detection for audit trail
      if (hasPII) {
        const logEntry = this.logging.log('llm-pii-detection');
        await logEntry.write({
          severity: 'WARNING',
          text: 'PII detected in LLM prompt',
          findings: piiFindings,
          timestamp: new Date().toISOString()
        });
      }

      return {
        hasPII,
        findings: piiFindings
      };
    } catch (error) {
      console.error('PII detection failed:', error);
      // Fail-safe: reject if detection fails
      return {
        hasPII: true,
        findings: []
      };
    }
  }

  /**
   * Redacts PII from text using DLP
   */
  async redactText(text: string): Promise<string> {
    try {
      const parent = `projects/${this.PROJECT_ID}/locations/global`;

      const inspectConfig = {
        infoTypes: [
          { name: 'US_SOCIAL_SECURITY_NUMBER' },
          { name: 'EMAIL_ADDRESS' },
          { name: 'PHONE_NUMBER' },
          { name: 'CREDIT_CARD_NUMBER' }
        ],
        minLikelihood: this.MIN_LIKELIHOOD
      };

      const redactConfig = {
        infoTypeTransformations: {
          transformations: [
            {
              primitiveTransformation: {
                characterMaskConfig: {
                  maskingCharacter: '#',
                  numberToMask: 0
                }
              }
            }
          ]
        }
      };

      const item = { value: text };

      const [response] = await this.dlpClient.redactContent({
        parent,
        inspectConfig,
        item,
        redactConfig
      });

      return response.item?.value || text;
    } catch (error) {
      console.error('PII redaction failed:', error);
      // Return original text on failure
      return text;
    }
  }

  /**
   * Process LLM prompt with PII filtering
   * Returns safe prompt and audit info
   */
  async processLLMPrompt(prompt: string): Promise<{
    safePrompt: string;
    audit: PIIDetectionResult;
  }> {
    const detection = await this.inspectPrompt(prompt);

    if (detection.hasPII) {
      const redacted = await this.redactText(prompt);
      return {
        safePrompt: redacted,
        audit: { ...detection, redactedText: redacted }
      };
    }

    return {
      safePrompt: prompt,
      audit: detection
    };
  }
}

// Example usage
async function main() {
  const filter = new LLMPIIFilter('your-project-id');

  const testPrompts = [
    "My SSN is 123-45-6789 and email is test@example.com",
    "Can you help me with my account balance?",
    "Call me at 555-0123 or reach out to john.doe@company.com"
  ];

  for (const prompt of testPrompts) {
    console.log(`\nOriginal: ${prompt}`);
    const result = await filter.processLLMPrompt(prompt);
    console.log(`Safe: ${result.safePrompt}`);
    console.log(`PII Found: ${result.audit.hasPII}`);
    if (result.audit.findings.length > 0) {
      console.log('Findings:', result.audit.findings);
    }
  }
}

// Run if this file is executed directly
if (require.main === module) {
  main().catch(console.error);
}

export { LLMPIIFilter, PIIDetectionResult };

Common Pitfalls

Avoid these critical mistakes that compromise PII detection effectiveness:

• Regex-only detection: Relying solely on regex patterns without context validation leads to high false positive rates (e.g., matching numbers that look like SSNs but aren’t)
• Missing audit logging: Not implementing proper audit logging for PII detection events creates compliance gaps and inability to demonstrate due diligence
• Edge case failures: Failing to handle PII in images, scanned documents, or encoded formats (Base64, URL-encoded)
• Overly broad patterns: Using patterns that match legitimate data (e.g., phone numbers in product codes, dates in historical records)
• Performance neglect: Not considering real-time PII detection impact on application latency, especially for large documents
• International formats: Ignoring international PII formats (EU SSNs, non-US phone numbers, different credit card formats) in global applications
• Unencrypted storage: Storing PII detection results without encryption or proper access controls
• No fail-safe behavior: Not implementing fail-safe when detection services are unavailable
• Static patterns: Failing to regularly update detection patterns and models as new PII types emerge or regulations change
• Insufficient testing: Not testing detection accuracy with real-world data samples, leading to production failures

Quick Reference

Detection Method Comparison

Method	Precision	Latency	Best For	Limitations
Regex	95-99%	less than 10ms	Structured PII (SSN, CC)	Brittle, no context
NER	75-90%	50-200ms	Unstructured (names, orgs)	Lower precision, resource-heavy
Hybrid	95-99%	50-250ms	Comprehensive coverage	Complex implementation
Cloud DLP	95-99%	100-300ms	Scale, compliance, auditing	Cost, network dependency

Cloud Service Pricing (Verified)

Service	Metric	Price	Source
Google Cloud DLP	Inspection API	Pay-per-use	cloud.google.com/sensitive-data-protection/docs
AWS Comprehend PII	100 characters	$0.0001 (300 char min)	aws.amazon.com/comprehend/pricing
Google Document AI	1,000 pages (1-5M)	$1.50	cloud.google.com/document-ai/pricing
Model Armor	LLM prompt/response filtering	Configurable templates	docs.cloud.google.com/model-armor/overview

LLM API Pricing (Context)

Model	Input/1M tokens	Output/1M tokens	Context Window
GPT-4o	$5.00	$15.00	128K
GPT-4o-mini	$0.15	$0.60	128K
Claude 3.5 Sonnet	$3.00	$15.00	200K
Claude 3.5 Haiku	$1.25	$5.00	200K

PII Entity Categories

Category	Examples	Detection Method	Redaction Strategy
Personal Names	John Smith, Dr. Jane Doe	NER, Regex	[NAME] or [PERSON]
Contact Info	email@example.com, 555-0123	Regex, NER	[EMAIL], [PHONE]
Government IDs	123-45-6789, 987-65-4321	Regex	[SSN], [ID]
Financial Data	4532-1234-5678-9010	Regex	[CREDIT_CARD]
Location	123 Main St, Springfield	NER, Regex	[ADDRESS], [LOCATION]
Healthcare	MRN: 123456, DOB: 01/15/1980	Regex, NER	[MRN], [DOB]

Widget

PII detector demo (text → detected entities + redacted output)

Interactive widget derived from “PII Detection & Redaction: Automatic Sensitive Data Handling” that lets readers explore pii detector demo (text → detected entities + redacted output).

Key models to cover:

• Anthropic claude-3-5-sonnet (tier: general) — refreshed 2024-11-15
• OpenAI gpt-4o-mini (tier: balanced) — refreshed 2024-10-10
• Anthropic haiku-3.5 (tier: throughput) — refreshed 2024-11-15

Widget metrics to capture: user_selections, calculated_monthly_cost, comparison_delta.

Data sources: model-catalog.json, retrieved-pricing.

Related Resources

Google Cloud DLP Documentation Comprehensive guide to DLP API with 200+ infoType detectors

AWS Comprehend PII Detection Managed NLP service for PII detection in text

Azure AI Language PII Detection Cloud-based PII detection and redaction service

Private AI Documentation 50+ supported entity types for multi-regulation compliance

Next Steps

Implementation Checklist

Before deploying PII detection in production, verify:

• ✅ All PII types relevant to your domain are covered
• ✅ Regex patterns tested against false positive scenarios
• ✅ NER models trained/fine-tuned for your domain
• ✅ Cloud service integration with proper authentication
• ✅ Audit logging implemented and accessible
• ✅ Fail-safe behavior when detection fails
• ✅ Regular monitoring and pattern updates scheduled
• ✅ Compliance requirements documented and met

For production deployment, start with regex patterns for immediate protection, then layer in NER models and cloud services as your needs scale. Always test with real data samples and monitor false positive rates to continuously improve detection accuracy.