Stefan Gasser [Tue, 23 Jun 2026 18:52:25 +0000 (20:52 +0200)]
Limit masking scans to input roles (#115)
Default PII and secrets detection to scan only user-controlled roles (user, tool, function, mcp) instead of every message role. Infer per-span roles in the Codex and Anthropic extractors so harness-injected context (environment_context, system-reminder, Codex AGENTS.md) and system/assistant content stay unscanned.
Restrict the dashboard preview to scanned roles after masking.
Stefan Gasser [Tue, 23 Jun 2026 08:55:05 +0000 (10:55 +0200)]
Stop seeding config.yaml into new Conductor workspaces (#110)
Remove file_include_globs so each workspace regenerates config.yaml from config.example.yaml in setup instead of inheriting a stale copy. The inherited copy carried a fixed detector_url that didn't match the per-workspace detector port, leaving the proxy pointed at the wrong detector. The remaining globs only covered an empty .env, so the setting is dropped entirely.
Stefan Gasser [Tue, 23 Jun 2026 07:31:37 +0000 (09:31 +0200)]
Track browser-extension requests separately in the dashboard (#107)
Add a `source` column to request logs so `/api/mask` calls sent with the
`X-PasteGuard-Source: browser-extension` header are counted and badged
separately from direct API calls and provider-proxied requests.
- Backfill `source` from `provider` for existing rows via migration
- Split stats into api_requests vs browser_extension_requests
- Replace the route-only Provider column with a unified Source column
- Add an Extension stat card
Stefan Gasser [Tue, 23 Jun 2026 06:34:57 +0000 (08:34 +0200)]
Rename masking whitelist config to allowlist (#104)
Rename the masking whitelist config key and its associated schema,
types, and helper (filterAllowlistedEntities) to use allowlist
terminology. Updates docs and the example config to match.
Stefan Gasser [Tue, 23 Jun 2026 06:04:47 +0000 (08:04 +0200)]
Add configurable masking denylist and regex whitelist (#101)
* Add configurable masking denylist
* Align whitelist pattern config
* Harden denylist/whitelist masking
- Merge denylist matches additively so they never shrink detector coverage
- Skip denylist matches inside existing placeholders to avoid corrupting secret/PII masks
- Anchor regex whitelist to the full entity so a partial match can't unmask larger PII
- Skip detection when PII detection is off and no denylist is configured
- Reject regex patterns that match the empty string at config load
- Reuse the conflict-resolver overlap helper; update docs and tests
* Fix streaming unmask when a placeholder delimiter splits across chunks
findPartialPlaceholderStart only buffered when the full "[[" delimiter appeared
within one chunk. If a stream chunk ended with a lone "[" (the first half of
"[["), it was emitted as safe, so the placeholder was never reassembled and
leaked to the client un-restored. Buffer a trailing partial of the start
delimiter too. Restores PII and secrets placeholders that the upstream model
tokenizes across the "[[" boundary.
* Remove ReDoS caveat from PII detection docs
* Simplify partial-placeholder detection and cover the closing-delimiter split
Replace the single-iteration loop in findPartialPlaceholderStart with a direct
trailing-bracket check, and add tests for a placeholder whose closing "]]" is
split across stream chunks.
* Exclude denylist matches by known placeholders; de-magic the match score
Replace the placeholder-shape regex heuristic with exact exclusion against the
real placeholders carried over from secrets masking: secretPlaceholders() is
threaded through detectPII/analyzeRequest and passed in /api/mask, so a denylist
pattern can no longer match (and corrupt) the internals of an existing
secret/PII placeholder. Also replace the vestigial denylist match score 2 with a
named DENYLIST_MATCH_SCORE constant.
* Require knownPlaceholders on detectPII so routes can't skip placeholder protection
* Clarify whitelist/denylist docs and drop Claude Code wording from the default
Stefan Gasser [Mon, 22 Jun 2026 10:29:05 +0000 (12:29 +0200)]
Replace Presidio with an open-source PII detector (#100)
Replace the Microsoft Presidio analyzer with a self-hosted open-source PII detector (Python/FastAPI, detector/) shipped in the all-in-one image: multilingual GLiNER NER plus a deterministic regex/checksum layer for structured identifiers.
- Add VAT_CODE (EU VAT, checksum-validated); member-state prefixes only, case-insensitive and overlap-safe so a label or word can't hide a valid number.
- /api/mask: detect secrets before PII so a connection string isn't partly masked as an email (matches the provider routes).
- Rename presidio_url to detector_url; language-agnostic detection; CPU-only torch; detector CI (ruff/pyright/pytest).
- Bump version to 0.5.0.
Stefan Gasser [Wed, 17 Jun 2026 20:24:19 +0000 (22:24 +0200)]
Add PII detection accuracy benchmark (#99)
Adds a corpus-driven PII accuracy benchmark against the Presidio /analyze endpoint, plus pl/ro precision negatives, non-western PERSON coverage, and address-like LOCATION cases.
Stefan Gasser [Wed, 10 Jun 2026 12:19:59 +0000 (14:19 +0200)]
Expose logging and secrets_detection in /info, remove unused log_content
Follow-ups from #91:
- /info now includes secrets_detection and logging sections so users can
verify their config is loaded (the issue reporter could not).
- The dashboard fallback message no longer claims log_masked_content is
false when content is simply absent for another reason.
- Remove the log_content option: it was defined in config and documented,
but no code ever read it. Logging raw request/response content would
contradict the privacy guarantees, so it is removed rather than
implemented. Zod strips unknown keys, so existing configs that still
set log_content keep loading.
Stefan Gasser [Wed, 10 Jun 2026 12:03:49 +0000 (14:03 +0200)]
Fix masked content not logged when secrets are detected
Masked content was never stored whenever secrets were detected, ignoring
the log_masked_content setting. The dashboard then showed a misleading
"Masked content not logged (log_masked_content: false)" message.
With secrets_detection.action "mask" (the default), maskedContent already
has both PII and secrets replaced by placeholders by the time it reaches
the logger, so it is safe to store. Gate on log_masked_content plus
whether detected secrets were actually masked, via a pure
shouldLogMaskedContent helper. The secrets-masked condition keeps route
mode with action "route_local" safe: there secrets are intentionally left
raw for the trusted local provider and must never be persisted.
This also enforces log_masked_content centrally for the openai/anthropic/
codex routes, which previously passed masked content unconditionally.
The helper lives in its own module so the unit test can import it without
tripping over the wholesale logger mock used by other route tests.
Stefan Gasser [Fri, 13 Mar 2026 07:54:28 +0000 (08:54 +0100)]
fix: use UID 1000 for container user to match typical Linux host users (#77)
Fixes permission denied errors when mounting volumes. Most Linux desktop
users have UID 1000, so bind-mounted directories now work without manual
chown.
* fix: preserve cache_control in Anthropic schemas to restore prompt caching
* fix: add biome-ignore for passthrough tests, extend fix to OpenAI
- Add biome-ignore comments to suppress noExplicitAny in tests
(required for testing unknown field preservation)
- Add .passthrough() to OpenAI schemas for consistency
(OpenAIMessageSchema, OpenAIContentPartSchema)
- Format Anthropic schemas to match project style
* test: add OpenAI passthrough tests for consistency with Anthropic
- Schema tests for name, tool_calls, audio content, unknown fields
- Extractor tests for field preservation through applyMasked
Stefan Gasser [Fri, 20 Feb 2026 20:52:52 +0000 (21:52 +0100)]
Fix SIGILL crash on CPUs without AVX2 support (#70) (#71)
Copy Bun binary from official oven/bun:1-slim image instead of using
the install script. The official images use baseline builds for x64,
which only require SSE4.2 and work on older/low-power CPUs like the
Intel Atom C3558R that lack AVX2 instructions.
Also fixes compatibility with updated presidio-analyzer base image:
- Use USER root for build, then switch to non-root (UID 1001) for runtime
- Fix Presidio config paths (/app/presidio_analyzer/conf/)
- Move PasteGuard to /pasteguard to avoid overwriting Presidio's /app
- Update supervisord to run without root privileges
- Update volume mount paths in docker-compose.yml and docs
Stefan Gasser [Mon, 9 Feb 2026 08:05:13 +0000 (09:05 +0100)]
Fix missing Presidio recognizers for URL, US_SSN, CRYPTO, etc. (#69)
The config generator only included 6 recognizers, missing standard ones
like UrlRecognizer, UsSsnRecognizer, CryptoRecognizer. This caused
detection failures when users enabled these entity types.
Changes:
- Add GLOBAL_RECOGNIZERS for pattern-based detection (7 recognizers)
- Add LANGUAGE_RECOGNIZERS for language-specific detection
- Only load language-specific recognizers when that language is configured
- EN: US + UK recognizers (8)
- ES: Spanish NIF/NIE (2)
- IT: Italian documents (5)
- PL: Polish PESEL (1)
- KO: Korean RRN (1)
Stefan Gasser [Sat, 31 Jan 2026 21:50:00 +0000 (22:50 +0100)]
Restructure docs around use cases (#66)
* Restructure docs around use cases
Replace the single Integrations page with four dedicated use case pages
(Browser Extension, Coding Tools, Apps & SDKs, Self-Hosted) so each
audience has a clear path through the docs.
- Rewrite introduction around four user contexts with cards
- Simplify quickstart to be tool-neutral
- Add Use Cases navigation group in mint.json
- Update README links to point to new docs pages
- Replace demo.gif with comparison.png
- Remove duplicate h1 headings (Mintlify renders frontmatter title)
- Remove Copilot from platform lists per positioning doc
* Sync docs and README with website messaging
- Update tagline to "AI gets the context. Not your secrets."
- Add trust statement and detection stats (30+ types, 24 languages)
- Rename use cases to match website: Chat, Apps, Coding Tools, API Integration
- Reorder use cases consistently across docs and README
- Replace comparison image with refined rose/teal design
- Add light/dark mode image support for GitHub and Mintlify
* Improve dark mode text contrast in comparison image
Stefan Gasser [Mon, 26 Jan 2026 08:26:19 +0000 (09:26 +0100)]
Increase Presidio startup timeout for multi-language images (#62)
The EU image loads 9 spaCy language models which takes ~2 minutes.
The previous 30-second timeout caused PasteGuard to restart multiple
times before Presidio was ready.
- Increase default timeout from 30s to 180s
- Add PRESIDIO_STARTUP_TIMEOUT env var for customization
- Document startup time in config and docs
Stefan Gasser [Mon, 26 Jan 2026 08:18:03 +0000 (09:18 +0100)]
Rename API_KEY_OPENAI to API_KEY_SK with expanded pattern (#61)
- Rename API_KEY_OPENAI to API_KEY_SK for broader coverage
- Expand pattern from /sk-.../ to /sk[-_].../ to match both hyphen and underscore
- Reduce minimum length from 45 to 20 chars for Stripe compatibility
- Now detects: OpenAI, Anthropic, Stripe, RevenueCat, and similar sk-prefixed keys
- Add tests for Stripe (sk_test_, sk_live_), Anthropic (sk-ant-), RevenueCat (sk_)
- Update all documentation and config examples
Stefan Gasser [Mon, 26 Jan 2026 07:13:59 +0000 (08:13 +0100)]
Fix entity extraction and improve API consistency
- Use direct placeholder lookup instead of fragile substring matching
- Make error details always an array for consistent response format
- Add languageFallback field to response
- Add tests for error paths and edge cases
Stefan Gasser [Wed, 21 Jan 2026 17:31:02 +0000 (18:31 +0100)]
Refactor logging interfaces for simpler data structures
- Simplify PIILogData: entityTypes string[] instead of allEntities objects
- Simplify SecretsLogData: types string[] instead of matches objects
- Move mapping logic into toPIILogData/toSecretsLogData converters
- Update api.ts to use createLogData() instead of manual construction
Stefan Gasser [Wed, 21 Jan 2026 06:56:56 +0000 (07:56 +0100)]
Add source tracking to dashboard
- Add API requests counter to stats section
- Add Source column to logs table with badges
- API badge (amber) for /api/mask requests
- PROXY badge (gray) for OpenAI/Anthropic proxy requests
- Backward compatible: existing logs show as PROXY
Stefan Gasser [Wed, 21 Jan 2026 06:14:05 +0000 (07:14 +0100)]
Add generic /api/mask endpoint for standalone text masking
Adds a new POST /api/mask endpoint that can be used by browser extensions,
CLIs, or any client that needs to mask PII and secrets independently of
the OpenAI/Anthropic proxy routes.
Features:
- Detects and masks both PII and secrets (configurable via detect param)
- Returns context mapping for client-side unmasking
- Supports multi-turn conversations via startFrom counters
- Auto-detects language or accepts explicit language parameter
- Logs requests to dashboard for visibility
- Rejects whitespace-only text input
- Consistent error handling for both PII and secrets detection
Stefan Gasser [Tue, 20 Jan 2026 22:40:11 +0000 (23:40 +0100)]
Replace generic provider/LLM terminology with OpenAI or Anthropic (#56)
- Use specific names instead of "provider" or "LLM" in user-facing text
- Keep "local LLM" for Ollama/vLLM references
- Update README, docs, and configuration files
Stefan Gasser [Tue, 20 Jan 2026 22:06:58 +0000 (23:06 +0100)]
Add Anthropic API support (#51)
* Add Anthropic provider
- Add /anthropic/v1/messages endpoint with full API compatibility
- Support OAuth tokens from Claude Code for subscription users
- Provider-agnostic text extraction for PII/secrets masking
- Support streaming and non-streaming responses
- Remove unused cloud provider health checks (only local services need them)
* Add Anthropic provider documentation
- Update README and introduction to mention Anthropic support
- Add Claude Code and Anthropic SDK to integrations
- Document Anthropic provider config with OAuth support
- Create separate API reference pages for OpenAI and Anthropic
- Update navigation structure
* Improve provider error messages in logs
- Add errorMessage getter to parse OpenAI/Anthropic error formats
- Log parsed error message instead of generic "Provider error"
* Update docs wording for multi-provider support
- Clarify OpenAI and Anthropic APIs with compatible providers
- Note Anthropic endpoint is mask mode only (route mode coming)
* Add route mode support for Anthropic endpoint
- Add callLocalAnthropic function for Ollama's Anthropic API
- Update Anthropic route to support route mode with local provider
- Update docs to reflect both mask and route mode support
* Add Anthropic brand color to dashboard provider badges
* Fix duplicate /v1 prefix in Anthropic proxy wildcard handler
The path variable already contains the full path after stripping the
/anthropic prefix (e.g., /v1/messages or /api/foo). Adding /v1 again
caused double prefixes for v1 paths and incorrect paths for non-v1
endpoints like /api/event_logging/batch.
* Add role field to Anthropic extractor for scan_roles filtering
* Remove OAuth token reading, use transparent header forwarding
- Delete oauth.ts - no longer read tokens from local storage
- Simplify client.ts to forward all auth headers transparently
- Simplify anthropic.ts wildcard handler
- Add Claude Code system prompt to default whitelist
- Whitelist merges user entries with default (not replaces)
* Simplify wildcard proxies to fully transparent passthrough
Stefan Gasser [Tue, 20 Jan 2026 20:52:02 +0000 (21:52 +0100)]
Add whitelist config for masking exclusions (#53)
Adds masking.whitelist config option to exclude specific text patterns
from PII masking. Useful for preventing false positives on known text
like company names or product identifiers.
- Add whitelist property to MaskingSchema (default: empty array)
- Add filterWhitelistedEntities function to filter detected PII
- Patterns match if detected text is contained in whitelist entry
or whitelist entry is contained in detected text
Stefan Gasser [Tue, 20 Jan 2026 19:07:03 +0000 (20:07 +0100)]
Add scan_roles config for role-based PII/secrets filtering (#52)
Allows configuring which message roles to scan for PII and secrets.
By default all roles are scanned (existing behavior). When scan_roles
is set, only messages with matching roles are sent to Presidio.
Use case: Skip scanning large system prompts to reduce API calls and
avoid false positives on app-controlled content.
- Add role field to TextSpan type
- OpenAI extractor populates role from message
- PII detector filters by scan_roles before Presidio calls
- Secrets detector filters by scan_roles before detection
Stefan Gasser [Mon, 19 Jan 2026 17:30:22 +0000 (18:30 +0100)]
Refactor for multi-provider architecture (#49)
Reorganizes the codebase to support multiple LLM providers with a clean,
extensible architecture. This is a preparatory refactor that improves
code organization without adding new provider support.
New patterns:
- Provider-agnostic text extraction with TextExtractor interface
- Shared error handling with ProviderError class
- Centralized timeout constants in src/constants/timeouts.ts
- Unified logging helpers in src/routes/utils.ts
Removed:
- src/services/decision.ts (logic moved to service layer)
- src/providers/openai-client.ts (replaced by src/providers/openai/client.ts)
Stefan Gasser [Sat, 17 Jan 2026 19:32:54 +0000 (20:32 +0100)]
Add per-part PII/secrets detection for multimodal messages (#47)
Each text block in multimodal messages is now analyzed separately
rather than concatenating all text together. This enables precise
masking while preserving message structure.
Changes:
- Per-part detection for both PII and secrets
- Symmetric pii/ and secrets/ module structure
- Shared utilities in utils/message-transform.ts
- Rename "redact" → "mask" for consistency
- Centralize MaskResult, Span interfaces
Stefan Gasser [Sat, 17 Jan 2026 13:10:36 +0000 (14:10 +0100)]
All-in-one Docker image with prebuilt language support (#42)
* All-in-one Docker image with dev mode support
Simplify deployment with a single container that includes both the proxy
and Presidio PII detection. No git clone required - just docker run.
Changes:
- Combined Dockerfile: Multi-stage build with Presidio + Bun in one container
- supervisord: Process manager for running both services
- Prebuilt images: ghcr.io/sgasser/pasteguard:en (~2.7GB) and :eu (~12GB)
- Release workflow: GitHub Actions to build en/eu images on version tags
- Dev mode: docker compose up presidio -d for local development with hot-reload
- Updated docs: curl-based installation, no cloning needed for production
- Language models: Changed from _md to _lg for better PII detection accuracy
Image tags:
- en (default/latest): English only
- eu: European languages (en, de, es, fr, it, nl, pl, pt, ro)
Keep docker-compose.yml in root for convenience (docker compose up works).
Update all path references in workflow, configs, and docs.
* Fix CI: update Dockerfile path
* Include config.example.yaml in Docker image for zero-config startup
* Simplify docs: zero-config quickstart
* Add missing ENV detection entities to config example
Add ENV_PASSWORD, ENV_SECRET, and CONNECTION_STRING to the
secrets_detection entities section to match what's documented
and implemented.
* Auto-configure languages per Docker image
- Add PASTEGUARD_LANGUAGES env var to Dockerfile (set from LANGUAGES build arg)
- Update config.example.yaml to use env var with fallback to 'en'
- Support comma-separated string for languages config (e.g., "en,de,fr")
- EN image now auto-enables English, EU image auto-enables all 9 EU languages
Users can still override via config.yaml with array syntax if needed.
* Update docs: languages are auto-configured per image
* Clarify runtime vs build-time env vars in docs
* Update docs: languages are auto-configured per Docker image
* Remove confusing env var override example from docs
* List specific EU languages in docs instead of 'All 9'
Config changes:
- Rename providers.upstream to providers.openai for clarity
- Remove routing config section (simplified to: PII → local, no PII → openai)
- Move local provider to top-level config (not under providers)
- Change default secrets action from block to redact
Proxy changes:
- Replace specific /models route with wildcard /* proxy
- Supports all OpenAI endpoints: /models, /embeddings, /audio/*, etc.
Documentation:
- Update all docs to reflect new config structure
- Remove docs/api-reference/models.mdx (now covered by wildcard proxy)
Stefan Gasser [Fri, 16 Jan 2026 22:30:36 +0000 (23:30 +0100)]
fix: log and display API errors in dashboard (#40)
Fixes #35
- Add LLMError class to preserve upstream status code and body
- Add status_code and error_message columns to request logs
- Add Status column to dashboard with OK/error badges
- Pass through upstream errors (429, 401, etc.) with original status
- Return OpenAI-compatible JSON format for all error responses
- Set X-PasteGuard headers consistently via Hono context
Stefan Gasser [Fri, 16 Jan 2026 16:11:50 +0000 (17:11 +0100)]
Fix overlapping entity conflict resolution (#33)
Implement Presidio-style conflict resolution for overlapping PII entities.
Problem: Presidio returns overlapping entities (e.g., both "Eric" and "Eric's")
which caused text corruption during masking.
Solution: Two-phase algorithm matching Presidio's Anonymizer logic:
1. Merge overlapping entities of same type (expand boundaries, keep highest score)
2. Remove conflicting entities of different types (contained or lower score loses)
- Add conflict-resolver.ts with resolveConflicts() and resolveConflictsSimple()
- Replace old removeOverlappingEntities with new conflict resolver
- Add 18 tests covering all conflict scenarios
- Delete old entities.ts (replaced by conflict-resolver.ts)
Stefan Gasser [Fri, 16 Jan 2026 14:09:32 +0000 (15:09 +0100)]
Fix overlapping entity masking (#33)
Presidio can return overlapping PII detections (e.g., both "Eric" and
"Eric's" as separate PERSON entities). When both were masked, the string
positions became invalid, corrupting the output.
Added removeOverlappingEntities() to filter overlaps before masking:
- Sort by start position (longer first if same start)
- Keep only non-overlapping entities
Stefan Gasser [Fri, 16 Jan 2026 15:40:04 +0000 (16:40 +0100)]
fix: use [[]] delimiters for placeholders to prevent HTML encoding issues (#38)
Fixes #36 - HTML-encoded placeholders now unmask correctly
Changes:
- Changed placeholder format from <TYPE_N> to [[TYPE_N]]
- Created src/constants/placeholders.ts as single source of truth
- Removed configurable redact_placeholder (was bug - streaming hardcoded [[)
- Updated dashboard regex for yellow highlighting
- Added tests for HTML/JSON/URL contexts
The [[]] delimiters are safe in HTML, JSON, and URLs - they don't get
entity-encoded like <> did.
Stefan Gasser [Fri, 16 Jan 2026 13:30:11 +0000 (14:30 +0100)]
feat(dashboard): display secrets detection in logs table (#30)
Add Secrets column to the Recent Requests table showing detected
secret types (e.g., OPENSSH_PRIVATE_KEY, PEM_PRIVATE_KEY) as red
badges. The data was already available from the API but not displayed.
- Add Secrets column header between PII Entities and Scan Time
- Parse secrets_types and secrets_detected from API response
- Display secret types as error-colored badges
- Add CSS utilities for error color variants
- Update colspan values for detail rows
Stefan Gasser [Mon, 12 Jan 2026 15:48:24 +0000 (16:48 +0100)]
Exclude test files from secret scanning (#31)
Test files contain intentional fake secrets for testing secret detection.
This prevents false positive alerts for MongoDB connection strings and
other test fixtures in src/secrets/detect.test.ts.
* Add PatternDetector and DetectionResult interfaces for secrets detection registry
* Move pattern detection utility to new patterns/utils.ts module
* Refactor secrets detection using a registry system
- Create privateKeysDetector, apiKeysDetector, tokensDetector modules
- Refactor detectSecrets() to use the pattern registry
- Re-export types from detect.ts for backwards compatibility
* Change default secrets_detection action to redaction
Hint: The example config still shows `action: block` explicitly, with a comment noting
that `redact` is the default action if not specified
* Implement new pattern detector and add corresponding SecretEntityType options
* Register new detector and extend test suite accordingly
* Add new entity types to config.ts
* Update docs and example config
* Add environment variables section to secrets detection docs
Stefan Gasser [Mon, 12 Jan 2026 07:13:59 +0000 (08:13 +0100)]
Always run language detection even with single language configured (#26)
Previously, language detection was skipped when only one language was
configured, returning the configured language directly. This made it
impossible to detect misconfiguration (e.g., only EN configured but
receiving DE text).
Now language detection always runs, providing:
- Actual detected language in logs (detectedLanguage field)
- Confidence score for debugging
- usedFallback=true when detected language isn't configured
Performance impact is negligible (~0.01-0.05ms per detection).
Stefan Gasser [Sun, 11 Jan 2026 18:42:09 +0000 (19:42 +0100)]
Fix PII detection to scan all message roles (#25)
Previously, PII detection only scanned the last user message initially,
then did a full scan only if PII was found. This caused PII in system
messages (e.g., RAG context from PDFs) to be missed entirely when the
user message contained no PII.
Changes:
- Consolidate analyzeMessages() to always scan all messages
- Scan system, developer, user, and assistant roles
- Remove analyzeAllMessages() as it's no longer needed
- Simplify decision.ts by removing the redundant full scan call
This ensures PII in system messages (common in RAG patterns) is properly
detected and masked before being sent to upstream LLMs.
Max Wolf [Sun, 11 Jan 2026 17:55:34 +0000 (18:55 +0100)]
Refactor secrets detection into pattern registry (#18)
* Add PatternDetector and DetectionResult interfaces for secrets detection registry
* Move all interfaces to patterns/types.ts and use the existing SecretesDetectionResult interface instead of the new DetectionResult
* Move pattern detection utility to new patterns/utils.ts module
* Refactor secrets detection using a registry system
- Create privateKeysDetector, apiKeysDetector, tokensDetector modules
- Refactor detectSecrets() to use the pattern registry
- Re-export types from detect.ts for backwards compatibility
* Change default secrets_detection action to redaction
Hint: The example config still shows `action: block` explicitly, with a comment noting
that `redact` is the default action if not specified
mkroemer [Sat, 10 Jan 2026 08:21:33 +0000 (09:21 +0100)]
You are right, and I (and claude) addressed these issues.
1. Secrets redaction (proxy.ts):
- Add per-part offset tracking to prevent partial secret leaks
- Filter and adjust redaction positions for each text part
- Fixes issue where 'sk-proj-' prefix could remain visible
2. PII masking (masking.ts):
- Actually mask array content instead of returning original
- Add per-part offset tracking for accurate entity positions
- Properly handle multimodal arrays with text and images
3. Tests:
- Add content.test.ts for text extraction utilities
- Add multimodal.test.ts with integration tests
mkroemer [Fri, 9 Jan 2026 20:06:49 +0000 (21:06 +0100)]
fix: Update type system for multimodal content support
- Update ChatMessage interface to accept MessageContent type
- Fix masking.ts to handle multimodal content
- Fix redact.ts to extract text before processing
- Fix import order and formatting per biome linter
mkroemer [Fri, 9 Jan 2026 19:58:56 +0000 (20:58 +0100)]
fix: Add support for multimodal content (text + images)
Fixes crashes when processing OpenAI Chat Completion requests with
multimodal content (array format). Previously, the code assumed message
content is always a string, causing Presidio errors and 502/503 responses
when LibreChat Agents sent vision requests.
Changes:
- Add extractTextContent() utility to safely extract text from both
string and array content formats
- Update PII detection to handle multimodal messages
- Update secrets detection to extract text from array content
- Update message redaction to preserve images while redacting text
- Fix dashboard display to show readable content instead of [object Object]
This enables full support for OpenAI's multimodal API format while
maintaining PII/secrets protection on text portions.
Resolves issues with LibreChat Agents feature returning 502 errors.
maximiliancw [Fri, 9 Jan 2026 16:10:15 +0000 (17:10 +0100)]
Refine bearer token and JWT token detection patterns to enforce minimum character requirements;
Updated tests to reflect changes in bearer token format