System Topology

A deep dive into the Kubernetes-native data plane and multi-layered detection pipeline of the Offline Security Auditor (LogShield).

01. Control Plane & Sidecar Injection

The architecture operates natively within a Kubernetes cluster using a Mutating Admission Webhook. When a developer deploys a new Pod to a namespace labeled with logshield.io/enabled=true, the control plane intercepts the request.

It dynamically injects the LogShield Sidecar container and required shared volumes (\`shared-logs\`, \`sidecar-data\`) directly into the Pod specification. This zero-config approach ensures that the primary application requires no code changes to have its logs securely audited and masked.

02. Data Plane: Secret Confidence Score (SCS)

High-Speed Filtration

The pipeline begins with a Preprocessor that recursively decodes nested JSON, URLs, and Base64 strings. To maintain extreme throughput (over 50k lines/sec), we implemented an Aho-Corasick automaton. This Candidate Finder rapidly filters out non-sensitive lines in linear time, preventing expensive regex evaluation on safe data.

Contextual Proximity

The Context Engine performs proximity analysis on the text surrounding potential secrets. It weighs positive context (e.g., password=, Authorization: Bearer) against negative context (e.g., trace_id=, request_hash=) to feed feature vectors to the scoring model.

ML Scorer & Vault Integration

The final stage uses a trained Machine Learning model evaluating features like Shannon entropy, token structure, regex matches (600+ patterns), and contextual proximity to generate a Secret Confidence Score (0-100). Scores ≥80 trigger redaction. The original secret is sent to SecureReveal—an encrypted local SQLite vault—leaving behind a safe reference such as [REDACTED:SEC-1234:SCS=94] for future auditing.

03. Secret Lineage & Tracking

One of the hardest problems in distributed microservices is tracing where a secret originated without exposing the secret itself. LogShield solves this via SecretLineage.

Upon detecting a secret, the sidecar generates a keyed HMAC fingerprint. This fingerprint acts as an immutable identifier for that exact secret string. As the secret traverses different services (and gets logged by them), LogShield masks the secret but emits the same fingerprint. This allows security teams to map the blast radius and sprawl of a leaked credential across the entire cluster without ever viewing the plaintext.