Skip to main content

Denial Event Pipeline

Every request that the proxy rejects produces a denial event. These events are queued separately from usage events and written to ClickHouse for audit, analytics, and alerting.

Denial Points

The proxy has 8 distinct points where a request can be denied:

Pre-Authentication (before key lookup)

DenialReasonHTTP Status
Missing API keyNo Authorization: Bearer header present401
Invalid key prefixKey does not match expected prefix format401
Hash not foundHMAC hash of the key does not exist in KV401

Post-Authentication (key found but request rejected)

DenialReasonHTTP Status
Inactive keyKey exists but is disabled/revoked403
Unknown providerProvider path segment not recognised400
Provider blockedKey's policy does not allow this provider403
Dimension validationX-ASO-* headers fail schema validation400
Model blockedKey's policy blocks this specific model403

Dedicated Queue

Denial events use a separate queue (denial-events-{env}) from usage events (usage-events-{env}). This separation ensures:

  • Denial spikes (e.g., a misconfigured client hammering with bad keys) do not delay usage event processing
  • Each consumer can scale independently
  • Monitoring and alerting can be scoped per queue

Denial Consumer

The denial consumer (apps/denial-consumer) is a Cloudflare Worker bound to the denial-events-{env} queue. It receives batches of denial events and writes them to ClickHouse.

The consumer is simple compared to the usage consumer — no pricing enrichment is needed. It validates the event schema, adds server-side timestamps, and inserts into ClickHouse.

Denial Event Fields

FieldTypeDescription
event_idstringUnique event ID (UUID) for deduplication
typestringDenial type (e.g., missing_key, inactive_key, provider_blocked)
reasonstringHuman-readable denial reason
http_statusnumberHTTP status code returned to the client
tenant_idstringTenant ID (null for pre-auth denials)
api_key_idstringAPI key ID (null for pre-auth denials)
providerstringRequested provider (null if not parsed)
modelstringRequested model (null if not parsed from body)
dimsobjectDimension headers present on the request
timestampdatetimeWhen the denial occurred
envstringEnvironment (dev or prod)
source_ipstringClient IP (hashed for privacy)
user_agentstringClient user-agent header

ClickHouse Table

Denial events are stored in a dedicated ClickHouse table partitioned by month and ordered by (tenant_id, timestamp). The event_id field is used as the deduplication key to handle at-least-once delivery from the queue.

Pre-auth denials (where tenant_id is null) are stored with a sentinel tenant ID for queryability.