Correct, cheap brand attribution from one raw stream

Problem

A brand-intelligence product needed near real-time reputation monitoring for retail brands: pull raw social posts, enrich each one with AI sentiment, and hand a mobile app a clean, per-brand view of how a brand is being talked about. One brand equals one account. The backend computes every metric; the app only presents.

The hard part was never the sentiment model. It was attribution. Every post had to land against the right brand, from a single raw, unfiltered stream, without draining the budget that fed it.

Constraint

Two pressures pulled in opposite directions.

The upstream field identities were opaque and dirty. Multiple clients collided inside the same field, so a naive read mixed brands together. Getting attribution right the obvious way meant wide groupBy queries, and those query patterns were expensive enough to drain the very stream they read from.

So the requirement was not "accurate attribution" or "cheap attribution." It was both, at once, on live data.

Architecture decision

I split the problem into two layers instead of paying for one heavy query.

Canonical ingestion. The inbound payload is treated as a scope carrier, not as a query to run verbatim. Ingestion builds its own fixed, lean query from that scope, which keeps custom fields lossless and the metric set consistent regardless of how messy the source shape is. The pipeline owns its cost profile instead of inheriting the caller's.

Prefix routing. Brand attribution is derived from the prefix of a per-message tag rather than from a heavy groupBy. Each message routes into a role bucket (own brand, competitor, topic) off that prefix. When the routing map is empty, the pipeline falls back to legacy behavior, so the feature is fully backward compatible and opens behind a config flag. Design for the many, ship for the one.

Result

A single pull attributed 3,725 posts at 99% accuracy and computed roughly 16.7M in total reach. The backend runs about 600 tests green, a live probe measured prefix attribution at 97%, and the mobile app is live on TestFlight.

The sentiment layer itself is a production LLM integration: raw posts in, structured JSON metrics out, consumed directly by the app with no human in the loop. The engineering was making that output trustworthy at stream scale, validated, correctly attributed, and cheap enough to run continuously.

The attribution moved from an expensive, brand-mixing read to a cheap, prefix-driven route that stayed correct on the raw stream, without a rewrite and without breaking the existing path.