The AI Pipeline That Turns Search Results Into Trustworthy Competitor Intelligence

At a glance

• An automated, self-healing AI pipeline that turns raw search data into ranked competitor intelligence
• Wrong-firm error rate driven from 40% down to 0% through evidence-based model selection
• Unit cost locked in at roughly $0.002 per firm, so $100 covers roughly 48,000 firms

Westwise Analytics needed to know its competition, automatically, accurately, and at scale. On its own, a search result says nothing about what a firm actually does, where it practices, or how seriously it competes. Producing that picture by hand, reading each website, classifying its services, mapping its territory, does not scale to the volumes the business needs.

The prize was replacing that manual research with an AI system that reads each firm's own website and produces a trustworthy, scored profile on its own. Two risks stood in the way. Trust: an AI that confidently names the wrong firm corrupts every field in that record. Cost: uncontrolled AI usage that makes scaling to millions of firms financially unviable.

‍

A naive automation would have introduced its own problems

The AI could get it wrong, occasionally identifying the wrong firm or returning malformed answers, and a single wrong-firm result poisons every field in that record. Quality could break silently, with a small instruction change quietly dropping records with no visible error. Cheaper-looking shortcuts could cost more, forcing a second AI step just to interpret raw page code. And the system could run blind on a fixed timer, wasting runs whether or not fresh data existed.

‍

Trust and cost were the two risks,
_{not one}

For Westwise, the failure mode wasn't a missing capability. It was a system that could look confident while being wrong, and a system that could get expensive before anyone noticed. At the volumes this business needs to operate at, that's not an inconvenience, it's a dataset nobody can stand behind.

Every stage the pipeline relies on, connected into one automated system:

‍

Why the standard answer didn't fit

The instinctive fix is to hire analysts to do this by hand, or point an AI at it without checking its work. Manual research doesn't scale to tens of thousands of firms. An unchecked AI scales, but silently corrupts the dataset the moment it names the wrong firm with confidence.

Westwise didn't need faster manual research or a naive automation. They needed a pipeline that used AI only where it could be trusted, validated every output before it reached the warehouse, and stayed cheap enough to run at real volume.

‍

What Orbit built

The architecture runs on one constraint: every result is validated before it's trusted, and every failure is caught and retried automatically rather than lost.

• An AI enrichment step that reads each firm's website in a single call and produces a scored profile, no separate scraping stage
• Evidence-based model selection, benchmarking two candidate models head to head and choosing the one with zero wrong-firm errors, not the larger, more expensive default
• A validation layer that checks every result against a strict format before it's ever stored, malformed answers never reach the warehouse
• Event-driven batching that runs only on fresh data, exactly once, so nothing gets processed twice
• Automatic retry and recovery, with failed records held safely and promoted the moment they succeed
• Safe-to-tune AI instructions, with the parts that protect data quality locked so a prompt edit can never silently break the output

‍

Tech Stack:

‍

How the rollout happened_‍

1. Chose the model on evidence

Both candidate models were scored on a like-for-like accuracy test before either one touched production. The larger model produced wrong-firm errors on 40% of records. Gemini 2.5 Flash produced none, and it was cheaper too.

2. Engineered the cost down

Every approach was benchmarked on real websites before being locked in. Reading the page directly settled at roughly $0.002 per firm, an optional live-search add-on was tested and switched off, and two alternative crawling services were evaluated and rejected for cost or quality reasons.

3. Built the self-healing loop

The pipeline moved from a blind daily timer to event-driven automation that runs only when fresh data is confirmed ready. Failure handling was added so any record the AI can't process is captured safely and retried automatically, with de-duplication ensuring the system never pays to analyze the same firm twice.

4. Wired up observability

Every AI call is tracked for cost, speed, and accuracy from day one. Prompts are versioned and traceable, so improvements can be tested without engineering changes, and failure reasons surface clearly instead of disappearing into logs.

‍

What changed for the team

Before, competitor intelligence meant analysts manually reading websites, classifying services, and scoring threats by hand, a process that couldn't scale past a fraction of the firms the business needed to track.

• Competitor profiles get scored automatically, at roughly $0.002 per firm
• Wrong-firm errors dropped from 40% to 0% on the evaluation set
• Failed records recover on their own, with no manual re-processing
• Cost and quality are visible per call, not discovered after the fact
• The dataset scales from one firm to ten million without the unit economics changing

The data was always there, sitting in search results nobody had time to read one by one. Orbit built the pipeline that reads it, scores it, and proves it's right, automatically.