Every purchase, point, and campaign click at Market4You is now visible the second it happens

At a glance

• MySQL production data captured through change data capture, no application-layer impact
• Every order, loyalty point, and customer update available in ClickHouse within seconds
• Custom schema automation keeps analytics in sync with production without manual migrations

‍

Market4You runs a loyalty and e-commerce platform where every purchase, point redemption, and campaign click is both a transaction and a signal. The production MySQL database handled the application workload reliably: orders, loyalty events, customer records, redemptions. What it wasn't built for was also answering analytical questions on top of that same live workload.

‍

The Infrastructure Problem

Every time the business needed a loyalty analytics view, a campaign performance number, or store-level GMV, someone had to query the production database directly. That meant analytical load competing with the traffic actually running checkouts and redemptions in real time.

There was no changelog. No structured record of what had changed and when. Just the current state of a row, with no way to reconstruct history or trace how a number got to where it is.

‍

Why the Existing Setup Could Not Keep Up

Adding more reporting on top of the production database doesn't solve this. It adds more analytical load to a system that already has a job, and it still leaves nothing decoupled between the two. As Market4You scaled, that pressure would only grow, and it doesn't give any real path to schema evolution as the product changes.

What was actually needed was a separation: production data captured the instant it changes, moved safely into a system purpose-built for analytical queries, without ever touching how the application itself runs.

‍

What Datum Labs Built

We connected directly to MySQL's binary log, the built-in changelog of every insert, update, and delete, with zero impact on the application. Every change becomes a structured event the moment it happens and flows through Kafka, a durable, replayable stream sitting between the source and the analytics layer. If the analytics database is ever briefly unavailable, events queue in Kafka and replay from the last known point. Nothing is lost.

ClickHouse receives that stream and serves it back out in under a second, even at scale, using columnar storage purpose-built for analytical workloads. Loyalty dashboards, campaign attribution, and store performance dashboards query ClickHouse directly, with no intermediate reporting layer standing between the data and the answer.

The harder problem was schema drift. MySQL and ClickHouse don't share a schema language, and every new column or table on the production side would normally require a manual migration on the analytics side. We built automated schema mapping scripts that generate the correct ClickHouse structure directly from the MySQL schema, so evolution on the production side propagates automatically instead of becoming an engineering task every time.

‍

What Changed

• Loyalty analytics update in real time: points, redemptions, and customer segments reflect activity within seconds of it happening
• Campaign performance is tracked as it happens, which offers drove purchases, attributed and visible immediately
• Store-level GMV and order counts update continuously, not on an overnight or scheduled basis
• Schema evolution on the production database no longer requires a manual analytics migration
• Resilience is built in: if the analytics layer goes down, nothing is lost, events queue and catch up automatically on recovery

Nobody on the team is querying production directly to answer a business question anymore. The analytics layer does that job now, and it does it in seconds.

‍

Built to Extend as the Business Grows

Every part of this stack, the Kafka streaming layer, the schema automation, the orchestration on Nomad, is designed to onboard new tables and new data sources without re-architecture. A new table in production means a new topic and an automatically generated ClickHouse table, not a project.

For a business where every transaction and every redemption is also an analytics event, that separation between "the system that runs the store" and "the system that understands it" isn't optional. It's what makes both of them trustworthy.

‍