Skip to main content

Peliqan

change-data-capture-diagram-feature-image

Change data capture diagram: the three CDC methods

InfoGraphics

change-data-capture-diagram

Change data capture is how a pipeline notices that a row changed without re-reading the whole table. There are three ways to do it, they are not equivalent, and the diagram below shows what each one watches and what it costs the source system.

The three methods

  1. 1Query-based CDC polls a timestamp column. Select everything where updated_at is newer than the last run. It works on almost any source and needs no special access, which is why it is so common. It also cannot see deletes, and it misses changes that do not update the timestamp.
  2. 2Trigger-based CDC writes every change to a shadow table. A database trigger records inserts, updates and deletes as they happen, so nothing is missed. The cost is that the trigger runs inside the source transaction, adding write load to the production system your business depends on.
  3. 3Log-based CDC reads the database’s own write-ahead log. The log already exists for crash recovery, so reading it adds no load to the tables and captures every change in commit order, deletes included. It needs elevated database access, which is the main reason teams do not start here.

Why deletes are the deciding factor

A cancelled order that is deleted in the source but never deleted downstream keeps counting toward revenue, quietly, for as long as that table exists. Query-based CDC cannot see it. Teams usually discover this during a reconciliation, months after the gap opened, which is why ‘does it capture deletes’ is the first question to ask of any replication setup.

Where CDC is actually used

  • Keeping a warehouse in sync with production without a nightly full reload.
  • Migrating a database with minimal downtime, by replaying changes while the old system still runs.
  • Feeding streaming pipelines and event-driven systems from an existing relational database.
  • Maintaining an audit history of what changed and when, which the source table itself usually does not keep.

How CDC relates to ELT

CDC is the extract step done incrementally. The transform still happens afterwards, in the warehouse, which is the pattern drawn on our ETL vs ELT diagram. Where it fits in the wider flow is on the data pipeline architecture diagram.

What this looks like in Peliqan

Peliqan syncs incrementally from 300+ sources, using log-based capture where the source database supports it and the API’s own change endpoints where it does not, so you get incremental loads without maintaining capture logic per source.

Use this diagram wherever you like

The diagram is free to use, including commercially, as long as there is a visible link back to this page. Download the PNG for slides and documents, or the SVG if you want to edit the labels. No email required. You can browse the rest of the set in the Peliqan diagram library.

Ready to build this on your own data? Get started with Peliqan.

FAQs

Change data capture, or CDC, means identifying and moving only the rows that changed in a source system instead of re-reading the whole table. It is what makes incremental syncs and near-real-time replication possible.

Three are in common use: timestamp or query-based CDC polls an updated_at column, trigger-based CDC records changes through database triggers, and log-based CDC reads the database’s write-ahead log. Only the latter two capture deletes.

In ETL and ELT, CDC is the extract step done incrementally. The transform still happens afterwards, usually in the warehouse, so CDC governs how much data moves rather than how it is modelled.

In SQL databases CDC is usually log-based, reading the transaction log the engine already writes – the PostgreSQL WAL or the MySQL binlog. SQL Server also ships a built-in feature called Change Data Capture that records row changes to system tables.

Get instant access to all your company data

Connect 300+ sources, serve any BI tool, and give every AI agent one governed endpoint to read, and write back where the app supports it.