Skip to content

Phase 3: Extract Inventory

Status: In progress
Risk: Medium — CQRS introduces a read/write split; CDC pipeline is the critical path.
Rollback cost: Medium — Redis projection must be drained and reconciled; consumer lag reconciliation required before re-routing to monolith.

Goal

Extract the Inventory context with its CQRS architecture. The key challenge is the Oracle → Postgres CDC pipeline and the dual-write window, which involves both the PostgreSQL write model and the Redis read projection.

What Changes

Concern Before After
Inventory data store Oracle (shared) PostgreSQL write model + Redis read cache
Stock level reads Direct Oracle queries Redis GET (sub-1ms)
Stock reservations Monolith Oracle transactions Inventory Service with row-level locking
/v1/inventory/* handler Monolith Java EE Inventory Service (TypeScript/Fastify)
Stock events None StockReserved, StockReleased on Kafka

Extraction Sequence

  1. Bootstrap write model: Debezium CDC from Oracle inventory table → Postgres.
  2. Bootstrap read model: Kafka consumer reads CDC stream and populates Redis projection.
  3. Verify CQRS parity: Reconciliation job confirms Redis stock levels match Postgres stock levels (tolerance: 0).
  4. Dual-write enabled: Inventory Service writes to Postgres; relay writes to Oracle.
  5. Canary start: Kong routes 1% of /v1/inventory/* to Inventory Service.
  6. Canary ramp: 1% → 10% → 50% → 100% (24-hour hold at each weight).
  7. Dual-write disabled: At 100%, relay stopped.
  8. 30-day stability window: Must be stable before Phase 4 (Orders) begins.

Critical Path: CDC Pipeline

The CDC pipeline must drain (lag < 1 second for 30 minutes) before the traffic canary starts. If the pipeline falls behind during the canary, the reconciliation job halts canary advancement automatically.

Monitor: kafka_consumer_group_lag{group="inventory-bootstrap-consumer"} in Grafana.

Go / No-Go Criteria

  • CDC lag < 1 second for 30 minutes before canary starts
  • Stock level divergence = 0 (zero oversell tolerance — INV-INV-001)
  • Inventory Service read p99 ≤ 100ms (Redis projection, all canary weights)
  • StockReserved events appearing in Kafka for all order-driven reservations

Rollback

  1. Kong: set /v1/inventory/* weight to 0 (monolith receives 100%).
  2. If dual-write active: Oracle is consistent. Verify Postgres matches Oracle before stopping relay.
  3. Redis projection: drain by stopping Kafka consumer; it is rebuilt from the Kafka topic on next start.
  4. Trigger: SLA burn rate > 2× for 15 minutes, Kafka lag > 10 seconds for 5 minutes, or oversell event detected.