SAP B1 → Data Warehouse Nightly CDC Sync

Nightly delta extraction from SAP Business One into a data warehouse (SQL/BigQuery). Uses UpdateDate-based CDC to pull changed documents, always re-pulls header + lines together (child rows can change without a header delete), normalizes DocDate/CreateDate timezones, and loads staging via a batch load endpoint.

Steps in this flow

  1. Schedule: Nightly ETL
  2. Read Last CDC Watermark
  3. SL Login (read login, 401→relogin)
  4. SL: GET /Invoices (delta + $expand lines)
  5. For Each Changed Document
  6. Normalize TZ (DocDate/CreateDate) + Header+Lines
  7. Load → DWH Staging (upsert doc+lines)
  8. Persist New Watermark (after successful load)
  9. Log Sync Summary

All templates