SAP B1 → Data Warehouse Nightly CDC Sync
Nightly delta extraction from SAP Business One into a data warehouse (SQL/BigQuery). Uses UpdateDate-based CDC to pull changed documents, always re-pulls header + lines together (child rows can change without a header delete), normalizes DocDate/CreateDate timezones, and loads staging via a batch load endpoint.
Steps in this flow
- Schedule: Nightly ETL
- Read Last CDC Watermark
- SL Login (read login, 401→relogin)
- SL: GET /Invoices (delta + $expand lines)
- For Each Changed Document
- Normalize TZ (DocDate/CreateDate) + Header+Lines
- Load → DWH Staging (upsert doc+lines)
- Persist New Watermark (after successful load)
- Log Sync Summary