Architecture · Webhooks & sync
Webhooks & delta sync with a unified affiliate API
Updated June 22, 2026 · ~15 min read
Polling a normalized list API on a fixed cadence is the reliable baseline for affiliate data pipelines. It is also expensive during quiet periods and slow during flash promos. Push notifications — webhooks that describe what changed after a sync pass — let you run delta workers instead of full-table scans. This guide covers polling vs push, cache invalidation, idempotent handlers, and how Feedico meters webhook payloads with resource units.
Reference flow
1. Upstream sync
Feedico pulls CJ / Awin / Impact on your credentials
2. Delta outbox
Changed firm & coupon ids enqueue affiliate.delta
3. Your worker
Verify signature → upsert → bust cache tags
Webhooks describe what changed; they do not replace your obligation to validate rows against programme rules before displaying them.
Polling vs push
| Approach | Strengths | Weaknesses |
|---|---|---|
| Fixed-interval polling | Simple cron; works on all plans; easy to reason about | Wastes API quota when nothing changed; slower promo freshness |
| Webhook-triggered delta | Pull only after upstream sync; lower median latency | Requires HTTPS endpoint, signature verification, retry handling |
| Hybrid (recommended) | Webhooks for speed + nightly full reconcile for safety | Two code paths to test — still cheaper than N network clients |
List endpoints are documented on the coupon API landing; merchant dimension sync uses the same pagination semantics on affiliate API list routes.
Delta payload shape (affiliate.delta v1)
Payloads are versioned and scoped to a provider and property_id. Id arrays reference Feedico internal ids — fetch full rows via list APIs or your warehouse join keys (provider + externalCouponId).
{
"webhook_version": 1,
"event": "affiliate.delta",
"occurred_at": "2026-06-20T14:22:11.000Z",
"provider": "cj_affiliate",
"property_id": "12844",
"networks": {
"upserted": ["88341", "88342"],
"inactivated": ["88101"]
},
"coupons": {
"upserted": ["1849201", "1849202", "1849203"],
"inactivated": ["1840001"]
},
"truncated": false
}Webhook resource units (metering pattern)
Feedico meters outbound webhooks with resource units: each entity id listed in networks.upserted, networks.inactivated, coupons.upserted, or coupons.inactivatedcounts as one unit against your plan's monthly webhook budget. Plans also cap deliveries per UTC day.
| Scenario | System behavior |
|---|---|
| Small delta (10 ids) | Full id lists delivered; units = 10 |
| Large sync exceeds remaining monthly units | Payload trimmed to budget; truncated: true set |
| Daily delivery cap reached (UTC) | Events queue in outbox for next UTC day |
| Worker receives truncated payload | Run targeted paginate with provider filter + fetchedAt window |
Design workers to handle truncation gracefully — it is a backpressure signal, not a failure. Monitor usage in the dashboard webhook panel alongside API request meters.
Idempotent worker design
Webhook providers retry on non-2xx responses; at-least-once delivery is the norm. Handlers must tolerate duplicates:
- Verify HMAC signature before parsing JSON.
- Upsert warehouse rows on composite keys — never blind insert.
- Process inactivation lists before assuming upserts cover full state.
- Log webhook delivery id + payload hash for support correlation.
- Return 2xx only after work is enqueued or committed — not merely received.
async function handleAffiliateDelta(payload) {
// 1. Verify signature (HMAC) before this function runs
for (const networkId of payload.networks.upserted) {
await upsertNetworkFromApi(networkId);
}
for (const couponId of payload.coupons.upserted) {
await upsertCouponFromApi(couponId);
}
for (const couponId of payload.coupons.inactivated) {
await markCouponInactive(couponId);
}
if (payload.truncated) {
await paginateProviderDelta(payload.provider, payload.property_id);
}
await bustCacheTags(payload.provider);
}Cache invalidation
Frontends should not call list APIs on every page view. Typical invalidation tags:
provider:{slug}— bust merchant lists filtered by CJ vs Awin.merchant:{externalMerchantKey}— coupon carousels on store pages.homepage-deals— aggregate materialized views after large deltas.
CDN and edge caches need explicit tags — webhook handlers should map id lists to tags via your warehouse index before purging.
Recommended hybrid architecture
- Baseline cron: full paginate
/me/coupons+/me/networksnightly. - Webhook path: on
affiliate.delta, enqueue incremental job for the provider in the payload. - Truncation fallback: if
truncated, run provider-scoped list pull with modest pageSize. - Validation: alert if coded coupon count drops sharply vs previous run (often auth or filter misconfiguration).
- Publish: invalidate cache tags; optional fan-out to internal microservices.
Failure modes to plan for
Endpoint down during promo
Queued deliveries resume; nightly cron still reconciles.
Signature mismatch
Reject request; rotate secret if compromise suspected.
Partial warehouse write
Use transactions per batch; never delete-all-then-insert.
Unit budget exhausted mid-month
Expect more truncated payloads; increase plan or reduce noisy properties.
Warehouse integration
The coupon warehouse ETL guide documents full paginate workers into SQLite or MySQL. Add a webhook consumer that translates id lists into targeted upserts or marks rows inactive — same composite keys documented in schema normalization. For industry context on why push layers matter, see affiliate API fragmentation 2026.
Frequently asked questions
- Should I replace polling with webhooks entirely?
- No. Use webhooks as an acceleration layer on eligible plans: they tell you when to pull deltas. Keep a scheduled full reconcile (daily or weekly) to catch missed deliveries, truncated payloads, or endpoint downtime.
- What is an affiliate.delta webhook?
- Feedico emits versioned JSON (webhook_version: 1, event: affiliate.delta) listing network and coupon ids that were upserted or marked inactive after a sync pass for a given provider and property. Your worker fetches or applies changes idempotently.
- What are webhook resource units?
- Each changed entity id referenced in a delivered webhook payload counts as one resource unit against your plan monthly cap. Large syncs may truncate id lists to fit remaining budget; truncated: true signals you should run a targeted list pull.
- How do I verify webhook authenticity?
- Configure a signing secret in the dashboard. Verify HMAC signatures on incoming requests before enqueueing work. Treat retries as duplicates — upsert handlers must be idempotent.
- How does this relate to coupon warehouse ETL?
- Warehouses usually poll on a cadence. Webhooks reduce unnecessary full paginations by triggering incremental jobs when Feedico finishes a provider sync. Pair both: webhooks for freshness, cron for reconciliation.
- What if my plan does not include webhooks?
- Polling POST /api/v1/me/coupons and /me/networks on a fixed schedule remains the supported baseline. Design workers with exponential backoff on rate limits and compare row counts run-over-run for anomaly detection.
You need programme approval and compliant use at each affiliate network. Feedico provides the integration layer - not a substitute for network terms.