Category: Integrations

Your order sync ran fine all week. Then at 3am on Saturday an access token expired. Your store kept taking wholesale orders. Nothing reached the ERP.

No email went out. The dashboard stayed green, because a job that never starts cannot turn red.

On Monday your warehouse manager asks why the pick list is short. Someone opens the logs. Forty orders are sitting in the store with no match downstream, and the two oldest were promised for Monday delivery.

The short answer: Integrations are built for the day they work, not the night they stop. Add alerts that reach a person with the order reference attached, a dead letter queue somebody reviews, a daily count comparing orders in your store against orders downstream, and a heartbeat for every scheduled job. Monitoring is what turns an integration into an automation.

A silent integration failure is a break in an automated flow that stops work moving without producing any signal a human being actually sees.

Almost every integration is tested on the happy path. An order is created, the API accepts it, the record appears downstream, everyone signs off. That test proves the flow works when both systems are awake, the token is valid and nothing is under load. It proves nothing about the other case, and the other case is the one that costs money.

Stated plainly: an automation without monitoring is a manual process with extra delay. A person still finds the problem. They just find it later, with more orders affected, and after your buyers have noticed.

How does an order flow stop without telling you?

Each of these breaks differently, and each one needs a different thing watching it. The middle column is what the failure looks like from the outside, which is usually all you have.

Failure mode	What it looks like	How you detect it
Access token expired or revoked	Everything stops at once. Calls return 401 or 403	Alert specifically on authentication errors. Put credential expiry dates in a calendar
Webhook delivered once, never retried	One or two orders missing, the rest perfectly fine	Log a receipt per event. Reconcile daily counts
Queue blocked behind one poisoned message	Everything after a certain timestamp stops, nothing before it	Alarm on queue depth and on the age of the oldest message
Rate limiting, HTTP 429	Intermittent gaps, worse on busy days and during bulk edits	Count 429 responses. Alert when retries are exhausted, not on the first one
Schema or required field changed at one end	One order type fails, every other type works	Validate the payload and alert on validation errors by type
Timeout mid-write	A half-written record, or a second copy of the same order	Reconcile field by field. Flag orders with no line items or no totals
Scheduled job did not run	Nothing. No errors, no records, no signal at all	A heartbeat. Alert on the absence of a run, not on a failed one

Notice the pattern. Five of these produce an error somewhere. Two produce nothing, and those two are the expensive ones. Error alerts can only tell you about work that was attempted.

The 429 row is a documented, ordinary condition rather than an emergency. RFC 6585 defines 429 Too Many Requests as the signal that a client has sent too many requests in a given period, and says the response may include a Retry-After header telling you how long to wait. Shopify's API rate limit documentation describes a leaky bucket that refills at a fixed rate and instructs apps to stop calling and back off when they are throttled. Handled properly, 429 costs you nothing. Handled as a hard failure, it drops orders on your busiest days.

The timeout row deserves its own reading, because a write that succeeded but reported failure is how the same wholesale order ends up in your system twice. That mechanism is covered in why the same wholesale order arrives twice.

What do the platforms actually promise when delivery fails?

Two documented examples, both from systems B2B teams genuinely run. These are published behaviors, not estimates.

Shopify webhooks stop trying, then stop existing. Shopify's webhook troubleshooting documentation states that Shopify retries failed webhook calls up to eight times over a four-hour period, that your endpoint has five seconds to respond before a delivery counts as failed, and that after eight failed attempts Shopify stops attempting delivery. It goes further: if failures persist, the subscription is removed, and removed subscriptions receive no deliveries until you create them again.

Read that as an operations fact. A four-hour outage on your receiving endpoint that starts at 3am is enough to lose every order in that window permanently, and enough to leave you with no subscription at all by morning. Nothing on your side errors, because nothing on your side is called. The only controls that catch this are a heartbeat on the receiver and a daily count.

Accounting tokens expire on a clock, not on an event. Xero's OAuth 2.0 FAQ documents a 30-minute access token and refresh tokens that expire after 60 days if unused, after which the user has to reauthorize the app. That is a dated fuse on every connection you have. An integration that goes quiet over a slow trading period can pass 60 days without anyone touching it, and the reconnection then needs a human with the right login.

Both cases share a shape. The vendor documented exactly how it fails. Nobody on the buying side read it during the build, because the build was about making it work.

Third-party logistics providers add their own version of this, covered separately in 3PL integration errors.

What does a sync that stopped on Friday night cost?

Numbers first, then the part you cannot invoice. These are illustrative. Put in your own order volume and hourly rate.

Say your order flow stops at 21:00 on Friday and somebody spots it at 09:00 on Monday.

Line	Illustrative number
Wholesale orders stuck over the weekend	18
Manual re-entry, 8 minutes each	2.4 hours
Re-entry cost at €35 per hour	€84
Orders that missed Monday's 3PL cutoff	18
Orders short-shipped because stock levels never came back to the store	3
Credit notes and reshipments at €40 each	€120
Customer service time chasing it, 3 hours	€105
Visible total	about €309

Illustrative figures. Redo the arithmetic with yours.

The €309 is not the problem. The problem is 18 orders that now ship on Tuesday instead of Monday, to buyers who run replenishment on a schedule and rate you on exactly this. One of them is your largest account.

And the cost scales with detection time, not with order volume. The same failure caught at 07:00 on Saturday costs one phone call and a token refresh.

What does good monitoring look like?

Five things. None of them require a monitoring platform.

1. Every failure lands somewhere a human sees, with the order reference attached. A Slack channel or a shared inbox is fine. The message must carry the order number or purchase order reference, the target system, and the error. An alert that says "sync failed" sends someone into the logs. An alert that says "order WH-4182 rejected by ERP: SKU not found" can be acted on from a phone.

2. A dead letter queue that is reviewed, not just written to. The pattern is standard infrastructure. Amazon's SQS dead-letter queue documentation describes a queue that receives messages which failed processing after a configured number of attempts, set by maxReceiveCount, so they can be isolated and diagnosed rather than lost, and it points to setting a CloudWatch alarm on messages arriving there. That alarm is the part teams skip. Whatever you build on, the rule is the same: failed messages need somewhere to land, and a named person needs a standing 10-minute slot twice a week to open it. A dead letter queue nobody opens is a folder where orders go to be forgotten.

3. A daily reconciliation count. Its own section, below.

4. A heartbeat for anything scheduled. If a job should run every hour, have it report that it ran, then alert when the report does not arrive. This is the only way to detect a job that silently did not start, and it is usually a few lines of code.

5. A named owner and a short runbook. One page. What this integration does, what breaks most often, how to replay a failed message, who to call at the other end, and how to enter an order by hand if the flow is down for a day. Name a person, not a team. Teams do not get out of bed.

Why does daily reconciliation catch what error alerts miss?

Error alerts watch the work that was attempted. Reconciliation watches the work that should have existed.

The control is simple. Once a day, count the orders created in your store yesterday. Count the orders downstream with a matching reference. Compare. If the numbers differ, list the references that are missing.

Two numbers and a list. That catches the webhook that exhausted its eight Shopify retries at 4am, the job that never ran and the queue that quietly stalled, because none of those leave an error on your side to alert on.

Run it in the morning, before your dispatch cutoff, so there is still time to act. Count in both directions if orders can also be created downstream.

If you are scoping a new connection, this belongs in the build, not in a later phase. It is on our B2B e-commerce integration checklist for that reason.

How do you avoid alert fatigue?

The failure mode after no alerts is too many alerts.

This is well documented ground. Google's SRE book chapter on monitoring distributed systems puts it bluntly: when pages occur too frequently, people second-guess, skim or ignore incoming alerts, sometimes ignoring a real page masked by the noise. It also sets the test worth stealing, that every page should be actionable, and that effective alerting systems have good signal and very low noise.

Translate that into three rules for an order flow:

• Alert on the outcome, not the attempt. A single 429 followed by a successful retry is not news. Retries exhausted is news.
• Aggregate the repeats. Fifty orders rejected for the same reason is one alert with a count, not fifty alerts.
• Every alert needs an action. If nobody would do anything differently on receiving it, it is a log line.

Then hold yourself to a number. If your alert channel gets more than a handful of messages on a normal day, the noise is the bug, and fixing it comes before adding anything new.

When is monitoring not worth building?

Say no when:

• The volume is tiny. Five orders a week through a flow where a person already checks every one. They are the monitoring.
• The flow is not time critical. A nightly marketing export that lands a day late costs nothing. Reconcile it weekly and skip the alerting.
• You would build a monitoring system instead of a webhook. The first version is a message into a channel you already use.
• The integration is being replaced this quarter. Instrument the one that survives.
• Nobody will own it. Alerts with no owner are noise with a timestamp.

How do you measure whether it worked?

Write these down now, before you change anything.

• Time to detect. From the moment a flow stops to the moment a person knows. This is the number the whole exercise moves.
• Silent failures found by reconciliation. Breaks that produced no error at all. Above zero means alerting alone was never enough.
• Alerts per normal day, and the share somebody acted on.
• Age of the oldest item in the dead letter queue.
• Orders affected per incident. This falls as time to detect falls.

Check again after a few normal weeks. Time to detect dropping from two days to twenty minutes is the result, even in a month where nothing broke.

Do this in the next hour

1. List every integration that moves orders, stock or invoices. Most teams find more than they expected.
2. Open the vendor documentation for each one and write down what it does when your endpoint fails, and when its credentials expire. Two of those answers will surprise you.
3. Add one alert to the flow with the highest order value, with the order reference in the message.
4. Add the daily reconciliation count for that same flow.
5. Put a name and a phone number next to it.

That is a morning's work, and it covers the failures that cost the most. It is the same Find, Prioritize, Automate methodology applied to reliability instead of speed, and it is how we build software integrations in the first place.

Not sure what your current setup does when it fails? Book a 30-minute call and we will go through one flow with you.

FAQ

What is a silent integration failure?

It is a break in an automated flow that stops work moving without producing a signal anyone sees. Common examples are a scheduled job that never started, a webhook whose retries were exhausted overnight, and a queue stalled behind one bad message. None of these generate an error on your side to alert on, which is why counting records in both systems catches them and error alerting does not.

How do I monitor webhooks for my e-commerce store?

Log a receipt for every event you accept, including the event identifier, so you can tell a repeat from a new one. Alert when processing fails after retries are exhausted, not on the first failure. Then compare daily counts between the sending system and the receiving one, because a delivery that was abandoned after its final retry leaves no trace on your side at all.

How many times does Shopify retry a failed webhook?

Shopify's documentation states that it retries failed webhook calls up to eight times over a four-hour period, and that your endpoint must respond within five seconds or the delivery counts as failed. After eight failed attempts Shopify stops attempting delivery, and if failures persist the subscription is removed and has to be created again. Treat a four-hour outage as permanent order loss.

What is a dead letter queue and do I need one?

A dead letter queue is where messages go after they fail processing repeatedly, so they are isolated for diagnosis instead of lost. You need one as soon as failing a message means losing an order. The important part is the review: alarm on messages arriving there, and book a short recurring slot for a named person to clear it, because a dead letter queue nobody opens is just a place orders go to be forgotten.

How often should I reconcile orders between my store and my ERP?

Daily is right for most B2B teams, run in the morning and before your dispatch cutoff so there is time to act on what it finds. Count orders created yesterday in the store, count the matching records downstream, and list any references that are missing. If your order volume is high or your margin per order is large, run it more often.

How do I stop integration alerts from becoming noise?

Alert on outcomes rather than attempts, so a retry that succeeded stays silent and only exhausted retries reach a person. Aggregate repeats into one message with a count. Then apply the test from Google's SRE practice: every alert should be actionable. If nobody would do anything differently on receiving it, it belongs in a log, not in the channel people are supposed to read.

Sources

• Shopify: troubleshoot webhooks supports the eight retry attempts over four hours, the five-second response deadline, and removal of the subscription when failures persist.
• Shopify: API rate limits supports the leaky bucket model, the 429 throttle response and the instruction to stop calling and back off.
• IETF RFC 6585: Additional HTTP Status Codes supports the definition of 429 Too Many Requests and the Retry-After header.
• Xero: OAuth 2.0 FAQ supports the 30-minute access token and the 60-day expiry of unused refresh tokens requiring reauthorization.
• Amazon SQS: using dead-letter queues supports the dead letter queue pattern, maxReceiveCount, and alarming on messages that arrive there.
• Google SRE Book: Monitoring Distributed Systems supports the alert fatigue argument, the "every page should be actionable" test, and good signal with very low noise.
• Google Cloud Pub/Sub: subscription overview supports at-least-once delivery as the default behaviour of message systems, and the requirement that consumers be idempotent.

A buyer calls about a credit note. They received two pallets of the same order, on the same day, from you.

Your system shows two orders, three minutes apart. Same purchase order number, same lines, same delivery address. Everyone assumes somebody clicked twice.

Usually nobody did. The sender never got a confirmation and tried again, and the second attempt landed on a write that had already succeeded. Then the warehouse did exactly what it was told, twice.

The short answer: Duplicate orders are almost always a retry problem, not a data quality problem. A write succeeds, the confirmation never comes back, and something tries again. Fix it by sending a stable reference on every write, upserting against that reference instead of inserting blindly, and storing the downstream identifier back on the source order.

An idempotency key is a unique value the sender attaches to a write so that repeating the same request produces the same single result instead of a second record.

Why are duplicate orders a retry problem, not a data problem?

Teams reach for the wrong tool here. They add a duplicate report, a cleanup script, a rule in the ERP, a person who checks the order list every morning. All of that treats the symptom.

The cause is almost always a system that could not tell whether its last write worked.

The IETF working group draft on the Idempotency-Key HTTP header field describes the situation exactly: a client sends a POST request and the request times out, leaving the client uncertain about the status of the resource. It does not know whether the record was created, whether the server finished processing, or whether it can safely retry. Repeating the request, the draft notes, can result in duplication.

That is your duplicate order, written down as a protocol problem years before it reached your warehouse. The sender had two bad options and picked the one that ships twice instead of not at all.

Which means the fix belongs at the write, not in the cleanup.

What are the six ways the same order gets created twice?

Cause	Tell-tale sign	Fix
Request timed out after the write succeeded	Two records seconds apart, identical payload, only one logged confirmation	Idempotency key on the request
At-least-once delivery replaying the same event	The same event identifier processed more than once, often minutes apart	Record processed event identifiers and ignore repeats
Batch job rerun after a partial failure	A block of duplicates all created within the same minute	Upsert on the external reference instead of insert
Two writers: two integrations, or one integration and one person	Duplicates with different creators, or slightly different field formatting	One system owns the write. The others read
Double submit on a manual entry screen	Two records within seconds, same user, same session	Disable the button on submit and key the write server side
Order edited so the downstream system creates a new record	The original plus a near-identical record with one line changed	Update by reference on edit. Make cancel and replace explicit

Five of these six are machine behavior. Only one involves a person clicking twice, and it is the one everybody blames first.

Worth noting where the duplicates land. If your warehouse or third-party logistics provider receives orders through its own connection, a duplicate can reach the dock without ever appearing twice in your ERP. That class of problem is covered in 3PL integration errors. And if purchase orders arrive as PDFs that get read automatically, the same order emailed twice by a buyer looks like two orders unless you match on the purchase order number, which is part of AI order entry accuracy.

What do real systems document about duplicates?

Two concrete, published examples. Neither is an edge case. Both are the documented normal behavior of systems B2B teams run every day.

Shopify tells you to expect duplicate webhooks and gives you the tool to drop them. Shopify's guidance on ignoring duplicate webhooks states that your app might receive the same webhook more than once, for example after a network timeout or a retry, and recommends two options: process webhooks using idempotent operations so receiving the same one twice does not change the outcome, or extract the X-Shopify-Webhook-Id header, check it against a persistent store, and skip processing if you have seen it. It also distinguishes that header from X-Shopify-Event-Id, which correlates deliveries originating from the same merchant action, so more than one subscription to the same topic produces separate deliveries you can still tie back to one event.

The retry side is documented too. Shopify's webhook troubleshooting page states that failed calls are retried up to eight times over a four-hour period, and that your endpoint has five seconds to respond before a delivery counts as failed. A slow endpoint that finishes the work in six seconds has created the order and reported failure. Shopify then retries, correctly, and you have two.

Stripe replays the original result instead of doing the work again. Stripe's idempotent requests documentation describes the pattern in full: the client generates a unique key, sends it in an Idempotency-Key header, and Stripe saves the status code and body of the first request made with that key, whether it succeeded or failed. Subsequent requests with the same key return that same saved result. Three details matter for order integrations. Keys are up to 255 characters. The idempotency layer compares incoming parameters against the original request and errors if they differ, which stops a changed order quietly reusing an old key. And keys can be removed after they are at least 24 hours old, after which a reused key generates a new request.

That last point is the one to take home. An idempotency key protects you against retries within a short window. It is not a permanent record of what you have already created. For an order that can be resent tomorrow, you also need a stable reference stored on both sides.

Message systems say the same thing in their own words. Google Cloud's Pub/Sub subscription overview states that at-least-once delivery with no ordering guarantees is the default on all subscription types, that messages may be delivered more than once, and that accommodating this requires your subscriber to be idempotent. Their exactly-once delivery feature exists, but only for pull subscriptions. Push and export subscriptions do not support it.

Duplicates are not a defect in these systems. They are the advertised behavior, and the receiver is expected to handle them.

What are the fixes, in order of value?

Do them in this order. Each one is cheaper than the one that follows it, and each one catches what the previous one misses.

1. Put an idempotency key or stable external reference on every write. One value derived from the order itself, not from the moment of sending. Your store order ID is usually the right choice. A timestamp or a random value generated per attempt is the wrong choice, because a retry generates a new one and you are back where you started.
2. Upsert against that reference instead of inserting blindly. Look for a record with this reference. If it exists, update it. If it does not, create it. One decision, one place, every time. This is the single change that removes most duplicates, and it works even for the batch job rerun that idempotency keys do not cover.
3. Store the downstream identifier back on the source record. When the ERP returns its order number, write it onto the store order. Now every order carries proof it was synced, a human can answer "did this go through?" in one click, and your reconciliation has something to join on.
4. Add a natural key check as a safety net. Customer, plus purchase order number, plus order date. Flag matches for review rather than blocking them, because a buyer occasionally does order the same thing twice in a day and that order is real.
5. Run a duplicate report somebody actually reads. Weekly, one screen, grouped by natural key. Its job is not cleanup. Its job is to tell you that steps one to four have a hole in them.

Steps one and two are a build task. Steps three to five are operational. Do not let the operational ones substitute for the build ones, which is the trade most teams make by accident.

Pair this with detection. A duplicate and a missing order come from the same class of plumbing problem, and the daily count that catches one catches the other, as covered in what happens when your order flow stops at 3am.

What does a duplicate cost once it reaches the warehouse?

This is why prevention beats cleanup, and the gap is not close.

A duplicate caught in the order list is a few minutes of somebody's attention. The same duplicate caught after dispatch is a physical object in a van, and it comes back as a return, a credit note and a conversation with a buyer who now wonders what else you get wrong.

Illustrative cost of one duplicate that shipped as a pallet:

Line	Illustrative cost
Second pick and pack, 12 minutes at €35 per hour	€7
Outbound freight	€45
Return freight	€55
Receiving and put away, 20 minutes	€12
Credit note and bookkeeping, 15 minutes	€9
Customer service, 20 minutes	€12
Total	about €140

Illustrative figures. Use your own freight rates and hourly cost.

Now compare the same duplicate caught at different points:

Where the duplicate is caught	Illustrative cost
At the write, by an idempotency key or upsert	€0
In the order list, before picking	about €3
After picking, before dispatch	about €19
After it shipped	about €140

At four shipped duplicates a month, that is roughly €560 a month, or €6,720 a year, before you count the buyer's opinion of you. Those are illustrative numbers. The ratio is the point, not the euros: preventing it costs a build task once, and cleaning it up costs a bill every time.

What do you do about the duplicates already in your system?

Fix the cause first. Otherwise you will clean the same list every month and call it maintenance.

Then:

1. Find them with the natural key. Group orders by customer, purchase order number and date, and list groups with more than one record. Do this as a report, not as a delete script.
2. Sort by what happened physically. Not shipped, shipped, or shipped and returned. These are three different jobs, and only the first one is easy.
3. Cancel, do not delete. Cancel the unshipped copy so the audit trail, the stock movement and the accounting entry all survive. Deleted orders break reconciliation later and hide the evidence you need to prove the cause is fixed.
4. Reconcile the money separately. A shipped duplicate has already touched stock and possibly an invoice. Route those to whoever owns your ledger rather than fixing them in the order screen.
5. Count what you found. That number is your baseline, and you will want it in six weeks.

When is chasing duplicates not worth it?

Say no when:

• It has happened twice this year. Two duplicates a year on 3,000 orders does not justify rebuilding a write path. Add the natural key check and move on.
• The duplicate is real. Some buyers genuinely reorder the same items the same week. Flag those for review. Never auto-merge them.
• Orders still arrive by email and get typed in. A duplicate here is a people and process problem. Fixing the order intake is worth more than fixing the write, and that comes earlier in the B2B e-commerce automation sequence.
• The connector is not yours to change. If a packaged integration does a blind insert and offers no setting, your leverage is a check on the receiving side, not a rewrite.
• Nobody owns the write path. Without an owner, step one never gets built and you inherit a permanent duplicate report instead.

How do you measure whether it worked?

Write these down before you change anything.

• Duplicates per 1,000 orders. The headline number. Rates, not counts, so growth does not flatter you.
• Where each one was caught, using the four stages in the cost table above. Duplicates moving from "after it shipped" to "at the write" is the result, even before the rate falls.
• Credit notes attributed to duplicates, per month.
• Share of orders carrying a downstream identifier. If this is not close to 100%, your reconciliation cannot see what is missing.
• Retries per day on the order write. Falling retries mean you fixed the timeout, which is the root cause behind the most common duplicate of all.

Check again after a few normal weeks. If the rate is unchanged but every duplicate is now caught before picking, you still removed most of the cost.

Do this this week

1. Pull last quarter's orders and group them by customer, purchase order number and date. Count the duplicates. Most teams find more than they expected.
2. For each one, work out which of the six causes produced it. The timestamps tell you more than the payloads.
3. Check whether your order write sends a stable reference today. If it sends nothing, that is your first build task.
4. Turn the blind insert into an upsert on that reference.
5. Write the downstream order number back onto the source order, so tomorrow's question has a one-click answer.

That is the whole fix, and it is smaller than the cleanup process it replaces. It is how we approach software integrations generally: make the write safe once, rather than watching it forever.

Not sure where yours come from? Book a 30-minute call and we will trace one duplicate with you.

FAQ

What causes duplicate orders in an e-commerce integration?

Most duplicates come from retries, not from bad data. A write succeeds but the confirmation is lost to a timeout, so the sender tries again. The other common causes are at-least-once event delivery replaying the same message, a batch job rerun after a partial failure, two systems writing the same order, a double submit on a manual entry screen, and an order edit that creates a new downstream record.

What is an idempotency key and how does it stop duplicate orders?

It is a unique value the sender attaches to a write so the receiver recognizes a retry of the same request. Stripe's documentation describes the pattern well: the status code and body of the first request are saved against the key, and later requests with that key return the same saved result instead of doing the work again. Derive the key from the order itself, never from the moment of sending.

Do idempotency keys expire?

Yes, and this catches people out. Stripe documents that keys can be removed once they are at least 24 hours old, and that reusing a pruned key generates a new request. So an idempotency key protects you against retries within a short window, not against the same order being resent next week. For that you need a stable external reference stored on both sides and an upsert rather than an insert.

Why do webhooks deliver the same order event twice?

Because at-least-once delivery is the normal guarantee. Google Cloud's Pub/Sub documentation states that at-least-once delivery is the default on all subscription types and that consumers must be idempotent. Shopify's documentation says your app might receive the same webhook more than once after a network timeout or retry, and recommends deduplicating on the X-Shopify-Webhook-Id header or making your processing idempotent.

Should I delete duplicate orders from my system?

Cancel them rather than deleting them. Cancelling keeps the audit trail, the stock movement and the accounting entry intact, while deletion hides the evidence and breaks reconciliation later. Handle unshipped duplicates in the order screen, and route anything already shipped or invoiced to whoever owns your ledger, because stock and money have already moved.

How do I tell a duplicate from a genuine repeat order?

Compare the customer, the purchase order number and the order date, then look at the gap between the two records. Records created seconds or minutes apart with identical lines are almost always a retry. A genuine repeat usually carries a different purchase order number, a different delivery date or a changed quantity. Flag close matches for a person to review instead of merging them automatically.

Sources

• IETF: The Idempotency-Key HTTP Header Field, draft 07 supports the definition of an idempotency key and the timed-out POST problem that leaves a client unable to tell whether its write succeeded.
• Stripe: idempotent requests supports the Idempotency-Key header, the saved and replayed first result, the 255 character limit, the parameter comparison error, and the removal of keys after at least 24 hours.
• Shopify: ignore duplicate webhooks supports deduplication on X-Shopify-Webhook-Id, correlation via X-Shopify-Event-Id, and the recommendation to process webhooks idempotently.
• Shopify: troubleshoot webhooks supports the eight retry attempts over four hours and the five-second response deadline that turns a slow endpoint into a duplicate.
• Google Cloud Pub/Sub: subscription overview supports at-least-once delivery as the default on all subscription types and the requirement that subscribers be idempotent.
• Google Cloud Pub/Sub: exactly-once delivery supports the point that exactly-once delivery is limited to pull subscriptions, and is not available for push or export subscriptions.