Your webhook handler receives a Stripe invoice.paid event. It verifies the signature, looks up the customer in your database, provisions their subscription, sends a confirmation email, updates your analytics service, and returns 200.
It works in development. It works in staging. Then it hits production, the database is slow, the email API times out, and the handler takes 35 seconds to respond. Stripe sees the timeout, retries, and your handler processes the same invoice twice. The customer gets two confirmation emails. Your analytics count the payment double.
A developer on the Shopify forums reported a 77% webhook failure rate because their handler did all this work synchronously. Shopify gave them 5 seconds. They needed 12.
The fix is simple: your webhook endpoint should do three things and nothing else.
The verify-enqueue-ACK pattern
Verify the signature
Confirm the request came from the provider. Reject it immediately if the signature is invalid. This is the only validation that belongs in the handler.
Enqueue the event
Drop the raw event payload onto a queue — Redis, SQS, Bull, Inngest, a database table. Don't parse it. Don't act on it.
Return 200
Send back a 200 (or 202) immediately. The provider marks the delivery as successful. No retries.
All business logic — database writes, API calls, email sends — runs in a background worker that pulls from the queue.
// ❌ The slow handler
app.post("/webhooks/stripe", async (req, res) => {
const event = stripe.webhooks.constructEvent(
req.body, req.headers["stripe-signature"]!, WEBHOOK_SECRET
);
// All of this runs before the response
const customer = await db.customers.findByStripeId(event.data.object.customer);
await db.subscriptions.activate(customer.id, event.data.object);
await emailService.sendConfirmation(customer.email);
await analytics.track("payment_received", { amount: event.data.object.amount_paid });
res.sendStatus(200); // 8 seconds later
});// ✅ The fast handler
app.post("/webhooks/stripe", async (req, res) => {
try {
stripe.webhooks.constructEvent(
req.body, req.headers["stripe-signature"]!, WEBHOOK_SECRET
);
} catch {
return res.status(400).send("Invalid signature");
}
await queue.add("stripe-webhook", {
payload: req.body,
receivedAt: Date.now(),
});
res.sendStatus(200); // < 50ms
});The fast handler finishes in under 50 milliseconds. The slow one takes however long the slowest downstream service needs.
Why this matters: provider timeout rules
Every webhook provider enforces a response deadline. Miss it and they retry — which can mean duplicate processing if your handler is slow but eventually succeeds.
| Provider | Timeout | Retry behavior |
|---|---|---|
| Stripe | 20 seconds | Up to 16 retries over 3 days |
| Shopify | 5 seconds | 19 retries over 48 hours; removes subscription after consecutive failures |
| GitHub | 10 seconds | 3 retries with exponential backoff |
| Twilio | 15 seconds | Marks delivery failed, no auto-retry |
| Slack | 3 seconds | 3 retries at 1-minute intervals |
| Paddle | 20 seconds | Retries with exponential backoff up to 10 times |
Slack is the strictest — 3 seconds. If your handler queries a database and makes an API call, you're already over budget.
Shopify is the most punishing. After 8 consecutive failures, it removes your webhook subscription entirely. Your app stops receiving events with no notification.
The background worker
The worker pulls events from the queue and processes them without time pressure. It can retry failed jobs, implement idempotency checks, and take as long as it needs.
// worker.ts — processes events from the queue
worker.process("stripe-webhook", async (job) => {
const event = JSON.parse(job.data.payload);
// Idempotency check: skip if already processed
const existing = await db.processedEvents.findByEventId(event.id);
if (existing) return;
switch (event.type) {
case "invoice.paid":
await handleInvoicePaid(event.data.object);
break;
case "customer.subscription.deleted":
await handleSubscriptionCanceled(event.data.object);
break;
}
// Mark as processed
await db.processedEvents.insert({ eventId: event.id, processedAt: new Date() });
});This worker can take 30 seconds per event without affecting webhook delivery. If the database is slow, the job stays in the queue and retries later.
What to use as a queue
You don't need Kafka. Pick whatever your stack already runs:
| Stack | Queue option |
|---|---|
| Node.js + Redis | BullMQ |
| Serverless | Inngest, Trigger.dev |
| AWS | SQS + Lambda |
| Any SQL database | A webhook_events table with a processed_at column |
The database-as-queue approach is the simplest starting point. Insert the event, have a cron or worker poll for unprocessed rows, process them, and mark them done. It works until you hit hundreds of events per second — by then you'll know if you need Redis or SQS.
Test the pattern with webhooks.cc
The architecture is only as reliable as your tests. These four tests cover the critical failure modes: slow handlers, bad signatures, duplicate delivery, and provider-specific payloads.
Install the SDK:
npm install @webhooks-cc/sdkTest 1: Handler responds under the timeout
The most common failure. Your handler works, but it's too slow for the provider. captureDuring sends a real webhook through your handler and captures the response so you can assert on timing.
import { WebhooksCC } from "@webhooks-cc/sdk";
import { captureDuring } from "@webhooks-cc/sdk/testing";
import { describe, it, expect } from "vitest";
const client = new WebhooksCC({ apiKey: process.env.WHK_API_KEY! });
describe("webhook handler", () => {
it("responds within 3 seconds", async () => {
const start = Date.now();
const [request] = await captureDuring(
client,
async (endpoint) => {
// Send a signed Stripe payload to your handler
await client.sendTo(`http://localhost:3000/webhooks/stripe`, {
provider: "stripe",
template: "invoice.paid",
secret: process.env.STRIPE_WEBHOOK_SECRET!,
});
},
{ expiresIn: "5m", timeout: "10s" }
);
const elapsed = Date.now() - start;
expect(request).toBeDefined();
expect(elapsed).toBeLessThan(3000); // Slack's limit — the strictest
});
});If this test fails, your handler is doing too much work before responding.
Test 2: Invalid signatures get rejected
Your handler should return 400 for bad signatures — not enqueue garbage.
it("rejects requests with invalid signatures", async () => {
const endpoint = await client.endpoints.create({
name: "sig-test",
expiresIn: "5m",
});
// Send a webhook signed with the WRONG secret
await client.sendTo(`http://localhost:3000/webhooks/stripe`, {
provider: "stripe",
template: "invoice.paid",
secret: "whsec_wrong_secret",
});
// Give the handler a moment, then check the queue is empty
await new Promise((r) => setTimeout(r, 1000));
const queueSize = await queue.getWaitingCount();
expect(queueSize).toBe(0);
await client.endpoints.delete(endpoint.slug);
});Test 3: Duplicate delivery doesn't duplicate processing
Providers retry. Your worker must handle the same event arriving twice. Send the same payload twice and verify your idempotency logic catches the duplicate.
it("processes the same event only once", async () => {
const endpoint = await client.endpoints.create({
name: "idempotency-test",
expiresIn: "5m",
});
const webhookOptions = {
provider: "stripe" as const,
template: "invoice.paid" as const,
secret: process.env.STRIPE_WEBHOOK_SECRET!,
};
// Send the same event twice
await client.sendTo("http://localhost:3000/webhooks/stripe", webhookOptions);
await client.sendTo("http://localhost:3000/webhooks/stripe", webhookOptions);
// Wait for the worker to process both
await new Promise((r) => setTimeout(r, 3000));
// The subscription should only be activated once
const activations = await db.subscriptions.countActivations(TEST_CUSTOMER_ID);
expect(activations).toBe(1);
await client.endpoints.delete(endpoint.slug);
});Test 4: Full round-trip with the flow builder
The SDK's flow() method chains the entire test sequence into one call: create an endpoint, send a signed webhook, wait for capture, verify the signature, and clean up.
it("handles a complete Stripe webhook flow", async () => {
const result = await client
.flow()
.createEndpoint({ expiresIn: "5m" })
.sendTemplate({
provider: "stripe",
template: "checkout.session.completed",
secret: process.env.STRIPE_WEBHOOK_SECRET!,
})
.waitForCapture({ timeout: "10s" })
.verifySignature({
provider: "stripe",
secret: process.env.STRIPE_WEBHOOK_SECRET!,
})
.cleanup()
.run();
expect(result.verification?.valid).toBe(true);
expect(result.request).toBeDefined();
expect(result.cleanedUp).toBe(true);
});This test confirms the webhook was signed correctly, delivered, captured, and verifiable — in five lines.
Run these tests across providers
The SDK supports templates for Stripe, GitHub, Shopify, Twilio, Slack, Paddle, and Linear. Swap the provider and template fields to test your handler against each one:
const providers = [
{ provider: "stripe", template: "invoice.paid", secret: STRIPE_SECRET },
{ provider: "github", template: "push", secret: GITHUB_SECRET },
{ provider: "shopify", template: "orders/create", secret: SHOPIFY_SECRET },
] as const;
for (const { provider, template, secret } of providers) {
it(`handles ${provider} ${template}`, async () => {
const result = await client
.flow()
.createEndpoint({ expiresIn: "5m" })
.sendTemplate({ provider, template, secret })
.waitForCapture({ timeout: "10s" })
.verifySignature({ provider, secret })
.cleanup()
.run();
expect(result.verification?.valid).toBe(true);
});
}When you don't need a queue
Not every webhook handler needs this pattern. Skip it when:
- Your handler does one fast thing. A single database upsert that takes 20ms doesn't need a queue.
- The provider has generous timeouts and built-in retry. If you have 20 seconds and the provider retries reliably, a slightly slow handler is fine.
- You're using a serverless platform with built-in queuing. Inngest, Trigger.dev, and similar tools handle the enqueue step for you — your function is the worker.
Add the queue when your handler does multiple things, calls external APIs, or runs close to the provider's timeout window.
SDK Reference
Matchers, signature verification, and testing helpers for webhook integration tests.