Delivery and retries

What counts as success

Any 2xx. Custral reads the status code and nothing else: the response body is ignored, and a redirect is not followed. Anything else, including a redirect, a timeout or a connection failure, is a failed attempt and is retried.

Your response	Outcome
`200`, `201`, `202`, `204`	Success. Delivery closed.
`3xx`	Failed as `redirect_not_followed`. Retried against the URL you registered.
`4xx`	Failed. Retried.
`5xx`	Failed. Retried.
No response within 10 seconds	Failed. Retried.

Acknowledge first, work second. The 10 second timeout covers your whole response, so push slow work onto your own queue and return 200 immediately. A handler that calls three other APIs before responding will eventually cross the line, and Custral will retry an event you already processed.

Redirects are not followed

A delivery is a signed POST. Following a redirect would send that signed body to whatever address the Location header names, so Custral stops at the 3xx, records it, and retries the URL you registered. It never requests the Location. The common causes are an http:// URL on a server that forces https://, a trailing slash your framework adds or strips, and an apex domain that redirects to www. Register the URL your endpoint redirects to.

The retry schedule

Custral makes up to 8 attempts over about 24 hours: the first as soon as the event fires, then 7 retries with a longer wait before each one. The short waits cover a dropped connection or a process restart. The long ones cover a deploy or an outage that runs for hours.

Attempt	Wait after the previous failure	Roughly this long after the first attempt
1	None	Immediately
2	5 seconds	5 seconds
3	30 seconds	35 seconds
4	2 minutes	2 minutes 35 seconds
5	10 minutes	12 minutes 35 seconds
6	1 hour	1 hour 13 minutes
7	5 hours	6 hours 13 minutes
8	18 hours	24 hours 13 minutes

Each wait starts when the previous attempt fails, so an attempt that hits the 10 second timeout adds those 10 seconds to the times above. After an outage, a delivery can arrive hours after your endpoint is back online.

If all 8 attempts fail, the delivery is not tried again and there is no way to replay it by hand. That takes an outage, or a handler that keeps returning errors, lasting longer than about a day. If you cannot afford to miss an event, reconcile from the API after a long outage.

Two failures skip the retries entirely, because retrying cannot help:

The endpoint’s URL, or an address its hostname resolves to, does not pass the safety check.
The subscription was deleted or set inactive after the event was queued.

Your handler must be idempotent

Assume every event can arrive more than once. A retry after a timeout is the common case: your server did the work, the response was slow, Custral never saw it, and the same delivery lands again. Deduplicate on the envelope’s id, which is stable across every attempt of one delivery:

custral.on("record.created", async (event) => {
  // `event.id` is the whd_… delivery id, the same on every retry.
  const fresh = await claim(event.id);
  if (!fresh) return; // Already handled.

  await doTheWork(event.data);
});

createdAt is not stable across retries. It is stamped per attempt, so it is useful for measuring delivery latency and useless as a deduplication key.

Order is not guaranteed

Deliveries are processed concurrently, so two events about the same record can arrive out of order. Do not infer sequence from arrival. If order matters, treat the payload as a notification that something changed and read the current state back:

custral.on("record.updated", async (event) => {
  const record = await custral.records.retrieve({
    object: "deals",
    id: event.data.recordId as string,
  });
  // `record` is authoritative. The payload told you to look.
});

Fan-out

One event delivers once per matching subscription. Three endpoints subscribed to record.created produce three deliveries with three separate ids, each retried independently. An endpoint that is down does not hold up the others.

The delivery log

Every attempt is recorded against the subscription before anything is sent, so a delivery survives a worker restart. Open an endpoint under Settings → Developers → Webhooks to read the last 100, newest first.

Field	What it tells you
`status`	`pending`, `success`, or `failed`
`attempts`	How many times it has been tried
`responseStatus`	The HTTP status your endpoint returned on the last attempt, or empty when it did not answer
`error`	A short code for what went wrong, such as `http_500` or `redirect_not_followed`. See Error codes.
`deliveredAt`	When it finally succeeded, if it did
`payload`	The `data` that was sent

A failed row is not final while attempts is below 8. Another attempt is still coming, unless error is webhook_inactive_or_missing or starts with unsafe_url:, which are never retried.

The log holds one row per delivery, not per attempt. responseStatus and error describe the most recent try, so you can see that a delivery failed three times and what the last failure was, but not the individual history of each attempt.

Error codes

error holds a code, never an error message. A failed connection’s message can include a network address, and everyone in the workspace can read the delivery log. Transport failures carry the underlying system code after a colon when there is one.

Code	What happened	Retried
`http_<status>`	Your endpoint answered with that status.	Yes
`redirect_not_followed`	Your endpoint answered with a `3xx`. See Redirects are not followed.	Yes
`timeout`	No response within 10 seconds.	Yes
`connection_failed`	The connection was refused, reset, or could not be made, such as `connection_failed:ECONNREFUSED`.	Yes
`tls_error`	The TLS handshake failed, such as `tls_error:CERT_HAS_EXPIRED`.	Yes
`dns_lookup_failed`	The hostname did not resolve, such as `dns_lookup_failed:ENOTFOUND`.	Yes
`delivery_failed`	Any other failure to deliver.	Yes
`unsafe_url:<reason>`	Custral refused the URL, or an address its hostname resolved to. See What Custral will and will not call.	No
`webhook_inactive_or_missing`	The endpoint was deleted or set inactive.	No

Limits worth knowing


Request timeout	10 seconds
Redirects	Not followed. Recorded as `redirect_not_followed` and retried.
Attempts	8, over about 24 hours
Delivery log retained	Last 100 per endpoint, newest first
Replay a delivery by hand	Not available yet
Auto-disable after repeated failure	Not implemented. A dead endpoint keeps being tried.

Getting started

Building a receiver

Event reference

Operations

What counts as success

Redirects are not followed

The retry schedule

Your handler must be idempotent

Order is not guaranteed

Fan-out

The delivery log

Error codes

Limits worth knowing

What’s next

Troubleshooting

Managing subscriptions

​What counts as success

​Redirects are not followed

​The retry schedule

​Your handler must be idempotent

​Order is not guaranteed

​Fan-out

​The delivery log

​Error codes

​Limits worth knowing

​What’s next

Troubleshooting

Managing subscriptions

What counts as success

Redirects are not followed

The retry schedule

Your handler must be idempotent

Order is not guaranteed

Fan-out

The delivery log

Error codes

Limits worth knowing

What’s next