Failure behavior

Delivery is at least once. Every failure below ends in one of two places: the delivery is retried with the same event ID, or the process exits so a supervisor restarts it. Nothing is lost silently, and nothing is sent without an attempt record.

Leases and crashesCopy Anchor Link

A worker claims a delivery by writing an attempt record with a lease token before it sends anything. The signed body is exactly the persisted body. If the worker dies, the lease expires after leaseMs (60 seconds by default, at least twice timeoutMs). The next worker to start a batch recovers expired leases: the attempt becomes abandoned, and the delivery returns to pending if retry budget remains, or becomes failed if it does not.

Lease tokens are fenced. A worker that wakes up late and tries to complete an attempt whose lease already expired gets stale; the result is discarded even if a replacement worker has since claimed and finished the same delivery. Two workers can never both record success for one attempt.

The recovered delivery is sent again with the same event.id and a fresh timestamp and signature. If the receiver accepted the first request and the worker died before recording it, the receiver sees a duplicate. This is the normal case the receiver deduplication exists for.

Database failuresCopy Anchor Link

The library opens no connections of its own. It uses the pool you pass in, so pool-level failures surface as rejections from run(), runOnce(), publish(), or check().

• Mid-batch disconnect. If PostgreSQL terminates the connection while a transaction owns the client, run() rejects with the driver error. Nothing is retried inside the library; the supervisor restarts the process and lease recovery handles the half-finished attempt. The example entrypoint logs webhooks.worker.failed and exits with code 1 without an unhandled error event.
• Idle disconnect. A connection terminated while the worker waits between polls produces the same exit. The pool's error is caught and reported, not thrown asynchronously.
• Blocked startup. A startup query waiting on a lock is cancelled by the statement timeout you configure. The process exits within that bound and closes its connection.
• Accepted TCP, no handshake. The connection timeout bounds a database that accepts the socket but never completes startup.

Set connectionTimeoutMillis, statement_timeout, and idle_in_transaction_session_timeout on the pool. The worker guide shows the example configuration.

Restarts and deploysCopy Anchor Link

• Graceful stop. Abort the worker's signal on SIGTERM. Claims stop immediately, active requests get the grace period, outcomes are written, then run() resolves. The operations rehearsal confirms that a delivery accepted by the receiver during shutdown is recorded succeeded with one attempt, not retried.
• Hard kill. SIGKILL leaves leases behind. They are recovered by the next worker after leaseMs. The rehearsal kills a worker during a backlog and checks that a replacement completes every delivery with no gaps and no duplicate receipts beyond what deduplication absorbs.
• Rolling deploy with a schema upgrade. Apply the upgrade SQL first. Processes on the previous package keep running when the release notes say so, and a restarted old process fails at check(). See the rollout order.
• Multiple workers. Workers coordinate through row locks and per-endpoint in-flight caps. A locked delivery is skipped, not waited for, so one slow endpoint does not stall a batch for other endpoints.

What is not coveredCopy Anchor Link

These behaviors are verified on a single PostgreSQL instance with local receivers. They do not establish throughput limits, behavior under database failover, or internet-path delivery characteristics. Treat the production checklist as the list of decisions that remain yours.

Where each claim is checkedCopy Anchor Link