Webhooks in VBWD, Part 2: Signed, Retried, and Self-Healing Delivery

Part 2 of a 3-part deep dive on the VBWD webhooks engine. Part 1 covered the event bus and the relay. This part covers the delivery half: how each webhook is signed, exactly what your endpoint receives, how you verify it, and what happens when delivery fails.

Emitting an event is the easy half. The hard, unglamorous half is delivering it to someone else's server reliably and securely — proving the payload really came from you, and behaving sensibly when the far end is slow, down, or lying. This is where a webhook system earns or loses trust, and where VBWD does its most careful work.

What your endpoint actually receives

Every delivery is a plain HTTP POST with a JSON body and four VBWD headers. The body is a stable envelope — the same shape for every event, with the event-specific data nested under data:

Example webhook request: a POST with X-VBWD-Signature (sha256=...), X-VBWD-Event, X-VBWD-Delivery-Id and X-VBWD-Timestamp headers, and a JSON envelope containing id, event_type, created_at and a nested data object — The envelope and headers your endpoint receives on every delivery.

The envelope is built once and serialised deterministically, because those exact bytes are what gets signed:

envelope = {
    "id": str(delivery.id),
    "event_type": delivery.event_type,
    "created_at": delivery.created_at.isoformat(),
    "data": delivery.event_payload or {},
}
body = json.dumps(envelope, sort_keys=True, default=str).encode("utf-8")

The four headers are X-VBWD-Signature, X-VBWD-Event, X-VBWD-Delivery-Id and X-VBWD-Timestamp. The event name and delivery id let you route and de-duplicate; the timestamp lets you reject stale replays; the signature is how you know it's genuinely from VBWD.

HMAC-SHA256 over the exact bytes

Each subscription has its own signing secret. VBWD computes an HMAC-SHA256 over the exact raw body bytes it sends, using that secret, and puts the result in the signature header prefixed with the algorithm name:

def compute_signature(secret: str, body: bytes) -> str:
    digest = hmac.new(secret.encode("utf-8"), body, hashlib.sha256).hexdigest()
    return f"sha256={digest}"

Your side verifies by recomputing the same HMAC over the received body and comparing in constant time — never with a plain ==, which can leak information through timing:

# On your server, verifying an incoming VBWD webhook
import hmac, hashlib

def verify(secret: str, raw_body: bytes, header_sig: str) -> bool:
    expected = "sha256=" + hmac.new(
        secret.encode(), raw_body, hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(expected, header_sig or "")

The one rule that matters: verify against the raw request body, before any framework re-serialises it. VBWD signs the precise bytes on the wire, and re-encoding JSON on your side (different key order, different whitespace) will produce a different signature and a failed check. Sign and verify the bytes, not the parsed object.

Every attempt is a record

A delivery isn't fire-and-forget — it's a tracked WebhookDelivery row that accumulates its own history. When the scheduler performs a POST, it records the outcome on the delivery: the HTTP status code, a (truncated) copy of the response body, the signature it sent, the attempt count, and — on failure — the error and when to try again.

class WebhookDelivery(BaseModel):
    __tablename__ = "vbwd_webhook_delivery"

    webhook_id    = db.Column(UUID, db.ForeignKey("vbwd_webhook.id"))
    event_type    = db.Column(db.String(255))
    event_payload = db.Column(JSONB)
    status        = db.Column(db.String(20), default="pending")  # pending|success|failed
    attempt_count = db.Column(db.Integer, default=0)
    max_attempts  = db.Column(db.Integer, default=5)
    next_attempt_at = db.Column(db.DateTime)
    response_code = db.Column(db.Integer)
    response_body = db.Column(db.Text)   # truncated
    error         = db.Column(db.Text)

A delivery counts as successful only on a 2xx response. Anything else — a 4xx, a 5xx, a timeout, a connection error — is a failure, recorded and scheduled for another try. Because every attempt is persisted, the admin can see exactly what happened to any event: what was sent, what came back, and how many times VBWD tried.

Retries with exponential backoff

When a delivery fails and hasn't exhausted its attempts, VBWD schedules the next try with exponential backoff, capped so it never waits absurdly long:

BACKOFF_BASE_SECONDS = 30
BACKOFF_CAP_SECONDS  = 6 * 60 * 60   # 6 hours

def backoff(attempt_count: int) -> timedelta:
    seconds = BACKOFF_BASE_SECONDS * (2 ** attempt_count)
    return timedelta(seconds=min(seconds, BACKOFF_CAP_SECONDS))

So a struggling endpoint is retried after roughly 30 seconds, then a minute, then two, four, and so on, up to a six-hour ceiling — giving a briefly-down server room to recover without hammering it, and without giving up on the first hiccup. After the configured maximum attempts (five by default), the delivery is marked failed and left alone.

Self-healing: subscriptions that disable themselves

Retries handle a bad delivery. A subscription that is persistently broken — a deleted endpoint, a rotated URL nobody updated — is a different problem, and VBWD handles it too. Each subscription tracks consecutive_failure_count; once it crosses the threshold (five consecutive failed deliveries), the subscription is automatically disabled and surfaced to the admin with a failed status:

@property
def status(self) -> str:
    if self.is_active:
        return "active"
    if (self.consecutive_failure_count or 0) >= AUTO_DISABLE_FAILURE_THRESHOLD:
        return "failed"
    return "inactive"

This keeps a dead endpoint from generating an infinite backlog of doomed deliveries, and it turns "your webhooks silently stopped working weeks ago" into a visible, actionable state in the admin. A single successful delivery resets the counter, so a transient outage doesn't permanently penalise an otherwise-healthy subscriber.

The whole safety story, in one line each

Authenticity — HMAC-SHA256 over the exact body, per-subscription secret, constant-time verification.
Integrity — deterministic envelope; you verify the bytes on the wire.
Reliability — every attempt persisted; retries with capped exponential backoff.
Resilience — a webhook failure never breaks event emission; bad subscriptions auto-disable.

In Part 3 we'll go from mechanism to practice: how VBWD's own plugins publish their events, how a plugin registers new event types without core knowing about them, and how you register and test a webhook through the admin API.

Want to see this delivering your real events into your real systems? Request an enterprise installation.

Webhooks in VBWD — the series:Part 1: The engine · Part 2: Safe delivery (this part) · Part 3: Plugins on the bus