My Go alerting service sat between Cloud Monitoring and on-call, and its whole job was to validate an alert, persist it, and reply 2xx before doing anything slow. The largest tenant on the Genesis platform I work on runs 613 resources from a 10,000 plus line generated config. A campaign reply that arrives twice and a follow-up draft that sends itself are the same two failures in different clothes: the record moves twice, or the machine acts before a human looks.
I have not wired a Resend campaign to a CRM, and I have not built their scheduling app. What I have shipped is the machinery underneath both: an alerting webhook in Go that had to acknowledge fast and apply once, and a double entry ledger core in Node and TypeScript where Postgres refuses the forbidden write. This post pulls those two together into the shape I would use for inbound replies, webhooks, and agent drafts: split receive from process and key on the sender id, then hold every generated action behind a human gate.
The sender retries, so your endpoint must not do the work
Cloud Monitoring posts an alert over HTTP and expects a prompt 2xx. If it does not get one, it treats the delivery as failed and retries. That is correct behavior for a notification channel, and it means a slow handler does not delay the alert. It duplicates it. A campaign reply path has the same property: the provider retries a webhook, a contact replies twice from two threads, your own queue redelivers while you were deploying. Each arrival looks like a fresh event until you give it an identity.
The endpoint therefore does three fast things and nothing else: validate the payload, persist the raw event keyed on the sender id, and reply success. Everything slow runs later in a worker: resolving the tenant, linking the reply to the right contact and thread, updating the record, proposing a follow-up. In the alerting service the sender id was the upstream incident id, stable across retries and across the lifetime of the condition. Duplicates became a counter increment instead of a second page, with no cache and no TTL to tune. Anything I minted on receipt would have named my transport, not their event.
In Go the receive half has this shape:
func handleAlert(w http.ResponseWriter, r *http.Request) {
var a Alert
if err := json.NewDecoder(r.Body).Decode(&a); err != nil {
http.Error(w, "bad payload", http.StatusBadRequest)
return
}
if a.IncidentID == "" {
http.Error(w, "missing incident id", http.StatusBadRequest)
return
}
inserted, err := db.UpsertIncident(r.Context(), a.IncidentID, a.Payload)
if err != nil {
http.Error(w, "store failed", http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusOK)
if inserted {
queue.Publish("alerts.process", map[string]string{"incident_id": a.IncidentID})
}
}Signature checks stay in the handler because unsigned traffic never reaches storage. Dedupe stays in storage because only storage sees every attempt. The worker then applies each incident once, and the full split is the one I wrote up here: https://tech.indra.codes/posts/the-webhook-must-reply-in-seconds-the-useful-work-takes-longer/
Your database holds a copy, so reconcile it against the source
Stripe holds the subscription. Your database holds a copy scoped by tenant, and every bug lives in the gap between them. A CRM wired to campaign email has the same shape even though I have not shipped that integration: the provider holds the thread, your tables hold the slice you render and act on, and the two drift the moment a retry, a manual edit, or a late webhook lands. When they disagree, the source wins, and the fix is a reconcile job, not a migration that edits history.
Every billing row carries its tenant because the query that forgets the tenant is the failure that dwarfs the rest. Keys lead with tenant id, reads filter on it, and the tenant resolves once at the edge and is stored at insert, never re-derived later. The ledger core I shipped in Node and TypeScript used the same discipline for money: integer minor units edge to edge with one explicit rounding step, and a deferred trigger that refuses commits whose legs do not sum to zero. Types helped readability, Postgres enforced, because every writer passes through it including the backfill someone runs by hand later. I wrote the ledger controls up here: https://tech.indra.codes/posts/the-app-balanced-the-ledger-the-database-refused-the-write/ and the subscription mirror here: https://tech.indra.codes/posts/stripe-holds-the-subscription-your-database-holds-a-copy/
The reconcile half has the same shape in any stack:
async function reconcileSubscription(tenantId: string, stripeSubId: string) {
const src = await stripe.subscriptions.retrieve(stripeSubId);
const copy = await db.findSubscription(tenantId, stripeSubId);
if (!copy || copy.status !== src.status || copy.plan !== src.items.data[0].price.id) {
await db.updateSubscription(tenantId, stripeSubId, {
status: src.status,
plan: src.items.data[0].price.id,
});
}
}Two deliveries can still describe one outcome, so the business key matters as much as the event id. The event id stops one delivery applying twice. A thread or invoice id stops two different events double counting one outcome. Each control needs a test that tries the forbidden write: insert one event twice and assert one applied row, apply two events for one thread and assert one update.
Generated text waits for a human the way generated code waits for a build
On the platform I work on, the application code for each service is generated, not typed. A model proposes Go files, a sandbox compiles and runs them with real credentials, and nothing ships until the build passes. Compile errors retry without a cap because a build error carries a file and a line. Run errors cap at five because past that the model chases its own tail. The loop itself caps at 25 for the model that will not stop calling tools, and a length finish reason is refused rather than returned as a silent truncation.
A follow-up agent needs the same gate with a human where the compiler was. The agent drafts, the queue holds, the human approves, and only the approval sends. The on-call triage service I built used the weaker version of this rule: the model wrote root cause analysis, the page went out first with what was already known, and the enrichment edited the message later. A model outage read as a missing paragraph, never as a missed page. For outbound email the rule is stricter because the blast radius is larger: no draft touches the provider until a person says so. I wrote the loop and its exit rules here: https://tech.indra.codes/posts/the-model-wrote-the-code-the-build-gate-was-the-product/
The hold has this shape in TypeScript:
async function proposeFollowUp(threadId: string, draftText: string) {
const draft = await db.insertDraft({
threadId,
body: draftText,
status: "pending_approval",
});
await queue.publish("drafts.review", { draftId: draft.id });
return draft;
}
async function approveDraft(draftId: string, approver: string) {
const draft = await db.getDraft(draftId);
if (draft.status !== "pending_approval") return draft;
return db.markApproved(draftId, approver);
}The draft row is append only until approval, and approval is an explicit transition with a name attached. Rejected drafts stay in the table as the audit of what the agent suggested and why a person said no. That table becomes the dataset that makes the next prompt better, grounded in rows rather than memory.
What I would tell someone building this
Acknowledge fast, apply once, key on their ids. Event id stops redelivery, thread id stops double counting, and anything you mint on receipt names your transport.
Rebuild your copy from the source or you own two truths. The wipe and rebuild path is the reconcile path, and plan changes fork anything less within a quarter.
Put the money rule in Postgres, not in review. A deferred balance check and a tenant key catch the branch no reviewer saw.
Let the agent draft, let the human send. Generation without a gate ships a first draft, and for outbound mail the gate is a person with a name on the approval.