“Where is my money?”: Support Agent with Asymmetric Release Gates
Design and implementation of an in-transit payments support agent governed by a formal behavior specification without out-of-band prompts, validated across a 120-ticket test harness with zero tolerance for false promises.
MODELED DEFLECTION
18.8%
Realism discount
RELEASE GATE
0 Violations
On C2.1–C2.6 & C2.8
COST / 1K TICKETS
$1.80
Economy-C Model
P50 LATENCY
480 ms
Optimized inference
01. Problem Statement & CFPB Regulation
In international remittances, peak friction happens when a transfer is flagged 'in transit'. Standard chatbots placate users with empty empathy or invent delivery timestamps, triggering regulatory complaints and chargeback disputes.
Injection of Statutory Facts (CFPB Remittance Transfer Rule):
- Mandatory cancellation window: 30 minutes (provided funds have not been picked up).
- Statutory refund timeline: maximum 3 business days.
- Dispute windows: 180 days to report an error and 90 days for formal resolution.
Regulatory facts live as structured data injected into the spec, never in stochastic LLM memory.
02. Operating Prototype (Live Simulator)
behavior_spec.v1Select a test scenario to inspect conversational behavior in real time alongside JSON contract extraction and guardrail enforcement.
{
"intent": "STATUS_INQUIRY",
"confidence": 0.98,
"hard_constraints_checked": [
"C2.1_PASS",
"C2.2_PASS",
"C2.8_PASS"
],
"handoff_triggered": false
}03. Evaluation Test Harness & Release Gate
The release gate is asymmetric: zero violations across critical constraints (C2.1–C2.6 and C2.8) across the 120-ticket golden test set. In cross-border remittances, a single hallucinated arrival date or privacy breach costs far more than ten uncontained tickets.
| Model | Cost / 1k | p50 Latency | Pass Rate | Gate Violations | Release Decision |
|---|---|---|---|---|---|
| Frontier-A | 14,20 USD | 1.850 ms | 94,1% | 1 violation (C2.1) | BLOCKED |
| Frontier-B | 11,50 USD | 1.200 ms | 91,6% | 0 violations | DISCARDED (COST) |
| Economy-C (Guarded) | 1,80 USD | 480 ms | 89,2% | 0 violations | RELEASE APPROVED |
04. Post-Mortem: Failure Gallery & Gate Hardening
Failure T014: Hallucinated arrival timestamp (C2.1)
“Don't worry, it will arrive Monday by 6:00 PM at the latest” The naive model responded with this timestamp when the core banking engine had no confirmation.
Solution: Strict prohibition against outputting time units whenever system_eta == null.
Post-Run Addition: Third-Party Privacy (C2.8)
In the initial run, a recipient with a valid transaction ID prompted the model to disclose the sender's source amount in 3 edge cases.
Solution: The release gate was hardened post-run by adding C2.8 to mandate sender session validation.
05. Economic Impact: Discounted Deflection Modeling
In contrast to the 30–60% deflection routinely marketed by AI vendors, this model enforces an explicit realism discount for multi-turn support noise and reweights against Case 1 corpus mix, projecting ~115,000 USD in sustainable annual direct support savings.
Next Step • First-Party Data Decision
From technical agent specifications and evaluations to digital product roadmap prioritization backed by authentic feedback and RICE.