Show the value before entering the number
Testing whether product discovery should come before recipient verification in international mobile top-ups.
Today, the phone number unlocks the available top-up catalogue. I designed a narrow, product-first MVP to test whether showing illustrative recipient benefits before asking for the number could increase completed purchases. The proposed A/B experiment would determine whether that gain is large enough—and the mismatch risk low enough—to justify a full integration.
Decision
Invest in a scalable catalogue or keep the number-first flow?
Primary metric
Completed purchases per eligible assigned visitor.
Main guardrail
Offer mismatch after number verification.
Core business question
Would showing the recipient's potential benefit before asking for their phone number generate enough additional completed purchases to justify building a scalable, accurate operator-and-product discovery experience?
The cost of a full solution comes from differences across destination countries, operators, product types, recipient amounts, bundle benefits, validity periods, pricing, exchange-related presentation and number-specific promotions. We want evidence of commercial value before funding that integration and operational complexity.
The problem
The current number-first flow protects accuracy: once we know the recipient's number, we can identify the operator and query eligible products. But it also asks for an action before a new visitor can see what they might buy, what they would pay, and what the recipient would get. The hypothesis is that this ordering suppresses purchase conversion—not that the phone field has already been proven to cause abandonment.
Where a visitor may drop
- 1Visitor arrives
- 2Wants to understand recipient benefit
- 3Encounters phone field
- 4Might leave before seeing the offer
The investment risk
Building a global pre-number catalogue upfront would require maintaining changing products, operator rules, pricing and promotions. The experiment tests whether that investment is justified before committing to it.
The two experiences
Both variants lead to the same underlying purchase. The only difference is the order in which value and verification appear.
- 1US sender selects Colombia or lands on the destination entry page.
- 2Page asks for the recipient phone number before showing offers.
- 3Number is verified; the operator is identified.
- 4Eligible products and confirmed prices are shown.
- 5Customer chooses a product and proceeds to checkout.
- 1Same eligible visitor arrives at the same entry point.
- 2Page displays curated, illustrative products for Colombia grouped by fictional operator.
- 3Each card distinguishes price, benefit type, validity, operator and 'example only' status.
- 4Visitor expresses interest in one offer, then the page asks for the recipient number.
- 5System checks operator and real eligibility through the existing number-based lookup.
- 6Before checkout, the verified final price, product, benefit and validity are shown.
- 7If the offer differs or is unavailable, the change is explained — never a silent substitution.
Customer-facing microcopy for B
“Explore example top-ups by operator. Enter the recipient's number to confirm the available products, exact benefits and final price before payment.”
Try it — A / B interactive demo
Toggle between the two experiences, filter by operator, and pick a demo recipient token to drive the verification state. No real lookup, payment or personal data is involved.
Explore example top-ups by operator
Enter the recipient's number to confirm the available products, exact benefits and final price before payment.
You pay
USD 5
illustrative until verified
Recipient gets
COP 19,000 airtime
You pay
USD 8
illustrative until verified
Recipient gets
3 GB · Valid for 15 days
You pay
USD 12
illustrative until verified
Recipient gets
5 GB + 200 SMS · Valid for 30 days
You pay
USD 10
illustrative until verified
Recipient gets
COP 39,000 airtime
You pay
USD 7
illustrative until verified
Recipient gets
2 GB · Valid for 10 days
You pay
USD 15
illustrative until verified
Recipient gets
8 GB + 300 SMS · Valid for 30 days
Experiment design
Hypothesis
For eligible US-to-Colombia top-up visitors, a curated product-first flow will increase completed purchases per assigned eligible visitor versus the current number-first flow, without causing an unacceptable rate of product, benefit or price discrepancies after recipient verification.
Setup
- Unit of assignment: eligible visitor or stable user ID, with persistent assignment.
- Allocation: 50/50 A and B. Returning users keep their assignment.
- Entry population: visitors entering the same US-to-Colombia journey, eligible to complete the same final purchase in both variants.
- Analyze by assigned group (intention-to-treat); do not restrict to those who click an offer or enter a number.
- Both variants measure the same completed-purchase event, with the same attribution window.
- Comparing A and B measures the whole product-first experience, not the isolated effect of the phone field.
- Before interpreting: check event completeness, assignment persistence and sample-ratio mismatch.
- Pre-register metrics, decision criteria and maximum duration before launch.
Metric tree
Primary business outcome
Purchase conversion rate = unique eligible assigned visitors completing a confirmed top-up purchase within the attribution window / all unique eligible assigned visitors.
Diagnostic funnel
- Entry → product seen
- Entry → offer selected
- Entry → recipient verification attempted
- Verification attempted → verified offer found
- Verified offer shown → checkout started
- Checkout started → purchase completed
Guardrails
- Offer mismatch rate
- Post-verification abandonment
- Checkout errors & failed purchase rate
- Support contacts / refunds
Sample-size planning examplePlanning example — not a calculation from site traffic.
| Baseline conversion | 5.0% |
| Minimum effect worth detecting | 6.0% (B) |
| Absolute / relative change | +1 pp / +20% |
| Two-sided alpha | 0.05 |
| Power | 80% |
| Allocation | 1:1 |
| Required sample | 8,143 per variant · 16,285 total |
| Method | Two-proportions power (Cohen's h) |
Sensitivity: with the same assumptions but a target of 5.0% → 5.5%, approximately 31,217 visitors per variant are needed.
Duration = planned total sample / eligible assigned visitors per day, plus the attribution window. If daily traffic is unknown, duration cannot be estimated without eligible-traffic data.
Commercial success and the investment decision
An increase in clicks is not commercial success. Neither is a purchase lift on its own. The question is whether the incremental contribution margin covers build and operating costs within an agreed payback period. Adjust the assumptions below to see how the decision changes.
Margin is net revenue attributable to top-up purchases minus the directly attributable variable costs.
Projected outcome
- Additional purchases / year
- 1,000
- Incremental contribution margin / year
- USD 3,000
- Net of annual operating cost
- USD -2,000
No operating break-even at current assumptions
Projected annual contribution does not exceed annual operating cost, so the build never pays back under these inputs.
Illustrative break-even example
Planning example — not a calculation from site traffic.B improves conversion in this hypothetical example, but the modeled USD 3,000 annual incremental contribution does not even cover the USD 5,000 annual operating cost. Therefore, this result would NOT justify funding the full build under these assumptions.
A +1 percentage-point conversion improvement is the illustrative minimum effect used for sample-size planning. It is NOT automatically the minimum commercially viable effect.
What scaling would cost
MVP test costs are not scaled-product costs. Manual updates and a narrow approved snapshot are test-only shortcuts, not evidence that a production catalogue would be inexpensive to operate.
One-off build
- Product discovery and design.
- Engineering integration with supplier product feeds.
- Operator and product mapping.
- Number-based eligibility verification.
- Price and benefit confirmation before payment.
- Experimentation and analytics instrumentation.
- QA across destinations, operators, product types, currencies and failure states.
Ongoing operation
- Catalogue updates and offer expiry handling.
- Price, benefit and validity monitoring.
- Promotion eligibility and localization.
- Supplier changes and incidents.
- Customer support caused by offer mismatch.
- Maintenance, monitoring and compliance review.
Commercial success rule
“We would consider funding a phased build only if the A/B test shows a credible increase in completed purchases, the offer-mismatch guardrail stays within its pre-agreed limit, AND a conservative contribution-margin forecast covers build and ongoing operating costs within the company's agreed payback period.”
- If B has more browsing but no purchase lift: do not scale; diagnose where the funnel loses users.
- If B lifts purchases but exceeds the mismatch threshold: stop or narrow B; fix offer accuracy first.
- If purchase results are inconclusive: do not call it 'no effect'; check power, measurement and scope.
- If a critical price/benefit misrepresentation occurs: pause B for investigation rather than waiting for significance.
- Example proposed thresholds (calibrate with real data): at least +1 pp absolute lift; critical mismatch limit 3% of attempted verifications.
Decision trail & outcome scenarios
Why these choices
1Why test before full build?
Full multi-country / multi-operator integration versus a scoped MVP: choose the scoped MVP to test commercial value first.
2Why one corridor?
Broad coverage versus US payer / Colombia recipient: choose one corridor with stable, maintainable offers and enough eligible traffic. The choice is hypothetical and must be checked against real data.
3Why curated illustrative offers?
A dynamic global catalogue versus a limited approved snapshot: choose the limited catalogue with operational ownership and strict confirmation before checkout.
4Why purchases instead of clicks?
Clicks show interest; completed purchases are the business outcome.
5Why a trust guardrail?
Pre-number browsing could create false expectations that only surface after verification.
6Reversal / failure scenario
Illustrative learning scenario, not an observed failure: higher offer clicks but no purchase lift would reverse the urge to scale, and point investigation at the verification handoff instead.
What would you decide?
Three illustrative branches, each labeled simulated.
What would you decide?
If the experiment shows…
Purchases improve enough, mismatch stays below the agreed threshold, and a conservative contribution-margin forecast covers build and operating costs within the agreed payback period.
Then the decision is…
Commission a phased technical discovery, supplier data audit and cost estimate for the tested corridor. Verify cost assumptions before broader rollout — do not claim production rollout is already warranted.
Outcome scenarios
A — Purchases up, guardrails held, economics support payback
Fund a phased technical discovery and staged build for the tested corridor; verify cost assumptions before broader rollout.
B — Purchases up, guardrails held, but economics do not cover cost
Do not fund the full catalogue. Test a cheaper way to communicate recipient value, or find a corridor with stronger economics.
C — Offer clicks up, completed purchases flat
Not commercial success. Diagnose whether verification or offer mismatch moved abandonment later in the funnel.
D — Purchases up but the offer-mismatch guardrail fails
Stop or narrow the treatment. Trust is not something the revenue lift can buy.
E — Negative or inconclusive
Document whether the test was adequately powered, correctly assigned and instrumented. State which investment was avoided or deferred without inventing a monetary saving.
Negative-result decision memo (example)Planning example — not a calculation from site traffic.
- What we expected
- A product-first flow would raise completed purchases without breaching the mismatch guardrail.
- What the simulated analysis suggests
- Interest rises but completed purchases do not clearly improve — presented as a scenario, not a real result.
- Was the test sound?
- Confirm adequate power, clean persistent assignment and complete tracking before drawing conclusions.
- New friction?
- Check whether real offer mismatch created friction after number entry.
- Investment avoided or postponed
- A full catalogue build is deferred — no quantified saving is claimed without a real cost estimate.
- Hypothesis that survived
- People may want visibility before sharing a number, but this particular product-first implementation did not increase completed purchases.
- Next cheapest test
- Show concise recipient-value examples beside the phone input, without implying guaranteed availability.
- Evidence distinction
- 'No evidence of improvement' is not the same as 'evidence of no meaningful improvement.'
Final decision record
- Experimental outcome
- Simulated / not observed
- Purchase-conversion effect
- Illustrative only
- Trust guardrail
- Proposed threshold and illustrative outcome
- Incremental contribution margin
- Assumptions and range
- One-off build cost
- Illustrative estimate
- Annual operating cost
- Illustrative estimate
- Payback horizon
- Business-defined assumption
- Recommendation
- Build in phases / run a cheaper follow-up test / do not invest
- What would reverse this
- A change in traffic, margin, conversion lift, offer accuracy or build cost
Limits & learning
Technical boundaries of the test MVP
01
Portfolio demo
Local deterministic data → illustrative operator/product cards → simulated verification states → mock checkout completion. No supplier connection, real payments, phone processing or actual A/B randomization.
02
Proposed production experiment
Small approved snapshot for one corridor and a few operator/product IDs → scheduled manual freshness checks → hide stale/unknown offers → existing number-based eligibility and final-price lookup → existing checkout → event logging and kill switch.
03
Full product, only if justified
Reliable supplier integrations, expiry/invalidations, operator detection, number portability, promotion eligibility, pricing/FX display, monitoring, operational ownership, compliance review and scaled coverage.
Scope labels used throughout
Temporary choices vs. scaled capabilities
| Temporary choices made to learn | Capabilities required for a scaled product |
|---|---|
| Hardcoded, curated offer snapshot | Reliable supplier product-feed integration |
| Manual freshness checks by an owner | Automated expiry, invalidation and monitoring |
| Single corridor (US → Colombia) | Multi-country, multi-operator coverage |
| Simulated verification states | Existing number-based eligibility and final-price lookup |
| Mock checkout completion | Live checkout, payments and reconciliation |
Risks & rules
- Supplier test numbers may not represent all real numbers; never infer universal eligibility from them.
- Operator ownership and product availability may vary by number.
- Prices, payout amounts, validity and bundle entitlements can change.
- Promotions may be number-specific or time-limited: do not display them before eligibility verification unless qualified.
- Every cache entry needs source, last_checked_at, valid_until, currency, operator, product ID, product type and verification status.
- Never use AI to invent pricing, benefits, availability, validity or FX rates. AI can help with synthetic copy, test fixtures or code; supplier-approved data and the final lookup determine what is sold.
- Hide or withdraw illustrative cards if freshness checks fail. No collection of real numbers or personal data in this demo.
Results / impact
If a real A/B test met the pre-agreed purchase-lift threshold while protecting offer accuracy, the next decision would be to estimate full-build cost, ongoing catalogue-maintenance cost and contribution-margin impact.
Confidence in actual business impact: not established — no live test has been conducted.
“The MVP was not designed to solve global catalogue complexity. It was designed to find out whether solving it would create enough customer and commercial value to warrant the investment.”
“An experiment can be successful even when the treatment loses: avoiding an unjustified integration is a valuable product decision. This portfolio case does not claim that such savings were realized.”
What I would do differently
If a curated catalogue produces many mismatches, reduce the scope further and validate simpler ways of communicating value before revisiting a catalogue.
Hypothetical portfolio exercise; all offers, data and experiment outcomes in this demo are simulated. No confidential employer or supplier information is used.