Retention Signals
An explainable dashboard that turns emerging customer feedback into trust-risk alerts and product decisions.
A hypothetical international calling, subscriptions, top-up, and remittances service receives feedback across app stores and support channels. Teams can see complaints, but struggle to distinguish a gradual retention issue from a critical trust-risk event that needs immediate intervention.
Primary decision
Investigate, contain, or prioritize
Evidence sources
Public feedback patterns + simulated product telemetry
Focus scenario
Unexpected auto-top-up charges after a subscription-flow change
The product problem
Teams often react to the loudest recent complaint. That approach misses two things: whether a topic is genuinely worsening over time, and whether it signals a high-severity loss of financial trust.
- 1Customer sees or experiences an unexpected top-up charge.
- 2They contact support, leave negative feedback, request a refund, or disable auto-top-up.
- 3Product teams receive fragmented signals across channels.
- 4The dashboard detects the pattern, surfaces evidence, and recommends the appropriate level of response.
Design principles
01
Trust before volume
Financial-control signals can outrank larger but lower-severity issues.
02
Trends, not snapshots
Compare current rates against historical baselines, not the loudest recent complaint.
03
Evidence before causality
Alerts trigger investigation; they do not prove root cause.
04
Different problems, different actions
Contain an incident, investigate a hypothesis, or prioritize a roadmap initiative.
Interactive prototype
The dashboard
Filter the view, switch topics, and click a point on the chart to inspect a single week. The unexpected-charge scenario is the most complete flow; other topics use simplified but coherent simulated data.
Churn & Trust Signal Monitor
Simulated monitoring surface. Filters update every panel below.
Showing the default trust-risk scenario.
Trust-risk alerts
1 active
Escalation recommended
Feedback mentioning unexpected charges
6.7%
+140% vs. 8-week baseline
Auto-top-up disable rate
8.1%
+3.5pp vs. control
Refund-request rate
3.4%
+1.8pp vs. control
Classification confidence
89%
reviewed agreement
Negative-feedback distribution over time
12 weekly observations · release line shown
Click a topic to select it; double-click to add or remove it from the chart.
Detail · W12
Unexpected auto-top-up charges — 31 signals · 6.7% of negative feedback
Trust risk corroborated: unexpected auto-top-up charges
Feedback about unexpected charges increased from 2.8% to 6.7% of negative feedback after the subscription-flow update. Simulated telemetry shows a higher seven-day auto-top-up disable rate for users exposed to the new flow.
Where is the signal concentrated?
Share of affected feedback for the unexpected-charge topic, by segment.
Likely linked to updated subscription flow
Validate localized consent copy
No equivalent increase
Stable comparison group
Geography, origin/destination corridor, exact billing logs, and customer-level behavior would require internal production data. They are represented only as simulated fields in this portfolio prototype.
How the team would validate the alert
A trust-risk alert is a hypothesis. In production, the team would validate it with a controlled comparison rather than by comparing app versions after the fact.
Previous subscription flow.
Updated subscription flow with auto-top-up setup.
Eligible users: Android users with auto-top-up already enabled. Primary guardrail metric: seven-day auto-top-up disable rate. Secondary signals: refund requests, support contacts about unexpected charges, and qualitative feedback. This is a simulated production-validation plan, not a real experiment result.
| Metric | Variant A | Variant B | Difference | Interpretation |
|---|---|---|---|---|
| Auto-top-up disable rate (7-day) | 4.6% | 8.1% | +3.5pp | Potential loss of trust; investigate and consider stopping rollout |
| Refund-request rate | 1.6% | 3.4% | +1.8pp | Secondary corroborating signal |
| Unexpected-charge support contacts | 2.1% | 5.3% | +3.2pp | Consistent with the feedback trend |
Methodology: An A/B test within the same app version is more reliable than comparing app versions alone, because both groups experience the same date, platform, campaigns, and external conditions. Results still require checks for instrumentation, exposure, and segment imbalance.
Decision record
Pause further rollout of the updated subscription flow for affected Android segments.
- Trigger
- Unexpected-charge feedback rose 2.4× above its historical baseline after the release.
- Corroboration
- Simulated telemetry shows increased auto-top-up disabling and refund requests among exposed users.
- Risk
- High financial-trust risk, even if the absolute number of affected users is limited.
- Immediate action
- Halt rollout; audit consent events, default states, billing execution, confirmation copy, and localization.
- Customer action
- Identify potentially affected users and route remediation through supervised support and billing processes.
- Product follow-up
- Redesign the auto-top-up set-up and management experience around explicit control and clear pre-charge information.
- What we are not concluding
- Feedback and experiment signals do not alone prove that charges were invalid or that the flow caused every cancellation.
- Owner
- Product + Billing + Support + Engineering
- Review date
- One week after containment
Prioritization logic
Retention Signals does not rank every issue with one universal number. It first classifies whether a pattern is an incident, a trust-risk alert, a discovery opportunity, or a monitor-only trend.
Core-service reliability spike
IncidentEngineering investigation now
Money or consent signal + corroboration
Trust-risk alertContain and investigate
Persistent high-volume pain without a critical spike
Discovery opportunityResearch and roadmap evaluation
Low-volume or uncertain emerging pattern
MonitorCollect more evidence
Opportunity score: For non-incident opportunities, teams consider trust impact first, then reach, trend, and evidence confidence. Scores support discussion; they do not replace judgment.
Data & methodology
Simulated- Public review patterns can be used to identify and classify recurring customer problems across languages.
- The dashboard's event-level telemetry, release metadata, experiment results, and user analytics are simulated to demonstrate the decision workflow.
- Topic classification uses a pre-defined taxonomy with a manual-review sample.
- Do not treat a review, a topic label, or an alert as proof of causality.
- A topic needs at least 3 independent qualitative signals to enter the report; a high-severity alert requires a historical increase and validation through internal telemetry in a real environment.
- This case does not use confidential data from any real employer.
Outcome
The prototype demonstrates a PM operating model rather than a claimed business result: detect a signal, inspect its distribution and evidence, validate it with appropriate internal data, contain potential harm, and convert verified learning into a product decision.