PRODUCT PORTFOLIO: 4. OPTIMIZE
I love tinkering, in pursuit of perfection.
Perhaps my favorite part of product management is refining the product: discovering common customer obstacles and unblocking them, or finding ways to optimize around the edges. I am endlessly curious, and I am always convinced there’s a better way.
My favorite tools of the trade: data tracking and A/B testing.
EXAMPLES OF DATA ANALYSIS & A/B TESTING
Wallet stickiness analysis
I analyzed a big data set of Zoona’s wallet (current/checking account) customers, to try to determine whether there were certain actions that correlated with a new customer remaining active in the future-- in other words, whether there was a “sticky” point of activity..
It turned out that customers who completed 3+ transactions in their first 30 days were way more likely to remain active later. Knowing that allowed the Growth, Product, and Marketing teams to all work to drive that sort of behavior. It also allowed the teams to de-prioritize efforts on customers past the 30 day mark.
Wallet app registrations
THE PROBLEM: Soon after launching Zoona Wallet (a cheque/current account), we saw that many people who downloaded the Wallet app did not register. This was curious because there was literally nothing else to do when you first open the app.
THE INSIGHT: From user testing, we learned that, when customers saw the button “Create a wallet,” they assumed they had to first register for an account at a Zoona branch, and then afterwards they could create a mobile banking wallet to link to the account. This was how all their other Zambian banking apps worked.
For Zoona Wallet, however, we wanted customers would register themselves and then top up the account at any branch or Zoona affiliate.
THE EXPERIMENT: Using Apptimize, we tested two different button text options for two weeks, with the goal of increasing the number of registrations.
(1) “Get Started” — which seems less committal than “Create a Wallet” and also suggests that you don’t need to start with a Zoona branch
(2) “Register for Wallet” — as the word “register” came out in our user testing as much clearer than “create”
THE RESULTS:
“Get Started” led to the most people starting the registration (+18% from “Create”), but over half then abandoned the process, perhaps because they didn’t realize what they were in for.
“Register for Wallet,” meanwhile, still led to a healthy 12% increase in registrations initiated, but more importantly, led to a whopping 62% increase in completed registrations.
Quite a big win for such a small change!
Sunga savings reminders experiments
THE PROBLEM: After beta testing Sunga (a savings/bank account product) in one Zambian town, we saw in our data that many people opened their accounts by depositing substantial sums, but then didn’t touch Sunga for weeks.
THE INSIGHT: We called some of these customers to ask them why, and time and again, they said they had simply forgotten to deposit their extra cash. We were skeptical, but decided to test their alleged forgetfulness by sending text message (SMS) savings reminders. I looked into the behavioral economics literature, to see what similar experiments and learnings had been gathered before.
THE EXPERIMENTS: Over the next couple months, we ran several iterative multivariate experiments to test hypotheses like:
General principle: Weekly savings reminders would lead to more frequent account activity (deposits and withdrawals), and higher overall deposit balance. Result: Yes, reminders worked.
Frequency: Weekly savings reminders were more effective than fortnightly or monthly reminders. Result: Weekly was best.
Content: Reminders (“don’t forget”) were better than financial literacy nudges (“savings is important”). Result: Yes, reminders worked way better. Financial literacy was seen as patronizing and performed worse than no SMS.
Content: Weekly savings reminders were more effective when given a target round number goal (e.g. “Deposit K15 this week, to reach a K500 savings balance.”). Result: No, generic reminder worked better.
I aim to create segmented, targeted product experiences.
As part of refining my product, I segment users and then create journeys specifically designed for their needs. Sometimes, that requires different features, but often, it just requires thoughtful onboarding and product messaging.
Mapping out Zoona Wallet onboarding
Customers often abandon products quite quickly, which means onboarding them at the beginning is incredibly important.
At Zoona, we realized we were not properly onboarding our Wallet customers. I was asked to rethink it. On the right is the resulting initial brainstorm. Previous learnings are in yellow, product interactions are in grey, and product communications (mainly SMS) are in green.
My optimization toolbox
I’m pretty quick to pick up new software, but as of now, I’ve used:
For analytics: GA, Kissmetrics, Crazy Egg, MixPanel, Appsee, Qlikview
For data analysis: SQL, Alteryx, SPSS Modeler, Excel
For out-of-the-box A/B testing: Optimizely, Apptimize, Google Play
For custom-built A/B tests: Wasabi, Airflow
For product communications tests: Mailchimp
I have also explored (not yet used, but would love the opportunity to!):
Data unification and segmentation: Segment, mParticle
Onboarding/notifications: Leanplum, OneSignal, Customer.io
BI dashboards: Looker, Tableau
THE REST OF MY PRODUCT PORTFOLIO:
PORTFOLIO INTRO
1. LEARN
2. DESIGN & BUILD
3. LAUNCH
4. OPTIMIZE