This was posted in 2012 on my personal blog. It unpacks the analysis we did to understand why we were churning customers at my first SaaS business, SocialWOD, and what we did about it.
I was stressing.
We had seen 40% of our customers cancel, and it was eating me up inside.
Each of those (former) customers had:
- decided they had the problem SocialWOD solves (workout tracking for every gym member at your gym, just by snapping and emailing a photo of your results whiteboard)
- decided they needed to solve that problem
- done research on solutions that might solve their problem
- decided that SocialWOD was compelling enough to try
- decided to take out their credit card to pay SocialWOD
- used and promoted SocialWOD to their members
After investing that much mental and emotional energy, they then they decided to cancel.
Argh.
We were trying to understand what the key differences were between our customers that cancelled, and our customers that didn’t.
We hop on the phone every week or two to call up customers, so qualitative data was no problem.
But we wanted to see what we could learn from our data.
Enter the Cohort Analysis
A cohort analysis helps you see how behavior varies across different cohorts of customers.
In this case, we were interested in the behavior of the cohort of customers who HAD cancelled, vs the behavior of those that hadn’t. What differences were there in each cohort’s usage of our product, and could we steer more gyms to do things like those gyms that had stayed on as customers?
I had read high-level stuff about cohort analyses, but it was this 90 minute cohort analysis class in NY with genius marketer Cassie Lancellotti-Young that really opened my eyes to how to run a proper one.
What Tools to use
Cassie recommended using Excel, and provided a template to use as the basis for a cohort analysis.
She mentioned tools like Kissmetrics and Mixpanel, but they were harder to use, less flexible, and required more overhead than Excel.
Given that likely all of the useful data you’ll want to analyze lives in your application’s database, her suggestion made a ton of sense to me.
How to start
We started by asking questions.
In this case, we were trying to answer this question:
“What are the major differences between customers that cancel, and customers that don’t?”
Pull some data
The next step was to write a ginormous SQL query pulling out all the data that could possibly help us answer this question. Here’s some of the relevant data we pulled:
- The month a customer joined in
- The amount a customer paid per month
- Free trial length
- The number of a gym's members that claimed their SocialWOD profiles
- # days after paying before canceling their service
- whether a gym uses SocialWOD to post their workouts to their gym's Facebook page (where the gym's members hang out)
All in all, there were 51 different fields of data that we pulled.
Analyze the raw data
I ran pivot tables comparing the difference between gyms that cancelled and gyms that didn’t.
I learned that customers who cancelled:
- had about half as many of their members claim a SocialWOD profile (so they could engage with our product)
- used the "post our workout results to my gym's Facebook page" feature about half as much as gyms who remained customers
- paid about 23% more than gyms who stayed
- cancelled their subscription after 61 days on average
Square the quantitative with the qualitative
We have a spreadsheet that details cancellation reasons for every customer, and I’d estimate we’ve talked to at least half of the customers who quit. The two most common reasons we kept hearing about why gyms quit were:
- our athletes aren't using it
- it's too expensive. SocialWOD is a nice-to-have (we're working on that ;) ). A member CRM is essentially a must-have for a gym over a certain size. The price for a member CRM is lower that it should be imho, and it sets a strong price anchor in the mind of a customer (i.e. "I *need* a CRM and it costs $X. I don't *need* SocialWOD, but it costs $1.5X.")
Given that the quantitative data suggested high member usage and lower price were big drivers of customer retention, the qualitative and quantitative data told a pretty strong story about what to do to cut down on cancellations.
Acting on the results
Here’s what we decided to do about each learning:
1. A cancelled customer had about half as many of their members claim a SocialWOD profile (so they could engage with our product).
Previously, our customers would tell their members to sign up for SocialWOD by posting an announcement about SocialWOD on their gym’s Facebook page, writing on their gym’s blog (read by most members), talking about SocialWOD at the gym, etc.
We figured there’s nothing easier to act on than an email describing benefits and a call-to-action. So we improved our onboarding to help a gym owner export a CSV of their members’ email addresses to send to us.
Once they do, we email each member a description of how SocialWOD benefits them, along with a link to click to sign up.
2. A cancelled customer used the “post our workout results to my gym’s Facebook page” feature about half as much as gyms who remained customers.
We improved our onboarding so that customers who haven’t set up posting to Facebook receive an email detailing the benefits, providing a video, and a strong CTA to do so.
3. A cancelled customer paid about 23% more than gyms who stayed
We built some technology than makes processing photos cheaper for larger gyms, and passed those savings on to both our existing and new customers. Those changes enabled us to drop prices by 15-60%, depending on price plan.
4. A customer cancelled their subscription after 61 days on average
We haven’t acted on this yet, but I can imagine sending gyms that meet some “likely to cancel” criteria an offer around day 45 of their membership to lock in for a year at a heavy discount.
The Outcome
Since implementing changes 1-3 two months ago, we’ve seen our cancellation rate drop from 40% to 5% - an 87.5% decrease. We’re going to run another cohort analysis in a couple months to isolate the impact of each change as it’s still too early to know the long-term impact of these changes, but in the short-term both Ryan and I are stoked about stopping the hemorrhaging.