Building a customer churn model


Hi, this is Ben Rigby, CEO of Sparked, here with Cody Chapman, our Lead Data Scientist, and we’re going to talk today about building a customer churn model. Exciting stuff. Cody, we’ve written four phases up on the board here.

Phase one is defining churn. What are some of the possibilities?



First, it’s really important that we have a clear definition of churn. If we’re a subscription business, this is very easy. We can just look at the contracts data to see if an account has renewed or churned at any given point. This is the simplest case.

A more complex definition is what we call “Activity Churn.” If you’re an Ecommerce business or mobile app, for example, you can label an account as “churned” if you haven’t seen it during a given period of time: if you haven’t seen it in six months, for example. Defining churn in this way is trickier, because the customer could come back at any point. But you just have to pick a timeframe that’s beyond when most customers have returned historically.



Okay, so two ways to define churn: subscription-based and activity-based.






Phase two: what kind of data is interesting for us to look at when building a churn model?



When we’re first building a model, it’s interesting to look at all of the data: billing data, usage data, and static account data such as what country they’re from, what type of plan they’re on, all of this different data. Then you’d use your domain expertise to prune it down into a more manageable set of signals. We don’t want to be feeding thousands and thousands of signals into the machine learning models. We can do that, but it’s more useful to prune in advance.



Okay, so it’s really not a kitchen sink approach. It’s more a matter of feeding in data that we think will be potentially predictive?



Exactly. It’s a combination of using your business expertise to define a potentially significant set of factors and then allowing the model to pick out what is actually statistically significant from that set.



Okay, phase three: building a classification model. What are some of the possibilities?



Yeah, so now we have churn definition and the pruned set of data. We then feed this data into a model that learns which factors are significant to churn. Black Box models tend to do a little bit of magic behind the scenes and give you a nice prediction out the back, but we don’t know exactly what’s happening in the process. Examples of these are support vector machines (SVMs) and gradient boosting machines (GBMs). There are a number of possibilities here.



An alternative is to use a more interpretable method such as logistic regression. It’s more transparent. It’ll give you a human-understandable result.



Okay, so we got the data, we got definition of churn and we’ve chosen our classification model. What’s the final step?



Now that we’ve chosen our classification model, we feed all of our data in and we let the model learn. And then we can use it for the fun part, which is the forecasting. The model will analyze the data from each account and will forecast an individualized churn probability for each account.



Okay, so at the end of the day, we have a list of all of the accounts or all of the customers with a predicted risk score?



That’s exactly right. How likely they are to churn at their next opportunity.



Okay, and those are the basics for building a customer churn model. Thanks, Cody.



Thank you.



All right. See ya.


What’s new in Customer Success – Thursday, May 5

Hi and welcome to our weekly roundup about what’s new in Customer Success. Each week, we scour the web for the best and brightest, so that you don’t have to! Below, are the top articles from companies, pontificators, and all the vendors.

– Jeff Nickerson, Roundup Curator at Sparked 

From the companies and pontificators:

The Science Behind Customer Churn
This article from Luc Burgelman CEO of NGDATA covers the basic principles of using data to predict potential Customer Churn. He explains that to gain a competitive advantage today companies need to embrace customer intelligence management and make use of real-time big data and machine learning. Love it.

The Ultimate Guide to Calculating Customer Lifetime Value
Do you know how valuable your customers are to your business? Social Media Strategist, Warren Knight gives a great breakdown on how and why CLV is so important to calculate accurately. Read it.

Land, Expand, Retain
Tomasz Tunguz, VC at Redpoint, shows how SaaS companies and startups can communicate their financials using the metrics that are organic to the business. Doing so may make it easier for everyone from investors to operators understand what’s going on. Love it.

From the vendors:

How to Become Customer-Centric like Trunk Club, Invision & Atlassian
Nichole Elizabeth DeMeré from writes about the virtues of becoming “truly customer-centric.” She shares insights from these 3 companies, including a case of “rampant customer success.” Love it.

How Trustpilot Analyzes the Entire Customer Experience
Another good one from It’s a Q&A with Ole Dallerup of Trustpilot. Ole explains that while there are many reporting tools available, if you can’t merge that data with your behavioral data, you may not be able to gain the valuable customer insights that you’re seeking. Totally. Love it.

How to Calculate and Improve Customer Retention Rate
From The SalesForce service hub, this is a nice roundup of retention related data and benchmarks, including an infographic to tack up on your wall to remind you of the formula (er, one of the formulas) to calculate retention rate. Read it.

Top Executives Weigh In On Lessons Learned In Customer Success From the Totango blog we found this recap of the Customer Success Summit 2016 where top execs discuss the valuable lessons they’ve learned through the evolution of customer success at their companies. Good stuff. Watch it.

3 Things Customer Success Management is NOT Another from Totango – Kevin Garcia, Marketing Manager defines Customer Success as a function of what it is not. More good stuff. Read it.

From us at Sparked:

When daily activity is not a good thing
There are a few cases when high levels of customer activity do not correlate with retention. Watch our latest 2-minute short where Sparked CEO talks about these edge cases. LOVE IT


Sparked is the least expensive and quickest-to-setup solution for Customer Success teams. Sets up in 30 mins. We’d love to meet you.



When daily activity is not a good thing

In our last two minute short, we talked about how you can orient your Customer Success efforts around two key customer-focused objectives:

  • More logins (boosting daily activity)
  • More revenue

I mentioned that there were a few edge-case exceptions. Well, a few of you wrote in to ask what those edge cases are, so thought I’d cover them today.

So, in what cases should you be concerned about higher rates of user activity? We’ve come across four edge cases. Of course, the nature of edge cases is that there are always more of them… way out on the long tail. So our list isn’t exhaustive, but it’s pretty good:

Edge case #1:
User is very busy extracting data from your tool so that they can quit. We see this case most commonly with online tools that have a reporting or archiving feature. Here, the user is preparing to pull out of the tool, getting everything they can get out before they quit or stop using it.

At first glance, this user looks like a great customer. They’re very very busy, so the tendency is to think: great, this user is getting a lot of value. But if you look at their usage pattern, you’ll find that they’re too busy. Their level of activity is way above their average over the past few months. This is a case where an early-warning system is a really good idea. You want to catch this high risk user before they start to extract and leave.

Edge case #2:
User is very confused – and clicking around a lot in frustration. We see this case where the product has a compicated user interface. Typically this kind of user won’t stick around for long though. They’ll have trouble getting to value and will quit pretty fast.

Edge case #3:
User is looking at your pricing or account management page. This is a very common pattern in advance of a quit. They’re checking out the pricing options and wondering why they’re paying so much for something they aren’t liking much.

Edge case #4:
Your product doesn’t lend itself to daily usage. There are certain products that just don’t lend themselves to a daily, weekly, or even monthly usage pattern. These are apps with high seasonality or peak usage patterns around certain times, like birthdays or holidays. For these apps, daily usage would be a peculiar pattern.

And there you have it, four edge case exceptions – when driving daily usage doesn’t make sense or isn’t a good thing.

The 2 keys to Customer Success

If you’ve been reading up on Customer Success, you’ve probably seen a number of articles that outline the keys to Customer Success. There are posts about 10-step programs, 12 guiding principles, and even a manifesto. While these approaches aren’t wrong (they’re actually pretty good), they are complex.

At Sparked, we don’t think it needs to be that complex. If you’re in a role where you’re working to deliver more value to your customers and revenue to the business, you can narrow your objectives down to two:

  • Customers should login more
  • Customers should spend more

Ok, so why does this work?

First, let’s talk about logins – otherwise known as daily actives or monthly actives (DAUs or MAUs). After looking at the data from many different companies across industries, we know that customers who use a product frequently are those who are retained over the long term. It’s really that simple. The reason is that people who use a product a lot are getting a lot of value from it. Of course, there are a few exceptions, but they really are edge cases. So, from the customer-value perspective, you can really just boil it down to: are they using it?

BUT, that’s not all you have to care about. Because at the end of the day, you want your business to thrive. So you also need to look at how much money the customer is spending with you (or generating for you, if you’ve got an advertising model). Luckily, if you’re doing well on objective #1, this one falls in line. Customers who are logging in every day or every week are much more likely to find your product indispensable. So when up-sell, cross-sell, or renewal time comes, they’re primed for the purchase.

It’s worth noting that these are key objectives – not metrics. You can formulate your metrics around the specifics of your business. For your business, DAUs might make more sense than MAUs. In fact, when we look at the data, we find that weekly actives are often most significant to retention. But WAUs sounds kind of funny. Maybe that’s the reason we don’t see this metric used very often.

As for spending more, you might care about monthly recurring revenue, average basket size, or lifetime value. In fact, it’s probably lifetime value that you’ll care about most. If so, it means that sometimes you win when the customer spends less per month, but sticks with you for longer. That’s more valuable for the business in the long term.

And there you have it. Two key objectives for Customer Success. See you next time.

What does a Customer Success Manager do?

Here’s another one in our 2 minute short series about Customer Success fundamentals. So, what does a Customer Success Manager (or CSM) do?

To review the definition of Customer Success, it’s: the business function that ensures customers realize value throughout their journey with your company.

Logically given this definition, the Customer Success Manager ensures that customers continues to realize value after their initial purchase of a product or service.

The activities of a CSM vary, depending on the type of business and orientation of the Customer Success team. Typically, the CS team will be attached more closely to either Customer Service or Sales.

When attached to a service function, the Customer Success Manager is concerned with on-boarding, adoption, retention, and satisfaction. When attached to sales, the CSM is more concerned with renewals, cross-sales, and up-sales. Of course, all of these functions are interrelated. Overall, a customer who is realizing value from the product is going to one that spends more money over time.

It’s important to note that the CSM is not a customer service rep or a salesperson. These roles have very specific day-to-day activities that may not be concerned with customer value. The CSM is 100% focused on customer value – and therefore is often focused more on proactive activities, versus fire-fighting or selling.

And there you have it, a very simple definition of the Customer Success Manager role.

See you next time.

What is Customer Success?

Today, we’re going to cover the fundamentals, because we’re seeing a lot of you come to our website with one simple question: What is the definition of Customer Success?

Simply: Customer Success is the business function that ensures customers realize value throughout their journey with your company.

Of course, if you’re doing a good job at Customer Success, the journey will be a long one and your business will earn a good financial return throughout.

So, is Customer Success something new? No, not really. The best companies have been practicing Customer Success for eons. The archetypal example is the small shop owner who greets you by name, knows your order, and helps you find something new that you’ll love.

What is new is applying the “Customer Success” label to these activities. And although it sounds like a small thing, applying a label can be a transformative moment. Because all of these things that you kind of knew were good ideas, start to become documented, tested, and organized under one umbrella.

And so it is with Customer Success, where there are now functional areas at companies with titles like VP of Customer Success and Customer Success Manager; there are products (like ours at Sparked); best practices and lots of thinking being done under this Customer Success umbrella. So, it turns out that the label is really useful. Rather than a bunch of ad-hoc practices, we have a corpus of practices and a community of practitioners doing “Customer Success.”

And there you have it! Customer Success defined. See you next time.

What’s new in Customer Success – Thursday, April 21

Hi and welcome to our weekly roundup about what’s new in Customer Success. Each week, we scour the web for the best and brightest, so that you don’t have to! Below, are the top articles from companies, pontificators, and all the vendors.

– Jeff Nickerson, Roundup Curator at Sparked 

From the companies and pontificators:

5 customer-centric metrics to implement today
Joel Elkind, Head of Customer Success at Custora, shares five Key Performance Indicators that you should be measuring if you want to put customers at the center of your decision-making. Read it.

Using Data to Drive Sales and Marketing Results
Eric Estrella, Senior Consultant at Sales Benchmark Index, explains that you need to understand the revenue potential by account, product, and buyer. Because when you have incomplete information, resources are not allocated correctly and performance falls short. Read it.

9 Vital Customer Analytics Every Manager Should Know
Bernard Marr from Forbes writes about how customer analytics are vital for assessing how well your business acquires, retains, satisfies, and engages their customers effectively. Love it.
We Want You Back – How To Prove It To Your Customers
COO of CustomerBliss, Jeanne Bliss, gives us some great examples of what to do when the inevitable happens, and you have suffered a failure that causes the disappointment of your customers. Check out these tips and if you ever need to show your customers that you want them back. Read it.
The Role of Text Analytics in Managing Customer Loyalty and Churn
The CTO of Attensity, Mark Schmelzenbach, discusses how new rules for assessing customer loyalty, value, and churn require more data from the customer journey. Love it.

From the vendors:

The Secret To Unleashing Data-Driven Creativity
From The Amplitude blog we found this great article on finding the one metric that matters most for your team to focus it’s efforts. On finding your OMTM. Love it.
The Causes of SaaS Churn and How to Prevent It
John Rode of Ptero shares this great infographic showing how the seeds of SAAS churn are planted early and deep. Read it.

You’re the New VP of Customer Success. Now What?
From Gainsight – Dan Steinman, Chief Customer Officer gives you some tips if you’ve just landed that new gig as VP of Customer Success. Read it.
Totango Summit: 5 Heads of ‘Customer Success’ Share Lessons Learned
Megan Headly, who is the research director at TrustRadius attended the Totango Customer Success Summit and interviewed several leaders in Customer Success about the trends and changes they’ve observed. Read it. 
25 Tools to Integrate Customer Success into Your Business
Nichole Elizabeth DeMeré from brings us an awesomely comprehensive list of high-level tools for running your Customer Success efforts. Love it.

From us at Sparked:

How Fitbit is *rocking* Customer Success
This is the last in our series about Fitbit and Customer Success. In addition to providing fertile ground for making analogies, as a business, Fitbit is doing a phenomenal job at Customer Success. In this last video we look at Fitbit as a case study. Love love it.


Sparked is the least expensive and quickest-to-setup solution for Customer Success teams. Sets up in 30 mins. We’d love to meet you. 



How Fitbit is *rocking* Customer Success

This is the last in our series about Fitbit and Customer Success. In the last three videos, we talked about how Fitbit can teach us (almost) everything we need to know about Customer Success and made analogies between personal fitness trackers and Customer Success.

This time, we’re going to look at how Fitbit is actually doing Customer Success. Because in addition to providing fertile ground for making analogies, as a business, Fitbit is actually doing a phenomenal job at Customer Success. So in this last video, we’re going to look at Fitbit as a case study in Customer Success.

To frame the mechanics of their business, here’s how it works:

  • They sell a personal fitness tracker, which is a physical device that connects to an app on your phone or website on your PC
  • They’ve got about 10 different models priced from $49 to $250, in addition to a variety of accessories, like watch bands
  • They’ve also got a premium subscription service for $50/yr

In sum, they’re selling a hardware device with a number of optional accessories and a subscription service all supported by a mobile app and website.

As a B2C company, they’re selling their product to millions of people and at a relatively low price point – so we know off the bat that their Customer Success strategy is going to be automated, scalable, and low cost. Whether you’ve got millions of customers or hundreds, the objectives of Customer Success are going to be the same:

  • More logins (daily activity)
  • More revenue

Of course, we know that customers who login more and spend more are those that are getting more value from your product. That’s a given. So, what is Fitbit doing to achieve these two Customer Success objectives. Let’s see:

First, let’s look at how they’re driving logins (daily activity):

  • They’re sending lifecycle emails, related to the product puchased. Some examples:
    – “Today is the day! Your order has been shipped.” – Getting me excited about my purchase
    – “Say hello to your Blaze” – Welcoming me to the product
    – “Blaze 101” – a weekly email teaching me how to use the product
    These are there most impactful tactics – driving directly to value and ongoing usage.
  • These are interspersed with more general emails that apply to all customers:
    – “Check out the 2 newest features from Fitbit”
    – “Find your perfect step goal”
    – “Be good to your heart with these 12 heart-healthy foods”
    These are a little less focused, but still driving to more frequent usage and value.
  • And then there are automated emails related directly to usage of the product, such as this one:
    – “Your battery is low: here’s what you can do.” – Here, they’re preventing a negative – battery death, which can be a big problem with battery powered hardware devices – as we know from my other video on the Arlo security cam.
  • Finally, outside of communications to the customer, they’re making ongoing improvements to the product itself: – They’ve gone from a very simple pedometer in 2011 to a product that gives you all kinds of reasons to check in every day…and even hourly
    – With these new features, come many opportunities for Fitbit to bring cusotmers back to the device or app, such as:
    – Weekly progress report emails
    – Badge notifications
    – And messages or challenges from friends
    Product improvements are obviously more of a long term activity and involve many aspects of the business. That said, you can see that their teams are working hard to drive daily usage, which is perhaps the prime factor in cultivating successful customers (for most businesses). As a Customer Success leader, you’ll want to be involved in helping the product team build features that drive daily activity and value.

Now let’s look at driving revenue:

  • They’re doing it through cross-sell and up-sell emails:
    – “The new Fitbit Blaze is finally here” – This one worked on me – I bought the Blaze
    – “Fitbit Alta is now available”
    – “Fitbit Blaze and Alta now available” – As you can see here, they’ve made a bit of a mistake, since I’d already purchased the Blaze… which is a problem with their drip email campaign rules not totally being in sync

And finally, let’s look at their customer service, which straddles both primary objectives. To ensure that customers are using the product every day and are in a position to purchase more of it, customer service has got to ensure that customers are not having any problems that would prevent them from doing so. So, let’s look at Fitbit’s customer service as it relates to an issue I had with my Fitbit Zip.

  • Back in 2013, I purchased a Fitbit Zip – and earlier this year, I lost the piece that keeps the battery in, so I contacted customer support to see if I could get a new one
  • The response I received was that my Fitbit was out of warranty, so they couldn’t replace the piece, but they could give me a 25% discount on a future purchase
  • I took the 25% and bought the much more expense and feature-rich Fitbit Blaze

This is a great example for how they took a customer problem – a potentially fatal problem in terms of Customer Success, in that the product was no longer working at all – and turned it around into a revenue opportunity. To boot, since their product had improved so much since the last time I purchased a Fitbit, it also resulted in dramatically boosting daily activity.

All around, Fitbit is doing an excellent job at driving daily activity and revenue, while fighting fires with their customer service. I give them an “A” for B2C Customer Success at scale. And that concludes our series on How Fitbit teaches you (almost) everything you need to know about Customer Success. Hope you enjoyed and we’ll see you next time.

What you can learn about Customer Success from Fitbit’s “drinking-water” goal

This is the third in our series about how the Fitbit teaches you (almost) everything you need to know about Customer Success.

Today, we’re going to talk about Fitbit’s secondary goals. Fitbit’s core mechanic is taking steps (which we talk about in another video), but there are a number of secondary goals you can track – all of which contribute to your overall health and fitness.

You can set goals around:

  • Sleep
  • Calories eaten
  • Active minutes per day
  • Various forms of exercise like biking and running
  • And even a goal for your daily water consumption

Some of these goals may be relevant to you and others not. You can see that I haven’t logged any water consumption – as it’s not a goal that is important to me.

For the purposes of your Customer Success efforts, you’ll want to set one or two core behavioral goals for your customers. These will be your primary success objectives, similar to the 10,000 steps a day goal in Fitbit. We recommend that you set your core Customer Success goals to: logging-in often and spending more.

Beyond your core goals, you’ll want to set a number of secondary goals all of which help you create a more fit customer base. Just as the water goal didn’t apply to me, some of these goals may be relevant only for a specific customer segment. For example:

  • A real estate service may have different feature-usage goals for its brokers and consumers
  • Or a b2b company with high value accounts, might have a specific content consumption goal just for decision makers or C-level users

Each of these secondary goals should aim to reinforce a known successful behavior for a particular customer segment.

Over the years, Sparked has looked at many datasets and interviewed numerous teams about their customer success efforts. Across the board, we’ve found that the teams that set up specific and measurable goals for their customers, are those that win.

Sparked Customer Fitness Feed – 4/5/16

The Sparked Customer Fitness Feed delivers *AWESOME* articles from around the Web to help customer success teams build a more fit customer base. Fit customers login more, spend more, and love your product more. Unfit customers don’t use your product and eventually churn out (gasp!). Let’s get’em fit.


Customer Success Team Planning & Cost Benchmarks

As most Customer Success leaders tend to be much more operations-focused as opposed to accounting-focused, it’s the right question to be asking. Well, we heard you loud and clear! At our recent Chief Customer Officer Summit, I asked a series of survey questions in order to capture these important Customer Success Cost benchmarks. View the full article

How The Business With The Best Customer Focus Always Wins

It doesn’t matter whether you have a shop, a restaurant, or a bank. Without customers, your business doesn’t exist. And thanks to the ease and connectivity of social media and online review sites, your customers hold more and more of the cards. In the Age of the Customer, the business with the best customer experience is the one that is most likely to win. View the full article

This is the difference between statistics and data science

Director of Data Scientist at Twitch talks about the intersection of marketing, product, data science, and stats. Great background for the Customer Success leader. View the full article

Doing Data Science Right — Your Most Common Questions Answered

Instacart VP of Data Science and former LinkedIn data leader have an engaging discussion about using data science to build better products. View the full article

Customer Experience and a Framework for Quality

“The cost of quality is the expense of doing things wrong.” That’s a quote that may sound like a blast from the past but it’s just as relevant today, if not more so, when thinking about quality and the quickly evolving definition of customer experience. View the full article

Curated by Jeff Nickerson and the Sparked team. Got feedback? Send an email to

Sparked is the least expensive and quickest-to-setup solution for customer success teams. We help you cultivate fit customers. Give it a try.