“Lean thinking” has transformed the way startups pursue growth, and has even permeated traditional management research. One of the cornerstones of this approach is the need for measurement driven decision making. This is because of the extremely uncertain situation startups find themselves in; a product that aims to disrupt will necessarily be confronted with a lack of relevant data (precisely it’s never been tried before). Whereas traditional businesses can rely on industry reports, conventional benchmarks and historical market trends, none of this is relevant to a disruptive startup, making the pursuit of good data even more crucial to growing a workable business model.
This pursuit of measurement often leads founders to fall prey to distinctly un-lean (fat?) thinking when confronted by the proliferation of types of metrics available. This usually involves time-and-cash strapped founders getting bogged down in tracking too many data points and being distracted by vanity metrics.
This problem is the focus of the book “Lean Analytics” by Alistair Croll and Ben Yoskovitz, which explains how the right approach to analytics can be the deciding factor between success and failure. At Bullet, we’re focussed on making startups better at what they do, so we were dying to get some insight from Alistair around early-stage growth metrics. He was kind enough to answer some questions we came up with after watching an interview he did with Software Advice. Our Q+A is below, but first here’s the video:
As you can see, Alistair has a lot of experience with startup metrics and has a lot of expertise to share. Here are the 4 burning questions about metrics he answered for us:
Eoghan: In the interview you explained how young startups often don’t know what business they’re in. Do you have any practical tips for startups in a stage of product flux to avoid focussing on the wrong data points?
Alistair: This is a strangely Zen-like attitude. Founders need to be passionate about solving a problem. But do you know what you call solving a problem nobody else has? A hobby. Similarly, if you’re just building a product that you want, but not planning on creating a business model, you may as well use Kickstarter or Indiegogo.
The purpose of a startup is to create a sustainable business model. Your product is just a means to that end. With that in mind, it’s much easier to abandon assumptions you may have made if they aren’t getting you closer to that goal. You might change from a product to a service; from an old market to a young one; from natural to artificial; from viral to paid acquisition—whatever you need to do to find sustainability.
The other point Ben makes often is that a pivot isn’t an excuse to change your mind. If someone tells me they’re pivoting, I ask them what learning made them do so. If they can’t answer, they’re not pivoting—their flailing about aimlessly.
E: Have you noticed a disconnect/conflict between product and sales metrics in SaaS businesses?
A: Good question. This is the topic of a blog I’m writing for Lean Analytics, which will be out shortly.
There’s a disconnect, but often it’s intentional. As a user, your goals may not be that of your SaaS provider. Consider, for example, Expensify. I love it. I rely on it. But the UI by which you upload, crop, and rotate receipts you’ve scanned is atrocious. So the product metric—usability, is bad.
On the other hand, Expensify makes money from a scanning service where, rather than you having to do it yourself, they charge you a small amount to do so. The business metric—revenue—is encouraged by a weak product metric.
This is a natural tension. It happens in all product categories. For example, the LinkedIn app for iPad is beautiful. But when you’re reading a story in it, you can’t copy a link to the iPad clipboard. This drives me nuts—I’m playing in LinkedIn’s walled garden. But for LinkedIn, this is a way of keeping sharing and interaction within their application, encouraging shares within LinkedIn itself rather than, say, on Twitter or email.
If you step back from these frustrations, you’ll often find that there isn’t a disconnect. There’s a balance. It’s a hard one to strike: too onerous, and you alienate users. Not tough enough, and you can’t force people to upgrade or pay in the way you hope, and your business model crumbles.
E: If you were launching a SaaS startup tomorrow, what 5 metrics would you keep your eye on?
A: Well, I wouldn’t. I’d pick one. That’s the core idea of the One Metric That Matters.
In the early stages, I’d count how many people I’d talked to; then how many people agreed with the solution I was proposing. But during this “empathy” stage, I’d be much more concerned with qualitative information to ensure I’d found a real problem.
As I was building the first version of the product, I might look at the pent-up demand—for example, using a signup page from Launchrock or the kind of queue that Mailbox used. Once I opened up the beta, I’d care about what percent of the mailing list enrolled when I told them to.
At that point, I’d switch to stickiness. This is about engagement—how many of the users adopt the service as their new way of doing things. This might be as simple as whether they return each day; or more sophisticated stuff like whether they’re using advanced features, or moving along the learning curve and completing tasks more efficiently with less uses of the Undo button.
Once I knew I had a sticky product, I’d move to virality (cycle time, viral coefficient.) Then as I had decent word of mouth acting as a force multiplier for each user I added, I’d start to look at revenues, and pouring a third of the revenues back into customer acquisition. In the revenue stage I care about customer acquisition cost, customer lifetime value, and the time it takes to pay back the acquisition cost.
In the final stage, which we call scale, I care about things like channel revenues and margins, and the adoption via APIs or application exchanges related to the product.
The point is that I don’t care about all of these. I care about one at a time, diligently and methodically, so I can optimize that part of the business and grow in a deliberate, measured way rather than scaling prematurely.
E: You’ve spent years working as an entrepreneur, what is the single biggest lessons you learned re: metrics?
A: It’s hard to see the forest for the trees. When you’re in the middle of the fight, it can seem overwhelming. But good leaders learn to step way back and understand what really matters. This is the same skill as task management—we all have a thousand things a day we need to do, but few of us know the one thing each day that will have a real impact on our lives or our businesses.
This is valuable advice for any founder. Are you pivoting based upon tested assumptions? Are you building a product you’re certain people will pay for? Do you know what your “one metric that matters” is? Do your decisions strike a balance between sales and product metrics? Does your “one metric that matters” fit with the stage of growth you’re at?
If you answered no to any of the above questions, you’re doing it wrong. Data is good, but drowning yourself in a sea of useless information is going to confound your search for product/market fit. This is particularly important for young, bootstrapped startups, who need to be extra-judicious about where they allocate their limited resources.