“We were getting lots of positive feedback about Bullet, but we just weren’t getting the sign-ups. As a freemium product you rely heavily on word of mouth. We had a problem.”

Questions about why our product is just not cutting it.

At this point, we were getting tons of positive feedback from our customers but Bullet wasn’t spreading as we had projected. Being a freemium product, its premium features need to sell for it to drive growth.

Asking questions about your business keeps the feeling of forward momentum going. Even if most of the time you’re just moving from left to right and back again.

But, asking questions when the mood is low can be difficult. We discovered this first hand.

The low mood was as a result of the dilemma of good reviews, high uptake of the free product but little return from the premium offerings.

We couldn’t figure out whether the low sales were, as a result, a faulty product, a poor landing page or simply a marketing problem. Even though we had done some testing, it was time to start ticking possible issues off a list with tangle data.

Standard Reply from Customers

Looking at your own business objectively is a hard thing to do. How many times have you heard founders say “we have a great product or blog, people love it, but they’re just not signing up”?.

Objectiveness is a blind spot for every entrepreneur, and that can be dangerous.

Take Away:

Being objective is hard because most founders tend to start companies in order to scratch their own itch. This makes the company personal, therefore evaluating where you’re going wrong can be hard. Find objective ways to 

Find ways to ask questions that invite people to be objective, or negative. “Tell me 3 things you don’t like about Bullet”.

So we decided our first focus would be clearly understanding what customers thought of our product. Using tangible data and not some assumptions. Our goal is to create a great product for small business users, so getting feedback from such users was key.

We had done tons of interviews before, and we personally emailed the first 4000 people who signed up to Bullet.

We also kept on improving our product by adding possible features requested by our users. We fixed bugs and made changes that made the product better.

But here we were, still lost on why we were not making as many sales and we hoped to.

We realised we needed something very unemotive, something that gave our product an objective, quantitative score. We looked around and came across Net Promoter Score (NPS).

Take Away:

Talk to your customers and refrain from asking them what they liked, you probably already know the answer to this. Instead, ask them what the didn’t like. This invites them to give you honest feedback without sounding offensive.

You live in ‘founder world’, you know where everything is, what your service is about and what the end result will be. Your customer, on the other hand, lives in ‘customer world’. They’ve no idea what’s going to happen when they buy your product/service. So try and look through their eyes by requesting their feedback constantly.

Customer scores are grouped as follows:

• Promoters (score 9-10): These are loyal enthusiasts who will keep buying your product and refer others, fueling growth.

• Passives (score 7-8): These are satisfied but unenthusiastic customers who are vulnerable to what your competitors have to offer.

• Detractors (score 0-6): These are unhappy customers who can damage your brand and impede growth through negative word-of-mouth.

It’s important to note that NPS isn’t the answer to all your questions, but, it’s an important starting point.

Take Away:

This is a brutal way of testing your product. Get a score below 9 and you get punished. Make sure you test a good customer segment. Keep away from power users, they’ll be bias.

Founder doc conversation between John & Peter on picking an NPS product

Having looked around at a couple of products such as Survey Monkey and Satmetrix we settled on a super light modern product called Delighted (we’ve no connection with them).

We looked at who we were going to target for the survey; infrequent users, power users, and cloud app users.

We’re an accounting product so some people like a contractor might create one invoice a month, this doesn’t mean he’s not an engaged user. Hence the decision to include infrequent users.

We ignored heavy users because we didn’t want to talk to people who’d confidence in Bullet that would really be a vanity test unless of course, they had to use your product.

Animation of Delighted NPS Software

All of these sections will provide you with some good insights, although power users will just pat your back.

We decided to focus on a segment of 800 users, those that have signed in 3-5 times in the last 3 months. The user or reader has to have being a little engaged with your product as the NPS is about your product or service.

Now we had a good number of customers, who weren’t power users, let’s see what they had to say.

Take Away:

Make sure the audience you pick will answer your problem and make sure you have over 50 customers to test. You want a nice number to rule out pops.

“Just over an hour in looks like 75 is our score. Here is are the benchmarks for 2014. Great news. Now where the fuck are our customers? But, that’s a great tangle score. And we’re only starting our big UI update.”

Email from Joel of Delighted.com

The good feedback started to roll in, which was all about our product. This brightened up our mood, as our product is a key part of a freemium sell, it has to sell itself.

Here is a sample quote of what a Promoter said:

Net Promoter Score 10 quote promoters

Remember promoters are the ones that will push your business.

Passives are a score of 8-7. Again nothing jumped out too bad but these customers were as in love with Bullet as the promoters verbally, they just didn’t score us in the 9-10 bracket.

Net Promoter Score 8 passives quote

Detractors are from 6 down to 0. We only got about 16 detractors, and some of them were people giving us a 1 thinking it was first place (top marks), but we also got some people hating the product.

We knew some of them as x customers and ignored most of them.

Net Promoter Score 6 detractors quote

Are we starting to separate the wood from the trees?

Take Away:

The NPS shows you how similar some of the positive comments are between promoters and passives are but the scoring splits them out. This adds some quantitative reality.

View of Bullet customers out voting Apple, Netflix, Amazon with a Net Promoter Score

We got a huge score compared to other supercharged companies which was a great high five and testament that we’d built a good product.

But, it didn’t answer the question as to why people weren’t signing up (remember we’re testing word of mouth).

Delighted Net Promoter dashboard showing Promoters

74% of the customers we surveyed were Promoters, so why weren’t we growing at a similar percentage when we’d 7.4 people in every 10 loving Bullet?

To add to make matters worse, our passive customer feedback were mostly in the 7-8 bracket, so just below promoters. Their comments were all positive as well.

This had us perplexed, but something good had happened. We realised that:

• We have a good product; that’s core to our business, and removed the founder ‘product beer goggles’.

• On-boarding was working; we’d picked people who had had little to some engagement with the product and ignored power users.

• We could probably work better on providing better perceived value. The value was there – they’re just not seeing it.

• We had to fix our marketing site.

• It might be more about marketing after all.

Take Away:

Sometimes the answers create more questions, but that’s OK, you’re moving forward. Get a clear idea of what you want to understand and be honest with how you ask about it.

Saying that we we’re 100% happy with it, again challenging our founder glasses.

We realised that we were falling behind a bit building virality into our product. Project Management tools have virality built in as you’ve to invite people into the project for the product to work.

As for accounts… well you’re not going to invite you neighbour to have a look at your invoices!

From the results you could easily argue that we needed to advertise more, or amplify word of mouth. But, again we wanted to product to be the key driver growth.

NPS is telling us the people onboard are happy. So we need to put effort in onboarding more people.

So we decided to interview people who didn’t fully onboard.

We did some deep research on how best to do this, as again you don’t want to lead people nor do you want a polite pat on the back.

The results from this questionnaire lead us to rebuild our marketing site from copy up and also add some features to our on-boarding process.

In the next post, I’ll explain the questions we asked, why we asked them, the results we got and what we changed.

Be sure to sign-up to our blog, but let’s continue and see how you can apply this to your company.

Take Away:

Really try to get into the head of your customer, again don’t fight it. 

You might not be a freemium product business, you might be running a blog or an online shop. But, the same principles apply.

You’re trying to separate the wood from the trees, you want to see the truth, remove the people who pat you on the back as well as the haters and just focus on standard customers.

I believe you’re always better off starting wide and then narrow your questions down. That way you mitigate yourself from making assumptions.

Asking a big group also removes spikes from vocal positive people and negative people.

These exercises are great, cheap and easy to deploy. They allow any business, big or small, traffic or no traffic to test their offering. “Would you share my blog content”, “Is my Amazon store selling the right goods for your cake decorating business’ and so on.

So in conclusion:

• Ask yourself what answer you want to find.

• Pick a customer segment that will best answer that.

• Execute a simple survey. From experience, NPS is the simplest.

• Analyse the responses objectively.

• Don’t ignore any further questions that arise from the survey.

• Repeat.

How would you apply this to your business? Let me know in teh comments below. I always respond.