In the last days of January, the number of Plumbr users exceeded an important milestone – we now have more than 1,000 users! As we started offering Plumbr as a public beta in the beginning of November last year, we can say that „collecting“ the first 1,000 customers took us 3 months.
Let’s dive into the numbers (we are big fans of factual proof!), and elaborate on:
The beginning was easy. The founders were working in a large IT service provider Webmedia and throughout their day to day Java development jobs experienced tens of different memory-related problems every year. For reasons still unknown the founders became respected specialists in the company and got bogged down by memory troubleshooting issues of different projects. As this got boring to handle manually, they went ahead and created a solution to automate it.
Hence, the first users where the tens (hundreds?) of Java projects developed within Webmedia. The memory patterns provided a very valuable pool to study, and to develop the first version of the leak detection and reporting algorithms. When the internal beta version was launched, we advertised the tool to the Java developers of the company and recruited the first 40 users.
Now to the sad part – those users did not have many problems with memory leaks at that moment. We received data of… well, 2 memory leaks. We didn’t feel comfortable with that knowledge base and started looking for ways to get more data.
Where does one find out whether the problem they are solving is real? Apparently you can Google for it. Or – to be more precise – you can check the Google Keyword Tool. It told us that 135 000 searches for „java.lang.OutOfMemory“ are fired every month. Wow. We had hit the jackpot!
But since we are from Estonia, and by common belief Estonians are slow by nature, we didn’t dare to raise a huge pile of money and pour it to Google’s pockets in exchange of targeted Adwords ads. Instead we decided to grow at a rate where we could interact with the users and improve the product based on their feedback.
Still, it takes months to get the initial PageRank and start receiving organic traffic. How can we enforce this without paying for the users? By luck, one possible solution found us by itself. A friend posted a link to our web page in the Hackernews channel. We only found out about it because the Google Analytics Real Time window (requires login, but is an awesome feature!) suddenly started showing intense traffic. So we picked up on this and posted a link to reddit/r/programming ourselves. Voila – we got 2,000 unique visitors between November 8th and 10th, 2011. Those 2,000 first visitors converted into 194 registrations and 146 downloads.
From there on, until the end of January (when we hit the 1,000 registration mark) we’ve had 26,000 unique web site visitors. This gives us a 3,8% visitor -> registration conversion rate. Not the best conversion rate out there, but at least we had found some channels that we could use for building awareness of the product, and reached the growth rate that we had dreamt of in the beginning.
The graph below is based on data gathered by Google Analytics. It is a bit skewed: for example, a Hackernews post is automatically tweeted by several bots, and top posts receive extra tweet coverage – and that is why some visitors who were directed to the site via Hackernews, are actually shown as Twitter (t.co) referrals. Nevertheless, I think that the distribution is a pretty good illustration:
It has not been only us advertising in those sites. Unproportional advertising of your own site would conflict the policy of several of them and we would probably be quickly banned as spammers. Fortunately, there have been other visitors who liked our content and shared it, so I think more than half of the blog posts have actually been featured in at least one of those link sharing sites.
If you know of any other communities to whom our content would be interesting, please let us know in the comments. Or even better – share the post yourself! (We will pick it up in the referral statistics, don’t worry)
Of course, getting a thousand registrations is great, and certainly marks a milestone. But a clever reader has wondered since the beginning – how much have those 1000 users actually used the product, and what might be the bottlenecks they face that inhibit larger usage? Let us look at the the numbers and elaborate on this next.
But first – for people who have not used Plumbr before – this is a quick overview of how your experience with Plumbr typically flows:
The above effectively describes our usage funnel. And our goal, naturally, is to direct as many users as possible into the later stages of the funnel.
We get feedback from the users in three ways. All of them provide us different data, and for reaching true insights, we need to combine all three.
Let’s visualize the data we have collected:
It seems that out of 1,000 registered users 263 actually started the tool. Shocking, isn’t it?
There is one technical catch though. If we look at the report opening data, we see that 18% of the „open report“ events come from people who have never sent us the „run Plumbr“ event. The reason is that some of the users use Plumbr on servers that forbid outgoing traffic, and the release upgrade check never reaches us.
Assuming that people behind firewalls get (and open) leak reports with the same distribution as the ones who are not, let’s take that 18% into account when analysing our funnel:
Still, shocking. What could be the possible reasons of the low 31% conversion rate?
From the qualitative data we estimate that a whole 30% of people intend to try Plumbr (they download it), but never do because something distracts them and they just add it to their task list.
A fair amount of users probably need Plumbr in an environment that is not supported yet. Many people have asked for Java 5 support (this is one of the top priorities in the list), but we have also received reports of Plumbr not working on Websphere and some Glassfish vesions, with Groovy and some environments.
There are definitely also users who just don’t know how to modify JVM start-up parameters and for whom the installation instruction is too complex. Some of them contact us, but every participation (in)equality study (see e.g. the 90–9–1 principle) shows you that 90% don’t. Let’s use a best guess and accommodate these assumptions on our funnel graph as well:
These are the biggest barriers we could think of, and that were supported by the qualitative feedback. If you think that we have missed something important, please let us know in the comments below!
We have several ideas on what to do to raise the „new visitors->registrations“ conversion on our web page. We need to improve the landing pages (the home page and the blog post page, where more than 90% of the people arrive to plumbr.eu). We need to make a better and more clear offering, explain better what Plumbr is, and publish references and testimonials of current users. We are also planning to A/B test different value proposition and Call-to-actions (I will write a separate blog post on this). We follow closely Peep Laja’s blog on website conversion and have more ideas than resources to try them out.
However, the web conversion rate is only the tip of the iceberg. What we really need to improve is the conversion to product usage. Without it, two-thirds of every effort made to increasing web site traffic or traffic->download conversion would be lost. And without a bigger user base we might take wrong product development decisions.
From the reasoning above we concluded that although we liked to think that our installation process is very smooth and easy, it is still too complex – complex enough that people who download, don’t get „hooked“ and give up using Plumbr before they get started.
So, we plan to investigate and implement:
We released Plumbr 1.0RC4 on Tuesday, and have dived into the development of the next release by now, implementing the aforementioned directions.
For our blog readers – my blog post pipeline includes topics on customer/user segmentation, channel based conversion analysis, our product licensing plans, and – of course – follow-ups to the above-mentioned topics. Follow our twitter feed to be notified when the posts are published!
And, of course, if you got interested in trying out Plumbr yourself, just tell us your e-mail address and you’ll be flown into our usage funnel instantly!