When “gut feelings” aren’t enough: using data to plan your product roadmap

When I began working on Front a few years ago, I didn’t anticipate how often I’d have to rely on my gut to plan and prioritize our product roadmap. I’d heard and read everywhere how important it is to be data-driven, to test everything, and to be skeptical of your instincts.

But being data-driven in an early stage company is really difficult—you can imagine why. You’re doing something new, you don’t have all the data, you don’t have the time, optimizations don’t move the needle right away, and many decisions seem obvious.

As your company grows, though, the planning landscape changes dramatically. Your access to meaningful data improves, you have a better understanding of your market and customers, and decisions become more complex and consequential. Data becomes a critical part of your planning process so that you know you’re building the right things, at the right time, in the right way.

It’s important to set realistic expectations and boundaries around the purpose of data in your plans.

Here’s how to get started with data-driven planning to shape your product roadmap—and business.

Plan ahead for planning

Of course, transitioning from early-stage, gut-based planning to a more structured, data-driven approach doesn’t happen overnight. The key is laying your data foundations early and then iterating over time.

At Front, we’re still making the transition. But because many mentors regretted investing in their data infrastructure too late, I made a deliberate decision to lay those foundations early on. To help train and build our “data muscles,” we also meet regularly with a team of data experts from our lead investor, Social Capital, that gives us feedback and advice on our data collection and analysis.

Define success metrics

So far, the most noticeable way data has impacted our planning process and product roadmap is in how we define our goals. Data can be an incredibly powerful communication tool, but even the most informed product teams can skew or misunderstand the purpose of a new feature because of a miscommunication.

For example, we have a project titled Search Improvements on our product roadmap. That seems straightforward enough, but it actually has a few different interpretations:

• “Make search results more accurate.”
• “Make search syntax more flexible.”
• “Make the search engine return results faster.”
• “Make the search engine search more data.”

With a short deadline, it may be tempting for a team to address just one of these goals or the one they can solve most quickly (say, making the search engine return results faster). But through further analysis you might learn that what users actually struggle with is syntax. They redo similar searches over and over, hoping that the engine will eventually return the results they’re looking for.

When we ask those users how we could improve Front, they say, “Improve the search!” However, a speed improvement wouldn’t help them at all—they just want better results. With this supporting analysis, we can tie our Search Improvements product plan to a metric, and our goal becomes crystal clear: “Improve search syntax recognition so that re-searching decreases by 20%.” When you quantify your goal, your team will have one north star to work toward with nothing lost in translation.

Data isn’t a cure-all

At the same time, keep a healthy skepticism about your data. As an analyst will tell you, correlation does not mean causation. In layman’s terms, it means that it’s never easy to get to the root cause of an issue.

Say churn is increasing, and you find that daily logins started decreasing around the same time. Which one caused the other? Should you be running campaigns to drive more frequent engagement? Could they both be caused by a third phenomenon, like decreased product stability? Sometimes, the data won’t be able to tell you—you’ll need to rely on your best judgment, test often, and build in frequent reviews to see what tactics are causing these metrics to shift in the direction you want.

Clear-cut answers are disappointingly rare—your role as a manager or project lead is to make judgments based on limited, nuanced, or even inconclusive analysis.

It’s important to set realistic expectations and boundaries around the purpose of data in your plans. Clear-cut answers are disappointingly rare—your role as a manager or project lead is to make judgments based on limited, nuanced, or even inconclusive analysis. You could debate most questions about your product or business indefinitely. When putting together everything from quarterly business goals to engineering sprint plans to product briefs, define the few key metrics you’re trying to drive, your targets, and the strategy for hitting those targets. Then step away from the dashboards.

Give your business a checkup

Planning with data is like going to your annual doctor’s appointment. When things are going great, you feel like you don’t even need it. Why bother? You’ve never felt better. Incorporating your business and product metrics into your regular planning cycles is like those regular checkups.

When you incorporate regular milestones into your project plans, you give yourself a cushion to catch mistakes or misjudgments before investing too much time on the wrong initiatives. And if (or when) you miss your targets, you can get things back on track quickly with confidence and insight.

Here’s where you can start. Every company will have key metrics unique to their business or product, but incorporating the following metrics into your product plans is a must-do for any growing company.

• Revenue: This one is pretty obvious—a growing company needs to make more money.
• More users means more people getting value out of your product. Signups can be artificially inflated compared to MAUs if you aren’t focusing enough on engagement.
• DAU (Daily Active Users): Ideally, you don’t want people to log in once a month; you want them to come back regularly so they experience the value of your product more often.
• Count of Core Action: By core action, I mean the single action your product was designed to do best: for Front, it’s “replying to a message.” User growth should correlate with the “count of core action taken.” Otherwise, customers might be buying your product for a reason you’re unaware of.
• Churn: Acquisition metrics can often mask a churn problem. It’s important to look at how long the average customer sticks around as the starting point for identifying churn drivers.

Establishing baselines by gathering this data and tracking how new features and processes impact them will ease your transition into data-driven planning. Even if they don’t seem to be moving (yet!), keep an eye on them. As you learn more, you can plan new features and angle your operations to amplify the behaviors that drive these metrics.

Your vision will survive

Getting started with data is daunting, and there’s a popular concern among some founders and leaders that being too data-driven might cause you to compromise your vision. That somehow looking at the data will force you to change course radically and lose track of what you really are or where you’re going.

The good news is: that doesn’t happen. Data doesn’t have the power to reshape your product or business from the ground up. It’s simply an objective tool you can use to measure progress—and that’s nothing you should be afraid of.

Mathilde Collin is co-founder and CEO of Front.