Case Study: My “product fail” thanks to neglecting data I didn’t even seek
What about products that don’t value data? Not investing my time to understand the impact of data resulted in serious consequences for one of my supposedly ready-to-launch products.
I made a self-study product in 2012 for consumers but had limited the audience too much by excluding a key segment from a technological perspective. Why did I exclude a key segment if I wanted to make a successful product?
The reality is I didn’t make the product for others at first — I made it for myself in 2010 to get through my law school exams, and it worked really well, but only on the Windows operating system. It was a couple of years later that I thought I could really get it out there.
Aside from the fact that users reported getting antivirus notices due to the anti-reverse engineering protection I added to the application file, a lot of people asked me, “is there a Mac version?” Deep down, I knew I should have made a web application so that a question like that would never get asked, but I just wanted to get the software out there. I wrote the following in February 2018 to reflect on the experience:
Product Management tips: if you’re going to release publicly the software you used to pass law school, don’t make it Windows-only when your market prefers Macs and smartphones, and don’t integrate anti-reverse engineering and IP protection that trigger anti-virus alerts.
The combination of users dealing both with antivirus notices and with my not building the software for a wider audience meant the product was not going to take off… unless I would address those points head-on. My opinion that “if people like it, they will get it” would never matter because the facts were clear — I didn’t make an accessible solution, and the effort needed to get it to the right state was significant.
This “product fail” is less about data science insights than it is about how I neglected to carry out and apply user research in a serious manner. I could have surveyed hundreds of students, but I already knew the answer — I should have made a web application that all students could access. I later went down this route, but I changed my priorities before releasing the web version.
Had I released it, I would have simplified reaching and benefiting my consumers, and I could have gone further and added user analytics onto the website. For example, by investing in social media advertising to gain exposure to the product, followed by applying A/B testing (serving different versions of the website to different visitors to assess favorability of one version over another), I would have gained a lot of insightful user analytics.
I would have determined what works better for the product by factoring in the “law of large numbers” — meaning, if I get a ton of data, I can have more confidence than not about changes to make to the product. Additionally, I would be able to validate my product strategy if my users would interact with all of the key features regularly.
While that’s data science applied to user research and testing (which it totally should be) to enhance product development, products like Impactoria and Loio have data science as part of their value proposition to their customers.
Case Study: Autonomous Driving Object Detection
Ridwan had some additional interesting points that I believe further support my main idea that opinions with data behind them are more powerful than without:
As a budding data scientist, the use of data science at the intersection of product management is quite evident. As someone that has utilized energy data to build client solutions that decreased their energy usage, it would be the same instance here except the solution this time is the actual product. The data gathered from the product’s use can help gather key insights into what features work, which don’t, and what features should be added.
He’s reflecting on the user analytics part of product management in the above, and he goes further in the following to express how the actual product itself can be based on continuous data analysis:
For example, in regards to how an autonomous vehicle “sees” objects on the road, it has to be able to recognize what the initial object is. I built a model part of an independent study that can recognize different traffic signs, lights, and other vehicles on the road. This is a key feature in how an autonomous car like a Tesla, “sees”, the road. It collects the data on what it is seeing and then analyzes the information in order to make its next turn.
Ridwan continues by providing an analogy that simplifies how the model works:
It’s like teaching a baby that this is an apple and then you keep showing the baby different types of apples and at different angles and sizes. Eventually, the baby will be able to tell you its an apple and know that it’s meant to be eaten. Just like when an autonomous vehicle sees a “Stop” sign and will know it’s time to stop. This data can then be analyzed to see if the car stopped, if it kept driving, or what else can be improved.
He concludes by noting that this analysis makes the product, the vehicle in this case, safer to use. The analysis “helps product managers understand what is lacking and what is good about the current version. Data is the current and can continue to be harnessed in order to build better products for all.”
Data and strategy together are a force to validate product value hypotheses and go forward
This brings me back to the direct product value gained by the consumer with products like Impactoria and Loio.
Although the way Impactoria analyzes data is different from how Loio does it, there is a common thread in both startups’ approaches: they use data analysis to achieve better outcomes for their end-users and, as a result, are validated in their respective strategies.
Whether they assess user analytics to improve their products or their own products learn the most optimal way to add value through applying ML and artificial intelligence techniques, the conclusion is ignoring data wouldn’t make the products better. In fact, ignoring data could make it worse or even fail.
Instead, by listening to the data, they are listening to the market.
Making your product fit the market is a solid strategy.
Because then it won’t be just, like, your opinion, man.
This is a personal blog. The views and opinions expressed in this article are those of the author and do not represent those of people, institutions or organizations that the author may or may not be associated with in a professional or personal capacity. All information is provided on an as-is basis.
