Some of the most successful web-scale companies of the past decade have developed advanced experimentation platforms in-house to make better decisions with data (see Google, Netflix, Airbnb, Uber, Twitter, Spotify, Pinterest, DoorDash). Experimentation may be the clearest way for data science teams to drive bottom-line impact (i.e., revenue). Why do so many companies build their own experimentation stacks instead of leveraging an off-the-shelf solution?
We spoke with data and experimentation teams across startups and larger companies and found that there are (at least generally) six major components of a modern experimentation stack:
In 2020, after validating that experimentation was a revenue driver for many web-scale companies with strong executive support, our investment team set out to research the systems that existed for high-growth companies to support experimentation cultures. We found that the off-the-shelf options lacked analytical rigor for deep product experimentation or were overly complex to integrate. A few other interesting things we found:
Along the way, we met Chetan Sharma. Che spent his career building experimentation tooling at B2C and B2B companies like Airbnb and Webflow, experiencing firsthand the impact that data-driven culture has on product development. When we met, he was developing a vision around making experimentation accessible to any company – so that any person could test new product ideas (after all, Eppo stands for “Every Paid Person’s Opinion”). Among his previous work, he created the widely popular Airbnb Knowledge Repo. He’s committed not only to helping companies run experiments to make product decisions but also to helping teams build and access institutional knowledge gleaned from experiments. In many ways, Che is an archetypical Amplify founder — he encountered a problem, designed a solution, and wanted to make that solution accessible to the rest of the world.
Over the course of several months, we got to know Che more and were particularly drawn to his approach targeting two largely unaddressed areas: the statistical engine and analysis UI. From our prior research in this space, we believed this was the most unsolved part of the experimentation stack and required someone with deep statistical know-how to execute. When Che vocalized that he was ready to start a company, we jumped at the opportunity to partner with him.
Che started Eppo in late 2020 and quickly assembled a team of all-stars with experience at companies like Strava, Slack, and Snowflake. Historically, top-notch, high-frequency experimentation was only reserved for companies who could afford to build systems in-house and staff an internal team to support the infrastructure. Eppo flips this narrative. The Eppo team designed an experimentation platform that allows companies to connect directly to their data warehouse and leverage a statistical engine and analytics UI to handle advanced use cases (e.g. guardrail metrics, automate power analysis, CUPED). With Eppo you get:
Since its founding, they have brought on a wide range of customers with different data volumes and use cases including Cameo, Foxtrot, and Netlify. Eppo is the tool that every company should use in their experimentation stack – which is why we’re thrilled to announce our seed and Series A investment in the company!