Let’s have an honest conversation about conversion.
My evolving relationship with conversion
I’ve had a problem with conversion since 2018.
This problem stems from hundreds, if not thousands, of optimized A/B tests and a product culture that really (let’s be honest with ourselves) only cares about conversion.
Conversion is a maddening yet exhilarating treadmill; it keeps you hooked in search of the next improvement, and the pursuit never ends.
Before we get too far, let’s define conversion in an e-commerce product context:
The product’s effectiveness at getting the customer to take a desired action.
There are a few elements to this:
Effectiveness: This indicates how well the experience works. If it’s highly effective, more customers will want to take the desired action.
Desired action: The action the business wants the customer to take, usually to make a purchase.
Now that we’re on the same page, let’s return to my problem with conversion.
I’ve worked hard to change Expedia’s conversion culture and ultimately failed.
Here’s my story.
In 2018, I was finding my footing as a product manager. I was getting the hang of A/B testing and understanding the established cultural guardrails.
The unspoken rule was that conversion is king and we should measure and optimize for it in all our tests.
I disagreed because I believed we should focus on improving the experience, not getting more people to make purchases.
So I started digging. How can we measure our success differently?
I was initially stumped, but after chatting with a colleague, I discovered micro-metrics.
Micro-metrics provide the closest measurement for the change made.
I was excited to try this for more clarity on the impact of the specific product changes and to achieve a higher A/B test conclusive rate.
As an example, if customers struggle to find lower-priced options, we could increase the visibility of the low-price filters. The metric should be engagement with that filter. This will help us understand if the core problem is truly solved. The purpose of solving this problem is not to increase the conversion rate but to help customers find what they are looking for.
Micro-metrics are useful because they provide the closest measurement to the specific change made.
It’s problematic because filter engagement improvement doesn’t matter to the business. What matters to the business is making money.
Micro-metrics create a Wild West situation for the product where every change is optimized for a different metric. This is an anti-pattern for product strategy that can cause conflicts and confusion for the team and customers.
Micro-metrics are best as secondary metrics. They indicate behavior but do not represent the ultimate behavior we want to change.
We tried micro-metrics for a few months. We improved our A/B test conclusive rate dramatically, which increased the team’s ability to glean learnings. However, the business metrics were lacking.
My assumption going into the micro-metrics endeavor was that conversion was not worth chasing because so often the changes we made had nothing to do with getting customers to make a purchase. I still believe this to be true.
However, there is a naivety to this assumption.
I felt that going after making money was greedy and dirty. And it can be. But the entire premise of a business is to make money. I worked for a business. Making money is what the business cares about and the incentive structure is geared to that purpose.
One job of the product manager is to figure out which behaviors the customer is taking and how those behaviors can be measured in a way that meaningfully ladders up to the business making money. If a product manager can clearly make the connection that an increase in filter engagement leads to revenue, great!
When I started down this route I was early in my young product career and had a very idealistic view of building cool products. This meant I had lost sight of the business. Creating an incredible product with the greatest experience is a worthwhile endeavor and can be done in tandem with making money. And while products exist to solve problems, make no mistake, products being built by companies exist to make money.
I realized this later in my career than I’m proud to admit, but I got there. I still had this weird feeling about conversion. What was it?
I recall a new product leader at Expedia saying that conversion would be the top metric for our organization. In a Q&A, someone asked the leader:
“How will we keep the customer front and center with conversion as our top metric?”
The leader responded:
“I see conversion as a customer-facing metric. If conversion improves, the experience is easier for customers to get what they want.”
I felt alarms going off in my head. It wasn’t the metric itself that troubled me, it was the framing!
Conversion is framed as a customer-facing metric. If the experience improves over time, more customers will want to buy, increasing conversion.
I reject this premise for two reasons:
Conversion is a company-facing efficiency metric
The experience itself ought to be measured differently
Conversion is a company-facing efficiency metric, not customer-facing.
Here’s why.
If a company wants to increase its conversion rate, it will enact various tactics to reduce experience friction and overall customer acquisition costs. The more efficient the experience is at converting customers, the less money the company needs to spend acquiring them.
There is an argument to be made that for existing customers, if the experience has less friction, it is a better customer experience. That may be true. However, reducing friction doesn’t mean the experience is improving; it just means it’s easier to pay the company.
The experience should be measured via a different metric not focused on making the company money. This doesn’t mean the company should avoid talking about making money (more on being honest in a bit…). Financial metrics can be looked at in tandem with customer-experience metrics.
This is where metrics like Net Promoter Score and Ease of Use Scores come into play. These metrics indicate customer opinions on the experience and its improvement over time.
There’s a push and pull between the efficiency metric (conversion) and the experience metrics (NPS, Ease of Use).
My belief is that you need to have efficiency and experience metrics serving as guardrails for one another. If efficiency is hyper-optimized, the experience may suffer and vice-versa.
I’ve realized my problem with conversion isn’t about the metric. It’s about our honesty as a product team and business.
We’re here to make money.
Let’s be honest and call it what it is.
Businesses want to be hyper-efficient to increase revenue and reduce customer acquisition costs. I have no issue with this.
When discussing conversion, let’s focus on what it is, not what it isn’t. Let’s not fool ourselves into thinking conversion is all about the customer.
In 2018, I had a naive approach to conversion. I believed we were chasing the wrong metric.
My current position in 2024 is more nuanced. I’m good with conversion. We’re friends again.
I’ve become more honest with myself about my role as a product manager. It’s to improve the customer experience, yes, but it’s primarily about making money for the company to operate and continue delivering great experiences.
Making money and being efficient aren’t bad. They can feel a bit dirty at times, so there’s a need to connect these to actual human beings. This drives empathy. All good things.
And that empathy should be used to understand how to solve problems. Those problems, though need to be thought of in the context of the business and the business objectives (ie. make money). How can you leverage that customer empathy and create a solution that drives the business objective?
That is the ultimate challenge of a product manager.
Conversion sits at the epicenter of that challenge in e-commerce.
So bask in the challenge.