Optimization - More Than a Means to an End?

Over the past year, I have written numerous articles that hopefully have either helped your new Optimization program(s) hit the ground running properly or, in the case of existing programs, take it to the next level and magnify your impact and value to the broader organization. These have included topics such as Increasing the Value of Your Optimization Program, Building an Optimization Roadmap, and Quantifying the Impact of Your Optimization Program.

As an industry, let’s take a step back for a moment. The one word in common in the articles mentioned above is OPTIMIZATION. But what does Optimization even mean? Many senior leaders equate Optimization to CRO (Conversion Rate Optimization). In other words, if it doesn’t impact conversion rates, then it’s just not that important. Stated another way, everything we do in our Optimization program must have a direct impact on conversion rates and/or revenue. Others say Optimization is just A/B testing. Run a few tests here and there on the site, and bam! You have an Optimization program.

Clearly, these definitions don’t smell right.  Not only that, they also narrow the scope and potential impact that a first in class Optimization program can have in an organization.

Let’s look at Optimization from a different angle – job performance. How do you measure the success of your Director of Optimization? And how does he or she measure the team’s performance? Is it the overall conversion rate increase on the site? The number of tests run on the site? Nailing down a solid understanding of what optimization means will make your job much easier when it comes to measuring the impact your Director is having on the organization. It will help your employees as they would be very clear on what their goals are and how they will be measured against those goals.  

And I think you may be surprised at some of those measurement traits.

Now, let’s take another step back, and crack a few dictionaries open (well, at least the online ones lol). Let’s break down the word Optimization apart from our industry, so that we don’t limit the scope of what it really means. Then we can bring it back into play and see how that may change the scope and potential impact of your Optimization program and how you measure performance of your employees.

Here’s the definition of “optimize” from Merriam Webster:

I really like this definition here, because it forces you to pause and think a bit more strategically about what your Director of Optimization and her team should really be doing. What are they trying to perfect or make more effective? The entire web site? Just the conversion funnel (think about the CRO statement above about Optimization being the same as CRO). Perfect only certain pages?

This also begs the question, is Optimization a means to an end or an end in and of itself? Is it good enough to optimize your “null search results” page on your site simply for the sake of making it as functional as possible for your end user? Or are you doing it to improve conversions? Does the former automatically drive the latter? Does your senior leadership ask about conversion and revenue impact for every test and discard the test if there was none?

Here's another definition from Britannica:

It’s fairly similar to first one but uses the word “good” instead of “perfect” which I appreciate a bit more. The other thing that came to mind here is that if you are trying to make something as good as possible, wouldn’t it be important to know the purpose of that “something” is?

A great exercise to recommend to your Director of Optimization is to have her team go through each page category on the site and define the purpose of each page. This information may already be available from product owners and such in the organization. If not, here’s a format that can be used to help get your team started.

I think your Optimization Director and team will learn two important things:

1.     Not every page can or should be optimized towards order rate and revenue

2.     It is possible to Optimize the entire purchase funnel starting at the entry point and moving page type by page type towards the end of the funnel

For example, the purpose of your product detail page could be having the user add the product to their cart. If the user does that, then wasn’t the page optimized for that particular user? What if the user changes his mind the next day due to a sale from a different web site? Does that mean your product page was not optimized? There are many down funnel things that can happen that your team needs to take into account when developing the proper KPI to measure the success of a test and call a statistically significant winner.

EACH PAGE TYPE IN THE SHOPPING PROCESS SHOULD BE OPTIMIZED PROPERLY

Here's another question for you to think about. Let’s say you run an A/B test on your Product Detail Page. The test variant increased Add to Cart rate 5% as compared to the control group. Conversion rate and revenue impact was inconclusive, meaning you can’t determine either way if this test has an impact on those down funnel metrics.

Did you successfully optimize this page? It’s such an important question. Here’s why – I have sat in countless executive meetings where a test such as this one was discussed. And often times I would hear, “That’s great, but if it didn’t impact conversion rates or revenue, then let’s move on.” Such a devasting answer – and wrong answer – to this test. In the truest sense of the definition, your Optimization team did its job perfectly!

HIGH LEVEL TAKE AWAYS 

·      Agree and communicate on the definition of Optimization

·      Reinforce that definition when presenting results to senior leadership

·      Measure your Director of Optimization and his or her team based upon that definition

·      Use the “smell test” -  i.e., it’s hard to optimize for revenue on a page that is too far upstream from the purchase

This article has more questions than answers, and that is by design. For our industry to continue to evolve, I argue that Optimization is more than a means to an end. It should be approached as the end itself, for whichever page or process on your site you are trying to improve.  We need to continue to question what we are doing, and how we are doing it in order to stay relevant and the value to our organizations and end users.

My hope is that these questions will trigger useful discussions with your Director of Optimization. And then in turn, you are able to take great strides with your program this year and add even more value to your organization that you already are doing. You should also be able to measure the performance of your Director and his or her Optimization team directly to the foundation of what Optimization means.

Happy Testing!!!

 Jason