Improve conversion rates through research and experimentation.

21 November 2022 min Optimisation

By combining quantitative and qualitative datasets, you can improve customer experience through effective experimentation.

Conversion rate optimisation (CRO) is the process of increasing the percentage of users on a website or mobile app to fulfil a particular business goal. Many CRO techniques and practices are utilised in this field to optimise the user experience and can be categorised into either research or experimentation. 

Typically, the latter plays a heavier role, but research is key to testing and optimising for a holistic data-driven approach.

It’s an iterative process

What does the research and experimentation breakdown look like? For a fluid approach to CRO, an iterative process is required; one that consists of steps to be actioned to validate a hypothesis.

<p><span class="TextRun SCXW65193086 BCX0" data-contrast="auto" id="isPasted" lang="EN-GB"><span class="NormalTextRun SCXW65193086 BCX0">This can be broken down into the following phases:</span></span><span class="EOP SCXW65193086 BCX0" data-ccp-props='{"335559738":240,"335559739":240}'>&nbsp;</span></p><p><strong><span style="color: rgb(241, 85, 35);">1. Observation:</span><span style="color: rgb(0, 0, 0);">&nbsp;</span></strong><span style="color: rgb(0, 0, 0);">W</span><span style="color: rgb(0, 0, 0);">hat is the problem you are trying to solve? What business KPIs are you addressing and across what parts of your website or app?</span></p><p><strong><span style="color: rgb(241, 85, 35);">2. Research:</span><span style="color: rgb(0, 0, 0);">&nbsp;</span></strong><span style="color: rgb(0, 0, 0);">Q</span>uantitative and qualitative analysis to understand the problem and identify opportunities for implementation and experimentation&nbsp;</p><p><strong id="isPasted"><span style="color: rgb(241, 85, 35);">3. Hypothesis:</span><span style="color: rgb(0, 0, 0);">&nbsp;</span></strong><span style="color: rgb(0, 0, 0);">B</span><span style="color: rgb(0, 0, 0);">ased on research, create actionable hypotheses to predict the outcome of the experiment</span></p><p id="isPasted"><strong><span style="color: rgb(254, 82, 11);">4. Experiment:&nbsp;</span></strong><span class="NormalTextRun SCXW173617718 BCX0" id="isPasted">&nbsp;Test the hypothesis in the live environment</span><span class="NormalTextRun SCXW173617718 BCX0">&nbsp;(</span><span class="NormalTextRun SCXW173617718 BCX0">A/B testing</span><span class="NormalTextRun SCXW173617718 BCX0">&nbsp;</span><span class="NormalTextRun SCXW173617718 BCX0">is&nbsp;</span><span class="NormalTextRun SCXW173617718 BCX0">the process&nbsp;</span><span class="NormalTextRun SCXW173617718 BCX0">of comparing two&nbsp;</span><span class="NormalTextRun SCXW173617718 BCX0">or more&nbsp;</span><span class="NormalTextRun SCXW173617718 BCX0">versions of a webpage or app against each other to determine which one performs better</span><span class="NormalTextRun SCXW173617718 BCX0">)</span></p><p><strong><span style="color: rgb(254, 82, 11);">5. Analysis:&nbsp;</span></strong><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" id="isPasted" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">Review experiment data&nbsp;</span></span><span class="TrackedChange SCXW83845737 BCX0"><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">to</span></span></span><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">&nbsp;understand&nbsp;</span></span><span class="TrackedChange SCXW83845737 BCX0"><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">the&nbsp;</span></span></span><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">impact&nbsp;</span></span><span class="TrackedChange SCXW83845737 BCX0"><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">on&nbsp;</span></span></span><span class="TextRun SCXW83845737 BCX0" data-contrast="auto" lang="EN-GB"><span class="NormalTextRun SCXW83845737 BCX0">key metrics</span></span>&nbsp;</p><p><strong id="isPasted"><span style="color: rgb(241, 85, 35);">6. Conclusion:&nbsp;</span></strong>Review next step opportunities; implementation and/or iteration</p>

This can be broken down into the following phases: 

1. Observation: What is the problem you are trying to solve? What business KPIs are you addressing and across what parts of your website or app?

2. Research: Quantitative and qualitative analysis to understand the problem and identify opportunities for implementation and experimentation 

3. Hypothesis: Based on research, create actionable hypotheses to predict the outcome of the experiment

4. Experiment:  Test the hypothesis in the live environment (A/B testing is the process of comparing two or more versions of a webpage or app against each other to determine which one performs better)

5. Analysis: Review experiment data to understand the impact on key metrics 

6. Conclusion: Review next step opportunities; implementation and/or iteration

What are the benefits?

The primary benefit of CRO is the quantifiable return on investment (ROI). 

Running A/B/n tests will give you conclusive statistics on uplifts against a control group. This concrete analysis differs from other business units that don’t allow for an exact measure of success.

CRO program offers additional benefits to quantifiable ROI, such as: 

  • Deeper customer insights: expand and cement your understanding of how your audience behaves through top-to-bottom auditing and experimentation
  • Continuous optimisation: through data-led insights based on research and experimentation, your website and contents can be constantly improved and refined by a structured, iterative process 

  • Real-time results: optimal performance of product launches and marketing campaigns arise from real-time data-led decisions as opposed to pre and post-campaign analysisremoving the element of doubt from your business decisions  

Always do your research

To run an effective experimentation program, you need to understand your audience. i.e., what are the customer’s intentions and how are they flowing through your website? Therefore, a combination of quantitative (what is currently happening) and qualitative (why it’s happening) research is required.

<h2 id="isPasted"><strong><span style="color: rgb(254, 82, 11);">Quantitative research: what is happening?</span></strong></h2><p>Typically, your quantitative research will come through your web analytics platform. Your aim should be to understand lay-of-the-land performance including conversion rates and engagement metrics, overlaid with a variety of dimensions such as channel, device and page performance. From there, you can begin to focus on specific pages and explore funnel performance to identify optimum and underperforming journeys.</p><p><strong><span style="color: rgb(254, 82, 11);">Qualitative research: why is it happening?&nbsp;</span></strong></p><p>Heatmapping will help to bridge the gap between quantitative and qualitative analysis and offer insights into how customers are interacting with your content and their potential friction points. These insights plus session recordings, polls/surveys, user testing, competitor research and further heuristic analysis, will give you a holistic understanding of audience behaviours</p>

Quantitative research: what is happening?

Typically, your quantitative research will come through your web analytics platform. Your aim should be to understand lay-of-the-land performance including conversion rates and engagement metrics, overlaid with a variety of dimensions such as channel, device and page performance. From there, you can begin to focus on specific pages and explore funnel performance to identify optimum and underperforming journeys.

Qualitative research: why is it happening? 

Heatmapping will help to bridge the gap between quantitative and qualitative analysis and offer insights into how customers are interacting with your content and their potential friction points. These insights plus session recordings, polls/surveys, user testing, competitor research and further heuristic analysis, will give you a holistic understanding of audience behaviours

Using your research to inform experimentation

Once you’ve collated your research, you can start forming hypotheses for A/B/n testing. Depending on your hypothesis, you may want to test changes such as messaging, imagery, content order or complete page redesigns.  

For testing with multiple variables, the option exists for multivariate testing if the traffic volume is there. This will reduce the need to run iterative experiments but requires significantly higher volumes of traffic to be successful. 

All experiments should be deployed and monitored through a suitable A/B testing platform and left running until the data has achieved statistical significance - the point at which your result is guaranteed to be a success or not.

Back to the research

Research and analysis are not limited to the start of a CRO program. The iterative nature of CRO means ongoing research, particularly when analysing results.

When analysing A/B/n tests, regardless of whether they were successful or not, consider the following criteria: 

  • Key metrics and conversion rates: detail every key metric relevant to the test, while noting that success to a primary metric might be a detriment to another. Ensure all possibilities are explored, including page flows. 
  • Additional dimensions: a test may be successful overall, but does this success translate to a specific device? How about for new vs returning users? Demographics? Review all dimensions that could be impacted when segmented out to take advantage of further opportunities for building personalised experiences. 

Consider documenting all this information in a report that includes: experiment observation, hypothesis, statistically significant results and next steps. 

In summary

Only through the iterative, ongoing process of research and experimentation can genuine CRO be achieved. When implemented successfully, CRO will optimise the user-journey and improve ROI.  However, changes made to conversion funnels without data-led research can prove disastrous. 

The technical and practical processes involved in performing this research can be daunting if you’ve never done it before, so we’re here to help. Our team of optimisation experts can help you to create a truly effective CRO strategy, so contact us today. 

CRO Experimentation Matrix

Google Optimize is sunsetting on 30 September

Which alternate provider should you go with? The Conversion Rate Optimisation (CRO) Experimentation Matrix allows you to easily compare vendors across the complex landscape.

Is your CRO strategy effective?

Insights & News

Insights 1.

Understanding third-party cookie deprecation and Google's Privacy Sandbox

How do you get ahead of third-party cookie deprecation with Google’s Privacy Sandbox?
Insights 1.

Resolution Digital wins The a2 Milk Company’s website account

The a2 Milk Company (a2MC) has partnered with leading digital-first marketing agency, Resolution Digital as its dedicated global website agency, following a competitive pitch process.
Insights 1.

Resolution Digital wins El Jannah

From feeding hungry locals in Granville during the late 90s, El Jannah has quickly derived a cult following for its legendary Lebanese-Australian charcoal chicken and even more renowned garlic sauce.

Stay informed with our latest insights & news

Our Offices

Sydney.

Gadigal Country
Bay 7, 2 Locomotive Street
South Eveleigh, NSW, 2015

Visit your Sydney office

Melbourne.

Wurundjeri Country
Level 6, 650 Chapel Street
South Yarra, VIC, 3141

Visit your Melbourne office

Brisbane.

Turrbal and Jagera Country
200 Adelaide Street
Brisbane City, QLD, 4000

Visit your Brisbane office
l

Contact Us