The perils of making CX and CRO changes without valid customer insight
You wouldn’t buy a used car without first checking the tyres, electrics, fluid levels and much more. And you’d probably want a test drive unless you are one of life’s risk takers.
But in the world of ecommerce and digital product design, some companies make decisions that can have huge impact on the customer experience (CX), conversions and the bottom line without making basic checks and running tests for possible unintended outcomes.
You may as well throw a dart at a target blindfolded and expect to hit a bullseye.
Even the world’s most successful companies make missteps by following intuition instead of data. Take Google’s short sightedness (pardon the pun) regarding Google Glass. The product lacked genuine user-centred design and a mass market use-case. If the company had run more tests with bigger focus groups and listened to customer feedback it might not have had such a failure on its hands.
Snap stumbled with a specific UX design failure in its 2018 update. A redesign changed the app interface to create a divide between friend’s content and branded content was heavily criticised as confusing. The backlash, including a stinging single tweet from Snap user Kylie Jenner, significantly impacted the UX and led to a drop in user engagement and stock price.
A more recent example is the CrowdStrike Tech outage, caused by a faulty software update and estimated to have cost Fortune 500 companies $5.4 billion. A regular update process would usually include rigorous testing to catch bugs, experimenting on various types of machines and taking the route of small, incremental roll outs rather than the massive blast out from Crowdstrike.
These instances emphasise how dangerous it is to make significant changes to websites and the granular user experience (UX) without careful checks and valid customer insight. Any business making big changes to their site without testing for evidence is stepping through a one-way door. It’ll be costly or even impossible to return to your start point. A two-way door decision is built on pre-tested possible outcomes and robust data and is much easier to step back through and reverse quickly.
5 Guidelines to staying on the straight and narrow
For anyone with commercial, CX and Conversion Rate Optimisation (CRO) responsibilities, there are guidelines to help avoid damaging debacles.
· Always A/B test any changes to the user experience to validate assumptions and make sure your decisions are based on data. A/B testing keeps your work customer-centric, without which the risk to user engagement and conversion rates is high.
· Don’t be dissuaded from testing due to perceived complexities or a lack of understanding – there are partners who can help set up and run tests or just provide advice to get you up and running.
· Measure the impact of incremental changes at each step so you understand how these small changes helped optimise conversion rates. Being able to show improvements, no matter how small, also helps build the business case for more CRO investment as well.
· Make sure there is a user feedback loop in place, so you can quickly understand how any changes have been received and what the criticisms are (if any).
· Avoid costly disruption and downtime by refusing the ’big bang’ approach to website redesign. It might have a ‘wow’ factor but it’s much safer to have a strategy of iterative improvements through constant testing and optimisation.
Airbnb: An Inspiring experimenter
There are many excellent business examples to both inspire best practice and illustrate how a curious, experiment-focused mindset can drive growth.
Airbnb is the poster child for embracing a culture of test and learn and iterative development from launch to deliver a customer-focused service. Among its first hires was a data scientist and the company embeds data scientists within product teams.
Decisions on changes to the UX and customer service are uniformly based on data-driven insights, while the company insists on continuous testing as a bedrock principle and regularly puts elements of its platform through checks to ensure optimal CX improvements. User feedback on any changes is highly valued and incorporated into the process of continuous refinement.
Former Airbnb data scientist Lindsay Pettingill summed up the mindset when he said: “I find the strongest teams operate with data at the outset – they don't just call over the data scientist when it's time to do a post-facto analysis of how XYZ experiment went. Some of the best product ideas we came up with on Airbnb's early growth team came from data scientists. At one point the Growth team started bi-weekly experiment reviews where product team members would present on experiments they ran, with a focus on lessons learned, mistakes and next steps. Knowledge sharing was off the charts."
We know speed to market is essential for staying competitive and you want to introduce new products and features at pace but the risks of implementing any unproven hypothesis can send your brand spinning off the road and into the ditch.
For further reading check out our blog What is A/B Testing And Why You Need To Do It and if you’ve recently been given CRO and UX responsibilities we highly recommend reading our report Fast Track Your CX Success: A 100 Day Plan For New Ecommerce Leaders [where can they find it?].
And if you want advice on CRO best practices or have a specific project in mind then do get in touch.
