trbo Insights – Jan’s Favorite Feature

In our blog series, we asked our colleagues what their favorite feature in the trbo tool is. After we got fascinating insights on click-in channels from Jihad and Jennifer showed us the advantages of a countdown, Jan now shares his favorite feature: testing.

When you think about a typical day at work, which customer requests give you the most pleasure?

When a customer asks for testing, I’m on the trigger and happy to tackle the project together with them. It takes some preparation to set up a test properly, but in the end testing delivers insights that have a significant impact on the performance of the online store and can provide the customer with measurable added value. Each form of insight is one more step that brings me closer to the goal of improving performance.

Why do you particularly like to use trbo Testing? 

Our testing options make it very easy to run a test. We can set up a campaign, generate a zero group for it with just one click and then get measurable results within a very short time. In parallel, tested variants can be viewed quickly and in a user-friendly way, and the results can be clearly evaluated. Testing simplifies the decision-making process on the question: ‘Which measures are preferred by my target group on my website’? It takes the decision out of speculating on measurements and backs them up with empirically measurable data instead. trbo testing therefore directly helps to customize your website to the exact needs of your customers.

Can you explain the testing function in more detail?

Testing in general describes the process of comparing different measures on a website and showing different variants to various users. From the measured KPIs of the individual user groups, successful and less successful actions can then be defined. These can be, for example, the modification and positioning of elements, the replacement of certain wording or user-based product recommendations. Each form of change can be predefined. Which action or changes work more successfully for a website can be tested and evaluated against each other. So before my customer guesses what works better for their users, I prefer to set up a test with them and empirically evaluate which variant works best for the overall performance of the website in terms of predefined KPIs. We distinguish between the simple A/B test, in which a variant is tested against a zero group, the multivariant test, in which several measures are tested against each other, and the multi-armed bandit test, in which the distribution of tested variants is already dynamically adjusted to user behavior, depending on which variant performs better.

At which point in the user journey is testing best suited and why?

Testing is basically well suited at any point on a website. The entry points on a website differ fundamentally and depend on the industry and the product. I therefore try to pick up each user at their initial touchpoint in the best possible way. Basically, tests throughout the site add value and tell us about user preferences. Therefore, it is particularly important to ensure an easy user journey for the customer close to the point of purchase. Product detail pages (PDPs) and the checkout thus give the greatest scope for various tests. Uplifts are particularly crucial here and therefore have a direct impact on the performance of a website. However, you should not test too much at the same time. Only this way you will get clear results that do not influence each other.

An online retailer wants to find out how a user’s buying behavior changes as a result of product recommendations on different sites. What would you advise them to do before starting the test? 

Product recommendations are a popular way to increase the conversion rate. In order to find out at which point a recommendation influences the user’s buying behavior, this should be tested in advance. Before setting up each test, it is therefore important to think about what exactly is to be tested. Depending on this, the setup of the individual variants is adjusted. Only if a test setup is clearly defined in advance can we evaluate it properly afterwards. 

→ Definition of KPIs: Which KPIs are weighted more severely in the evaluation, clicks, conversions or impressions? These KPIs must be identified and tracked properly in all variants. Downstream, it is pointless to interpret KPIs that have not been predefined.

→ How many variants do I analyze? Depending on the number of variants, the interpretation of the results is important in addition to the longer runtime of the test. The more variants there are, the more they may differ within different KPIs (clicks, conversion, average order value). It is important to clearly distinguish between the effects and impacts of the individual variants and to interpret them correctly. Just because a variant ‘performs’ better in terms of a KPI does not necessarily mean that it is also the most user-friendly variant.

→ User allocation: When interpreting the KPIs, a reference value is always provided against which we can measure an improvement or deterioration. Particularly with regard to the analysis of conversion, it is advantageous to use the zero group to measure what has changed in terms of global sales. Since we measure every conversion once a user has received an impression, we rarely know what really led to the purchase completion. We can therefore only interpret conversion changes via the differences to the status quo, i.e. the zero group, and attribute them to a variant. 

→ Keep testing: After the test is before the test. Variants can always be optimized. Once a test has been completed, I would recommend continuing to optimize the successful variant. A follow-up test is then set up for this purpose, which can be expanded even further. 

Why is testing so important in e-commerce? What chances does it open up for online retailers?

Just as in brick-and-mortar retail, where customers are attracted by a pleasant atmosphere, good customer advice, and fair offers, online retailers are constantly striving to offer users a special shopping experience. The best way to achieve this often varies significantly. But testing is one of the most effective ways to find out. You no longer have to guess and decide gut-based what will go down better with your customer, but can be progressively guided along the right path based on the results of the test.