How do i A/B test search with Google Tag Manager?

At Klevu, often, we are asked if our customers can perform A/B testing to check if the Klevu Search is better than their existing search solution. What the customers usually want to know is how much the Klevu Search is helping them in search led conversions.

Google Tag Manager (aka GTM), is one of the popular tools that allow you to monitor consumer activities on a website, including what they search, click and buy. From the GTM’s perspective, monitoring such activities is a process called setting up “tags” in GTM. For the every tag setup in GTM, Google provides a snippet of code to be included on your website. This code is responsible for sending the required data to Google.

In this document, we provide information on setting up GTM to perform A/B testing for Klevu Search.

It is, basically, a three steps process:

1. Setting up a Google Tag Manager (GTM) account
2. Setting up GTM testing environment (i.e. obtaining snippets of codes to include on your website)
3. Making minor changes to the Klevu Search codebase to enable/disable Klevu Search based on a parameter configured by the GTM code.

Setting up a Google Tag Manager account

1. If you already have a GTM account, please login into it. Otherwise, please follow the steps mentioned here to create a new GTM account.
2. Once the account is created, you need to add a container for your website. The container is just a collection of tags that you like to add to your website (e.g. for measuring analytics, adwords conversions, etc). Please follow the steps as mentioned here to create a new container for your website.

Setting up an A/B Test in GTM

1. Login into your GTM account
2. Choose the respective account for your website and the container
3. Click on the tags > Add a new tag > Custom HTML Tag
   
   
4. Under the "Configure Tag", paste the following code and click on the “Continue” button.
5. Under the “Fire On” section, click on the “More” > “NEW” button
6. A new window with the title "Create Trigger" will open. Here, under the "Choose Event" section, please click on the "Custom Event" button.
   
   
7. Set the event name to call_gtmtesting.
8. Click on the "filters" button and configure the condition as: event — equals — call_gtmtesting.
9. Click on the "Create Trigger" and “Save Trigger” buttons.
10. Click on the "Create Tag" button.
    
    
11. Click on the “Save Tag” button.
12. At this stage, please go to your google analytics account (i.e. https://analytics.google.com) and find out the google analytics code for your website. The code should look like UA-XXXXXXXX-X. You will need this in the next steps.
13. Once you have the Google Analytics code, now is the time to configure parameters for your A/B test. Please go to http://abtestguide.com/gtmtesting/createtest.htmland fill up the details as described below.

    Test Details Test ID: eg: AB-Klevu01 Test name: Klevu Search AB Test Control Description: Klevu Search Enabled Code: var klevu_search_disable=false; Variation B Description: Klevu Search Disabled Code: var klevu_search_disable=true; Integrations Analytics: Select the option “Universal Analytics” Property ID: here, provide your analytics code, the one you had copied from your google analytics account (e.g. UA - XXXXXXXX-XX). Event Category: AB Test Targeting This section allows you to distribute your traffic between Klevu and the other search module(s) you have on your website. Exposure Rate: 100% Traffic Control: 50% (Klevu search will be activated to 50% of your customers) Traffic Variation B: 50% (The other search option on your website will be activated to rest of the 50% of customers)

14. Click on the Generate AB test Code button. This will generate a code at the bottom of the page. You need to copy this code and generate another tag in your GTM manager.
    
    
15. Add another custom tag in your GTM account (i.e. Click on the tags > Add a new tag > Custom HTML Tag) and name it “GMT Testing”.
16. Copy the code from step 14 and paste it in the HTML area of “Configure Tag”.
17. Under the section “Fire On”, click on “More” > “New” > “Custom Event” and select the “Page View” option.
18. Under the “Configure Trigger”, select the “DOM Ready” option.
19. Under the “Fire On” section, select the “All Page Views” option and click on the “Create Trigger” button.
    
    
20. Click on the “Create Tag” button.
    
    
21. Make sure to publish your container by clicking on the “Publish” button at the top right corner of your page. This will make the test live on your website.

Changes in the Klevu’s code base

If you have followed all the steps so far, when you view the source of your website (i.e. right click anywhere on the page > view source), you will see that for some sessions, GTM would insert the following javascript variable on the page. var klevu_search_disabled=false; 

For the other session, it would be
var klevu_search_disabled=true;

You need to ensure that you add the following if statement surrounding the Klevu’s javascript.

You’re done!

Any questions or need any help? Please feel free to write to us at support@klevu.com