Friday, November 14, 2014

50/50 Traffic Routing Methodology for Google Site AB Testing

The purpose of this article is to document one method for AB Testing in a Google Site.

One of the joys of working in a large organization is having to UX within the organization’s parameters or constraints. In my case, we have several teams making use of Google Sites as part of our internal communications networks. The usages vary from sharing team calendars, install schedules, processes and procedures, to displaying link farms and quick information. The UX team that I’m a part of is one that serves the business both internally (backstage user interfaces and processes) and externally (company websites, mobile apps, etc…)

Providing UX research methods to internal teams using Google Sites can be challenging. Even more challenging is figuring out how to conduct specifically an AB test on a frequently used page that lives on an internal Google site.

Ok, so before we get going too far… What is AB Testing? Wikipedia describes AB testing as “jargon for a randomized experiment with two variants, A and B, which are the control and treatment in the controlled experiment”... Yikes! That was a bit “science-y”. Basically, an AB test is where you have an original page design and a new or several new page designs that you want to put in competition with each other to see which outperforms the rest. Seems basic enough right?

As I began climbing down this rabbit hole, I discovered something great… Google Analytics has a method for AB testing built into it called ‘Content Experiments’. where you have the capability to have Google automatically equally serve out your page variations. This is fantastic news!!!… for everyone who is wanting to do AB testing on their non-Google Site web pages.

The problem I ran into was that in order to set up a google experiment, I needed to place a block of JavaScript into the head of the originating web page (the “A page"). Google Site pages do not expose the “<head>” tag in any of the website management menus. Now what? Maybe I can use the “Google Website Optimizer”? Nope… It’s been decommissioned. DANG!

At this point, I have a hypothesis that users of a particular Google Site are spending more time than necessary searching for forms that are most useful to them. I also suspect that some users are giving up before they even get to the forms page.

If Google isn't going to provide me with what I need, then I must come up with a workaround. A new method is born...

50/50 Traffic Routing for Google Site AB Testing
50/50 Traffic Routing for AB Testing is not an exact science. it’s more of a hit-and-miss-hope-for-the-best kind of effort. One of the challenges in split testing is trying to be as objective as possible. What we want is to throw the same number of users at one design as we do the other design/s, and then after a period of time, compare the numbers (such as drop rates and time on page).

Getting an even spread of traffic going to the different page design variations is the challenge. Based on my own web research, it seems as though the most popular way to determine how best to direct traffic relies on tracking ip address, storing information in databases, or in some instances even more intrusive measures (i.e. fingerprints, blood samples, dental records, just, kidding, etc...). Some solutions relied on a “random” mechanism that would send people to one design or another randomly. That seemed a bit unintentional for my liking. For appropriate AB testing, there needs to be a bit more purpose, or control.

The solution used in the 50/50 Traffic Routing for Google Site AB Testing is to serve up a variation based on the date. There are 365 days in a year (“odd days” and “even days”). If a person were to send all the traffic to one design one day, all the traffic to another design a different day by way of a "Redirect page", and then alternate back and forth as the calendar marches on, they would then accomplish a pretty even spread.

Like I said… Not an exact science, but a decent effort.

Putting it together
After understanding what I was up against, and then putting some thought into potential solutions, it was time to put it all together. I will now attempt to describe the process I went through to accomplish this AB testing.

Step 1 - Ecosystem
It’s always good to understand the ecosystem that you are about to “pimp” before you take actions to begin “pimping” it. In my case, the Google Site that I was focusing on was only accessible to internal employees. The site is accessible from an employee's home by simply logging into their work profile, but again… an authorized person must be logged in, meaning that I can’t really make use of an outside testing company’s slick plugin or web based solution. My solution must be able to work within this constraint… and it does.

In order to employ this methodology, you’ll need to gain the necessary Permissions to be able to add/edit/delete site pages “at will”. Luckily, my wife Katie runs the site that I’m working on, so getting this access was a breeze.

Step 2 - Get Google Analytics Running
There are a lot of analytics solutions out there, but when it comes to Google Sites, I’m going to stand firm on the belief that Google Analytics is best in terms of ease-of-use and integration. I recommend that if this is your first time setting up Google Analytics with a google site, that you follow the instructions in this Google Support article:

Step 3 - Get your links
At this point, I decided that I was only going to come up with one page design variation. Since I know that I’ll need to put the link to the “B page” in some code later, I decided that this was a good time to get the page ready to be used. Since I don’t really know what the “B page” will look like yet, the easiest thing I could think of to do in order to get a link to the page was to simply go to the page that I will be focusing on and make a copy.

After the copy was made, I went into the sharing options for my new page and captured the hyperlink to my clipboard. I then pasted it into a blank notepad as a way to save it for later use. Since you have the sharing options open, it’s a good idea to verify that all the permissions match what is configured for the “A page” design. They should match by default, but you never can be too sure with Google.

Step 4 - Prime the pumps
If you think you will have a large number of people visiting your “A page” and “B page” during your AB Testing, maybe it’d be best to allow Analytics to run just on the “A” page for a few weeks or months to validate that theory. This is what I am having to do with my test. Without this valuable up-front intelligence, designing a useful “B page” would be heavily reliant on Genius Design Guerrilla Warfare.

Typically quantitative tests, such as split tests are really valuable when considering raw numerical statistics. The more users using a page, the better the result of the test.

Step 5 - Get your redirect set up
At the heart of the 50/50 Traffic Routing method is a simple “Redirect” page. In other words, we’ll need to have a custom HTML page placed somewhere that will push part of the traffic one way and part of the traffic a different way.

In my case, I worked with my intranet team buddies to have them place my "Redirect page" in the intranet (which is only available to internal employees at our company). Here’s the html/JavaScript code that I used. This "Redirect page" can be made easily using notepad. Open a blank notepad, type all the text in the screenshot, and then save it as "redirect.html". Replace the text that reads "{replace this with URL from step 3 for 'A' page}" and "{replace this with URL from step 3 for 'B'page}" with the urls for those pages.

Once the redirect has been built, place it in the process flow so that the link to the “A page” is replaced with a link to the redirect page. Once this change is live and in place, the redirect will begin to use the logic within it to determine which page to serve up. At this point in time, my “A page” and “B page” were identical in appearance.

ALERT! Since I originally posted this, I discovered that people who wanted to access this section of the site from home were unable to… There was too high of security on the redirect page because it was housed within the Intranet. I ended up setting up my redirect to be served out publicly from my personal Google Drive. Here’s what I did to accomplish that:

Step 6 - UCD Process
As is best practice, I try my best to adhere to the user-centered design process which is made up of three main phases: 1) research phase, 2) lofi design phase, and 3) hifi design phase.

Research phase -
As described above, it was necessary for me to allow Google Analytics run for a period of time. Because my efforts were part of a training exercise, it was not in my teams budget to put a full Research effort towards a new design.

Lofi design phase -
During the lofi design phase, I created a number of possible sketches for solving the problem described in my hypothesis above. After iteration, I decided on one that I thought would be best. During this phase, I focused on the environment constraints, existing design usability focal points, and traffic flow patterns.

Hifi design phase -
The hifi design phase is where I take my design and build it out on the existing “B page”. I make special effort to avoid laziness when it comes to styling and spacing. In this phase, my goal is simply to come up with a design that beats the old. Lastly, save the design.

All the ducks are in a row now. All the cogs are in place. It’s now time to run the AB test.

Step 7 - Run the AB Test
Once I completed placing the new “B page” into position, I set up some calendar events. To get that somewhat even 50/50 spread of traffic and to have some valuable information, I thought it would be best to run the tests for at least 2 weeks. 1 month would have been better, but honestly this is just a training exercise, so I opted to cut the testing scope down.

Step 8 - Debrief
At the end of the settled upon date range for running your AB test, it is time to “harvest those crops!”. Capture all the stats that occurred during range and begin analyzing. In comparing the two page variations, it should be clear which is the winner depending on your logistics goals. Adjust the link from the start page that was pointing to the redirect so that it is now pointing at the winner.

All Done! Good work team!!!

No comments:

Post a Comment

Yes, please comment.