“Two roads diverged in a wood, and I— I took the one less traveled by, And that has made all the difference.” - Robert Frost Robert Frost laments that he may choose only one path to travel, and makes the decision based solely on his limited knowledge about which path has been traveled less. Thankfully, as a User Experience team, we not only get to create the paths we send our users down, but also get to observe how many users choose each one. When used intelligently, this leads us to insights and opportunities for improvements at each step along the way. One method we use to control and monitor our users’ traffic is A/B testing. In an A/B test, we create a new version of an existing page within a flow in an attempt to see if we can improve the outcome. This outcome we are monitoring is generally referred to as a “conversion.” A new design is built with a hypothesis in mind which should theoretically produce more conversions. Examples of conversions could be things like selling more widgets, attracting more clicks, or increasing the percentage of signups per visit. The UX/UI team at ZirMed recently used A/B testing within our patient portal, where millions of transactions account for millions of dollars in payments each year, meaning we have more than enough traffic for an effective A/B path analysis. Within our portal, patients have the option to either pay individual statements, or to sign up for a full Patient Notebook account where they can view and pay future statements online, communicate with their doctors’ offices, and update their information in a single location. At the end of the “pay single statement” flow, we give users the opportunity to sign up for a full account. This particular A/B test was designed to optimize the number of users who sign up for a full account on the Receipt page after completing the payment flow. We created a variation of the Receipt page to test whether text saying “You’re almost done” followed by a “Continue” button would outperform the current page which featured a visually attractive banner and the message to “Sign Up for Patient Notebook”. See the screenshots below: Current Variation A very important part of A/B testing is that only one element of the flow was changed. This is so any difference in the conversion rate can be directly attributed to the variation. Each step before the receipt, and each step of the signup flow after the receipt, went unchanged to ensure that we were measuring the effectiveness of the changed element and nothing else. We designed the experiment to randomly distribute traffic to each design equally. Over the course of thousands of visits, random distribution negates the influence of outside factors such as gender, time of day, amount owed on statement, or any other factor not considered part of the test. With our variation set up, and our traffic being distributed evenly, patterns began to emerge: We collected the data into a chart which shows how many users we saw (sessions), and how many conversions each path generated. This experiment gathered data from over 5,000 visits to gather data. The path with the original banner led 2.9% of users to sign up for Patient Notebook. The path with the text-based variation, on the other hand, generated almost 3 times as many, at 8.1%! Seeing such a dramatic difference, we could be confident that directing all traffic toward the text-based variation would yield more signups. It’s important to note that A/B tests reveal how many users perform a certain action from a specific design, but not why. We can theorize reasons for the difference, but without talking to users we can’t really be sure. And ultimately, the underlying reason behind the effectiveness of a design is secondary to the quantitative fact that it just IS more effective at converting clients. To answer those deeper “why” questions, we use qualitative analysis techniques such as taskbases assessment and contextual inquiries, which the UX team leverages on site with users on an ongoing basis as well. Wherever there is an opportunity for a more effective design, there is an opportunity for my team to create a diverging path and monitor the results. Robert Frost’s decision to take the path less traveled hints at the possibility of missed opportunities due to his ability to take only a single path, but by using modern technology and data-driven decisionmaking, ZirMed is able to make sure that the path most traveled is the right one every time.
© Copyright 2018