How to increase conversion
Taught by Peep Laja of ConversionXL
Master Class: Conversion through Research
Andrew: This session is about how research can increase your conversions. The session is lead by Peep Laja. He is the founder of Markitekt, a firm that helps companies make more money through conversion optimization. He also famously blogs about conversion optimization on his site, ConversionXL. Peep, thanks for teaching us here.Peep: Thanks for having me.Andrew: I want to set up what we’re going to learn by showing an example of what can be done. That example comes from a site that you were working with, that was actually doing okay. You said to me before we started recording, “Okay is actually a bigger problem than poorly done or badly done with big mistakes.” Why is okay a problem?Peep: If you have a site that sucks [??] terrible, the design is amateur and it has all of these problems, it’s easy to find and fix those problems. You can just look at the site and say, “Hey, this sucks, that sucks, and this sucks.” and it’s not that hard. What is hard is taking a good site already making money, with no obvious no-brainers, and then improving it. That is hard work.
Andrew: What did you to do to this site that allowed you to grow their sales like this? Let’s bring up the numbers. I know this is a lot of data for people to see on their screen, but the big picture is that users went down but the conversion rate increased significantly. It’s a 45% conversion rate increase. Revenue shot up about 30% and so on. Can you give me an overview of how you were able to get it? Then in this session we’re going to go step-by-step over the process that you’ve taken, and that others who are watching us can take. What did you do to get the numbers that we looked at?
Peep: We used a multi-step process. We knew this site must have problems, we just don’t know what they are. We need to be data driven, figure out what we think is wrong with it, then see if our hunches are true. We have quantitative data to evaluate and reevaluate our hypothesis once we have a list of maybe a hundred or two hundred items wrong with this site. There were so many problems that we identified. For instance, the site was not responsive even though the mobile traffic was like 20%. Mobile experience was terrible. We decided to completely redesign the site using the data. We kept what was working and we changed what was not working. The end result was 30% more transactions with less traffic.
Andrew: Peep, you did one before we started, that most people want to just do a complete redesign. They look at their site, they learn a whole lot, and they realize, “My site is junky, I missed out on all these things that everyone else has learned over the last few months and years. I’m going to redesign it using everything I learned.” You said, “Don’t do that!” Why not?
Peep: Most radical redesign is a bad idea. The reason is that if you change everything at once, inevitably some things will get better, but some things will get worse. They cancel each other out and you don’t know what’s what. What made it better? What made it worse? You see it all of the time in news, where a major e commerce site has a million dollar redesign, but revenue per visitor is down 30%. They then roll back to the old site after spending all that money and all the time. A better alternative is to just improve your site one specific item at a time through proper scientific…
Andrew: Is that what you did on this site whose data we were looking at?
Peep: No. In their case, improving the site by one item at a time would have taken a year. That’s too slow. Since there were so many severe problems we decided to redesign everything. However, we did not redesign it based on clients personal preferences or designers creative ideas. Actually, we did not even include a designer until we had full wire frames done, based on the research data, based on talking to the users, and based on analytics data. Then we finally brought in the designer and said, “here are the fully detailed wire frames. Make it look awesome.”
Andrew: We’re going to show how that process plays out and how other people can use it. Can you say publicly whose stats that these are that we see up on my screen now?
Peep: It’s a national e commerce site called National Allergy.
Andrew: National Allergy. Let’s go over to the big board and take a look at the big steps that we’re going to be taking to give our audience this kind of result. And the first one is, well you’re saying, “Look, your opinion doesn’t matter.” But you also, Peep, are saying that you’re opinion doesn’t matter either. And in fact,
Andrew: As much experience as you had doing this, you keep getting surprised by what works and what doesn’t. Let’s take a look at a version…
Peep: All the time.
Andrew: …of your site.
Andrew: This is your site. I clearly, taking a look at this, can say, “This makes sense. I could see why conversions would be high.” You’ve got the box asking for an email address. You have the “share” buttons. You have clear description of what’s coming up and talk to me about why you thought this would work.
Peep: When I published the book on Amazon, and I did some research about the name, for the name of the book and I named the book “How to Build Web Sites That Sell.”
Peep: And I thought, “Hey since, you know, I did this study on the name of the book, I’ll use this same name here ’cause I know it works.” But of course, typically you want to have, you know, some more copies, some bullet points with benefits, some sub headline, an [??] offer, show social proof with a light button and of course…
Peep: …I want to have…
Andrew: So you did add that. I see bullet points right there in the center with those green check marks next to them. Yep.
Peep: Yep and then on the side, I had the tweets and the likes and all that stuff, so people would not feel that they’re the only idiots using this site.
Peep: And I said, “Ah, that’s great!” But then I tested it. Because when I looked at the data of what people do on this page, the conversion rate was about five per cent for that opt in box which I wanted to be more. And when I looked at the scroll map, actually a lot of people were not really scrolling for further down and nobody was clicking on the social shares.
So, if it’s not motivation, it’s friction. That’s what I thought. So, how about I get rid of the social share? How about I get rid of most of the copy? Make the page narrower, you know, less tall. And I…
Andrew: [??] work with that?
Peep: I had a version before where I even had a “latest blog article excerpt here”, which I also removed. And then, you can see that’s the next iteration and this doubled the conversions on my home page. So, right now it’s converting at around ten per cent. So you only see a tag line, head line, and an offer. That’s it.
Andrew: All right, and you keep wanting us to think this way. It’s not that we should remove Facebook buttons. It’s, that’s not the big message here. The big message is, “You don’t know and what you think doesn’t matter, it’s only a matter of testing.” And everything.
Peep: Right. [??] I have run so many A/B tests and when I have to predict which A/B test, which variation will win, I get it right maybe 60, 70 per cent of time, which is only slightly better than flipping a coin.
Andrew: And what’s shocking to me about that is that you are the guy that other people turn to when they want to learn about optimization, and when they want to hire somebody…
Andrew: …to fix their sites conversion and if you can’t do it, there’s no nope for the rest of us. We have to test, yes.
Peep: So the point is, the conversion optimization is not a library of layouts that always work. If you think conversion optimization is tactics, “Do this, this works. Don’t do this, this never works.”
You’re wrong. Conversion optimization is a repeatable, systematic process. So, it’s about knowing how to use the process that gets you the results, not about what works. So like a site like “WhichTestWon”…
Peep: Their test library is useless for your purposes because what worked for that other guy will not work for you.
Andrew: All right, let’s go back to the big board here and the next big point is to, “Conduct experience based assessments of your site.”
Andrew: And what you mean by this is, the experience, tell me if I’m wrong. The experience that a user is likely to go through, you want us to walk through it like a tourist visiting a foreign city and look around and look for what?
Peep: So, a client brings me in to start working on a new Web site.
Peep: And this assessment is the very first thing that I do, always. I walk through the site as their end user, while knowing what is their business objective. So let’s say it’s an e-commerce sight.
So, on the home page, I want to see, you know, “Where am I? What can I do here? Why should I do it?” You know, is it there? And also, the goal of the home page is to get people off the home page.
So, I’m assessing the content of the home page for clarity. Value, like are they communicating a value proposition. Distraction, like if you have these sliders, you know, carousels changing every two seconds. That’s distracting. I will not pay attention to anything else except for the blinking. . .
Andrew: I’m waiting to see what the next slide is going to be.
Moving backgrounds or stuff like that.
Can we take a look at a site through your eyes? Here’s one. This is one of your clients. When you’re going through this with your tourist eyes so to speak what are you looking for?
Peep: First of all I need to know what is it we want to do? What is our business objective? In this case it’s obviously to book a hotel room. We want people to use that widget to find something.
Also a secondary goal is to get people to become Marriott rewards members because rewards members convert many times higher than regular visitors. It’s six seven times more. They don’t do comparison shopping. They are a really profitable segment.
We want to have more rewards members. Knowing these two things I’m starting to go through this page. First thing clarity. Where am I, what can I do here, why should I do it. Well it’s Marriott so it’s different. Most people know Marriott. Maybe it’s OK. There’s no valuable proposition here.
However, Marriott is also not cheap. Why book through Marriott? That information is not here. No, my saying that does not mean that is the truth and they need the text here. What I do here is I write down, “they are not communicating a compelling value proposition.”
They are not giving me a reason to book my stay with Marriott. This is what I call an area of interest. Now I want to go and seek data to either validate or invalidate this claim that they need to communicate the value proposition. I can go test it right away.
Of course I don’t know what the right value proposition should be. I can generate random ideas but random ideas hardly work. That’s one. Number two, looking at the page again I see in the top right corner it says, “Not a member? Join now.” Member of what?
Andrew: Right. Our people going look at that and understand that means join the Marriott rewards program. Frankly are they even going to see that at all. You’re saying. . .
Peep: I see it and why would I click it? Why would I want to join? The same thing as the log-in bar underneath the image they also have join now.
Andrew: We’re looking at right here.
Peep: Nothing is done here to increase my motivation to take action.
Andrew: There it’s hard to see.
Peep: Is it that becoming a rewards member saves 20% off of your next stay? Well that would have value right?
Peep: Well that huge background image there a lovely hotel I assume. That is the most visually dominant thing on this page. We want people to search for a hotel yet the picture is bigger. Is it a distraction? It’s possibly a distraction. I’m going to start to look at the orange couches. I write down, “the image is a possible distraction.”
Maybe if the search was bigger it would dominate over other things and more people would perform a search.
Andrew: Okay. It’s just going through and saying What is their goal primary and secondary? What are the potential obstacles to getting to that goal? You just look at it with fresh eyes and see not what they are but what you think they might be. You don’t walk away saying that little button is obviously a problem we should make it bigger. You walk away saying here’s an issue that we can examine later with data.
Peep: I’m identifying areas of interest. That review of the page is also very systematic. It’s not about this sucks and this sucks it’s about clarity, value, motivation, distraction and friction. If we want people to action (a) we want to increase their motivation for doing that and we want to reduce friction meaning make it easier to take action.
It’s a systematic process and it’s best with multiple people, not just you because it’s kind of a brainstorming thing. More eyes see more things.
Andrew: All right. Let’s go back to the big board here. Next thing we want to be aware of is to gather qualitative data using surveys. I’ve got one of your surveys here. Here it is. Let me open it up. Bringing back our web browser here.
Peep: The survey is a great way to get data for some of our findings. Excuse me. So for instance we want people to book hotel rooms. What is it that matters to them the most when it comes to booking hotel rooms? Is it how beautiful the room is? Is it the price? Is it something else? We don’t know. So we want to find out what are the buttons that we can push to get them to book more hotel space or what’s keeping them? A large portion of people come to the page and they leave without booking the room. Why? What’s holding them back? So again with the survey we can figure out what are some of those obstacles.
Andrew: Okay. And do you want us to just put a list of obstacles on maybe in radio boxes on a list so people can select the ones that matter the most to them?
Peep: You would think it would be so easy. The problem with that is that you don’t know what the problem is so if you create a bunch of radio boxes, buttons, or check boxes, you are not opening yourself up to new possible problems that they might be experiencing. So I always do only open-ended questions. Voice of the customer. So when I ask the question, “What made you almost not book the hotel room?” I want to see what they write. Or “What matters to you for booking a hotel room, when you’re booking a hotel room? What do you pay attention to?” I want to see their specific wording because that wording I can use as copy writing on the page itself. And if I will ask a stupid . . .
Andrew: All right. So let’s, let’s do. . . Oh, sorry. I was just going to say let’s take a look at it on one of your surveys.
Peep: Yeah. Yeah.
Andrew: I’m opening up a browser again. What system do you use to create these surveys for Marriott?
Peep: This is Typeform. So mostly I use Typeform just because they’re, it’s beautiful.
Andrew: Typeform, for anyone who’s out there who wants to use it. And so we hit enter or click the start button and I can see it says, “Tell us about yourself.” I’m zooming in a bit. Then “What made you choose Marriott.com to book your hotel room? What matters . . . ”
Peep: So this survey goes out to people who recently booked the hotel stay. So these are actual paying customers. So we know that this is our target audience because they actually spend money. So we survey them when they still freshly remember their booking experience but before they actually stay at the hotel so their hotel stay experience would not cloud the website experience.
Andrew: Ah. Okay. And I see here, “What hesitations did you have before finalizing your booking on Marriott? Did you have any room booking questions that you couldn’t find answers to on the site?” Do people actually fill out surveys like this?
Peep: A typical response, well we always incentivize, so everybody who fills out the survey gets something.
Andrew: I see.
Peep: So what it is depends on the client. It can be money. Money can be of course expensive. It’s easier to give coupons or you know, something cheap but valuable.
Andrew: I see, that doesn’t cost money. . .
Peep: And typically we get a response. . .
Andrew: . . . but does have a big value for the audience.
Peep: Exactly right. So for many companies they can give away like free eBooks or downloadables or free month of their software, something that doesn’t cost a huge amount of money but is valuable to the user. So we always incentivize people. We want to get a minimum of 100 responses. Otherwise, you know, the sample size is not big enough and one voice can become too dominant. We also do not want more than 250 because if you have more than 250 open ended responses it takes huge amount of time to go through them to and to analyze the responses. And the insight, the extra insight that you get is not really there. I mean it’s minimal. So we want 100 to 250 responses. Usually we need, if we want 100 responses we need to e-mail at least 500 people.
Andrew: That’s not bad, 20%.
Peep: Something like that, yeah.
Andrew: But I guess they’re fresh. . .
Peep: So it’s not like your regular e-mail list where it has a bunch of freeloaders on it. These are people who paid you money. So they have a relationship with you. So they’re, you know, they feel closer to your brand. So 10-20% response rate is typical.
Andrew: All right. Let’s go back to the big board because there’s so many other ways to learn from our customers and to research what they’re looking for and what they can’t find.
Peep: With qualitative data, for instance, we had a client called [??] where we wanted to find out what is the main problem why they are not buying stuff and analytics data showed us huge drop of rates, we couldn’t figure out why, and there selling pool parts, and we learned that Peep: it’s mostly old men, you know renaissance men, do it yourself, and B: the main reason for not buying a pool part was because they didn’t know what was the right part for their pool. Major insight for you know for using, for improving conversion. So you can get such high quality valuable insight out of qualitative surveys that you can’t get from analytics data.
Andrew: You know what? I can see how someone who’s a do-it-yourselfer would be blocked if he can’t figure out what the tool or what the part is the right one for his pool, and it also maybe makes him feel a little bit weak because he came into this process thinking that he can fix the pool and now he can’t even figure out if it’s the right part for the pool.
Peep: Exactly, what kind of a pool pump you know and all that stuff yeah.
Andrew: And Google Analytics wouldn’t tell you. Alright I can see why you’d survey.
On to the Big Board to the next point that I was going to say, which is to review live chat transcripts. Here’s what that looks like on one of your partner sites, one of your client sites excuse me, this is National Allergy. It’s obviously just that box that pops up, we’ve seen it a million times when we go to different sites, it asks users, “Do you have any issues? Do you have any questions?” And these boxes save the chat programs, the chat conversations.
Pep: The exact live chat software does not matter because they’re all pretty much the same. So what matters is that people are usually using live chat to ask pre-sales questions. Like they have a lot of specific questions that’s keeping them from buying something, so those questions are highly valuable. I also like to just talk to support people, but with live chat transcripts I usually get, you know depending on the volume, I want to read through the last 30 day’s worth of chats, and sometimes I also take the live chat transcript and put it in a tool like Wordle, Wordle.net I believe it is, and it just highlights the bigger, the most common words and it gives insight.
So I want to understand with the live chat transcripts what are the most common questions and doubts. So for this analogy one of the most common questions was, “Does this specific, particular product alleviate my specific allergy that I have?” Some are allergic to dust mites, some are allergic to bee pollen, maybe they have asthma you know whatever, whatever the condition is they care about their specific condition.
So once we learn that, when we understood that that’s…that helps people make a purchasing decision, once they know that, “Yes this cures my pain,” or, “Alleviates my problem,” we did two things. Peep: for each product, or most products, where applicable we added on the product, onto the product page we added, “This relieves allergy from dust mites,” for instance, and at the same time we added a thing on to the main menu. Do you have a screenshot of…?
Andrew: Yeah actually, here let’s take a look in the browser, bring up my browser right now. Here is, I was doing the search for it, this is National Allergy, I just looked at all cotton mite proof pillows and of course it says, “Dust mites,” here and you’re clear about what it’s there for, but you also said that there’s some other section, this one.
Pep: So yeah underneath the logo, yeah, there’s a drop down, and it’s automatically open once you go to the homepage, and so here people can just choose their specific condition, and this was when we redesigned it we added it, it wasn’t there before. So we were also like, “Hey are people going to actually use it? Will it make a difference?” And now what we know, because we have analytics data on it, that yes people use it, not as many as we would maybe like. I think that right now the number of all traffic on the site 15% of the traffic uses that functionality, but those who do convert four or five times better than those people who don’t use it. So it makes a huge difference.
So now we’re like, “Yes our hypothesis was correct,” because we had that insight from a live chat transcript.
Now our next task is that okay we know this functionality works not the questions, “How can I get more people to use it?” And this you know requires AB Testing, different ideas…
Andrew: Was the open drop down menu was it from the start that you added it on?
Andrew: You did, so you said, “All right we can’t just hide it we have to both highlight by putting in a different color,” here’s what it looks like…let’s bring that up, and, “we have to highlight it by putting in a different color, and open up that menu on the homepage because it’s so important.”
Peep: You know a funny thing, we also split tested an orange version, so instead of the light blue we had orange. It really [stirred] up much more, but it actually lowered conversions. Revenue went down. We were like, “What? Is our hypothesis is incorrect?” Then we noticed that actually the amount of people using this widget did not increase. It stayed exactly the same. What it impacted negatively was people who did not use that feature because it was a distraction.
Andrew: I see.
Peep: People who stopped paying attention to the products, the categories, and the stuff, they just looked and was like, “Ah, this annoying orange thing.”
Andrew: Strange how little differences like that have so much impact. Onto the next point which is you suggesting we call customers that match our ideal customer profile. Now you don’t necessarily mean our real customers, just the ones that match our profile. As a result of doing that, here’s a before and after of how National Allergy site looked. Let me zoom in again. This is what it was before the change, and this is what is after. It’s easy to say, “Well, it’s prettier after.” But you want us to notice something else here. What are we looking at?
Peep: We called up these people. These are mostly middle-age and old-school people. That part of National Allergies marketing is that they send this paper catalog to doctor’s offices. If somebody has an allergy condition they go see the doctor and the doctor says, “Hey, you need this allergy relief product from this catalog.” They give them this specific product to buy. We called up a bunch of their customers. I think we interviewed 15, if I’m not mistaken. We had like twenty 30 minute talks about how they would like to buy, and how they buy allergy relief products in general. The biggest competitor for National Allergy is Amazon, because that’s their go-to source, to by anything really. We wanted to understand…
Andrew: Why do people even come to National Allergy when there’s an Amazon?
Peep: Exactly right. We wanted to figure it out. What we found out is that when the doctor told them to buy something, they felt some sort of anxiety that they needed to get the specific thing that the doctor told them to get. When we were asking them, “Hey, did you consider buying the same stuff from Amazon because maybe it seemed cheaper there?” They said, “Well no, the doctor told me this National Allergy is a professional medical website, not like Amazon.” That was interesting. We started to probe that with other people that we interviewed, about this doctor authority, professional, medical, and serious. That inspired the new version of the homepage, where you see the doctor.
Also, it’s where you see the copy there. Underneath the headline it says, “Thousands of doctors are recommending National Allergy to their patients.” We’re using this doctor thing as something that sets National Allergy apart from Amazon and other competitors. This is a serious medical site. If you want an allergy relief solution that actually works, you come to National Allergy.
Andrew: The only way that you were able to find it out was by talking to customers and asking them, “Why would you use a site like this? What troubles did you have with it?”
Peep: Exactly right. Why would you prefer National Allergy to other sites?
Andrew: Should we not talk to our current customers? You’re saying talk to people who are like our ideal customers?
Peep: You want to talk to both. You want to talk to people who actually bought from you, because you want to understand why they bought from you and not the other guy. What made them think that your site is the right site? You want to understand those reasons. So when we interviewed people, ten of the people were actually customers. They had actually purchased products. If you’re talking to ideal customers, nobody’s more ideal than somebody who paid you money.
Andrew: I see. So it’s not that you’re saying, “Talk to people who are like your ideal customer.” You’re saying, “Talk to your customers who match your ideal customer avatar, or talk to others who haven’t bought from you who should be, because they match your ideal customer.” And through that you’ll see why people buy and why they don’t buy. That’s how you ended up with a doctor on the homepage. And with a site that emphasizes the trustworthiness of this site that has medical connections, as opposed to emphasizing free, here this is what you used to emphasize, what they used to emphasize, free shipping on over $50 and the diversity of their collection.
Andrew: Right so that…
Peep: So when you have a site like this, which is like most ecommerce sites, they sell products that also other ecommerce sites sell. Like everyone sells Samsung TV’s right? So why should somebody buy from you? So you have two options really, either to be better or to be different and it’s very hard to be better, and it’s not that hard to be different.
Andrew: All right let’s go back to the big board, the next one is to recruit 10 to 15 people who match our target audience and you’re suggesting that we use a site, this is where user testing.com comes in for you am I right?
Peep: Exactly, so UserTesting.com is just one of the options so you don’t have to use it and you can also do user testing in your office. You get people to your office, sit them down at the computer, because the main idea is that you have your target audience use your site, or the site in question, and comment everything out loud, whatever they’re saying, they’re doing, and you want to give them scenarios, and you want to give them three types of scenarios. A, you want to give them a broad task. So a broad task would be find a pair of pants you like. I’m not telling them what they should click on, and so on, it’s very broad. So then I’m observing how they go about doing it and then commenting everything they say and do.
The second type of task you give them is very highly specific task. So let’s say, “Find dark jeans, Hugo Boss brand, Size 34, under $50,” highly specific, and I want to see how they go about. Are they successful, are they frustrated, is it easy to find something that is highly specific?
And the third type of task is just buy it. So you basically go through the checkout process. I actually take out a credit card and type in the numbers…
Andrew: I see.
Peep: …and buy something.
Andrew: And so we can either do it ourselves by having somebody walk into our office, sit down or sit down at a coffee shop, and watch them but you want us to have them do specific things and talk out loud what they’re thinking as they’re doing it so we can see what’s going on in their minds.
Peep: Exactly right.
Andrew: And if you use…
Peep: And you want to give them just tasks and you don’t want to tell them what to do, you don’t want to ask questions, “So do you think this page is secure?” If they’re not bringing up security it’s not an issue right? So we had a case with a client, Marriott.com, and after surveying Marriott customers and we wanted to understand use case, “Why are you going to stay in Marriott?” A number one use case, “I’m traveling for business, I’m attending a conference, I’m visiting an office in a city,” a different city and usually so they have a specific destination, like a specific reason.
So we created a task for our user test, a scenario, “You’re attending a conference at the United Nations Headquarters in Manhattan, New York, please find two hotels that are closest to that location.” And then we just observed how people go about that task, and so what people did was that they figured out what was the address for United Nations, something-something United Nations Way, or something, and so they typed in the exact address for Marriott.com and once they clicked search the search results come from, came from Abu Dhabi United Arab Emirates.
Andrew: Let’s do that actually. Wait, here we go, bringing up Marriott’s site. If I copy the address, I’ll zoom in here so we can kind of show it. This is what it looks like when you type it in. Then hit find. This time it didn’t do it, weird. We tested it before, but it does show here United Arab Emirates is the location as opposed to United States even though, I think I did it, I’m 90% sure I did it. Let’s do it right here, I pasted it 760 United Nations Plaza, New York, New York, hit and then New York, New York, zip code 10017, and then comma United States, hit find…This time it’s not showing anything but the problem is the same one that we noticed earlier when we tested this, which is it’s asking for, it’s looking in the United Arab Emirates and so you wouldn’t have known that unless you watched someone struggle with it?
Peep: Right, their competitor, Hilton, does not have this problem, it’s easy to find, you can just punch in the street address, and Marriott has this functionality in their advanced search settings somewhere, but this is a problem that we found through user testing. We have, of course, nobody find a client Marriott about it, but you know how it goes in huge organizations change is slow.
Andrew: How long have you been with them?
Peep: It’s about insight.
Andrew: How long have you been with Marriott? Are they a new client?
Peep: We started, we went through this whole conversion research process with them in August and September, and then we…
Andrew: Okay so just a few months ago.
Peep: A few months ago, exactly, and it’s a pretty good site. We actually struggled to find things wrong with it, but you know if you gather a bunch of data and analyze the data you’ll find stuff. So we found 75 issues, some more severe than others, and they implemented some of them, some are being tested, some are things are not implemented, not really up to me but just today I saw an email, or yesterday, in the inbox where they say, “Hey we want to do another round because we changed a bunch of stuff so let’s do more analysis here.” So they were happy with that.
Andrew: The cool thing is this, I had to struggle to find, the first time I searched on Marriott I found the exact same page that we just showed. The second time it was gone because they were AB Testing and I was seeing one of their variations, and so we do see even as you and I are talking and looking at their site that they’re constantly testing. The other thing I noticed is you said that they do how much in sales on their site, roughly?
Peep: Their online volume is $7 billion.
Andrew: So even a small percent, you were saying, a fraction of 1% could be a huge lift for them?
Peep: My god yes. You know I had a chat with their, one of their conversion managers and they said a year ago they installed a Java Script snippet onto their site that slowed the page down enough that the conversions dropped by 0.25% percent and that cost a loss of like $100 million.
Peep: So scale.
Andrew: Scary and exciting too, because if you could give them even a small lift that you’ve way more than made up for the cost that they’re spending on your firm.
Andrew: Alright, the pen ultimate point is use web analytics to find out which pages are costing you the most, and there was a company that worked with truckers that had an issue that you uncovered this way.
Andrew: What did you do?
Peep: Well with Google Analytics the main thing that you need to find is where are you leaking money. So if you have an ecommerce site you know if people get to the category page are they then, how many are proceeding to the product page? You know maybe that’s where the flow is stuck, or maybe they get to their product page but they’re not adding to the cart, or all your checkout funnel. So you need to find out where are they dropping out and then you figure out why, and again be also specific pages that are causing the problem.
So yeah I had a client, I still do, that is kind of a job service for truck drivers and they had a five step resume creation process, so if you wanted to apply for a trucking job you needed to fill out a form that was on five pages, five step form, huge, huge. So in Google Analytics we saw that in step three massive amounts of people are dropping out. They had filled out two pages worth of information already so they must have been invested, but page three they’re dropping out. We look at the form on page three can’t find a problem. We were suspecting technical problems so we ran cross browser testing, nothing, everything was working.
So at the same time we were using a heat mapping tool that also records user sessions, so we were watching user session replay videos. You can use tools like Inspectlet, or Session Cam, or Click Tail, and so we were, we took three, four hours, and just watched a bunch of videos of users that had dropped out in step three.
So what we noticed was that, first of all there’s a question about your criminal history, and we discovered that a lot of the people have criminal records, and you know many had more than one thing, assault and whatnot, and then they get to a question that says, “Do you have references?” And the form needs three mandatory references.
Andrew: Is what it looked like on the form.
Peep: So nothing complicated just name, relationship, phone number, right? But these people didn’t have references; maybe because they had a criminal history, maybe because they were fired from their previous job, we don’t know that, that we don’t know. But what we found out was that personal reference that they didn’t want to fill out either because they didn’t want to or they didn’t have any. So we told the client “Hey, this is what we found, this is an insight. We think this personal references are a problem.” And the client said, “We can get rid of it. Well, talking about getting rid of stuff, this five step form, you know process is ridiculous.”
And of course you know he’s saying “Well we can’t change it, because this is the requirements of trucking companies, their HR wants to see all of that information,” and so back and forth with the client, and say “Well, why don’t we just get the initial lead in, somebody interested in the job, and then we get the rest of the information later on either through phone or send them an additional form. So we through discussions we ended up with something completely different down the line where we actually instead of a five page funnel, we have a six field form.
Andrew: Here it is.
Peep: So it seems like a no-brainer; yes less form fields, higher conversions, right like everybody knows that [??] talks about it, but the hard thing about this form fields is, it is the business requirement. So if all the data is needed to apply for the job; so how do you remove form fields, because you need the data? So you need to work you know, change something on the back, and change on how you do business or so.
Andrew: And in this case it’s a short form, because it meant they also had to follow-up with phone calls, so it looks like in some ways your costs are going up. But you only understood that, because you understood their overall business.
Peep: Exactly right, yeah.
Andrew: Yeah, so it’s not as easy as just get rid of the fields; life will be good. Get rid of the fields but then you also have to spend some money to follow-up with people, but I guess as you looked at it, you realized it was worth it, but what a simpler process as I look at it.
Peep: And you know, form fields, sometimes you want to add form fields. You want to add friction to the process for instance my website, my agency website Markitekt.com we had so many leads coming in, and most were unqualified, could not afford us. So I spent sometimes 30 minutes on the phone, 45 minutes on the phone-call with somebody and we get to the budget finally, and they say “Well I have $800.00” “What? My call with you cost more than that.”
So how do we, because you don’t want to start a conversation with money, because you want to build value first. So a simple change with it, we added a drop-down box with budget options where the lowest budget was I think $5,000.00 or something. So that immediately, overnight eliminated all unqualified leads, because people self qualify.
Andrew: Yeah, here’s what it looks like now, right here. Oh wow, less than a quarter million is one and, so do you take people who spend less than a quarter million? Oh sorry, this is annual revenue. Do you take people who have less than a quarter million in revenue?
Peep: It depends, sometimes we do, and we have multiple forms here, so we have a design form where we actually ask about your budget, or if you want monthly optimization then we’ll ask you about your online revenue, because basically we want to understand if we increase your revenue five percent per month, how much is five percent worth to you? So I want to have my clients get three-four-five times return on what they pay me.
Andrew: All right, final point is, now that we’ve got all these issues, you want us to list them out, categorize them, and rank them. Why do we need to make this list?
Peep: So once you go through all this qualitative-quantitative research process, you will end up with a huge list of issues. So a typical website I have 30 pages full of issues, so now I mean where do we start from? What’s more important and you know, how should we tackle these issues? So first thing you want to do is you want to categorize all the issues.
Some of those issues are instrumentation issues, for instance instrumentation means that something that needs to be measured with analytics, it’s either not being measured. A typical problem let’s say on E-Commerce site is that they’re not measuring at the cart clicks, they’re measuring visits to cart page which is not the same thing. So the issue might be at the cart clicks are not recorded in Google Analytics, that’s an instrumentation issue. Second category is test
So we found an obvious problem with an obvious solution, but you know we know that my opinion does not matter so we need to test it not just change it, so we’ll put in a testing category. Sometimes we know there’s an obvious problem. For instance, people don’t know which part is the right part for their pool right? It’s a problem, but what is the solution to the problem? Now there is no obvious solution there could be many ways to solve the problem, so that type of problem we put into hypothesize category, obvious problem not obvious solution.
Peep: I see.
Andrew: A fourth kind of category is just effing do it. So a no brainer, so you’re font size is eight pixels light gray on gray background, you know people can’t read it. No need to test it just make your font size bigger and darker, right increase contrast. So just do it and the fifth type of issue is investigate.
So we have identified an issue, for instance, conversion rate for Internet Explorer 10 is only to 20% compared to Internet Explorer 11 and 9, why? We don’t know, so there might be some cross browser compatibility issues so a developer needs to go and investigate. So that’s investigate category.
So once we have categorized these issues we know who to send it to. All the instrumentation issues we’ll send it to an analytics guy, or developer. All the test and hypothesize things we’ll send to our optimization team. All the just do it things goes to whoever you know changing stuff on the website, and so you know how to organize the work, and now even if your categories don’t fit into five categories.
Still you might have 30 things in one category, well where do you start? So you want to prioritize each issue from, we usually use five to one ranking, so five means it’s a severe problem costing you money and a huge amount of your traffic is exposed to this problem. So for instance, in Marriott the address search, you know the search is the most prevalent thing, huge amount of visitors exposed to the problem so we’ll probably rank it around four, and also then up to one.
One is the minor usability issue that should get fixed eventually, not a high priority. So an example could be that people on your about page are not finding your, I don’t know, your address. You know maybe they don’t, not too many people have that use case but it should be fixed eventually you know?
So once you have that, and then you should have all these issues in kind of a spreadsheet, we use just Google Doc’s.
Exactly right. So we have an issue and what the issue is, the bucket which is the category, back on why is this an issue, action which is what should we do about the issue, and rating, and you can also add another column here for responsible. So Google Analytics Bounce Rate Info is wrong that’s an instrumentation issue. Well, Susie in the Analytics Department can fix that, or we are not recording free-trial downloads well we should send it to Linda; she’s taking care of all the event tracking for whatever downloads right? So we can add specific people and now we have an action plan.
So when I start, when I have a conversation with start-ups who want to do testing, but don’t know where to begin there number one question is, “What do I test, I don’t know?” Well now you have 30 pages full of stuff to do, you have no question about what to test, but once you go through this research process you, you know you can be confident in your way forward.
Peep: All right and the way to do it is to conduct those experience based assessments to gather those qualitative pieces through surveys, I’m look at the list here. In fact, I might as well just show it up on the screen, and there it is. Look at the chat transcripts, call your customers, get people who are in your target audience, look at analytics, and then once you have that list prioritize, assign, and then get to work.
This has been a real meaty conversation here, but I still want more. Because, there’s something about conversion, especially when it’s explained right by someone who’s done a lot, and I don’t know many people in the world who’ve done more than you and have researched it as well as you do. It’s just fascinating to see what you should do and how much those little tweaks can impact the business. Your business, if anyone wants to check it out, it’s Markitekt.com, but I’m going to suggest that they check out the blog first. They can register, they can read your articles, and you’ve got them going back, if I remember right, when did you start 2011?
Andrew: 2011 yeah.
Peep: 2011, and they’re really interesting articles here. It’s so great to have you on here. Is this the best place that I should be sending people to sign up right here with your site?
Andrew: That is right yeah. Conversion XL you know if I may say so myself it is the best conversion blog out there. When I started it I thought everything else is kind of mediocre out there so we work really hard to create awesome content.
Yeah and you keep it really interesting here. That’s ConversionXL.com. Peep, thank you so much for doing this session with me here today.
Peep: Thank you for having me, Andrew.
Andrew: You bet. Thank you all for being a part of it.