Posts

Showing posts with the label A/B Testing

11 Tips for Improving Customer Experience and Driving Conversions

Image
Struggling to drive conversions?  The issue might be with customer experience. After having worked with several brands, big and small, I can assure you that you don't have to make sweeping changes to drive better results. Many times even small changes and little bit process can lead to happy customers and big impacts. In this post I have complied 11 tips that you can use today. If you need help then don't hesitate to reach out to me. Easy to fill forms  – How many times have you come across a form field where you don’t remember what the field was about?  Many designers/developers use the default text in the form filed as the filed label. Once you tab into that field, the default text is gone and now you can’t figure out what that field was about.  That is a very bad design which will likely cause customer frustration and kill conversions. No more unnecessary form field formatting and validations - Other than Captcha validation, you are likely using form field valida...

2 A/B Testing Lessons Learned from Amazon Video

Image
It is no secret that Amazon is a data driven organization.  The culture of testing is ingrained into pretty much everything Amazon does. However, a company like Amazon also makes mistake as I learned from my recent experience with Amazon Videos. I have two Amazon Fire TV Sticks, one attached to each TV that I have. Both are tied to the same Amazon account.  Using my Amazon Video account, I started watching a movie on one TV, turned it off halfway through and then a day later tried to watch it on the other TV. Guess what? I could not find that movie in any of the obvious menu options. I was expecting it to show up in my stream or at least in the same place where I found it last time.  Nope.  Amazon changed the order of movies on me. It’s not like it added new movies to the lineup and this movie got pushed down. They just reordered the existing movie selection. It appeared to me that some kind of movie display sorting experiments was going on. However, that experiment,...

Email Personalization Not Working? Read This.

Image
I am a big believer in targeting and personalization and have written extensively about it in this blog. However, targeting and personalization is not “set it and forget it” strategy. It needs to be continually tested to make sure that it is working and driving value. Sometimes you have to test and see if “no-personalization” will yield better results than personalization. To make my point, let’s take an example: I get a weekly promotional email from a prominent marketing company. They personalize the email subject line (good so far). Here are few subject lines How [XYZ Company] Should Kickoff New Clients  Ensure that [XYZ Company] delivers great PPC campaign The rules of online branding that [XYZ Company] needs to know Note: Instead of [XYZ Company] they use the name of the company I used to work with. However there is an issue with the way they do personalization. Do you see the issue? The issue is that they use the name of the company I USED TO WORK FOR . I left that c...

Underline the Clickable Text and Link the Pictures

Image
Sometimes there are things that you know are not right and need to be fixed immediately rather than testing and waiting for the result. Such was a case that I recently encountered and the results were amazing. Recently, I was working with a client who sells people expertise (I am purposely vague here because I don’t want to reveal the name of the client). The “Expert” search is the start of the process and then “Expert Search result” page is a second step in the process. On the expert search page, each expert is listed with a small blurb and the name of the expert was linked to the next step of the “checkout process”. When analyzing the “checkout” process we noticed that there were 2 things missing The name of Expert was not underlined – it was linked to the next step in the “checkout” funnel but not underlined. Also, that was the only call to action on that page i.e. to click on the experts name to go to the next step. The picture of the Expert was static and had no link whatsoever...

Significance of Statistically Significant Results in A/B Testing

Image
You are running an A/B test (or multivariate test) but are in a hurry to make a decision to pick the winning page. Should you pick a winner based on initial few days of the data before your tool has actually declared a clear winner? This question comes up quite often during the conversations with the clients. I always warn against such an approach and advice to be patient and get statistically significant results before pulling a plug on an underperforming variation o declaring a winner. I wanted to share few graphs with you to illustrate my point. The following example is from a test that I am currently running. First Two Weeks : If you make a decision without completely getting through the test, you will pick “Yellow” as the winner while declaring “Blue” a loser. Month Later : There does not seem to be a clear winner. Few More Days Later : Seems like “Blue” is trending higher. Should you pick “Blue” now? Finally : We don’t have a clear winner yet. As you can see, picking a w...

Are Form Validations Invalidating Your Conversions?

Image
Form validations are a great way to ensure that the data you collect from your users is clean and in the right format. However form validations can also frequently lead to lost conversions. CAPTCHA is one such type of validation . Some validation put unnecessary formatting burden on your customer/visitor leading them to abandon your forms and go to your competitors. A lot of data formatting can be done via client side JavaScript or backend processing without putting the customer through a lot of pain. I am not suggesting to remove all required data and data formatting requirement, all I am saying is that be if you can handled formatting with scripts/code then do not make your customer do it. Let the users finish the form. Your analytics tool will show you the abandonment rate of your forms. What it won’t tell you is how many of those abandonments were a result of form validations. Most of the analyst/optimization specialist will suggest reducing the number of fields and/or conducting A...

Is CAPTCHA Eating Up Your Conversions?

Image
CAPTCHA acronym for “Completely Automated Public Turing test to tell Computers and Humans Apart” is a way for sites to block spam. According to Wikipedia it is “A CAPTCHA or Captcha (pronounced /ˈkæptʃə/) is a type of challenge-response test used in computing to ensure that the response is not generated by a computer. The process usually involves one computer (a server) asking a user to complete a simple test which the computer is able to generate and grade. Because other computers are unable to solve the CAPTCHA, any user entering a correct solution is presumed to be human. Thus, it is sometimes described as a reverse Turing test, because it is administered by a machine and targeted to a human, in contrast to the standard Turing test that is typically administered by a human and targeted to a machine. A common type of CAPTCHA requires that the user type letters or digits from a distorted image that appears on the screen.” Below is an example of a CAPTCHA on craigslist. Though CAPTCHA ...

Widemile Optimize: A/B and Multivariate Tool

Image
In my post titled “ A/B and Multivariate Testing Landscape ”, I reported on a variety of tools that marketers and analytics professionals were using for A/B and multivariate testing. As I’ve predicted the interest in testing and targeting/segmentation continues to grow. This was also validated by the Web Analytics Association’s “Outlook 2009: Survey Report”. I just got through a good discussion and demo of the newest release of Widemile Optimize with Bob Garcia, VP Business Development for Widemile. I’m impressed with the new release of Widemile Optimization product that he demoed; this is the first Widemile release for corporate users. Widemile had released an Agency Edition of the solution last fall and the new release includes advanced segmentation and a refined user interface. You might be wondering who Widemile is as they don’t have the same brand recognition as Optimost or Omniture Test&Target. Widemile has been around for several years and was one of the vendors listed in ...