MaxBLOG

Forrester has published its first overview of the online multivariate testing vendor landscape. Their analysis shows a high level of maturity in the market with testing being a must-have for online marketers.

John Lovett’s paper opens, "Online testing is a critical component of an efficient marketing optimization solution in the burgeoning field of web site optimization"

Forrester surveyed client-side businesses and give a wealth of insight into both the biggest benefits of testing and some of the challenges faced. Of the marketers surveyed, 48% felt that the increase in conversion rate they had achieved was the top benefit of testing. A significant number also cited customer satisfaction as having improved. Reduction in the risk and guesswork associated with making changes online isn’t covered in the report but is a recurring theme in the discussions we have with our clients.

Within the vendor comparisons, Maxymiser is identified by Forrester as one of only two companies offering a full suite of methodologies and service levels. What’s more, we appear as the firm with the most people dedicated to testing worldwide. Some larger firms for whom testing is only a small element of overall business declined to give figures. Our sole focus at Maxymiser is helping our clients to proactively manage conversion rate and we’re proud of that and the results they have achieved.

We were happy to see that managed service offerings, comprising test strategy, implementation support and analysis receive praise. Forrester notes that clients engaging for such services gain greater uplift from their engagements which mirrors our own experience at the coal face.

"While evidence shows that testing can result in improvements regardless of the method employed, organizations that leverage a managed service model achieve greater gains. On average those gains are two percentage points higher, which could equate to millions of dollars for conversion based web sites" Forrester, October 2009

This increased effectiveness as a result of leveraging external expertise in testing is something that is mirrored in our own day to day work with marketers. Our experience is that marketers who combine external expertise and a testing tool to create a conversion management strategy rather than adopting a tool and driving it in-house benefit from 30% more uplift. Marketers we work with also tell us that adding test design, creative and deployment skills to the managed service model for testing to create a full service conversion management offer creates yet more benefit by enabling them to get the right tests live and in a timely manner.

At our recent client seminar, both Chris Dalrymple of ASDA Wal-Mart and Martin Smith of easyCar commented that they benefit from leveraging external expertise. A full service allows them to get more of the right tests done and to conduct testing without placing high demands on busy marketing and technical teams.

If you are yet to include testing in your online marketing strategy, Forrester’s Online Testing Vendor Landscape report gives a great level of insight into how marketers are deploying multivariate testing, their motivations and some of the challenges they face.