Evan Schuman's StorefrontBacktalk

Best Buy Does OK, Costco Tanks. When Keynote Systems started looking at the mobile sites from major retailers late last year, the veteran mobile and Web site test and measurement firm knew that these sites would be a lot slower than their wired Web counterparts. But some at Keynote were caught off-guard by just how slow some of the major retailers’ mobile sites were..

To put this difference into context, Keynote argues that a wired Web site should, on average, be able to deliver a page—especially the site’s homepage—within two seconds. For mobile, Keynote said, users should tolerate sites that are about twice as slow, or about 4 seconds on average. In its examination of 10 major E-tail sites—Amazon, Barnes & Noble, Best Buy, Costco, Dell, Foot Locker, Musician’s Friend, Sears, Target and Walmart—the very fastest site (Best Buy) averaged more than twice Keynote’s acceptable slow estimate, crawling in at 8.3 seconds. Again, that was the fastest mobile site. The slowest site delivered its average page in 34 seconds.

When the company tested mobile search, it found similar slowpokes. Wired Web searches can often take a small fraction of a second. The fastest mobile search response Keynote found was from Walmart, at 4.5 seconds, while the slowest of the sites came in at 38 seconds.

Ken Harker, a Keynote senior manager for mobile and Internet technologies, said the differences between the mobile sites of the strongest and the weakest of these 10 sites were many, but they mostly revolved around how seriously that chain took mobile.

Harker said the biggest factor, logically enough, is how much data the retailer makes the user download before the page can be displayed. Best Buy, for example, Harker said, only had three elements on its mobile homepage. “It’s stripped down,” he said.

Compare that to the slowest site, which had an impressive 70 elements on its homepage, including 40 images, Flash and “more than 10” JavaScript elements. “They’re not really making an effort to trim down the site for the mobile market. Images, JavaScript, stylesheets, HTML elements—it adds up very quickly. They’re not doing any optimization for mobile customers at all,” Harker said, adding that Best Buy made extensive use of cache, which the slowest site did not do.

Keynote officials—including Harker—steadfastly refused to identify which site was the slowest. That said, points made by Keynote while discussing the study pretty much eliminated all of the tested retailers other than Costco from being candidates for the slowest performing site. Of the 10 retailers named, Keynote said those that topped any category did not do especially poorly in any of the other categories. That rules out Best Buy (which came in number one in homepage performance), Walmart (number one in search), Foot Locker (number one in product information performance), Sears (#2 in homepage delivery), Barnes & Noble (#2 in both homepage delivery and product information delivery) and Amazon (#3 in search results).

That accounts for six sites, leaving four: Costco, Dell, Musician’s Friend and Target. And Target, Musician’s Friend and Dell all have homepages that clearly have far fewer than 40 images. That leaves only Costco, which has far more images on its homepage. This seems to concur with a report from Citi that identified Costco as one of the less sophisticated retailers when it comes to using technology.

In earlier studies on mobile performance and wired Web performance, some sites that fared poorly defended their position, saying that it’s neither fair nor rational to put all sites on equal footing. For example, they say, a site that is pitching multimedia for a very young audience may need to have more graphics, more Flash animation, and more movement and activity than what some might consider necessary for a more utilitarian site such as a hardware store.

Although that is certainly a legitimate position, the flip side is also true. Wouldn’t it be the case that those attention-deficient, easily bored shoppers are likely to be the first ones to abandon your site if it takes too long to load? A better mobile approach might be to offer quickly served minimalistic homepages—so shoppers get on the site quickly—and then offer explicitly described links for multimedia. That way, customers’ waiting expectations are properly set, but only after they are safely inside your mobile store.