E-Commerce Performance Often Checks Out During Checkout

Written by Evan Schuman
July 12th, 2005

In analyzing more than 6,500 data points from the top 19 e-commerce sites, a new Keynote Systems report found sites that were too slow, failed too often and often glitched during the crucial checkout period.

In analyzing more than 6,500 data points from each of 22 top e-commerce sites, a new Keynote Systems report found that many sites were too slow, failed too often and often glitched during the crucial checkout period.

The sites evaluated were, Barnes & Noble, Best Buy,, Circuit City, Costco, Dell, Crate and Barrel, eBay, Eddie Bauer, Gap, JCPenney, L.L. Bean, Macy’s, Nordstrom, Office Depot,, Staples, Target, Wal-Mart, Williams-Sonoma and Yahoo Shopping.

The results didn’t show any one e-commerce site doing well in all areas, but the company did release the companies that fared the best in selected categories.

For example, Office Depot and Staples delivered the best response times, with Office Depot on top for average T1 response time (with Staples coming in second) and Staples on top for average analog dial-up response time (with Office Depot coming in second and Costco coming in third).

For site reliability, Eddie Bauer came out on top, with Barnes & Noble coming in second and Staples third. For overall service level ranking, the top five were Williams-Sonoma, Wal-Mart, Eddie Bauer, Staples, Circuit City and Best Buy.

What struck Ben Rushlo?one of the report’s authors who serves as Keynote’s senior competitive researcher?most intriguing was the huge difference between some of the larger sites and how poorly some of the weakest performers fared.

During the one month of testing (roughly 480 hours of Web performance evaluated), the unidentified site with the weakest reliability was down for about 10 hours. Compare those numbers with that category’s top performer?Eddie Bauer?”which had not a single error for the entire month that we measured them,” Rushlo said.

The evaluation considered any improper response (such as clicked link not delivering the correct page or delivering a “page not found” message) an error, Rushlo said.

Ten or more errors constituted a failure. If more than 30 percent of the clicks generate an error during one hour, that would be considered a site outage or the site being down.

The industry average for successful clicks is about 98.6, Rushlo said, adding that the top 11 of the e-commerce sites Keynote evaluated averaged 99.5 percent, with Eddie Bauer at 100 percent, Wal-Mart at 99.6 percent and Williams-Sonoma at 99.3 percent.

Rushlo said the report’s results strongly suggested that many of these sites were not actively watching their own performance or many of the glitches would have been repaired.

“Some of the worst sites had such glaring issues that it was clear that no one was captaining the ship,” he said. “There needs to be an internal team that knows the site inside and out and is constantly testing and monitoring.”

When he confronts managers of sites that fared poorly, “it’s very typical to have someone say, ‘We haven’t measured that,'” Rushlo said.

“When we work with a customer, quite frequently they’ll have a half-person dedicated” to tracking Web performance.

Rushlo said the final segment of an e-commerce site is often where they found the greatest problems.

“The checkout portion was the least reliable, the most error-prone,” he said. “There needs to be more focus there. That’s literally where you are making your money.”

It’s not surprising that the checkout area is the most problematic because it is also by far the most complicated part of the typical e-commerce site.

Until the checkout, most sites are simple catalog displays, with lots of product images and pricelists.

It’s not until checkout that most sites bring in log-ins, security or payment system mechanisms that might include third-party credit card companies as well as possible tie-ins with the supply chain and delivery options and tracking numbers.

“Of all the pieces of the sites that we measured, it’s the most complex because you’re using things that are more error-prone,” Rushlo said, adding that two retailers?Eddie Bauer and Barnes & Noble?performed 100 percent on checkout.

Another focus of the report was page download times.

Rushlo said Keynote’s latest figures show analog dial-up connections still representing some 45 percent of all e-commerce users today.

Given that almost half of all e-commerce site visitors are likely dialing in and not using any form of broadband, Rushlo said he was surprised how little work was being done?through wise image choice, cache, compression, etc.?to keep pages small.

For a dial-up user, Keynote found average page download times of 30 seconds.

The smallest page sizes were at Staples, Office Depot and Circuit City, Rushlo said, with those pages averaging about 45K.

The worst offenders in that category had pages at 201K, more than four times as large, Rushlo said.

Page size “is one area where most companies do not have a handle,” he said.

On the customer experience side, Keynote found a very different set of retailers faring well. strongly dominated there, taking the top spot for customer satisfaction (followed by Nordstrom and Barnes & Noble), conversion impact/loyalty (followed by eBay and Barnes & Noble) and visual appeal (followed by Lands’ End and Eddie Bauer).

The key consumer perception drivers were improved site search capabilities, checkout processes that displayed total cost?including taxes and shipping before asking for a credit card?ease of navigation, and the increasing popularity of guest checkout, said Lance Jones, a Keynote senior analyst for competitive intelligence.

Guest checkout is where the e-commerce site gives customers the choice of registering, logging in or doing neither and just being allowed to make purchases directly with a credit card.


Comments are closed.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.