Evan Schuman's StorefrontBacktalk

With the surge of U.S. E-Commerce traffic on Thanksgiving night and all day Friday, the so-called Black Friday apparently caught several E-Commerce giants by surprise, with Lowe’s, Macys and Victoria’s Secret especially hard hit, according to statistics released this weekend from Keynote Competitive Research, a firm that tracks Web site performance.

But those three sites were far from alone, said Shawn White, Keynote’s director of external operations. "Our monitoring of retail site performance has shown a definite slow-down in site performance over the course of Black Friday for many leading sites," White said. "Almost a third of the thirty leading retail sites we monitor for our holiday shopping index experienced significant slow-downs that impacted the product search and check-out processes – and presumably will impact online sales."

For E-Commerce directors, the problem of site slowdown—as opposed to an outright outage—can be especially frustrating. Some site executives are unaware of the problem because they see sales not only continue to come in, but to increase sharply. The problem is that they can’t how many additional sales they might have made had the site responded more quickly.

How long is too long for a site to respond to a mouseclick? That depends on the individual shopper and that shopper’s patience. White cited the popular 8-second rule but many factors influence how lenient shoppers will be.

Is the content literally exclusive, such as a shoppers’ checking account balances from their online bank? The consumer can’t get that data elsewhere so they typically have little choice but to wait.

Is the site especially and acceptably image-intensive, with images that the customers truly want to see? Victoria’s Secret, for example, showed a huge slowdown Thursday night—from a 5-second response to a 15-second response—but White speculated that their customers might expect—and want to see–extensive images so they might be more tolerant of the delays.

Another interesting aspect of the Keynote results was their attempt to isolate where on the site the delays were happening.

In general, White said, homepage performances were good, but delays crept in much later in the purchase process. It was typically when the site visit moved from product description pages to either search or checkout, which is when sites move transactions to other servers and—quite often—to other sites entirely. That might include Fedex.com for shipping rates and tracking numbers, for example, or to their credit card processor for a Visa purchase or to a database for sales tax lookups, etc.

With so many comparison shopping sites this season, searching has become one of the most popular application types, but the millions of SKUs involved today also make it one of the most CPU-intensive.

While Keynote said that "5 percent to 10 percent slowdowns in traffic on Black Friday and Cyber Monday are common and likely will not impact the consumer’s online experience or check-out and sales" for most, Keynote added that "the worst performing sites on Black Friday" were taking as much as four times as long to respond, which White said "will lead to consumers abandoning a product search or check-out."

Macy’s, for example, saw site performance from its typical 12 seconds to about 15-20 seconds. "But it’s not happening on their homepage," White said. "It’s happening when (site visitors) are actually searching the site."

On the one hand, some performance degradations may not be the direct fault of the retail site. How can one blame Walmart.com when its card processor is overloaded and therefore takes a long time to approve a purchase? Easily, White said.

Major retailers can and should "put pressure on suppliers, partners" to optimize their own systems so that the E-Commerce site’s performance is not impacted, White said. He adding that when negotiating service level agreements (SLAs) with those suppliers/partners, attention should be placed on guaranteed response times during holidays and other anticipated high-traffic periods.

Among some of the other hardest hit etailers were Lowe’s—which saw its response time increase threefold and that it "used to be 100 percent (rapid response) but it’s now fluctuating between 20 percent and 30 percent—OfficeDepot (went from 10 seconds to 25 seconds), Buy.com and Borders, White said.

The Borders performance drop ("We’re seeing their performance slowing down 100 percent," White said) is more difficult to explain because Border’s online efforts are still completely handled by Amazon.com, which fared much better in the survey.

But White said Keynote did see that while Borders slowdowns were more severe than Amazon’s, they did happen at the exact same times as Amazon slowdowns. This raised the possibility that Borders’ traffic at Amazon might be limited to smaller number of servers or perhaps slower servers, White said. "The Borders transactions overall are faster than the Amazon ones—five seconds faster—but the Borders spikes are more significant," he said. "Amazon went from 20 seconds to 30 seconds. Borders is averaging about 15 seconds, but when it was an issue, it shot up to 36 seconds."

White said the Borders’ slowdown was also not on its homepage, but happened during its checkout process.

The traditional Black Friday and "Cyber Monday" traffic patterns are blurring, with traffic hikes happening earlier. For the last few years, neither day actually represents the height of purchase activity.

This year’s retail predictions suggest 2007 will have one of the weakest retail sales gains years since 2002, the worst of the dot-com-implosion years. The National Retail Federation on Sunday predicted that Monday will show a sharp increase, with "72 million consumers planning to shop online from home or at work tomorrow, up from 60.7 million in 2006 and 59.0 million in 2005. The (NRF) survey found that 31.9 percent of adults will shop on Cyber Monday, up 17.3 percent over last year."