Evan Schuman's StorefrontBacktalk

When a major site performance research site reported last week that the Oriental Trading Company Web site had a major meltdown through all of August, Oriental Trading officials raised questions about whether the report was correct. On Monday (Sept. 15), two leading site traffic tracking firms—including the firm that issued the original report—backed Oriental Trading’s position, to varying degrees.

The original report came from Gomez. And now Gomez is stepping back from its own report, according to report author Matt Poepsel, who serves as the Gomez Vice President of Performance Strategies.

"In layman’s terms, I believe the figures were incorrect," Poepsel said. "In this very unusual case, the (Oriental Trading Company) metrics were characterized by the system as unavailable for reasons that may not have reflected the experience of a typical end user. This can happen, for example, when a management system implements hard deadlines or errors in terms of page download times, error responses, page locations, page sizes or other administrative actions. While the system classified some (Oriental Trading Company) measurements as unavailable, an end user may or may not have actually abandoned the page that was being delivered. Although our benchmarks aim to serve as a proxy for the end-user experience, in this case I do not believe that proxy was an accurate reflection of (Oriental Trading Company) end-user experience for that period."

Poepsel said an internal Gomez review "may involve changes on our end."

Part of the confusion seemed to be that, on Aug. 2, Oriental Trading moved its Web server from one co-location facility to another, a transition that was wrapped by Aug. 3. The site sustained a "big outage" on Aug. 2—from 2 AM to 5 PM—but was fully back the next day, according to Brian Moen, Oriental’s VP for E-Commerce.

But Gomez’s software continued to "see" an outage all month, raising the possibility that it was looking at an IP address that was no longer active.

A Gomez competitor, Keynote Systems, reported this week that its software monitoring showed consistent uptime for Oriental Trading, other than from Aug. 1 through Aug. 3.