Wall Street Not Alone In Crashing Tuesday

Written by Evan Schuman
March 2nd, 2007

The abrupt 200-point plunge in stock prices this week sent investors running to sell shares and Web admin struggling to keep sites from crashing.

As people looked at the incredible strain placed on Web servers this week when the market took a massive tumble, a frequently-heard comparison was with retailers fighting holiday shoppers.

There are two reasons such a comparison doesn?t work. First, the start and likely end of E-Commerce heightened purchases are precisely known and known for more than a year in advance. Tuesday’s plummet came with no warning.

The second reason, though, involves consequences. A consumer shopping who is delayed a couple of minutes will be annoyed and perhaps might even jump to a competing site. An investor who is similarly delayed?especially around the 4 PM Cinderella hour, when carriage share sales orders turn into pumpkin powerless keystrokes?can literally lose a fortune. And, of course, most investors are locked into using one broker (wherever their accounts reside) and they can’t switch on a whim when server performance slows, pointed out Abelardo Gonzalez, a Web performance manager at Keynote Systems.

How slow did things get? Well, that depends on the site examined. After all, it’s times like these that separate the men from the bulls.

The two top firms that track Web traffic are Keynote and Gomez. Although the two firms’ methodologies differ?making direct comparisons difficult?both agree that eTrade did quite well during the trading onslaught, while investment houses with much longer histories?including Fidelity Investments and Vanguard?fared poorly.

But again, how poorly? On a good day, the typical trading site should respond with a few seconds. A day before the plunge (Feb. 26), for example, Gomez found Scottrade responding with an average of 3.7 seconds, ETrade at 4.6 seconds, TradeKing at 8.5 seconds with old-timers typically delivering weaker figures, with Charles Schwab at 11.5, Vanguard at 11.5, Fidelity at 19.4 and Wells Fargo at about 22 seconds. Please remember those are typical figures for the day beforethe plunge.

On the day of the plunge, it wasn’t only stock prices that were on a freefall. Vanguard’s 11.5 second response dropped to an amazingly bad 208.43 seconds. Yes, that’s almost 3.5 minutes. That’s just an average. Gomez clocked Vanguard as slow as 350.2 seconds, which is almost six minutes.

Imagine trying to execute a trade and having to wait almost six minutes for the site to respond. Do you assume the site has crashed? Do you dare try again, risking a double order?

Vanguard was the worst on Gomez’s radar, but Fidelity (with an average 96 second response time) and Banc of America (averaging at about 82 seconds) held their own in the lousy response department.

Why the difference? Matt Poepsel, vice president of performance strategy at Gomez, believes much of it involves upbringing. eTrade and some of the other younger sites “really cut their teeth on the Web. The understand the importance” of response time and “they built their infrastructure the right way.”

The challenge for some of the larger sites also involves the inability to solve these kinds of problems by throwing money at it. “There are certainly some things you can do with capacity, but there are not an unlimited number of things you can do,” Poepsel said.

Keynote’s Abelardo Gonzalez pretty much agreed, although Keynote released very few specific performance numbers. “Fidelity did have places where they went over 40 seconds. The slowest was about a minute,” he said.

Gonzalez said that although response times are critical, he thought a potentially even more important number to watch are failed transactions, citing the day showing a 25 percent decrease in successful transactions.

Keynote also cautioned Web managers to not draw the wrong conclusions. Some site managers want to look at the worst performance they’ve seen and load-test for double that. Gonzalez suggested that testing for seven times the peak figure might sometimes make more sense.

On the one hand, it’s easy to dismiss weak site performance during these market plunges as inevitable, on the rationale that Web sites must prepare for what is reasonable and likely. Paying for an infrastructure that is designed to assume that every day is a mad rush wouldn’t be very responsible, the argument?typically advanced by people who think cheapness is a virtue?goes.

That argument might sound convincing were it not for the existence of sites like eTrade. They prove that it can be done and not necessarily with a break-the-bank budget. How often have young Web operations found ways to deliver performance in a way that the established companies simply knew couldn’t be done?

Giving an online site a true Web-centric architecture, infrastructure and attitude? Now that’s a gameplan I’d invest in.


Comments are closed.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.