At Wal-Mart, World’s Largest Retail Data Warehouse Gets Even Larger

Written by Evan Schuman
October 13th, 2004

It’s only fitting that the largest retailer should have the world’s largest database, but at more than one-half a petabyte, that’s a lot of information, even for Wal-Mart.

The vendor that is supporting those many bytes of data?NCR’s Teradata division?begged for the extraordinary permission from the normally secretive Wal-Mart to announce this achievement Wednesday to make a point: It is arguing that its systems can scale without hiccups even at an extreme number.

But Wal-Mart being Wal-Mart, it’s not saying much. While confirming that it does even now have the world’s largest datawarehouse?and that it permitted its supplier to announce that?it won’t say anything other than “to acknowledge an important milestone,” said Gus Whitcomb, Wal-Mart’s director of corporate communications. He referred questions to Teradata, saying it’s their announcement.

Beyond issuing a news release that Wal-Mart is “increasing its lead as the largest retail data warehouse in the world,” it gave no details as to the size or specifics. The “more than 500 terabytes” figure came from a source who didn’t want a name or a company linked to the figure.

The statement did, however, point out that this massive data warehouse is not solely a customer CRM system, but also serves as the base for Wal-Mart’s Retail Link decision-support system between Wal-Mart and its suppliers. Retail Link allows suppliers to access large amounts of online, real-time, item-level data to help those suppliers improve operations.

Back at Teradata, officials are prohibited from discussing what they have done for Wal-Mart, but one vice president did take the opportunity to argue what it means from an IT perspective.

“The issues we encounter at Wal-Mart are really not all that different from smaller retail data warehouses,” said Rob Berman, vice president of Teradata’s retail operations. He contrasted Wal-Mart’s current data warehouse size with its earliest stage, when it was literally less than one-thousandth of its current size.

“When Wal-Mart started with a 320-GByte data warehouse, it used one database administrator [DBA]. Today, the number of DBAs is still fewer than five,” Berman said.

Unlike a typical database that can get slower as it expands?and requires more time to complete backups and virus scans, for example?Berman argues that Teradata’s approach sidesteps those growth issues. “Our system is nearly 100 percent linear-scalable. It’s designed to scale without the management restrictions of other databases.”

How so? “Every time we add a node, we add an equal amount of bandwidth,” he said. “Every time we add a component of processing power, we add another component of bandwidth. We just grow the highway. Every time they grow in DASD [direct-access storage device], we add I/O bandwidth.”


Comments are closed.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.