Evan Schuman's StorefrontBacktalk

But it was the inventory problems that killed the experiment. “It simply lost too much money. It would scan the wrong item and it wouldn’t be caught at the door,” the source said. “We only mark things up 7 to 10 percent, so it hurt.”

Age-restricted items, another headache for all self-checkout systems, were also a problem with the Costco trial. The system would pause for an age-related item and need approval from a store employee, but it often timed out before help could arrive. “We just tried to adjust it to give us more time, but that didn’t work,” the Costco exec said.

Self-checkout systems have historically had this love/hate relationship with ROI. For customers who use it properly, self-checkout can help offload small orders and improve how many customers can be processed with the same number of associates. But as the members-per-hour numbers at Costco demonstrated, if self-checkout experiences another delays, it doesn’t necessarily help.

Last month, the Wal-Mart Chief Financial Officer tried to associate a boost in self-checkout lanes with a reduction in cashier wages, but couldn’t quite get the numbers to add up.

Self-checkout generates two very different types of shrink (deliberate and accidental), but it’s almost impossible to differentiate the two when customers are trying to leave the store. Indeed, those accidental shrinks (sounds like a great movie title: The Accidental Shrink) are often not even the fault of the customer as much as the fault of the store or, more precisely, store systems. Even implying that such customers might have been shoplifting is a disaster.

And yet, letting real shoplifters go without consequences is inviting many more theft attempts. Granted, loss prevention has many ways of dealing with this problem (footage of the facial expressions and actions at the point of self-checkout, shared records with other area retailers of people who have this before, etc.). But it’s easy to see why store managers are not universally falling in love with self-checkout. (And if you place a customer on the suspect list and that customer finds out, things will get ugly there, too.)

Costco’s experiment with self-checkout—which is apparently still happening in some stores—is important and was the right move for the $88 billion chain. It says something good about Costco’s IT strategy that the members-only chain was willing to experiment with self-checkout. And it says something even better that it let the trial happen for two years, to really see if the initial results would improve with tweaks and experience. Some chains would have pulled the plug much more quickly—often too quickly. It takes a wise chain to say, “If we’re going to test something, let’s really give it a chance to work.”

But the big issue with self-checkout is that it’s less a matter of technology and more a matter of how well it meshes with a particular group of customers, who have a particular preference for certain types of products. Demographics play a key role in how much shoppers in that neighborhood want to interact with associates, how much true fraud exists and basket sizes. (Generally, the larger the average basket size, the less effective shelf-checkout is.)

It’s bad enough that mobile payment and other factors are making shoplift prosecutions more difficult, while customer expectations of self-checkout keep diverging—those who like it are fine with it and those who hate it really hate it.

But when losses drop and associate efficiency rises as soon as the self-checkout lanes are pulled, it’s pretty good evidence that at least in this store, for this community, it was never going to work.