Evan Schuman's StorefrontBacktalk

Bringing Mobile Commerce in-store—for everything from mobile payments to mobile checkout—just hit another snag. Researchers have developed a way by which thieves could capture video of users typing on a smartphone screen from as much as 14 feet away inside a store, and then automatically extract passwords, PINs, payment-card numbers or other sensitive data from the video, which can be captured over the phone user’s shoulder or even in the reflection of a user’s sunglasses.

Shoulder surfing is an unavoidable problem with smartphones in a crowd, but 14 feet away feels like enough distance to be safe from video eavesdropping. And although there are workarounds to block the attack—turn down the screen brightness or turn off the pop-up keypress confirmation that makes the iPhone’s virtual keyboard so much easier to use—it’s unlikely that customers will be willing to do that. Store associates, on the other hand, might want to do both those things, along with being careful to guard their screens and watch for customers who happen to be using video cameras nearby. The biggest question is whether they can be convinced to do anything.

The researchers, from the University of North Carolina at Chapel Hill, used consumer-grade cameras (ranging from a $1,000 Canon camcorder to a $90 Kodak PlayTouch) aimed at users keying in text on iPhones indoors, outdoors and on a moving bus. With more than a little algorithmic magic involving some computer vision techniques, researchers were able to reconstruct the text that users typed at better than 94 percent accuracy—certainly good enough to put passwords and PINs at risk.

The secret to their success was the pop-up feedback with each character typed. “Although the on-screen text is essentially unreadable, the pop-out event provides a strong visual cue to help identify the letter that was tapped,” the researchers wrote.

That made it possible to capture text from a distance even when the phone’s screen was reflected in a user’s sunglasses—a scenario that would usually be safe from a thief looking on. (It probably helped that computer processing is likely to be better at correcting for the distorted reflection in the lenses of sunglasses.)

As with all academic papers on security topics, there’s not quite enough information for thieves to simply lift the researchers’ work and put it immediately to evil use. But now thieves know it can be done. And even if the results aren’t as good as those that the researchers got, they may still be good enough to pose a security problem.

And for customers, there’s not much retailers can do.