Evan Schuman's StorefrontBacktalk

Think that keeping customer and payment-card data in-house but using the cloud for other projects will keep you safe when it comes to PCI and privacy? Don’t be so sure. As Best Buy learned when its IT shop first discovered the cloud, the ease of starting a project in the cloud also means it’s easy to avoid IT governance. Unless every developer, every business manager and every executive understands that card data in the cloud is strictly forbidden—no matter what—someone will decide it’s safe enough for analyzing historical data or adding some quick capacity to deal with an E-Commerce site glitch.

Good luck keeping that sort of thing from happening on your cloud. Even better luck keeping it from happening if some clever business-side power user discovers he can extract CRM or POS data and rent cloud time on his own to crank it through a Hadoop application.

But you don’t even need a cloud to get into trouble. Say some programmer has hacked together a workaround to solve a problem for a business unit or a partner—maybe it’s as simple as using a cookie in an unusual way, because that’s the quickest way to solve the problem. A few years later, your privacy policy has changed and now prohibits using that type of cookie. All the conventional cookie code has been disabled. But that unusual cookie code isn’t where it’s supposed to be, so it isn’t found until some consumer privacy advocate stumbles upon it and raises a stink.

The more third parties, developers and users that have access to your systems—or support your E-Commerce site—the more complicated the problem is. It requires plenty of governance and discipline to keep everything inside the PCI- and privacy-safe bounds, even under normal circumstances. And even then, governance only helps going forward. Old patchwork will remain hidden until it surfaces to give you problems.

And then there are times that aren’t normal—like any time your E-Commerce site staggers unexpectedly. Keeping the site up becomes the top priority of the E-Commerce operations team. The rules that don’t go out the window can get very badly bent, especially if the crisis happens during a major sale or on Black Friday (ask Target how that comes down).

That’s panic-mode time, when mistakes are easy to make, code reviews are minimal and IT governance is trumped by whatever works. Ironically, though it’s hard to backtrace everything that’s done to keep a site up in a crisis, at least the post-crisis cleanup teams know they’re looking for potential problems. It’s much harder to find the old, one-off workarounds that are problems waiting to happen.

The “correct” answers—governance, discipline, communication—still only go so far. As Google discovered, you can actively watch out for those messy, unsignaled side effects—but you still have to be prepared to get the bad news.