Best Buy’s Cloud: Wild West Gives Way To Making The Same Data Mistakes Again
Written by Frank Hayes and Evan SchumanMany chains have seen the cloud as a nice way to get unlimited data storage on the cheap. But Best Buy’s initial cloud efforts revealed something much more fun: a lawless area where IT management didn’t have any rules.
A funny thing happened, though: “Everybody has always said that if we could do the datacenter over again, we’d make no mistakes and everything would be perfect. It would be this incredible Utopian datacenter, except that we’re all making the same mistakes that we made in the datacenter originally, because you go to the cloud like the Wild West,” said Thomas Kelly, Best Buy’s enterprise architect for cloud services.
Kelly, who made his comments at a VentureBeat conference last week in Redwood City, Calif., spoke initially of the short-lived joy of lack of regulation. “That which was not regulated by arch gov [architecture governance] is technically something you don’t have to tell arch gov about. So it was hilarious, because people would say ‘oh, we’ll put it on the cloud.’ We had architects who were interested in the cloud, or we had a consultancy or a vendor that was interested in the cloud, so we ended up with 50 or so applications running on the cloud with absolutely no governance whatsoever,” he said.
That couldn’t last. But once it was time to bring governance back in—not on a project-by-project basis but to build an enterprise infrastructure in the cloud—it was possible to rethink both the hows and whys of IT governance.
“For the first time, we have a clean slate,” Kelly said. “What if we brought all of the existing cloud applications under governance and refactored them into—before it’s too late—a cohesive enterprise infrastructure? We laid down the requirements for a core repository, an ETL2 dataview, mandates for atomic service development. We brought in a really powerhouse gateway with L7. We selectively take advantage of pretty much all of the Amazon data services, for example. We run an L2-backed EMS bus to our back-end datacenter, so our cloud is actually now becoming a hybrid cloud where we’re directly connected for localization to our back-end bus.”
In short, Best Buy didn’t just shove some easy IT functions up into the cloud. It didn’t even simply come up with an IT architecture more suitable to the cloud than the one the company had used for decades. Instead, Best Buy rethought not only governance, but many of the assumptions that had never before needed to be questioned about how to do retail IT.
The problem isn’t just that the Wild West of the cloud needs a sheriff. It’s also what types of rules are appropriate, and what considerations drive them. That’s going to take some trial and error. Fortunately, it’s relatively cheap trial and error. Case in point: Capacity management.
“We are legitimately in a situation where we can be doing as much as 10 to 15 times the amount of volume on our data systems in November than we are in the middle of the summertime,” Kelly said. “We were always building out to Black Friday. However, Black Friday is one day a year, and you have to pay for software and hardware and everything else all year long.”
For online retailers, one month a year is insane for capacity requirements. Historically, IT has had to pay for all that capacity the rest of the time, too. With the cloud, that’s no longer necessary. Scaling up and back isn’t a trivial process. But it beats acquiring, paying for, setting up and maintaining the necessary hardware and software to scale up without the cloud.
That difference in acquisition grief doesn’t stop there.