Social Unstructured Data Is Not Unusable

Written by Evan Schuman
January 21st, 2010

Just as certain a fact as stating that many of today’s social network sites will be gone in two years is the fact that new social sites—invariably much more niche and focused—will replace them. Hidden in plain sight within the millions of posts in dozens of languages of these huge number of sites is every trend, every individual customer profile and every hint of what customers will buy—and perhaps even their desired price range—that your chain could ever wish for. There’s only one problem: There is no simple spreadsheet-friendly way to access that data.

You can read it without limits. But to automate that process and to process the data in a way to get anything meaningful out of it, that’s difficult. We are deluged with products and services that are trying to solve problems that hardly anyone has ever experienced. Who will be the first to conquer this one? Many companies—including SAP and Oracle—are trying to figure it out. But they typically try to fall back on algorithms and filters. The software needed is closer to what the CIA and the NSA use to parse billions of phone calls and E-mail messages while trying to figure out plots. It’s much closer to artificial intelligence than cryptography. Military satellite technology eventually came to consumers in the form of GPS. How long will it take for AI to visit the local retail chain, where software will peruse the world to find out the best assortment to be displayed tomorrow?


3 Comments | Read Social Unstructured Data Is Not Unusable

  1. Michelle de Haaff Says:

    This information is hugely valuable. We analyze it for real use and action with retailers everyday. There is so much real-action insights in social media…
    Some examples:
    *Cries for help! – are cusotmers complaining about something online that you can answer? We find it, analyze it and route it to people to get involved in the conversation.
    *Deep sentiment analysis – What do people like? What don’t they like? What is contributing to them deciding to buy or not buy?
    *Product quality and issues – Is there a product or a product feature that is having an issue? Impacting product quality? Something we should alert customer service about?

    Thanks for writing this!
    -Michelle de Haaff
    CMO Attensity Group

  2. Fabien Tiburce/Compliantia Says:

    Until the semantic web comes to be (if it ever does), there are ways retailers can collect and process unstructured user data. For example ratings and tags, particularly semantic tags (controlled vocabularies) do provide some level of structure. The rest can be mined using probability models, the way search engines do it. While it’s easy to overlook, Google is powered by a probability engine, not semantics.

  3. Evan Schuman Says:

    Fabien’s right, but that’s a lot easier for data you can influence (such as your own Facebook area or comments within your site) as opposed to things that are entirely out of your control, such as other people’s discussion forums or social pages. For that matter, social sites you’ve never heard of two continents away. For data that is unstructured and out there in many places, well, that requires yet a different strategy. But there’s gold out there–free for the taking–for those creative enough to find ways to get at it.


StorefrontBacktalk delivers the latest retail technology news & analysis. Join more than 60,000 retail IT leaders who subscribe to our free weekly email. Sign up today!

Most Recent Comments

Why Did Gonzales Hackers Like European Cards So Much Better?

I am still unclear about the core point here-- why higher value of European cards. Supply and demand, yes, makes sense. But the fact that the cards were chip and pin (EMV) should make them less valuable because that demonstrably reduces the ability to use them fraudulently. Did the author mean that the chip and pin cards could be used in a country where EMV is not implemented--the US--and this mis-match make it easier to us them since the issuing banks may not have as robust anti-fraud controls as non-EMV banks because they assumed EMV would do the fraud prevention for them Read more...
Two possible reasons that I can think of and have seen in the past - 1) Cards issued by European banks when used online cross border don't usually support AVS checks. So, when a European card is used with a billing address that's in the US, an ecom merchant wouldn't necessarily know that the shipping zip code doesn't match the billing code. 2) Also, in offline chip countries the card determines whether or not a transaction is approved, not the issuer. In my experience, European issuers haven't developed the same checks on authorization requests as US issuers. So, these cards might be more valuable because they are more likely to get approved. Read more...
A smart card slot in terminals doesn't mean there is a reader or that the reader is activated. Then, activated reader or not, the U.S. processors don't have apps certified or ready to load into those terminals to accept and process smart card transactions just yet. Don't get your card(t) before the terminal (horse). Read more...
The marketplace does speak. More fraud capacity translates to higher value for the stolen data. Because nearly 100% of all US transactions are authorized online in real time, we have less fraud regardless of whether the card is Magstripe only or chip and PIn. Hence, $10 prices for US cards vs $25 for the European counterparts. Read more...
@David True. The European cards have both an EMV chip AND a mag stripe. Europeans may generally use the chip for their transactions, but the insecure stripe remains vulnerable to skimming, whether it be from a false front on an ATM or a dishonest waiter with a handheld skimmer. If their stripe is skimmed, the track data can still be cloned and used fraudulently in the United States. If European banks only detect fraud from 9-5 GMT, that might explain why American criminals prefer them over American bank issued cards, who have fraud detection in place 24x7. Read more...

Our apologies. Due to legal and security copyright issues, we can't facilitate the printing of Premium Content. If you absolutely need a hard copy, please contact customer service.