So what’s it all about?

If you haven’t heard of it yet, then you will do soon. I don’t say that simply because you are reading about it at this very moment but because big data looks set to be one of the key business and IT trends in 2012. There is more data being created everyday nowadays than the total amount of data created prior to 2003. What’s more we can now capture data from so many more sources than we could before and we can get it in both structured and unstructured forms.

In the quest for competitive advantage in what are still bleak economic times, organisations are turning to these large or complex repositories of corporate and external data to uncover trends, statistics, and other actionable information to help make better decisions. Those data sets, along with their associated tools, platforms, and analytics, are often referred to as 'big data'.

Cheaper data storage, new big data tools and technologies, and even cloud computing, are now allowing this data to be stored and analysed to enrich an organisation’s processes, functions and decisions.

Sounds like business intelligence...

Yeah, it’s a lot more than that though. Traditionally, organisations have been more than able to cope with simple, structured, tabular data. That’s your bread and butter business intelligence process – looking at data in spreadsheets and documents to garner insight into the market and your business. Big data is taking that on to grander expanses.

The term big data can often be a misleading one. It does not strictly refer to sheer quantity of data; no it’s much more than that. It covers the ‘three ‘v’s’ – volume, velocity and variety – of the data your organisation is dealing with, as well as the complexity of said data.

So I can use it even if I don’t have lots of data?

You better believe it.

Is it just for the big boys?

Big data is not reserved for the huge worldwide enterprises; its uses expand far beyond this right down to small, innovative startups. Big data is a means of taking data received in a number of formats, whether it is spreadsheets, PDFs or social media feeds, and being able to analyse them collaboratively, offering invaluable business insight. Moreover, the big data tools that are being created with haste allow for this analysis to be conducted in a time critical fashion and then delivered in a range of files or graphs depending on the nature of the information and who needs it.

In fact, like cloud, big data could be a playing field leveller – allowing small companies to use exactly the same analytical tools and processes as the corporates. The expansion of Apache’s open source  Hadoop big data platform and the tools being built by some of the technology heavyweights such as Microsoft, EMC and IBM, has meant that big data is an affordable tool for most organisations.

Why is it such a big deal?

Well, you certainly don’t need me to tell you that the current economic outlook is more than a little austere. As such, any means of getting an edge on your competitors, regardless of the sector you operate in, must be seized with both hands. Big data is one such way to get the upper hand. It can help you pull together information from a range of sources and in a variety of forms and get the latest and most accurate information about your marketplace.

This in turn can help you better target potential customers, keep current clients happy or decide where you should be investing your budget. To stay afloat right now, let alone thrive, you must squeeze every drop of value out of the information available to you. Having a database full of files that aren’t being analysed is merely throwing money away. Big data can address this.

Who is using it and what for?

Many of the big names in the online and retail sectors have been using big data for some time.  Take eBay, for example, the online auctioning site gathers huge amounts of information about its users. Not only does it record every bid and transaction made, it also collates records of each specific customer. It can see the kinds of items you look at, assess how you bid – whether you like to ‘buy it now’ or sweep in with a last minute bid to pick a fellow bidder’s pocket – and can examine how long you spend on each page before clicking on a new link.

Such information offers eBay great value – it can make its marketing campaigns far more specialised to the individual and sell its advertising space at an even greater premium. The likes of Boots and Tesco with their clubcards, as well as social media giants like Facebook, Twitter and LinkedIn, are other companies that are doing similar things.
 

Does it have anything to do with the cloud?

Yes and no. Big data is separate from the cloud in that it doesn’t need to be delivered in or via the cloud. Rather, like business intelligence tools, big data can be performed in-house. However, the cloud is a valuable enabling tool and it sees as though over time more and more big data analysis will take place in the cloud.

The cloud offers transparency, cost savings and real time analytics that marry very well with big data. Moreover, with the volume of data that is being stored, it is likely that the cloud will be used as a means of accessing and performing analysis on this data without needing the databases on site.

How can I learn more about big data and how I could use it?

Great question. Conveniently, the Business Leadership Exchange (The Cloud Circle’s parent company) has launched the Big Data Insight Group. This community will cut through the hype surrounding big data and deliver expert insight, advice and guidance for your organisation to harness these valuable technologies.

Our aim is to enable members to build long-term relationships with key practitioners and suppliers in the marketplace helping them accelerate their learning, to network with their peers and source the best solutions available for their organisation.

Members already signed up to the Big Data Insight Group include:

BBC, Home Office, John Lewis, Endsleigh Insurance, Hays, PepsiCo, GSK, Flight Centre, TNT, Vodafone, Nokia, Telefonica, Institute of Cancer Research, Tesco, Holiday Extras… and many more.

Whether you are only just hearing about 'big data' or have been realising the benefits for some time, we’d love to welcome you to the community. Membership is free for practitioners following qualification. Click here to sign up now.