Sunday, 29 June 2014

Big Data, Analytics and Insights

What is Big Data
Big data is one of the most important topic today.  So what is "big data"?  Does it mean "data" has become big?

For my point of view, I feel that the data that can be stored in digital form has increased.  Information has always been there, from stone inscriptions to those written on books, but the readers were limited, owing to knowledge, access, language, distance and the time period up to which it could be preserved.

Today data that can be stored and accessed has increased in quantity.  All of us here have email ids, send emails, have mobile devices, send SMSes, many of us have installed Whats App and use it, tweet and use Facebook.  We use these to exchange information - data.  All this data is stored somewhere, hence we are able to search for a mail which we received four years ago.  We are able to see a photograph of our friend's child.  We get an SMS from Railways giving the PNR number and requesting not to waste paper, an instant message from a Bank intimating that the EMI for Housing Loan has been deducted, a message in Facebook that a movie that you wanted to see is not so good and that a shop your regularly visit has opened a store in your neighbourhood.  You wish your friend on her Birthday, and Thank Facebook for the timely reminder that it gave you.  You too, tweet, write a blog and post your favourite photo in Facebook.  The newspapers and magazines are available over the Internet, so it is not necessary for you to spend time with them in 'paper' form early in the  morning.  With all this, we get to know that the 'Volume' of data has increased. 

By the time I complete this post, so many others would be also be adding other data by posting photographs, marking 'Like' for a garment brand, tweeting comments on a product, greeting a friend for passing an examination, booking railway tickets, transferring money from a bank account using NEFT etc.  The data that already exists keeps changing very fast (Velocity)

To store all these Varieties of data - written text, photographs, number of Likes for a garment brand, railway ticket booking, managing bank balance and transfer of money securely, the systems should be appropriately equipped.

In some of the above examples, the Veracity (how true the data is) plays a very important role. Incorrectly tagging a person in Facebook is a common example false data.

Hence Big Data is a huge Volume of data, that keeps on increasing, of different Varieties and the Velocity at which this data changes is significant.  Veracity of this data has to be ensured.  We realize its importance based on the Value it provides.

Why Big Data?
Consciously or sub consciously we have been using analyzed Big Data.  Google displays the search results using maximum access as the criteria.  Facebook and LinkedIn do show us your "Could be" Friends.

Big Data is not there just because the extent of storage capability has increased.  It is there because it can be analyzed to provide us valuable Insights. 

For example, after I Publish a Post, Blogger allows me to view valuable Statistics.  It provides details on number of times each post has been read, read count based on countries, browsers, operating systems and the link using which the reader read to the Post, which provides me some insights.  As soon as I publish, the read count starts increasing slowly, it increases sharply once I give the link in Facebook, and it decreases after the post is a week old or so.

When an individual is so much interested in knowing the number of readers, it is obvious that Governments, Banks, Insurance, and Retailers would want to know Insights which would help them improve their services, and / or maximize their profits.

Actionable Insights
1. Some of the actions that Banks take based on the Insights might have already been experienced or observed.  A Bank in which you have your salary account (or a good amount as balance) is  ready to give you a Personal Loan without provision of documents.  The organization is also treats you as a Privileged customer, thereby ensuring that you are satisfied with their service and will remain their customer.

2. When you are regularly consulting a Diabetes hospital, the hospital staff call you up when your next Consultation is due.

3. A reputed food chain has started offering Vegetarian food after getting to know closely that most of the potential customers are Vegetarians.

Given below are other examples.
1. On knowing that a store is running short of a particular drug, the Pharma company can immediately replenish the stock there.  If there are many such stores run short of the same drug, the company may need to decide whether the production of the drug has to be increased.

2. A particular garment brand compared itself with its competitors using details from a social website and found that its performance has dramatically reduced.  It then started replenishing stores with stocks based on the geography and reducing prices to make the clothes suit the style of the locals and affordable to them.  Also it started advertising through the social website and adding new designs.

3. Currently, organizations have started recognizing the importance of big data and analysis and have predicted that the industry would require a number of Data Scientists over the years to come. (Predictive analysis).

With Big Data and Analysis playing a key role in almost all domains, we need to understand the components that are used to obtain the actionable insights and the opportunities that it provides us.

The below links will enable you get a good understanding of Big Data and Insights.
What is Big Data Analysis
Big Data - What it is and how it matters
IBM Big Data Use Cases
IBM Big Data in Action


Raghavan said...

An informative and interesting blog by the author....enlightening common man to know much about data,network, and other connected matters..Useful to all...

Arunkumar Narayanan said...

very good article ..

Post a Comment