A way to a modern BI world with minimal human interventions
Big data is the cutting edge technology in the competitive Data Management world. These parallel processing platforms are truly based on unstructured and loosely coupled data, such as paper-based material that has been digitalized, feeds from social networks can be captured (tweets, FB posts, etc.,) blog entries, RSS feeds, audio recordings, etc., that’s the direction in which our digital world is heading towards.
In an analysis context, Big Data, Business Intelligence, and Data Warehousing are three interrelated topics. Business Intelligence cannot exist without either a Data Warehousing or Big Data foundation
Business Intelligence, refers to technologies, applications, and practices for the collection, integration, analysis, and presentation of business information.
BI helps the users make the right decisions, based on available information.
Most of the Data Warehouse projects dealt with so-called structured data/semi-structured data (flat files, spreadsheets in a certain format). Certainly one would not reckon unstructured data feeds/inputs to turn into an understandable/coherent structure.
Big Data analysis is nothing but building the right structure out of chaos to make proper business decisions after interpreting and analyzing the results. In other word, big data analysis provides rapid analytics in seconds by analyzing the unstructured data sets
One of the myths on big data is, it’s only useful for large enterprises and not meant for small or medium organizations. It’s not about storing the data, it’s all about accessing the data and organizing the information in an understandable format. Sometimes a short message or a sentence can cost organizations millions if they do not interpret the data, how they need it.
The point about storing the data has evolved a lot and it keeps growing. We moved from tape based storage mechanisms to large database storage systems in about 30 years of span. My smart mobile have more memory (2 GB of RAM) than the first computer RAM that I have seen (say 8 MB).
Thanks to advanced research gigs to get the infrastructure into living for today’s world for faster querying in natural language (example: Google Search: just type anything and you get the tons of information available right in front of you in just seconds)
Some jaw dropping big data facts for you,
- Google alone processes on average over 40 thousand search queries per second, making it over 3.5 billion in a single day.
- Every minute we send 204 million emails, generate 1.8 million FB likes, 278 thousand tweets, and upload 200,000 photos to FB.
- It is expected that by 2020 the amount of digital information in existence will have grown from 3.2 zettabytes today to 40 zettabytes
If I go back to my childhood days I still remember Aladdin and the magic lamp story. Aladdin trapped in a cave and he rubs the lamp in despair and a powerful genie appears who is destined to fulfill the wishes of the person holding the lamp. Aladdin orders him to take home and the genie does it.
Of course that’s a story and I never thought that becomes true if you touch a magic lamp and wish something good and genie does it for you. However, I have experienced and it happened to all of us, just think about the smart mobile with google apps. If you’re lost somewhere/directionless to back home, we can still find a best possible way to reach our destination with the help of great google map app, isn’t that amazing?
Have you ever wonder how iPhone SIRI understands you? That’s nothing but natural language processing and many other technologies like data mashups, question analysis, and machine learning.
Artificial Intelligence, is the field that studies the synthesis and analysis of computational agents that act intelligently.
AI makes decisions for the users
IBM’s Watson technology and its power to respond to queries presented in natural language have received much attention in recent times. It was true that Watson competed on the quiz show “Jeopardy!” in 2011. Watson evolved from IBM’s DeepQA project and the Deep Blue chess program that defeated chess champion Garry Kasparov in 1997.
We’re familiar with how basic business intelligence and descriptive analytic technologies such as query, reporting, and BI can help us analyse what has happened in the past and how data mining and predictive analytics techniques can help predict what may occur in the future. I strongly believe Artificial Intelligence could play a vital role in future advanced analytics to the beyond steps of predictive analytics.
Inclusion of Artificial Intelligence into data analytics or big data implementations yield new insights and may lead to decision automation and making of actual decisions with minimum human intervention. Although AI systems will continue to evolve and improve their decision-making capabilities, we will still need human intervention to handle unforeseen exceptions.
In fact, machine learning algorithms are widely used in Data Mining and Predictive analytics.
Microsoft released self-service Power BI Office 365 with natural language search capabilities.
- Machine learning gets boost from growing big data ecosystem