The CIA admits to monitoring Twitter, Facebook, and blogs. This shouldn't surprise anyone, but the tools they’re using might be enough to get you to rethink posting anything personal or ever checking in...
The new reality for Business Intelligence and Big Data
Book Promo
Secrets of the Big Data Revolution
Real-world success stories and roadmaps to get you there.
Commentary
This report is big business, it's important to remember it drives 100's of millions in corporate Business Intelligence(BI) spending, and has a hidden side to it. Although Gartner stays well away from saying it, there is a definite favorability for products from companies Gartner does business with. A quick look on Quora shows you exactly what that looks like: more time with business partners products and loosening of rules for inclusion.
You know about Big Data and its potential, how it creates greater understanding of our world, reduces waste and misuse of resources, and dramatically increases efficiency. The level of information available is growing like never before, and hiding in all that data are the insights, ideas, and explanations needed to reach that promised potential; it’s just waiting to be discovered, and there’s the problem.
Data Stories
What would it be like to have a doctor who’s always up on the latest research and has learned about treatments from over 1.5 million previous cases? It would look alot like Watson, IBM’s Jeopardy! playing supercomputer that’s getting ready to roll out with an all new look and a Memorial Sloan Kettering Cancer Center education in oncology.
What has been both feared and expected is finally on its way: Facebook is building a better search; they're opening their vast stores of user data and giving us the ability to discover what’s inside. Lars Rasmussen, the mind that brought us Google Maps, is now hard at work creating Facebook’s new Graph Search, and from the looks of it, it’s going to put unprecedented power in the hands of its users.
The mission statement of most police departments includes something like this, “our goal is to increase public safety, prevent crime, and protect human life.” With sufficient records of criminal behavior and analytics tools like Crimespotting, Cities can have the ability to predict when and where crimes are likely to take place and dispatch accordingly.
Prior to President Obama’s lopsided victory over Mitt Romney, the data science team behind Obama’s campaign was already making waves in the data industry. Now They’re the toast of the town and in the middle of a fight for power.
I remember walking into a Fortune 100 company that wanted more insight into the contacts they had with their customers, the day after installing our sweet new Web 2.0-y business intelligence application, the director who oversaw customer contacts wanted to see what it could do. (This was in the Web 2.0 heyday, where “Web 2.0” was the magic pill that would fix all of your business problems and fix your back problems at the same time.)
“So, what can it tell me?” he asked me.
“What do you want to know?” was my reply.
And that, ladies and gentlemen, is the problem with business intelligence in a nutshell.
That is why Business Intelligence is going to die.
I’m fortunate to have sold that company to Cisco back in 2007, and I still have a certain fondness for it, but that product was a bastion of old-school business intelligence. It was very, very good at giving you answers to your questions. But that type of product is going to die.
"But," you ask, "isn’t data hotter than hot!?" 4 out of the past 5 years CIOs surveyed by Gartner Research labeled data strategies as the most important area for development, and news about data mining and analytics is finding its way into mainstream media. So yes, data is our hottest commodity and that’s why Business intelligence(BI) is in its deathbed.
You know about Big Data and its potential, how it creates greater understanding of our world, reduces waste and misuse of resources, and dramatically increases efficiency. The level of information available is growing like never before, and hiding in all that data are the insights, ideas, and explanations needed to reach that promised potential; it’s just waiting to be discovered, and there’s the problem.
Traditional BI cannot keep up with the data and it can’t keep up with the transforming landscape of the data industry, and most importantly, it can only answer questions that you already know you should be asking.
The Crushing Weight of Big Data.
Traditional business intelligence vendors struggle with Big Data. Their tools aren't designed to sift through massive data warehouses, and they can’t identify what is meaningful or what should be ignored. They simply present reports and dashboards on top of your data, in response to questions you tell it you need to know the answers to. It turns the answers into a pretty mess of charts, graphs, and grids that you can read on a daily, weekly, or monthly basis.
Millions went into the development of some of these programs, but they were designed for days gone by. Now companies are attempting to update for Big Data by duct-taping on tools like Hadoop. But the fact remains that they are built for a paradigm where you have to know the question before you can get an answer.
But what if you don’t know the questions you should be asking? Because, quite frankly, that is where all of the interesting stuff is.
And let’s be clear that when we talk about Business Intelligence we’re really talking about a class of product, not the intelligence that comes from data, directed towards business. There’s a clear class of software that emerged in the 1990’s and 2000’s that is labeled “business intelligence”, and that’s what we’re talking about. This includes things such as dashboards, emailed reports, OLAP cubes to an extent, and so on. “Business intelligence” in the sense of the intelligence derived from data that’s applicable to your business is clearly not dying, that’s the whole point.
The Transforming Landscape
People have recently become aware of the world-changing power of data to make and remake industries. And, truth be told, that movement is still in its infancy, we’re just now starting to understand the ramifications in society, business, and culture. The volume of data quickly shows BI for what it is: outdated. BI simply cannot put Big Data to work for its customers.
People see the potential and they want to use the tools that unlock it, but a short look at any of the current BI tools shows just how impossible that is. BI was designed for IT and CIOs, if you want to use it you need to be trained. But as society at large moves towards a data-driven approach to, well, everything, a much larger audience for data tools is emerging: a whole new type of customer, non-technical, and a much larger market that traditional BI tools just can’t reach.
Beyond the new market, What consumers want from BI has changed. Traditional BI is fantastic at building reports and monitoring key performance indicators, but it uses dashboards, grids, and tables primarily to show the data. This type of reporting is fantastic if you’re technical and understand it, but for the average businessperson it’s reminiscent of that calc class they barely passed.
Business Intelligence’s coup de grace
With tools ill suited for the work, a new consumer unforgiving towards complex interfaces, and reporting styles unfriendly to typical users, BI is looking at a bleak future as the gap between what is needed and what is offered expands.
Fortunately, there are new tools becoming available which fill this gap. Companies like Tableau are creating solutions which make it easy to explore data and find answers. We classify these types of tools as "Data Discovery" tools--they make it easy (or at least, easier) to find the interesting data within the mountains of available data.
The analytics market is bifurcating--splitting into two segments: the old "business intelligence" market and the new "data discovery" market. Gartner Research speaks to this in their latest Business Intelligence Magic Quadrant report:
We agree completely with Gartner, and we believe that this trend will only accelerate over the medium-to-long-term. Data discovery capabilities will become increasingly important as the amount of data continues to increase and the traditional business intelligence fails to keep up.
Back in 2007 this didn’t help much, I didn’t have a good answer for the director. Today it’s a slightly different story: the tools are improving, becoming smarter. There are tools becoming available that can help mine for the nuggets of gold in the sea of data. That’s called Data Discovery, and that’s what is killing business intelligence.
For more information on Data Discovery get this free report today:
Jason