You are here

The Future of Big Data and Analytics

We are in the midst of a seismic shift in the analytics world. Industries are waking up to the reality that data is an asset, and not a simple storage necessity. This is a shift from “here, this needs to be done--go take care of it for me” to “how do I unlock the value in this”.

Image 

This realization is beginning to manifest itself in the marketplace, with the newer analytics tools focusing more on deriving insight from data than running reports and dashboards. Which is of course, the correct way to look at this problem or any problem--focus on the end goal, don’t focus on the “how”: keep your eye on the prize.

However, the tools available today are primarily old-school business intelligence tools that are being retrofitted as fast as possible to try to ride this wave. Whether or not they will actually succeed or a new generation of tools will need to emerge is still an open question.

With that in mind here are 5 developmental directions to watch:

Your Data will tell you what is Interesting

Current BI is exactly the opposite: you tell your tool what you’re interested in. The coming generation will tell you what’s interesting.

Personalization will have a lot to do with this. Think about how revolutionary Pandora was for music when you first tried it--it actually learns what you find interesting! This is the direction analytics is heading.

The systems themselves will also get better at automatic learning and pattern recognition. This will enable them to spot outliers, trend changes, and other interesting data that, today, must be spotted by a human being.

And it’s needed: one thing that everyone can agree on is that the amount of data in the world is growing beyond our ability to compute it. You can’t manage it all by looking through it. The haystack is simply too big. You will need magnets to pull the needles out automatically.

Wild Visualizations

Charts and graphs have been around for a long time. And there’s a reason for that: they’re good at conveying meaning in an easy-to-grasp way.

But again, they were developed as a way to understand much smaller amounts of data than we’re looking at now. New ways of looking at data--and especially interacting with it--are being developed. Right now we’re seeing the first wave of this, where designers are trying things out and some work, some don’t. Things like infographics, interactive Web pages, and some of the newer intelligence tools are examples of trying new ways of looking at data.

We expect to see this trend accelerate. And as the field becomes more popular and more vital to the world economy, the brains and time spent on this problem will lead to breakthroughs in new ways to look at and understand data.

For example, how do you feel about looking at your data in an immersive planetarium-like environment?


 

Self Serve Intelligence

The ability to use analytics will no longer be a specialized skill, and we are already seeing the beginings of this with data discovery. It will be a common and everyday thing for executives on down the line to use as part of their daily and hourly workflow. This will be especially pervasive in knowledge industry.


This is slightly counter-intuitive because the amount of data being used is growing by the day, and yet the tools will be simpler? Yes, that’s right.

As the amount of data increases it is forcing a complete re-evaluation of what intelligence tools need to be. The complex user interfaces with miles of toolbars and byzantine menu systems are no longer adequate--they were built for a world with much less data.

Which brings us to the new types of interfaces that are needed:



Natural and Intuitive Data Interaction

Business intelligence is one of the last bastions of enterprise-y software simply because of its complexity. You know you’re using business intelligence software if half the screen is taken by menus and a passerby would have no idea what you're looking at.

 

Natural interfaces such as touch, voice, and gestures abstract away the complexity. The more functionality you can make intuitive the more productive a user can be.

Natural interfaces such as touch, voice, and gestures abstract away the complexity. The more functionality you can make intuitive the more productive a user can be, because they won’t have to learn how to use it. When you can literally ask the question “what demographic should I focus on to increase sales?” You can find valuable information without training or expert help. If you first have to be trained to navigate your data and transform it to answer your questions, you may never learn get there..

Apple did the world a great service by training people to use touch and coming up with intuitive--and, now, universally accepted--touch gestures. Apple’s Siri is important because it is training people to use voice. Microsoft’s Kinect is important because it’s training people to use body gestures. All of these modes of input combined create an interactive environment that lets you play with your data--explore it and interact with it.

 

You will think with your eyes and follow your nose to the data you’re looking for, even if you’re not sure of the right question to ask.

Data will be *fun*.

 

Collaborative

A few years ago you couldn’t escape “collaboration” as a buzzword. The idea was that by enabling people to work together you freed them to pool their creative and mental resources. Hopefully, of course, resulting in a synergistic explosion of ideas and good decision.

Collaboration was, in a sense, a way to deal with a lot of information--Big Data--before Big Data got really Big. Unfortunately the amount of data that needs to be evaluated is simply too big even for large numbers of people to efficiently look at.

The new types of interfaces (as explained above) will alleviate that problem. But there was a core value in collaboration that doesn’t go away just because it’s no longer “the new black”. Two heads are still better than one, if they’re in the right place at the right time.

When tools allow people to use intuitive interfaces to not only explore their data and discover from it but also to allow more than one person to do this at the same time, the results will be nothing short of magical. Imagine being able to look at data in interesting ways, exploring it and sharing what you’re looking at in real-time with colleagues from around the world. Enabling people to work on the same thing but giving them the freedom to explore independently gives you the best of both worlds.

In Summary

Most of these changes will happen within the next 3 to 5 years. Some of them are happening already. But all signs point to a type of inevitability for them--they are at the intersection of two or more larger trends driving our society towards a new type of information consumption. One that’s very different from what you think of today.

Data Discovery tools roundup 2012

Data Stories

How to increase sales by looking at your customer data

Are you a B2C company interested in increasing sales? If you have some form of customer data, you may be in luck.

IBM's Watson and the future of Healthcare Analytics

What would it be like to have a doctor who’s always up on the latest research and has learned about treatments from over 1.5 million previous cases? It would look alot like Watson, IBM’s Jeopardy! playing supercomputer that’s getting ready to roll out with an all new look and a Memorial Sloan Kettering Cancer Center education in oncology.

How Facebook's Graph Search will affect Google, Technology, and Privacy

What has been both feared and expected is finally on its way: Facebook is building a better search; they're opening their vast stores of user data and giving us the ability to discover what’s inside. Lars Rasmussen, the mind that brought us Google Maps, is now hard at work creating Facebook’s new Graph Search, and from the looks of it, it’s going to put unprecedented power in the hands of its users.

How Big Data and Analytics will Change Society.

The mission statement of most police departments includes something like this, “our goal is to increase public safety, prevent crime, and protect human life.” With sufficient records of criminal behavior and analytics tools like Crimespotting, Cities can have the ability to predict when and where crimes are likely to take place and dispatch accordingly.

The Big Data Revolution

Want to learn more about how big data is changing business and how you can take advantage? Pick up our latest book: The Big Data Revolution.

"With everyone talking about Big Data and Data Science, its tough to know whom to listen to and how to make sense of it all. This book cuts through the noise and presents the reader with a roadmap for success. Whether you've been in this space for a while or are just coming up to speed, Secrets of the Big Data Revolution is a must read."
-Chris Crosby, CEO of Inflection Point Global


Trend Watch

What Higher Education Teaches us about Data-Driven Customer Retention

Rio Salado Community College is currently optimizing their student retention through focused testing and they are finding some truly telling results. We can learn a lot about customer retention and segmenting by studying what they've done.

The Growing Collaborative Consumption Market

Remember the Big Data mantra? how Big Data will enable us to better understand everything, reduce waste, and improve efficiency? Well honestly, without concrete examples it fades into the mass of voices shouting about how great the world is going to be. So lets take a moment to talk about collaborative consumption and it’s implications.

Andreas Weigend on the Future of Social Data

Dr. Weigend, Stanford Professor and former Chief Data Scientist at Amazon, tells us about social data and it's implications.

Big Data and Government Transparency

A wealth of government data is available to us today on .gov sites and private sites across the web. If we analyzed this data properly, we could build a rich understanding of how our government works and how it could be improved. But as the big data challenge dictates, the chokepoint is consumption.

Commentary

Gartner splits the 2014 Business Intelligence Magic Quadrant in two.

In an interesting turn, Gartner decided this year to split the annual Magic Quadrant for Business Intelligence and Analytics Platforms.

New Gartner Magic Quadrant: Advanced Analytics Platforms

The big story this year is how Gartner split the Business

How RoboCharm is using data to optimize customer interactions

Let’s face it, the ultimate goal for any use of data is to drive profits, and more often than not that comes back to learning how to enga