You are here

What is Data Discovery?

One of the most important new trends in the business intelligence industry is Data Discovery. It's a departure from traditional business intelligence in that it emphasizes interactive, visual analytics rather than static reporting.

The goal of data discovery is to work with and enable humans, allowing them to use their intuition to find meaningful and important information in their data. This process usually consists of asking questions of the data in some way, seeing results visually, and refining the questions. Contrast this with the traditional approach which is for information consumers to ask questions, which causes reports to be developed, which are then fed to the consumer, which may generate more questions, which will generate more reports.

The reason data discovery is gaining so much momentum is because it allows information consumers to move much faster. The answer to a question arrives immediately and can be thrown away in favor of a better question, and this can be repeated indefinitely, there is no lead time. Traditional business intelligence requires development time, which causes the questions to be "stickier"--if the question is wrong you're often hesistant to throw away the original work and start over, so the report is tweaked and refined until some semblance of an answer can be found, and at that point a new question can be asked and the process starts over again. Data discovery allows users to throw away work if it proves to be unuseful, it makes insight both disposable and a renewable resource.

Image

This is process is often referred to as "exploratory analytics" or "investigative analytics" due to its iterative process and the way you "follow your nose" through your data. It's easily the most radical shift that business intelligence has seen in the past 20 years. Data discovery embodies using technology to augment human capabilities, which is very often proven to be more effective than humans alone, or technology alone. Shyam Sankar recently gave a wonderful TED talk on this topic and why it is important, it would be worth your while to watch it:

Because of this symbiotic workflow--you might even say necessitated by it--data discovery tools are often much easier to use than traditional business intelligence tools. They're intended to be used by the end users of the information, not by an IT department or developer, and so much of the complexity has been abstracted away and made invisible. It's much more complex to develop these tools, but much, much easier to use them.

For example in the hands of an advanced user, excel could be considered a data discovery tool--it does in fact allow easy navigation of data and quick, interactive question asking. In fact Microsoft often touts it as a data discovery tool. However, it fails the smell test in that it requires extensive training in Excel to be able to proficiently use it in a "real" data discovery method, and most business users don't have that much time to invest.

Data discovery tools often tend to be more visual and interactive than traditional reporting is. They employ radical new data visualization methods--charts, graphs, infographics, and so on--to display the results and prompt the user to new insights and ideas. In fact, data discovery has often been referred to as "visual data mining".

The most exciting aspect of data discovery in our opinion is the trend towards simplicity and ease of use, which will open up the wonders of analytics to a much wider audience over time.

Image

Get your free Data Discovery Tools Roundup report

Image
We respect your email privacy.

Jason Kolb

Data Stories

How to increase sales by looking at your customer data

Are you a B2C company interested in increasing sales? If you have some form of customer data, you may be in luck.

IBM's Watson and the future of Healthcare Analytics

What would it be like to have a doctor who’s always up on the latest research and has learned about treatments from over 1.5 million previous cases? It would look alot like Watson, IBM’s Jeopardy! playing supercomputer that’s getting ready to roll out with an all new look and a Memorial Sloan Kettering Cancer Center education in oncology.

How Facebook's Graph Search will affect Google, Technology, and Privacy

What has been both feared and expected is finally on its way: Facebook is building a better search; they're opening their vast stores of user data and giving us the ability to discover what’s inside. Lars Rasmussen, the mind that brought us Google Maps, is now hard at work creating Facebook’s new Graph Search, and from the looks of it, it’s going to put unprecedented power in the hands of its users.

How Big Data and Analytics will Change Society.

The mission statement of most police departments includes something like this, “our goal is to increase public safety, prevent crime, and protect human life.” With sufficient records of criminal behavior and analytics tools like Crimespotting, Cities can have the ability to predict when and where crimes are likely to take place and dispatch accordingly.

The Big Data Revolution

Want to learn more about how big data is changing business and how you can take advantage? Pick up our latest book: The Big Data Revolution.

"With everyone talking about Big Data and Data Science, its tough to know whom to listen to and how to make sense of it all. This book cuts through the noise and presents the reader with a roadmap for success. Whether you've been in this space for a while or are just coming up to speed, Secrets of the Big Data Revolution is a must read."
-Chris Crosby, CEO of Inflection Point Global


Trend Watch

What Higher Education Teaches us about Data-Driven Customer Retention

Rio Salado Community College is currently optimizing their student retention through focused testing and they are finding some truly telling results. We can learn a lot about customer retention and segmenting by studying what they've done.

The Growing Collaborative Consumption Market

Remember the Big Data mantra? how Big Data will enable us to better understand everything, reduce waste, and improve efficiency? Well honestly, without concrete examples it fades into the mass of voices shouting about how great the world is going to be. So lets take a moment to talk about collaborative consumption and it’s implications.

Andreas Weigend on the Future of Social Data

Dr. Weigend, Stanford Professor and former Chief Data Scientist at Amazon, tells us about social data and it's implications.

Big Data and Government Transparency

A wealth of government data is available to us today on .gov sites and private sites across the web. If we analyzed this data properly, we could build a rich understanding of how our government works and how it could be improved. But as the big data challenge dictates, the chokepoint is consumption.

Commentary

Gartner splits the 2014 Business Intelligence Magic Quadrant in two.

In an interesting turn, Gartner decided this year to split the annual Magic Quadrant for Business Intelligence and Analytics Platforms.

New Gartner Magic Quadrant: Advanced Analytics Platforms

The big story this year is how Gartner split the Business

How RoboCharm is using data to optimize customer interactions

Let’s face it, the ultimate goal for any use of data is to drive profits, and more often than not that comes back to learning how to enga