Big Data and Business Intelligence: Riding the Revolution of Relativity and Resilience

Part of the excitement of being a BA is understanding leadership’s vision for organizational evolution and success. That success is often times all about realizing the opportunities at hand in relation to the organization’s data. Getting at that data is the tricky part.

When Einstein created his “Theory of Relativity” it was just that, a theory. It took years before his theory would be proven to show the relationships between time and space. It is the same way with project work, there is the idea and in relation to the project goals and objectives there is the need to prove the expected results.

The other  tricky part is in getting others to agree that the data, or the findings, mean something. It is no secret to the world of analysis how easy it is to have two people look at the same data and perceive two totally different stories. That is what makes the role of the BA so exciting, as it is full of challenges that can make a difference. It is also a bit scary as there are so many unknowns. Plus, there is more data available today than ever before and the amount of digital information in the world is more than doubling every two years.

Because business analysis involves “doing analysis”, knowing what data to use as well as what possibilities reside in the data is often a challenge. Understanding the concept of BI (Business Intelligence) can reveal plenty of useful information to the skilled analyst – information that is all relative – as the story the data tells is all in the way that the analyst connects the dots.

Understanding how to handle Big Data can take a BA to places they never thought existed.  However, there are differing opinions when it comes to using data to glean information. There are those who would question the significance of BI, who claim the data is “dirty” or “stale”.  So “Is Business Intelligence an oxymoron or is Big Data and BI really the wave of the future?”

The term “Big Data” surfaced as a result of all the massive amounts of data that were being stored. This affected the need for bigger storage capacity as time went by and which, because of its size, became more difficult to analyze and manage using the same old database tools.

The definition of Big Data is a bit more pragmatic, as it can be described as that vast frontier of information that every organization has the ability to cultivate and aggregate. It is comprised of the three “Vs”; volume, velocity and varieties of data that are considered to be too large, as it changes too quickly, and it is often unstructured. It is this same data that needs to be stored, processed, and accessed in order to make an organization more effective in operations, more strategic in its decision-making, more financially responsible, and more thoughtful when reducing risk and serving customers.

BI (Business Intelligence) is defined a bit differently. It is “the set of techniques and tools for the transformation of raw data into meaningful and useful information for business analysis purposes.” In fact, the idea that “intelligence” can be derived from data is pretty amazing – as data can reveal new insights toward solving problems and offer up amazing solutions.

There are six ways of utilizing business intelligence:

–   Discovering new perspectives and trends in the industry
–   Dashboards to drill into reporting aspects and managing metrics
–   Operational maturity (repeatable processes need to be in place)
–   Multi-dimensional analytics (slice-dice/pivot tables)
–   Mobile BI
–   Alerts and notifications

The utilization of business intelligence is the first step in defining real business value. By utilizing the data the data begins to make connections on its own. Take the Google search engine for instance, the data is being indexed alphabetically, when you trigger a search the engine does all the work, connecting the dots by indexing keywords to  provide answers to the question.

There are few things that out live humans; those things are music, literature, and art… and then there is data. Data is likely to outlive everyone. But, just like art, data does not have any meaning unless someone discovers it and gives it meaning.

Einstein’s Theory of Relativity was given meaning and importance only when several other astronomers discovered a way to prove it by using mathematics and data from a series of eclipses.

Since the beginning of time mankind has been interested in discovering patterns. Another example is the patterns of the stars in the night sky and how the creation of zodiacs mean something to the viewers, but the constellations change as a result of different perspectives depending on the hemisphere in which the stars are observed.  There are patterns in weather, patterns in language, patterns in animal and human behavior, patterns in geology and the formation of rocks, and it goes on and on. Now the world of analysis is exploding with the patterns discovered in the combination of data sets.

The business demands of today are constantly looking to find new customers and see what trends are evolving. For that reason an organization might look to hire a Data Scientist, or a statistician…yet, every analyst could be that scientist by becoming more aware of how important the data is to its organization.

The folks coming out of MBA programs are the new data analysts.  Academia has taken the time to train these students on the latest technologies and greatest revelations concerning innovation. These students are learning and trying out the newest tools for building dashboards, retrospective analytics, pivot table comparisons, predictive analytics for discovering trends, rapid processing of parallel data streams, and more. Which makes BI is every bit as exciting as watching the stars in the night sky! Because, it is all about spinning the data, discovering the patterns to determine a model, and arriving at a way to support all that hypothetical and critical thinking…all the “What ifs!”

Today, data experts are using the term “mashup”, which is a combination of data sets to achieve a particular goal. Mashups have also been defined as a type of technique that uses multiple sources of web-based data to make any project concept more meaningful. In other words the data was just sitting there – all alone – yet when it was merged with other data it suddenly had new meaning.

Here is a fun mashup example that comes from an open government data gala held in California. This gala was a government app contest targeting innovative uses of refreshed state data which included over 400 major data sources and millions of public records. There were five winners, one of which was called “California Cage Fight” and it allowed state residents to compare county data pertaining to population growth, per capita income, unemployment, and new housing developments, in comparison to any other county across the State of California. Just choose the two counties to compare and a graph would display providing the type of information the user wanted to know.

These mashups are techniques that BAs were not able to use ten years ago because the data wasn’t there and neither were the tools to manage the data, query the data, and compile it, let alone the ability to store all that data.

There are so many career opportunities for a business analyst, and these opportunities – just like the data – will keep on growing. It is all a matter of what the BA wants to make of their career. The future of business analysis is full of possibilities. These possibilities span the needs of every industry, and every level of government, organizations both big and small (public and private), career ladders both high and wide, and the data sits in the center of it all.

For financial analysts, sports analysts, insurance and risk management analysts, political analysts, business analysts, web analysts – and so many more types of analysts – the data is just waiting for new discoveries to be made.  The potential to find the answer to all those organizational unsolved mysteries – those so called “problems” or “opportunities” – is sitting inside all that relative data.

Along with the relationships that are discovered between various data sources there is the movement toward resiliency. Resiliency is the new path for essential technologies, it goes hand in hand with innovation as companies are choosing to evolve as their customer demands evolve. And being able to trust the data is a big deal. In organizations with fewer than 20 unique data sources there is 70% confidence level in the data. As the number of unique data sources increase the confidence level goes down. However, where data governance and good Master Data Management efforts exist valuable information can be shared with the executives quickly and can make the difference in four out of five times.

Technology plays a big role with the newest trends in data management and reporting. And these trends are leading to mini revolutions in the workplace….freeing up employees in ways that change lives. The business demands that utilize these trends span from digital data to cloud redundancy to mobile device technologies using GIS/GPS geospatial location-aware devices – all of which deal with data. That is why it is so important to be resilient as that data will also be resilient. Playing with the data can result in telling stories to further organizational success and encourage organizational resiliency as the “information age” marches forward in to the “age of data mashups” where the master data analyst brings the data together to deliver the “punch”  to give the data meaning.


  1. I think this is among the most vital info for me. And i’m glad reading your article.
    But should remark on few general things, The site style is ideal, the articles is really nice : D.
    Good job, cheers

Speak Your Mind