Getting Value from Big Data – How, When and Why?

The Big Data revolution is reshaping business in ways that reach far beyond the technology worlds of databases and data management. From refined data on parts and processes that is raising product manufacturing quality to a new level, to the use of tracking data from store shelves to recalibrate supply chains, businesses and business processes are being transformed by Big Data analytics.

Because the revolution starts with how we obtain, store, and work with data, it is expressed in terms familiar to data scientists, database managers, and other IT specialists. These professionals speak in term of The Three V's that drive technology developments in handling Big Data:

Getting Value from Big Data

  • Volume.  This is the "V" that gives Big Data its name. The sheer quantity of data now available far exceeds what firms could work with even a few years ago.
  • Velocity.  Not only is there more data than before; it is data on the move, constantly shifting as parts move through manufacturing, merchandise moves through the supply chain, or customers move through social media.
  • Variety.  More data and more frequently changing data also means more new kinds of data. Some of it resembles familiar "structured" business data; other data takes new forms that must be organized and managed on their own terms.

But for businesses the most important "V" behind Big Data is the fourth:

Value.  Big Data creates new business value by providing new insights. These range from manufacturing and supply chain information to market information gleaned from social media traffic, to medical insights resulting from detailed comparison of millions of electronic medical records (EMRs).

The London-based Centre For Economics and Business Research (CEBR) estimated last year that manufacturing and retail, along with professional services would show the greatest impact from Big Data analytics. Health care, telecoms, transportation and logistics, and investment banking were also expected to show a major impact. The CEBR study looked specifically at Britain, but Big Data is a global revolution, and it is having a global impact.

Companies looking at the value potential of Big Data analytics face three challenges:

  • How to move into a framework capable of handling Big Data, allowing it to be mined for analytics insights
  • When to make the move
  • Why to make the move


How to Move

The challenge of working with Big Data begins with storing and accessing it – which means it all starts with databases and database technology. Broadly speaking there are two kinds of database systems that you will hear about:

One is traditional warehoused data, stored in relational databases, and queried using the SQL query language. The other is less structured data, often stored in distributed systems such as Hadoop, and queried using NoSQL (Not only SQL) procedures.

Both of these storage environments have their advantages. SQL databases are strongly administered, making them secure and reliable. On the other hand, they can only store highly regularized sorts of data – account records are a familiar example. NoSQL databases are more flexible, but also more specialized and developer-centric.

Moving in either or both of these directions is not just a technology choice: It must begin with an idea of what sort of value the firm hopes to gain from Big Data analytics. What parts of the business are not using analytics now, but could gain from analytics insights? A successful move into Big Data analytics begins with a value proposition.

Once the value proposition has been defined, implementing Big Data analytics involves three stages:

  • Acquire Data: Harness existing data sources and add new ones from a variety of potential data streams. For example, social media are adding new types of financial trading information, while implementation and comparison of electronic medical records (EMRs) are bringing new health data points to light.
  • Organize Data: Once data is obtained it must be warehoused in ways that make it readily available, whether this involves SQL or relational databases, NoSQL databases, or a combination of the two.
  • Analyze Data: Once data is obtained and readily available, it must be cross-compared to search for actionable insights. Network analytics, text analytics, visualization, and other technologies, involving both hardware and software, may be called on as analytical tools to aid in this process.

Data Analysis is an engine that will bring Value to the businesses after Big Data project implementation.  There are many directions how collected and organized data could be viewed to derive the maximum value from recorded clicks, parts, or movements. There are many different technics available (link to Email#7), and they consists from Descriptive analytics, Predictive analytics, and Prescriptive analytics.

  • Descriptive analytics provides a picture of current conditions in some data environment. With descriptive analytics you can track what is happening now, not wait until it has already happened before you can respond to it.
  • Predictive analytics compares current conditions with historical trends data to offer a projected view of future conditions. When predictive analytics are available, you can respond to potential problems, such as traffic bottlenecks, before they turn into actual delays and disruptions.
  • Prescriptive analytics combines predictive analytics results with models of business goals to offer recommended courses of action to business decision makers. Companies can ramp up production of products that are likely to be strong sellers, or reroute shipments to avoid anticipated delays.

Prescriptive analytics may take the form of business guidance, such as recommending proposed courses of action. Or, in some cases, prescriptive analytics can be automated to directly guide a business process. For example, predictive analytics can provide automatic adjustments in supply chain logistics or order fulfillment to ensure that goods are available to satisfy customer demand – and ensure satisfied customers.


When to Move

Because data warehousing and data management are both technology-intensive processes, many firms move into Big Data analytics as a direct result of technology challenges.

Thus The Data Warehousing Institute, or TDWI, speaks of IT "pain points" that emerge as existing hardware and software systems reach their limits in handling the growing load of available data. As IT upgrades its capabilities, it opens up new challenges – and possibilities – to the parent companies.

This approach is often called the IT-driven approach into Big Data analytics. And it is having an impact, particularly on industries such as retailing. As IT departments upgrade, retailers gain access not just to more customer data, but to new kinds of customer data, and the ripples are being felt across the retail scene:

  • Thus, for example, data-driven marketing  has come front and center for many brick and mortar retailers this year, as supply chain data analytics moves from "descriptive" to "prescriptive" and retailers scramble to keep up.
  • Similarly, Big Data is already replacing mobile as the hot topic for the 2014 National Retail Federation (NRF) show, again as firms and their IT departments respond to the fast-moving analytics playing field.

For many firms the IT-driven approach to Big Data analytics is working well. But this approach to analytics is essentially a reactive approach. It is like buying a truck, and discovering that now you can make customer deliveries. New technology opens the door to new possibilities.

The IT-driven approach, however, is not the only way to go. Many firms are not waiting for IT necessity to drive technology advancement. Instead they are giving IT the new tools, then calling on IT to make the most of those tools by identifying and applying analytics to new sources and types of data.

This more proactive approach to Big Data analytics has been called the business-driven approach. Across industries, business leaders are not just passively waiting for technology to force their hand. Instead they are taking active steps to harness new technology in order to develop new business possibilities.


Why to Move

What are some of the reasons for firms to adopt the business-driven approach to Big Data analytics, rather than waiting to be pushed along by IT-driven technology requirements?

The reasons for tackling Big Data analytics are practically endless – as varied as the new types of data to which analytics are being applied. Retailers can reach out to customers at the moment they walk past a store. Manufacturers can track individual supply shipments, or watch overall traffic patterns and clear bottlenecks before they happen. Security specialists can detect suspicious or anomalous events and take immediate protective action.

Big Data analytics is not just about keeping up with technology progress. It is about using new technology both to answer old questions and to ask entirely new questions. Analytics tools allow business leaders to explore patterns and scope out trends.

With appropriate tools you can now look at long-suspected but previously undetectable relationships to see if they actually play out as expected. To take just one example, data from "smart" vending machines at sports venues or other event-centric locations is now being matched up with event schedules to provide new insights into consumer demand patterns.

Do audiences at rock concerts buy different soft drinks than basketball fans at games? Venue operators and concession vendors no longer need to guess, or rely on uncertain survey data. Big Data analytics can put solid data in their hands. Result: Happier customers, and increased sales.

The impact of Big Data analytics has been most immediate and dramatic in the retail and manufacturing Industries, where new information is most readily available in forms that firms can put to work directly. But there are hardly any industries that won't be transformed by the Big Data analytics revolution. Some of the specific industry impacts are:

  • Retail. From managing supply chain and inventory to gaining marketing insights and reaching out directly to customers, Big Data analytics is transforming the retail world.
  • Manufacturing. Every step of the manufacturing process, from shipping to receiving to manufacturing equipment to communications with line workers, is being reshaped by sensor and data analytics technology.
  • Financial. The financial industry is gaining access to a vast new store of market information, ranging from movement of goods to social media data. Big Data analytics is also providing new layers of security for financial transactions.
  • Advertising and Marketing. The era of "mass" marketing is giving way to an era of precision information about customers, and what they are looking for in the marketplace.
  • Medical and Healthcare. As the use of electronic medical records (EMRs) takes hold, we will gain access to a vast new wealth of information about individual patient results – a revolution that could transform our ability to treat diseases more effectively – or prevent them before they take hold.
  • Leisure and Entertainment. Big Data analytics is not only offering a wealth of new consumer information about leisure and entertainment choices. It is also providing new insights into familiar pastimes, as sabermetrics has transformed our understanding of baseball. These technologies also have the potential to create whole new entertainment industries, combining elements of social media, gaming, and Hollywood-style production.
  • Government. Practically all government services can be targeted more precisely, and delivered more efficiently, by harnessing the power of Big Data analytics. The result: Improved public services at reduced cost.
  • Security. Both public and private security, from front-line police work to cybersecurity, are benefiting from the wealth of information provided by Big Data analytics. Security can now respond proactively to incipient threats, on the streets as well as across our computer networks.

As the Big Data analytics revolution continues to take hold, more and more industries will join this list.


Taking On the Analytics Challenge

Many firms are not waiting for IT "pain points" to drive their advance into Big Data analytics. Nor do firms need to be at the mercy of technology vendors. A growing range of educational resources are available to guide business leaders past the challenges of Big Data Volume, Velocity, and Variety, to reach the goal of enhanced Value for their businesses.

Let GRT Corporation be your Sherpa, aiding you in scaling the mountain peaks of Big Data analytics, and putting the value of its insights to work for your business or organization.


To learn more about how GRT can help you meet your goals, contact us.