The new year is the perfect time for fresh beginnings. As a rule – or just an overwhelming coincidence – we all have an innate desire to implement changes in our day-to-day lives that set us up for success in whatever we choose. There is no reason that this same courtesy does not have to extend to your business practices, especially those that govern how we do or do not interact with enterprise Big Data stores.
The term Big Data has been all the rage for years now with no sign of it going away. As technology provides us newer, better, and more innovative methods for gathering information on ourselves, our companies, our employees, and our customers, we are faced with newer and more complex challenges – such as making sure we use this resource for good rather than allowing it to overwhelm us. The knowledge of patterns and practices that come from analyzing Big Data can have tremendous impact on the ways a business operates, how it modifies its practices to fit the environment and its customers, and more. Big Data can help us save money and make even more on top of it. We just have to use it properly.
Let’s explore tips for how to get the most of your Big Data this year, tasks you can add to your own yearly goals that are designed to better both your data’s performance and your own.
1. Turn Big Data into "Small Data".
A huge benefit of Big Data is that it allows you to look at vast quantities of information, analyzing patterns based on these massive stores. The downside is that it often shows you everything when you are really only interested in one thing. By embracing Small Data, you can drill down to a subset of this resource with the intention of only looking at an individual product, pattern, customer group, etc. It helps in better optimizing the use and flow of the resource being analyzed – putting your team in a position to enact real change in a shorter amount of time.
2. Evaluate your Data Recovery plan.
With more data, comes a greater responsibility for ensuring its safety. Do you know what data recovery practices your company employs? Does your team actively backup data as often as it should? It is better to know these things now rather than stressing to figure them out in the event a server - or even worse, servers - should crash.
Our top points to consider when evaluating data recovery include:
Where are backups stored?
How often should backups be made? What is the workflow for creating a backup?
Is there a limitation to how much disk/server space is allowed for each backup? What about for multiple backups?
Will this limitation mean that historical backups cannot be maintained or must be maintained elsewhere?
Who has access to backups? What team or employee should be contacted in the event a previous backup is needed?
What is the process for restoring from a backup? How much time does it take, and how quickly can it be implemented?
What are examples of red flags users should be aware of that tell them they need to restore from a backup (e.g., corrupt data, server crash)?
3. Get real when it comes to "Real-Time" data.
One of the largest sources of Big Data is real-time data. This source knows no bounds when it comes to industry, including just about anything. Seriously. Heartbeat data from your Apple Watch, location of field crews for a utility company, pressure reading for a pipeline, and number of Netflix shows binged watched are all examples of real-time information. While having access to this can be beneficial, this type of data requires you to perform frequent refreshes to pull down the most current information. It’s just the nature of the data. Unfortunately, the process is costly - both in time and resources (manpower and monetary).
To cut down on these costs, keep data mining and refreshes to the time period you’re studying. If you are evaluating an application’s usage from 6am to 3pm, only pull information for that schedule. Likewise, if the frequency of the resource generating this data does not require you pulling data down every 30 seconds, reconsider scheduling refreshes for every 15, 30, or even 60 minutes. It can be expensive to refresh, process, and analyze such large amounts of information so frequently. Make sure the time frames and rate at which data is refreshed accurately reflect the nature of the data itself.
4. Employ data analytics to identify business problems, not just fulfill consumer goals.
We like to think that the sole purpose of Big Data and the insights it gives us is to reimagine the ways we can make more money and better serve our customers. While that is an admirable goal to have for your data sleuthing, one of the greatest uses of this information can actually help us help ourselves. Analyzing big data with the intent of uncovering business practices that are not being implemented properly, need to be updated, or are just altogether outdated is like taking a long look in the mirror. It can be tough, but it is incredibly worthwhile. After all, streamlining your own business processes makes it easier for you to actually help your consumers.
A few areas this sees a huge impact are developing training for employees, evaluating administrative workflows, determining employee productivity, managing enterprise spatial data, and more. By identifying where your team could use improvement, Big Data has instead helped you to create new goals that serve your business internally – ensuring the cogs of the machine are well greased and continuing to run smoothly.
5. Ensure Big Data is structured - and that its structure corresponds with your own business processes.
There are many misconceptions about Big Data – the biggest being that there is no rhyme or reason to how it is stored, managed, or interacted with. In reality, the only difference between this information and say, the data you manage in your own Documents folder on your local drive, is that it exceeds your expectations of the three V’s - That is, Volume, Velocity, and Variety.
Regardless of the depth or breadth of data, there should be a structure to ensure that is easy to care for, quick to access, and simple to recover should it be required.
Take time this year to identify the current infrastructure employed by your team and your organization. Determine if the structure being used now reflects the rules outlined on a team and/or corporate level. Even better? Dig deeper and assess if these rules are working for those who interact with the data most. For example, does the naming scheme require dates input as MONTH/DAY/YEAR but it is encouraged in the remainder of the company to use the YEAR/MONTH/DAY format? It is a small example, but even things like this can be reevaluated with a plan made to update the process – and historical data along with it. Consider ways to make these massive stores of data more manageable. Remember, data infrastructure itself is as important as the workflows and business practices behind it.
6. Reevaluate your approach to data analysis - data driven or hypothesis driven?
Whether or not you realize it, we typically approach data analysis in one of two ways. We are either data driven or hypothesis driven. A data driven approach means that you look at everything – every pixel, cell, scrap of information – to discover what patterns and secrets lie in the fold. It can be effective but messy, resembling searching for a needle in a haystack – where the needle can be any size, shape, or color and the haystack can be size of a football field. Hypothesis driven, on the other hand, is when we approach the data with an idea of what it might tell us. It provides a direction for analysis, making the task much less overwhelming.
An example of hypothesis driven data analysis is when a utility company in a right to choose area (i.e., consumers can choose the company from which to buy power as opposed to being assigned a provider based on boundaries) wants to know why their enrollment has dropped in the last 6 months. At their disposal, they have data that shows the rates, company size, crew response times, promotions, and enrollment count for all utilities in the area. They hypothesize that the availability of promotions by competitors – and the timing of their own current promotions ending – is to blame for this. By looking at the enrollment and promotions alongside the other factors, they are able to prove this hypothesis true or false. If false, they can adjust their prediction and more deeply explore the other contributing factors.
The approach you take will likely differ based on the data and circumstances at hand. Just ensure that you and your team are exploring the most efficient ways to get everything you need out of this information and your own business practices.