Big data is the fuel that drives the many digital transformation initiatives that are underway. These initiatives lean into machine learning, which forms the building blocks for strategic programs rooted in artificial intelligence (AI). Findings from a 2021 McKinsey survey indicate that AI adoption is steadily rising, with 56 percent of respondents reporting adoption in at least one function in the business. (1)
Add to this equation the scalable multi-cloud infrastructures that open the world of data to high-powered systems, networks, storage, applications, and services. Organizations are presented with a lifetime of significant opportunities. Most companies tend to use a mix of multiple clouds and on-premises platforms for AI, similar to what they use for their everyday IT workloads. But the high performers extracting the most value from their AI initiatives use cloud infrastructure much more than their peers do: 64 percent of their AI workloads run on public or hybrid cloud, compared with 44 percent at other companies. (2)
What organizations can accomplish with these capabilities is totally up to them. Some accomplishments may come by chance. However, when a strategy, a plan, and clear actions are placed behind the allocated resources, it’s entirely feasible to accomplish some big things with considerable value.
Let’s break down this idea of what it means to accomplish something significant with your data.
ACCOMPLISH: Assess -> Catalog -> Combine -> Observe -> Mechanize -> Picture -> Learn -> Integrate -> Secure -> Harvest
Assess
What data do you have, and what additional data do you need to create or curate? Where does the data reside? Who has access to it? What can you legally and ethically do with it? Does where you operate, collect, store, and use the data change this picture? Do geographical boundaries and societal norms change this picture? How well do your business values align with what you’re trying to accomplish?
These aspects of assessing the data must first be explored to ensure that the organization can balance the risk-vs-reward and return-on-investment equations. More often than ever before, consumers and customers are paying attention to how well organizations present themselves on the world stage. (find ESG citation)
Catalog
Following the assessment to determine what exists, cataloging the data based on its source and form is essential.
Data comes from many sources:
-
- Human-authored data (drawings, diagrams, pictures)
- User-provided data (surveys, reviews, social posts)
- Machine captured data (events, transactions, logs)
- Algorithm-derived data (risk scores, fraud alerts)
Data presents itself in various forms:
-
- Core data (people, places, things)
- Transaction data (events, transactions)
- Reference data (categories, sub-categories)
- Metadata (characteristics, interpretations)
- Unstructured data (objects, blobs)
This documented view will help the business, operational, and technical teams determine what processes, tools, and methods will be needed down the line.
Combine
The true power of data comes with the notion found in the saying, “the whole is greater than the sum of its parts.”
This step can be somewhat exploratory by combining, correlating, and creating new slices of information using the collection of data sets already on hand.
What knowledge might be gained by pairing the movement of customers within a geographic region with the data collected from sensors connected to civilian services throughout a smart city? By mixing and matching data with code and algorithms, signals for when and where value might exist will begin to emerge.
Observe
With the combined data sets in hand, you can analyze and observe, looking for interesting patterns that could tell a value story to the business.
While the organization will undoubtedly want to evaluate the value side of the equation here, it’s equally important to look back to the first step in this exercise to ensure that the scenarios you are about to embark upon will align well with the assessment results; be sure to adhere to any guidelines and policies put forth at the beginning of the program.
Mechanize
With scenarios and stories surfacing, it likely makes sense to automate some of the actions and decisions further to help speed up the observation and analysis process.
The objectives, guidelines, and policies defined and documented earlier can be converted into code. Machine learning algorithms can be used to begin finding the most effective and efficient methods through which the data can be curated, analyzed, and put into value-based stories. It’s these stories that we’re after; once we can find a way to get these stories, we can begin to wrap the business around them.
Picture
Once a meaningful story surfaces, it’s time to paint a picture that can be read and analyzed by both humans and machines. Visualizing the problem, challenge, and solution can go a long way in figuring out the best ways to utilize the data you have on hand.
This picture can communicate the objectives, the expected outcomes, and the potential risks with the executive leadership team, business unit leaders, and the operational and technical teams that will see the program through.
Learn
With the picture in hand and a plan in place, it’s time to do something with the story. What is the best path forward to extract the most value from your data with the best return possible and least risk involved?
- Replicate the story at scale, bringing it to more situations in more locations
- Configure and tune it to make it an even better story
- Apply the story to different parts of the business
- Discard the story as it does not provide a good return; the investment would be better applied elsewhere
Once the program has a goal, a plan, and is appropriately staffed, it can take action.
Integrate
Now that you know what you want to do with the story, it’s time to integrate the data, algorithms, functions, applications, and infrastructure to achieve the desired outcomes. You’re close to extracting the value of your data, but there’s one more critical step before moving on to something else.
- What business objectives need to be modified?
- What operational workflows need to be adjusted or re-configured?
- What infrastructure and application implementations need to change to support the workflows?
- What changes are required in team skills and make-up to handle the new business processes?
Secure
The inputs and outputs of the new business workflows that are driven by data must maintain confidentiality, integrity, and availability in support of the very first step you took in this process: assessment. If the systems or data are compromised, the risks could outweigh the return. It’s imperative to define acceptable use policies, not just for humans but also the machines. The procedures must also be monitored and policies enforced. A clear path for escalation must be defined for the cases where something goes off the rails.
Harvest
With the entire loop coming to a close, it’s time to reap the rewards from the investments you’ve made in your data and the systems that generate, capture, store, and use it. Don’t forget that this is a living process and that the business is only as good as its latest accomplishment – and success will be based on an ongoing assessment of what data you have and when you can use it.
Citations
1,2: https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Analytics/Our%20Insights/Global%20survey%20The%20state%20of%20AI%20in%202021/Global-survey-The-state-of-AI-in-2021.pdf