Manufacturing’s Data Dilemma: Regaining control in an Industry 4.0 world

If you feel frustrated and paralyzed by the experience of data and reports of your manufacturing operation then this 10 minute article is for you. It covers the modern data struggles manufacturers are facing, the root causes, and the missing paradigm shift to move forward. If it is useful then share it, post it, and send it to others. Manufacturers deserve a better data experience!

Introduction ​

Manufacturing is neck deep in data. This isn't a new phenomenon. As the industry has moved into its current era — often referred to as Industry 4.0 — there has been a push for generating and collecting as much data as possible. Over the last couple decades much of the industry has bought into this paradigm shift and invested heavily in doing precisely that. The goals through all of this are the same things that have driven manufacturing innovation for centuries: higher efficiency, less supply chain risk, and more agility in responding to market needs. Yet some manufacturing operations find themselves stuck in limbo part way through this transition, heavily invested in this new future but struggling to realize the benefits.

This article aims to help leaders and decision makers in manufacturing who are facing this transition struggle. Over the next five minutes we'll look at common issues faced, root causes, the missing paradigm shift piece, and the decisions that must be made to move forward.

The Data Challenges of Industry 4.0 ​

"Never forget, a tool is only as useful as the amount it benefits its users." — Urban Dynamics #1 strategic advice on tooling selection

Let's start with the data challenges manufacturers face with Industry 4.0. Data collected today generally spans many different systems and departments, including ERP (Enterprise Resource Planning), MES (Manufacturing Execution System), SCADA (Supervisory Control and Data Acquisition) systems, CRM (Customer Relation Management), and more. Conversations in this world tend to focus on vendor technologies involved or the stages of data pipelines, but instead we're going to focus on the business operations and value layers along with the gaps experienced there.

There are five common issues seen at the business operations & value layers:

1. Data Volume: Driving much of the below points is the exponential increase in how much data is generated and collected. This is not necessarily a negative, as more data can be used for more insights, but the 100x or even 1,000x amount of data manufacturers are seeing is overwhelming. This can lead to data utilization actually decreasing as users of it feel that the sheer size is intimidating and unclear how to work with it.
2. Data Silo'ing: With so many different functions collecting data into their own systems, it's unsurprising that data becomes very silo'ed. Trying to track down root causes for a recent increase in warranty claims on a product could require sifting through your data in your customer support SaaS solution, separate CRM SaaS solution, self-run ERP system running at your headquarters, multiple different SCADA systems across your plants, plus an extra fun addition of random business Excel spreadsheets. A ton of effort is spent just extracting and correlating data across these systems, even if the final analysis is quick.
3. Data Access: Building on top of the issue of Data Silo'ing is Data Access. The more partitioned data is, the harder is it for people to access it. Often times this results in human intermediaries for data access. As an example, if a business analyst is trying to make a key decision they have to manually email contacts at each plant to pull data they need and wait hours or days to get responses, each with an Excel sheet in a different format. This is only made worse by older systems that have highly limited access due to security concerns. This human process also makes any security auditing of data access impossible as data is given out at the discretion of the person receiving emails asking for it.
4. Limitation from Vendor Platforms: Vendor solutions are not designed for your business, they're designed for many businesses. This means that there is always going to be a gap between what you ideally want it to do and what it does. One of two things occur: (1) accept these limitations and work within them or (2) over time different parts of the business customize them. The first option is the simple one but it rarely occurs and instead business units frequently do the second by customizing these systems. Operators customize the SCADA implementation to their needs. Procurement builds custom reporting on top of the ERP system. Customer Support does the same for the CRM system. This leads to our next issue.
5. Paralysis from Legacy Dependencies & Customizations: As time passes systems tend to not get upgraded because the business functions that depend on these custom pieces refuse change due to the effort and/or downtime of doing so. This may be because the customizations that aren't compatible with these upgrades or simply low prioritization. Regardless, if this occurs then one day you will find yourself being told that some of your systems are 5 or 10 or even 15 years out of date. It becomes paralyzing as no vendor or internal person knows how to unravel this giant problem that's built up over time. Adding insult to injury, employees may threaten to revolt at the suggestion of change as they've worked hard to make their workflows effective on these systems and are rightfully concerned about the operational impact from any change. So no one touches it and it just gets worse each year.

The Root Cause: Coupling Data Generation, Collection, and Utilization ​

Data flows through three steps: generation, collection, and utilization. The root cause of the issues from the prior section is when these become coupled. Let's look at a quick example to highlight this. Let's say your factory buys a new piece of equipment as part of the manufacturing line. This new piece of equipment generates a ton of data as it runs and that data is sent to a server running software provided by that same vendor which collects that data. When it's time to analyze that data, you have to log into that vendor provided software to query, interact with, and run reports against that data. This is the common scenario today but the couplings from this are problematic.

Some very common challenges occur from this high degree of coupling between these three phases of data. Should picking a more efficient factory floor component be stopped by employees being scared of losing the old system's data reporting UI? No, but that can happen with coupling. Should picking the best safety monitoring solution be determined by whether it does or does not integrate with Qlik? No, but with coupling it's bound to happen.

This idea of couplings between data generation, collection, and utilization may seem like a problem factory operators must accept with using any vendor solution. We at Urban Dynamics have talked to some companies that have viewed this as an inevitability for over a decade now. However, as the next section will walk through, there is a solution to this problem.

The Missing Paradigm Shift: Owning a Data Platform ​

Over time, as technology has evolved in manufacturing so too have the internal capabilities needed by manufacturing. A century ago it was common for manufacturing plants to have to solve power generation themselves, with massive boilers and steam systems as the way power was generated and used in a plant. Today many plants run off of the standard electric grid and no longer need such capabilities while new ones now exist no one could have imagined a century ago, such as robotics. A new capability that manufacturers must now think about in our Industry 4.0 world is owning a Data Platform.

An internal Data Platform allows businesses to take back control of data collection and utilization. This is a move away from a turn key vendor solution to a platform that supports the business's exact needs and their evolution over time. Think of this as the difference between renting a home and owning a home. A rental has a recurring fee for your usage and limitations on that usage while you live there. Meanwhile, ownership has a higher upfront cost but allows you to do exactly what you want with your home. An internal Data Platform is like this, except instead of your home it's all the business's data.

There are a few key outcomes from owning your own Data Platform:

1. All Reporting can be Done in One Place: Ideally, plant operators and customer support staff should all be pulling data from this singular Data Platform. This means training and onboarding for this is simplified, as people don't need to be trained across numerous systems for reporting. Additionally, business functions now have the ability to correlate data across business units at no cost. Looking at a recent increase in warranty claims and want to look for correlations from factors when the failed units were manufactured? All the data is available, just start running queries to look for connections. To be clear, transitioning to doing reporting through a single Data Platform is an incremental process that can take years. But this is the goal to work towards where there's the biggest payoff across the business for all its data.
2. Systems can Evolve Separate from Reporting: Have a system that's end-of-life that you need to implement a replacement for but reports from that system are mission critical to your operations? No problem. With all the data copied into the Data Platform, reporting and analysis can continue to function without impact and reports can be updated to pull in data from both the prior and new system in a unified manner.
3. Not Limited by Vendor Capabilities: Ever been shopping for a solution to a problem on your factory floor only to get the disappointing news it doesn't integrate your ERP or some other system you'd want to use it with? By owning your own Data Platform you are no longer limited by this. You can create your own integration between this solution and the Data Platform so the exact info you want is pulled and then you're able to query it with all the data across the business.
4. Alerting Capabilities: Alerting is a key operational need to rapidly respond to high risk scenarios. With a unified data platform alerting can be done across the business so the right people are made aware immediately when data comes in showing concerning patterns.
5. Can Efficiently Scale to Large Data Volumes: Data Platforms should be designed to easily handle hundreds of petabytes of data (a petabyte is a thousand terabytes). Solutions like Google Cloud Platform's BigQuery costs a third of a penny ($0.003) per month for a gigabyte of data with an average storage compression ratio. That means 100 TB of data only costs ~$300/month to store in BigQuery with other Data Platform options having similar costs.

Conclusion: It's a Skillset, Not a Purchase ​

Through this article we've looked at the data challenges manufacturers face, the root cause of it, and the paradigm shift needed to realize the benefits of this new data rich phase of the manufacturing industry. The one point to reinforce in closing is that this data capability is not a purchase but a skillset. Regardless of how what vendors you use or technologies you integrate with, the core path forward is having data capabilities as a skillset for your business. That skillset can be something you either (1) staff yourself if you have strong technology capabilities or (2) work with a long term service partner that has it as a core skillset. Either way, this is how manufacturers can face their current data dilemma and regain control of the data centric world they've bought into but struggled to realize the benefits of.

If you found this useful then share it, post it, and send it to others. The world needs better data experiences. And if you want to go deeper or discuss the exact problems you're facing, don't hesitate to talk with us — we always enjoy discussing interesting problems.

Thanks and hope this helps make data better for your life and those you work with!