The Cost of “Good Enough” Data: Why the Fundamentals Still Matter

My latest book, The Cost of “Good Enough” Data: Why Modern Architectures Fail at Scale, examines a problem that has been building quietly across enterprise IT for years: we have become increasingly willing to accept data that is merely “good enough.” And that approach is becoming increasingly expensive.

Organizations today are investing heavily in AI, analytics, cloud computing, data lakes, data lakehouses, streaming platforms, and increasingly complicated data pipelines. Yet underneath all of this technology lies something much more fundamental: the data itself. If that data is incomplete, inconsistent, duplicated, poorly governed, stale, or inadequately understood, no amount of sophisticated technology layered on top of it can completely compensate. Indeed, AI has changed the economics of data quality because errors that once affected a report or dashboard can now propagate through automated recommendations, decisions, and business processes.

This is where the mainframe perspective becomes particularly interesting.

Those of us who have spent our careers working with Db2 for z/OS and other mainframe technologies grew up in an environment where data management was taken very seriously. Mission-critical systems could not routinely tolerate questionable integrity, undocumented changes, unreliable recovery, uncontrolled access, or sloppy transaction processing. The mainframe disciplines surrounding ACID transactions, database administration, security, recovery, change management, performance engineering, and operational control evolved because the workloads demanded them.

Of course, mainframe environments are not perfect. Bad designs, poor SQL, inadequate governance, and questionable data can exist anywhere. But there is something important to learn from the engineering culture that developed around systems of record.

Accuracy matters. Consistency matters. Recovery matters. Lineage matters. Governance matters. And somebody has to be accountable for the data.

As enterprises distributed their data across warehouses, lakes, cloud platforms, SaaS applications, analytical databases, streaming systems, and countless pipelines, some of that discipline was weakened or lost. Copies multiplied. Ownership became less clear. Transformations accumulated. Metadata became inconsistent. And organizations accumulated what I refer to as data debt — the growing cost and risk created by compromises that seemed reasonable when they were originally made. The book examines how that debt can undermine AI initiatives, increase operational costs, introduce compliance risk, and impair business decision-making.

AI is now exposing that debt.

An AI model does not magically know which customer record is authoritative, whether replicated Db2 data is current, whether a transformation changed the meaning of a field, or whether a piece of information satisfies the organization's governance requirements. It consumes what the data infrastructure provides.

And this brings us back to the mainframe.

For many large enterprises, some of their most valuable, accurate, and carefully controlled business data still resides in Db2 for z/OS and other IBM Z systems. That should not be viewed as an impediment to modernization. Quite the opposite. Trusted systems of record can become an enormously valuable foundation for analytics and AI — provided organizations build architectures that preserve the integrity, context, security, and governance of that data as it moves into newer environments.

That is really one of the central themes of The Cost of “Good Enough” Data. The book is not an argument against the cloud, or any of the newer distributed architectures. It is an argument against complacency. The technology has changed, the scale has changed, and the pace has certainly changed, but the fundamental principles of sound data management have not.

Modernization should not mean abandoning the disciplines we learned building reliable enterprise systems. It should mean applying those disciplines to modern architectures.

Because whether your data resides in Db2 for z/OS, PostgreSQL, a cloud database, a data lakehouse, or all of them at once, the same basic truth applies:

If the data cannot be trusted, neither can anything you build with it.

And in the age of AI, settling for “good enough” data may be more costly than ever.

Why "Cloud-Only" Strategies Fail for Mission-Critical Workloads

 

The Future Isn't Cloud or Mainframe—It's Cloud and Mainframe

For more than a decade, many organizations have pursued what sounded like an irresistible vision: move everything to the cloud. The promise was compelling—lower costs, greater agility, virtually unlimited scalability, and freedom from maintaining expensive on-premises infrastructure. Industry analysts proclaimed that the data center was dead, and vendors eagerly promoted "cloud-first" and, eventually, "cloud-only" strategies as the inevitable future of enterprise computing.

Today, reality is beginning to catch up with the marketing. An increasing number of organizations are discovering that a cloud-only strategy works quite well for some workloads, but not for all of them. In particular, mission-critical operational systems continue to demand characteristics that are difficult, expensive, or simply impractical to reproduce entirely in the cloud.

The issue isn't whether the cloud is valuable. It certainly is. The real question is whether it should become the exclusive home for every enterprise workload. For many organizations, the answer is no.

Mission-Critical Means Different Things

One of the biggest mistakes I see is treating every application as though it has identical requirements. Of course, they don't. A marketing website has very different availability requirements than a bank's transaction processing system. A departmental reporting application is not held to the same standards as an airline reservation system. An internal collaboration tool does not carry the same consequences as a healthcare claims processing application.

Mission-critical systems typically require:

• Continuous availability
• Extremely high transaction throughput
• Consistent response times
• ACID transaction integrity
• Comprehensive security
• Extensive auditability
• Predictable operational behavior
• Recovery measured in seconds—not hours

These characteristics have defined enterprise computing for decades.

They're also the reason IBM Z, Db2, IMS, and CICS continue to run some of the world's largest businesses.

Reliability Is an Architectural Property

One misconception is that reliability can simply be purchased. Experienced users know that it can't. Reliability emerges from architecture, operational discipline, software maturity, governance, and decades of engineering refinement.

Mainframe systems have evolved specifically to deliver extraordinary levels of availability while processing millions of business transactions every day. And that is no accident! It is the result of engineering decisions that have been refined over generations of hardware and software.

Simply moving an application to the cloud does not automatically transfer those architectural qualities.

Latency Matters More Than People Think

Cloud advocates often focus on compute capacity. But mission-critical applications care just as much about latency. It is not uncommon for financial transactions, for example, to involve multiple application servers, databases, authentication services, security components, messaging systems, and external interfaces. 

Adding unnecessary network hops introduces delay. Sometimes that delay is insignificant, but sometimes it isn't.

When thousands of transactions occur every second, small increases in latency accumulate into measurable business costs.

The Economics Are Often Misunderstood

Cloud providers have become extraordinarily efficient. But that doesn't mean every workload becomes less expensive after migration. Consider:

• Operational databases generate enormous volumes of I/O.
• Large enterprises execute billions of SQL statements.
• Data is replicated.
• Backups are created.
• Logs are archived.
• Storage grows.
• Network traffic increases.

High availability configurations multiply infrastructure requirements. And organizations often discover that the monthly operating expense exceeds what they originally projected.

This is one reason cloud repatriation has become an increasingly common topic. Some workloads simply cost less to operate where they already run efficiently.

Data Gravity Is Real

One of my recurring themes is data gravity. Applications can move relatively easily but it is not always that easy for data. Operational databases become deeply integrated with hundreds or even thousands of applications. Moving those databases affects:

• Batch processing
• Replication
• CDC pipelines
• Reporting
• AI platforms
• Security controls
• Regulatory compliance
• Disaster recovery
• Data governance

In other words, the database is rarely an isolated component. It sits at the center of an enterprise ecosystem.

Ignoring that reality leads to expensive modernization projects that deliver far less value than expected.

AI Is Changing the Conversation

Ironically, artificial intelligence has strengthened the business case for retaining mission-critical operational systems. Why is that so? Well, it is because AI depends upon trusted data.

Large language models are becoming commodities... trusted enterprise information is not. Organizations increasingly recognize that their operational databases represent their most valuable information assets.

Db2 databases running on IBM Z often contain decades of highly governed, audited, transactionally consistent business information. The same can be said for IMS databases, which yes, still exist (though are less common).

The trusted data stored in Db2 and IMS is exactly the kind of data AI needs.

Moving it unnecessarily introduces complexity, synchronization challenges, governance concerns, and additional cost.

Modernization Does Not Mean Migration

Perhaps the most important lesson is this: modernization is not synonymous with migration.

Modernization can include:

• API enablement
• Event streaming
• Change Data Capture
• Hybrid cloud integration
• AI enablement
• REST interfaces
• Data virtualization
• Kubernetes integration

Notice what isn't required. Moving the operational database.

Many organizations are discovering that exposing trusted operational data through modern interfaces produces better business outcomes than relocating the underlying systems.

The Hybrid Enterprise Has Already Won

The enterprise architectures I encounter today rarely resemble the all-or-nothing visions presented a decade ago. Most commonly, they are hybrid. Operational systems remain on platforms where they perform best. Cloud platforms provide elasticity for analytics, AI experimentation, web applications, and new digital services. Streaming technologies connect everything together. APIs expose business capabilities. Data virtualization reduces unnecessary duplication. AI consumes trusted enterprise information regardless of where that information physically resides.

That isn't a compromise... it is a good architecture.

Final Thoughts

Technology decisions should always begin with business requirements not industry trends and hype. The cloud is an extraordinary platform for many workloads. But so is the mainframe!

The objective should never be to force every application into a single architectural model. It should be to place each workload where it delivers the greatest business value while minimizing risk, cost, and operational complexity.

After more than forty years working with enterprise databases, I've learned that the most successful architectures rarely chase the latest trend. Instead, they build upon proven strengths while selectively adopting new technologies where those technologies genuinely improve the business.

For mission-critical workloads, that often means embracing a hybrid strategy that combines the resilience and transactional integrity of IBM Z and Db2 with the flexibility and innovation of cloud services.

That may not be as catchy as "cloud-only," but it is far more likely to succeed.