Why "Cloud-Only" Strategies Fail for Mission-Critical Workloads

 

The Future Isn't Cloud or Mainframe—It's Cloud and Mainframe

For more than a decade, many organizations have pursued what sounded like an irresistible vision: move everything to the cloud. The promise was compelling—lower costs, greater agility, virtually unlimited scalability, and freedom from maintaining expensive on-premises infrastructure. Industry analysts proclaimed that the data center was dead, and vendors eagerly promoted "cloud-first" and, eventually, "cloud-only" strategies as the inevitable future of enterprise computing.

Today, reality is beginning to catch up with the marketing. An increasing number of organizations are discovering that a cloud-only strategy works quite well for some workloads, but not for all of them. In particular, mission-critical operational systems continue to demand characteristics that are difficult, expensive, or simply impractical to reproduce entirely in the cloud.

The issue isn't whether the cloud is valuable. It certainly is. The real question is whether it should become the exclusive home for every enterprise workload. For many organizations, the answer is no.

Mission-Critical Means Different Things

One of the biggest mistakes I see is treating every application as though it has identical requirements. Of course, they don't. A marketing website has very different availability requirements than a bank's transaction processing system. A departmental reporting application is not held to the same standards as an airline reservation system. An internal collaboration tool does not carry the same consequences as a healthcare claims processing application.

Mission-critical systems typically require:

• Continuous availability
• Extremely high transaction throughput
• Consistent response times
• ACID transaction integrity
• Comprehensive security
• Extensive auditability
• Predictable operational behavior
• Recovery measured in seconds—not hours

These characteristics have defined enterprise computing for decades.

They're also the reason IBM Z, Db2, IMS, and CICS continue to run some of the world's largest businesses.

Reliability Is an Architectural Property

One misconception is that reliability can simply be purchased. Experienced users know that it can't. Reliability emerges from architecture, operational discipline, software maturity, governance, and decades of engineering refinement.

Mainframe systems have evolved specifically to deliver extraordinary levels of availability while processing millions of business transactions every day. And that is no accident! It is the result of engineering decisions that have been refined over generations of hardware and software.

Simply moving an application to the cloud does not automatically transfer those architectural qualities.

Latency Matters More Than People Think

Cloud advocates often focus on compute capacity. But mission-critical applications care just as much about latency. It is not uncommon for financial transactions, for example, to involve multiple application servers, databases, authentication services, security components, messaging systems, and external interfaces. 

Adding unnecessary network hops introduces delay. Sometimes that delay is insignificant, but sometimes it isn't.

When thousands of transactions occur every second, small increases in latency accumulate into measurable business costs.

The Economics Are Often Misunderstood

Cloud providers have become extraordinarily efficient. But that doesn't mean every workload becomes less expensive after migration. Consider:

• Operational databases generate enormous volumes of I/O.
• Large enterprises execute billions of SQL statements.
• Data is replicated.
• Backups are created.
• Logs are archived.
• Storage grows.
• Network traffic increases.

High availability configurations multiply infrastructure requirements. And organizations often discover that the monthly operating expense exceeds what they originally projected.

This is one reason cloud repatriation has become an increasingly common topic. Some workloads simply cost less to operate where they already run efficiently.

Data Gravity Is Real

One of my recurring themes is data gravity. Applications can move relatively easily but it is not always that easy for data. Operational databases become deeply integrated with hundreds or even thousands of applications. Moving those databases affects:

• Batch processing
• Replication
• CDC pipelines
• Reporting
• AI platforms
• Security controls
• Regulatory compliance
• Disaster recovery
• Data governance

In other words, the database is rarely an isolated component. It sits at the center of an enterprise ecosystem.

Ignoring that reality leads to expensive modernization projects that deliver far less value than expected.

AI Is Changing the Conversation

Ironically, artificial intelligence has strengthened the business case for retaining mission-critical operational systems. Why is that so? Well, it is because AI depends upon trusted data.

Large language models are becoming commodities... trusted enterprise information is not. Organizations increasingly recognize that their operational databases represent their most valuable information assets.

Db2 databases running on IBM Z often contain decades of highly governed, audited, transactionally consistent business information. The same can be said for IMS databases, which yes, still exist (though are less common).

The trusted data stored in Db2 and IMS is exactly the kind of data AI needs.

Moving it unnecessarily introduces complexity, synchronization challenges, governance concerns, and additional cost.

Modernization Does Not Mean Migration

Perhaps the most important lesson is this: modernization is not synonymous with migration.

Modernization can include:

• API enablement
• Event streaming
• Change Data Capture
• Hybrid cloud integration
• AI enablement
• REST interfaces
• Data virtualization
• Kubernetes integration

Notice what isn't required. Moving the operational database.

Many organizations are discovering that exposing trusted operational data through modern interfaces produces better business outcomes than relocating the underlying systems.

The Hybrid Enterprise Has Already Won

The enterprise architectures I encounter today rarely resemble the all-or-nothing visions presented a decade ago. Most commonly, they are hybrid. Operational systems remain on platforms where they perform best. Cloud platforms provide elasticity for analytics, AI experimentation, web applications, and new digital services. Streaming technologies connect everything together. APIs expose business capabilities. Data virtualization reduces unnecessary duplication. AI consumes trusted enterprise information regardless of where that information physically resides.

That isn't a compromise... it is a good architecture.

Final Thoughts

Technology decisions should always begin with business requirements not industry trends and hype. The cloud is an extraordinary platform for many workloads. But so is the mainframe!

The objective should never be to force every application into a single architectural model. It should be to place each workload where it delivers the greatest business value while minimizing risk, cost, and operational complexity.

After more than forty years working with enterprise databases, I've learned that the most successful architectures rarely chase the latest trend. Instead, they build upon proven strengths while selectively adopting new technologies where those technologies genuinely improve the business.

For mission-critical workloads, that often means embracing a hybrid strategy that combines the resilience and transactional integrity of IBM Z and Db2 with the flexibility and innovation of cloud services.

That may not be as catchy as "cloud-only," but it is far more likely to succeed.

Don't Forget About IBM Redbooks!

Many of my readers are probably aware of IBM redbooks, but for those who are not, here is a quick introduction to a valuable free technical resource. 

If you've been working with any IBM technology for any period of time you've probably become familiar with the IBM redbook. These are supplemental manuals that IBM employees and customers write and distribute free-of-charge over the Internet at http://www.redbooks.ibm.com/. There are redbooks on just about any technical topic that you might imagine - (and for the many Db2 zealots who follow my blog, 

IBM sponsor redbooks and guides the projects that are put together to write them. Some are written by IBMers but many are written by IBM's customers who are actual technology practitioners. Redbooks are written via residencies. A residency is an intensive, multi-week work effort where small teams explore and document product implementation, integration and operations. Each team is directed by an IBMer from their technical publications group. The team will consist of professionals from IBM field and development, from IBM Business Partners, from customers, and from the marketplace in general. So, you too, can research and write an IBM redbook! Of course, you have to make sure that you have the expertise, willingness, and time to work on the project. 

But more importantly, anyone can enjoy the results. IBM publishes all of its redbooks on the web in Adobe PDF format. And anyone can download any redbook for free! So all you need is an Internet connection and the free Adobe Reader software to start reading redbooks. 

Some of my favorite redbooks over the years have addressed topics like stored procedure development, IDAA, improving availability with parallel sysplex, Db2 13 for z/OS performance (there's usually one of these for each new Db2 version), and even AI and watsonx! 

And don't worry if your shop is not a big blue shop, you still might benefit from perusing the redbooks that are available. Yes, there are many redbooks on Db2, IBM Z, watsonx, WebSphere, CICS, and other IBM software products, but there are also redbooks on Linux, Windows, and other non-IBM software. There are even several SQL Server and Oracle redbooks. And these books are always very high quality - I've never read one that is sub-par... and the price is right. 

So, if you haven't checked out the IBM redbook library before - or even if you have, but not lately - take a moment to click over to the redbook site and search for a topic of your interest. You might be surprised at what is available there... 

Db2 Locking Explained in 90 Seconds

Think your enterprise application is running slow because of "aging hardware" or "insufficient CPU"? Think again. In my experience, I’ve found that the vast majority of performance problems blamed on slow infrastructure are actually caused by a silent, internal killer: locking. Or more precisely, inefficient coding built without an understanding of locking!

Let’s talk about Db2 locking, which is one of the most misunderstood performance topics in enterprise systems.

When Db2 locks a resource, it isn't doing it to annoy you; it’s protecting data integrity. It ensures that concurrent transactions don't overwrite each other’s work or read uncommitted data (preventing dirty reads, non-repeatable reads, and phantoms). But when locks escalate or linger unnecessarily, your entire system can grind to a screeching halt.

If you only have 90 seconds, here is the fundamental loop of a locking problem:

1. A transaction requests a lock: It could be a row lock, a page lock, or a table space lock depending on your bind parameters and data modifications.

2. The lock is held too long: If the application does too much processing before committing, or if the access path is inefficient, the transaction hoards that lock.

3. Other transactions must wait: Subsequent tasks trying to touch (read/update) that same resource enter a lock-wait state.

4. The Bottleneck: Accumulate enough waiting tasks, and you get an enterprise-level bottleneck. To the end-user, it looks like the database has frozen.

Why More Hardware Won't Fix It

The classic knee-jerk reaction to this kind of slowdown is throwing money at the problem: adding more CPU, faster storage, or upgrading the mainframe/server capacity.

But if Transaction A is waiting for Transaction B to release a lock, giving Transaction A a faster processor just means it will arrive at the brick wall a microsecond quicker. It still has to wait. The fix isn't more CPU... it’s changing how your data and code interact.

The True Fixes for Lock Contention

If you want to eliminate locking bottlenecks, focus your tuning efforts on these three pillars:

• Better Indexing: If Db2 has to perform a table space scan because an index is missing, it will lock vastly more data (potentially the entire table space) than if it could precisely target a single row via an index. Good indexes keep your locks small and surgical.

• Shorter Units of Work: Keep your transactions tight. Don't fetch data, perform complex business logic, call external APIs, and then commit. Do your non-database work outside the transaction bounds so the lock is held for milliseconds, not seconds.

• Proper Commit Frequency: If you are running batch processes or massive update loops without regular COMMIT statements, you are building an operational dam. Frequent, structured commits flush the log buffers and release held locks, keeping the data flowing smoothly for concurrent users.

The Bottom Line

If your team isn't actively monitoring lock contention, lock timeouts, and deadlocks using Db2 statistics and traces, you are likely tuning the wrong problem.

The next time someone complains that Db2 is "running slow," step away from the infrastructure metrics. Open your monitoring tools, look at the lock-wait times, and find out who is holding the key to the castle while everyone else is standing outside in the cold.

And if you are looking for more details on locking, I wrote a 17-part series on this blog awhile ago that is still pertinent. You can access it here (this is a link to part 17 that has links to the previous 16 parts, too).

What are your favorite techniques for tracking down elusive lock-waits in Db2? Drop a comment below and let’s discuss!

The Most Overlooked Db2 Performance Metric

 If I had to identify a single most overlooked Db2 performance metric, it would be GETPAGEs.

Many organizations focus on CPU utilization, elapsed time, or buffer pool hit ratios because those metrics are easy to understand and frequently appear on dashboards. But getpages often reveal performance problems long before those other metrics become alarming.

In Db2 for z/OS, a GETPAGE represents a request by Db2 to access a page of data or an index page. If the page is already in the buffer pool, no physical I/O is required. If not, Db2 must read it from disk. Either way, every GETPAGE consumes CPU resources.

Why GETPAGEs Matter

Because every SQL statement generates GETPAGE requests, the number of GETPAGEs is a fundamental measure of how much work Db2 performs to execute a workload. The more pages Db2 must examine to satisfy a query, the more work it performs. Reducing unnecessary GETPAGEs often leads directly to lower CPU consumption and better application performance.

High GETPAGE counts usually indicate one or more of the following:

• Inefficient index usage
• Poor clustering
• Table scans
• Low filter factors
• Redundant index probes
• Accessing more columns or rows than necessary

Even when all pages are found in the buffer pool, millions of unnecessary GETPAGEs still consume CPU.

The Hidden CPU Consumer

I've worked with organizations that upgraded processors because CPU costs continued to rise. After analyzing the workload, we discovered that many critical SQL statements were generating ten or even one hundred times more GETPAGEs than necessary.

A simple index redesign or SQL rewrite reduced GETPAGE counts dramatically. And CPU usage dropped without any hardware changes.

That's why I often tell clients: "Every unnecessary GETPAGE results in costly unneeded work."

Don't Just Count GETPAGEs

The raw number of GETPAGEs by itself does not tell the whole story. Instead, monitor:

• GETPAGEs per transaction
• GETPAGEs per SQL statement
• GETPAGEs per row returned
• Trends over time
• The highest GETPAGE-consuming applications

A workload processing twice as many transactions should naturally generate more GETPAGEs. The real warning sign is when GETPAGEs per transaction steadily increase. That usually means SQL or access paths have degraded.

GETPAGEs Point to Root Causes

Unlike CPU utilization, which tells you that work is occurring, GETPAGEs often explain why.

For example:

Symptom	What High GETPAGEs May Indicate
High CPU	Excessive index or table page accesses
Long elapsed time	Inefficient access path
Buffer pool pressure	Poor locality of reference
Lock contention	Long-running scans
Increased zIIP usage	More work being offloaded, but still excessive overall processing

My Rule of Thumb 👍

After more than four decades tuning Db2 systems, one principle has remained remarkably consistent:

Watch the work, not just the time.

Elapsed time can fluctuate because of concurrency. CPU depends on hardware generation. I/O depends on storage technology.

But GETPAGEs measure the amount of work Db2 is performing. If you reduce the work, you almost always improve performance.

Other Frequently Overlooked Metrics

Although GETPAGEs are my top choice, several other metrics deserve more attention:

• Synchronous read percentage — Indicates how often Db2 must wait for I/O instead of benefiting from prefetch.
• Pages read per GETPAGE — Helps evaluate buffer pool effectiveness.
• Lock suspension time — Often more meaningful than simply counting lock waits.
• Class 3 suspension time — Shows where Db2 is waiting (I/O, locks, logging, etc.).
• RID pool failures — Can reveal access path problems that are otherwise difficult to diagnose.
• Sort overflows — Indicate insufficient sort memory or inefficient SQL.
• Index leaf page split rates — A useful indicator of index maintenance issues and clustering degradation.

If I were teaching a new Db2 performance analyst, I'd spend less time looking at CPU graphs and more time asking, "Why is Db2 doing so much work?" In many cases, the answer begins with GETPAGEs.

Db2 for z/OS Lock Escalation: When Fine-Grained Locking Becomes a Problem

One of the primary goals of Db2 for z/OS locking is to maximize concurrency while maintaining data integrity. In a perfect world, every application would acquire only the locks it needs, hold them for the shortest possible duration, and release them promptly. But the real world is not always perfect. Sometimes an application acquires so many locks that Db2 decides it is more efficient to replace those many locks with a single, larger lock. This process is known as lock escalation.

Lock escalation is one of those Db2 behaviors that every DBA should understand because when it occurs unexpectedly, it can have a significant impact on application performance and availability.

What Is Lock Escalation?

Lock escalation occurs when Db2 replaces numerous row, page, or LOB locks with a single table space or partition lock.

Imagine an application updating hundreds of thousands of rows. Instead of managing and tracking an enormous number of individual locks, Db2 may determine that maintaining all of those locks consumes too much storage and processing overhead. Rather than continue managing thousands of granular locks, Db2 escalates them to a larger lock.

For example:

• 50,000 row locks become one table space lock.
• Thousands of page locks become one partition lock.
• Many LOB locks become a higher-level lock.

From Db2's perspective, lock escalation can reduce lock management overhead. From the application's perspective, however, lock escalation reduces concurrency because other applications may now be blocked from accessing a much larger portion of the data.

Why Does Db2 Escalate Locks?

Db2 lock escalation is generally driven by one of two conditions:

LOCKMAX Threshold Reached

The most common cause is the LOCKMAX parameter. LOCKMAX is set at the tablespace level and it defines the maximum number of page, row, or LOB locks that can be held for a table space or partition before Db2 attempts escalation.

The value can be:

• A specific number (ranging from 0 to 2,147,483,647)
• 0 (means lock escalation is disabled)
• SYSTEM (use subsystem default NUMLKTS)

You can find the value of LOCKMAX in the Db2 Catalog by reviewing the MAXROWS column of SYSIBM.SYSTABLESPACE.

When the threshold is exceeded, Db2 attempts escalation. For example, if LOCKMAX is set to 10,000 and an application acquires its 10,001st lock, Db2 attempts to escalate.

Lock Storage Shortage

Db2 also monitors lock storage consumption.

Even if LOCKMAX is not reached, Db2 may escalate locks when lock storage resources become constrained. This protects the subsystem from excessive lock memory consumption.

In these cases, escalation is a defensive measure designed to preserve overall system stability.

What Happens During Escalation?

Suppose an application holds 25,000 row locks and that is also the value of LOCKMAX. When the next row lock is requested while updating a table Db2 attempts to replace all 25,000 + 1 locks with a higher-level lock, typically:

• Exclusive table space lock for updates
• Share table space lock for read activity

If Db2 successfully acquires the higher-level lock:

1. The individual locks are released.
2. The table space or partition lock is acquired.
3. Processing continues.

The problem is that other applications may now be blocked from accessing data that previously would have remained available through row-level concurrency. A single poorly designed batch job can suddenly become a bottleneck for dozens or hundreds of online transactions.

Why Lock Escalation Can Be Dangerous

Many DBAs think lock escalation is merely a locking event. In reality, it is often an application design warning signal.

Consider a CICS transaction that normally updates ten rows. No issue. Now consider a batch job (running concurrently with the transactions) that updates five million rows under one unit of work. Without frequent commits, the job accumulates massive numbers of locks. Eventually escalation occurs. The consequences may be dire, including:

• Increased lock contention
• Application timeouts
• Deadlocks
• Reduced concurrency
• Unexpected outages for online users

In production environments, lock escalation frequently becomes visible only after users begin reporting delays.

Common Causes

Over the years, I have found that lock escalation is usually symptomatic of one or more underlying issues. And it is usually an application design/coding issue.

Infrequent Commits

Not issuing sufficient (or any COMMITs) is probably the most common cause. Applications that process large volumes of data without committing work accumulate locks continuously. And the locks are not released until a COMMIT is issued (or the program ends).

A batch job committing every 100,000 rows will typically consume far more lock resources than one committing every 1,000 rows.

I have written about Bachelor Programming Syndrome before (check the link), which is just my way of saying don’t fear committing. In general, I recommend that you plan to issue COMMITs in every batch program. You can structure the logic so that the COMMIT processing is contingent on a parameter passed to the program. This approach enables an analyst to modify COMMIT frequency, or even turn off COMMIT processing, as the concurrency needs of the application varies.

Mass Updates and Deletes

Large-scale data modification operations naturally acquire large numbers of locks.

Examples include:

DELETE FROM CUSTOMER_HISTORY
WHERE CREATE_DATE < CURRENT DATE - 5 YEARS;

or

UPDATE ACCOUNT
SET STATUS = 'I'
WHERE LAST_ACTIVITY_DATE < CURRENT DATE - 3 YEARS;

These operations can quickly exceed escalation thresholds.

Poor Access Paths

Inefficient access paths may cause Db2 to examine and lock far more rows or pages than intended. An application expected to update 100 rows might actually scan millions due to a missing or ineffective index.

Excessively Large Units of Work

The larger the unit of work, the greater the lock accumulation. Applications that hold locks for extended periods are prime candidates for escalation. Again, parameterized control of COMMIT frequency makes it easier to manage and optimize concurrency without requiring a program change.

Detecting Lock Escalation

Fortunately, Db2 provides several ways to identify escalation activity. DBAs should monitor:

• IFCID traces
• Db2 statistics reports
• Accounting reports
• Performance monitor alerts
• System messages

A sudden increase in lock waits often points directly to escalation activity. When troubleshooting, examine:

• Which object escalated
• Which application triggered escalation
• COMMIT frequency
• Number of locks acquired
• Access path efficiency

The goal is not merely to identify that escalation occurred, but to determine why.

Preventing Lock Escalation

The best strategy is usually prevention rather than accommodation.

Improve Commit Frequency

Frequent commits reduce lock accumulation.

This is often the single most effective corrective action.

Tune SQL

Efficient SQL accesses fewer pages and rows, reducing lock requirements.

Better indexing and improved access paths often eliminate escalation problems entirely.

Adjust LOCKMAX

In some situations, increasing LOCKMAX may be appropriate.

However, simply raising thresholds without understanding the underlying workload can mask deeper application issues.

Use Partitioning

Partition-level locking can significantly reduce the scope of lock contention.

An escalated partition lock is generally less disruptive than a full table space lock.

Be Cautious with LOCKMAX 0

Setting LOCKMAX to 0 disables lock escalation. This prevents escalation, but it does not eliminate lock consumption.

If applications accumulate excessive locks, other resource constraints may emerge. Therefore, LOCKMAX 0 should be used only after careful analysis.

The DBA Perspective

One lesson I have learned repeatedly is that lock escalation is rarely the root problem. It is usually a symptom. When escalation occurs, Db2 is telling you something important. Specifically:

"This application is holding more locks than I am comfortable managing efficiently."

The correct response is usually not to disable escalation or simply raise thresholds. Instead, investigate the workload. Examine COMMIT frequency. Review SQL efficiency. Analyze access paths. Understand the business process generating the activity.

In many cases, the real solution lies in better application design rather than lock configuration.

Lock escalation exists to protect Db2. But when it appears regularly in your environment, it is often signaling an opportunity to improve performance, scalability, and concurrency. Wise DBAs treat lock escalation not as a nuisance, but as valuable diagnostic information about the health of their applications and workloads.