Don't Forget About IBM Redbooks!

Many of my readers are probably aware of IBM redbooks, but for those who are not, here is a quick introduction to a valuable free technical resource. 

If you've been working with any IBM technology for any period of time you've probably become familiar with the IBM redbook. These are supplemental manuals that IBM employees and customers write and distribute free-of-charge over the Internet at http://www.redbooks.ibm.com/. There are redbooks on just about any technical topic that you might imagine - (and for the many Db2 zealots who follow my blog, 

IBM sponsor redbooks and guides the projects that are put together to write them. Some are written by IBMers but many are written by IBM's customers who are actual technology practitioners. Redbooks are written via residencies. A residency is an intensive, multi-week work effort where small teams explore and document product implementation, integration and operations. Each team is directed by an IBMer from their technical publications group. The team will consist of professionals from IBM field and development, from IBM Business Partners, from customers, and from the marketplace in general. So, you too, can research and write an IBM redbook! Of course, you have to make sure that you have the expertise, willingness, and time to work on the project. 

But more importantly, anyone can enjoy the results. IBM publishes all of its redbooks on the web in Adobe PDF format. And anyone can download any redbook for free! So all you need is an Internet connection and the free Adobe Reader software to start reading redbooks. 

Some of my favorite redbooks over the years have addressed topics like stored procedure development, IDAA, improving availability with parallel sysplex, Db2 13 for z/OS performance (there's usually one of these for each new Db2 version), and even AI and watsonx! 

And don't worry if your shop is not a big blue shop, you still might benefit from perusing the redbooks that are available. Yes, there are many redbooks on Db2, IBM Z, watsonx, WebSphere, CICS, and other IBM software products, but there are also redbooks on Linux, Windows, and other non-IBM software. There are even several SQL Server and Oracle redbooks. And these books are always very high quality - I've never read one that is sub-par... and the price is right. 

So, if you haven't checked out the IBM redbook library before - or even if you have, but not lately - take a moment to click over to the redbook site and search for a topic of your interest. You might be surprised at what is available there... 

Db2 Locking Explained in 90 Seconds

Think your enterprise application is running slow because of "aging hardware" or "insufficient CPU"? Think again. In my experience, I’ve found that the vast majority of performance problems blamed on slow infrastructure are actually caused by a silent, internal killer: locking. Or more precisely, inefficient coding built without an understanding of locking!

Let’s talk about Db2 locking, which is one of the most misunderstood performance topics in enterprise systems.

When Db2 locks a resource, it isn't doing it to annoy you; it’s protecting data integrity. It ensures that concurrent transactions don't overwrite each other’s work or read uncommitted data (preventing dirty reads, non-repeatable reads, and phantoms). But when locks escalate or linger unnecessarily, your entire system can grind to a screeching halt.

If you only have 90 seconds, here is the fundamental loop of a locking problem:

1. A transaction requests a lock: It could be a row lock, a page lock, or a table space lock depending on your bind parameters and data modifications.

2. The lock is held too long: If the application does too much processing before committing, or if the access path is inefficient, the transaction hoards that lock.

3. Other transactions must wait: Subsequent tasks trying to touch (read/update) that same resource enter a lock-wait state.

4. The Bottleneck: Accumulate enough waiting tasks, and you get an enterprise-level bottleneck. To the end-user, it looks like the database has frozen.

Why More Hardware Won't Fix It

The classic knee-jerk reaction to this kind of slowdown is throwing money at the problem: adding more CPU, faster storage, or upgrading the mainframe/server capacity.

But if Transaction A is waiting for Transaction B to release a lock, giving Transaction A a faster processor just means it will arrive at the brick wall a microsecond quicker. It still has to wait. The fix isn't more CPU... it’s changing how your data and code interact.

The True Fixes for Lock Contention

If you want to eliminate locking bottlenecks, focus your tuning efforts on these three pillars:

• Better Indexing: If Db2 has to perform a table space scan because an index is missing, it will lock vastly more data (potentially the entire table space) than if it could precisely target a single row via an index. Good indexes keep your locks small and surgical.

• Shorter Units of Work: Keep your transactions tight. Don't fetch data, perform complex business logic, call external APIs, and then commit. Do your non-database work outside the transaction bounds so the lock is held for milliseconds, not seconds.

• Proper Commit Frequency: If you are running batch processes or massive update loops without regular COMMIT statements, you are building an operational dam. Frequent, structured commits flush the log buffers and release held locks, keeping the data flowing smoothly for concurrent users.

The Bottom Line

If your team isn't actively monitoring lock contention, lock timeouts, and deadlocks using Db2 statistics and traces, you are likely tuning the wrong problem.

The next time someone complains that Db2 is "running slow," step away from the infrastructure metrics. Open your monitoring tools, look at the lock-wait times, and find out who is holding the key to the castle while everyone else is standing outside in the cold.

And if you are looking for more details on locking, I wrote a 17-part series on this blog awhile ago that is still pertinent. You can access it here (this is a link to part 17 that has links to the previous 16 parts, too).

What are your favorite techniques for tracking down elusive lock-waits in Db2? Drop a comment below and let’s discuss!

The Most Overlooked Db2 Performance Metric

 If I had to identify a single most overlooked Db2 performance metric, it would be GETPAGEs.

Many organizations focus on CPU utilization, elapsed time, or buffer pool hit ratios because those metrics are easy to understand and frequently appear on dashboards. But getpages often reveal performance problems long before those other metrics become alarming.

In Db2 for z/OS, a GETPAGE represents a request by Db2 to access a page of data or an index page. If the page is already in the buffer pool, no physical I/O is required. If not, Db2 must read it from disk. Either way, every GETPAGE consumes CPU resources.

Why GETPAGEs Matter

Because every SQL statement generates GETPAGE requests, the number of GETPAGEs is a fundamental measure of how much work Db2 performs to execute a workload. The more pages Db2 must examine to satisfy a query, the more work it performs. Reducing unnecessary GETPAGEs often leads directly to lower CPU consumption and better application performance.

High GETPAGE counts usually indicate one or more of the following:

• Inefficient index usage
• Poor clustering
• Table scans
• Low filter factors
• Redundant index probes
• Accessing more columns or rows than necessary

Even when all pages are found in the buffer pool, millions of unnecessary GETPAGEs still consume CPU.

The Hidden CPU Consumer

I've worked with organizations that upgraded processors because CPU costs continued to rise. After analyzing the workload, we discovered that many critical SQL statements were generating ten or even one hundred times more GETPAGEs than necessary.

A simple index redesign or SQL rewrite reduced GETPAGE counts dramatically. And CPU usage dropped without any hardware changes.

That's why I often tell clients: "Every unnecessary GETPAGE results in costly unneeded work."

Don't Just Count GETPAGEs

The raw number of GETPAGEs by itself does not tell the whole story. Instead, monitor:

• GETPAGEs per transaction
• GETPAGEs per SQL statement
• GETPAGEs per row returned
• Trends over time
• The highest GETPAGE-consuming applications

A workload processing twice as many transactions should naturally generate more GETPAGEs. The real warning sign is when GETPAGEs per transaction steadily increase. That usually means SQL or access paths have degraded.

GETPAGEs Point to Root Causes

Unlike CPU utilization, which tells you that work is occurring, GETPAGEs often explain why.

For example:

Symptom	What High GETPAGEs May Indicate
High CPU	Excessive index or table page accesses
Long elapsed time	Inefficient access path
Buffer pool pressure	Poor locality of reference
Lock contention	Long-running scans
Increased zIIP usage	More work being offloaded, but still excessive overall processing

My Rule of Thumb 👍

After more than four decades tuning Db2 systems, one principle has remained remarkably consistent:

Watch the work, not just the time.

Elapsed time can fluctuate because of concurrency. CPU depends on hardware generation. I/O depends on storage technology.

But GETPAGEs measure the amount of work Db2 is performing. If you reduce the work, you almost always improve performance.

Other Frequently Overlooked Metrics

Although GETPAGEs are my top choice, several other metrics deserve more attention:

• Synchronous read percentage — Indicates how often Db2 must wait for I/O instead of benefiting from prefetch.
• Pages read per GETPAGE — Helps evaluate buffer pool effectiveness.
• Lock suspension time — Often more meaningful than simply counting lock waits.
• Class 3 suspension time — Shows where Db2 is waiting (I/O, locks, logging, etc.).
• RID pool failures — Can reveal access path problems that are otherwise difficult to diagnose.
• Sort overflows — Indicate insufficient sort memory or inefficient SQL.
• Index leaf page split rates — A useful indicator of index maintenance issues and clustering degradation.

If I were teaching a new Db2 performance analyst, I'd spend less time looking at CPU graphs and more time asking, "Why is Db2 doing so much work?" In many cases, the answer begins with GETPAGEs.

Db2 for z/OS Lock Escalation: When Fine-Grained Locking Becomes a Problem

One of the primary goals of Db2 for z/OS locking is to maximize concurrency while maintaining data integrity. In a perfect world, every application would acquire only the locks it needs, hold them for the shortest possible duration, and release them promptly. But the real world is not always perfect. Sometimes an application acquires so many locks that Db2 decides it is more efficient to replace those many locks with a single, larger lock. This process is known as lock escalation.

Lock escalation is one of those Db2 behaviors that every DBA should understand because when it occurs unexpectedly, it can have a significant impact on application performance and availability.

What Is Lock Escalation?

Lock escalation occurs when Db2 replaces numerous row, page, or LOB locks with a single table space or partition lock.

Imagine an application updating hundreds of thousands of rows. Instead of managing and tracking an enormous number of individual locks, Db2 may determine that maintaining all of those locks consumes too much storage and processing overhead. Rather than continue managing thousands of granular locks, Db2 escalates them to a larger lock.

For example:

• 50,000 row locks become one table space lock.
• Thousands of page locks become one partition lock.
• Many LOB locks become a higher-level lock.

From Db2's perspective, lock escalation can reduce lock management overhead. From the application's perspective, however, lock escalation reduces concurrency because other applications may now be blocked from accessing a much larger portion of the data.

Why Does Db2 Escalate Locks?

Db2 lock escalation is generally driven by one of two conditions:

LOCKMAX Threshold Reached

The most common cause is the LOCKMAX parameter. LOCKMAX is set at the tablespace level and it defines the maximum number of page, row, or LOB locks that can be held for a table space or partition before Db2 attempts escalation.

The value can be:

• A specific number (ranging from 0 to 2,147,483,647)
• 0 (means lock escalation is disabled)
• SYSTEM (use subsystem default NUMLKTS)

You can find the value of LOCKMAX in the Db2 Catalog by reviewing the MAXROWS column of SYSIBM.SYSTABLESPACE.

When the threshold is exceeded, Db2 attempts escalation. For example, if LOCKMAX is set to 10,000 and an application acquires its 10,001st lock, Db2 attempts to escalate.

Lock Storage Shortage

Db2 also monitors lock storage consumption.

Even if LOCKMAX is not reached, Db2 may escalate locks when lock storage resources become constrained. This protects the subsystem from excessive lock memory consumption.

In these cases, escalation is a defensive measure designed to preserve overall system stability.

What Happens During Escalation?

Suppose an application holds 25,000 row locks and that is also the value of LOCKMAX. When the next row lock is requested while updating a table Db2 attempts to replace all 25,000 + 1 locks with a higher-level lock, typically:

• Exclusive table space lock for updates
• Share table space lock for read activity

If Db2 successfully acquires the higher-level lock:

1. The individual locks are released.
2. The table space or partition lock is acquired.
3. Processing continues.

The problem is that other applications may now be blocked from accessing data that previously would have remained available through row-level concurrency. A single poorly designed batch job can suddenly become a bottleneck for dozens or hundreds of online transactions.

Why Lock Escalation Can Be Dangerous

Many DBAs think lock escalation is merely a locking event. In reality, it is often an application design warning signal.

Consider a CICS transaction that normally updates ten rows. No issue. Now consider a batch job (running concurrently with the transactions) that updates five million rows under one unit of work. Without frequent commits, the job accumulates massive numbers of locks. Eventually escalation occurs. The consequences may be dire, including:

• Increased lock contention
• Application timeouts
• Deadlocks
• Reduced concurrency
• Unexpected outages for online users

In production environments, lock escalation frequently becomes visible only after users begin reporting delays.

Common Causes

Over the years, I have found that lock escalation is usually symptomatic of one or more underlying issues. And it is usually an application design/coding issue.

Infrequent Commits

Not issuing sufficient (or any COMMITs) is probably the most common cause. Applications that process large volumes of data without committing work accumulate locks continuously. And the locks are not released until a COMMIT is issued (or the program ends).

A batch job committing every 100,000 rows will typically consume far more lock resources than one committing every 1,000 rows.

I have written about Bachelor Programming Syndrome before (check the link), which is just my way of saying don’t fear committing. In general, I recommend that you plan to issue COMMITs in every batch program. You can structure the logic so that the COMMIT processing is contingent on a parameter passed to the program. This approach enables an analyst to modify COMMIT frequency, or even turn off COMMIT processing, as the concurrency needs of the application varies.

Mass Updates and Deletes

Large-scale data modification operations naturally acquire large numbers of locks.

Examples include:

DELETE FROM CUSTOMER_HISTORY
WHERE CREATE_DATE < CURRENT DATE - 5 YEARS;

or

UPDATE ACCOUNT
SET STATUS = 'I'
WHERE LAST_ACTIVITY_DATE < CURRENT DATE - 3 YEARS;

These operations can quickly exceed escalation thresholds.

Poor Access Paths

Inefficient access paths may cause Db2 to examine and lock far more rows or pages than intended. An application expected to update 100 rows might actually scan millions due to a missing or ineffective index.

Excessively Large Units of Work

The larger the unit of work, the greater the lock accumulation. Applications that hold locks for extended periods are prime candidates for escalation. Again, parameterized control of COMMIT frequency makes it easier to manage and optimize concurrency without requiring a program change.

Detecting Lock Escalation

Fortunately, Db2 provides several ways to identify escalation activity. DBAs should monitor:

• IFCID traces
• Db2 statistics reports
• Accounting reports
• Performance monitor alerts
• System messages

A sudden increase in lock waits often points directly to escalation activity. When troubleshooting, examine:

• Which object escalated
• Which application triggered escalation
• COMMIT frequency
• Number of locks acquired
• Access path efficiency

The goal is not merely to identify that escalation occurred, but to determine why.

Preventing Lock Escalation

The best strategy is usually prevention rather than accommodation.

Improve Commit Frequency

Frequent commits reduce lock accumulation.

This is often the single most effective corrective action.

Tune SQL

Efficient SQL accesses fewer pages and rows, reducing lock requirements.

Better indexing and improved access paths often eliminate escalation problems entirely.

Adjust LOCKMAX

In some situations, increasing LOCKMAX may be appropriate.

However, simply raising thresholds without understanding the underlying workload can mask deeper application issues.

Use Partitioning

Partition-level locking can significantly reduce the scope of lock contention.

An escalated partition lock is generally less disruptive than a full table space lock.

Be Cautious with LOCKMAX 0

Setting LOCKMAX to 0 disables lock escalation. This prevents escalation, but it does not eliminate lock consumption.

If applications accumulate excessive locks, other resource constraints may emerge. Therefore, LOCKMAX 0 should be used only after careful analysis.

The DBA Perspective

One lesson I have learned repeatedly is that lock escalation is rarely the root problem. It is usually a symptom. When escalation occurs, Db2 is telling you something important. Specifically:

"This application is holding more locks than I am comfortable managing efficiently."

The correct response is usually not to disable escalation or simply raise thresholds. Instead, investigate the workload. Examine COMMIT frequency. Review SQL efficiency. Analyze access paths. Understand the business process generating the activity.

In many cases, the real solution lies in better application design rather than lock configuration.

Lock escalation exists to protect Db2. But when it appears regularly in your environment, it is often signaling an opportunity to improve performance, scalability, and concurrency. Wise DBAs treat lock escalation not as a nuisance, but as valuable diagnostic information about the health of their applications and workloads.

 

Concurrency vs. Throughput in Db2

As Db2 DBAs we often find ourselves chasing two elusive targets: speed and capacity. We want our systems to handle everything all at once, and we want it done yesterday. But as database architectures evolve and workloads grow increasingly complex, DBAs must constantly manage the delicate, often misunderstood relationship between two fundamental metrics: Concurrency and Throughput.

It’s easy to mistake one for the other, or to assume that maximizing one automatically boosts the other. But the two are not the same. Let’s break down what these terms really mean for your Db2 for z/OS subsystems and why balancing them is the key to a healthy production environment.

Defining the Duo

Before we look at how they interact, let’s establish a clear baseline for both concepts.

• Concurrency is the database's ability to handle multiple interactive sessions, transactions, or users at the exact same time. It’s a measure of simultaneous access. Think of it as the number of lanes on a highway. More lanes typically mean that more cars can be on the road simultaneously.

• Throughput is the actual amount of work successfully completed by the database per unit of time (e.g., transactions per second, rows processed per minute). If concurrency is the number of lanes on the highway, throughput is the number of cars that actually pass through the toll booth every hour.

In an ideal world, as concurrency increases, throughput rises right along with it. But database systems don't operate in a vacuum. They are bound by physical limitations: CPU, memory, I/O bandwidth, and, most importantly, locking and latching mechanisms.

The Concurrency Curve: When More Becomes Less

If you increase the number of concurrent threads entering your Db2 subsystem, throughput will generally scale linearly, at least up to a point. But eventually you hit a tipping point that is referred to as the "knee of the curve."

Beyond this point, adding more concurrent users doesn't get more work done. Instead, it breeds contention.

When too many transactions fight for the same resources, Db2 spends more time managing the queue than doing actual work. You’ll start to see symptoms like:

• High internal latch contention.

• Increased lock wait times and, worst-case scenario, spikes in deadlocks and timeouts.

• Elongated class 3 suspension times in your accounting reports.

At this stage, you haven't plateaued your throughput... you’ve actually degraded it. You are burning CPU just to manage the traffic jam.

Db2 Mechanisms to Balance the Scale

Maximizing throughput while maintaining healthy concurrency requires an understanding of Db2’s internal mechanics. Here are the core areas where DBAs can tip the scales back in their favor:

1. Lock Avoidance and Isolation Levels

Locks are the primary gatekeepers of concurrency. The best way to improve concurrency is to avoid locking altogether when safe. Ensure your application packages are bound with CURRENTDATA(NO) to allow Db2 to use lock avoidance techniques. Furthermore, when appropriate, consider dirty reads. If an application can tolerate reading uncommitted data, UNCOMMITTED READ (UR) blows the doors wide open for concurrency because it doesn't acquire read locks.

Note: it has been my experience that far too many programs and SQL statements use UR. Although removing locks with UR improves concurrency, it can damage data quality if used inappropriately.

2. Thread Management (MAX REMOTE ACTIVE / MAX ONLINE)

It is not an uncommon misconception that letting every single request hit the engine at once improves throughput. It really doesn’t. Leveraging Db2’s thread pooling and setting intelligent limits on MAX REMOTE ACTIVE (for distributed workloads via DDF) ensures that work is queued efficiently before it can thrash the engine.

3. Page-Level vs. Row-Level Locking

Row-level locking (LOCKSIZE ROW) sounds like a silver bullet for concurrency because it minimizes the footprint of a lock. However, it comes with a steep price tag: you might experience considerable CPU overhead for lock acquisition and management. If your throughput is CPU-bound, using LOCKSIZE PAGE (combined with smart page splitting and small row sizes) might actually increase your overall throughput by freeing up CPU cycles.

The trade-off between row and page locks depends on understanding the nature of the application accessing the data and its current execution profile. If there are contention problems when accessing a table space that is currently set to LOCKSIZE PAGE you might consider altering it to LOCKSIZE ROW and then monitoring the impact on performance, resource consumption, and concurrency.

The Bottom Line

Mainframes are designed to process massive, mind-boggling volumes of concurrent data better than almost anything else on earth. But the laws of database physics still apply.

To achieve maximum throughput, you cannot simply crank the concurrency dial to eleven and hope for the best. You must monitor your buffer pools, optimize your indexing to minimize table scans, design your applications for short commit scopes, and precisely configure your subsystem parameters.

Remember: True database performance isn't about how many transactions you let through the door at once; it's about how fast you can successfully get them out the exit.