If your organization uses a mainframe or you are interested in modern mainframe computing issues, be sure to register for and join me in my webinar for GT Software, titled Mainframe Modernization: The Why and How. on Tuesday, October 29, 2019 from 12:00 PM - 1:00 PM CDT.
This webinar will discuss the rich heritage of the mainframe and the value of the applications and systems that have been written over many decades. Organizations rely on these legacy systems and the business knowledge built into these applications drive their businesses.
But an application created 20 or more years ago will not be as accessible to modern users as it should be. Digital transformation that enables users to access applications and data quickly is the norm, but this requires modernizing access to the rich data and processes on the mainframe.
This presentation will expose the value proposition of the mainframe, and look at the trends driving its usage and capabilities. I will look at the IT infrastructure challenges including changing technology, cloud adoption, legacy applications, and development trends. And look at tactics to achieve mainframe modernization amid complexity and change.
So if mainframes are your thing, or you just want to learn more about the state of the modern mainframe, be sure to sign up and attend!
Monday, October 14, 2019
Tuesday, September 17, 2019
IBM Unleashes the z15 Mainframe
In New York City, on September 12, 2019, IBM announced the
latest and greatest iteration of its Z systems mainframe computing platform,
the IBM z15. And I was lucky enough to be there for the unveiling.
The official IBM announcement letter can be found here if you want to dive into the details. But before you go there, consier first reading what I have to say about it below.
Before going any further, here I am with the new z15 in New York… don’t we make a
handsome couple?
The event was held at 3 World Trade Center in lower Manhattan. Ross Mauri, General Manager of IBM Z, kicked off the event extolling the unprecedented security delivered by the z15 with encryption everywhere and the data privacy passports. He claims that the IBM z15 is the most secure platform you can get, and the new capabilities back that up. Mauri also acknowledged that "there's always the next big thing in technology" but stated that "IBM is innovating and leading by anticipating customer needs to ensure the on-going relevance of the mainframe."
And there is a lot to like about the new IBM z15 platform,
both for long-time users and those embracing the platform for new development.
IBM is embracing the multicloud
approach and reminding everybody that the mainframe is a vital component of multicloud
for many organizations.
But modern
infrastructure with the latest application development techniques is not as
simple as throw out the old and bring in the new. I mean, let’s face it, if you
have a mainframe with possibly hundreds or thousands of man years of work
invested in it, are you really going to take the time to re-code all of that
mission-critical work just to have it on a “new” platform? Rewriting
applications that work today cannot be the priority for serious businesses!
Especially when the modern mainframe is as new as it gets, runs all of that
legacy code that runs your business, and also supports new cloud apps and development,
too.
The IBM Z
works perfectly as a part of your multicloud development strategy. The cloud
promises an open, flexible world. But your most critical workloads also need to
run securely and without interruption. To accomplish both objectives you must support
cloud with an underlying IT infrastructure. And for Fortune 500
companies and other large organizations, the multicloud includes the mainframe
as part of the enabling infrastructure.
What’s
New
The new
IBM z15 is housed in a convenient 19 inch rack, and that means it can be
integrated into a standard rack. So you get all the benefit and strengths of
the mainframe while fitting into the size expected by a standard data center.
Did you know that there are more transistors in the new
IBM z15 chip than there are people in the world! Inside the IBM z15
processor chip, there are 15.6 miles of wires, 9.2 billion transistors and 26.2
billion wiring connections — all of which allow a single z15 server to process
1 trillion web transactions per day.
The
mainframe is the ideal platform for many organizations. It provides the
resiliency, security, and agility needed to power, secure, and integrate your
hybrid cloud. And it capably, securely, and efficiently runs your
transactions and the batch workload required to keep your business humming. IBM
used to talk about five 9s of availability (that is 99.999%) but with the new
IBM z15, IBM can deliver seven 9s (that is 99.99999%)! That is 3.16 seconds of
downtime per year, or only 60.48 milliseconds of downtime per week. Now that is
impressive!
The primary new features that are worth your time to investigate further, and that were highlighted by IBM at the kickoff
event are:
- Encryption everywhere which protects your data anywhere, even after it leaves your system, with new IBM Data Privacy Passports, which delivers privacy by policy.
- Cloud native development that simplifies life for developers as they build and modernize applications using standard tools, including new support for Red Hat OpenShift. This enables you to both modernize the apps you have and to deploy new ones using the tools of your choice.
- IBM Z Instant Recovery can reduce the impact of planned and unplanned downtime. Instant Recovery can speed the return to your pre-shutdown SLAs by up to 2x.
The flexibility of the z15 is
noteworthy, too. The new IBM z15 provides the flexibility to implement 1 frame...
or up to 4 frames, as your capacity needs dictate.
And did you know it can run multiple operating systems, not just
z/OS? The IBM Z platform can run z/OS, Linux on Z, z/VM, z/VSE, and z/TPF. This
enables organizations to run legacy applications and modern, specialist ones
using the operating system of their choice. Indeed, convenience and flexibility
are hallmarks of the IBM Z platform.
The IBM z15 is a modern platform for all of your processing needs. And that is backed up not just by IBM, but also a brand new survery from BMC Software, in their 14th annual mainframe survey for 2019. The survey shows that 93% are confident in the combined long-term and new workload strength of the IBM Z platform, the strongest showing since 2013! Other highlights inlcude a majority thinking that mainframe growth will continue, along with increasing MIPS/MSU consumption... not to mention that the mainframe is handling increases in data volume, number of databases, and transaction volume. If you are working with mainframes in any way, be sure to check out the new BMC Mainframe Survey.
Indeed, with the new IBM z15 things are looking great for the mainframe and those that rely upon it to power their digital business.
Wednesday, September 04, 2019
The Power of Data Masking for Data Protection
Data privacy regulations and the desire to
protect sensitive data requires methods to mask production data for test
purposes. Data masking tools create structurally similar data that is not the
same as the actual data, but can be used by application systems the same way as
the actual data. The capability to mask data is important to be in compliance
with regulations like GDPR and PCI-DSS, which place restrictions on how personally identifiable information (PII) can
be used.
But I also wanted to
share a new video produced by UBS Hainer that explains how data masking can help
you to stay compliant and protect your sensitive data. It is well worth your
time to watch this 2 minute video if you need to better address the protection
of sensitive data at your shop.
Data masking is not a simple task, and as the video helps to explain, there is much to consider. To effectively mask your data requires a well-thought-out process and method for implementation to achieve success. As such, a tool like BCV5 Masking Tool can simplify how you address your Db2 data protection requirements. It provides dozens of easy to use masking algorithms implemented using Db2 user-defined functions. It ensures that the same actual value is translated to the same masked value every time. And the value will be a plausible value that works the same as the data it is masking. The tool understands thing like referential integrity, unique constraints, related data, and so on.
A reliable method of automating the process of
data masking that understands all of the complicated issues and solves them is
clearly needed. And this where UBS Hainer’s BCV5 Masking Tool excels.
UBS Hainer’s Masking Tool
for BCV5 (their test data management solution) offers robust masking of Db2 for
z/OS data. I wrote about this capability previously on the blog last year (see Data
Masking: An Imperative for Compliance and Governance, November 12, 2018), and
if you are looking for a concise, yet thorough overview of the product’s data
masking capabilities I point you to that blog post.
So why am I talking
about data masking again? Well, it is a thorny problem that many organizations
are still struggling with. As much as 80% of sensitive data resides in environments
used for development, testing, and reporting. That is a lot of data that is
ripe for exposure.
Click to watch the video |
Data masking is not a simple task, and as the video helps to explain, there is much to consider. To effectively mask your data requires a well-thought-out process and method for implementation to achieve success. As such, a tool like BCV5 Masking Tool can simplify how you address your Db2 data protection requirements. It provides dozens of easy to use masking algorithms implemented using Db2 user-defined functions. It ensures that the same actual value is translated to the same masked value every time. And the value will be a plausible value that works the same as the data it is masking. The tool understands thing like referential integrity, unique constraints, related data, and so on.
Thursday, August 15, 2019
BMC AMI for DevOps Intelligently Integrates Db2 for z/OS Schema Changes
Organizations of all types and sizes have adopted a DevOps approach to building applications because it effectively implements small and frequent code changes using agile development techniques. This approach can significantly improve the time to value for application development. The DevOps approach is quite mature on distributed platforms, but it is also gaining traction on the mainframe.
As mainframe development teams begin to rely on DevOps practices more extensively, the need arises to incorporate Db2 for z/OS database changes. This capacity has been lacking until recently, requiring manual intervention by the DBA team to analyze and approve schema changes. This, of course, slows things down, the exact opposite of the desired impact of DevOps. But now BMC has introduced a new solution that brings automated Db2 schema changes to DevOps, namely BMC AMI for DevOps.
BMC AMI for DevOps is designed to integrate into the DevOps tooling that your developers are already using. It integrates with the Jenkins Pipeline tool suite to provide an automated method of receiving, analyzing, and implementing Db2 schema changes as part of an application update.
By integrating with your application orchestration tools AMI for DevOps can capture the necessary database changes required to move from test to production. But it does not just apply these changes; it enforces and ensures best practices using built-in intelligence and automated communication between development and database administration.
The ability to enforce best practices is driven by BMC’s Automated Mainframe Intelligence (AMI), which is policy driven. The AMI capability builds much of the DBA oversight for schema changes into the DevOps pipeline, enforcing database design best practices as you go instead of requiring in-depth manual DBA oversight.
Incorporating a database design advisory capability into the process offloads manual, error-prone tasks to the computer. This integrated automation enables automatic evaluation of Db2 database schema change requests to streamline the DBA approval process and remove the manual processes that inhibit continuous delivery of application functionality.
Furthermore, consider that intelligent database administration functionality can be used to help alleviate the loss of expertise resulting from an aging, retiring workforce. This is a significant challenge for many organizations in the mainframe world.
But let’s not forget the developers. The goal of adopting a DevOps approach on the mainframe is to speed up application development, but at the same time it is important that we do not forgo the safeguards built into mainframe development and operations. So you need a streamlined DevOps process—powered by intelligent automation—in which application developers do not have to wait around for DBA reviews and responses. A self-service model with built-in communication and intelligence such as provided by AMI for DevOps delivers this capability.
The Bottom Line
BMC AMI for DevOps helps you to bring DevOps to the mainframe by integrating Db2 for z/OS schema changes into established and existing DevOps orchestration processes. This means you can use BMC AMI for DevOps to deliver the speed of development required by agile techniques used for modern application delivery without abandoning the safeguards required by DBAs to assure the accuracy of the database changes for assuring availability and reliability of the production system. And developers gain more self-service capability for Db2 schema changes using a well-defined pipeline process.
As mainframe development teams begin to rely on DevOps practices more extensively, the need arises to incorporate Db2 for z/OS database changes. This capacity has been lacking until recently, requiring manual intervention by the DBA team to analyze and approve schema changes. This, of course, slows things down, the exact opposite of the desired impact of DevOps. But now BMC has introduced a new solution that brings automated Db2 schema changes to DevOps, namely BMC AMI for DevOps.
BMC AMI for DevOps is designed to integrate into the DevOps tooling that your developers are already using. It integrates with the Jenkins Pipeline tool suite to provide an automated method of receiving, analyzing, and implementing Db2 schema changes as part of an application update.
By integrating with your application orchestration tools AMI for DevOps can capture the necessary database changes required to move from test to production. But it does not just apply these changes; it enforces and ensures best practices using built-in intelligence and automated communication between development and database administration.
The ability to enforce best practices is driven by BMC’s Automated Mainframe Intelligence (AMI), which is policy driven. The AMI capability builds much of the DBA oversight for schema changes into the DevOps pipeline, enforcing database design best practices as you go instead of requiring in-depth manual DBA oversight.
Incorporating a database design advisory capability into the process offloads manual, error-prone tasks to the computer. This integrated automation enables automatic evaluation of Db2 database schema change requests to streamline the DBA approval process and remove the manual processes that inhibit continuous delivery of application functionality.
Furthermore, consider that intelligent database administration functionality can be used to help alleviate the loss of expertise resulting from an aging, retiring workforce. This is a significant challenge for many organizations in the mainframe world.
But let’s not forget the developers. The goal of adopting a DevOps approach on the mainframe is to speed up application development, but at the same time it is important that we do not forgo the safeguards built into mainframe development and operations. So you need a streamlined DevOps process—powered by intelligent automation—in which application developers do not have to wait around for DBA reviews and responses. A self-service model with built-in communication and intelligence such as provided by AMI for DevOps delivers this capability.
The Bottom Line
BMC AMI for DevOps helps you to bring DevOps to the mainframe by integrating Db2 for z/OS schema changes into established and existing DevOps orchestration processes. This means you can use BMC AMI for DevOps to deliver the speed of development required by agile techniques used for modern application delivery without abandoning the safeguards required by DBAs to assure the accuracy of the database changes for assuring availability and reliability of the production system. And developers gain more self-service capability for Db2 schema changes using a well-defined pipeline process.
Thursday, August 01, 2019
DevOps is Coming to Db2 for z/OS
Mainframe development teams are relying on DevOps
practices more extensively, bringing the need to incorporate Db2 for z/OS
database changes into the toolset that is supporting their software development
lifecycle (SDLC).
But most mainframe
professionals have only heard a little about DevOps and are not really savvy as
to what it entails. DevOps is an amalgamation of Development and Operations.
The goal of DevOps is to increase collaboration between developers and
operational support and management professionals, with the desired outcome of
faster, more accurate software delivery.
DevOps typically relies
on agile development, coupled with a collaborative approach between development
and operations personnel during all stages of the application development
lifecycle. The DevOps approach results in small and frequent code changes and it
can significantly reduce the lead time for changes, lower the rate of failure,
and reduce the mean time to recovery when errors are encountered. These are all
desirable qualities, especially as organizations are embracing digital
transformation driven by the 24/7 expectations of users and customers to access
data and apps at any time from any device.
The need to be able to
survive and thrive in the new digital economy has caused organizations to adopt
new and faster methods of developing, testing and delivering application
software. Moving from a waterfall software development methodology to an agile
methodology is one way that organizations are speeding the time-to-delivery of
their software development. Incorporating a DevOps approach is another.
Instead of long software
development projects that may not deliver value for months, or perhaps even
years (common using the Waterfall development methodology) an agile DevOps
approach delivers value quickly, and then incrementally over time. DevOps
enables the continuous delivery of new functionality demanded by customers in
the digital economy.
Succeeding with DevOps,
however, requires a cultural shift in which all groups within IT work in
collaboration with one another, and where management endorses and cultivates
this cultural change. Because DevOps relies upon incremental development and
rapid software delivery, your IT department can only thrive if there is a
culture of accountability, collaboration, and team responsibility for desired
business outcomes. Furthermore, it requires solid, integrated automated tooling
to facilitate the SDLC from development, through testing, to delivery. Creating
such an environment and culture can be challenging.
With DevOps the result
will be a constantly repeating cycle of continuous development, continuous
integration and continuous deployment. This is typically depicted graphically
as the infinity symbol such as in Figure 1 (below).
Note, however, that this particular iteration of the DevOps infinity graphic calls out the participation of both the application and the database. This is an important, though often lacking, detail that should be stressed when adopting DevOps practices.
The
Mainframe and DevOps
The adoption of DevOps has,
until now, been much slower within mainframe development teams than for
distributed and cloud application development. The staid nature of mainframe
development and support, coupled with a glass house mentality, and a rigid
production turnover process contribute to the delayed adoption of DevOps on the
mainframe. This is not surprising as mainframes mostly are used by large
organizations running mission critical workloads with an aversion to any kind
of change and risk-averse.
Additionally, the
traditional waterfall development methodology has been used by most mainframe
software developers for multiple decades, whereas DevOps is closely aligned
with an agile approach, which differs significantly from waterfall.
Notwithstanding all of
these barriers to acceptance of DevOps on the mainframe, mainframe developers can,
and in some cases already do successfully utilize a DevOps approach.
Technically speaking, the mainframe is just another platform and there is
nothing inherent in its design or usage that obviates the ability to
participate in a DevOps approach to application development and delivery.
What about
Db2 for z/OS?
Integrating database
change into the application delivery lifecycle can be a stumbling block on the
road to DevOps success. Development teams focus on application code, as they
should, and typically view database structure changes as ancillary to their coding
efforts. In most application development projects, it is not the programmer’s
responsibility to administer the database and modify database structures. But
applications rely on the database being designed, implemented, and changed in
accordance with the needs of the business and the code.
This means that many
development projects have automated their SDLC tool chain to speed up the
delivery of applications. This is the “Dev” portion of DevOps. But the
requisite automation and tooling has not been as pervasively implemented to
speed up the delivery of database changes. This is the “Ops” portion of DevOps.
And this is changing.
A big consideration is
that the manner in which change is applied to applications differs from how
database changes are applied. That means each must be managed using different
techniques and probably different tools. When an application program changes,
the code is compiled, and the load module is migrated from test to production.
The old load module is saved for posterity in case the change needs to be
backed out, but the change is a wholesale replacement of the executable code.
Database changes are
different. The database is an entire configuration in each environment and
changes get migrated. There is no wholesale replacement of the database
structures. DDL commands are issued to ALTER, DROP, and CREATE the changes to
the database structures as needed.
From the perspective of
database changes on Db2 for z/OS, DBAs need the ability to modify all the
database objects supported by Db2 for z/OS. Supporting Db2 for z/OS using
DevOps requires tooling that understands both Db2 for z/OS and the DevOps
methodology and toolchain. And the tooling must understand how changes are
made, as well as any underlying changes that may be required to effectively
implement the database change. Some types of database changes are intrusive,
requiring a complicated series of unloads, metadata captures, drops, creates,
loads, and additional steps to implement. The tooling must be capable of making
any of these changes in an automated way that the DBA trusts.
Fortunately, for
organizations adopting DevOps on the mainframe with Db2, there is a solution
for integrating Db2 database change into the DevOps toolchain: BMC
AMI DevOps for Db2. BMC AMI DevOps for Db2 integrates with Jenkins, an
application development orchestration tool, to automatically research and
determine database schema change requirements, to streamline the review and
approval process, and to safely implement the database schema changes making
development and operations teams more efficient and agile.
Monday, July 29, 2019
Webinar: DevOps and Database Change Management for Db2 for z/OS - August 13, 2019
DevOps practices are gaining popularity on all development platforms and the mainframe is no exception. DevOps relies heavily on agile development and automated software delivery. However, the ability to integrate and orchestrate database changes has lagged.
To learn more about DevOps, change management, and Db2 for z/OS, I am delivering a webinar on this topic along with John Barry of BMC. We will discusses issues including an overview of DevOps, the requirements for database change management, and an introduction to BMC’s new AMI DevOps for Db2 that solves the change management dilemma for Db2 for z/OS development.
You can register today to attend the webinar on August 13, 2019 (Noon Central) at https://event.webcasts.com/starthere.jsp?ei=1251892&tp_key=3ff9b7af72.
Tuesday, July 16, 2019
Proud to be an IBM Champion
Just a quick post today about the IBM Champions program, which if you haven't heard about, is a special program run by IBM to recognize and reward non-IBM thought leaders for their work associated with IBM products and communities.
IBM publishes the list of IBM Champions annually and the title is valid for one year. So, champions must be nominated each year to maintain their status.
I want to thank IBM for running such a wonderful program and for all they have done to help recognize those of us in the trenches using IBM's technology. I have been named an IBM Champion for Data and Analytics again this year... for the 10th time. So IBM bestowed upon me this Acclaim badge:
As an IBM Champion I have had the opportunity to interact with IBM folks and with other IBM Champions at events, webinars, and in person, and it has definitely helped to enrich my professional life.
Although the majority of IBM Champions focus on data and analytics, the program is not just for data people! IBM names champions in each of the following nine categories:
Thanks again, IBM... and congratulations to all of this year's IBM Champions.
IBM publishes the list of IBM Champions annually and the title is valid for one year. So, champions must be nominated each year to maintain their status.
I want to thank IBM for running such a wonderful program and for all they have done to help recognize those of us in the trenches using IBM's technology. I have been named an IBM Champion for Data and Analytics again this year... for the 10th time. So IBM bestowed upon me this Acclaim badge:
As an IBM Champion I have had the opportunity to interact with IBM folks and with other IBM Champions at events, webinars, and in person, and it has definitely helped to enrich my professional life.
Although the majority of IBM Champions focus on data and analytics, the program is not just for data people! IBM names champions in each of the following nine categories:
- Data & Analytics
- Cloud
- Collaboration Solutions
- Power Systems
- Storage
- IBM Z
- Watson IoT
- Blockchain
- Security
Thanks again, IBM... and congratulations to all of this year's IBM Champions.
Wednesday, July 10, 2019
There’s a New Db2 12 for z/OS Function Level (505)
In late June 2019, IBM
delivered more great new capabilities with the latest new function level for
Db2 12 for z/OS, Function
Level 505 (or FL505).
If you do not know
what a function level is, then you probably aren’t yet on Version 12, because
function levels are how new capabilities are being rolled out for Db2 12 and
beyond. It is how IBM has enabled a continuous delivery model for Db2 functionality.
You can learn more about function
levels here.
Although the first
link above goes into all of the gory details of the new functionality, I will
take a bit of time to summarize the highlights of this new function level.
The first thing that
will appeal to most Db2 users is improved performance. And FL505 delivers improved
performance in two areas: HTAP and RUNSTATS.
- For HTAP, FL505 improves the latency between Db2 and the IBM Analytics Accelerator (sometimes called IDAA). Nobody likes latency and these improvements can enable transactional and analytical applications to see the same data.
- For RUNSTATS, FL505 makes the default RUNSTATS behavior to use page sampling for universal table spaces (unless the RUNSTATS specification explicitly states TABLESAMPLE SYSTEM with a value other than AUTO). This will boost RUNSTATS performance. (A nice description of this is provided by Peter Hartmann here.)
FL505 also delivers REBIND phase-in for executing packages. Waiting for a package to be idle (not running) has long been a deterrent to rebinding. Now, you can REBIND a package while it is running. Db2 makes this happen by creating a new copy of the package. When the REBIND completes, new executions of the package will use the newly rebound package and the threads already running with the old package continue to do so successfully until completion.
We also get some new built-in functions (BIFs)
in FL505, for encrypting and decrypting data using key labels. You may be aware
that Db2 already had
functions for encryption and decryption but these functions, introduced
back in V9 were not very capable because they required you to provide and
manage a password to decrypt the data. The new functions work with key labels:
encrypting plain text using ENCRYPT_DATAKEY to a block of encrypted text using
a specified algorithm and key label; and decrypting with DECRYPT_DATAKEY to return
the block of data decrypted to the specified data type.
And with FL505 we finally get additional
functionality for DECFLOAT data type. The DECFLOAT data type was introduced in
DB2 9 for z/OS, but it is not widely used because of some shortcoming. But
first, what is DECFLOAT? Well, DECFLOAT is basically a combination of DECIMAL
and floating-point data types, that is a decimal floating-point or DECFLOAT data
type. Specified as DECXFLOAT(n), where the value of n can be either 16
or or 34, representing the number of significant digits that can be
stored. A decimal floating-point value is an IEEE 754r number with a decimal
point and it can be useful to store and manage very large numbers.
So what is the improvement? Quite simply, it is now possible to specify columns defined as DECFLOAT in an index and as a key in a primary key or a unique key. Unfortunately, there is still no support for DECFLOAT usage in COBOL programs, which will likely continue to hinder its uptake in many shops.
So what is the improvement? Quite simply, it is now possible to specify columns defined as DECFLOAT in an index and as a key in a primary key or a unique key. Unfortunately, there is still no support for DECFLOAT usage in COBOL programs, which will likely continue to hinder its uptake in many shops.
And finally, FL505 improves temporal support for triggers. It delivers the capability to
reference system temporal tables and archive-enabled tables in the WHEN clause
of your triggers.
Summary
IBM is using function
levels to deliver significant new capabilities for Db2 12 for z/OS. It is
important for you and your organization to keep up-to-date on this new
functionality and to determine where and when it makes sense to introduce it into
your Db2 databases and applications.
Also, be aware that if
you are not currently running at FL504, moving to FL505 activates all earlier
function levels. You can find a list of all the current function levels
here.
Monday, June 17, 2019
Two Types of Db2 for z/OS Temporary Tables: Declared vs. Created
Db2 for z/OS has provided the ability to create temporary tables since way back in Version 5. But the initial functionality was practical only in certain circumstances due to some inherent limitations. The first type of temporary table supported by Db2 is now known as a created temporary table.
But IBM’s support of temporary tables expanded since (in Version 7) and Db2 offers two different types of temporary tables: created and declared.
Why use Temporary Tables?
Before we delve into these two types of temporary tables, let’s first consider why anybody would want or need to use a temporary table in the first place.
One potential usage of temporary tables is to store intermediate SQL results. Consider, for example, if the results of one query need to be used in a subsequent query. Instead of rerunning the first query (or combining it with the subsequent query), the results of the first query can be stored in a temporary table. Then the temporary table can be joined into the second query without incurring the overhead of rerunning the first query. This is particularly useful if the first query is particularly complex or inefficient.
An additional use case is when a query result set needs to be returned more than once during the execution of the same program. Consider this scenario: a complex multi-table join is coded that consumes a lot of resources to run. Furthermore, that join needs to be run three times during the course of the program. Instead of running the join three times you can run it once and populate a temporary table with the results. The next two times you can simply read the temporary table which might be more efficient than re-executing the complex, resource-consuming multi-table join.
Temporary tables also can be useful for enabling non-relational data to be processed using SQL. For example, you can create a global temporary table that is populated with IMS data (or any other non-relational data source) by a program. Then during the course of that program, the temporary table (that contains the heretofore non-relational data) can be accessed by SQL statements and even joined to other Db2 tables. The same could be done for data from a flat file, VSAM, IDMS, or any other non-relational data.
Another reason for temporary tables is to make conversion from other relational products easier.
Now let’s examine the two types of temporary tables supported by DB2.
Created Temporary Tables
A created temporary table exists only as long as the process that uses it. Temporary tables are created using the CREATE GLOBAL TEMPORARY TABLE statement. When created, the schema for the table is stored in the Db2 system catalog (SYSIBM.SYSTABLES) just like any other table, but the TYPE column is set to 'G' to indicate a global temporary table. Created temporary tables are sometimes referred to as global temporary tables – but this is confusing since declared temporary tables are also referred to as global declared tables.
It is important to remember that a created global temporary table must be created using a DDL CREATE statement before it can be used in any program.
A created temporary table is instantiated when it is referenced in an OPEN, SELECT INTO, INSERT, or DELETE statement, not when it is created. Each application process that uses the temporary table creates a new instance of the table for its use. When using a created temporary table, keep the following in mind:
· Because they are not persistent, some typical database operations including locking, logging, and recovery do not apply to created temporary tables.
· Indexes can not be created on created temporary tables so all access is by a complete table scan.
· Constraints can not be created on created temporary tables.
· A null is the only default value permitted for columns of a created temporary table.
· Created temporary tables can not be referenced by Db2 utilities.
· Created temporary tables can not be specified as the object of an UPDATE statement.
· When deleting from a created temporary table, all rows must be deleted.
· Although views can be created on created temporary tables, the WITH CHECK OPTION can not be specified.
Work file data sets are used to manage the data of created temporary tables. The work database (DSNDB07) is used as storage for processing SQL statements that require working storage – not just for created temporary tables. So if you are using created temporary tables be sure to examine the Db2 Installation Guide for tactics to estimate the disk storage required for temporary work files.
When a temporary work file result table is populated using an INSERT statement, it uses work file space. No other process can use the same work file space as that temporary work file table until the table goes away. The space is reclaimed when the application process commits or rolls back, or when it is deallocated, depending which RELEASE option was used when the plan or package was bound. It is a good idea to keep the work files in a separate buffer pool to make it easier to monitor.
Declared Temporary Tables
The second type of Db2 temporary table is the temporary tables. It is different than a created temporary table and overcomes many of their limitations. The first significant difference between declared and created temporary tables is that declared temporary tables are specified using a DECLARE statement in an application program – and not using a DDL CREATE statement. Because they are not persistent they do not have descriptions in the DB2 Catalog.
Additionally, declared temporary tables offer significant features and functionality not provided by created temporary tables. Consider:
· Declared temporary tables can have indexes and CHECK
constraints defined on them.
constraints defined on them.
· You can issue UPDATE statements and positioned DELETE statements against a declared temporary table.
· You can implicitly define the columns of a declared temporary table and use the result table from a SELECT.
To “create” an instance of a declared temporary table you must issue the DECLARE GLOBAL TEMPORARY TABLE statement inside of an application program. That instance of the declared temporary table is known only to the process that issues the DECLARE statement. Multiple concurrent programs can be executing using the same declared temporary table name because each program will have its own copy of the temporary table.
But there is more work required to use a declared temporary table than there is to use a created temporary table. Before you can declare temporary tables you must create a temporary database and table spaces for them to use. This is accomplished by specifying the AS TEMP clause on a CREATE DATABASE statement. Then, you must create segmented table spaces in the temporary database. Only one temporary database for declared temporary tables is permitted per Db2 subsystem.
When a DECLARE GLOBAL TEMPORARY TABLE statement is issued, Db2 will create an empty instance of the temporary table in the temporary table space. INSERT statements are used to populate the temporary table. Once inserted, the data can be accessed, modified, or deleted. When the program completes, DB2 will drop the instance of the temporary table. Also, be aware that users of temporary tables must have been granted USE authority on the temporary table space.
The following example shows a DECLARE statement that can be issued from an application program (assuming the temporary database and table spaces already have been defined):
DECLARE GLOBAL TEMPORARY TABLE TEMP_EMP
(EMPNO CHAR(6) NOT NULL,
FIRSTNME VARCHAR(12) NOT NULL,
MIDINIT CHAR(1) NOT NULL,
LASTNAME VARCHAR(15) NOT NULL,
WORKDEPT CHAR(3),
PHONENO CHAR(4)
);
This creates a declared temporary table named TEMP_EMP.
Additionally, you can use the LIKE clause to DECLARE a temporary table that uses the same schema definition as another currently existing table. You can use the INCLUDING IDENTITY COLUMN ATTRIBUTES clause to copy the IDENTITY columns as well. For example:
DECLARE GLOBAL TEMPORARY TABLE TEMP_PROJ LIKE DSN8C10.PROJ
ON COMMIT PRESERVE ROWS;
Notice the ON COMMIT PRESERVE ROWS clause in the previous example. The ON COMMIT clause specifies what action Db2 is to take with the data in the declared temporary table when the program issues a COMMIT statement. There are two options: PRESERVE or DELETE rows.
Specifying PRESERVE ROWS indicates that the rows of the table are to be kept. Beware, though, that the PRESERVE ROWS option impacts thread reuse. You will not be able to reuse threads for any application process that contains, at its most recent COMMIT, an active declared temporary table defined using the PRESERVE ROWS option of the ON COMMIT clause. The other option, which is the default, is ON COMMIT DELETE ROWS. In that case all of the rows of the table are deleted as long as there are no cursors defined using WITH HOLD.
Scrollable Cursors
A scrollable enables cursor provides the ability to scroll forward and backward through the data once the cursor is open. Using only SQL, the programmer can navigate up and down the cursor results. Although this blog post is not about scrollable cursors, it is important to know that there are two types of scrollable cursors: static and dynamic. Static scrollable cursors use declared temporary tables.
So, keep in mind, even if you do not choose to use temporary tables in your application programs, you may need to implement them to support scrollable cursors.
Declared Temporary Table Storage
Before using declared temporary tables, the temporary database and temporary table spaces must be defined to store the temporary data. For example:
CREATE DATABASE TEMPDB AS TEMP;
CREATE TABLESPACE TEMPTS IN TEMPDB
SEGSIZE 4
BUFFERPOOL BP7;
SEGSIZE 4
BUFFERPOOL BP7;
The table space is created as a temporary table space by virtue of it being in the temporary database.
The page size of the temporary table space must be large enough to hold the longest row in the declared temporary table. The size of a row in the declared temporary table might be considerably larger then the size of the row in the table for which the scrollable cursor is used. As with a regular table, the size of the row depends on the number of columns that are stored in the declared temporary table and the size of each column.
An in-depth discussion of calculating the storage requirements for declared temporary table table spaces is provided in the Db2 Installation Guide. Be sure to refer to that manual before implementing declared temporary tables or any features that rely on declared temporary tables (e.g. static scrollable cursors).
Keep in mind, too, that when there is more than one temporary table space defined to the Db2 subsystem, Db2 will select which temporary table spaces it will use for scrollable cursor processing.
Declare or Create?
With all of the limitations of created temporary tables why would anyone still want to use them instead of declared temporary tables?
Well, there are a few potential problems with declared temporary tables, too. First of all, the SYSPACKDEP catalog table will not show dependencies for declared temporary tables, but it will for created temporary tables. Secondly, some DBAs are leery of allowing database structures to be created by application programmers inside of an application program. With limited DDL and database design knowledge it may not be wise to trust programmers to get the table structure correct. Furthermore, the additional management of the temporary database and table spaces can become an administrative burden.
So, created temporary tables are still useful – in the right situations. They should be considered primarily when no updating of temporary data is needed and access to the temporary data is purely sequential.
Synopsis
Db2 provides two options for handling temporary data in tables: created and declared temporary tables. The wise Db2 professional will understand the capabilities and shortcomings of each type of temporary table – and deploy the correct type for each specific situation.
Subscribe to:
Posts (Atom)