Mixing Db2 Database Administration with DevOps - Part 2: The DevOps Toolchain

Adopting a DevOps approach to application development is all about moving faster. Automating the multitude of processes required during software development and management is a core part of increasing speed and enabling faster development.

As mentioned in the first part of this series, a DevOps toolchain is used to automate the development process. A toolchain is a set of software tools used to perform a complex development task... or to create a software product. The software tools that comprise a toolchain typically are executed sequentially with the output or state of one tool becoming the input for the next, but this is not a hard-and-fast requirement.

A DevOps toolchain therefore is a set of tools that interoperate with one another to aid in the delivery, development, and management of software applications throughout the SDLC. The following components typically are involved in putting together a useful DevOps toolchain.

Orchestration enables the automation of coordinating and managing the SDLC. Whereas automation refers to a single task or capability, orchestration automates a complex, multi-step process. The orchestration tool drives the entire DevOps process. Jenkins is the most popular DevOps orchestration tool.

Continuous Integration (CI) and Continuous Delivery (CD) are important components of the DevOps toolchain. Typically, CI and CD are implemented into collaboration tools that provide a dashboard for integration and delivery of software. Bamboo and Jenkins offers CI and CD capabilities. 

Tools that deliver Configuration and Resource Management provide an automated method for maintaining your systems and software infrastructure in a consistent state. This can include servers, storage, networking, and software, with the goal of managing the environment in a known, desired state. Examples of configuration management tools include Chef, Ansible, and Puppet.

Source Control software is used to manage who can change code, who is changing code, and track those changes. It enables developers to work collectively on a complex software project without impacting code changes other developers are making. GitHub is the most popular modern source control tool for DevOps projects.

Collaboration tools aid in the scheduling and tracking of code sprints by delivering transparency to the process for all stakeholders. Many different types of tools can fall into this category, including communication, project management, and service management software. Tools that help developers to catalog and track issues help to speed delivery by improving the response taking corrective action. 

Whenever code is being written, it must also be tested. Automated Software Testing tools can improve the speed and quality of software by quickly identifying defects and validating the accuracy of code. 

Deployment tools automate the migration and implementation of application code throughout your environments. It facilitates rapid feedback and continuous delivery in agile development while providing the audit trails, versioning and approvals needed in production. A popular deployment tool IBM UrbanCode Deploy is an example of a popular deployment tool for DevOps.

A Container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.  Using a container developers can incorporate everything required to run an application to enable it to be portable, without being impacted by differences across multiple environments. Docker is the most popular container software. Kubernetes is frequently used with Docker as a container orchestration system that automates application deployment, scaling, and management.  

Monitoring software that can oversee the performance and operation of applications and services is a crucial component of a DevOps pipeline. There are many types of monitoring software including IT infrastructure (operating system, network, and database); the end-user response and experience; and for the performance and functionality of the actual applications themselves. 

Mixing Db2 Database Administration with DevOps - Part 1: Intro to DevOps

Organizations of all types are being impacted by the transformation to the modern digital economy. This digital transformation, driven by the 24/7 expectations of users and customers to access data and apps at any time, from any device, requires the ability to rapidly change technology and software. This means that today’s businesses have to deliver and improve the services and software used by their customers faster than ever before.

To survive and thrive in the new digital economy requires that organizations adopt new and faster methods of developing, testing and delivering application software. The clear trend for modern application development is the use of agile development and DevOps techniques. Why have these techniques been adopted for software development over long-standing techniques like the waterfall approach? It all comes down to improved collaboration, resulting in faster development and therefore a quicker return on investment.

An agile development methodology involves the organization of the team and goals such that software development is iterative, incremental, and evolutionary. Using an agile approach, development and testing activities are undertaken concurrently, unlike the traditional Waterfall development model. Large software projects are broken into pieces such that immediate value can be delivered quickly, in smaller pieces, instead of waiting for a monolithic product to be completed. With continuous testing and integration, the smaller software pieces can be integrated into a larger, final deliverable.

Another key methodology used to speed up the delivery of software is the DevOps approach, which results in small and frequent code changes. Its name is an amalgamation of Development and Operations. As such, DevOps relies on agile development, coupled with a collaborative approach between development and operations personnel during all stages of the application development lifecycle. Such an approach can significantly reduce the lead time for changes, lower the rate of failure, and reduce the mean-time-to-recovery when errors are encountered. With such benefits that can accrue, it is no wonder that DevOps and continuous delivery are gaining in popularity.

Instead of long software development projects that may not deliver value for months, or perhaps even years (such as are common using the Waterfall development methodology), an agile DevOps approach delivers value quickly, and then incrementally over time. DevOps enables the continuous delivery of new functionality demanded by customers in the digital economy.

Succeeding with DevOps however, requires a cultural shift in which all groups within IT work in collaboration with one another, and where management endorses and cultivates this cultural change. Because DevOps relies upon incremental development and rapid software delivery, your IT department can only thrive if there is a culture of accountability, collaboration, and team responsibility for desired business outcomes. Creating such a culture is not easy. 

The goal of DevOps is a constantly repeating cycle of continuous development, continuous integration and continuous deployment. To achieve such a high level of continuous delivery also requires in-depth automation of the entire development lifecycle. This is often referred to as the DevOps toolchain.

Happy Holidays 2023

Did you know that there are over two dozen holidays celebrated across the world between November 1st and January 15th? 

 

 

So in the spirit of the season, I want to wish all of my readers everywhere a Happy Holiday season.

 

This will be the last post I make here this year.

I hope that you will take some time to celebrate whichever holidays you observe... and that you enjoy this time of year with your family and friends. 

 

And let's meet back here next year to talk some more about our favorite DBMS... Db2!

Understanding Db2 Messages

Have you ever been confronted with a Db2 message and needed help deciphering what it is trying to tell you? 

I'm sure most of us have. But let's back up a moment and define what a Db2 message actually is. In Db2 for z/OS, a message refers to a notification from the Db2 subsystem that is conveying information about the status, events, errors, or other conditions within the Db2 environment. Db2 for z/OS generates messages to report various aspects of its operation, and these messages can be critical for monitoring, diagnosing, and resolving issues in the database system.

There is an entire manual devoted to describing Db2 Messages, titled, appropriately enough, Db2 for z/OS Messages. Db2 messages serve several purposes:

• Informational Messages: These messages provide general information about the status or activities of the Db2 subsystem. They might include details about ongoing processes, successful completion of operations, or other relevant information.
• Warning Messages: Warning messages indicate that an operation completed with a potential issue or that there's a condition that might require attention. While not necessarily errors, warnings prompt users to review and possibly take corrective actions.
• Error Messages: Error messages signify that a problem occurred during the execution of a Db2 operation. These messages provide details about the nature of the error and often include information to help identify the cause.
• Diagnostic Messages: Diagnostic messages provide additional details that can be useful for troubleshooting and debugging purposes. They offer insights into the internal workings of Db2 and can assist database administrators in identifying and resolving problems.

Where do you find Db2 messages? Well, the can be found in various places, including: system logs, job output, Db2 message files, and output from SQL queries or commands.

You can recognize a Db2 message because it will start with the three-letter indicator DSN. DSN is the product identifier used internally by IBM for Db2. Db2 for z/OS messages follow a specific format that includes important information about the message type, severity, and details about the message. 

Db2 messages are identified by unique message numbers, which are eight to ten characters long. Db2 for z/OS message identifiers have the following format:

      DSNSnnnnnI

The first three characters, as we have mentioned, is the three-character message identifier, which in Db2 for z/OS is DSN.

The next single character, shown by the S above, is the subcomponent identifier. This character identifies the subcomponent of Db2 that issued the message. Each subcomponent has an associated hexadecimal identifier (hex ID), which is the hexadecimal representation of the subcomponent identifier. These identifiers (as of Db2 13 for z/OS) are as follows:

A    Call attachment facility and some Db2 supplied stored procedures

B    Buffer manager

E    TSO attachment facility 

F    Message generator 

G    Database descriptor manager 

H    Precompiler, DSNH CLIST 

I    Data Manager

J    Recovery log manager

L    DDF

M    IMS Attachment Facility

P    Data space manager 

Q   MQListener

R    Recovery manager 

S    Storage manager 

T    Service controller, install 

U    Utilities U 

V    Agent services manager 

W    Instrumentation facility

X    Relational data system 

Y    Initialization procedures 

Z    System parameter manager

1    Service facilities

3    Subsystem support subcomponent

5    Db2 Accessories Suite for z/OS

7    Group manager 

8    Sample applications 

9    General command processor

The next three to five characters (beginning at the fifth character and indicated by nnnnn) is the numeric identifier of the message. This identifier is unique within each subcomponent.

The final character of the message identifier (indicated by I in the example above) is the type code. This is sometimes thought of a severity code, but most Db2 messages use I for the type code, regardless of the severity or whether an action is required. Some older Db2 messages use other type codes, but keep in mind that the type code does not necessarily reflect the severity of the message. 

What Does the Message Mean?

To understand the meaning of a message you will need to look up the message identifier in the Db2 Messages manual. The manual is broken down into chapters, with each chapter devoted to a subcomponent. So, what if you receive a DSNJ994I Db2 message?

Well, we know that the J means this is a recovery log manager error. If we look this up in the manual (Chapter 8 for Db2 13 for z/OS), we see the following explanation:

    VSAM OPEN failed with the indicated ACB error-code
    for the indicated dd-name.

This is a time to contact your system programmer because Db2 cannot open the underyling VSAM data set.

Summing Things Up

Understanding Db2 messages is a crucial aspect of being able to effectively program, manage, monitor, and resolve Db2 problems. Be sure to have the Db2 manuals and documentation available (either online or downloaded to your computer) to be able to retrieve detailed information about the various messages and their meanings when you need them.

The Db2 for z/OS Messages manual provides comprehensive information about Db2 messages and it is a valuable resource for troubleshooting and understanding system behavior.

My Top Ten Db2 Performance Posts

Regular readers of this blog will notice that the past few months have been consumed by my posts outlining my Top Ten Db2 Performance tips.

So, it being December and the end of the year, I thought it would be a good idea to publish a post with links to each of the Top Ten tips... so here it is!

Number 1 - Analyze Query Execution Plans

Number 2 - Optimize Your Database Design

Number 3 - Efficient Indexing Strategies

Number 4 - Effective Memory Configuration

Number 5 - Workload Management and Query Optimization

Number 6 - Monitoring and Performance Metrics

Number 7 - Configuration Parameters and Tuning

Number 8 - I/O Optimization

Number 9 - Regular Maintenance and Housekeeping

Number 10 - Stay Up-to-Date

Of course, a list such as this will not be comprehensive in terms of listing everything that you may encounter on your path to efficient Db2 performance... but it can serve as a good place to start that journey.

Let me know your thoughts on anything in this list... or share your performance stories and headaches with us here on the blog!