Wednesday, September 02, 2020

The Benefits of In-Memory Processing

One area that most organizations can benefit from is by better using system memory more effectively. This is so because accessing and manipulating data in memory is more efficient than doing so from disk.

Think about it… There are three aspects of computing that impact the performance and cost of applications: CPU usage, I/O, and concurrency. When the same amount of work is performed by the computer using fewer I/O operations, CPU savings occur and less hardware is needed to do the same work. A typical I/O operation (read/write) involves accessing or modifying data on disk systems; disks are mechanical and have latency – that is, it takes time to first locate the data and then read or write it.

There are many other factors involved in I/O processing that involve overhead and can increase costs, all depending upon the system and type of storage you are using. For example, each I/O consists of a multitude of background system processes, all of which contribute to the cost of an I/O operation (as highlighted in Figure 1 below). It is not my intent to define each of these processes but to highlight the in-depth nature of the processing that goes on behind-the-scenes that contributes to the costly nature of an I/O operation.


Figure 1. The Cost of an I/O

So, you can reduce the time it takes to process your mainframe workload by more effectively using memory. You can take advantage of things like increased parallelism for sorts and improve single-threaded performance of complex queries when you have more memory available to use. And for OLTP workloads, large memory provides substantial latency reduction, which leads to significant response time reductions and increased transaction rates.

The most efficient way to access data is, of course, in-memory access. Disk access is orders-of-magnitude less efficient than access data from memory. Memory access is usually measured in microseconds, whereas disk access is measured in milliseconds. (Note that 1 millisecond equals 1000 microseconds.)

The IBM z15 has layers of on-chip and on-board cache that can improve the performance of your application workloads. We can view memory usage on the mainframe as a pyramid, from the slowest to the fastest, as shown in Figure 2. As we go up the pyramid, performance improves; from the slowest techniques (like tape) to the fastest (core cache). But this diagram drives home our core point even further: that system memory is faster than disk and buffering techniques.


Figure 2. The Mainframe Memory Pyramid

So how can we make better use of memory to avoid disk processing and improve performance? Although there are several different ways to adopt in-memory processing for your applications, one of the best methods can be to utilize a product. One such product is the IBM Z Table Accelerator.

 IBM Z Table Accelerator is an in-memory table accelerator that can improve application performance and reduces operational cost by utilizing system memory. Using it can help your organization to focus development efforts more on revenue-generating business activity, and less on other less efficient methods of optimizing applications. It is ideal for organizations that need to squeeze every ounce of power from their mainframe systems to maximize performance and transaction throughput while minimizing system resource usage at the application level. You can use it to optimize the performance of all types of data, whether from flat files, VSAM, Db2, or even IMS.

 So how does it work? Well, typically a small percentage of your data is accessed and used a large percentage of the time. Think about it in terms of the 80/20 Rule (or the Pareto Principle).  About 80% of your data is accessed only 20% of the time, and 20% of your data is accessed 80% of the time.

The data that you are accessing most frequently is usually reference data that is used by multiple business transactions. By focusing on this data and optimizing it you can gain significant benefits. This is where the IBM Z Table Accelerator comes into play. By copying some of the most often accessed data into the accelerator, which uses high-performance in-memory tables, significant performance gains can be achieved. That said, it is only a small portion of the data that gets copied from the system of record (e.g. Db2, VSAM, etc.) into the accelerator.

High-performance in-memory technology products -- such as IBM Z Table Accelerator -- use system memory. Sometimes, if the data is small enough, it can make it into the L3-L4 cache. This can be hard to predict, but when it occurs things get even faster.

Every customer deployment is different, but using IBM Z Table Accelerator to optimize in-memory data access can provide a tremendous performance boost.

A Use Case: Tailor-Fit Pricing

Let’s pause for a moment and consider a possible use case for IBM Z Table Accelerator.

In 2019, IBM announced Tailored Fit Pricing (TFP), with the goal of simplifying mainframe software pricing and billing. IBM designed TFP as a more predictable, cloud-like pricing model than its traditional pricing based on a rolling-four-hour-average of usage. Without getting into all of the details, TFP eliminates tracking and charging based on monthly usage and instead charges a consistent monthly bill based on the previous year’s usage (plus growth).

It is important to note that last point: TFP is based on last year’s usage. So you can reduce your bill next year by reducing your usage this year, before you convert to TFP. Therefore, it makes a lot of sense to reduce your software bills to the lowest point possible the year before the move to TFP.

 

So what does this have to do with IBM Z Table Accelerator? Well, adopting techniques to access data in-memory can lower MSU usage – and therefore your monthly software bill. Using IBM Z Table Accelerator to optimize your reference data in-memory before moving to TFP can help you to lower your software bills and better prepare you for the transition to Tailored Fit Pricing.

 

 --------------------

 If you’d like to learn more about IBM Z Table Accelerator there is an upcoming SHARE webinar on September 15, 2020, that goes into some more details about the offering. It is titled Digital Transformation IncludesGetting The Most Out of Your Mainframe: click the link for details and to register to attend.


 

 

 

 

 

Friday, August 28, 2020

IBM Db2 on Cloud News

If you are thinking about, or have already adopted Db2 in the cloud, there is some recent news you should know about. But before we explore that news, let’s take a look at the quick highlights of using Db2 in the cloud.


IBM’s Db2 on Cloud offering is a fully-managed operational data store running the IBM Db2 11.5 engine offering 24x7x365 availability. So if you know Db2 on Linux, Unix, and Windows platforms, you know Db2 on Cloud… but there’s more.

Db2 on Cloud runs containerized Db2 on with a dedicated DevOps team managing the maintenance and updates required to run your mission-critical workloads. This includes features like seamless data federation, point-in-time recovery, HADR with multizone region support and independent scaling. So many of the administrative burdens of managing Db2 on-premises are handled by IBM in the cloud.

Now if you know me, and have been reading my “stuff” on cloud and DBA, you know that this does not mean that you can entirely offload you DBA. But it is cool and it does help, especially with DBA teams being stressed to their limits these days.

So yes, you can run Db2 on Cloud! And there are many good reasons to consider doing so, such as scalability, pay-as-you-use pricing, and to take advantage of managed services.

OK, So What is New?

I promised some news in the title of this blog post and so far we have just set the stage by examining IBM’s cloud offering of Db2 (albeit at a high level). So, what’s new?

Well, IBM is revamping its pricing plans. Before digging into the news, you need to know that IBM offers two high-level pricing plan options.

  • The Lite plan uses a shared multi-tenant system designed for application development and evaluation of IBM Db2 on Cloud. It is offered free-of-charge, without any time limitations.
  • Enterprise plans are for usage and deployment of business applications and systems. It includes one database per service instance with 4 vCPU x 16 GB RAM x 20 GB storage on dedicated compute slices with the option to use a three-HA-node configuration with multizone region support. Pricing starts at $989/month.

What is new is that on August 19 IBM introduced two new plans, the Enterprise non-HA plan, and the Standard non-HA plan.  This means that there are now four options, other than the free Lite plan: Enterprise HA, Enterprise non-HA, Standard HA, and Standard non-HA.

As is typical with IBM pricing, it is not really all that simple and it is getting more complex.  But options are always good (I think).

So what is this Standard plan that does not appear on the IBM Db2 on Cloud: Pricing page? Well, we can find this on the IBM Db2 on Cloud catalog page Here we see that (as one might expect) it is a lower-cost option between Lite and Enterprise starting at 8 GB RAM with 20 GB storage.

IBM also noted that IBM Db2 on Cloud is now available in the following six data centers: Dallas, Frankfurt, Tokyo, London, Sydney, and Washington. And your instances can be provisioned either with or without the Oracle compatibility feature.

It is important to note that IBM also notes that customers on older, legacy plans (how about that, cloud legacy already) will be required to upgrade their to one of the newer plans.

 Summary

So, there are more options to choose from with your Db2 on Cloud implementations. And if you have an older plan take some time to familiarize yourself with the new pricing plan options and be ready to choose accordingly for your workload requirements.

Tuesday, August 18, 2020

Navigating the IBM COBOL 4.2 End of Service Waters: Chart a course to benefit your business

 

Surprisingly, COBOL has been in the news a lot recently, due to its significant usage in many federal govern-ment and state systems, most recently with unemployment systems, being in the news. With the global COVID-19 pandemic, those unemployment systems were stressed like never before with a 1600% increase in traffic (Government Computer News, May 12, 2020) as those impacted by the pandemic filed claims.

Nevertheless, there is another impending event that will likely pull COBOL back into the news as IBM withdraws older versions of the COBOL compiler from service. All IBM product versions go through a lifecycle that starts with GA (general availability), after some time moves to EOM (end of marketing) where IBM no longer sells that version, and ends with EOS (end of support) where IBM no longer supports that product or version. It is at this point that most customers will need to decide to stop using that product or upgrade to a newer version because IBM will no longer fix or support EOS products or versions.

Of course, code that was compiled using an unsupported COBOL compiler will continue to run, but it is not wise to use unsupported software for important, mission-critical software, such as is usually written using COBOL. And you need to be aware of interoperability issues if you rely on more than one version of the COBOL compiler.

So what is going on in the world of COBOL that will require your attention? First of all, earlier this year on April 30, 2020, IBM withdrew support for Enterprise COBOL 5.1 and 5.2. And Enterprise COBOL 4.2 will be withdrawn from service on April 30, 2022 – just about two years from now.

So now is the time for your organization to think about its migration strategy.

Why is COBOL still being used?

Sometimes people who do not work in a mainframe environment are surprised that COBOL is still being used. But it is, and it is not just a fringe language. COBOL is a language that was designed for business data processing, and it is extremely well-suited for that purpose. It provides features for manipulating data and printing reports that are common business requirements. COBOL was purposely designed for applications that perform transaction processing like payroll, banking, airline booking, etc. You put data in, process that data, and send results out.

COBOL was invented in 1959, so its history stretches back over 60 years; a lot of time for organizations to build complex applications to support their business. And IBM has delivered new capabilities and features over the years that enable organizations to keep up to date as they maintain their application portfolio.

So, COBOL is in wide use across many industries.

A majority of global financial transactions are processed using COBOL, including processing 85 percent of the world’s ATM swipes. According to Reuters, almost 3 trillion dollars in DAILY commerce flows through COBOL systems!

The reality is that more than 30 billion COBOL transactions run every day. And there are more than 220 billion lines of COBOL in use today. COBOL is not dead…

 What’s new in COBOL 6

With COBOL 5.1 and 5.2 already out of support, and COBOL 4.2 soon to follow, one migration path is to Enterprise COBOL 6, and IBM has already delivered three releases of it: 6.1, 6.2, and 6.3. There are some nice new features that are in the latest version(s) of IBM Enterprise COBOL, including:

  • Compile and runtime support delivering performance improvements for z15 hardware and z/OS 2.4 operating system
  • Increased compiler capacity making it possible to compile and optimize larger programs (6.1)
  • 64-bit (AMODE 64) support in this compiler enables users to process large data tables that require greater than 2 GB of addressing space (6.3)
  • JSON support (6.1) including JSON PARSE statement (6.2)
  • Support for many new features from the COBOL 2002/2014 programming standards including new statements like ALLOCATE, FREE and INITIALIZE; addition of Dynamic Length elementary items; conditional compilation using the DEFINE compiler option, and more
  • Many new compiler options
  • Improved usability with USS

At the same time, there are concerns that need to be considered if and when you migrate to version 6. One example is that the new compiler will take longer to compile programs than earlier versions – from 5 to 12 times longer depending on the optimization level. There are also additional work data sets required and additional memory considerations that need to be addressed to ensure the compiler works properly. As much as 20 times more memory may be needed to compile than with earlier versions of the compiler.

Some additional compatibility issues to keep in mind are that your executables are required to be stored in PDSE data sets and that COBOL 6 programs cannot call or be called by OS/VS COBOL programs.

And of course, one of the biggest issues when migrating from COBOL 4.2 to a new version of COBOL is the possibility of invalid data – even if you have not changed your data or your program (other than re-compiling in COBOL 6). This happens because the new code generator may optimize the code differently. That is to say, you can get different generated code sequences for the same COBOL source with COBOL 6 than with 4.2 and earlier versions of COBOL. While this can help minimize CPU usage (a good thing) it can cause invalid data to be processed differently, causing different behavior at runtime (a bad thing).

Whether you will experience invalid data processing issues depends on your specific data and how your programmers coded to access it. Some examples of processing that may cause invalid data issues include invalid data in numeric USAGE DISPLAY data items; parameter/argument size mismatches; using TRUNC() with binary data values having more digits than they are defined for in working storage; and data items that are used before they have been assigned a value.

Migration considerations

Keep in mind that migration will be a lengthy process for any medium-to-large organization, mostly due to testing application behavior after compilation, and comparing it to pre-compilation behavior. You need to develop a plan that best suits your organization’s requirements and work to implement it in the roughly 2-year timeframe before IBM Enterprise COBOL 4.2 goes out of support.

Things to consider:

  • Gartner research shows that “huge ‘all-or-nothing’ modernization programs often fail to meet expectations”
  • What is your current state? Which COBOL compilers are you using and what is your end goal (6.1, 6.2, 6.3)?
  • Remember that compiled programs will continue to run, so it may not be imperative to re-compile everything prior to the end of support date. Of course, it can be difficult to keep track of what has been converted and what has not if you do not have a plan moving forward other than “convert when the program has to be changed at some point.” And it can become difficult to keep track of all the requirements and incompatibilities for multiple versions of COBOL if you do not plan for, and eventually convert to a newer compiler version.
  • Do you have the COBOL talent and knowledge not only to convert but to continue supporting your existing portfolio of COBOL applications?
  • Enterprise application portfolios can be quite large, making it difficult to effectively discover and map all of the dependencies. Consider using tools to help. 

Migration challenges and Options to Consider

As you put your plan together, you might consider converting some of your COBOL applications to Java. An impending event such as the end of support for a compiler is a prime opportunity for doing so. But why might you want to convert your COBOL programs to Java?

Well, it can be difficult to obtain and keep skilled COBOL programmers. As COBOL coders age and retire, there are fewer and fewer programmers with the needed skills to manage and maintain all of the COBOL programs out there. At the same time, there are many skilled Java programmers available on the market, and universities are churning out more every year.

Additionally, Java code is portable, so if you ever want to move it to another platform it is much easier to do that with Java than with COBOL. Furthermore, it is easier to adopt cloud technologies and gain the benefits of elastic compute with Java programs.

Cost reduction can be another valid reason to consider converting from COBOL to Java. Java programs can be run on zIIP processors, which can reduce the cost of running your applications. A workload that runs on zIIPs is not subject to IBM (and most ISV) licensing charges... and, as every mainframe shop knows, the cost of software rises as capacity on the mainframe rises. But if capacity can be redirected to a zIIP processor, then software license charges do not accrue - at least for that workload.

Additional benefits of zIIPs include:

  • They are significantly cheaper to acquire than standard CPs
  • When workload is redirected to a zIIP it frees up capacity on the standard CP

So, there are many reasons to consider converting at least some of your COBOL programs to Java. Some may be worried about Java performance, but Java performance is similar to COBOL these days; in other words, most of the performance issues of the past have been resolved. Furthermore, there are many tools to help you develop, manage, and test your Java code, both on the mainframe and other platforms.

Keeping in mind the concerns about “all-or-nothing” conversions, most organizations will be working toward a mix of COBOL migrations and Java conversions, with a mix of COBOL and Java being the end results. As you plan for this be sure to analyze and select appropriate candidate programs and applications for conversion to Java. There are tools that can analyze program functionality to assist you in choosing which the best candidates. For example, you may want to avoid converting programs that frequently call other COBOL programs and programs that use pre-relational DBMS technologies (such as IDMS and IMS).

How to convert COBOL to Java

At this point, you may be thinking, “Sure, I can see the merit in converting some of my programs to Java, but how can I do that? I don’t have the time for my developers to re-create COBOL programs in Java going line-by-line!” Of course, you don’t!

This is where an automated tool comes in handy. The CloudFrame Migration Suite provides code conversion tools, automation, and DevOps integration to deliver very maintainable, object-oriented Java that can integrate with modern technology available within your open architecture.  It can be used to refactor COBOL source code to Java without changing data, schedulers, and other infrastructure components. It is fully automated and seamlessly integrates with the change management systems you already use on the mainframe.

The Java code generated by CloudFrame will operate the same as your COBOL and produce the same output. There are even options you can use to maintain the COBOL 4.2 treatment of data, thereby avoiding the invalid data issues that can occur when you migrate to COBOL 6. This can help to reduce project testing and remediation time.

It is also possible to use CloudFrame to refactor your COBOL programs to Java but keep maintaining the code in COBOL. Such an approach, as described in this blog post (Consider Cross-Compiling COBOL to Java to Reduce Costs), can allow you to keep using your COBOL programmers for maintenance but to gain the zIIP eligibility of Java when you run the code.

Upcoming Webinar

To learn more about COBOL migration, modernization considerations, and how CloudFrame can help you to achieve your modernization goals, be sure to attend CloudFrame’s upcoming webinar where I will be participating on a panel along with Venkat Pillay (CEO and founder of CloudFrame) and Dale Vecchio (industry analyst and former Gartner research VP). The webinar, titled Navigating the COBOL 4.2 End of Support(EOS) Waters: An expert panel discusses the best course of action to benefit your business will be held on September 23, 2020 at 11:00 AM Eastern time. Be sure to register and attend!

Summary

Users of IBM Enterprise COBOL 4.2 need to be aware of the imminent end of service date in April 2022 and make appropriate plans for migrating off of the older compiler.

This can be a great opportunity to consider what should remain COBOL and where the opportunities to modernize to Java are.  Learn how CloudFrame can help you navigate that journey.

Friday, August 07, 2020

The Virtual North American IDUG Db2 Tech Conference 2020

The IDUG North America Virtual Conference is happening now... and it runs through the end of next week (August 14, 2020), so there is still time to register, attend, and hear about some great Db2 "stuff!"
Originally, the event was planned as a weeklong conference to be held early in June in Dallas, Texas. But with the pandemic, IDUG changed its plans and turned this year's North American IDUG into a virtual event. The conference has actually been running since July 20, 2020, when the event kicked off, with new content (labs, workshops, and sessions) being released every week since.
What this means to you is that there is still time to take advantage of all the great, online Db2 content that IDUG has made available virtually! The IDUG Virtual Db2 Tech Conference is not free; there is a nominal cost of $199 to participate and attend. But this is a bargain considering the regular cost of attending an IDUG event (not just the event cost, but also travel and lodging).
I've been participating in the event for the past couple weeks and I have to say, there is great content available. It may be a bit more difficult to stay focused on the event as you participate from your home office, though. At least, I found it to be. There are interruptions and distractions that are not there when you participate live, on-site. Furthermore, the camaraderie of an in-person event is lost when it is just you and your computer. 
But these are minor quibbles. Overall, there is a lot of great stuff on offer from this virtual IDUG event that make it well worth the nominal fee being charged. The event includes 60+ sessions, live Q&A with industry experts and leaders, opportunities for engagement with your favorite vendors and each other, and most importantly, cutting-edge technical education streaming straight to your home or office. Your registration also includes a complimentary premium membership so you can access exclusive IDUG content all year long.  
One final thing to share with you is that my pre-recorded session, The Plight of the Modern DBA, will be available starting Monday, August 9, 2020 at 8:00 AM Easter time. I hope you'll take the time to give it a listen and share your thoughts with me. I’ll be participating in a Q&A session on Friday, August 14, 2020, at 2:15 PM Eastern time, so you can stop by and ask anything you'd like!
More information on the 2020 IDUG virtual Db2 Tech Conference can be found here.

Wednesday, July 22, 2020

Mainframe Earnings Up Big for IBM


This week IBM announced earnings results for 2020Q2 and the company reported Systems revenues of $1.9 billion, up 6 percent, led by IBM Z, which was up 69 percent… the mainframe is a shining jewel in IBM’s earnings.

Note that the Systems category comprises IBM systems hardware and operating system software, including mainframe, Power systems, storage, etc.

Other bright spots for IBM include total cloud revenue of $6.3 billion, up 30 percent and Red Hat revenue up 17 percent.

After reporting the earnings, IBM shares rose by as much as 6 percent in extended trading on Monday due to the overall better-than-expected results.

Overall, earnings were $2.18 per share, adjusted, vs. $2.07 per share as expected by analysts.