DB2PORTAL Blog

A Forced Tour of Duty

2012-01-26T14:44:00.000-06:00

Mainframe developers are well aware of the security, scalability, and reliability of mainframe computer systems and applications. Unfortunately, though, the bulk of new programmers and IT personnel are not mainframe literate. This should change. But maybe not for the reasons you are thinking.

Yes, I am a mainframe bigot. I readily admit that. In my humble opinion there is no finer platform for mission critical software development than the good ol’ mainframe. And that is why every new programmer should have to work a tour of duty on mainframe systems and applications after graduating from college.

Why would I recommend such a thing? Well, it is because of the robust system management processes and procedures which are in place and working extremely well within every mainframe shop in the world. This is simply not the case for Windows, Unix, and other platforms. By working on mainframe systems newbies will learn the correct IT discipline for managing mission critical software.

What do I mean by that? How about a couple of examples: It should not be an acceptable practice to just insert a CD and indiscriminately install software onto a production machine. Mainframe systems have well-documented and enforced change management procedures that need to be followed before any software is installed into a production environment.

Nor should it be acceptable to just flip the switch and reboot the server. Mainframe systems have safeguards against such practices. And mainframes rarely, if ever, need to be restarted because the system is hung or because of a software glitch. Or put in words PC dudes can understand: there is no mainframe “blue screen of death.” Indeed, months, sometimes years, can go by without having to power down and re-IPL the mainframe.

And don’t even think about trying to get around security protocols. In mainframe shops there is an entire group of people in the operations department responsible for protecting and securing mainframe systems, applications, and data. Security should not be the afterthought that it is in the Windows world.

Ever wonder why there are no mainframe viruses? A properly secured operating system and environment make such a beast extremely unlikely. And with much of the world’s most important and sensitive data residing on mainframes, don’t you think the hackers out there would just love to crack into those mainframes more frequently?

Project planning, configuration management, capacity planning, job scheduling and automation, storage management, database administration, operations management, and so on – all are managed and required in every mainframe site I’ve ever been involved with. When no mainframe is involved many of these things are afterthoughts, if they’re even thought of at all.

Growing up in a PC world is a big part of the problem. Although there may be many things to snark about with regard to personal computers, one of the biggest is that they were never designed to be used the way that mainframes are used. Yet we call a sufficiently “pumped-up” PC a server – and then try to treat it like we treat mainframes. Oh, we may turn it on its side and tape a piece of paper on it bearing a phrase like “Do Not Shut Off – This is the Production Server”… but that is a far cry from the glass house that we’ve built to nourish and feed the mainframe environment.

Now to be fair, strides are being made to improve the infrastructure and best practices for managing distributed systems. Some organizations have built an infrastructure around their distributed applications that rivals the mainframe glass house. But this is more the exception than the rule. With time, of course, the policies, practices, and procedures for managing distributed systems will improve to mainframe levels.

But the bottom line is that today’s distributed systems – that is, Linux, Unix, and Windows-based systems – typically do not deliver the stability, availability, security, or performance of mainframe systems. As such, a forced tour of duty supporting or developing applications for a mainframe would do every IT professional a whole world of good.

Row and Column Access Control in DB2 Version 10

2012-01-17T14:40:00.002-06:00

Rowand column access control enables you to manage access to a table at the level ofa row, a column, or both. It enables you to build policies for the particularsof which data can be accessed by specific users, groups, or roles. Row accesscan be controlled using row permissions and column access control can beaccomplished using column masks.

Row andcolumn access control differs from multilevel security in that it is integratedinto the database system. All applications and tools that use SQL to access thedatabase are automatically subject to the same control. Sensitive data need notbe filtered at the application level when row and column access control is inplace.

Prior to rowpermissions and column masks, row and column level security was implemented inDB2 using views or stored procedures. Using views and stored procedures is a viable approach for simple requirements, but it breaks down as a solutionfor more complex requirements. When a large number of views are builtto support your security needs, it can be difficult to administer as the viewsneed to be updated and maintained.

Let’s seehow row permissions and column masks can be used to improve upon row- andcolumn-level security.

RowPermissions: Row Access Control

A rowpermission must be created and activated to be enforced. The structure of apermission will be familiar to anyone who is used to coding SQL statements. TheCREATE PERMISSION statement is used to create a row permission.

Let’sconsider an example using a banking system. Assume that bank tellers should only be able to accesscustomers from their local branch. But customer service representatives (CSRs)should be allowed to access all customer data. Assume further, that secondaryauthids are setup such that tellers have a secondary authid of TELLER, and CSRshave a secondary authid of CSR. Given this scenario, the following row permissions can becreated to institute these policies:

CREATE PERMISSION TELLER_ROW_ACCESS
ON     CUST
FOR ROWS WHERE VERIFY_GROUP_FOR_USER(SESSION_USER, ′TELLER′) = 1
AND
BRANCH = (SELECT HOME_BRANCH
          FROM   INTERNAL_INFO
          WHERE  EMP_ID = SESSION_USER)
ENFORCED FOR ALL ACCESS
ENABLE;

COMMIT;

CREATE PERMISSION CSR_ROW_ACCESS
ON     CUST
FOR ROWS WHERE VERIFY_GROUP_FOR_USER(SESSION_USER, ′CSR′) = 1
ENFORCED FOR ALL ACCESS
ENABLE;

COMMIT;





These row permissions will not be enforced, however, until they are activated by alteringthe table, for example:

ALTER TABLE CUST
 ACTIVATE ROW ACCESS CONTROL;

COMMIT;

With the row permissions in force, when tellers SELECT from the CUST table they will only be able to “see” customer data for their branch, whereas customer service representatives can see all customer data.

These row permission definitions use the VERIFY_GROUP_FOR_USERbuilt-in function. This function returns a valueindicating whether the primary authid and the secondary authids that areassociated with the first argument are in the authorization names specified inthe list of the second argument.

Data Masking: Column Access Control

Column access control allows you to manage access to atable with filtering and data masking. As with a row permission, a column maskmust be created and activated before it can be enforced. The column maskdefines the rules to be used for masking values returned for a specifiedcolumn.

You use the CREATE MASK statement to create a column mask.Multiplecolumn masks can be created for a table, but each column can have only onemask. The table and column must exist before the mask can be created.

For example, you can create a mask for employee social securitynumbers (assuming the table name is EMP and the column name is SSN) as follows:

CREATE MASK SSNMASK
ON     EMP
FOR COLUMN SSN RETURN
  CASE
    WHEN (VERIFY_GROUP_FOR_USER(SESSION_USER, ′PAYROLL′) = 1)
    THEN SSN
    WHEN (VERIFY_GROUP_FOR_USER(SESSION_USER, ′HR′) = 1)
    THEN ′XXX-XX-′ || SUBSTR(SSN,8,4)
    ELSE NULL
  END
ENABLE;

COMMIT;

This mask will return the actual data when accessed by a user in accounting, a version with the first 5 digits masked when access by human resources, and null for anyone else. Of course, column access control must be activated for the table before any mask will be enforced:

ALTER TABLE EMP
 ACTIVATE COLUMN ACCESS CONTROL;
 
COMMIT;

Summary

Using row and column access control a security administrator canenforce detailed security policies for the databases under their control.

Seasons Greetings

2011-12-20T22:11:00.003-06:00

Here's wishing all of my readers a very happy holiday season... Be safe and enjoy the holidays and we'll see you again in 2012!

Tune That SQL to Improve DB2 Performance!

2011-11-19T23:37:00.001-06:00

Structured Query Language, better known as SQL, is apowerful tool for manipulating data. It is used in virtually every relationaldatabase management system on the market today, not just by DB2, but also by Oracle,Sybase, MySQL, and Microsoft SQL Server.

SQL is a high-level language that provides a greater degreeof abstraction than do procedural languages. Most programming languages requirethat the programmer navigate data structures. The navigation information isencoded in the program and is difficult to change after it has been programmed.

SQL is different. It is designed so that the programmer canspecify what data is needed, and not how to retrieve it. A DB2 applicationprogrammer will use SQL to define data selection criteria. DB2 analyzes SQL andformulates data-navigational instructions “behind the scenes.” Thesedata-navigational instructions are called accesspaths. By having the DBMS determine the optimal access path to the data, aheavy burden is removed from the programmer. The database has a betterunderstanding of the state of the data it stores, and thereby can produce amore efficient and dynamic access path to the data. The result is that SQL,used properly, can provide for quicker application development.

Quick application development is a double-edged sword. Whileit can mean reduced application development time and lowered costs, it can alsomean that testing and performance tuning are not thoroughly done. The task oftuning the database as well as optimizing the SQL typically falls to thedatabase administrator (DBA).

The DB2 environment and its host system can be tuned toachieve a certain level of performance improvement, but the greatest potentialfor performance improvement comes from analyzing the SQL code itself and makingchanges to improve speed and efficiency. The consensus among SQL performanceexperts is that 80% or more of database performance problems are caused byimproperly written and un-tuned SQL.

SQL Query Tuning

SQL is not merely a query language. It can also define datastructures, control access to the data, and insert, modify, and delete data. Consolidatingthese functions into a single language eases communication between differenttypes of users.

SQL is, by nature, quite flexible. It uses a free-formstructure that gives the user the ability to develop SQL statements in a waybest suited to the given user. Each SQL request is parsed by the DBMS beforeexecution to check for proper syntax and to optimize the request. Therefore,SQL statements do not need to start in any given column and can be strungtogether on one line or broken apart on several lines. Any SQL request could beformulated in a number of different but functionally equivalent ways.

SQL’s flexibility makes it intrinsically simple, butflexibility is not always a good thing when it comes to performance. Differentbut equivalent SQL formulations can result in extremely variable performance. Inthis section, we’ll talk about some of the tools within DB2 to help optimizeperformance and we’ll get into some of the things to watch for in the codeitself.

Queries Built for Speed

When you are writing your SQL statements to access DB2 data,keep in mind the three fundamental guidelines listed in this section. These aresimple, yet important rules to follow when writing your SQL statements. Ofcourse, SQL performance is a complex topic and to understand every nuance ofhow SQL performs can take a lifetime. That said, adhering to the followingsimple rules puts you on the right track to achieving high-performing DB2applications.

Always provide only the exact columns that you need to retrieve in the SELECT-list of each SQL SELECT statement.
Another way of stating this is “do not use SELECT *”. The shorthand SELECT * means retrieve all columns from the table(s) being accessed. This is fine for quick and dirty queries but is bad practice for inclusion in application programs because:

DB2 tables may need to be changed in the future to include additional columns. SELECT * will retrieve those new columns, too, and your program may not be capable of handling the additional data without requiring time-consuming changes.
DB2 will consume additional resources for every column that requested to be returned. If the program does not need the data, it should not ask for it. Even if the program needs every column, it is better to explicitly ask for each column by name in the SQL statement for clarity and to avoid the previous pitfall.

Of course, simply avoiding SELECT *is not sufficient. You also have to avoid returning certain columns…

Do not ask for what you already know.
This may sound simplistic, but most programmers violate this rule at one time or another. For example, consider what is wrong with this simple query:

SELECT LASTNAME, FIRST_NAME, JOB_CODE,DEPTNO

FROM EMP

WHERE JOB_CODE = 'A'

AND DEPTNO = 'D01';

Look at the SELECT-list. There arefour columns specified but only two of them are needed. We know that JOB_CODEwill be A and DEPTNO will be D01 because we told DB2 to only return those rowsusing the WHERE clauses. Every column that DB2 has to access and return to ourprogram adds overhead. Yes, it is a small amount of overhead here, but thisstatement may be run hundreds, or even thousands, of times a day. And thatsmall overhead adds up to significant overhead.

Use the WHERE clause to filter data in the SQL instead of bringing it all into your program to filter.
This too is a common rookie mistake. It is much better for DB2 to filter the data before returning it to your program. This is so because DB2 uses additional I/O and CPU resources to obtain each row of data. The fewer rows passed to your program, the more efficient your SQL will be.

Look for IF-THEN-ELSE logic or CASEstatements immediately following the FETCH statements in your applicationprograms. If the conditional logic code is analyzing columns that you justretrieved from DB2, try to remove them from the host language code instead buildingthe tests into WHERE clauses in your SQL statements. Doing so will improveperformance.

Follow good SQL coding practices (like these threeguidelines), and you’ll start seeing a performance improvement in your DB2applications. To further tune the code, you’ll need to understand how toleverage the optimizer, update statistics, and manage indexes.

Leveraging the Optimizer

The optimizer is the heart and soul of DB2. It analyzes SQLstatements and determines the most efficient access path available forsatisfying each statement. It accomplishes this by parsing the SQL statement todetermine which tables and columns must be accessed. It then queries systeminformation and statistics stored in the DB2 system catalog to determine thebest method of accomplishing the tasks necessary to satisfy the SQL request.

The optimizer is essentially an expert system for accessingDB2 data. An expert system is a set of standard rules that when combined withsituational data can return an expert opinion. For example, a medical expertsystem takes the set of rules determining which medication is useful for whichillness, combines it with data describing the symptoms of ailments, and appliesthat knowledge base to a list of input symptoms. The DB2 optimizer rendersexpert opinions on data retrieval methods based on the situational data housedin DB2’s system catalog and a query input in SQL format.

The notion of optimizing data access in the DBMS is one ofthe most powerful capabilities of DB2. Remember, access to DB2 data is achievedby telling DB2 what to retrieve, not how to retrieve it. Regardless of how thedata is physically stored and manipulated, DB2 and SQL can still access thatdata. This separation of access criteria from physical storage characteristicsis called physical data independence.DB2’s optimizer is the component that accomplishes this physical dataindependence.

If indexes are removed, DB2 can still access the data(albeit less efficiently). If a column is added to the table being accessed,the data can still be manipulated by DB2 without changing the program code.This is all possible because the physical access paths to DB2 data are notcoded by programmers in application programs, but are generated by DB2.

Compare this with non-DBMS systems in which the programmermust know the physical structure of the data. If there is an index, theprogrammer must write appropriate code so that the index is used. If the indexis removed, the program will not work unless changes are made. Not so with DB2and SQL. All this flexibility is attributable to DB2’s capability to optimizedata manipulation requests automatically.

The optimizer performs complex calculations based on a hostof information. To simplify the functionality of the optimizer, you can pictureit as performing a four-step process:

Receive and verify the syntax of the SQL statement.
Analyze the environment and optimize the method of satisfying the SQL statement.
Create machine-readable instructions to execute the optimized SQL.
Execute the instructions or store them for future execution.

The second step of this process is the most intriguing. Howdoes the optimizer decide how to execute the vast array of SQL statements thatcan be sent its way?

The optimizer has many types of strategies for optimizingSQL. How does it choose which of these strategies to use in the optimizedaccess paths? IBM does not publish the actual, in-depth details of how theoptimizer determines the best access path, but the optimizer is a cost-basedoptimizer. This means that the optimizer will always attempt to formulate anaccess path for each query that reduces overall cost. To accomplish this, theDB2 optimizer applies query cost formulas that evaluate and weigh four factorsfor each potential access path: the CPU cost, the I/O cost, statisticalinformation in the DB2 system catalog, and the actual SQL statement.

The Importance of Statistics

Without the statistics stored in DB2’s system catalog, theoptimizer will have a difficult time optimizing anything. These statisticsprovide the optimizer with information about the state of the tables that willbe accessed by the SQL statement that is being optimized. The types ofstatistical information stored in the system catalog include:

Information about tables including the total number of rows, information about compression, and total number of pages.
Information about columns including number of discrete values for the column and the distribution range of values stored in the column.
Information about table spaces including the number of active pages.
Current status of the index including whether an index exists or not, the organization of the index (number of leaf pages and number of levels), the number of discrete values for the index key, and whether the index is clustered.
Information about the table space and partitions.

Statistics are gathered and stored in DB2’s system catalogwhen the RUNSTATS utility is executed. Be sure to work with your DBA to ensurethat statistics are accumulated at the appropriate time, especially in aproduction environment.

Index for Performance

Perhaps the single most important thing that can be done toassure optimal DB2 application performance is creating correct indexes for yourtables based on the queries used by your applications. Of course, this iseasier said than done. But we can start with some basics. For example, considerthe following SQL statement:

SELECT LASTNAME,SALARY

FROM EMP

WHERE EMPNO ='000010'

AND DEPTNO = 'D01';

What index or indexes would make sense for this simplequery? The short answer is “it depends.” Let’s discuss what it depends upon!First, think about all of the possible indexes that could be created. Yourfirst short list probably looks something like this:

Index1on EMPNO
Index2on DEPTNO
Index3on EMPNO and DEPTNO

This is a good start and Index3 is probably the best of the lot. It allowsDB2 to use the index to immediately lookup the row or rows that satisfy the twosimple predicates in the WHERE clause. Of course, if you already have a lot ofindexes on the EMP table you might want to examine the impact of creating yetanother index on the table. Factors to consider include:

Modificationimpact
Columnsin the existing indexes
Importanceof a particular query

Modification Impact

DB2 will automatically maintain every index that you create. This meansthat every INSERT and every DELETE to this table will cause data to be insertedand deleted not just from the table, but also from its indexes. And if youUPDATE the value of a column that is in an index, the index will also beupdated. So, indexes speed the process of retrieval but slow down modification.

Columns in the Existing Indexes

If an index already exists on EMPNO or DEPTNO it might notbe wise to create another index on the combination. However, it might makesense to change the other index to add the missing column. But not alwaysbecause the order of the columns in the index can make a big difference inaccess path selection and performance, depending on the query. Furthermore, if indexes already exist for bothcolumns, DB2 potentially can use them both to satisfy this query so creatinganother index may not be necessary.

Importance of this Particular Query

The more important the query the more you may want to tuneby index creation. For example, if you are coding a query that will be runevery day by the CIO, you will want to make sure that it performs optimally.Who wants to risk a call from the CIO complaining about performance? So buildingindexes for that particular query is very important. On the other hand, a queryfor a low-level clerk may not necessarily be weighted as high, so that querymay have to make due with the indexes that already exist. Of course, thedecision will depend on the importance of the application to the business – notjust on the importance of the user of the application. An additional criterionto factor into your decision is how often the query is run. The more frequentlythe query needs to be executed during the day, the more beneficial it becomesto create an index to optimize it.

There is much more to index design than we have covered sofar. For example, you might consider index overloading to achieve index onlyaccess. If all of the data that a SQL query asks for is contained in the index,DB2 may be able to satisfy the request using only the index. Consider ourprevious SQL statement. We asked for LASTNAME and SALARY given informationabout EMPNO and DEPTNO. And we also started by creating an index on theEMPNO and DEPTNO columns. If we include LASTNAME and SALARY in the index aswell then we never need to access the EMP table because all of the data we needexists in the index. This technique can significantly improve performancebecause it cuts down on the number of I/O requests.

Keep in mind, though, that it is not prudent (or evenpossible) to make every query an index only access. This technique should besaved for particularly troublesome or important SQL statements. And you shouldalways examine the impact on other queries and programs when deciding whetherto add columns to any index.

Summary

Properly tuned SQL and a well-tuned DB2 environmentcan yield noticeable performance improvements. These can mean faster responsetime for DB2 applications, a better user experience, and faster throughput. Thekey is a combination of programming practice, system optimization, andeffective use of software tools to automate simulation and code analysis.

IBM Information on Demand 2011: Day Four (#IODGC)

2011-10-26T23:37:00.004-05:00

The highlights of my fourth day at the IOD conference in Las Vegas was the general session with Michael Lewis and Billy Beane.

Billy Beane is the the general manager (as well as a minority owner) of the Oakland Athletics. Michael Lewis is the author of the book, Moneyball, that outlines how Beane revolutionized baseball analytics by focusing on different statistics than the traditional RBI and batting average. Indeed, the A's analysis showed that on-base percentage and slugging percentage were better predictors of offesnive success, and therefore translated into more wins. Additionally, because other teams were not focusing on those stats it would be easier for a small market team like the A's to acquire talent based on them and compete with the "big boys" like the New York Yankees and Boston Red Sox.

Lewis and Beane were informative and entertaining. Lewis started with a funny tale about waiting to talk to the A's players and seeing them as they walked naked from the showers. He said if you just lined these guys naked, up against a wall, you'd never think they were professional athletes.When Lewis mentioned this to Beane, Beane replied that that was basically the point. He told him "We're in the market for players whose value the market doesn't grasp..."

After this conversation Lewis continued to observe the team operations for awhile. And the light bulb came on. Lewis told Beane "Aha... I see what you are doing here. You are arbitraging the mispricing of baseball players." He recognized it because he had covered Wall Street in the past.

When asked if it took courage to rely on the statistics like he did, Beane countered that it really didn't. With a small market team he had no money to compete against the major market teams using traditional measurement analytics. So, it made sense to use the new statistics that were backed up by rigorous analytics and compete in a non-traditional way.

Beane also discussed how baseball tends to get the 8 best teams in the playoffs each year because they play 162 games and the better teams tend to win over longer periods of time. But in the post season, with best of 5 or best of 7 series, anybody can win. The nugget of wisdom passed on by Beane in this story: "Never make decisions based on short term results." To make his point, Beane said that this year most people would agree that the Philadelphia Phillies were the best team in baseball... but they lost to the St. Louis Cardinals in a best of 5 series in the National League Division Series.

The interview with Lewis and Beane tied in well to the overall theme of the IOD conference, which focused on gaining insight from information through analytics. And that is what Beane achieved and Lewis documented in Moneyball (which is now a major motion picture showing at a theatre near you).

Speaking of motion pictures, not major ones by Hollywood standards but perhaps by DB2 users standards, be sure to keep checking in on the daily IOD video blog that I am hosting at http://www.softbase.com/blog. Today's video blog offers up an interview with Suresh Sane, Database Architect at DST Systems in Kansas City and three-time best user speaker at IDUG.

IBM Information on Demand 2011: Day Three (#IODGC)

2011-10-26T01:07:00.001-05:00

Day three at the IOD conference in Las Vegas started off with a Big Data focus. The first thing in the morning was the general session which began, as usual with a series of factoids and interesting statistics. The highlights of which, in my opinion, were these:

There are over 34,000 Google searches done every second. Which is a huge number, but not unbelieveable....
1 in 3 business leaders make decisions based on data they don't trust. Personally, I think the other 2 are just kidding themselves.
Top 2 leadership challenges according to IBM study: increasing complexity and exploding data volume. (Hey, what about small budgets?)
And in a survey of IOD conference attendees, 55 percent say that the relationship between business and IT is getting better... let's hope... at some shops it couldn't get any worse, could it?

At any rate, the general session kicked off and Katty Kay of BBC America was the emcee again. She did a great job at hosting today's general session (and yesterday's, too). But I have to say, the general sessions are nowhere near as entertaining as they have been at previous IOD events.

Steve Mills then took the stage. For those who don't know Mr. Mills, he is Senior Vice President and Group Executive - Software and Systems. Mills provided an overview of Big Data from an IBM perspective. Mills kicked things off with a pithy quote saying "Everybody is talking about big data these days, as if data wasn't already big." True... he then went on to outline the big data challenge as the 3Vs: variety, velocity, volume. Not a bad start, but a few Vs short in my opinion... should include vicinity and validity.

Another piece of wisdom from Mills' keynote was this: Big Data is not a single structure, but many structures. He also stated that Big Data must be an integral part of the enterprise data platform...

Mills also discussed various examples of big data solutions that IBM was working with customers on, including my personal favorite, analyzing massive volumes of "space weather" data in motion. But perhaps a more "down to Earth" example can be found in IBM helping to analyze sensor data in offshore oil rigs where more than 2 TB of data is being processed on a daily basis.

Later in the day I attended a panel on customer sentiment analysis. Professor Jonathan Taplin, Director of USC's Annenberg Innovation Lab talked about analyzing social media data to measure customer sentiment around various areas including film, fashion, and even the World Series. The information uncovered helped identify movies that would tank and several film studios began working with the Lab to identify customer sentiment earlier in the cycle. After all, how does it help a movie studeo to learn that a major motion picture is about to tank on the Thursday before it opens? The studios worked with the Lab to learn about negative sentiment earlier so the studios could try to reverse the sentiment through marketing efforts.

There are tremendous spinoff opportunities for this type of sentiment analysis because it gives greaterinsight into customer behavior. It can potentially be applied to other areas, too, such as measuring employee sentiment, or perhaps citizen sentiment to help predict and uncover events such as the Arab Spring.

The key take away is to realize that this is a different type of data with low latency, real time applications. It is not the type of data that we will be storing long term in databases or data warehouses.

According to Taplin, the next challenge is realtime analysis using IBM's InfoSphere Streams product.

I also attended a fantastic lunch provided by IBM for IBM Champions. Thank you IBM for the nice spread and recognition. I am proud to be an IBM Champion!

Today was also the day of my presentation and I delivered my DB2 for z/OS Performance Tuning Roadmap to a packed audience. I deliver 60 slides in just a little bit over an hour and the presentation seemed to be well received...

The conference ended with a fantastic concert from the band Train. I liked the band before this, but I really like them after the concert! Let me tell you, the lead singer Patrick Monahan has a heckuva set of pipes. The guy can flat out sing. This became abundantly clear not just in their stellar versions of thier hits (like "Meet Virginia", "Calling All Angels", and "Hey, Soul Sister") but also the fantastic cover versions (especially the "Ramble On/Walk On The Wild Side" mashup). Yes, Monahan did the vocals proud and can match Robert Plant note for note. I did not expect that. And their cover of "Dream On" was pretty good, too!

Finally, don't forget to keep checking in on the video blogs I am hosting for SoftBase Systems. Today's blog interview was with advanced SQL expert, Sheryl Larsen. Check it out by clicking here!

IBM Information on Demand 2011: Day Two (#IODGC)

2011-10-24T19:30:00.001-05:00

As promised, here is the second of my daily blogs from the IOD conference in Las Vegas. Today it was reported that the attendance at the event was the highest ever for an Information On Demand conference; there are more than 11,500 registered attendees.

The second day of the conference is when things really start humming with news and in-depth presentations. The day kicked off with the general session delivered by a hostof IBM executives and customers. Big data, business analytics, and gaininginsight into data was the theme of the session.The opening session was peppered with lots of interesteing facts and figures. For example, did you know that 90 percent of the world's data was created in just the last two years? Me neither... but there was no attribution to that nugget of information, so...

Otherhighlights of the day included the announcement of Cognos Mobile for the iPhone and iPad (afree trial is available on the iTune store)… and the other big product focus of the day was IBM InfoSphere BigInsights, a Hadoop-driven big data solution that can process hugeamounts of data very quickly and accurately. For more details on that offeringcheck out my Data Technology Today blog where I cover a customer implementationof this solution.

I also had the opportunity to chat with IBM's Bernie Spang, Director of Marketing, Database Software and Systems. We chatted about various things, starting with the uptake of DB2 10 for z/OS. Earlier in the day it was stated that the uptake of V10 has been faster than for V9 and I asked Bernie why that was. His answer made a lot of sense: skip-level migration support coupled with a clear performance boost out-of-the-box without having to change the database or the apps. I asked if he had metrics on how many customers had migrated, but he didn't have access to that. He said he would get back to me and when he does I will share that information with you all.

We also chatted quite a bit about the recently announ ced DB2 Analytics Accelator. Bernie thinks this is probably the announcement he is most excited about. For those of you who haven't heard about this great piece of technology, it is the second iteration of the Smart Analytics Optimizer (but that name is now dead). The DB2 Analytics Accelerator is built on Netezza technology and can be used to greatly improve the performance of DB2 for z/OS analytical queries without changing the SQL or any application code. There are multiple value points but Bernie pointed out the application transparency and the ability to keep the data on the z platform (no movement required) while accelerating the performance of analytical queries.

IBM views the competition as Oracle Exadata and Teradata, which makes sense. I asked Bernie if there were plans to incorporate the Oracle compatibility features of DB2 LUW in a future iteration of DB2 for z/OS, and he said that made sense. Of course, no one from IBM will commit to future functionality of an as yet to be announced version, but perhaps Vnext??? (that was me speaking there, no Bernie!)

Then I think I blew his mind when I passed a thought of mine past him. With Netezza being used as a component of an accelerator to improve DB2 analytical processing, has IBM given any thought to using IMS as a component of an accelerator to improve DB2's traditional OLTP processing. Not sure if that is even possible, but it should be worth a research project, right? Especially with IBM announcing IMS 12 at the conference today and the IBM boast that IMS 12 can achieve 61,000 transactions per second. That is impressive! But can the mismatch between relational and hierarchical be overcome in a useful manner to do it?

Finally, we chatted about Informix. As a DB2 bigot I am always at a loss for when to direct people to Informix instead of DB2. It just doesn't sound like something I would do! But Bernie offered a brief overview of Informix time series as something unique that certain customers should look into. An Informix customer uses time series for smart meter management of over 100 million smart meters. A month's worht of data - 4 terabytes - can be loaded and processed in less than 8 hours. And some queries perform from 30x to 60x faster.

OK, even to this DB2 bigot that sounds like an impressive capability. Kudos to Informix.

Finally, I'd like to direct my readers over to the video blog that I am hosting in conjunction with SoftBase Systems. I'll be interviewing DB2 luminaries daily, so tune in there at http://www.softbase.com/blog to view each daily submission!

Until tomorrow...

Information On Demand 2011: Day One (#IOD11)

2011-10-24T01:15:00.000-05:00

Well, the first day of the IOD conference is just about behind us. As usual, Sunday is a day to get acclimated to Vegas and the Mandalay Bay conference center. If you are here, I hope you brought some comfortable shoes, because you'll be doing a LOT of walking.

Typically, the highlight of the first day is the opening of the Expo Hall, and this year was no exception. The hall was jam-packed with IBM booths demonstrating and promoting all kinds of software, from DB2 to Informix to Analytics to InfoSphere to Big Data to Cloud and more. And there were also a large number of ISVs in the Expo Hall, too. It could take most of the week to visit all of the booths and learn about all the great technology on display.

But, of course, we won't be doing that. Tomorrow is the beginning of the educational sessions, kicking off in the morning with the general session, which this year is titled Turning Insight Into Action. Actually, that is the theme of this year's conference, too.

Word is that attendance is up this year over the 10,000 attendees at last year's conference. I haven't heard an official number yet, but I've heard rumors of more than 11,000 attendees this year.

As the week progresses, I will tweet (http://www.twitter.com/craigmullins) my experiences, and blog about the conference daily. So be sure to check back here, as well as on my Twitter feed, for the straight scoop from IOD.

To end today blog posting on a high note, here are a few facts about the latest IBM Information Management and Business Intelligence activities:

IBM projects $16 billion in business analytics software and services revenue by 2015
Over the past 5 years, IBM has invested more than $14 billion in 25 key acquisitions including Cognos, Netezza, and SPSS (and many others)
IBM is committed to researching advanced analytics technologies as demonstrated by Watson (who is here at the conference) and IBM's $100 millions investment to develop new tools toward tackling Big Data challenges.
Analytics software and services for IBM were up 17 percent in their second quarter

Also, remember that I will be videotaping highlights and interviews from the conference this year in conjunction with SoftBase Systems. You can find links to these videos as the become available daily at https://www.softbase.com/blog/.
Goodbye for now... Hope to see you all again tomorrow as we discuss day two of the conference...

Information On Demand 2011( #IODGC)

2011-10-22T22:01:00.003-05:00

Just a quick post today, Saturday, October 22nd 2011, to let everybody know that I will be blogging daily from the IOD conference in Las Vegas this week.

I'll try to keep my readers up-to-date on what is going on by posting my thoughts about the conference, covering the news and announcements that are made, and by working with SoftBase Systems to produce daily videos on the news of the day along with daily interviews of DB2 luminaries... so whether you can't make the conference this year, or can but want to keep abreast of things, keep checking back here for more daily details from IOD.

Let's start by letting everybody know that I will be presenting "IBM DB2 Performance Tuning Roadmap" on Tuesday, 10/25, at 2:00. I'm just one of 59 IBM Champions that will be presenting at this year's IOD conference. Here is a list if you are interested.

DB2 Developer's Guide, 6th edition

2011-10-18T15:08:00.003-05:00

I know a lot of my readers are waiting on the updated edition of my book, DB2 Developer's Guide. This short blog post is to let you know that the wait is almost over. The book will be published early next year and is available to be pre-ordered on amazon com.

The book has been completely updated and is now up-to-date with DB2 10 for z/OS. Just think of the things that have been added to DB2 since the last time the book was updated: Universal table spaces, pureXML, SECADM, hashes, new data types, INSTEAD OF triggers, temporal support, and much, much more.

Consider pre-ordering a copy today so you'll get it as soon as it comes off the presses!

IBM announces Smart Analytics System 5710

2011-09-28T19:50:00.002-05:00

Last week (September 2011), IBM announced the Smart Analytics System 5710, which is a database appliance for business intelligence and data analytics targeted at the SMB market. The IBM Smart Analytics System 5710 is based on IBM System x, runs Linux, and includes InfoSphere Warehouse Departmental Edition and Cognos 10 Business Intelligence Reporting and Query.

The announcement of this appliance was somewhat lost in the shuffle of Oracle's marketing blitz for its similar Oracle Database Appliance, also announced last week. But IBM's offering is geared and pre-configured for quick deployment of analytics and business intelligence capabilities.

The IBM Smart Analytics System 5710 is powered by the InfoSphere Warehouse Departmental Edition which is built on a DB2 data server, and features Optim Performance Manager, DB2 Workload Manager, Deep Compression and Multidimensional clustering.

The IBM Smart Analytics System 5710 provides key capabilities of reporting, analysis and dashboards to enable fast answers to key business questions delivered as a cost-effective solution designed for rapid deployment. It allows users to quickly extract maximum insight and value from multiple data sources and deliver a consistent view of information across all business channels.

It also provides cubing services giving users a multidimensional view of data stored in a relational database. Users can create, edit, import, export, and deploy cube models over the relational warehouse schema to perform deeper multi-dimensional analysis of multiple business variables improving both profitability and customer satisfaction. Cubing services also provide optimization techniques to dramatically improve the performance of OLAP queries.

Additionally, the powerful, yet simple, data mining capabilities enable integrated analytics of both structured and unstructured data in the system. Standard data mining models are supported and can be developed via drag and drop in an intuitive design environment.

So what does it cost? For a such a rich collection of software, the starting price is just under $50K. Furthermore, the new offering is part of the IBM Smart Analytics System family, which consists of solutions that span multiple hardware platforms and architectures, including the mainframe (System z).

DB2 10 for z/OS: For Developers Only!

2011-09-01T14:15:00.005-05:00

Today's blog post is to promote an upcoming FREE webinar that I will be delivering titled DB2 10: For Developers Only! The presentation is sponsored by the good folks at SoftBase Systems. and it will be conducted on Wednesday, September 14th, 2011, from 2:00 to 3:00 pm EST.

This presentation highlights the DB2 10 for z/OS enhancements that directly impact DB2 application developers. Every release of DB2 is chock full of new features and functionality and that can make it hard to focus on those things that are most helpful for programmers. So instead of scanning volumes of manuals, you can watch this presentation distills the DB2 10 information down to cover what should be most important to programmer/analysts.

Examples of areas this presentation will cover include:
• Binding issues and details for V10
• Temporal support with examples
• A new type of function
• New timestamp options and some improvements to existing SQL
• Implicit casting, access to currently committed data, and much more…

If you are a programmer wanting to learn more about DB2 10, or a DBA looking for the programmer’s perspective on DB2 10, this presentation should have something to offer you.

This is sort of a tradition for me... you may have heard me give a similar presentation for previous DB2 versions. Well, this webinar introduces a brand new presentation in this series, this time for DB2 Version 10...

So register today !

DB2 Symposium 2011 – Round Two

2011-08-24T09:57:00.001-05:00

Today's blog post is about the DB2 Symposium, a three day training event with one day seminars presented by well-known DB2 consultants. I was fortunate enough to be asked to participate this year by the primary organizer of the event, Klaas Brant. (Klaas is a respected DB2 consultant based in the Netherlands.). Earlier this year, the DB2 Symposium event was held in Dallas, TX and was well-received by attendees. So a second round is planned, this time in Chicago, IL!

What is the difference between DB2 Symposium and events like IDUG and IOD? Well, DB2 Symposium fills the gap between a conference and a multi-day training course. The DB2 Symposium is unique because you can participate for 1, 2, or 3 days, depending on your needs and budget.

Round two of the USA DB2 Symposium is happening soon, so you'll need to act fast if you want to participate. It occurs September 21-23, 2011 in the Chicago, Illinois area. More precisely, at the DoubleTree Hotel in Downers Grove, IL (in the Western suburbs of Chicago). Each day the training sessions start at 9.00am and end at around 5.00pm.

But registration on site is not possible, you must pre-register online... so plan ahead!

My session is on September 23rd and it is called DB2 Developer's Guide Comes Alive! This one day session, covers tips, techniques, and procedures you need to know in order to excel at administering and using DB2 on the mainframe. The material is based upon DB2 Developer's Guide, the best-selling DB2 for z/OS book on the market. Additionally, the course material will contain references to sections of the book for students to find additional material on each topic after the sessions. Topics to be covered will include:

A performance tuning roadmap for managing DB2 application, database and system performance. You will learn SQL coding and tuning techniques, guidance for database optimization and reorganization, coverage of buffer pool settings and parameters for performance.
Logical and physical database design recommendations for DB2, so you can build and maintain effective DB2 databases immediately. Includes discussion of standards, logical to physical translation, data types, usage of nulls, and more.
Information and guidance on BINDing and REBINDing, along with a discussion of the most important parameters.
Along the way we'll look at locking, access paths, statistics, indexing and more.
And even though the current edition of the book covers through DB2 V8, this course adds coverage of some of the newer features added to DB2 in versions 9 and 10 that can boost your productivity and performance.

If you own the book already, bring it along and I'll be happy to autograph it for you. And then you can use it along with the course materials... and if you don't own it already, you'll probably want to grab a copy after attending the seminar... you can always find a link to buy my books on the front page of my web site at http://www.craigsmullins.com.

So register for the DB2 Symposium today... and I'll see you in Chicago!

Plan to Attend IDUG This November to Improve Your DB2 Skills

2011-08-23T12:48:00.005-05:00

It is almost that time of year again, time for the annual European IDUG conference. And as anyone who has ever attended this conference can tell you, it will be chock full of news, education, and networking opportunities for DB2 programmers, DBAs, sysprogs, and users! And you can take advantage of great offers including FREE IBM certification training and DB2 migration workshops!

So be sure to register for the IDUG DB2 Tech Conference in Prague, Czech Republic, 13-18 November 2011, at the Clarion Congress Hotel Prague. If you register on or before 17 October 2011 and receive a discount of EUR 275.

IDUG is also extending the following special offers:

Multiple Delegate Discount
For every three individuals who register from the same organisation, a fourth may attend at the discounted rate of EUR 730.
Mentor Program
If you have attended three previous IDUG DB2 Conferences, you are eligible to bring a first-time colleague to Prague for an 80% discount off the full registration fee.
One-Day Educational Seminars - Friday, 18 November 2011
Registration for paid full conference delegates is EUR 450; the cost for just the one-day seminar is EUR 495. Select from the following session topics:
- DB2 10 for z/OS - In Depth, Phil Grainger, Cogito
- DB2 Intermediate and Advanced SQL, Daniel Luksetich, Yevich Lawson & Assoc Inc.
- I Didn't Know DB2 did THAT!, Bonnie Baker, Bonnie Baker Corporation
- Optimising DB2 for z/OS System Performance Using DB2 Statistics Trace, John Campbell, IBM Corporation
- Rocket Science: DB2 for LUW Performance Analysis and Tuning Workshop, Scott Hayes, DBI

Do You Still Use That DB2 Program?

2011-08-12T12:11:00.002-05:00

A recurring problem for programmers and IT folks in general is determining whether or not a particular program is still required. As your organization grows and the number of programs increases, keeping track of them all can become quite difficult.

As administration and management burdens increase, a common desire is to get rid of programs that are no longer being used. But it can be difficult to determine which programs are no longer used.

You can always “ask around,” but few IT professionals would be willing to delete anything based on such a general tactic. Another common method is to review performance reports or extracts from a performance warehouse. But perhaps your performance traces are not turned on all the time.

The question is probably more common in DB2 environments because of the plans and packages that consume storage and "sit around" taking up space if their associated program is no longer being used.

Well, for DB2 professionals this type of question becomes easier to answer once you migrate to DB2 10 for z/OS. DB2 maintains a new column, LASTUSED, in the DB2 Catalog. The column exists in both SYSIBM.SYSPACKAGE and SYSIBM.SYSPLAN and is defined as a DATE data type. The date is changed when the package header is requested from EDM. The column is also maintained for triggers and stored procedures. After BIND (REPLACE) this column is reset to the default value (CURRENT DATE).

This is similar to the LASTUSED column added to SYSIBM.SYSINDEXSPACESTATS in DB2 9, which is used to show the last time an index was used.

Of course, you will have to give it some time –- because you might have a program that is used only rarely, yet still used. Most shops have queries and programs that run quarterly, or even annually, but nevertheless are very important. So don’t just start freeing packages a month after you’ve migrated to DB2 10!

But it is good to know that we now have additional usage information at our fingertips in the DB2 Catalog, isn’t it?

Bad Database Standards

2011-07-25T11:26:00.002-05:00

Today's blog post is a metapost, of sorts. I am using my DB2 blog to point you to a series of posts I made in my other blog (Data and Technology Today). You see, I write for two blogs, this one that focuses on DB2 and mainframe topics, and another one on data and database management topics in general.

One of my more popular series of posts on Data and Technology Today was the one on bad database standards. The general idea of these posts is to dissect and criticize standards that are outdated, or simply wrong. The seven part series ran about a year ago and has generated a bunch of comments. And it will be useful to DB2 folks to read these posts, too.

So without any further ado, here are links to the seven bad database standards:

Take a moment to click through to these links and peruse the "bad" standards and, if you are so inclined, post a comment (either here on this blog, or over on the other one) to share your "favorite?" bad standards.

Updating DB2 Developer's Guide (6th edition)

2011-07-19T10:49:00.003-05:00

Just a brief blog post today to inform regular readers who might be concerned that I have not been blogging as frequently as in the past.

I am currently working on updating my book, DB2 Developer's Guide, for DB2 versions 9 and 10. This will be the sixth edition of the book and if all goes as planned, it should be available on IBM Press before the end of the year (2011).

Be sure to keep an eye on this blog and my web site (http://www.craigsmullins.com) for further details about the next edition of the book.

The PIECESIZE Clause

2011-06-19T16:52:00.002-05:00

One of the more troubling aspects of DB2 database design and creation is the non-partitioning index (NPI). Creating NPIs on tables in a partitioned table space can pose management and performance issues. Partitioned table spaces tend to be large and by their very design will span multiple underlying data sets. Any partitioning indexes will also span multiple data sets. But what happens when you need to define non-partitioning indexes on a table in a partitioned table space?

The PIECESIZE clause of the CREATE INDEX statement can be used during index creation to break an NPI into several data sets (or "pieces"). More accurately, the PIECESIZE clause specifies the largest data set size for a non-partitioned index. PIECESIZE can be specified in kilobytes, megabytes, or gigabytes. For example, the following statement will limit the size of individual data sets for the XACT2 index to 256 megabytes:

CREATE UNIQUE INDEX DSN8910.XACT2
ON DSN8910.ACT (ACTKWD ASC)
USING STOGROUP DSN8G910
PRIQTY 65536K
SECQTY 8192K
ERASE NO
BUFFERPOOL BP0
CLOSE NO
PIECESIZE 256M;

Basically, PIECESIZE is used to enable NPIs to be created on very large partitioned table spaces. It breaks apart the NPI into separate pieces that can be somewhat managed individually. Without PIECESIZE, NPIs would be quite difficult to manage and administer. Keep in mind, though, that PIECESIZE does not magically partition an NPI based on the partitioning scheme of the table space. This is a misperception held by some. So, if you have a partitioned table space with 4 partitions and then create an NPI with 4 pieces, the data in the NPI pieces will not match up with the data in the 4 partitions.

When using PIECESIZE, more data sets will be created and therefore you can obtain greater control over data set placement. Placing the pieces on separate disk devices can help to reduce I/O contention for SQL operations that access NPIs during read or update processing. The elapsed time improvement may be even greater when multiple tasks are accessing the NPI.

Separating the NPI into pieces allows for better performance of INSERT, UPDATE and DELETE processes by eliminating bottlenecks that can be caused by using only one data set for the index. The use of pieces also improves concurrency and performance of heavy INSERT, UPDATE, and DELETE processing against any size partitioned table space with NPIs.

Keep in mind that PIECESIZE is only a specification of the maximum amount of data that a piece (that is, a data set) can hold and not the actual allocation of storage, so PIECESIZE has no effect on primary and secondary space allocation. Each data set will max out at the PIECESIZE value, so specifying PRIQTY greater than PIECESIZE will waste space. But also make sure that you avoid setting the PIECESIZE too small. A new data set will be allocated each time the PIECESIZE threshold is reached. DB2 will increment the A001 component of the data set name each time. Ideally, the value of your primary quantity and secondary quantities should be evenly divisible into PIECESIZE to avoid wasting space.

To choose a PIECESIZE value, divide the overall size of the entire NPI by the number of data sets that you wish to have. For example, for an NPI that is 8 megabytes, you can arrive at 4 data sets for the NPI by specifying PIECESIZE 2M. Of course, if your NPI grows over 8 megabytes in total you will get additional data sets. Keep in mind that 32 pieces is the limit if the underlying table space is not defined with DSSIZE 4G or greater. The limit is 254 pieces if the table space is defined as DSSIZE 4G or greater.

Mainframe Specialty Processors

2011-06-01T11:54:00.003-05:00

Anyone who uses an IBM z Series mainframe has probably heard about zIIPs and zAAPs and other specialty processors. But maybe you haven't yet done any real investigation into what they are, what they do, and why they exist. So, with that in mind, let's take a brief journey into the world of specialty processors in today's blog entry!

Over the course of the past decade or so, IBM has introduced several different types of specialty processors. The basic idea of a specialty processor, is that it sits alongside the main CPUs and specific types of "special" workload is shuttled to the specialty processor to be run there, instead of on the primary CPU complex. Why is this useful or interesting to mainframe customers? Well, the specialty processor workload is not subject to IBM (as well as many ISVs) licensing charges... and, as any mainframer knows, the cost of software rises as capacity on the mainframe rises. But if capacity can be redirected to a specialty processor, then software license charges do not accrue -- at least for that workload.

And for VWLC customers, shuttling workload to a specialty processor can reduce the rolling four hour average and thereby decrease your monthly IBM software license bill.

Another benefit of the specialty processors is that can be cheaper to acquire than standard CPUs.

But specialty processors can only run certain types of workloads. There are four types of specialty processors:

ICF: Internal Coupling Facility - used for redirecting coupling facility cycles in a data sharing environment.
IFL: Integrated Facility for Linux - used for processing zLinux workload on an IBM mainframe.
zAAP: Application Assist Processor - used for Java workload
zIIP: Integrated Information Processor - used for processing certain, distributed database workloads.

When you activate any of these processors, some percentage of that type of workload can be redirected off of the main CP onto the specialty processor... but not 100% of the workload. It can be frustrating, particularly with the zIIP, to determine exactly what is redirected exactly when and exactly how much of it. In general, distributed DB2 for z/OS workload and XML processing can be redirected to zIIP processors.

Additionally, to run on a zIIP, the workload must run under an enclave SRB. So, code written to execute under a TCB will usually be unable to execute under an SRB without major changes. If you didn't understand that sentence, don't worry about it too much. Basically, IBM has enabled certain types of (mostly DB2) workload to run on zIIPs, and ISVs have enabled some of their code to run on zIIPs, too. If you are interested, more details about zIIPs can be found at this link.

Another interesting tidbit is that zAAP-eligible workloads can be run on zIIPs with IBM's zAAP on zIIP support. This can be a boon to some shops that only have zIIPs and no zAAPs. Now, with zAAP on zIIP support, you can use zIIP processors for both Java and distributed DB2 workloads. The combined eligible TCB and enclave SRB workloads might make the acquisition of a zIIP cost effective.This capability also provides more value for customers having only zIIP processors by making Java- and XML-based workloads eligible to run on existing zIIPs.

To take advantage of zAAP on zIIP, you need to be running z/OS V11.1 (or z/OS V1.9 or V1.10 with the PTFs for APAR OA27495) on a z9, z10, or z196 server.

Keep in mind, that the terms for specialty processors do not change. You can only have 1 zAAP and 1 zIIP per each general purpose processor. So, even if you have zAAP on zIIP configured, the chip is still a zIIP and you cannot have any more than 1 per general purpose processor.

The Bottom Line

The bottom line is that even though it can take some studying and research to understand their benefit and functionality, specialty processors can help to reduce the cost of mainframe computing... and that is a good thing!

What is an Enclave?

2011-06-01T11:51:00.001-05:00

If you are a DB2 professional dealing with distributed workload… or if you are enabling zIIP specialty processors… chances are you’ve heard the term “enclave” or “enclave SRB.” But just what is an enclave?

An enclave is a construct that represents a transaction or unit of work. Enclaves are a method of managing mainframe transactions for non-traditional workloads. You can think of an enclave as an anchor point for resource accumulation regardless of where the transaction is executing.

With traditional workloads it is relatively easy to map the resources consumed to the actual transaction doing the consumption. But with non-traditional workloads – web transactions, distributed processing, etc. – it is more difficult because the transaction can span platforms. Enclaves are used to overcome this difficulty by correlating closely to the end user’s view of the transaction.

So even though a non-traditional transaction can comprise multiple “pieces” spanning many server address spaces, and can share those address spaces with other transactions, the enclave gives you more effective control over the non-traditional workload.

If you are interested in more details on enclaves and how they are managed, read through Enclaves – Managing Business Transactions from IBM’s RMF Newsletter.

A Quick SQL Trick: Find The Number of Commas

2011-05-25T16:46:00.002-05:00

Today's blog post is a short one. I was recently asked how to return a count of specific characters in a text string column. For example, given a text string, return a count of the number of commas in the string.

This can be done using the LENGTH and REPLACE functions as follows:

SELECT LENGTH(TEXT_COLUMN) - LENGTH(REPLACE(TEXT_COLUMN, ',' ''))

The first LENGTH function simply returns the length of the text string. The second iteration of the LENGTH function in the expression returns the length of the text string after replacing the target character (in this case a comma) with a blank.

So, let's use a string literal to show a concrete example:

SELECT LENGTH('A,B,C,D') - LENGTH(REPLACE('A,B,C,D', ',', ''))

This will translate into 7 - 4... or 3. And there are three commas in the string.

When confronted with a problem like this it is usually a good idea to review the list of built-in SQL functions to see if you can accomplish your quest using SQL alone.

DB2 -- What's in a Name?

2011-05-13T17:06:00.002-05:00

Versions of DB2 exist for a large array of platforms, of which the mainframe (z/OS) is only one. Of course, it is my favorite one since I’ve been working on mainframe technology now for decades and have worked with DB2 since Version 1.

It used to be easy: DB2 meant IBM’s mainframe SQL database management system based on the relational model. But you can’t just say the term “DB2” any more and expect people to understand what you mean.

Today there are variations of DB2 that run on the iSeries (AS/400), on Linux, Unix, and Windows (LUW) platforms, and even one that runs on PDAs and smart phones called DB2 Everyplace. Not to mention the mainframe variations that run on z/OS, VM, and VSE.
These products are all collectively referred to by IBM as the DB2 Family. Individually, each DBMS is referred to as DB2, or sometimes DB2 Universal Database Server. There was a period of time when DB2 for LUW was called UDB and DB2 for z/OS was just called DB2. Then IBM tried to rebrand both as DB2 UDB. But that seems to have gone away several versions ago now.
The proper way to refer to any individual offering in the DB2 family is DB2 for (operating system) (for example, DB2 for z/OS or DB2 for Windows).

Different Code Bases

There are four distinct code bases for the products under the DB2 brand. The mainframe has its own code base, as does the iSeries, and VSE/VM. The fourth code base is for Linux, Unix, and Windows (LUW) platforms—and the other DB2 offerings (e.g. DB2 Everyplace) originate from this code base.

Having a separate code base means that each of these DB2 “products” was developed independently from the others. So, for example, the process used by DB2 for z/OS to optimize SQL differs from the process used by DB2 for Linux. Usually, though, the result is similar—an efficient SQL statement.

But keep in mind that there will be some differences between the DB2s.

Some of the Differences

It is obvious that the different DB2 products are not “plug and play” commodities simply because they all share the name DB2. There are some big differences among these products in their current releases. The biggest differences are relatively easy to detect and include the following:

Differences imposed due to operating system constraints
(OS/400 versus z/OS versus AIX)
Back-level compatibility issues
Workstation orientation differences such as GUI interfaces and drag-and-drop menus
Subsystem-centric implementation (z/OS) versus database-centric implementation (workstation)

Most of these differences are minor and easy to handle. Indeed, IBM has slowly but surely been making these disparate implementations of DB2 more and more alike with each new release and version. The interface (or API) by which most people access any of the DB2 Family is SQL and there is broad compatibility among the SQL implementations of the members of the DB2 Family (though not 100 percent, of course).

A misconception “out there” in DB2-land is that the LUW platform drives new features, but a review of the changes that have been introduced to DB2 over the past several versions and releases does not bear that out. Some features are introduced on the mainframe first; others on the distributed platforms first.

Of the basic differences mentioned earlier, the only one that might not be obvious is the focus of the DBMS implementation. DB2 for LUW is database-centric. This implies that each new database carries its own system catalog with it. Additionally, it is not possible to simply access tables across different databases; distributed access is required.

On z/OS, DB2 is subsystem-centric. A single system catalog spans databases. Each subsystem has a unique identification, and you can create multiple databases within it. Distributed requests are not required to access databases within the same subsystem (or, indeed, across multiple subsystems in a data-sharing environment).

Another concept that is different at the workstation level is that of a directory. The DB2 for z/OS Directory houses DBMS system-related information regarding DBD structure, skeleton plan and skeleton package tables, RBA log ranges, and utility control data. The information cannot be updated by the user but is managed and controlled by DB2.

At the workstation level, a directory is another matter altogether. For example, the directory structure used by DB2 for LUW controls the overall environment.

The System Database Directory identifies the databases that can be accessed from the workstation and contains an entry for each local and remote one. Each database entry contains the database name, alias, entry type, and location.
One Volume Database Directory is allocated per disk drive that contains a workstation database. Each entry identifies the location of a specific database on the drive.
The Workstation Directory is used to make a connection to a remote database server. It is used in conjunction with the Database Connection Services Directory to make a connection to a remote host server.
The Database Connection Services Directory is used by DB2 Connect to make a connection to a remote host server.

Not only is it possible for the user to update these directories, it is required. The workstation directories define the environment of DB2 for LUW. Without the proper information recorded in these directories, DB2 might not function in the desired manner. The information in these directories is somewhat analogous to DB2 for z/OS DSNZPARMs and the SYSDDF system catalog tables.

Database Structures

Not all the objects available to DB2 for z/OS users are supported at the workstation level. For example, hardware-specific DB2 objects such as table spaces and storage groups are not available for DB2 on other platforms, at least not in the same way that mainframers are used to dealing with them. Partitioning and segmenting as it is done on z/OS is not done on other platforms.

However, DB2 for LUW does provide a feature known as a segmented table. But this is not the same concept as a DB2 for z/OS segmented table space. DB2 for LUW segmented tables are used to span volumes, enabling DB2 to get around file size limitations.

The file structure used for databases differs from platform to platform. For example, DB2 for z/OS uses VSAM Linear Data Sets (LDS) or Entry Sequenced Data Sets (ESDS). A database deployed on DB2 for LUW uses two files for table data: one for normal data and a second to store long fields. These workstation files are flat files, not VSAM files.

Although tables are basically the same for all of the DB2 environments, not all of the DDL options are provided in all of the environments.

Optimizer Differences

One of the most significant benefits of relational databases is that they provide built-in optimization. The DB2 for z/OS optimizer is well-known to mainframe DB2 users, but how similar are the other DB2 optimizers?

DB2 for LUW uses the latest and greatest optimization technology from IBM -- the Starburst optimizer (which arose from IBM’s Almaden research lab). Starburst is a database optimization research project that has been covered quite extensively in the academic press.

As one example of the difference, consider that the DB2 for LUW optimizer has varying levels of optimization that can be selected by the user. This concept is not implemented in DB2 for z/OS.

Although some Starburst technology will find its way to DB2 for z/OS, the mainframe DB2 optimizer will not be completely replaced by Starburst technology. Doing so would not be wise because the DB2 for z/OS optimizer has been finely tuned for its environment over the course of almost three decades.

Another interesting tidbit is that DB2 for iSeries provides an access method for programmers in which they can bypass the relational engine. This is not encouraged, but it is available.

Other Differences

Other differences exist between the different implementations of DB2. Some of these are caused by the different release cycles IBM has created for the differing platforms. The bottom line is that you need to be aware that there are differences between the DB2s on different platforms. Whenever you use a specific implementation of DB2, you need to be aware of the features it supports that other DB2 platforms do not, as well as the features it does not support that other DB2 platforms do support.

Packaging and Naming Issues

The actual name of the DB2 edition can be tricky to master on non-mainframe platforms. On the mainframe you just say “I want DB2,” and that is what you get. Well, almost. You also have to decide whether you want IBM’s utilities or not, too.

But things are more difficult in the LUW world. The following packages are all available for DB2 on Linux, Unix, and Windows:

DB2 Workgroup Server Edition (WSE) is a multi-user, single host, DBMS at the departmental user. It should be deployed for smaller systems with a limited number of users.

DB2 Enterprise Server Edition (ESE) is the highest level of DB2 database version with intra-partition parallelism support (the database engine can process SQL statement segments in parallel), and inter-partition parallelism support (process a query in parallel across all of the nodes). ESE has Partitioning and Clustering options as additional add-on features. So, this is the enterprise DB2.

DB2 Advanced Enterprise Server Edition (AESE) sounds like a step up from ESE, and it is, kind of... but not really in terms of key DBMS technology. The advanced means that IBM integrates Optim and InfoSphere technologies into the product.

DB2 Express Edition is targeted at entry level users at a low price point. Small shops, partners, and new users can build applications on top of DB2 Express.

And DB2 Express-C is IBM’s “free” DBMS offering providing all the “core” capabilities of DB2 at no charge. So why use an open source DBMS when you can get a free version of DB2?
A handy comparison of the editions is available on IBM’s web site.

Summary

So you see, saying DB2 is no enough any more. Which DB2? They’re all great, but it can take some time to wrap your arms around all of this…

I'll Be Tweeting Live From IDUG

2011-04-29T16:15:00.003-05:00

For those of you who use Twitter, make sure you are following me next week (http://www.twitter.com/craigmullins) as I will be tweeting my experiences from the IDUG conference in Anaheim.

If you aren't planning to go, you can follow my Tweets to hear what is going on... and if you are attending the show, you can follow my Tweets to hear my perspective on things...

I arrive in Anaheim Tuesday afternoon, so I will miss the kickoff, but I'll be there the rest of the week.

100 Years of IBM

2011-04-26T17:14:00.002-05:00

If you have anything at all to do with computers or information technology, you have something to thank IBM for. Watch this video to find out what!

What About Surrogate Keys?

2011-04-04T11:30:00.004-05:00

As is so often the case with my blog, today's topic came about as the result of an e-mail question I received from a DBA I know. His question was this:

"A great debate rages here about the use of ‘synthetic’ keys. We read all sorts of articles on the wild wild web but none seem to address the database performance impacts of designs using synthetic keys. I wondered if you could point me to any information on this…"

If you've ever Googled the term "surrogate key" you know the hornet's nest of opinions that swirls around "out there" about the topic. For those who haven't heard the term, here is my attempt at a quick summary: a surrogate key is a generated unique value that is used as the primary key of a database table; database designers tend to consider surrogate keys when the natural key consists of many columns, is very long, or may need to change.

And here is the response I sent to my e-mail inquisitor:

I doubt that there is any “final word” on this topic. It has been raging on for years and years; so folks pro, others con. This Wikipedia article offers up a nice start: http://en.wikipedia.org/wiki/Surrogate_key

However, when I get to the performance area of this article I don’t think I agree. The article puts a lot of emphasis on there being fewer columns to join and therefore better performance.. If you’ve got an index on those multiple columns how much “worse” will the performance be, really? Sure, the SQL is more difficult to write, but will a join over 4 or 5 indexed columns perform that much worse than a join on one indexed column? I suppose as the number of columns required for the natural key increases the impact could be greater (e.g. 10 columns???)

I guess I can see the argument if you are swapping a variable length key with a surrogate having a fixed length key – that should improve things!

Furthermore consider this: the natural key columns are still going to be there, after all, they are naturally part of the data, right? So the surrogate (synthetic) key gets added to each row. This will likely reduce the number of rows per page (maybe not, but probably). And that, in turn, will negatively impact the performance of sequential access because more I/O will be required to read the “same” number of rows.

And what about the impact of adding data? If there are a significant number of new rows being added at the same time by different processes there will be locking issues as they all try to put the new data on the same page (unless, of course, your surrogate key is not a sequential number and is, instead, something like the microseconds portion of the current timestamp [that must be tested to avoid duplicates]).

The one thing that usually causes me to tend to favor natural keys is just that – they are natural. If the data is naturally occurring it becomes easier for end users to remember it and use it. If it is a randomly generated surrogate nobody will actually know the data. Yes, this can be masked to a great deal based on the manner in which you build your applications to access the data, but ad hoc access becomes quite difficult.

I guess the bottom line is that “it depends” on a lot of different things! No surprise there, I suppose.

Here are a few other resources with information (not so much on performance though) that you may or may not have reviewed already:

What do you think about natural keys versus surrogate keys? Surely some readers here have an opinion on this topic! If so, post them as comments...

DB2 Symposium 2011

2011-03-09T13:24:00.003-06:00

Today's blog post is about a great symposium dedicated to the topic of DB2. It is called, appropriately enough, the DB2 Symposium. DB2 Symposium is a three day training event with one day seminars presented by well-known DB2 consultants. I was fortunate enough to be asked to participate this year by the primary organizer of the event, Klaas Brant. For those of you who don't know him, Klaas is a well-respected DB2 consultant based in the Netherlands... and an all around great guy.

Why should I attend the DB2 Symposium you may ask? Don't IDUG and IOD provide everything I need in the way of events? Well, DB2 Symposium fills the gap between a conference and a multi-day training course. The DB2 Symposium is unique because you can participate for 1, 2, or 3 days, depending on your needs and budget.

Although it has not been to the USA the past few years, the DB2 Symposium is a regular, well-known event in Europe! And after a period of absence the DB2 Symposium is back in the USA.

The USA DB2 Symposium is happening soon, so you'll need to act fast if you want to participate. It occurs March 21-23, 2011 in the Dallas, Texas area. More precisely, at the Hilton Arlington (2401 East Lamar Boulevard, Arlington, Texas, USA 76006-7503). Each day the training sessions start at 9.00am and end at around 5.00pm.

But registration on site is not possible, you must pre-register online... so plan ahead!

My session is on March 21st and it is called DB2 Developer's Guide Comes Alive! This one day session, covers tips, techniques, and procedures you need to know in order to excel at administering and using DB2 on the mainframe.The material is based upon DB2 Developer's Guide, the best-selling DB2 for z/OS book on the market. Additionally, the course material will contain references to sections of the book for students to find additional material on each topic after the sessions. Topics to be covered will include:

A performance tuning roadmap for managing DB2 application, database and system performance. You will learn SQL coding and tuning techniques, guidance for database optimization and reorganization, coverage of buffer pool settings and parameters for performance.
Logical and physical database design recommendations for DB2, so you can build and maintain effective DB2 databases immediately. Includes discussion of standards, logical to physical translation, data types, usage of nulls, and more.
Information and guidance on BINDing and REBINDing, along with a discussion of the most important parameters.
Along the way we'll look at locking, access paths, statistics, indexing and more.
And even though the current edition of the book covers through DB2 V8, this course adds coverage of some of the newer features added to DB2 in versions 9 and 10 that can boost your productivity and performance.

So register for the DB2 Symposium today... and I'll see you in Dallas, pardner!

Not Your Standard Sorting Requirement

2011-02-21T13:46:00.002-06:00

Sometimes the requirements of a particular application dictate that data needs to be sorted using some irregular collating sequence. These odd needs sometimes cause developers to sit and scratch their heads for hours, searching for ways to make DB2 do something that seems to be "unnatural." But often you can create an answer just by understanding the problem and applying some creative SQL.

At this point, some of you might be asking "What the heck is he talking about?" Fair enough. Let’s take a look at an example to bring the issue into focus.

Assume that you have a table containing transactions, or some other type of interesting facts. The table has a CHAR(3) column containing an abbreviation for the name of the day on which the transaction happened; let’s call this column DAY_NAME. So, for example, the DAY_NAME column would contain MON for Monday data, and so on.

Now, let’s further assume that we want to write queries against this table that orders the results by DAY_NAME. We’d want Sunday first, followed by Monday, Tuesday, Wednesday, and so on. How can this be done?

Well, the first step is usually to write the first query that comes to mind, or something like this:

SELECT DAY_NAME, COL1, COL2 . . .
FROM TXN_TABLE
ORDER BY DAY_NAME;

Of course, the results will be sorted improperly. ORDER BY will sort the results alphabetically; in other words: FRI MON SAT SUN THU TUE WED

This is what I mean by an irregular sorting requirement. Here we have an example that occurs commonly enough, but without an obvious immediate solution. Furthermore, many businesses and applications have similar requirements for which the business needs dictate a different sort order than strictly alphabetical or numeric. So what is the solution here?

Of course, one solution would be to design the table with an additional numeric or alphabetic column that would sort properly. By this, I mean that we could add a DAY_NUM column that would be 1 for Sunday, 2 for Monday, and so on. But this requires a database design change, and it becomes possible for the DAY_NUM and DAY_NAME to get out of sync if you are not careful.

There is another solution that is both elegant and does not require any change to the database. To implement this solution, all you need is an understanding of SQL and SQL functions -- in this case, the LOCATE function. Consider this SQL:

SELECT DAY_NAME, COL1, COL2 . . .
FROM TXN_TABLE
ORDER BY LOCATE('SUNMONTUEWEDTHUFRISAT',DAY_NAME);

The trick here is to understand how the LOCATE function works. The LOCATE function returns the starting position of the first occurrence of one string within another string.

So, in our example, LOCATE finds the position of the DAY_NAME value within the string 'SUNMONTUEWEDTHUFRISAT', and returns the integer value of that position. So, if DAY_NAME is WED, the LOCATE function returns 10. (Note: Some other database systems have a similar function called INSTR.) Sunday would return 1, Monday 4, Tuesday 7, Wednesday 10, Thursday 13, Friday 16, and Saturday 19. This means that our results would be in the order we require.

Of course, you can go one step further if you’d like. Some queries may need to actually return the day of week. You can use the same technique with a twist to return the day of week value, given only the day’s name. To turn this into the appropriate day of the week number (that is, a value of 1 through 7), we divide by three, use the INT function on the result to return only the integer portion of the result, and then add one:

INT(LOCATE('SUNMONTUEWEDTHUFRISAT',DAY_NAME)/3) + 1;

Let’s use our previous example of Wednesday again. The LOCATE function returns the value 10. So, INT(10/3) = 3 and add 1 to get 4. And sure enough, Wednesday is the fourth day of the week.

Summary

With a sound understanding of the features of DB2 SQL and a little imagination many irregular requirements are achievable using nothing more than SQL!

View Naming Conventions

2011-02-10T13:42:00.003-06:00

Naming conventions sometimes instigate conflict within the world of DB2, especially when it comes to views. But, really, it should be very easy. Just always remember, that a view is a logical table. It consists of rows and columns, exactly the same as a DB2 table. A DB2 view can (syntactically) be used in SQL SELECT, UPDATE, DELETE, and INSERT statements in the same way that a DB2 table can. Furthermore, a view can be used functionally the same as a DB2 table (with certain limitations on updating as outlined in my article).

Therefore, shouldn't it stand to reason that views should be held to the same naming conventions as are used for tables? (As an aside, the same can be said for DB2 aliases and synonyms).

End users querying views don't need to know whether they are accessing a view or a table. That is the whole purpose of views. Why then, should we enforce an arbitrary naming standard, such as putting a V in the first or last position of a view name, on views?

DBAs and technical analysts, those individuals who have a need to differentiate between tables and views, can utilize the DB2 Catalog to determine which objects are views and which objects are tables.

Most users don't care whether they are using a table, view, synonym, or alias. They simply want to access the data. And, in a relational database, tables, views, synonyms, and aliases all logically appear to be identical to the end user: collections of rows and columns.

There are certain operations that cannot be performed on certain types of views, but the end users who need to know this will generally be sophisticated users. For example, very few shops allow end users to update any table they want using a report writer or query tool (e.g. QMF, SPUFI, etc.). Updates, deletions, and insertions (the operations which are not available to some views) are generally coded into application programs and executed in batch or via online transactions. Most end users need to query tables dynamically.

Now you tell me, which name will your typical end user remember more readily when he needs to access his marketing contacts: MKT_CONTACT or VMKTCT01?

TIMESTAMP versus DATE/TIME

2011-02-03T10:35:00.002-06:00

Consider a database design decision point where you need to store both date and time information on a single row in DB2. Is it better to use a single TIMESTAMP column or two columns, one DATE and the other TIME?

Well, of course, the answer is "it depends!" The correct solution will depend on several factors specific to your situation. Consider the following points before making your decision:

With DATE and TIME you must use two columns. TIMESTAMP uses one column, thereby simplifying data access and modification.

The combination of DATE and TIME columns requires 7 bytes of storage, while a TIMESTAMP column always requires 10 bytes of storage. Using the combination of DATE and TIME columns will save space.

TIMESTAMP provides greater time accuracy, down to the microsecond level. TIME provides accuracy only to the second level. If precision is important, use TIMESTAMP. Use TIME if you want to ensure that the actual time is NOT stored down to the microsecond level.

A TIMESTAMP can always be broken down into a DATE and a TIME component, after which you can treat the data just like DATE and TIME data.

Date and time arithmetic probably will be easier to implement using TIMESTAMP data instead of a combination of DATE and TIME. Subtracting one TIMESTAMP from another results in a TIMESTAMP duration. To calculate a duration using DATE and TIME columns, two subtraction operations must occur: one for the DATE column and one for the TIME column.

Formatting may be easier with DATE and TIME data. DB2 provides for the formatting of DATE and TIME columns via local DATE and TIME exits, the CHAR function, and the DATE and TIME precompiler options. If the date and time information is to be extracted and displayed on a report or by an online application, the availability of these DB2-provided facilities for DATE and TIME columns should be considered when making your decision.

Prior to DB2 V9, not much help was available for the formatting of TIMESTAMP columns.But DB2 9 for z/OS adds the TIMESTAMP_FORMAT function, which offers three different formats for displaying timestamp data.

Upon reviewing all of these details, and factoring in your usage requirements, you can then make an informed decision about whether to use one TIMESTAMP column, or two columns, one DATE and one TIME.

More Indicator Variables Available in DB2 10 for z/OS

2010-12-13T16:22:00.004-06:00

As you all should know by now, version 10 of DB2 doe z/OS is generally available and has been for a month or so now. As such, it is probably time that I start to blog about some of the features of the new release. But instead of starting with one of the bigger features, that you already may have heard about, I decided to start with a feature that has flown somewhat under the radar: extended indicator variables.

Those of you who write programs that deal with possibly null results should know what an indicator variable is. Basically, DB2 represents null in a special variable known as an indicator. An indicator is defined to DB2 for each column that can accept nulls. The indicator variable is transparent to the end user, but must be provided for when programming in a host language (such as COBOL or PL/I). If the indicator variable is less than zero, then the column to which it applies has returned NULL.

DB2 10 for z/OS enhances and extends the concept of an indicator variable so they can be used outside the scope of nullability. Consider the scenario where a program is being written to modify data. There are multiple combinations of columns that may need to be modified based on conditional programmatic processing. Maybe the program is for editing customer data. The customer has multiple columns that could be modified: name, address, telephone number, credit rating, etc. So the programmer codes up an online screen (e.g. CICS) with all of the data that can then be changed by the end user.

But what happens when the end user cracks the enter key to submit the changes? What actually changed and what stayed the same? Does the program check every value on the screen (perhaps hundreds) and build every UPDATE statement iteration for data that might have been changed? Unlikely, since that would require x! statements (where x is the total number of columns). For non-mathematicians a discussion of factorial can be found here (http://en.wikipedia.org/wiki/Factorial).

Yes, there are CICS options to help the programmer determine which values have changed (or simply save and compare). But until now, dealing with all the potential SQL statements could be problematic. Well, DB2 10 indicator variables come to the rescue. As of DB2 10 NFM you can use indicator variables to inform DB2 whether the value for an associated host variable has been supplied or not… and to specify how DB2 should handle the missing value.

This is an extended indicator variable. And it can be applied to host variables and parameter markers. Whether you will use extended indicator variables can be enabled at the package level, by using the EXTENDEDINDICATOR option of the BIND PACKAGE command. You can also enable extended indicator variables on a statement level for dynamic SQL by using the WITH EXTENDED INDICATORS attribute on the PREPARE statement.

How would this work? Well, extended indicator variables can be specified only for host variables that appear in the following situations:

The set assignment list of an UPDATE operation in UPDATE or MERGE statements
The values list of an INSERT operation in INSERT or MERGE statements
The select list of an INSERT statement in the FROM clause

OK, then, how would we use an extended indicator variable? By setting its value to tell DB2 how to proceeed. The following values are available:

0 (zero) or a positive integer: This indicates the first host identifier provides the value of this host variable reference and it is not null.
-1, -2, -3, -4, or -6: This indicates a null.
-5: If extended indicator variables are not enabled, this indicates a null; otherwise, a value of -5 indicates that the DEFAULT value is to be used for the target column for this host variable.
-7: If extended indicator variables are not enabled, this indicates a null; otherwise, a value of -7 indicates that the UNASSIGNED value is to be used for the target column for this host variable (in other words, treat it as if it were not specified in this statement).

For an INSERT, -5 and -7 settings for an extended indicator variable will end up with the same result. This is so because the INSERT statement works by inserting a default value for any column that is missing. On the other hand, for UPDATE and the UPDATE portion of a MERGE, setting the extended indicator variable to -5 leads to the column being update to the default value, but -7 leads to the update of the column not being applied.

With extended indicator variables then, there is no need for the application to re-send a column’s current value, or to know a column’s DEFAULT value. Which should make things easier for developers.

IBM Information On Demand 2010 - The Final Keynote

2010-10-28T13:58:00.003-05:00

The keynote session for the third day of the IOD conference features the authors of Freakonomics, Steven Levitt and Stephen Dubner. I've read their first book and it is an excellent read... I highly recommend it.

But, of course, there are the IBMers that must speak first. The session kicked off with a video on intelligence being infused into the devices we use in our everyday life. And this “smarter planer” improves our life in countless ways. Smart grids, smart healthcare, smart supply chains, etc. All of which make us more productive and effective not just in business, but in all aspects of our lives. IBM calls this the “Decade of Smart.”

The first part of the session then featured Mike Rhodin, Sr. VP IBM Software Solutions Group. He indicated that we are at the beginning of what is going to constitute massive changes to the way we look at and solve problems. He explained by talking about solutions for commerce that have changed over the last decade or so. The experience is vastly different today across the board. This is so in terms of how buying decisions are made, how buying is done, and how the transaction is completed.

But how can we know what the customer of the future wants? The idea now is to look at how you can leverage things like social media to perform “sentiment analysis” to engage in conversation. By making it a dialogue instead of a one way street we can start this transformation.

He talked about a Southwest flier who was dissatisfied by a delay and tweeted about it. A few days later a Southwest representative contacted him and offered him “something” to assuage his dissatisfaction. Although that is good, he said it would have been better if the Southwest rep was waiting for him at the gate at his destination. OK, but in my opinion, it would have been even better if the Southwest rep could have made contact before the plane took off (either on the plane or at the gate if they had not yet boarded).

Next up was Brenda Dietrich, VP Business Analytics and Mathematical Sciences and IBM Research. It was good to hear from someone in the research group because they don't get "out" to speak much. Dietrich espoused the global reach of IBM’s research group with 9 major offices across the world and many more co-laboratories, which are smaller labs with the goal of working with more local talent.

If it has to do with the future of technology, IBM Research is probably involved in it. Examples include nanotechnology, supercomputing and workload-optimized systems, cloud computing, and analytics.

The future of the “smarter planet” is at optimizing individual systems, like an electrical grid. And then developing systems of systems where those individual systems interact with other systems. For example, where the electrical grid interacts with the traffic grid. Additionally, today things are being digitized and we are analyzing and reacting to this information. We are moving toward using this information to model and predict outcomes.

She also discussed an analytics project called Smart Enterprise Executive Decision Support (SEEDS). It is a super-dashboard that IBM Research is working on. It incorporated a common data model with multiple IBM technologies to perform analytics that delivers better answers. Sounds exciting to me!

And IBM Research has even created a computer that plays Jeopardy! The video she played that demonstrated that was very impressive. This is especially so because it understood the questions in natural language, which is very difficult for computers to accomplish.

Then the Freakonomics dudes came out. And they were very entertaining. They took turns telling stories. Levitt is the economist and Dubner is the writer, but both were eloquent speakers who mixed information with humor extremely well. Dubner started out with my favorite story from their first book: how the legalization of abortion led to a decrease in crime. If that surprises you, you really need to read the book(s).

Levitt followed and told the story of how adding social security number to the tax forms caused 7 million children to vanish from the face of the Earth. It turns out that Americans are very immoral and had created children for the tax deduction. Levitt had troubles believing this until he talked to his father and was told that he himself had lost two brothers!

I would try to explain how they then moved from trying to teach monkeys to use money to a discussion of the proper pricing for prostitution services... but it would be far better if you read about it in their book(s). I know after seeing them that I am going to buy their new book, SuperFreakonomics.

All in all, though, it was a thoroughly entertaining and education final keynote session at the IOD show.

A Video from IOD, DB2 -- Monday 10/25

2010-10-26T15:58:00.001-05:00

This video was shot by Rebecca Bond at the IBM Information On Demand Conference 2010 in Las Vegas. She was interviewing DB2 folks on what they do and the benefit they get from attending IOD. I am the third interview... but don't just skip to me! Listen to Melanie and Fred, too!

News From The IOD Conference

2010-10-26T13:32:00.000-05:00

As usual, IBM has put out a number of press releases in conjunction with the Information On Demand conference, and I will use today’s blog to summarize some of the highlights of these releases.

First of all, IBM is rightly proud of the fact that more than 700 SAP clients have turned to IBM DB2 database software to manage heavy database workloads for improved performance… and, according to IBM, at a lower cost. By that they mean at a lower cost than Oracle. Even though the press release does not state that these SAP sites chose DB2 over Oracle, the IBM executive I spoke with yesterday made it clear that that was indeed the case.

This stampede of SAP customers over to DB2 should not be a surprise because DB2 is SAP’s preferred database software. This might be surprising given that SAP recently acquired Sybase, but IBM notes that seven Sybase runs SAP on DB2.

The press release goes on to call out several customers who are using DB2 with SAP and their reasons for doing so. For example, Reliance Life chose DB2 for the better transaction performance and faster access to data it delivered. Banco de Brasil, on the other hand, was looking to reduce power consumption and storage by consolidating its database management systems.

IBM also announced new software that helps clients automate content-centric processes and manage unstructured content. The highlight of this announcement is IBM Case Manager, software that integrates content and process management with advanced analytics, business rules, collaboration and social software.

IBM also enhanced its content analytics software. NTT DOCOMO of Japan is impressed with IBM’s offering. “With Content Analytics, we have an integrated view of all information that’s relevant to out business in one place regardless of where it’s stored,” said Makoto Ichise, Manager of Information Systems Department Group at NTT DOCOMO.

IBM also enhanced its Information Governance solutions and announced further usage of it InfoSphere Streams product for analyzing medical data to improve healthcare.

So IBM software keeps improving and helping us to better manage our data in a constantly changing world…

DB2 10 Technical Overview at IOD Conference

2010-10-25T20:04:00.002-05:00

Today I attended the IOD conference and had the opportunity to listen to Jeff Josten present an technical overview of DB2 10 for z/OS. Even though information and specifications have been trickling out on DB2 10 for z/OS over the course of the past year or so, this is the first DB2 10 presentation I have attended subsequent to the GA announcement IBM made last week (announced October 19th, General Availability on October 22nd, 2010). So I’m fairly certain that everything Jeff will talk about will be officially part of DB2 10, instead of just the rumors, hints and allegations proffered prior to the GA announcement. I don’t think anything to be covered will surprise me, but we’ll see.

Jeff started off saying that the technical strategy for DB2 10 was four-pronged:

Continuous availability

Performance and scalability

Ease of management

Advance application features

One of the key value propositions is the deep synergy with the System z hardware. Because the code does not need to be ported all over the place, it can take advantage of the hardware capabilities that bring improved performance and efficiency.

DB2 10 is being promoted as delivering 5 to 10 percent CPU batch and transaction performance improvement out-of-the-box; 20 percent for new workloads. And then an additional 10 percent when you start using new features. This seems to be the high-level talking point for DB2 10 – but that is okay, it is a very good one!

DB2 V8 will go out of service April 2012. But with the ability to go staright from V8 to 10, I’m betting a lot of V8 shops will skip 9 altogether and go right to 10. But it looks like you have just under 2 years to get off of V8, though.

DB2 10 takes advantage of many zEnterprise (the new mainframe announced a couple of months ago) features to deliver scalability. Examples include improved compression, cache optimization, blades for running the Smart Analytics Optimizer, etc.

Jeff mentioned that we’ll be able to support 10 times more users by avoiding memory constraints in DB2 10. That is a big scalability improvement!

DB2 10 went through the largest beta ever: 23 customers and more than 80 vendors. The focus was on testing production level workloads to ensure that the release is stable. That is different than past betas where the focus was on testing new features. And 22 of the beta customers are planning to go into production with DB2 10 next year. Impressive!

Before diving into the technical details, Jeff mentioned that not much has changed versus what IBM has been talking about over the past months. A couple late add features include hash performance, BIND performance, REBIND not required for packages flagged as private protocol (but they will fail if they actually use private protocol), and a new ZPARM for default SEGSIZE for DDL compatibility.

Post GA delivery items include APREUSE and APCOMPARE (for reusing access paths instead of using “hints”) because beta testing exposed some quality items, the ability to delete a data sharing member, inline LOBs for SPT01, online REORG concurrency for materializing deferred ALTERs, and some temporal enhancements (e.g. TIMESTAMP WITH TIMEZONE support).

High performance DBATs (DDF threads) were forced to be RELEASE COMMIT. This gave good storage usage but at the expense of CPU for releasing resources at COMMIT and putting the thread back on the queue. For DB2 10, customers can use RELEASE DEALLOCATE which will keep the thread assigned to the distributed requestor so the next time he comes in he can reuse the thread. This is a nice feature for shops with heavy distributed usage.

Another nice thing right out of the box is parallel index updating at INSERT. This used to be synchronous, each index modified one after the other. Now, the indexes can be modified in parallel, which should be a nice performance improvement for many shops!

There is also a new buffer pool option for a “fully in memory” object for reading the data and pinning it in buffers. Don’t know about you, but I’ve been wanting that for years.

Things that require a REBIND will include most access path enhancements, query parallelism improvements, IN list performance improvements, and Stage 2 predicates being pushed to Stage 1.

Things that require NFM include DB2 Catalog concurrency, compress on insert capability, most utility enhancements, LOB streaming between DDF and the rest of DB2, INSERT improvements for universal table spaces, faster FETCH and INSERT with lower virtual storage consumption, SQL Procedure Language performance improvements, efficient caching of dynamic SQL with literals, as well as a few other “things.”

And then there are things that require NFM and DBA work, such as hashing, index include columns, inline LOBs, DEFINNE NO for LOB and XML columns, MEMBER CLUSTER for universal table spaces, and online REORG for all DB2 Catalog and Directory table spaces.

So it looks like our favorite DBMS is continuing to grow and expand offering high performance functionality and features that are unparalleled in the industry. Indeed, there is a lot of great and exciting new “stuff” on the way in DB2 10.

Null Follow-up: IS [NOT] DISTINCT FROM

2010-10-07T16:15:00.000-05:00

After publishing the last blog post here on the topic of pesky problems that crop up when dealing with nulls, I received a comment lamenting that I did not address the IS [NOT] DISTINCT FROM clause. So today’s blog post will redress that failure.

First of all, IS [NOT] DISTINCT FROM is a relatively new predicate operator, introduced to DB2 for z/OS in Version 8. It is quite convenient to use in situations where you are looking to compare to columns that could contain NULL.

Before diving into the operator, let’s first discuss the problem it helps to solve. Two columns are not equal if both are NULL, that is because NULL is unknown and a NULL never equals anything else, not even another NULL. But sometimes you might want to treat NULLs as equivalent. In order to do that, you would have to code something like this in your WHERE clause:


WHERE COL1 = COL2
OR (COL1 IS NULL AND COL2 IS NULL)

This coding would cause DB2 to return all the rows where COL1 and COL2 are the same value, as well as all the rows where both COL1 and COL2 are NULL, effectively treating NULLs as equivalent. But this coding although relatively simply, can be unwieldy and perhaps, at least not at first blush, unintuitive.

Here comes the IS NOT DISTINCT FROM clause to the rescue. As of DB2 V8, the following clause is logically equivalent to the one above, but perhaps simpler to code and understand:


WHERE COL1 IS NOT DISTINCT FROM COL2

The same goes for checking a column against a host variable. You might try to code a clause specifying WHERE COL = :HV :hvind (host variable and indicator variable). But such a search condition would never be true when the value in that host variable is null, even if the host variable contains a null indicator. This is because one null does not equal another null - ever. Instead we’d have to code additional predicates: one to handle the non-null values and two others to ensure both COL1 and the :HV are both null. With the introduction of the IS NOT DISTINCT FROM predicate, the search condition could be simplified to just:


WHERE COL1 IS NOT DISTINCT FROM :HV :hvind

Null Troubles

2010-10-06T11:46:00.003-05:00

A null represents missing or unknown information at the column level. A null is not the same as 0 (zero) or blank. Null means no entry has been made for the column and it implies that the value is either unknown or not applicable.

DB2 supports null, and as such you can use null to can distinguish between a deliberate entry of 0 (for numerical columns) or a blank (for character columns) and an unknown or inapplicable entry (NULL for both numerical and character columns).

Nulls sometimes are inappropriately referred to as “null values.” Using the term value to describe a null is inaccurate because a null implies the lack of a value. Therefore, simply use the term null or nulls (without appending the term “value” or “values” to it).

DB2 represents null in a special “hidden” column known as an indicator. An indicator is defined to DB2 for each column that can accept nulls. The indicator variable is transparent to the end user, but must be provided for when programming in a host language (such as COBOL or PL/I).

Every column defined to a DB2 table must be designated as either allowing or disallowing nulls. A column is defined as nullable – meaning it can be set to NULL – in the table creation DDL. Null is the default if nothing is specified after the column name. To prohibit the column from being set to NULL you must explicitly specify NOT NULL after the column name. In the following sample table, COL1 and COL3 can be set to null, but not COL2, COL4, or COL5:


CREATE TABLE SAMPLE1
  (COL1   INTEGER,
   COL2   CHAR(10) NOT NULL,
   COL3   CHAR(5),
   COL4   DATE     NOT NULL WITH DEFAULT,
   COL5   TIME     NOT NULL);

What Are The Issues with Null?

The way in which nulls are processed usually is not intuitive to folks used to yes/no, on/off, thinking. With null data, answers are not true/false, but true/false/unknown. Remember, a null is not known. So when a null participates in a mathematical expression, the result is always null. That means that the answer to each of the following is NULL:

5 + NULL
NULL / 501324
102 – NULL
51235 * NULL
NULL**3
NULL + NULL
NULL/0

Yes, even that last one is null, even though the mathematician in us wants to say “error” because of division by zero. So nulls can be tricky to deal with.

Another interesting aspect of nulls is that the AVG, COUNT DISTINCT, SUM, MAX, and MIN functions omit column occurrences set to null. The COUNT(*) function, however, does not omit columns set to null because it operates on rows. Thus, AVG is not equal to SUM/COUNT(*) when the average is being computed for a column that can contain nulls. To clarify with an example, if the COMM column is nullable, the result of the following query:


    SELECT  AVG(COMM)
    FROM    DSN8810.EMP;

is not the same as for this query:


    SELECT  SUM(COMM)/COUNT(*)
    FROM    DSN8810.EMP;

But perhaps the more troubling aspect of this treatment of nulls is “What exactly do the results mean?” Shouldn’t a function that processes any NULLs at all return an answer of NULL, or unknown? Does skipping all columns that are NULL return a useful result? I think what is really needed is an option for these functions when they operate on nullable columns. Perhaps a switch that would allow three different modes of operation:

Return a NULL if any columns were null, which would be the default
Operate as it currently does, ignoring NULLs
Treat all NULLs as zeroes

At least that way users would have an option as to how NULLs are treated by functions. But this is not the case, so to avoid confusion, try to avoid allowing nulls in columns that must be processed using these functions whenever possible.

Here are some additional considerations regarding the rules of operation for nulls:

When a nullable column participates in an ORDER BY or GROUP BY clause, the returned nulls are grouped at the high end of the sort order.
Nulls are considered to be equal when duplicates are eliminated by SELECT DISTINCT or COUNT (DISTINCT column).
A unique index considers nulls to be equivalent and disallows duplicate entries because of the existence of nulls, unless the WHERE NOT NULL clause is specified in the index.
For comparison in a SELECT statement, two null columns are not considered equal. When a nullable column participates in a predicate in the WHERE or HAVING clause, the nulls that are encountered cause the comparison to evaluate to UNKNOWN.
When a nullable column participates in a calculation, the result is null.
Columns that participate in a primary key cannot be null.
To test for the existence of nulls, use the special predicate IS NULL in the WHERE clause of the SELECT statement. You cannot simply state WHERE column = NULL. You must state WHERE column IS NULL.
It is invalid to test if a column is <> NULL, or >= NULL. These are all meaningless because null is the absence of a value.

Examine these rules closely. ORDER BY, GROUP BY, DISTINCT, and unique indexes consider nulls to be equal and handle them accordingly. The SELECT statement, however, deems that the comparison of null columns is not equivalence, but unknown. This inconsistent handling of nulls is an anomaly that you must remember when using nulls.

Here are a couple of other issues to consider when nulls are involved.

Did you know it is possible to write SQL that returns a NULL even if you have no nullable columns in your database? Assume that there are no nullable columns in the EMP table (including SALARY) and then consider the following SQL:


SELECT SUM(SALARY)
FROM   EMP
WHERE  DEPTNO > 999;

The result of this query will be NULL if no DEPTNO exists that is greater than 999. So it is not feasible to try to design your way out of having to understand nulls!

Another troubling issue with NULLs is that some developers have incorrect expectations when using the NOT IN predicate with NULLs. Consider the following SQL:


SELECT C.color
FROM   Colors AS C 
WHERE  C.color NOT IN (SELECT P.color 
                       FROM   Products AS P);

If one of the products has its color set to NULL, then the result of the SELECT is the empty set, even if there are colors to which no other product is set.

Summary

Nulls are clearly one of the most misunderstood features of DB2 – indeed, of most SQL database systems. Although nulls can be confusing, you cannot bury your head in the sand and ignore nulls if you choose to use DB2 as your DBMS. Understanding what nulls are, and how best to use them, can help you to create usable DB2 databases and design useful and correct queries in your DB2 applications.

A Recommended New DB2 Book

2010-09-24T15:56:00.001-05:00

Judy Nall has performed a much-needed service for the DB2 for z/OS community by writing her new book, DB2 9 System Administration for z/OS: Certification Study Guide. There are many DB2 for z/OS books (heck, I wrote one myself) that cover programming, performance, and database administration details. But never before has there been one that focused on system administration and system programming.

Of course, the book is targeted at those looking to become an IBM Certified System Administrator for DB2 for z/OS. I have never taken the exams required for that certification, but the material in this book will go a long way toward making you a better system programmer for a mainframe DB2 environment.

Whereas some of the material can be found in greater detail in other books on the market, we must keep in mind that target market for the book. And the coverage of DB2 fundamentals and performance is well-written and hits the mark for systems folks. And the chapters on installation and migration, system backup and recovery, and systems operation and troubleshooting offer great systems-level knowledge not found in other DB2 for z/OS books.

So while DB2 9 System Administration for z/OS: Certification Study Guide is not for everyone, the people that it is for (systems programmers and systems DBAs) should enjoy it and benefit from the nice job Judy has done organizing and explaining the details of system administration for DB2 for z/OS.

Free DB2 Education Webinar Series

2010-08-26T10:15:00.006-05:00

Want to learn more about DB2 for z/OS but there is no money in the education budget? Can you spare an hour a week over the course of a month? Well then, you are in luck because SoftwareOnZ is sponsoring a series of DB2 webinars presented by yours truly, Craig S. Mullins…

Each webinar will be focused on a specific DB2 topic so you can pick and choose the ones that are most interesting to you – or attend them all and receive a certificate signed by me indicating that you have completed The DB2 Education Webinar Series.

The schedule and topics for these sessions follows:

September 28, 2010 – DB2 Access Paths: Surviving and Thriving

Binding your DB2 programs creates access paths that dictate how your applications will access DB2 data. But it can be tricky to understand exactly what is going on. There are many options and it can be difficult to select the proper ones… and to control when changes need to be made.

This presentation will clarify the BIND process, enabling you to manage DB2 application performance by controlling your DB2 access paths. And it will introduce a new, GUI-based product for managing when your programs need to be rebound.

October 5, 2010 – Optimizing DB2 Database Administration

DB2 DBAs are tasked with working in a complex technological environment, and as such, the DBA has to know many things about many things. This makes for busy days. How often have you asked yourself, “Where does the time go?”

Well, the more operational duties that can be automated and streamlined, the more effective a DBA can be. This presentation will address issues that every DB2 Database Administrator and/or DB2 Systems Programmer faces on a daily basis. And it will introduce a new tool, DB-Genie, that will reduce the amount of time, effort, and human error involved in maintaining DB2 databases.

October 12, 2010 – DB2 Storage: Don’t Ignore the Details!

For many DB2 professionals, storage management can be an afterthought. What with designing, building, and maintaining databases, assuring recoverability, monitoring performance, and so on, keeping track of where and how your databases are stored is not top of mind. But a storage problem can bring your databases and applications to a grinding halt, so it is not wise to ignore your storage needs.

This presentation will discuss the important storage-related details regarding DB2 for z/OS, including some of the newer storage options at your disposal. And we will also introduce a new web-based tool for monitoring all of your mainframe DB2 storage.

October 19, 2010 – The DB2 Application Developer’s Aid de Camp

Building DB2 application programs is a thankless job. And it can be difficult to ensure that you have a effective and efficient development environment for coding DB2 applications. Can you easily identify which tables are related to which… and what indexes are available so you code queries the right way the first time? Do you have the right data to test your programs? Can you make quick and dirty changes to just a few tables or rows without having to write yet another program?

This presentation will discuss the issues and difficulties that developers encounter on a daily basis as they build DB2 applications… and it will present a useful programmer-focused toolset for overcoming these difficulties.

Summary

Certainly there will be something of interest for every DB2 professional in at least one, if not all, of these complimentary web-based seminars.

So what’s stopping you? Sign up today!

DB2 Best Practices

2010-08-05T13:44:00.003-05:00

With today's blog entry I'm hoping to encourage some open-ended dialogue on best practices for DB2 database administration. Give the following questions some thought and if you've got something to share, post a comment!

What are the things that you do, or want to do, on a daily basis to manage your database infrastructure?

What things have you found to be most helpful to automate in administering your databases? Yes, I know that all the DBMS vendors are saying that they've created the "on demand" "lights-out" "24/7" database environment, but we all know that ain't so! So what have you done to automate (either using DBMS features, tools, or homegrown scripts) to keep an eye on things?

How have you ensured the recovery of your databases in the case of problems? Application problems? Against improper data entry or bad transactions? Disaster situations? And have you tested your disaster recovery plans? If so, how? And were they successful?

What type of auditing is done on your databases to track who has done what to what data? Do you audit all changes? To all applications, or just certain ones? Do you audit access, as well as modification? If so how?

How do you manage change? Do you use a change management tool or do it all by hand? Are database schema changes integrated with application changes? If so, how? If not, how do you coordinate things to keep the application synchronized with the databases?

What about DB2 storage management? Do you actively monitor disk usage of your DB2 table space and index spaces? Do you have alerts set so that you are notified if any object is nearing its maximum size? How about your VSAM data sets? Do you monitor extents and periodically consolidate? How do you do it... ALTER/REORG? Defrag utilities? Proactive defrag?

Is your performance management set up with triggers and farmed out to DBAs by exception or is it all reactive, with tuning tasks being done based on who complains the loudest?

Do you EXPLAIN every SQL statement before it goes into production? Does someone review the acess plans or are they just there to be reviewed in case of production performance problems? Do you rebind your programs periodically (for static SQL programs) as your data volume and statistics change, or do you just leave things alone until (or unless) someone complains?

When do you reorganize your data structures? On a pre-scheduled regular basis or based on database statistics? Or a combination of both? And how do you determine which are done using which method? What about your other DB2 utilities? Have you automated their scheduling or do you still manually build JCL?

How do you handle PTFs? Do you know which have been applied and which have not? And what impact that may be having on your database environment and applications? Do you have a standard for how often PTFs are applied?

How is security managed? Do the DBAs do all of the GRANTs and REVOKEs or is that job shared by security administrators? Are database logons coordinated across different DBMSs? Or could I have an operating system userid that is different from my SQL Server logon that is different than my Oracle logon -- with no capability of identifying that the user is the same user across the platforms?

How has regulatory compliance (e.g. PCI DSS, SOX, etc.) impacted your database administration activities? Have you had to purchase additional software to ensure compliance? How is compliance policed at your organization?

Just curious... Hope I get some responses!

Happy SYSADMIN Day

2010-07-30T09:31:00.002-05:00

To all of the system administrators out there (and I include DBAs and network admins in that group), HAPPY SYSADMIN DAY.

For those who are unaware if this very important holiday, every year on the last Friday of July responsible people everywhere celebrate Sysadmin Day. The idea is to show some appreciation for the folks who keep your systems up and running every day of the week. There is even a page with some gift ideas if you are so inclined to get something for your favorite sysadmin. (Personally, I prefer cash... .)

At the very least you can wish him/her a Happy Sysadmin Day today... and hoist a beer or two in his/her honor at the pub this evening...

Classics of Computer Literature

2010-07-13T10:25:00.005-05:00

Although the main focus of this blog is DB2 and mainframe software, I thought it would be worthwhile to take some time to recommend a few classic books for computer professionals. I am an avid reader of all kinds of books, not only on technology but on a wide variety of topics. Periodically I will use my blog to extol the virtues of some of my favorite books.

I'm starting with computer books as everyone reading this is probably in the field of IT. (...except maybe my Mom, hi Mom!) These books are not DBMS- or data-focused: I will recommend data and database books later, in some future blog posting.

So, here goes, my coverage of a nice starter set of 4 computer books that everyone should read...

&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;	Every computer professional should own a copy of Frederick P. Brooks Jr.’s seminal work, The Mythical Man-Month (Addison-Wesley Pub Co; ISBN: 0201835959). Brooks is best known as the father of the IBM System/360, the quintessential mainframe computer. He managed the projects that created the S/360 and its operating system. This book contains a wealth of knowledge about software project management including the now common-sense notion that adding manpower to a late software project just makes it later. The 20th anniversary edition of The Mythical Man-Month, published in 1995, contains a reprint of Brooks’ famous article “No Silver Bullet” as well as Brooks’ reflections on the twenty years since the book’s publication. If creating software is your discipline, you absolutely need to read and understand the tenets in this book.
&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;	Another essential book for technologists is Peopleware (Dorset House; ISBN: 0932633439) by Tom DeMarco and Timothy Lister. This book concentrates on the human aspect of project management and teams. If you believe that success is driven by technology more so than people, this book will change your misconceptions. Even though this book was written in the late 1980’s, it is still very pertinent to today’s software development projects. DeMarco is the author of several other revolutionary texts such as Structured Analysis and Design (Yourdon Press; ISBN: 0138543801). This book almost single-handedly introduced the concept of structured design into the computer programming lexicon. Today, structured analysis and design is almost completely taken for granted as the best way to approach the development of application programs.
&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;	If you are a systems analyst, application programmer, or software engineer then you will surely want Donald Knuth’s three volume series The Art of Computer Programming (Addison-Wesley Pub Co; ISBN: 0201485419). This multi-volume reference is certainly the definitive work on programming techniques. Knuth covers the algorithmic gamut in this three volume set, with the first volume devoted to fundamental algorithms (like trees and linked lists), a second volume devoted to semi-numerical algorithms (e.g. dealing with polynomials and primes), and a final volume dealing with sorting and searching. Even though a comprehensive reading and understanding of this entire set can be foreboding, all good programmers should have these techniques at their disposal. OK, I know, this is sort of cheating because it is a 3 book set, but so what... my blog... my rules!
&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;	Finally, I’d like to recommend a good book on the history of computing. The old maxim still stands: "Those who do not know history are doomed to repeat it." But most computer specialists are only dimly aware of the rich history of their chosen field. There are quite a few books available on computing hsitory and most provide coverage of the basics. A current favorite though, is The Universal History of Computing: From the Abacus to the Quantum Computer by Georges Ifrah. The book offers a comprehensive journey through the history of computing. Particularly interesting is the chronological summary offered up in Chapter 1. It starts out in 35000 BCE - the era from which we have discovered the first notched bones that were probably used for counting, and progresses into the modern era of computing. This book spans the complete history of information processing providing useful insight into the rise of the computer.

Now I don’t pretend to believe that these are the only classic books in IT literature, but I do know that they will provide a good, solid core foundation for your IT library. Books promote knowledge better than any other method at our disposal. And knowledge helps us do our jobs better. So close down that web connection and pick up a book. You’ll be glad you did.

Access Your DB2 Catalog "Poster" Online

2010-06-22T15:07:00.003-05:00

If you're anything like me, you're constantly looking for DB2 Catalog table and column names. For writing catalog queries, for examining statistics, for looking at your table and tablespace parameters, for many, many things. But it is not very easy to keep reaching for the DB2 manuals (Which manual is it in? Which appendix was that? Why did they put it there? Did they move it?)...

So, many of us gladly tacked up those posters from Platinum Technology that graphically depicted the DB2 Catalog... and then later similar posters from CA, Inc, and BMC Software... but those posters have grown is size (as has the DB2 Catalog)... and the posters have become less useful over the years because they contain less information and the smaller type.

Well, here comes a solution from zSystems and SoftwareOnZ: a free and very easy to use online DB2 Catalog reference application.

If you can use a web browser and a mouse then you can find the DB2 Catalog information you desire. Simply point and click on the appropriate table and you'll get its definition along with a listing of its columns and their data type and length. And you'll also get information about which columns participate in any indexes and what type of index (primary, unique, duplicate index).

Not only that, there is both a V8 and V9 edition of the DB2 Catalog so you can easily toggle back and forth between the two versions. That is very handy for sites that have some V8 subsystems and some V9 subsystems!

So if you are looking for a better way to view your DB2 Catalog information, be sure to check out the online DB2 Catalog reference from zSystems.

IDUG NA 2010, Days Two and Three

2010-05-13T17:04:00.003-05:00

I’ve been running around kinda busy the past couple of days here at IDUG in Tampa, so I got a bit behind in blogging about the conference. So, today I’m combining two days of thoughts into one blog post.

(For a summary of IDUG Day One, click here.)

I started off day two by attending Brent Gross’ presentation on extracting the most value from .NET and ODBC applications. Brent discussed some of the things to be aware of when developing with .NET, an important “thing” being awareness that .NET is designed to work in a disconnected data architecture. So applications will not go through data a row at a time but instead send the data to the application and let it process it there. As an old mainframe DBA that caused alarm bells to ring.

I also got the opportunity to hear Dave Beulke discuss Java DB2 developer performance best practices. Dave delivered a lot of quality information, including the importance of developing quality code because Java developers reuse code – and you don’t want bad code being reused everywhere, right?

Dave started out mentioning how Java programmer are usually very young and do not have a lot of database experience. So DBAs need to get some Java knowledge and work closely with Java developers to ensure proper development. He also emphasized the importance of understanding the object to relational mapping method.

From a performance perspective Dave noted the importance of understanding the distributed calls (how many, where located, and bandwidth issues), controlling commit scope, and making sure your servers have sufficient memory. He also indicated that it is important to be able to track how many times Java programs connect to the database. He suggested using a server connection pool and to be sure that threads are always timed out after a certain period of time.

And I’d be remiss if I didn’t note that Dave promoted the use of pureQuery, which can be used to turn dynamic JDBC into static requests. Using pureQuery can improve performance (perhaps as much as 25 percent), as well as simplifying debugging & maintenance.

Dave also discussed how Hibernate can cause performance problems. Which brings me to the first session I attended on day three, John Mallonee’s session titled Wake Up to Hibernate. Hibernate is a persistent layer that maps Java objects to relational tables. It provides an abstraction layer between DB2 and your program. And it can also be thought of as a code generator. Hibernate plugs into popular IDEs, such as Eclipse and Rational tools. It is open source, and part of JBoss Enterprise Middleware (JBoss is a division of Red Hat).

John walked attendees through Hibernate, discussing the Java API for persistence, its query capabilities (including HQL, or Hibernate Query Language), and configuration issues. Examples of things that are configurable include JDBC driver, connection URL, user name, DataSource, connection pool settings, SQL controls (logging, log formatting), and the mapping file location.

HQL abstracts SQL. It is supposed to simplify query coding, but from what I saw of it in the session, I am dubious. John warned, too, that when HQL is turned into SQL the SQL won’t necessarily look the way you are used to seeing it. He recommended to setup the configuration file such that it formats the generated SQL or it won’t be very readable. John noted that one good thing about HQL is that you cannot easily write code with literals in them; it forces you to use parameter markers.

OK, so why can Hibernate be problematic? John talked about four primary concerns:

SQL is obscured
performance can be bad with generated code
Hibernate does not immediately support new DB2 features
Learning curve can be high

But he also noted that as you learn more about these problems -- and how Hibernate works -- that things tend to improve. Finally (at least with regard to Hibernate) John recommends that you should consider using HQL for simple queries, native SQL for advanced queries, for special situations use JDBC, and to achieve the highest performance use native DB2 SQL (e.g. stored procedure).

I also attended two presentations on the DB2 for z/OS optimizer. Terry Purcell gave his usual standout performance on optimization techniques. I particularly enjoyed his advice on what to say when someone asks why the optimizer chose a particular path: “Because it thinks that is the lowest cost access path.” After all, the DB2 optimizer is a cost-based optimizer. So if it didn’t choose the “best” path then chances are you need to provide the optimizer with better statistics.

And Suresh Sane did a nice job in his presentation in discussing the optimization process and walking thru several case studies.

All-in-all, it has been a very productive IDUG conference… but then again, I didn’t expect it to be anything else! Tomorrow morning I deliver my presentation titled “The Return of the DB2 Top Ten Lists.” Many of you have seen my original DB2 top ten lists presentation, but this one is a brand new selection of top ten lists… and I’m looking forward to delivering it for the first time at IDUG…

IDUG Tampa 2010, Day One

2010-05-12T21:53:00.002-05:00

As usual, the North American IDUG conference is proving to be a hectic, yet enjoyable and informative time. The days are packed from morning til evening with technical sessions, networking, and running from here to there and back again.

Tuesday was the first day for normal IDUG sessions (the day-long seminars were moved to Monday this year), and the day was dominated (for me at least) by DB2 10 sessions. The spotlight session by Jeff Josten was an information-packed 90 minutes overview of DB2 10 that can only be described as drinking from a firehose. Myself and about 200 other curious attendees sat in attention as Jeff discussed the features that back up the themes of Versionn 10, which are efficiency, resilience, and growing new workloads on DB2 for z/OS.

Jeff didn’t share a GA date for the new version, nor would anyone else from IBM this week, but it has been strongly hinted that it could be before the end of the year (2010).

The biggest “thing” being touted by IBM about DB2 10 is the performance gains it delivers right out-of-the-box. Jeff discussed IBM’s performance objectives as historically being to deliver less than a 5% performance regression from release to release. But things have perked up recently. For DB2 9, most customers reported no regression or gain out of box. And the new goal is no longer containing regression, but delivering gain. For DB2 10, the expectation is that many customers will reduce CPU time 10% to 20% right out-of-the-box.

In IBM’s labs, Jeff indicated that the out-of-the-box CPU reduction numbers for traditional workloads are ranging from 5-10% and for newer workloads (e.g. TCP/IP, stored procedures) the improvement is as much as 20% in lab measurements. And when you start using new functionality, you can reasonably expect to see up to 10% CPU reduction. Of course, Jeff was careful to note that these are pre-GA numbers so things could change, even though there is no expectation that they will change.

Additionally, there is a lot of focus on scalability in DB2 10. Shops can expect to support 5x to 10x more concurrent users, up to 20,000 per subsystem. This is possible due to virtual storage relief: threads have been moved above the bar.

Jeff went on to cover a lot of additional new functionality to be delivered with DB2 10 including parellel index update during INSERT (which should speed up inserts against tables with multiple indexes), DB2’s usage of 1MB page size (z/OS) in buffer pools, multiple SQL access path and performance improvements, efficient caching of dynamic SQL with literals, LOB streaming between DDF and rest of DB2, Workfile spanned records (PBG), INSERT improvements for UTS, solid state disk monitoring and exploitation, temporal data support, timestamp data type improvements, and more.

Hash support is particularly interesting. With hashing you can get direct access to data with a single getpage instead of the multi-getpage approach of b-tree indexing. The targeted use case for hashes is for lookup of a row based upon primary key. The hashing algorithm is stored in the DB2 engine. Never fear, though, because you can still define additional indexes on hashed tables and the optimizer will understand and prefer hashed access when it is possible. (I hear the IMS DBAs out there laughing. DB2 DBAs are now going to need to understand space calculations for hash space and what collisions and overflow means.)

Next up was Roger Miller who covered DB2 10 from a database administration perspective. He began his session by referencing the extra detail that is available in the DB2 10 webcast presentation that Roger did about last month, which is available on the web.

Roger states that a lot of what is at the heart of DB2 10 is about making things easier for DBAs. And then to prove his point he talked for an hour about all of those things. Highlights included the reduced need for REORG, monitoring enhancements, hashing, and pureXML enhancements for usability, scalability, and performance.

A particularly interesting point made by Roger is that query parallelism these days is less about decreasing elapsed time and more about the ability to shuttle workload to a zIIP.

Roger also discussed the ability to skip V9 and go directly from V8 to V10. He also expressed concern that folks who choose to do this not ignore learning all about V9 when they do this. For example, RUNSTATS in V9 had key changes, so shops need to be careful to run RUNSTATS when moving to V10.

Roger also spoke about the significant changes to the DB2 Catalog and DB2 Directory in DB2 10. There are about 60 new table spaces, the links have been removed, inline LOBs are used in many places, and row level locking is used. These changes mean that online REORG works for everything in the catalog & the directory.

He also spoke about the various improvements to security administration in DB2 10. There is a new SECADM authority with no access to data and there is also a new option for DBADM without data access. Another nice new option is DBADM authority for every database in the subsystem. And then there is the ability to REVOKE without cascading, something that DB2 security administrators have been looking for for years!

Changing pace, I attended Billy Sundarrajan’s presentation on “De-mystifying JDBC Universal Drivers – for the z/OS DBA.” The reality is that more and more dynamic SQL applications are being implemented, so knowing about JDBC drivers is a necessity, not a luxury for the mainframe DBA.

Billy discussed the types of JDBC drivers and the installation issues involved. You can connect using a type 2 or type 4 driver. The Type 2 driver connects directly without DB2 Connect gateway; Type 4 driver connects thru DB2 Connect gateway.

He also discussed the benefits of setting end user variables for monitoring and the different properties that can be used for configuration.

Of course, I attended a few other sessions and spent some time at the exhibit hall and caught up with some old friends and… well, this is long enough of a post for the first day… check back tomorrow for a shorter (I promise) synopsis of day two.

IDUG in Tampa

2010-05-09T17:47:00.003-05:00

It is Sunday, May 9, 2010 and I'm posting a brief blog entry today to remind everyone about IDUG in Tampa this week. I will be attending (arrive Monday morning) and I will update my blog with the highlights of what is happening in Tampa this week... so be sure to check in regularly.

On Becoming a DBA

2010-04-30T00:23:00.000-05:00

Perhaps the most frequent question I am asked is: How can I become a DBA?

The answer, of course, depends a lot on what you are currently doing. Programmers who have developed applications using a database system are usually best-suited to becoming a DBA. They already know some of the trials and tribulations that can occur when accessing a database.

If you are a programmer and you want to become a DBA, you should ask yourself some hard questions before you pursue that path. First of all, are you willing to work additional, sometimes crazy, hours? Yes, I know that many programmers work more than 40 hours already, but the requirements of the DBA job can push people to their limits. It is not uncommon for DBAs to work late into the evening and on weekends; and you better be ready to handle technical calls at 2:00 a.m. when database applications fail.

Additionally, you need to ask yourself if you are insatiably curious. A good DBA must become a jack-of-all-trades. DBAs are expected to know everything about everything -- at least in terms of how it works with databases. From technical and business jargon to the latest management and technology fads, the DBA is expected to be "in the know." And do not expect any private time: A DBA must be prepared for interruptions at any time to answer any type of question -- and not just about databases, either.

And how are your people skills? The DBA, often respected as a database guru, is just as frequently criticized as a curmudgeon with vast technical knowledge but limited people skills. Just about every database programmer has his or her favorite DBA story. You know, those anecdotes that begin with "I had a problem..." and end with "and then he told me to stop bothering him and read the manual." DBAs simply do not have a "warm and fuzzy" image. However, this perception probably has more to do with the nature and scope of the job than with anything else. The DBMS spans the enterprise, effectively placing the DBA on call for the applications of the entire organization. As such, you will interact with many different people and take on many different roles. To be successful, you will need an easy-going and somewhat amiable manner.

Finally, you should ask yourself how adaptable you are. A day in the life of a DBA is usually quite hectic. The DBA maintains production and test environments, monitors active application development projects, attends strategy and design meetings, selects and evaluates new products and connects legacy systems to the Web. And, of course: Joe in Accounting just resubmitted that query from hell that's bringing the system to a halt. Can you do something about that? All of this can occur within a single workday. You must be able to embrace the chaos to succeed as a DBA.

So, if you want to become a DBA you should already have some experience with the DBMS, be willing to work long and crazy hours, have excellent communication and people skills, be adaptable and excel at organization. If that sounds like fun, you'll probably make a good DBA.

The Ever-Changing Role of the DBA

2010-04-22T12:35:00.002-05:00

Defining the job of DBA is getting to be increasingly difficult. Oh, most people know the rudimentary aspects of the job, namely keeping your organization's databases and applications running up to par. The DBA has to be the resident DBMS expert (whether that is DB2, Oracle or SQL Server, or most likely a combination of those). He or she has to be able to solve thorny performance problems, ensure backups are taken, recover and restore data when problems occur, make operational changes to database structures and, really, be able to tackle any issue that arises that is data-related.

The technical duties of the DBA are numerous. These duties span the realm of IT disciplines from database design to physical implementation and consistent, on-going monitoring of the database environment.

DBAs must possess the abilities to create, interpret, and communicate a logical data model and to create an efficient physical database design from a logical data model and application specifications. There are many subtle nuances involved that make these tasks more difficult than they sound. And this is only the very beginning. DBAs also need to be able to collect, store, manage, and query data about the data (metadata) in the database and disseminate it to developers that need the information to create effective application systems. This may involve repository management and administration duties, too.

After a physical database has been created from the data model, the DBA must be able to manage that database once it has been implemented. One major aspect of this management involves performance management. A proactive database monitoring approach is essential to ensure efficient database access. The DBA must be able to utilize the monitoring environment, interpret its statistics, and make changes to data structures, SQL, application logic, and the DBMS subsystem to optimize performance. And systems are not static, they can change quite dramatically over time. So the DBA must be able to predict growth based on application and data usage patterns and implement the necessary database changes to accommodate the growth.

And performance management is not just managing the DBMS and the system. The DBA must understand SQL used to access relational databases. Furthermore, the DBA must be able to review SQL and host language programs and to recommend changes for optimization. As databases are implemented with triggers, stored procedures, and user-defined functions, the DBA must be able to design, debug, implement, and maintain these code-based database objects as well.

Additionally, data in the database must be protected from hardware, software, system, and human failures. The ability to implement an appropriate database backup and recovery strategy based on data volatility and application availability requirements is required of DBAs. Backup and recovery is only a portion of the data protection story, though. DBAs must be able to design a database so that only accurate and appropriate data is entered and maintained - this involves creating and managing database constraints in the form of check constraints, rules, triggers, unique constraints, and referential integrity.

DBAs also are required to implement rigorous security schemes for production and test databases to ensure that only authorized users have access to data. As industry and governmental regulations multiply, the need to audit who did what to which data when is also a requirement for sensitive data in production systems – and the DBA must be involved in ensuring data auditability without impacting availability or performance.

And there is more! The DBA must possess knowledge of the rules of relational database management and the implementation of many different DBMS products. Also important is the ability to accurately communicate them to others. This is not a trivial task since each DBMS is different than the other and many organizations have multiple DBMS products (e.g., DB2, Oracle, SQL Server).

Remember, too, that the database does not exist in a vacuum. It must interact with other components of the IT infrastructure. As such, the DBA must be able to integrate database administration requirements and tasks with general systems management requirements and tasks such as network management, production control and scheduling, and problem resolution, to name just a few systems management disciplines. The capabilities of the DBA must extend to the applications that use databases, too. This is particularly important for complex ERP systems that interface differently with the DBMS. The DBA must be able to understand the requirements of the application users and to administer their databases to avoid interruption of business. This includes understanding how any ERP packages impact the business and how the databases used by those packages differ from traditional relational databases.

But Things Are Changing

So at a high level, DBAs are tasked with managing and assuring the integrity and efficiency of database systems. But keep in mind, too, that there are actually many different DBAs. Some focus on logical design; others focus on physical design; some DBAs specialize in building systems and others specialize in maintaining and tuning systems; and there are specialty DBAs and general-purpose DBAs. Truly, the job of DBA encompasses many roles.

Some organizations choose to split DBA responsibilities into separate jobs. Of course, this occurs most frequently in larger organizations, because smaller organizations often cannot afford the luxury of having multiple, specialty DBAs.

Still other companies simply hire DBAs to perform all of the tasks required to design, create, document, tune, and maintain the organization’s data, databases, and database management systems.

But no matter what "type" of DBA you happen to be, chances are that your role is changing and adapting to new types of computing and data requirements. Indeed, one of the biggest challenges for DBAs these days is the ongoing redefinition of the job roles and responsibilities.

The primary role of database "custodian," of course, continues to be the main emphasis of the job. But that is no longer sufficient for most organizations. The DBA is expected to take on numerous additional -- mostly technical -- roles. These can include writing application code, managing the application server, enterprise application integration, managing Web services, network administration and more.

If you compare the job description of DBAs across several organizations, it is likely that no two of them would match exactly. This is both good and bad. It is good because it continually challenges the technically-minded employees who tend to become DBAs. But it can be bad, too; because the job differs so much from company to company, it becomes more difficult to replace a DBA who leaves or retires. And no one can deny that database administration is a full-time, stressful job all on its own. But the stress level just keeps increasing as additional duties get tacked onto the DBA's "to do" list.

Summary

There are many jobs that DBAs perform and it can be confusing when you try to match job title up against the responsibilities of the job. Don't let your job title keep you from expanding into other, related disciplines. The more you know and the more you can do, the more employable you become... and that is important in this day and age!

Nominate Someone for the CA IDUG Award for Outstanding Work in DB2

2010-04-01T14:34:00.002-05:00

As many of you know, each year CA sponsors an award at IDUG to honor outstanding work with DB2. The only requirement for the award is that you and your company have used DB2, either on the mainframe or distributed system, in a novel, ground-breaking, or cutting-edge manner.

There is no requirement that an entrant organization be based in North America. There is also no requirement that an entrant organization license or use specific CA or other vendor database management tools or that he/she attend the IDUG conference in person. All nominations will be judged by a panel of independent DB2 consultants in conjunction with CA executives.

Nominations are now open through April 26, 2010. To learn more, visit ca.com/awards/db2.

The winner will be announced at the IDUG North America conference, May 10-14, 2010.

Each winner will receive:

One complimentary full IDUG conference pass for any IDUG Europe or IDUG North America conference held prior to December 2011
Half-day consulting engagements by each of the consultant judges within 12 months of award presentation
Expense reimbursement up to $1,500 either (a) the winner’s travel to an IDUG conference held prior to December 2011 or (b) travel by one of the DB2 consultants for on-site provision of the complimentary consulting services described above
Recognition plaque

Submit your project today and get recognized for your outstanding work. Visit ca.com/awards/db2.

What is Production Data?

2010-03-17T14:41:00.001-05:00

I received an interesting e-mail recently that made me stop and think a bit... so I thought I'd blog about it. Basically, the e-mail posed the question in the title of this blog entry – “What is production data?”

The e-mail read as follows:

I'm looking for a one paragraph definition of "production data". What do you think of this: "Production data is data recorded for the purpose of controlling/managing/reporting/researching events, processes or states."

I'm trying to get around the belief that data recorded by a development team to manage its projects and resources is somehow less than production data. To me it should be regarded as the development team's "production data" and so I'm looking for a definition that satisfactorily encompasses that belief, as well as encompassing regular business production data.

You know, I do not recall ever seeing an actual definition of the term “production data.” The above definition is a good starting point, but I do not think it is complete. The author of the e-mail makes a good point about different types of production data. The data used by an application development team to conduct their business (writing computer programs to support business processes) is definitely production data… to the application development team.

Here is my take on a definition:

Production data is information that is persistently stored and used by professionals to conduct business processes. It must be accurate, documented, and managed on an on-going basis to ensure its value to the organization.

I say information instead of data because the data must be defined and in context in order to be useful for production work. And I say persistent because even though there may be many forms of transitory data used by production processes, it is the data that is stored over periods of time that needs to be managed.

I think this definition should serve the needs of the e-mailer... and more. What do you think?

Did I miss anything?

Mainframes: The Safe IT Career Choice

2010-03-05T10:10:00.000-06:00

A recent Computerworld article (Bank of America touts mainframe work as a safe career) touts the mainframe as a safe haven for those considering a career in IT. This is an interesting article because the usual spiel you hear in industry trade rags is that the mainframe is dying and only a fool would work on such a platform. It is good to hear an alternate opinion on the matter in a journal as respected as Computerworld. (Of course, the fact that I agree with this opinion might have a little something to do with my cheer upon reading the article.)

One of the highlights of this particular article is the discussion of avialable mainframe jobs at sites such as Monster (764 jobs over 30 days) and Dice.com (1,200 ads over 30 days). These are significant numbers of jobs, especially in a down economy.

Another interesting tidbit from this piece is that "IBM says it's mainframe revenue has grown in eight of the last 13 quarters." This is impressive; consider the difficult servers market coupled with the impression that the platform is dying.

Speaking of the death of the mainframe, don't you believe it for a minute. People having been predicting the death of the mainframe since the advent of client/server in the late 1980s. That is more than 20 years! Think of all the things that have died in that timespan while the mainframe keeps on chugging away: IBM's PC business, Circuit City, Koogle peanut butter, public pay phones, Johnny Cash... the list is endless.

Some may counter that they recall reading about companies that were going to eliminate their mainframe. Well, yes, I'm sure you do remember those, I do, too. But do you recall reading many articles about companies that SUCCESSFULLY eliminated their mainframes? Many tried, few succeeded. Indeed, the re-Boot Hill web site provides examples of companies that tried to eliminate the mainframe but could not (hence, they had to re-boot). If you follow the link to the re-Boot Hill site click on the little tombstones to read the stories of failure.

So, the mainframe is a rock-solid platform, continues to grow, and is producing a significant number of job opportunities... what is not to like?

This blog has moved

2010-03-05T09:41:00.004-06:00

This blog is now located at http://db2portal.blogspot.com/.

You will be automatically redirected in 30 seconds, or you may click here.

For feed subscribers, please update your feed subscriptions to
http://db2portal.blogspot.com/atom.xml.

IBM Announces DB2 10 for z/OS Beta Program

2010-02-09T11:21:00.003-06:00

IBM announced the beta program for the next version of DB2 today, now "officially" known as DB2 10 (no more DB2 X). It is a closed beta program that will begin on March 12, 2010. That means you have to be selected by IBM to participate.

The announcement highlighted some of the areas of improvement to be delivered by DB2 10 for z/OS, and at the top of that list, to no one's surprise, is performance. DB2 10 promises to deliver out-of-the-box savings by improving operational efficiencies ranging from 5% to 10% out-of-the-box CPU savings for traditional workloads and up to 20% out-of-the-box CPU savings for nontraditional workloads.

Other areas called out by IBM in the announcement include

Improved business resiliency through scalability improvements and fewer outages (planned or unplanned).
Schema evolution or data definition on demand as well as query performance manageability enhancements support improved availability.
New features such as hash access, index include columns, inline large objects, parallel index updates, faster single row retrievals, work file in-memory, index list prefetch, 64-bit memory enhancements, use of the 1 MB page size of the System z10, buffer pools in memory, access path enhancements, member clustering for universal table spaces, efficient caching of dynamic SQL statements with literals, improved large object streaming, and SQL procedure language performance.
Rapid application and warehouse deployment for business growth including improved concurrency for data access, data management, and data definition.
The ability avoid an outage by adding active log data to a subsystem.
Improved application and data warehousing support including temporal data, a 64 bit ODBC driver, currerntly committed locking, implicit casting or loose typing, timestamp with time zone, variable timestamp precision, moving sum, and moving average.
Improvements to DB2's XML support including expanded pureXML, customer-driven performance and usability requirements, schema validation in the engine, binary XML exchange format, multiversioning, easy update of subparts of XML document, stored procedures, user-defined functions and triggers, XML index matching with date/timestamp, and a CHECK XML utility.
Enhanced query and reporting facilities, including QMF V10 with over 140 new analytical functions, support for HTML, PDF, and Flash reports, and more.

So it would seem that there is a lot of new functionality for us to begin to become acquainted with. As IBM rolls out more details, and customers begin to use the new version of DB2, we will examine some of these new features in more depth here on the DB2 Portal blog.

If you are interested in the beta program, the pre-requisite for DB2 10 is z/OS V1.10 (5694-A01) or later running in 64 bit mode. More information about the DB2 10 beta program is available on IBM's web site.

No GA date for DB2 10 has been announced.

IBM Manages the Data Lifecycle

2010-02-03T21:55:00.002-06:00

Data lifecycle is a somewhat new-ish term, at least in terms of what I plan to talk about in this blog posting. The data lifecycle – and data lifecycle management – deals with tracking, managing, and understanding data and metadata as it flows through organizations. From its inception…whether entered by a clerk or read via a feed or loaded from an external source, etc…through its various usages…whether to conduct business, analyze trends and patterns, and so on…tracked from system to system, application to application, and user to user…and finally through its end of life.

Not many companies today can track all of their important data and what happens to it throughout its entire lifecycle. But doing so is important. Having such a capability enables organizations to adapt and react, gaining a competitive advantage. Much can go awry as data moves throughout an organization. Schema changes, policy changes, regulations adapt, programs change, formats changes, and so on. Any of these things can cause data quality issues, which should be brought to the attention of the business analyst using the data. But how often is this done? Knowing the history of data and its related metadata can improve business processes. But it is a major task – both for businesses and IT vendors hoping to offer solutions.

Which brings me to today’s (February 3, 2010) announcements from IBM. Big Blue announced new data protection software, a line of consulting services and resources and previewed information monitoring software to help organizations expand their use of trusted information to improve decision making. These moves further bolster IBM’s already formidable arsenal of data lifecycle management solutions.

The data protection announcement was for Optim Data Redaction. This solution, engineered for unstructured data like Word documents and PDF files, automatically recognizes and removes sensitive content from documents and forms. For example, a customer’s credit scores in a loan document could be hidden from an office clerk, while still being visible to a loan officer. In today’s atmosphere of more and more stringent regulations, a data redaction solution is becoming a requirement. For example, PCI DSS industry standards dictate specific rules regarding the display of debit and credit card information on receipts and reports.

Optim Data Redaction is planned for general availability in March 2010.

The information monitoring announcement was for InfoSphere Business Monitor. This technology is based on a combination of work from IBM’s research group and technology gained when IBM acquired Guardium. Guardium is a database activity monitoring (or auditing) solution. InfoSphere Business Monitor tracks the quality and flow of an organization’s information and provides real-time alerts of potential flaws. For example, if a health insurance company was analyzing profit margins across different product lines (individual, group, HMO, Medicare, etc.), decision makers would immediately be alerted when a data feed from a specific geography was not successfully integrated.

InfoSphere Business Monitor is available as a technology preview; it is not generally available and no GA date was announced.

At the same time, IBM announced its intention to acquire Initiate Systems, a provider of data integrity software for information sharing among healthcare and government organizations. Initiate's software helps healthcare clients work more intelligently and efficiently with timely access to patient and clinical data. It also enables governments to share information across multiple agencies to better serve citizens. IBM plans to continue to support and enhance Initiate's technologies while helping clients take advantage of the broader IBM portfolio, specifically Cognos and InfoSphere solutions for BI and analytics. This acquisition bolsters IBM’s data lifecycle management offering along these verticals.

And all of today’s announcements serve to clarify IBM’s ascent to the throne within the realm of information and data lifecycle management.

Larry Sure Knows How to Get Press

2010-02-02T12:41:00.004-06:00

Going under the assumption (I assume) that no press is bad press, Oracle CEO Larry Ellison has attacked IBM's DB2... but he made several factual errors in his rant.

Here are some of the highlight (?) of the claims Ellison made about DB2 during a webcast last week.

Regarding TPC-C benchmarks, Ellison claims to have "(blown) the doors off of IBM. We crushed them." He went on to elaborate saying "In a machine that took up less than 10% the floor space, of IBM's record setting computer. We ran faster, we ran a lot faster: using a tiny fraction of the floor space, a tiny fraction of the power, cost less."

First of all, technicians working in trenches know that benchmarks are not indicative of real life performance. That aside, it is true that Oracle currently has the leading TPC-C benchmark result. Until late in 2009, DB2 enjoyed a massive 49% lead over Oracle. Oracle's most recent results give them a 25% lead (using more than six times as many CPU cores to do it).

Regarding the claim of using less space and power, this is due to Oracle using flash memory and comparing it with an IBM benchmark using conventional disk technology. If Oracle compared its benchmark to an IBM system using flash memory, these claims would not stand.

Later, Ellison claimed that "SAP chooses the Oracle Database to run under SAP in almost all their large accounts." As anyone who follows the computer industry knows, this claim is rather absurd. SAP's customers choose the DBMS to run, not SAP. And if SAP had anything to say about it, they would not recommend Oracle, their biggest competitor in the commercial business applications space. Furthermore, SAP favors DB2 for their own systems. They operate more than a thousand SAP systems, and all of those systems run on DB2.

Perhaps the silliest of Ellison's comments is this: "The Oracle Database scales out, IBM DB2 for Unix does not. Let me see, how many servers can IBM put together for an OLTP application? Let's see, how many can they group together? Um, one. They can have up to one server attacking really big jobs. When they need more capacity, they make that server bigger. And then they take the old server out, put a bigger one in. And when you've got the biggest server, that's it. That's all the can do for OLTP." Ellison also claimed that IBM "can't scale out, they can't do cloud, they can't do clusters, the can't do any of this."

I bet this surprised a lot of DB2 users doing these things with DB2! DB2 Parallel Edition was released in 1995, along with the capability to scale to a system of over a 100 Unix servers. DB2 LUW scalability is proven in many of the world's largest OLTP environments. Consider this press release talking about how DB2 LUW powers one of the largest OLTP systems in the world.

And what about that clustering claim? Evidently Mr. Ellison slept through 2009. IBM DB2 pureScale, released last year, offers powerful, efficient database clustering. For a cluster of 64 nodes, DB2 pureScale maintains 95% efficiency. At 128 nodes, DB2 pureScale maintains 84% efficiency. This is important because if you are growing a cluster to handle bigger workloads, you want your hardware to be doing productive work, not handling system overhead. On the other hand, Oracle RAC has a 100 server limit...

Ellison also made other far-out claims about IBM like "They're so far behind, I don't think they have any chance at all. I'm serious." Ellison also said "They are not competitive in the database business, except on the mainframe."

If this were true, why would Ellison spend any time thinking or talking about IBM. He must be worried, IMHO. Anyone with even a cursory knowledge of the computer industry has to admire IBM. They have led the industry in developing patents for the last 17 years. In 2009, IBM produced 4914 patents while Oracle did not even place in the top 50 patent leaders. A search of the US Patent office database reveals 1588 patents with "database" in the patent description while Oracle produced only 184 patents.

Hyperbole is one thing, but gross inaccuracy is another. In his latest tirade, Ellison is guilty of both. Oracle makes a good DBMS... pity its CEO doesn't think it can sell it on its own merits without making up stuff about the competition.

Some New Year's Resolutions for DBAs

2010-02-01T16:59:00.002-06:00

This is sort of a re-blogging (to coin a term). I first published this last month in the Data Management Today blog I wrote for NEON. Well, I no longer work for NEON and I'm not sure how long that blog will remain active, so I thought it might make sense to re-blog some of the pertinent entries here... so here goes with my New Year's Resolutions for DBAs blog entry...

At the beginning over every year many of us take the time to cobble together some resolutions for the coming year. We plan to lose weight, save money, stop smoking, and so on. Usually, it doesn’t take long before we’ve abandoned these resolutions. Perhaps we’d be wiser to make some business related resolutions. With that in mind, here are some thoughts on the New Year’s resolutions you might be wise to make as a DBA in 2010.

Are you insatiably curious? A good DBA must become a jack-of-all-trades. DBAs are expected to know everything about everything -- at least in terms of how it works with databases. From technical and business jargon to the latest management and technology fads, the DBA is expected to be "in the know." So perhaps “be more curious” would be a useful DBA resolution.

Most DBAs know that private time is a luxury we cannot afford. A DBA must be prepared for interruptions at any time to answer any type of question -- and not just about databases, either. With that in mind, how are your people skills? DBA are usually respected as a database guru, but also frequently criticized as a curmudgeon with limited people skills. Just about every database programmer has his or her favorite DBA story. You know, those anecdotes that begin with "I had a problem..." and end with "and then he told me to stop bothering him and read the manual." DBAs simply do not have a "warm and fuzzy" image. However, this perception probably has more to do with the nature and scope of the job than with anything else. The DBMS spans the enterprise, effectively placing the DBA on call for the applications of the entire organization. As such, you will interact with many different people and take on many different roles. To be successful, you will need an easy-going and somewhat amiable manner. So another good New Year’s resolution might be to “improve your people skills.” Take a Dale Carnegie course or start by reading Carnegie’s seminal book, How to Win Friends and Influence People.

How adaptable you are? A day in the life of a DBA is usually quite hectic. The DBA maintains production and test environments, monitors active application development projects, attends strategy and design meetings, selects and evaluates new products and connects legacy systems to the Web. And, of course: Joe in Accounting just resubmitted that query from hell that's bringing the system to a halt. Can you do something about that? All of this can occur within a single workday. You must be able to embrace the chaos to succeed as a DBA. So a third resolution might be to “roll with the punches” better – and without complaining!

Of course, you need to be organized and capable of succinct planning, too. Being able to plan for changes and implement new functionality is a key component of database administration. And although this may seem to clash with the need to be flexible and adaptable, it doesn't really. Not once you get used to it. You just need to prepare yourself to be adapatable and organize to incorporate change more rapidly than others. So my final suggestion for a 2010 New Year’s resolution is to adopt a planning methodology and stick to it. Buy a planner – either electronic or not – and use it this year. You might even consider taking a time management class.

If you keep all of these resolutions, just imagine how productive you will be in 2010. And then you can use 2011 to lose weight and save money and…

Which is better? "BETWEEN" vs "<=" and >"="

2010-01-25T12:08:00.002-06:00

This was a recent topic on the DB2-L mailing list so I thought I'd weigh in with my two cents worth on the topic.

As with most DB2 (and, indeed, IT) issues, the correct answer is "it depends!" Let's dig a bit deeper to explain what I mean.

From a maintainability perspective, BETWEEN is probably better. The BETWEEN predicate is easier to understand and code than the equivalent combination of the less than or equal to predicate (<=) and the greater than or equal to predicate (>=). In past releases, in many cases it was more efficient, too. But today the Optimizer recognizes the two formulations as equivalent and there usually is no performance benefit one way or the other. Performance reasons aside, one BETWEEN predicate is easier to understand and maintain than multiple <= and >= predicates. For this reason, I tend to favor using BETWEEN.

But not always. Consider the scenario of comparing a host variable to two columns. Usually BETWEEN is used to compare one column to two values, here shown using host variables:

WHERE COLUMN1 BETWEEN :HOST-VAR1 AND :HOST-VAR2

However, it is possible to use BETWEEN to compare one value to two columns, as shown:

WHERE :HOST-VAR BETWEEN COLUMN1 AND COLUMN2

This statement should be changed to

WHERE :HOST_VAR >= COLUMN1 and :HOST-VAR <= COLUMN2

The reason for this exception is that a BETWEEN formulation comparing a host variable to two columns is a Stage 2 predicate, whereas the preferred formulation is Stage 1. And we all know that Stage 1 outperforms Stage 2, right?

Remember too, that SQL is flexible and often the same results can be achieved using different SQL formulations. Sometimes one SQL statement will dramatically outperform a functionally equivalent SQL statement just because it is indexable and the other is not. For example, consider this SQL statement

SELECT EMPNO, FIRSTNME, MIDINIT, LASTNAME
FROM EMP
WHERE MIDINIT NOT BETWEEN 'A' AND 'G';

It is not indexable because it uses the NOT BETWEEN predicate. However, if we understand the data in the table and the desired results, perhaps we can reformulate the SQL to use indexable predicates, such as

SELECT EMPNO, FIRSTNME, MIDINIT, LASTNAME
FROM EMP
WHERE MIDINIT >= 'H';

Or we could code MIDINIT BETWEEN 'H' AND 'Z' in place of MIDINIT >= 'H'. Of course, for either of these solutions to work correctly we would need to know that MIDINIT never contained values that collate lower than the value 'A'.

So, as usual, there is no one size fits all answer to the question!

On Specialty Processors

2010-01-20T14:21:00.003-06:00

Today's blog entry is kind of a meta entry.

As many of my readers know I write two blogs, this one that is predominantly about DB2 for z/OS, and the Data Management Today blog, that focuses on data-related issues, database management, and industry trends.

Well, I've written a series of entries on my other blog about specialty processors (zIIPs, zAAPs, etc.) and related issues. Since DB2 for z/OS folks should find that information useful and relevant, I thought I'd write an entry pointing my readers here to the pertinent specialty processor entries at my Data Management Today blog, so here goes...

The first entry I'd like to call your attention to is titled Specialty Processors on the Mainframe. This piece is basically an introduction to the different types of specialty processors (zIIp, zAAP, IFL, ICF), and what they can be used for. It is a good place to start if you are new to specialty processors or are looking for an update.

The second entry worth a peak is titled simply zAAP on zIIP. As you may or may not know, IBM delivered the capability for zAAP work to run on a zIIP (with certain conditions). This blog entry provides a brief synopsis of the August 18, 2009 z/OS V1.11 announcement that introduced the new capability to enable zAAP-eligible workloads to run on zIIPs

Next up is What is an Enclave? If you are working with zIIPs you have probably heard the term Enclave SRB. And if you are doing any type of distributed workload you've probably heard about enclaves, too. This blog entry is for those who are new to the term, or are confused about it. It offers an explanatory definition of the term "enclave" and points you on to additional reference material for those interested.

My most recent post over there (January 19, 2010), titled What is Generosity Factor?, has been a popular one. This blog entry delves into the generosity factor imposed upon zIIP workload including definitions of geneorsity factor, qualified and eligible work, and a discussion of what it implies for ISV products.

Hope you find this material worthwhile... and thanks for your continued support of my blogs.

Evaluating a DBA Job Offer

2010-01-11T11:38:00.002-06:00

Today's blog entry is for DBAs. Although this blog is focused on mainframe DB2 the advice here applies to any DBA job... so pass the linik on to your Oracle and SQL Server friends...

As a DBA, it is almost inevitable that you will change jobs several times during your career. When making a job change, you will obviously consider requirements such as salary, bonus, benefits, frequency of reviews, and amount of vacation time. However, you also should consider how the company treats their DBAs. Different organizations place different value on the DBA job. It is imperative to your career development that you scout for progressive organizations that understand the complexity and ongoing learning requirements for the position.

Here are some useful questions to ask:

Does the company offer regular training for its DBAs to learn new DBMS features and functionality?

As a DBA you need to be well-versed on the latest and greatest features of the DBMSs you manage. And, on average, there will be a new version to contend with every 18 to 24 months. It is possible to learn the basics by reading the WHAT'S NEW manual and skimming through the voluminous, additional manuals, but some formal training is warranted to get the most out of a new version of the DBMS.

What about training for related technologies such as programming, networking, e-business, transaction management, message queueing, and the like?

DBAs are also called upon to administer more than just databases these days. A top notch employer will allow its DBAs to be trained in new technologies... as well as time for independent learning through reading books and articles.

Does the company allow DBAs to regularly attend local user groups? What about annual user groups at remote locations?

User groups are essential for networking with others who perform the same, or similar, job duties. By attending local user group meetings you can not only get inexpensive training through watching the presentations, but you can also learn exchange ideas with your peers.

Are there backup DBAs, or will you be the only one on call 24/7?

Nobody wants to be the only DBA on call, every night, all the time, on weekends, holidays, etc. And if there is no backup what happens if you take a vacation? Is it really a vacation if you have to carry a company cell phone everywhere you go?

Are there data administration and system administration organizations, or are the DBAs expected to perform all of these duties, too?

DBA is a full-time job but some organizations expect the DBA staff to handle data administration and system administration duties, too. Depending on the volume of work this might not be a deal breaker, but be cautious.

Does the DBA group view its relationship with application development groups as a partnership? Or is the relationship more antagonistic?

A partnership is essential in order to produce optimally performing database applications. And if you do not have applications that perform well, then the DBA job will be burdensome.

Are DBAs included in design reviews, budgeting discussions, and other high-level IT committees and functions?

The more involved the DBA team is in the overall IT strategy the better prepared the company's databases will be to support the required work... and the easier your job will be as a DBA.

The more YES answers you get to these questions, the more progressive the DBA environment is. Be sure to ask these questions during your interview. It will show that you have experience and that you care about your career. Be sure to research the answers later, too. Ask around about the company to those who used to work there and anyone you know (remember those user groups) that currently works there. Sometimes the answers given by the workers will not exactly match those given by the interviewer.

Keep in mind, too, that these are not the ONLY questions you need to ask.

And good luck with you DBA career!

Happy Holidays

2009-12-21T18:24:00.002-06:00

Just a brief posting today to wish everyone a very happy holiday season.

I am taking some down time thru the end of the year to visit family "up North."

So, until next year, may your databases run without a glitch and here's hoping you all have an enjoyable holiday!

Quick Thoughts on DB2 Performance

2009-12-16T15:59:00.003-06:00

Database performance problems are not caused by magic. Indeed, all performance problems are always caused by change. That statement flies in the face of what I normally say, which is “Almost never say always or never”… but in this case, it is true.

Think about it for a moment. If everything remains stable and unchanging in your environment, then why would performance vary? That’s right, it wouldn’t.

Something tangible must change before a performance problem can be experienced. The challenge of performance tuning is to find the source of the change, gauge its impact, and formulate a solution.

Change can take many forms, including the following:

Physical changes to the environment, such as a new CPU, new disk devices, or different tape drives.
Changes to system software, such as a new release of a product (for example, WebSphere, CICS, or even z/OS), the alteration of a product (for example, the addition of more or fewer CICS regions or an IMS SYSGEN), or a new product (for example, implementation of DFHSM). Also included is the installation of a new release or version of DB2, which can result in changes in access paths as well as utilization of new features.
Changes to the DB2 engine from maintenance releases and PTFs, which can change the optimizer (and sometimes introduce other new functionality).
Changes in system capacity. More or fewer jobs could be executing concurrently when the performance problem occurs. Or additional users may be banging away at your transactions.
Environmental changes, such as the implementation of client/server programs, the adoption of SOA, or other new technologies.
Database changes. This involves changes to any DB2 object, and ranges from adding a new column or an index to dropping and re-creating an object.
Changes to the application development methodology, such as usage of check constraints instead of application logic or the use of stored procedures.
Changes to application code, both SQL and host language code (COBOL, C, Java, etc.).

Although the majority of your performance problems are likely to be application-oriented, you must be prepared to explore any and all of these other areas when application tuning has little effect.

My advice is to be sure that you institute strict change control tracking across all areas of your IT infrastructure. That way, whenever you experience a performance problem, you will be able to track what has changed recently, along with who changed it and why. This is important because every DBA knows what the answer to the question “What changed?” will be… right?

It is always “nothing!”

And that cannot be true. Oh, it does not mean that the person answering is lying. He or she may not have changed anything. And it is not necessarily reasonable to expect an application developer to know what all could have changed…especially when what can impact DB2 performance spans so many areas of the IT infrastructure.

So do yourself… and your company a favor: be sure that you meticulously track each and every change to any aspect of your systems. Then – and this is where many shops break down – make sure that you have methods of tying all of the change information together in such a way that it can be queried and examined in the face of a performance problem.

Only then can you reasonably expect your DBAs rapidly to be able to track down and remedy DB2 performance problems… because only then will they have the pertinent information at their disposal.

A Short History of DB2 for z/OS – Part 2

2009-12-11T13:53:00.002-06:00

Today’s blog entry is a continuation of yesterday’s post in which we began a brief review of the history of DB2 for z/OS. That post covered Versions 1 through 3; so today we pick up our historical review with Version 4.

Version 4 was a very significant milestone in the history of DB2. It was highlighted by the introduction of Type 2 indexes, which removed the need to lock index pages (or subpages, now obsolete). Prior to V4, index locking was a particularly thorny performance problem that vexed many shops. And, of course, I’d be remiss if I did not discuss data sharing, which made its debut in V4. With data sharing, DB2 achieved new heights of scalability and availability unmatched within the realm of DBMS; it afforded users the ability to upgrade without an outage and to add new subsystems to a group “on the fly.” The new capabilities did not stop with there; V4 also introduced stored procedures, CP parallelism, performance improvements, and more. DB2 V4 was, indeed, a major milestone in the history of mainframe DB2.

In June 1997 DB2 Version 5 became generally available. It was the first DB2 version to be referred to as DB2 for OS/390 (previously it was DB2 for MVS). Not as significant as V4, we see the trend of even numbered releases being bigger and more significant than odd numbered releases (of course, this is just my opinion). V5 was touted by IBM as the e-business and BI version. It included Sysplex parallelism, prepared statement caching, reoptimization, online REORG, and conformance to the SQL-92 standard.

Version 6 brings us to 1999 and the introduction of the Universal Database term to the DB2 moniker. The “official” name of the product is now DB2 Universal Database for OS/390. And the Release Guide swelled to over 600 pages! Six categories of improvements were introduced with V6 spanning:

Object-relational extensions and active data
Network computing
Performance and availability
Capacity improvements
Data sharing enhancements
User productivity

The biggest of the new features were SQLJ, inline statistics, triggers, large objects (LOBs), user-defined functions, and distinct types.

Version 6 is also somewhat unique in that there was this “thing” typically referred to as the V6 refresh. It added functionality to DB2 without there being a new release or version. The new functionality in the refresh included SAVEPOINTs, identity columns, declared temporary tables, and performance enhancements (including star join).

March 2001 brings us to DB2 Version 7, another “smaller” version of DB2. Developed and released around the time of the Year 2000 hubbub, it offered much improved utilities and some nice new SQL functionality including scrollable cursors, limited FETCH, and row expressions. Unicode support was also introduced in Db2 V7. For a more detailed overview of V7 (and the V6 refresh) consult An Introduction to DB2 for OS/390 Version 7.

DB2 Version 8 followed, but not immediately. IBM took advantage of Y2K and the general desire of shop’s to avoid change during this period to take its time and deliver the most significant and feature-laden version of DB2 ever. V8 had more new lines of code than DB2 V1R1 had total lines of code!

I don’t want to get bogged down in recent history here outlining the features and functionality of DB2 releases that should be fresh in our memory (V8 and V9). If you really want some details on those refer to these links for them:

An Overview of DB2 for z/OS Version 8

DB2 9 for z/OS Features

Which brings us to today. Most shops should be either running Version 9 in production or planning their migration from V8 to V9. And we are all waiting with baited breath for DB2 X… which hopefully should be announced sometime next year.