IBM Information on Demand 2011: Day Three (#IODGC)

Day three at the IOD conference in Las Vegas started off with a Big Data focus. The first thing in the morning was the general session which began, as usual with a series of factoids and interesting statistics. The highlights of which, in my opinion, were these:

• There are over 34,000 Google searches done every second. Which is a huge number, but not unbelieveable....
• 1 in 3 business leaders make decisions based on data they don't trust. Personally, I think the other 2 are just kidding themselves.
• Top 2 leadership challenges according to IBM study: increasing complexity and exploding data volume. (Hey, what about small budgets?)
• And in a survey of IOD conference attendees, 55 percent say that the relationship between business and IT is getting better... let's hope... at some shops it couldn't get any worse, could it?

At any rate, the general session kicked off and Katty Kay of BBC America was the emcee again. She did a great job at hosting today's general session (and yesterday's, too). But I have to say, the general sessions are nowhere near as entertaining as they have been at previous IOD events.

Steve Mills then took the stage. For those who don't know Mr. Mills, he is Senior Vice President and Group Executive - Software and Systems. Mills provided an overview of Big Data from an IBM perspective. Mills kicked things off with a pithy quote saying "Everybody is talking about big data these days, as if data wasn't already big." True... he then went on to outline the big data challenge as the 3Vs: variety, velocity, volume. Not a bad start, but a few Vs short in my opinion... should include vicinity and validity.

Another piece of wisdom from Mills' keynote was this: Big Data is not a single structure, but many structures. He also stated that Big Data must be an integral part of the enterprise data platform...

Mills also discussed various examples of big data solutions that IBM was working with customers on, including my personal favorite, analyzing massive volumes of "space weather" data in motion. But perhaps a more "down to Earth" example can be found in IBM helping to analyze sensor data in offshore oil rigs where more than 2 TB of data is being processed on a daily basis.

Later in the day I attended a panel on customer sentiment analysis. Professor Jonathan Taplin, Director of USC's Annenberg Innovation Lab talked about analyzing social media data to measure customer sentiment around various areas including film, fashion, and even the World Series. The information uncovered helped identify movies that would tank and several film studios began working with the Lab to identify customer sentiment earlier in the cycle. After all, how does it help a movie studeo to learn that a major motion picture is about to tank on the Thursday before it opens? The studios worked with the Lab to learn about negative sentiment earlier so the studios could try to reverse the sentiment through marketing efforts.

There are tremendous spinoff opportunities for this type of sentiment analysis because it gives greater insight into customer behavior. It can potentially be applied to other areas, too, such as measuring employee sentiment, or perhaps citizen sentiment to help predict and uncover events such as the Arab Spring.

The key take away is to realize that this is a different type of data with low latency, real time applications. It is not the type of data that we will be storing long term in databases or data warehouses.

According to Taplin, the next challenge is realtime analysis using IBM's InfoSphere Streams product.

I also attended a fantastic lunch provided by IBM for IBM Champions. Thank you IBM for the nice spread and recognition. I am proud to be an IBM Champion!

Today was also the day of my presentation and I delivered my DB2 for z/OS Performance Tuning Roadmap to a packed audience. I deliver 60 slides in just a little bit over an hour and the presentation seemed to be well received...

The conference ended with a fantastic concert from the band Train. I liked the band before this, but I really like them after the concert! Let me tell you, the lead singer Patrick Monahan has a heckuva set of pipes. The guy can flat out sing. This became abundantly clear not just in their stellar versions of thier hits (like "Meet Virginia", "Calling All Angels", and "Hey, Soul Sister") but also the fantastic cover versions (especially the "Ramble On/Walk On The Wild Side" mashup). Yes, Monahan did the vocals proud and can match Robert Plant note for note. I did not expect that. And their cover of "Dream On" was pretty good, too!

Finally, don't forget to keep checking in on the video blogs I am hosting for SoftBase Systems. Today's blog interview was with advanced SQL expert, Sheryl Larsen. Check it out by clicking here!

IBM Information on Demand 2011: Day Two (#IODGC)

As promised, here is the second of my daily blogs from the IOD conference in Las Vegas. Today it was reported that the attendance at the event was the highest ever for an Information On Demand conference; there are more than 11,500 registered attendees.

The second day of the conference is when things really start humming with news and in-depth presentations. The day kicked off with the general session delivered by a host of IBM executives and customers. Big data, business analytics, and gaining insight into data was the theme of the session. The opening session was peppered with lots of interesteing facts and figures. For example, did you know that 90 percent of the world's data was created in just the last two years? Me neither... but there was no attribution to that nugget of information, so...

Other highlights of the day included the announcement of Cognos Mobile for the iPhone and iPad (a free trial is available on the iTune store)… and the other big product focus of the day was IBM InfoSphere BigInsights, a Hadoop-driven big data solution that can process huge amounts of data very quickly and accurately. For more details on that offering check out my Data Technology Today blog where I cover a customer implementation of this solution.

I also had the opportunity to chat with IBM's Bernie Spang, Director of Marketing, Database Software and Systems. We chatted about various things, starting with the uptake of DB2 10 for z/OS. Earlier in the day it was stated that the uptake of V10 has been faster than for V9 and I asked Bernie why that was. His answer made a lot of sense: skip-level migration support coupled with a clear performance boost out-of-the-box without having to change the database or the apps. I asked if he had metrics on how many customers had migrated, but he didn't have access to that. He said he would get back to me and when he does I will share that information with you all.

We also chatted quite a bit about the recently announ ced DB2 Analytics Accelator. Bernie thinks this is probably the announcement he is most excited about. For those of you who haven't heard about this great piece of technology, it is the second iteration of the Smart Analytics Optimizer (but that name is now dead). The DB2 Analytics Accelerator is built on Netezza technology and can be used to greatly improve the performance of DB2 for z/OS analytical queries without changing the SQL or any application code. There are multiple value points but Bernie pointed out the application transparency and the ability to keep the data on the z platform (no movement required) while accelerating the performance of analytical queries.

IBM views the competition as Oracle Exadata and Teradata, which makes sense. I asked Bernie if there were plans to incorporate the Oracle compatibility features of DB2 LUW in a future iteration of DB2 for z/OS, and he said that made sense. Of course, no one from IBM will commit to future functionality of an as yet to be announced  version, but perhaps Vnext??? (that was me speaking there, no Bernie!)

Then I think I blew his mind when I passed a thought of mine past him. With Netezza being used as a component of an accelerator to improve DB2 analytical processing, has IBM given any thought to using IMS as a component of an accelerator to improve DB2's traditional OLTP processing. Not sure if that is even possible, but it should be worth a research project, right? Especially with IBM announcing IMS 12 at the conference today and the IBM boast that IMS 12 can achieve 61,000 transactions per second. That is impressive! But can the mismatch between relational and hierarchical be overcome in a useful manner to do it?

Finally, we chatted about Informix. As a DB2 bigot I am always at a loss for when to direct people to Informix instead of DB2. It just doesn't sound like something I would do! But Bernie offered a brief overview of Informix time series as something unique that certain customers should look into. An Informix customer uses time series for smart meter management of over 100 million smart meters. A month's worht of data - 4 terabytes - can be loaded and processed in less than 8 hours. And some queries perform from 30x to 60x faster.

OK, even to this DB2 bigot that sounds like an impressive capability. Kudos to Informix.

Finally, I'd like to direct my readers over to the video blog that I am hosting in conjunction with SoftBase Systems. I'll be interviewing DB2 luminaries daily, so tune in there at http://www.softbase.com/blog to view each daily submission!

Until tomorrow...

Information On Demand 2011: Day One (#IOD11)

Well, the first day of the IOD conference is just about behind us. As usual, Sunday is a day to get acclimated to Vegas and the Mandalay Bay conference center. If you are here, I hope you brought some comfortable shoes, because you'll be doing a LOT of walking.

Typically, the highlight of the first day is the opening of the Expo Hall, and this year was no exception. The hall was jam-packed with IBM booths demonstrating and promoting all kinds of software, from DB2 to Informix to Analytics to InfoSphere to Big Data to Cloud and more. And there were also a large number of ISVs in the Expo Hall, too. It could take most of the week to visit all of the booths and learn about all the great technology on display.

But, of course, we won't be doing that. Tomorrow is the beginning of the educational sessions, kicking off in the morning with the general session, which this year is titled Turning Insight Into Action. Actually, that is the theme of this year's conference, too.

Word is that attendance is up this year over the 10,000 attendees at last year's conference. I haven't heard an official number yet, but I've heard rumors of more than 11,000 attendees this year.

As the week progresses, I will tweet (http://www.twitter.com/craigmullins) my experiences, and blog about the conference daily. So be sure to check back here, as well as on my Twitter feed, for the straight scoop from IOD.

To end today blog posting on a high note, here are a few facts about the latest IBM Information Management and Business Intelligence activities:

• IBM projects $16 billion in business analytics software and services revenue by 2015
• Over the past 5 years, IBM has invested more than $14 billion in 25 key acquisitions including Cognos, Netezza, and SPSS (and many others)
• IBM is committed to researching advanced analytics technologies as demonstrated by Watson (who is here at the conference) and IBM's $100 millions investment to develop new tools toward tackling Big Data challenges.
• Analytics software and services for IBM were up 17 percent in their second quarter

Also, remember that I will be videotaping highlights and interviews from the conference this year in conjunction with SoftBase Systems. You can find links to these videos as the become available daily at https://www.softbase.com/blog/.
Goodbye for now... Hope to see you all again tomorrow as we discuss day two of the conference...

Information On Demand 2011( #IODGC)

Just a quick post today, Saturday, October 22nd 2011, to let everybody know that I will be blogging daily from the IOD conference in Las Vegas this week.

I'll try to keep my readers up-to-date on what is going on by posting my thoughts about the conference, covering the news and announcements that are made, and by working with SoftBase Systems to produce daily videos on the news of the day along with daily interviews of DB2 luminaries... so whether you can't make the conference this year, or can but want to keep abreast of things, keep checking back here for more daily details from IOD.

Let's start by letting everybody know that I will be presenting "IBM DB2 Performance Tuning Roadmap" on Tuesday, 10/25, at 2:00. I'm just one of 59 IBM Champions that will be presenting at this year's IOD conference. Here is a list if you are interested.

DB2 Developer's Guide, 6th edition

I know a lot of my readers are waiting on the updated edition of my book, DB2 Developer's Guide. This short blog post is to let you know that the wait is almost over. The book will be published early next year and is available to be pre-ordered on amazon com.

The book has been completely updated and is now up-to-date with DB2 10 for z/OS. Just think of the things that have been added to DB2 since the last time the book was updated: Universal table spaces, pureXML, SECADM, hashes, new data types, INSTEAD OF triggers, temporal support, and much, much more.

Consider pre-ordering a copy today so you'll get it as soon as it comes off the presses!

IBM announces Smart Analytics System 5710

Last week (September 2011), IBM announced the Smart Analytics System 5710, which is a database appliance for business intelligence and data analytics targeted at the SMB market. The IBM Smart Analytics System 5710 is based on IBM System x, runs Linux, and includes InfoSphere Warehouse Departmental Edition and Cognos 10 Business Intelligence Reporting and Query.

The announcement of this appliance was somewhat lost in the shuffle of Oracle's marketing blitz for its similar Oracle Database Appliance, also announced last week. But IBM's offering is geared and pre-configured for quick deployment of analytics and business intelligence capabilities.

The IBM Smart Analytics System 5710 is powered by the InfoSphere Warehouse Departmental Edition which is built on a DB2 data server, and features Optim Performance Manager, DB2 Workload Manager, Deep Compression and Multidimensional clustering.

The IBM Smart Analytics System 5710 provides key capabilities of reporting, analysis and dashboards to enable fast answers to key business questions delivered as a cost-effective solution designed for rapid deployment. It allows users to quickly extract maximum insight and value from multiple data sources and deliver a consistent view of information across all business channels.

It also provides cubing services giving users a multidimensional view of data stored in a relational database. Users can create, edit, import, export, and deploy cube models over the relational warehouse schema to perform deeper multi-dimensional analysis of multiple business variables improving both profitability and customer satisfaction. Cubing services also provide optimization techniques to dramatically improve the performance of OLAP queries.

Additionally, the powerful, yet simple, data mining capabilities enable integrated analytics of both structured and unstructured data in the system. Standard data mining models are supported and can be developed via drag and drop in an intuitive design environment.

So what does it cost? For a such a rich collection of software, the starting price is just under $50K. Furthermore, the new offering is part of the IBM Smart Analytics System family, which consists of solutions that span multiple hardware platforms and architectures, including the mainframe (System z).

DB2 10 for z/OS: For Developers Only!

Today's blog post is to promote an upcoming FREE webinar that I will be delivering titled DB2 10: For Developers Only! The presentation is sponsored by the good folks at SoftBase Systems. and it will be conducted on Wednesday, September 14th, 2011, from 2:00 to 3:00 pm EST.

This presentation highlights the DB2 10 for z/OS enhancements that directly impact DB2 application developers. Every release of DB2 is chock full of new features and functionality and that can make it hard to focus on those things that are most helpful for programmers. So instead of scanning volumes of manuals, you can watch this presentation distills the DB2 10 information down to cover what should be most important to programmer/analysts.

Examples of areas this presentation will cover include:
• Binding issues and details for V10
• Temporal support with examples
• A new type of function
• New timestamp options and some improvements to existing SQL
• Implicit casting, access to currently committed data, and much more…

If you are a programmer wanting to learn more about DB2 10, or a DBA looking for the programmer’s perspective on DB2 10, this presentation should have something to offer you.

This is sort of a tradition for me... you may have heard me give a similar presentation for previous DB2 versions. Well, this webinar introduces a brand new presentation in this series, this time for DB2 Version 10...

So register today!

DB2 Symposium 2011 – Round Two

Today's blog post is about the DB2 Symposium, a three day training event with one day seminars presented by well-known DB2 consultants. I was fortunate enough to be asked to participate this year by the primary organizer of the event, Klaas Brant. (Klaas is a respected DB2 consultant based in the Netherlands.). Earlier this year, the DB2 Symposium event was held in Dallas, TX and was well-received by attendees. So a second round is planned, this time in Chicago, IL!

What is the difference between DB2 Symposium and events like IDUG and IOD? Well, DB2 Symposium fills the gap between a conference and a multi-day training course. The DB2 Symposium is unique because you can participate for 1, 2, or 3 days, depending on your needs and budget.

Round two of the USA DB2 Symposium is happening soon, so you'll need to act fast if you want to participate. It occurs September 21-23, 2011 in the Chicago, Illinois area. More precisely, at the DoubleTree Hotel in Downers Grove, IL (in the Western suburbs of Chicago). Each day the training sessions start at 9.00am and end at around 5.00pm.

But registration on site is not possible, you must pre-register online... so plan ahead!

My session is on September 23rd and it is called DB2 Developer's Guide Comes Alive! This one day session, covers tips, techniques, and procedures you need to know in order to excel at administering and using DB2 on the mainframe. The material is based upon DB2 Developer's Guide, the best-selling DB2 for z/OS book on the market. Additionally, the course material will contain references to sections of the book for students to find additional material on each topic after the sessions. Topics to be covered will include:

• A performance tuning roadmap for managing DB2 application, database and system performance. You will learn SQL coding and tuning techniques, guidance for database optimization and reorganization, coverage of buffer pool settings and parameters for performance.
• Logical and physical database design recommendations for DB2, so you can build and maintain effective DB2 databases immediately. Includes discussion of standards, logical to physical translation, data types, usage of nulls, and more.
• Information and guidance on BINDing and REBINDing, along with a discussion of the most important parameters.
• Along the way we'll look at locking, access paths, statistics, indexing and more.
• And even though the current edition of the book covers through DB2 V8, this course adds coverage of some of the newer features added to DB2 in versions 9 and 10 that can boost your productivity and performance.

If you own the book already, bring it along and I'll be happy to autograph it for you. And then you can use it along with the course materials... and if you don't own it already, you'll probably want to grab a copy after attending the seminar... you can always find a link to buy my books on the front page of my web site at http://www.craigsmullins.com.

So register for the DB2 Symposium today... and I'll see you in Chicago!

Plan to Attend IDUG This November to Improve Your DB2 Skills

It is almost that time of year again, time for the annual European IDUG conference. And as anyone who has ever attended this conference can tell you, it will be chock full of news, education, and networking opportunities for DB2 programmers, DBAs, sysprogs, and users! And you can take advantage of great offers including FREE IBM certification training and DB2 migration workshops!

So be sure to register for the IDUG DB2 Tech Conference in Prague, Czech Republic, 13-18 November 2011, at the Clarion Congress Hotel Prague. If you register on or before 17 October 2011 and receive a discount of EUR 275.

IDUG is also extending the following special offers:

• Multiple Delegate Discount
  For every three individuals who register from the same organisation, a fourth may attend at the discounted rate of EUR 730.
• Mentor Program
  If you have attended three previous IDUG DB2 Conferences, you are eligible to bring a first-time colleague to Prague for an 80% discount off the full registration fee.
• One-Day Educational Seminars - Friday, 18 November 2011
  Registration for paid full conference delegates is EUR 450; the cost for just the one-day seminar is EUR 495. Select from the following session topics:
  - DB2 10 for z/OS - In Depth, Phil Grainger, Cogito
  - DB2 Intermediate and Advanced SQL, Daniel Luksetich, Yevich Lawson & Assoc Inc.
  - I Didn't Know DB2 did THAT!, Bonnie Baker, Bonnie Baker Corporation
  - Optimising DB2 for z/OS System Performance Using DB2 Statistics Trace, John Campbell, IBM Corporation
  - Rocket Science: DB2 for LUW Performance Analysis and Tuning Workshop, Scott Hayes, DBI

Register Today!

Do You Still Use That DB2 Program?

A recurring problem for programmers and IT folks in general is determining whether or not a particular program is still required. As your organization grows and the number of programs increases, keeping track of them all can become quite difficult.

As administration and management burdens increase, a common desire is to get rid of programs that are no longer being used. But it can be difficult to determine which programs are no longer used.

You can always “ask around,” but few IT professionals would be willing to delete anything based on such a general tactic. Another common method is to review performance reports or extracts from a performance warehouse. But perhaps your performance traces are not turned on all the time.

The question is probably more common in DB2 environments because of the plans and packages that consume storage and "sit around" taking up space if their associated program is no longer being used.

Well, for DB2 professionals this type of question becomes easier to answer once you migrate to DB2 10 for z/OS. DB2 maintains a new column, LASTUSED, in the DB2 Catalog. The column exists in both SYSIBM.SYSPACKAGE and SYSIBM.SYSPLAN and is defined as a DATE data type. The date is changed when the package header is requested from EDM. The column is also maintained for triggers and stored procedures. After BIND (REPLACE) this column is reset to the default value (CURRENT DATE).

This is similar to the LASTUSED column added to SYSIBM.SYSINDEXSPACESTATS in DB2 9, which is used to show the last time an index was used.

Of course, you will have to give it some time –- because you might have a program that is used only rarely, yet still used. Most shops have queries and programs that run quarterly, or even annually, but nevertheless are very important. So don’t just start freeing packages a month after you’ve migrated to DB2 10!

But it is good to know that we now have additional usage information at our fingertips in the DB2 Catalog, isn’t it?