When Good Work Doesn’t Go To zIIP / zAAP Heaven

(Originally posted 2007-05-09.)

Whenever I talk about zIIPs and zAAPs I’m reminded of Zip Zap miniature radio-controlled cars. But (to get past the censors I have to say) zIIPs and zAAPs are not toys.

Here’s an extract from a post on MXG-L that Bernie Pierce made. (I reproduce it with his kind permission.)

If there is contention for the local lock, the cross memory local lock, the CMS lock or any other suspend lock, units of work that are zAAP or zIIP eligible will be dispatched on a standard processor as they are given the lock in periods of contention for the lock. When the lock is available, this processing does not apply; it applies only when the requestor is deferred due to lock unavailable. This overrides the IFAHONORPRIORITY=NO option. In extreme cases this may be an indication of possible scalability issues if the MIPS consumed by the application are expected to increase significantly in the future. One application I examined recently had so much local lock contention that one third of the zAAP eligible time was consumed on CPs with IFAHONORPRIORITY=NO. I would not be too concerned about 2-3%.

The customer asking the question to which Bernie responded obviously had 2-3% of the eligible work running on general-purpose engines (GCPs).

I think it’s worthwhile tracking this with the appropriate SMF 72 (Service-Class level) and SMF 30 (Address-Space level) instrumentation – but not to be too paranoid about it. It’s another example of where you need to dig below the marketing material to understand what’s really going on. And, through the life of zIIPs and zAAPs we’ll all learn a lot more about how they work and how work is scheduled on them.

A comment that may get me in trouble: The way zIIPs and zAAPs work has gotten complicated (to my mind) to the extent that I wonder how many people really know what’s going on and how to manage them.

The state of the art evolves… I’m told that if there were (hypothetically) to be a similar control to IFAHONORPRIORITY for zIIPs , the same considerations would apply to zIIP when NO is selected as for zIIPs.

DB2 Data Sharing Performance and Coupling Facility Enhancements for DB2 Version 8

(Originally posted 2007-05-08.)

Occasioned by some recent enhancements to MXG I thought it high time I talked about a couple of Coupling Facility commands that DB2 Version 8 can take advantage of:

(And thanks to Barry Merrill for the MXG information and the sample data I’m using here.)

MXG Version 25.04, May 7, 2007 provides support for these enhancements.

Coupling Facility Level (CFLEVEL) 13 introduced the WARM and RFCOM commands. These batch up pages for both the Write And Register process and the Read for Castout process. The former is for when pages have to be written to the coupling facility and the latter is for when pages are written out to disk by the castout owner at castout time. They’re intended to be more efficient than their single-page analogues.

DB2 still does single page actions – where appropriate.

DB2 Statistics Trace (SMF 100 in this case) provides instrumentation:

On the Castout side:

QBGLCM      RFCOM*REQUESTS*FOR*MULTIPLE*PAGES          3964
QBGLCR      RFCO*REQUESTS*FOR*SINGLE*PAGE              1900
QBGLRC      PAGES*CASTOUT                             31638

Using the above MXG variables, the number of pages castout using RFCOM is 31638 – 1900 or 29738 or 94%. The number of pages read for each RFCOM is 29738 / 3964 or 7.5.

Similarly one can do arithmetic with the WARM and WAR statistics:

QBGLMW      WARM*REQUESTS*FOR*MULTIPLE*PAGES           8066
QBGLWP      PAGES*WRITTEN*BY*WARM                     85449
QBGLWS      WAR*REQUESTS*FOR*SINGLE*PAGES             85714

Actually these last two numbers look suspicious: One might suspect that the number of WAR requests was 85714 – 85449 which actually is exceedingly small. Assuming the names are right (and they do reflect what’s in SDSNMACS) then one can conclude about half of the pages written were written by WAR and half by WARM (which is credible). The average number of pages written per WARM command is 85449 / 8066 or 10.6.

These are useful statistics as they show how these two CF commands (WARM and RFCOM) do make things more efficient by batching up requests. The other thing is I expect there will be conditions that prevent WARM and RFCOM from batching to this degree. Things in the category of “trickling”, for example. It might change the way we recommend configuring buffer pool and GBP thresholds.

I’d like to hear from other sites about their numbers. Especially the WAR / WARM ones.

UKCMG Conference, June 25-27, Whittlebury Hall, Northants

(Originally posted 2007-05-02.)

I’ve just submitted my three presentations for this great conference:

  • Much Ado About CPU
  • Memory Matters in 2007
  • DB2 Data Sharing Performance For Beginners

UKCMG is – for me – a great chance to catch up with customers, vendors and others in the industry. Its prime focus is Performance – multiplatform in fact – but we always have a thumpingly good showing of mainframe and DB2 topics. And this year IBM is a Platinum sponsor, which might explain why we have a nice booth for me to hang around. I wonder what our freebies will be. 🙂

And the website for UKCMG is UK Computer Measurement Group.

Gameframes

(Originally posted 2007-04-27.)

Hmmm: IBM press release on the gameframe.

I’d like to see a lot more detail on this before I go “whoopie!” but it does sound like an intriguing idea. It would be the first “specialty” engine to actually feature different hardware. Reminds of me of the 3090 Vector Facility but it’s really nothing like it.

And at this stage there’s nothing about what will actually be shipped.

Still, I’m happy to see this because:

  • Someone’s getting to have fun like this in the mainframe world.
  • Hosting gaming servers (for example World of Warcraft) and Virtual Worlds (for example Secondlife) is an interesting and rapidly growing niche.

Update November 2007

First, here’s another developerWorks blogger’s take on the “gameframe”.

Second, according to the keynote at UK GSE Conference in October, we’re talking more “adjunct processor” than “specialty engine”. So more akin to Crypto than to zIIP.

Third, I’m thinking SMF 70 is going to have to evolve again to cope with this. At least that’s what I’d argue for.

But then nothing’s been announced beyond the original announcement. So we’ll just have to wait and see.

Update May 2008

I sat in an internal presentation a couple of days by my good friend Carl Parris in Poughkeepsie – who is leading the technical side of the project. I think the “gameframe” is in great shape and it’s highly likely I’m going to have some practical involvement with it in the early days. I’m frankly psych’ed. 🙂

What I Learned From My Conference Evaluations

(Originally posted 2007-04-26.)

I was pleasantly surprised to see my conference evaluations so soon – given the European System z Technical Conference was only last week. Very professional to have the feedback straight away.

All the people who filled in the evaluation forms – and that exceeded 50% of the attendees in every session – were very kind, I feel. Here’s what I learned:

I still talk too fast, and still have highly idiomatic English. So, given my international role, I’ll definitely have to work on both of those. My next conference is the “take no prisoners with UK English” 🙂 UKCMG conference in June. Then I hope to be at the North American version of this current conference in the Autumn – where maybe I’ll have to adapt my style again. I have friends who can tell me if my English is still too idiomatic, though maybe they’re unable now to act as cyphers for the average North American sysprog. 🙂

I wonder why I talk too fast: Perhaps it’s nervousness (and I’ll admit to just a tad of that), perhaps it’s having too much material (and maybe I should cut it down a little), but I’m hoping it’s just enthusiasm and a desire to throw in “the kitchen sink”.

On “the kitchen sink” it’s very difficult to know what your audience expects of you. Clearly, just parroting the manuals isn’t good enough. It’s taken me a while to realise I deliver more than that – and perhaps to relax about the “technical credibility” thing. And since these are my own foils (with one exception this time) I shouldn’t feel I’m just parroting some standard IBM Marketing material. But I’d hate not to deliver the “kitchen sink”.

And I realise that for some readers the term “kitchen sink” may be unfamiliar: If you “throw in the kitchen sink” it means you “throw in everything you’ve got”.

I’m not really agonising about this stuff, mind. But I do care. So thanks for the evaluations (which were pretty positive) and thanks for the evaluation comments.

Finally, do any other speakers feel this way? I guess they must do: I’ve had dear friends desperately worried about presenting and how they went down with their “public”.

European System z Technical Conference – End of Day 5

(Originally posted 2007-04-20.)

Marna Walle: Recent Installation Enhancements and Directions

Marna gave a very good presentation, covering a wide range of topics in this area. Here are some highlights – at least from MY perspective:

In z/OS R.7:

  • z/OS R.7 UNIX introduced Dynamic Service Activation for SELECTED corrective PTFs. Only works for PTFs marked as “for dynamic activation” with ++HOLD REASON(DYNACT). Not all PTFs are eligible. Not meant for general activation but rather occasional cases and where it’s safe to.
  • z/OS R.7 Language Environment introduced a Run-Time Options PARMLIB member – which replaces the need to create assembler LE defaults modules. One advantage is that for enhancements to LE parameters (where the default is OK for you) you no longer need to modify your assembler options and assemble it again. This means no migration action. Copy the sample member and modify, rather than converting your assembler options modules. (The syntax is different.)

In z/OS R.8:

  • Can copy a DFSMS SCDS into an ACDS without activating it. This helps create an ACDS to be used elsewhere – for example in a disaster recovery situation.
  • The bpxmtext shell command has been enhanced to support zFS to allow you to find the meaning of a zFS reason code.
  • inittab (in /etc/inittab) is now supported, providing a single point of control for starting and restarting daemons and servers. Can have etc/inittab execute etc/rc.

Previewed for z/OS R.9:

  • You’ll be able to use TSO XMIT with DSNTYPE=LARGE data sets (which can contain more than 65,535 tracks). There are other TSO functions (such as LISTDSI in REXX / CLIST) for DSNTYPE=LARGE in R.9.
  • SDSF will expose a lot of its data through REXX variables. There will be some samples. I think this is one I might use myself as some of our code screenscrapes what this would provide properly.
  • Support for Coupling Facility “Maintenance Mode”.

There’s been a lot of work on installation simplification in recent releases. I’m NOT the one to critique these enhancements. There are, though, items for both ServerPac and SystemPac.

The IBM Health Checker for z/OS is going to support migration and exploitation checks. I’ll admit it’s not a tool I get to use but A LOT of effort has gone into it – and the journey continues. For example I’ve just spotted in the R.9 preview that checks will be able to be written in REXX. Now THAT’s something else I might try.

Martina Schmidt and Thomas Schulze: WebSphere Application Server Healthcheck for System z

This is a good general presentation on ensuring the environment and setup is right for WebSphere Application Server (WAS).

I’ve noticed this week that there’s a good German team of younger IBMers working on pre-sales work for the WebSphere brand. This presentation was given by two of them.

Thomas reminded me it takes in the neighbourhood of 20% to 30% more JVM heap to create the equivalent Java garbage collection behaviour (frequency and duration) with a 64-bit heap as was seen with 31-bit.

Thomas suggested that the thread/work pipeline should get thinner (lower limits) at each stage from inital arrival through to eg DB2. I’ll have to think about that one.

There will be free System z Optimised Applications (ZOA) 3-day workshops in Germany throughout 2007.

Jython is replacing JACL as the WebSphere scripting language. Jython is an “open” language, which is why installations are being encouraged to migrate to it.

Martina did a good review of connectivity options to eg CICS and IMS.

European System z Technical Conference – End of Day 4

(Originally posted 2007-04-19.)

Parwez Hamid: System z9 BC GA2 and System z9 EC GA3 Update

Available from 11 May, Driver Level 67 provides the BC GA2 / EC GA3 enhancements. The current one is 63. Parwez set the expectation the upgrade can be done non-disruptively if you plan carefully.

Common to BC and EC machines:

  • LPAR Group Capacity Limit. Group of LPARs on one footprint need not be in the same parallel sysplex. Requires z/OS Release 8.
  • Coupling Facility code (CFCC) Level 15. Exclusive to z9. Designed to provide a basis for future CF enhancements (which will require z/OS R.9). z/OS R.6 with OA17055 for function and OA17070 for RMF support. Right now provides more allowable tasks (112 vs 48). (I mentioned this as part of Joan’s presentation yesterday.)
  • System-Initiated CHPID Reconfiguration reduces operator burden when taking CHPIDs offline. z/OS R.6 with PTFs required.
  • Multi-path IPL.
  • CHPID type FCP (QDIO) and OSA performance enhancements. (I mentioned these as part of Connie’s presentation yesterday.)
  • QDIO Diagnostic Synchronisation (Connie again).
  • On/Off Capacity on Demand flexibility improved so may well provide more responsive activation of temporary capacity.
  • HMC Logon LDAP Authentication enhances security for the Hardware Management Console (HMC). There are other HMC enhancements.
  • The System Activity Display (SAD) now has the system’s power consumption. The example of a 10-IFL machine is 4.7kW. Parwez said the biggest z9 configuration only takes 18kW – when I asked if his example was realistic. Personally I think these numbers are VERY good. And here’s an example:

BC only:

  • R07 Capacity Backup (CBU) Enhancement allows conversion to the capacity limit of a S07. Note: Some R07 limitations (compared to S07) remain. S07 and R07 are the hardware models, of course.
  • New FICON Express4-2C SX card – has fewer channels than EC FICON Express4, to meet “BC customer” requirements.
  • Crypto Express2-1P single processor card for “BC customer” requirements, compared to EC card which has two processors.

Statements of Direction:

  • Removal of support for Dynamic ICF Expansion – which allows CF LPAR to expand from dedicated ICFs to use shared General-Purpose Engines (GCPs).
  • z9 is the last system which will support older than “n-2” technology – whether as the CF or another coupled system.
  • z/OS XML processing on zAAP and zIIP specialty engines.
  • IPSec encryption and decryption on zIIP is expected be available in August 2007 – using Enclave SRBs. zIIPs support CP Assist Crypto Function (CPACF) as all characterisable engines do. Support is in z/OS R.8 with PTFs.

Me: DB2 I/O with DS8000

2 people in my session – one a hardware storage vendor and the other a business partner who does DB2 education. So we had a nice wide-ranging chat using the foils.

Isabel Arnold: Attracting Young Developers to the Mainframe using Eclipse-Based Tooling

Websphere Developer for z is Eclipse-based, so has the usual Eclipse behaviours. There is a division of labour between the Eclipse-based workstation components and host components. I noticed EGL as one of the supported languages. I’m psyched enough to want to play with it! It would be fair, though, to note that things like refactoring for mainframe languages are not YET as advanced as those for java.

Hendrik Wörner: Websphere Portal V6 for z/OS

A good presentation on why z/OS is a good place to run Websphere Portal. Those advantages are mainly the same as for Websphere Application Server on z/OS.

Isabel Arnold: Mainframe Asset Discovery, Modernization and Integration

A very good SOA-oriented overview presentation, with more detail on componentisation. Websphere Studio Asset Analyser and Application Transformation Workbench work together to help with transformation.