One Service Definition To Rule Them All?

This is one of those posts where I have to be careful about what I say – to not throw a customer under the bus with over-specificity. So, bear with me on that score.

Actually it’s two customers, not one. But they have the same challenge: Managing Workload Manager (WLM) service definitions for multiple sysplexes.

These two customers have come to different conclusions, for now. I say “for now” because, in at least one case, the whole design is being reconsidered.

Some Basics

Even for those of us who know them well, the basics are worth restating:

• A sysplex can have one and only one service definition.
• A service definition cannot be shared between sysplexes, but it can be copied between them. These are separate installations and activations.
• In a sysplex different systems might not run work in all the service classes.
• The ISPF WLM application has been used since the beginning to edit service definitions.
• You can print the service definition from the ISPF WLM application.
• Relatively recently the z/OSMF WLM application became available – as a more modern (and more strategic) way of editing service definitions.
• You can export and import the service definition as XML – in both the ISPF and z/OSMF applications.

What I’ve Been Doing Recently With WLM Service Definitions

For many years I relied entirely on RMF Workload Activity SMF records (SMF 72-3) when tuning WLM implementations. This simply wasn’t (good) enough, so a fair number of years ago I started taking WLM service definitions in XML form from customers when working with them.

Why wasn’t RMF SMF good enough? Let’s review what it buys you:

• It tells you how service class periods behave, relative to their goals and with varying workload levels.
• It allows you to examine report class data, which often is a good substitute for SMF 30 address space information.

So I’ve done a lot of good work with SMF 72-3, much of which has surfaced as blog posts and in conference presentations. So have many others.

But it doesn’t tell you what work is in each service class, nor how it got there. Likewise with report classes. Admittedly, SMF 30 will tell you an address space’s service and report classes, but it won’t tell you why. Likewise, SMF 101 for DDF work will give you a service class (field QWACWLME) but it won’t tell you why, and it won’t tell you anything about report classes. And SMF won’t tell you about history, always a fascinating topic.

To understand the structure of the service definition you need the XML (or the print, but I don’t like the print version). So that’s what I’ve been looking at, increasingly keenly.

A Tale Of Two Customers

• Customer A has multiple sysplexes – on two machines. They sent me multiple XML service definitions – as they have one per sysplex.
• Customer B has multiple sysplexes – on more than two machines. They sent me a single XML service definition – so they are propagating a single service definition to multiple sysplexes.

As it happens, in both customers, there are application similarities between some of their sysplexes. In Customer B there are different application styles between the LPARs in a single sysplex – and this fact will become important later in this discussion.

So, which is right? The rest of this post is dedicated to discussing the pros and cons of each approach.

Multiple Service Definitions Versus A Single Propagated One

The first thing to say is if you have an approach that works now think carefully before changing it. It’ll be a lot of work. “A lot of work” is code for “opportunity for error” rather than “I’m too lazy”.

If you have one service definition for each sysplex you might have to make the same change multiple times. But this would have to be an “egregious” change – where it’s applicable to all the sysplexes. An example of this would be where you’d misclassified IRLM address spaces and now needed to move them all into SYSSTC (where they belong). There’s nothing controversial or specific about this one, and not much “it depends” to be had.

For changes that are only relevant or appropriate to one sysplex this is fine. For example, changing the duration of DDFHI Period 1 for a Production sysplex.

But if you had a DDFHI in each of several sysplexes this could become confusing – if each ended up with a different specification. Confusing but manageable.

If you had one service definition for a group of sysplexes you’d make a change in one sysplex and somehow propagate that change around. You’d use literally the same XML file (assuming you go the XML route), export it from the first sysplex, and import it into the rest. The mechanism, though, is less important than the implications.

Any change has to be “one size fits all”. Fine for the IRLM example above, but maybe not so good for eg a CICS Region Service Class velocity change. In the “not so good” case you’d have to evaluate the change for all the sysplexes and only if it were good (enough) for all of them make the change.

But, you know, “one size fits all” is a problem even within a sysplex: Customer B has different applications – at quite a coarse-grained level – in different LPARs within the same sysplex:

• The common componentry in the LPARs – e.g Db2 or MQ – probably warrants the same goals.
• The application-specific, or transaction management, componentry – e.g. Broker, CICS, IMS, Batch – probably doesn’t.

There are some ways of being “differential” within a service definition. They include:

• Treating one transaction manager differently from another.
• Classification rules that are based on system.
• Classification rules that are based on sysplex.

That latter – classification rules that are based on sysplex – is something I addressed very recently in my sd2html WLM Service Definition XML Formatter open source project. If you run it there’s a “Service Definition Statistics” table near the top of the HTML it produces. If there are classification rules based on sysplex name the specific sysplexes are listed towards the bottom of the table. This will save me time when looking at a customer’s service definition. You might use it , for example, to check whether such rules are still in place – when you thought they’d been removed. The specific use of the “Sysplex Name” qualifier will appear in either the Classification Groups or Classification Rules table, depending.

(I could use a good XML comparer, by the way. Because today I can’t easily tell the differences between Service Definition XML files – and I don’t think I should be teaching sd2html to do it for me.)

Conclusion

There is no conclusion, or at least no general firm conclusion; You really have to think it through, based on your own circumstances. Most notably two things:

• Whether “one size fits all” works for you.
• Whether operational procedures in WLM service definition management are effective.

But, still, it’s an interesting question. And it’s not hypothetical:

• Plenty of customers have separate Sysprog, Development, Pre-Production, and Production sysplexes. In fact I’d encourage that.
• Quite a few customers have separate Production sysplexes, especially outsourcers.

And this is why I find “Architecture” such fun. And why WLM is anything but simple – if you do it right.