Maskerade

(Originally posted 2018-01-19.)

So now I get to sit and relax for umpteen hours, while a 787 speeds me to California. So I can write something. And, yes, it’s been a while.

To tell you the truth, I’ve been working on back-to-back engagements and enjoying vacation for what seems like forever. And after a stint of that I generally I end up with a swirl of “if only I had time to write about that” topics in my brain. So here I am ”Going To California On A Big Jet Plane”[1] with time on my hands.

So I have this customer – and it’s not the first like it – who has a Parallel Sysplex with a pair of Coupling Facility LPARs and a couple of Production z/OS LPARs. Not the first, either, to have offline Coupling Facility links. But I didn’t know they had offline links until they told me. Shame on me! 🙂

If a customer tells me things I guess I should know I go on a hunt for how I could have figured it out from the data. And so it was in this case. And, having just been successful at writing code to detect offline links, I’m writing this to show you what’s possible and how.

Three Sources Of Link Data

SMF 74 Subtype 4 is the main Coupling Facility Activity record, and it has three sets of information about links:

• Local Facility Data Section
• Remote Facility Data Section
• Path Data Section

You get at least one 74–4 for each coupling facility from each z/OS connected to it. Ideally you’d enable 74–4 on all systems in the sysplex.[2]

I say “at least” as modern CFs tend to spill to a second and, occasionally, a third. So here’s another thing I’ve absorbed over the last few customer engagements: Customers nowadays have hundreds of structures in each coupling facility.[3]

Local Facility Data Section

You get one of these in the 74–4 record – whether the base or the overflow ones. It has to be there as it has the Coupling Facility name, as well as the machine it’s on.

It also has some relevant things which I’ll touch on now, and come back to later:

• Path masks
• Channel Path Acronyms
• Channel Path IDs (CHPIDs)
• Index into Path Data Sections and count of the number of sections

As examples of Channel Path Acronyms, I’ll take three:

• CIB – Infiniband
• CS5 – ICA-SR
• CL5 – Coupling Express Long Reach

Remote Facility Data Section

You get one or more of these, depending on how many CFs the one in the Local Facility Data Section has duplexing links to. The path-related information in this section is:

• Count of the number of paths
• Channel Path Acronyms
• CHPIDs
• Index into Path Data Sections and count of the number of sections

Pretty similar to Local Facility Data Section information. But not quite. The information is only for online paths.

Path Data Section

I’ve written about this section many times, most recently in And Latency Once More.

You get one section for each link under certain circumstances: The link has to be physical (no IC links) and modern enough (not ICB, for example). Unfortunately, it has to be online. So this section doesn’t help us in our quest for offline links.

And the reason I keep writing about this section is because it is full of value and surprises. But not this time. 😦

Actually There’s A Fourth

I include this for completeness[4]. SMF 73 gives information on CHPIDs – from a z/OS system.

For CF links this information is very basic. Mainly the CHPID, whether it’s shared, whether it’s online, and Channel Path Acronym. [5] For other types of channel, such as FICON, we get a lot more, useful, information.

There is a potential use for this data, quite apart from corroboration: If a CF-related channel path appears in SMF 73 but not in 74–4 it implies it’s not connected. Of course it would be strange if it managed to be online despite that. 🙂 I should probably code to this possibility.

The Truth Is Masked

In the Local Facility Data Section piece I mentioned path masks. It might not have seemed obvious but they are key to this. There are three 8–bit mask fields:

• Path Installed Mask
• Path Available Mask
• Patch Composite Mask

Each bit represents a different path and each path is described by the same bit position in all the masks.

These terms might be familiar in the context of disk controllers.

Consider the following worked example:

This is from a real customer. Reading from top to bottom.

1. The channel path acronyms speak of 4 CS5 (ICA-SR) paths and 4 CIB (Infiniband) paths. Note the order.
2. The Path Installed Mask (Hex FF) has all the bits turned on – so all 8 paths are installed.[6]
3. The CHPIDs are listed. The order here is again important.
4. The Path Available Mask (Hex B1) has some bits on and some off – 10110001 being the bit string. So only some of these paths are available, the rest being installed (in this case) but offline.
5. The Path Composite Mask (also Hex B1) reflects both the Installed and Available Masks.

Taken together, we conclude:

• All 4 Infiniband paths are installed but offline – CHPIDs 0E, 6A, 6F, and 78.
• All 4 ICA-SR paths are installed and online – CHPIDs 04, 24, 25, and C6.

Conclusion

For links between z/OS LPARs and CF LPARs we can see both the online and offline paths. For the CF to CF links we can only see the online ones.

There is an old joke. It goes like this, updated a little: “I wrote a long blog post as I didn’t have time to write a short one”. Well, I did have time to write a short one… 🙂

By the way “Maskerade” is the name of a Terry Pratchett Discworld book, in case you wondered if there was a cultural reference. High time I read it again.