Four Coupling Facilities

This isn’t following on from Three Billboards? 🙂 but rather Shared Coupling Facility CPU And DYNDISP from 2016. I’m not sure it adds much that’s new but this set of customer data was an opportunity too good to miss…

… It enables me to graph most of the Coupling Facility LPAR types you’re likely to see.

I won’t repeat the contents of the 2016 post but I will repeat one thing: There are two views of Coupling Facility CPU:

• SMF 70 Partition
• SMF 74–4 Coupling Facility

That 2016 post talked at length about the latter. This post is more about the former.

Different Coupling Facility LPAR Types

There are different kinds of coupling facility LPAR, and this customer has several of them:

• Dedicated
• Shared – without Dynamic Dispatch (DYNDISP=NO)
• Shared – with Dynamic Dispatch (DYNDISP=YES)
• Shared – with DYNDISP=THIN

The latter two are similar but, in essence, Thin Interrupts (DYNDISP=THIN) shortens the time a CF spends polling for work, releasing the physical CPU sooner. This is good for other ICF LPARs, but maybe not so good for the LPAR with DYNDISP=THIN.

While this customer’s data doesn’t exemplify all four types it is a useful set of data for illustrating some dynamics.

About This Customer’s Mainframe Estate

I’m only going to describe the relevant bits of the customer’s mainframe estate – and I’m going to remove the names.

There are four machines, each running a mixture of LPARs in sysplexes and monoplexes. The sysplex we were most interested in had four LPARs, one on each machine. Also four coupling facilities, again one on each machine. There were no external coupling facilities.

Those of you who know a bit about resilience are probably wondering about duplexing coupling facility structures but this post isn’t about that.

I don’t think it makes any difference but these are a mix of z13 and z14 machines.

We had SMF 70-1 and SMF 74-4 from these z/OS LPARs and the four coupling facilities, but little from the others.

Here are the four machines’ ICF processor pools, across the course of a day.

The top two look significantly different to the bottom two, don’t they?

Machines A And B

These two machines have multiple ICF LPARs, each with some kind of DYNDISP turned on. We can see that because they don’t use the whole of their PR/SM shares – as their utilisation from the PR/SM point of view is varying.

Each machine has two shared ICF processors.

We have SMF 74-4 for the blue LPARs. So we can see they are using DYNDISP=THIN. We can’t see this for the other LPARs as we don’t have SMF 74-4 for them. (SMF 70-1 doesn’t have a DYNDISP indicator.)

The blue LPARs are also much busier than the other LPARs in their pool. While one might consider dedicating one of the two ICF processors we wouldn’t ideally define an ICF LPAR with a single logical processor.

Machine C

Machine C looks different, doesn’t it?

Again we have 2 physical processors in the ICF pool.

Here we have four ICF LPARs, each with DYNDISP=NO. We know three facts to establish this:

• From SMF 70-1 we know that none of the LPARs has any dedicated ICF engines. We know this two ways:
  • As this graph shows, none of them has the share of a single ICF engine.
  • We have Dedicated and Shared processor information explicitly in SMF 70-1.
• These LPARs use their whole share – judging by the constant CPU use in SMF 70-1.
• From SMF 74-4 we know the blue LPAR in particular has DYNDISP=NO. (We don’t have SMF 74-4 for the others.)

Machine D

This looks similar to Machine C but it isn’t quite.

Yet again the ICF pool has 2 processors.

• The Red LPAR has dedicated processors – from both SMF 70-1 and SMF 74-4. DYNDISP doesn’t even come into it for this LPAR.
• The other LPARs have DYNDISP=NO, judging by their (SMF 70-1) behaviour.

A minor footnote: As I articulated in A Picture Of Dedication (in 2015) I sort the LPARs so the Dedicated ones appear at the bottom of the stack up. (Even below *PHYSICAL – which is a very thin blue veneer here but actually red in the case of the other three machines.)

But Wait, There’s More

When I started writing this post I thought it was just going to be about showing you four pretty pictures. But, partially because some blog posts get written over an extended period, something came up meanwhile that I think is worth sharing with you. Besides, electronic words are unlimited – even if your patience isn’t.

Having shown you some graphs that depict most of the ICF LPAR CPU situations I experimented with DYNDISP detection for ICF LPARs we don’t have SMF 74-4 for.

Got right, this could make the story of an ICF pool with shared logical engines much nearer to complete.

The current algorithm – now in our Production code – assumes DYNDISP if the ICF LPAR uses less than 95% of its share over the focus shift (set of hours). Otherwise it’s Dedicated or DYNDISP=NO. I still can’t tell whether it’s DYNDISP=THIN or YES without SMF 74-4.

While ideally I still want data from all z/OS LPARs and coupling facilities in a customer’s estate, this technique fills in some gaps quite nicely.

Well, it works for this customer, anyway… 🙂