Remember Memory

In Engineering – Part 8 – Remote Access Detection I talked about reasons why LPARs might go cross-drawer.

One reason I didn’t give was about memory.

(I should perhaps remind you that on modern machines a logical processor accessing Level 4 Cache or memory in another processor drawer incurs a significant performance penalty, showing up as an increase in Cycles Per Instruction (or CPI) in SMF 113. Engineering – Part 8 – Remote Access Detection talks about techniques to look at cross-drawer.)

I was reminded recently of something I already knew but hadn’t seen much incidence of: LPAR memory configuration leading to LPARs split across drawers.

An Example

Here is a theoretical example that illustrates the point:

A 2-drawer machine has 2TB of memory. (Actually this is a little modest these days.) There are three LPARs:

• LPAR A with 800GB of memory and fewer than half the number of online logical cores that would fit in a drawer.
• LPAR B with 400GB of memory and even fewer logical cores than LPAR A.
• LPAR C with 700GB of memory and again fewer logical cores than half a drawer’s worth.

(When counting cores you need to add together all the cores in all the pools. So, for z/OS that would be both GCPs and zIIPs.)

Ignore the cores, except to note that any pair of LPARs taken together has fewer cores than would fit in a drawer.

With two drawers and three LPARs PR/SM has an interesting challenge: There are three possible pairs of LPARs to try to fit into a drawer. (More than one LPAR In a drawer is not a problem, in principle.) The three combinations are:

• A + B: 1200GB
• A + C: 1500GB
• B + C: 1100GB

In fact all three LPARs add up to 1900GB, so there is enough memory on the machine to fit all three in the two drawers.

It is obvious that some LPAR will have to have memory allocated in both drawers.

PR/SM Decision Making

In the z17 Technical Guide I clarified the following thing: When allocating machine resources – memory and cores – PR/SM allocates them to the LPAR with the highest entitlement first, then the next highest, and so on. So far no clarification. The clarification is what the term entitlement means:

• For dedicated core LPARs think of it as the number of cores (in all pools).
• For shared core LPARs think of it as the number of cores’ worth of weight ( in all pools).

A general piece of advice is not to try too hard to second guess what PR/SM would do – but to aim for sensible LPAR design.

The example in this post does not represent the best possible design, though it might be necessary.

How Could This Situation Be Improved?

There are two main approaches, and each has its own downside:

1. Buy more memory. That obviously costs money – so maybe not popular. But when buying a new machine this sort of thing should be taken into account.
2. Reduce the memory footprint of the LPARs – but that has performance implications.

When I look at LPAR memory I generally see a lot spare – but not always. This can be established with SMF 71. But it is important to consider things like recovery scenarios: That apparently spare memory might be there to allow, for example, for a Db2 subsystem to be recovered, along with maybe some CICS regions. This is one reason I take an interest in Architecture and what uses installations put their hardware and software to.

Conclusion

LPAR Design is an interesting topic – but it can’t be treated as an exact science.

I would also note in closing two things:

• You can’t from RMF tell how much memory is purchased in each drawer. Vital Product Data (VPD) is something a Customer Engineer has access to – and it does document this.
• If an LPAR moves drawer it can take some time to move. The larger the LPAR’s memory (including Virtual Flash) the longer the more it can take – with a potential for cross-drawer memory access.

Perhaps you shouldn’t just assign all purchased memory to an LPAR.

Making Of

I’m writing this on a plane journey home, having spent the week with two rather sophisticated customers. Some of what I’ve written is inspired by them, some not. Writing, to me, is an assimilation of what I’ve experienced and thought about – often over many months. These two might just recognise themselves in bits of this but nobody else would. And the example is a simplified version of something real.

That last sentence was hard fought as I experimented with picking the iPad up and writing on it directly with the Apple Pencil. That didn’t work very well. In general, though, writing with the Pencil is getting a little easier.

And something is causing suggested text replacements to be better – and the acronym “LPAR” to be better recognised. It’s probably AI as I think it’s hallucinating words into existence. 😀