Born With A Measuring Spoon In Its Mouth

(Originally posted 2016-06-05.)

SMT really was born with a measuring spoon in its mouth.[1]

Let me rewind a few years…

So, when SMT (Simultaneous Multithreading) was being designed I was privileged to be on the periphery of discussions about how to instrument SMT. Things like CPU Utilisation get a little wierd in the SMT case, as you can imagine.

Now, I was only on the periphery of the discussions and they carried on without me in the run up to the announcement of z13 in early 2015. But the drift I caught was that the hardware was going to have to help out, essentially being self-metering.

Fast forward to now, just over a year after we first shipped z13. And now I come to warm over our CPU code to account for SMT.

Timely, huh? 🙂

Seriously, from where I sit I have to see real data from real customers before I can do serious development.[2]

Now, in mid–2016, there are lots of z13 customers. So it’s time to act.

Remember that SMT “only” affects zIIPs and IFLs. GCPs and ICF engines are not affected. So everything already works fine for GCPs. But obviously hiding behind the fact it’s only certain types of engines that support SMT is not a good thing to do.

Rewind again, but this time to the Autumn of 2015. I had the privilege of presenting a one day workshop on Performance for the ITSO in Europe.

A smallish section of this was about SMT, and an even smaller portion of it was about the measurements. So what I really wanted to impart with that material was the general sweep of the instrumentation; That some stuff was on a per-core basis and some on a per-logical-processor basis.

Now a core is the thing that can have multiple threads and it’s basically all PR/SM knows about. It’s z/OS that knows about logical processors or threads.

Here’s an example[3] that might explain the relationship between logical processors and logical cores:

In this example MVSA has 5 logical cores.

• Logical cores 0,1, and 2 have a single thread and are GCP cores.
• Logical cores 3 and 4 have two threads and are zIIP cores.

So SMT–2 clearly affects zIIPs and not GCPs, as the diagram shows.

Obviously logical cores get dispatched on physical cores by PR/SM.

In the workshop I showed (briefly) sample RMF reports – for PROCVIEW CPU (non-SMT) and PROCVIEW CORE (SMT 1 and SMT 2). By the way the support in RMF came with OA44101.

Back To The Present

Returning to the present moment I want to replicate that, and then put my own personal twist on it.

(Generally that’s the way to go: Replicate RMF and then progress beyond the product’s reporting.)

So, for most major numbers in an RMF report you have to derive them. The nice surprise with the SMT support is that the numbers are basically there. By “basically” I mean the worst you have to do is divide by 1024.[4]

So, for example the following fields are all there in the SMF 70–1 record (in the sole CPU Control Section).

• Maximum Capacity Factor
• Capacity Factor
• Average Thread Density

You get one each for GCPs, zIIPs and (gasp!) zAAPs.

I mention these by name in the hope the astute reader will recognise them as terms used in most performance materials related to SMT.

But the point is that no fancy derivation is necessary.

What Else Is New And Changed In SMF 70–1 For SMT

First the CPU Data Section (previously one per logical processor) is at the thread level, not the core level. (In fact, thread is synonymous with logical processor.)

So this now needs relating to the core. Here’s where the next change comes in: The new Logical Core Data Section.[5]

This has a number of aspects:

• It allows you to relate the logical processor / thread to the core.
• You get the Core Productivity number.
• You get the Core LPAR Busy time.

A question you’d probably like to be able to answer is “which LPARs on this machine have PROCVIEW CORE in effect?” The answer to this is found in the PR/SM Partition Data Section (one per LPAR): If field SMF70MTID is greater than 0 PROCVIEW CORE is in effect; Otherwise it’s PROCVIEW=CPU.

Finally, in the PR/SM Logical Processor Data Section (one per logical core) SMF70MTIT gives you the “Multithreading Idle Time in microseconds accumulated for all threads of a dispatched core. This field is only valid if SMF70MTID is not zero for this partition.”[6]

A Lot To Chew On

If you’d come to the conclusion there’s a lot to chew on here you’d be right. But at least we’re being spoon-fed.

To continue the malaphor, I’m still digesting this; The definitions are a little hazy in my brain, but at least I have a way of seeing how the data behaves in real customers.

And I have some thoughts on how to diagram things (as the picture above illustrates) and otherwise tell the story. More on this as I implement in my code; For now things are going to have to be hand-drawn.

Happy chomping!

And here’s a nice presentation on SMT to digest: IBM z Systems z13 Simultaneous Multi-Threading (R)Evolution by Daniel Rosa, IBM Poughkeepsie.