(Originally posted 2015-05-09.)
Just a brief one this time. A customer recently asked me how to detect Sysplexes sharing Infiniband links. [1]
It arose when discussing the information in the new(ish) Channel Path Data Section in RMF’s SMF 74 Subtype 4 record.
The question really boils down to “how do I detect in RMF / SMF different Sysplexes using the same identifiable link”? [2]
My first suggestion was the PCHID field R744HPCP (which we’ll return to presently). Seemed reasonable to me – as it had the word “Physical” in it. What could be more permanent and definitive? 🙂
Other fields in the section with “physicality” are Host Channel Adapter ID (R744HAID) and Host Channel Adapter Port ID (R744HAPN).
My friend Erik Bakker pointed out to me that Infiniband links don’t have PCHIDs but rather Adapter IDs and Port Numbers. If you read no further then the “take home” is to use Adapter ID and Port Number as the link identifiers.
But one mystery remained: Why am I seeing valid-looking [3] numbers in the “PCHID” field?
So I spoke to Dave Surman, who’s generally very helpful in these matters. He said “For coupling links that don’t have a PCHID (namely Infiniband links), the CHSC returns something called a VCHID in the PCHID location. It’s a virtual CHID representation used by the firmware, with no physical correlation.”
CHSC is, of course, the Channel Subsystem Call machine instruction.
Now of course I would’ve known all this if I’d read the following Redbook: Implementing and Managing InfiniBand Coupling Links on IBM System z. Well Erik probably had but, being a Performance guy I unfortunately hadn’t. Section 2.4.2 talks all about VCHIDs.
I was seeing values of hex “07xx” for VCHID in R744HPCP, by the way.
So if there is a moral of the story it’s “you can never get too close to the infrastructure you’re reporting on and trying to tune”. And that’s what a large chunk of this whole blog [4] has always been about.
In case my whole credibility on Coupling Facility links hasn’t been blown away 🙂 you might like these other posts of mine:
- What’s The Latency, Kenneth?
- The Effect Of CF Structure Distance
- The Missing Link?
- Coupling Facility Topology Information – A Continuing Journey
- System zEC12 CFLEVEL 18 RMF Instrumentation Improvements
-
The motivation for this is less about Performance and more about documentation, verification, detecting change, trouble-shooting, and “separation of concerns”. ↩
-
SMF being, of course, the first port of call for any self-respecting Performance person. ↩
-
Who knows what “valid looking” means? 🙂 ↩
-
I hate it when people say “blog” when they mean “post” (noun, not verb). In this case I definitely do mean the whole shooting match, not just a single post. ↩
Mainframe, Performance, Topics 
One thought on “Sysplexes Sharing Links”