Mainframe Performance Topics Podcast Episode 27 “And Another Thing”

So this is one of our longest episodes yet, but jam full of content which is very us. As usual, finding times when we can both be available to record was tough. What wasn’t difficult was finding material. I can say for myself what we talked about is a set of things I’ve wanted to talk about for a long time.

Anyhow, enjoy! And do keep feedback and “Ask MPT” questions coming.

Episode 27 “And Another Thing” Show Notes – The Full, Unexpurgated, Version

Follow up

  • Additional System Recovery Boost enhancements:

    • Sysplex partitioning recovery: Boosts all surviving systems in the sysplex as they recover and takes on additional workload following the planned or unplanned removal of a system from the sysplex.

    • CF structure recovery: Boosts all systems participating in CF structure recovery processing, including rebuild, duplexing failover, and reduplexing.

    • CF data-sharing member recovery: Boosts all systems participating in recovery following the termination of a CF locking data-sharing member

    • HyperSwap recovery: Boosts all systems participating in a HyperSwap recovery process to recover from the failure of a storage subsystem.

    • Existing functions: image-related boosts for IPL (60 min) and shutdown (30 min)

    • These are different in that they boost multiple systems, rather than the single one that the originally announced SRB would boost. These should be fairly rare occurrences – but really helpful when needed.

  • Two more portable software instances: Db2 and IMS added to Shopz on Aug 20, 2020, in addition to CICS (Dec 2019).


Martin was asked about which systems in a System Recovery Boost Speed Boost situation get their GCPs sped up to full speed. The answer is it’s only the LPARs participating in a boost period that get their GCPs sped up. For example, on a 6xx model the other LPARs don’t get sped up to 7xx speed.

Mainframe – Trying out Ansible for driving z/OSMF Workflows

  • Marna’s been learning about Ansible and how it can drive z/OSMF work. So far with z/OSMF Workflows, so terminology might not be exactly right from inexperience.

  • Found a driving system for Ansible system (ubuntu distribution Linux running on x86).

  • A lot of installs on this Linux distribution was necessary: python 3.7, ansible, ansible-galaxy collection for ibm.ibm_zos_zosmf .

  • Ansible galaxy is a public collection of modules. Run those modules from a playbook . While steps are called roles, like jobs with steps. Ansible has some sophistication where those roles can be run by different people.

  • Had to learn a little yaml, and understand the sample playbook which came with the collection, changing the playbook for my specific system.

    • This is where helped was necessary from two helpful gracious Poughkeepsie system testers (massive thanks to Marcos Barbieri and Daniel Gisolfi !!)

    • Learning about an Ansible with a staging inventory (a configuration file), andn learning which playbook messages were ok or not ok.

  • Encountered two kinds of problems:

    • Security issues connecting the Linux environment to a large Poughkeepsie System Test z/OSMF environment

      • Required changes to the playbook, and to environment files.
    • Workflow-related: duplicate instances when part of the playbook ran ok, and selecting an automated workflow.

  • Why learn Ansible when you are a z/OS person? Ansible gives us a very nice interface to get to z/OS resources via z/OSMF capabilities with a simple programming interface.

    • Also, if we want to get more people to use z/OS, and they are familiar with Linux, they probably will want to drive work to z/OS with some sort of familiar automation

    • Linux installation of products from a command line, and having to keep looking up command syntax isn’t that fun. Although pretty easy find with google.

    • The products all installed quickly and cleanly, however knowing that the dependencies were right was not obvious. Especially the python and Ansible levels.

  • Ansible Tower as the GUI is helpful, but Marna chose to learn Ansible in pieces from the ground up.

  • As it seems to always be: getting your job done comes down to what your search returns, no matter what you are doing – z/OS or Linux work. Or even programming.

Performance – So You Don’t Think You’re An Architect?

  • Brand new presentation for 2020, builds on the “who says what SMF is for?” slide in “How To Be A Better Performance Specialist”

    • Usually kick off an engagement with an Architecture discussion, which might be hours long!

    • Builds common understanding and trust

    • Techniques in this presentation well honed

  • Presentation talks about 3 layers

    • The Top Layer – Machines, Sysplexes And LPARs

      • For example a diagram showing all the LPARs and their definitions – by pool – for a machine. Together with their sysplexes.

        • Important to answer “what is the role of this LPAR?”

        • Important to understand how far apart machines are

        • Inactive LPARs often give availability clues

        • Driven by RMF CPU and Sysplex SMF

    • The Middle Layer – Workloads And Middleware

      • Spikes: e.g. HSM Daily spike, and actually patterns of utilisation in general

      • Restarts: IPLs, Db2, IMS, CICS Regions

      • Cloning: Things like identical CICS regions

      • Topology: e.g. for CICS what Db2 or MQ a region connects to, from SMF 30 Usage Data Section. Invert that and you have the Db2’s role and what connects to this Db2. Enclave statistics give DDF traffic, Db2 Datasharing group structure

      • Software Levels: From SMF 30 for CICS, Db2, MQ in Usage Data Section, often see IMS in transition, and limitations with Db2 12 onwards – function Levels eg M507 vs M500.

        • All this from SMF 30 and 74-2 XCF, run through Martin’s own tooling. The point of mentioning it is to get the ideas out.

        • MXG discussion ongoing – especially about SMF 30 Usage Data Section. Believe the issues have been ironed out.

    • The Bottom Layer – Application Components

      • Things like Db2 and MQ Accounting Trace: Give names in great detail, likewise CICS Monitor Trace (but these are very heavy but very valuable)

      • DDF topology can be handy

      • Data sets – from SMF 42-6, 14, 15, 16, 62, 64

    • Fulfilling the definiton Of Architecture?

      • Understand what’s really going on

      • In a way that’s especially valuable to real systems architects

  • Customers should consider this approach because you can keep an up to date view of the estate and inform your own capacity and performance role.

  • Martin will keep exploring this rich avenue because dabbling in architecture keeps him interested, and lets him find patterns among customers.

Topics – Notifications

  • They are unsolicited messages appearing on the device, generally interruptions whether wanted or not.

  • Can contain text and images, so really just messages. Taking a platform-neutral view follows.

  • Where do they come from?

    • Form all sorts of services, many on the web

    • Need polling applications on the device

    • Can drain the battery

  • What do you receive and where?

    • Some on Watch: For example from my bank immediately from a contactless payment, and from a washing machine.

    • Most on phone: Slack notifications, from when Google home wifi is out. Sometimes inappropriate, like at Carnegie Hall.

    • A few on iPad: For example WordPress notifications

    • Very few on Mac

    • Very few on web browser: Only when someone replies on favourite newsgroups

    • A key way of managing them is trying not to duplicate across multiple devices

    • IFTTT still a favorite of Marna’s, mentioned in Episode 23 “The Preview That We Do” Topics topic. Martin just subscribed to Premium, but at a low-ball price.

    • Constant interruptions not always welcome, especially those with sounds.

  • How do you manage them?

    • Try to reduce their number. Often have “Wood for the trees” problem.

    • On iOS at least apps can stack/group them

    • Many are just plain spammy

      • Many are defaulted to on when you install an app
    • Figure out which devices should get what and watch for “must always receive immediately”

    • Only the place they make sense, such as exercise chivvies, letting each app has its own settings

  • An advanced topic is how you can generate your own

    • IFTTT from them service (emails, messages), from the app (weather tomorrow).

    • On iOS the Pushcut app: a webhook from eg IFTTT can cause a notification

      • Pushcut webhook support includes the ability to trigger actions on the iOS device, using cURL to invoke webhooks
    • On Android we have MacroDroid.

    • If you have an Amazon Echo or related device you can use web hooks with the Notify Me skill to send notifications to the Echo

  • Lots of scope for things to go wrong with automation based on notifications

Customer requirements

  • Customer Requirement 138729 : z/OS Console message help support JES2 messages

    • Use case: Hover over a JES2 messages or other prefixed message to get the messages help.

    • IBM response: This RFE has been delivered via APAR PH24072 at end of June 2020. It’s available on both V2R3 and V2R4. The JES2 message prefix such as $ can be setup in “Configure Message Help” dialog in z/OSMF Console UI.

    • “Configure Message Help”, there is a location for your KC4Z hostname so you can can retrieve message help from the Knowledge Center when you hover over message ID.

    • Note might be usable in principle be used by other subsystems, but it was raised as a z/OSMF requirement for JES2.

On the blog

Published by Martin Packer

I'm a mainframe performance guy and have been for the past 35 years. But I play with lots of other technologies as well.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: