(Originally posted 2011-07-07.)

It’s a popular suggestion that what separates the truly exceptional person from the rest of us is 10,000 hours of "practice". In book form I’ve seen it twice – in Matthew Syed’s "Bounce" and in Malcolm Gladwell’s "Outliers". Actually, to be fair, Matthew acknowledges his original source so that’s actually only one distinct source. (Life Lesson aside: Trace ideas and "facts" back to see if they came from one place or are truly corroborated.)

The suggestion is that there’s no such thing as innate talent and that all that matters is practice – 10,000 hours of it. And this is often repeated now in public folklore.

I find this assertion problematic for a number of reasons – though I’m prepared to admit I could be wrong.

For a start I find it very hard to believe we all respond the same way to our experiences, that we have the same physiology and brain "wiring".

Secondly – and this is where the title of the post comes in – 10,000 hours of what? Actually the suggestion is that it is useful practice. The issue for me is that the usefulness of the practice is highly variable – depending on who we are, how motivated we are and how tired we are (and maybe many other variables besides).

Thirdly, there is what I call the categorisation problem: Take the example of someone who is a generalist in, say, Information Technology. They could easily gain 10,000 hours of experience, perhaps good experience, spread across their whole domain. Does this make them an expert? If they spent the whole 10,000 hours in a narrower area what kind of an expert does that make them? And if they spent two chunks of 5,000 hours in two areas are they not an expert in either? What if those two areas were abutting? What if they weren’t?

That previous paragraph had a lot of questions in it. Some are easy to answer, some less so. Maybe – and here I hope is the relevance of this whole "10,000 hours" idea – these sorts of questions allow one to "evaluate" one’s career. I put "evaluate" in quotes because I don’t mean this as a scorecard: There are very many valid paths through life. But, for example, discovering your 10,000 hours have been spent scattered across a wide range of topics might tell you you’re a generalist. But then you probably knew that. 🙂

A more difficult case is what to do when you discover you’ve spent 10,000 hours in the same area. With a low boredom threshold like mine there’s a premium on convincing yourself there’s been some diversity. 🙂 But has there really? Or the converse: 10,000 hours concentrated in one area, but is it really one area?

In my case – and this post isn’t really about me – I’ve convinced myself 🙂 of three things: That there’s plenty of variety, that the technology keeps evolving at a dizzying pace, and that my role has in any case morphed over time. Actually, I think a lot of us feel that way.  

But does all this change keep resetting the counter to zero hours? I’d maintain it didn’t: The way I learn and (I think) the way I incorporate situations into my experience base is accretive (adding on around the edges of what I already know).  So I don’t really think the counter ever resets: We just start new counters occasionally. But I wouldn’t count some of the new technology I dabble in as "start at zero" – despite how much it sounds like babbling. 🙂

I enjoy being in the babble phase. But, like most people, I worry about the quality of the work I produce in that phase. I particularly enjoy when I spot the experience beginning to build. That "10,000 hour"idea just might be motivational. And now for a gratuitous Metallica lyric. 🙂

"Trust I seek and I find in you
Every day for us something new
Open mind for a different view
and nothing else matters"

In the spirit of "10,000 hours" don’t you think that’s a great verse? Anyhow, keep logging those hours. Who’s to say "it’s all been a waste of time"? 🙂

| Minor edit 27 October to fix "whose" that should’ve been "who’s". Dunno how that crept in there. 😦

(Originally posted 2011-07-05.)

As you probably know Hardware Data Compression has been supported by MVS and IBM mainframes for around 20 years. In several recent batch studies I’ve conducted it’s been evident in a widespread way.

(In this post I’m not talking about DB2 compression of either flavour or VSAM compression – though some of the information here applies to these functions as well.)

It’s not as simple a deployment strategy as "turn it on for everything" though I think "turn it on for all sequential data sets large enough to stripe" may well be the flavour. There’s a certain logic to this: If striping can make sequential data access go faster then compressing the data will make it go even fasterer.

There’s a lot of truth in that. But consider one thing: Compression and (decompression) takes CPU cycles. In most software licencing schemes that translates into an increased software bill. For small-scale use that might not matter much. But used wholesale it could make a significant difference.

There’s another thing: If you cause a job that’s running in a CPU-constrained environment to burn more CPU through Compression the speed up may be disappointing. I’m not saying it’s worse than useless – I’ve no evidence of that – but it is a consideration.

My suspicion that the CPU cost can be quite high comes from seeing "low CPU" cases like DFSORT COPY operations and other simple reformatters burning far more CPU than I would have expected. (Quite often this includes a significant chunk of SRB time but not always. This is consistent with the way DFSMS/MVS does (de)compression.) There is no direct metric for the CPU cost of Compression: You have to use the overall CPU consumption and draw your own conclusions.

It sounds like I’m against using Hardware Data Compression. Actually I’m not: I’ve seen evidence of very good compression ratios (for example in SMF 14 and 15 records for non-VSAM data sets). And I know that in many of the cases I’ve seen the alternative might have been to write the data to tape, with all the handling that entails. Which brings us on to something else:

Quite a few of the scenarios I’ve seen have been one-writer-one-reader cases. You tend to think "Pipes" in those cases. You were expecting that, weren’t you. 🙂 but, seriously, the advantages are fairly obvious. One of the objections to Pipes has always been that it takes more CPU than writing data to disk or tape. But if you compress it* it’s not nearly so obvious that’s the case. It would be interesting to conduct an experiment.

This is pretty much an "it depends" situation. Aren’t they all? Actually, there’s quite possibly more speed advantage in getting the buffering and number of stripes right – as described in SG24-2557 Parallel Sysplex Batch Performance. (This book has lots of good advice on access method tuning – written by my residency teammates in 1995.)

In short, if you’re implementing Hardware Data Compression measure the impact and effectiveness and be open to the idea there may be other ways of achieving the speed up.

Finally, I said this wasn’t VSAM but note that SMF 64 also has a Compressed Data Statistics Section, just like SMF 14 and 15. My expectation is, however, that the access pattern to compressed VSAM is not sequential. (For QSAM and BSAM I expect it to be largely sequential.)

---

* Tape compression is a different matter – with the hard work being done in the control unit.

(Originally posted 2011-07-02.)

Following on from The Best Sort Is The One You Don’t Do here’s another reason for eliminating sorts. I think it’s worth a post in its own right.

(In this post, again, I’m talking about resequencing passes over data – not copying or merging.)

With a sort it’s possible the last record read in might be the first record written out. So you can never overlap input and output phases. (There might even be a phase between the end of the input phase and the beginning of the output phase – particularly if there are intermediate merges.)

A common pattern in a job is a processing step, then a sort step, then another processing step, and so on. Often the first processing step writes a data set the sort reads and the second processing step reads the sorted version of that. (It’s a separate question whether either processing step could have been performed by the sort, of course.) Let’s call the first processing step "W", the sort "S" and the second processing step "R".

Now consider what happens with BatchPipes/MVS. With Pipes you overlap the reader and the writer. That’s part of the benefit (along with I/O time reductions and, perhaps, the elimination of tape mounts). In the above scenario you can overlap W with the input phase of S – with a pipe. Likewise you can overlap the output phase of S with R. What you can’t do is overlap W with R – because you can’t overlap the two phases of S.

A pity but perhaps not a huge one. Let’s illustrate this with some numbers. Suppose:

• W runs for 10 minutes, writing all the while.
• S has an input phase of 5 minutes and an output phase of 5 minutes.
• R runs for 10 minutes, reading all the while.

The three steps between them take 10 + 5 + 5 +10 = 30 minutes.

Overlapping W with S’ input phase saves 5 minutes (as S’ input phase gets stretched to 10 minutes). Similarly overlapping S’ output phase with R saves 5 minutes. So we save 10 minutes overall.* We like to say "the sort gets done for free".

If we can eliminate the sort completely we can overlap S and W completely with a total saving of 20 minutes – 10 for the overlap and 10 for the sort removal. (Without Pipes we’d still see a 10 minute reduction just by removing the sort.)

So, this is another case where removing a sort is a valuable thing to achieve. But as I said in the referenced post that’s easier said than done.

---

* In this calculation you’ll notice I’ve assumed the I/O time reduction is zero and that there are no tape mounts eliminated. Generally there is some I/O reduction, of course. But it’s still a fair comparison. It’s just the benefits of Pipes are understated.

(Originally posted 2011-07-02.)

Have you ever had the suspicion a sort was unnecessary in your batch? I bet you have.

In recent Batch Performance studies I’ve had the suspicion that many of the sorts are unnecessary: Either they should be merges or not done at all. But how do you prove it?
 

But first, what do I mean by a sort not needing to be done at all?

Clearly if the data is reformatted then something has to be done to it – just maybe not a sort. It could be a copy or maybe a merge. So I’m really talking about the need to reorder the records. Reordering them is more expensive in terms of disk space (for sort work data sets), CPU and run time. So a sort is best avoided.

So how do you figure out if a sort needs doing? One way is to remove the sort and see what happens. Probably best not done in Production. 🙂 That’s a form of "destructive testing". But there is another way: Essentially running the test I’m about to outline in the time frame of the sort.
 

In a moment "choreography" but first the mechanics:

Testing If Data Is Already Sorted

For a DFSORT MERGE operation to be successful all the data sets being merged must already be sorted on the merge key(s). Otherwise you get a return code of 16 and a ICE068A message such as:

ICE068A 0 OUT OF SEQUENCE SORTIN01

We can use this to our advantage by attempting to merge the supposedly-already-sorted data set with a dummy file. (DD DUMMY will do just fine.) If the data set is already sorted the step will complete with a 0 return code. Otherwise, as I say, it will be RC=16.
 

It’s worth noting that the merge will fail immediately it detects a record out of sequence. This means a badly unsorted data set will fail fast. However a largely well sorted data set may not fail the test until it has been almost completely read: Perhaps the last record is the only one out of sequence.

How To Incorporate This Test

If the input data set is persistent you could either add the test step in just before the sort or at the end of the job. If it’s transient (perhaps temporary) it has to be tested before it goes away. In short you run the test when it’s still there but not otherwise updated.

One important aspect is the intrusiveness of the test: Breaking a BatchPipes/MVS pipe to do it is a particularly bad idea. Holding up processing for tape mounts is probably bad. And maybe running the test alongside the actual sort slows it down.

If you were tuning your batch window and suspected a sort was unnecessary you might allow the test to run a few dozen times to gain confidence that it’s not needed. But you might never be sufficiently confident.

One other thing: You might decide to always run the test and run the sort only if the test step ends with RC=16. But then a downstream step would have to access the right data set – presumably the SORTOUT from the test step or the SORTOUT from the sort, depending.

Concluding Thoughts

 

At the end of the day it’s better to know your batch well enough to avoid having to rely on tests like this. But in reality people don’t understand their batch that well, in my experience. Not a criticism, just an observation. And a situation caused by the complexity and longevity of the typical batchscape.

Maybe this technique is best used during development or in a test environment. As I said on Twitter yesterday I was thinking about ways of "injecting dye into the water" in a data flow sort of way. Maybe I’ll think of some additional "dye test" or "smoke test" (if you prefer) techniques. One other wrinkle might be to inject record sequence numbers into the data to see whether they’re preserved.

(Originally posted 2011-06-29.)

This may be stating the obvious – but I wonder to whom it actually is obvious…

I’ve been doing quite a lot of work with batch job timings and CPU recently. (Everything I’m about to say is equally true of steps.) It’s interesting to think about the effects of faster engines versus more engines (a question I haven’t been asked recently) and whether a customer needs more capacity or just faster engines (a question that has come into play).
 
There’s nothing very new about this but it is worth thinking about. And in particular what we can glean from SMF 30 Step- and Job-end records…
 
We know lots of things about a job’s timing and related stuff, most notably:

• When it started and ended, and hence the elapsed time.
• Where it ran.
  
• When it was read in and initiator delays.
• How much CPU it used – whether TCB or SRB.
• Whether it used a zIIP or zAAP or was eligible to but didn’t.
• How many disk or tape EXCPs it did (and how many tape mounts).
• Step condition codes.
  

But one thing we can’t know is whether a job met significant CPU queuing or not. At least not from Type 30. We might be able to infer something from DB2 Accounting Trace (SMF 101) but (to rehearse previous arguments):

• We can’t know if the Unaccounted For time (the most likely bucket) really is queuing time. (Generally it is but there are lots of edge cases where it’s something else.)
• We can encounter difficulties in tying up the 101 with the 30 – particularly for IMS.
• We have to take into account CP Query Parallelism.
• The job might not be DB2 at all.

Given we can’t readily establish the amount of time a job queued for CPU for it becomes difficult to establish whether more capacity would help the job. But there is some hope: "I have no idea" is probably not the best we can do:

• The job will be part of a WLM Service Class and so will – if the Service Class period has a velocity goal – have Delay For CPU samples. This is extremely broad brush but can tell you if a high CPU job is likely to suffer from queuing.
• If the EXCP count is high we can infer that a big chunk of the job’s time is for I/O.
• Variability of run time for similar CPU times and EXCP counts suggests the job sometimes gets held up for something.

So, suppose we kept capacity the same and, say, doubled the engine speed. Consider the case of a job that was 30% CPU. And ignore the n-way queuing effects of going from 2n engines to n. Then we might hazard the CPU time would halve and the elapsed time go down therefore by 15%.

Suppose we eliminated queuing (by and large). We can’t really say what the effect would be – except in our example job’s case it’s some fraction of the 70% that isn’t CPU. With the EXCP count we can "hand wave" a number, but it’s precisely that: a hand wave.

Some of the above is part of why I don’t like to give elapsed time speed up estimates. And why I’m not overly keen on answering the "faster versus more engines" question for batch. It’s actually worse than I’ve stated because individual job speed is hard to factor into the overall window’s outcome when migrating to a newer processor. (This same problem applies no matter which speed up you apply to individual jobs and steps, of course.)

But the "unknowable CPU queuing" fact plays into how to interpret other facts like (as in our example) the job is "only" 30% CPU. We don’t know whether it would’ve been 100% CPU without queuing or 30%. (Probably somewhere in between.) But we can use EXCP count, as I said, to help us guess.

For what it’s worth I rarely see jobs much above 30% or much below 10% CPU. I’d  say it was a bell curve around 20%. If true, this means most of the leverage is either in I/O time or CPU queuing. Though as a contrary data point I hear quite frequently of customers upgrading to faster engines and seeing their overall batch gets faster.

Welcome to my world of "it depends".:-)

(Originally posted 2011-07-19.)

Here’s another neat piece of algebra: A technique for summing series.

You know what b – a + c – b + d – c is. Right?

Suppose I were to write the same sum as:

(b – a) +

(c – b) +

(d – c)

The answer is still d – a. Right?

Now, suppose I re-label with s₀ = a, s₁ = b, s₂ = c and s₃ = d. We end up with:

(s₁ – s₀) +

(s₂ – s₁) +

(s₃ – s₂)

This is actually pretty scalable terminology as you can write s_r for any arbitrary value of r. And that’s one of the strengths of algebra: generalisation.

So let’s do that up to n:

(s₁ – s₀) +

(s₂ – s₁) +

…

(s_n – s_n-1)

which is, of course, s_n – s₀.

But what has that got to do with summing series?

If we can replace each (s_r – s_r-1) by a single term u_r you may see the relevance…

u₁ + u₂ + … + u_n = s_n – s₀.

The series summation boils down to a "simple" subtraction. The trick is to find these s terms, given the u terms. Let’s try it with an example.

 

---

Summing The Integers

 

This is the series 1, 2, 3, … , n.

The r’th u term is just r. u_r = r. So we now have to find the s_r term. Remember s_r – s_r-1 has to equal r.

Try s_r = r (r + 1).

Then s_r-1 = (r – 1) r   or   r (r – 1).

So s_r – s_r-1 = [(r + 1) – (r – 1)] r   or   2r. Not quite what we wanted. But we know – dividing by the factor of 2 – we should’ve guessed s_r = ½ r (r + 1).

So s_n – s₀ = ½ n (n + 1) – ½ 0 (0 + 1) = ½ n (n + 1) – 0 = ½ n (n + 1).

The sum of the first n integers being ½ n (n + 1) is a well-known result. Admittedly it could’ve been done another way. But it’s simple enough to show the method.

 

---

Another Example – Summing The Squares Of The Integers

 

This is the series 1, 4, 9, … , n² .

In this case we need to do something that will appear slightly perverse:

Rewrite r² as r ( r + 1) – r.

If you can split each of the terms in a series into two terms you can sum these sub terms. I just did the split. We already know how to sum the "r" portion. It’s ½ n (n + 1). So we need to sum the r (r+1) portion and subtract ½ n (n+1) from the result.

Try s_r = r (r + 1) (r + 2).

Again we need to find s_r-1.

It’s:

(r – 1) r (r + 1)

or, rearranging,

r (r+1) (r-1)

So

s_r – s_r-1 = [(r + 2) – (r – 1)] r (r + 1) or 3r (r + 1).

This is 3 times what we want so we should’ve guessed s_r = 1/3 r (r + 1) (r + 2).

So this portion of the sum is 1/3 n (n + 1) (n + 2) – 1/3 0 (0 + 1)(0 + 2) or 1/3 n (n + 1) (n + 2).

But we need to subtract ½ n (n + 1) from this:

1/3 n (n + 1) (n +2) – ½ n (n + 1) = 2/6 n (n + 1) (n +2) – 3/6 n (n + 1)

or

1/6 n (n + 1) [ 2 (n + 2) – 3] = 1/6 n (n + 1)( 2 n + 1) .

If you try it for a few values you’ll see it’s right. This isn’t such a well known result as for the sum of the integers.

 

---

 

I’m conscious there’s been some fiddliness here – which is where I normally fall down. 😦

But I think the "sum a series by converting it to a single subtraction" trick is a neat one – which is why I share it with you.

(Originally posted 2011-06-22.)

A need has arisen for pretty printing XML. This post includes some Python code to do it.

I’ve been working with the OpenOffice.org ODP presentation file format recently: I want to generate presentations from application code.

An ODP file is a structured zip file. Among other items it contains an XML file – content.xml – which describes the pages in the presentation. content.xml concatenates everything as one long line of text. It’s therefore very hard to read – without specialist XML parsing tools. And I’d like to – so I can teach my code to navigate and update it.

So the following is a short Python program to pretty print an XML file. It takes the XML file as standard input (stdin) and writes the formatted XML to standard output (stdout). If it can’t parse the XML it writes an appropriate error message to standard error (stderr).

While this isn’t a Python tutorial I’d like to outline how it works:

• I’m using the built-in xml.dom.minidom package to parse the XML and to pretty print it.
• I’m using the built in re regular expression package to remove trailing spaces on a line that minidom’s toprettyxml() seems to create.
• I’m using standard Python replace() to do some other substitutions to further clean up the XML.
• If the read-in XML isn’t valid minidom throws a ExpatError. In fact it uses xml.parsers.expat and this is what throws the error.

I’ll admit to a little disappointment that toprettyxml() doesn’t create quite as pretty an output file as I’d like. Hence the post-processing. Feel free to experiment with different values for the parameters to it. Also you might not feel the need for some of the post-processing.

Here’s the code. It’s more white space, comments and error handling than actual straight-through processing.

#!/usr/bin/python
from xml.dom import minidom
from xml.parsers.expat import ExpatError
import sys,re

# Edit the following to control pretty printing
indent="  "
newl=""
encoding="UTF-8"

# Regular expression to find trailing spaces before a newline
trails=re.compile(' *\n')

try:
  # Parse the XML - from stdin
  dom=minidom.parse(sys.stdin)
  
  # First-pass Pretty Print of the XML
  prettyXML=dom.toprettyxml(indent,newl,encoding)
  
  # Further clean ups
  prettyXML=prettyXML.replace("\t","")
  prettyXML=prettyXML.replace('"?><','"?>\n<')
  prettyXML=re.sub(trails,"\n",prettyXML)
  
  # Write XML to stdout
  sys.stdout.write(prettyXML)

except ExpatError as (expatError):
  sys.stderr.write("Bad XML: line "+str(expatError.lineno)+" offset "+str(expatError.offset)+"\n")

One question: Can this be run on z/OS? I believe it can: minidom is a core part of Jython and Jython certainly runs under z/OS. In fact, being a java jar I’d expect it to run on a zAAP (or zIIP with zAAP-on-zIIP). If you try it and get it working let us know.

(Originally posted 2011-06-21.)

Reading Now Your Embarrassing/Job-Threatening Facebook Photos Could Haunt You For Seven Years I think they’ve missed a point or two. Points which may be incidental to the thrust of the article but nonetheless are important ones:

Who’s to say what’s embarrassing or job-threatening?

Some things are obvious: In the post they talk about weapons ownership and racism as two things you wouldn’t want on your "record". Actually, the former example illustrates my point:

While racism is an obvious "no no" weapons ownership is in a grey area. Though personally I abjure weapons ownership there are many among you who feel differently. (And by espousing my position I’m exposing it – in the very previous sentence.) 🙂 But it shouldn’t be job threatening.

To repeat the first point: "Who’s to say what’s acceptable or not, embarrassing or not, job threatening or not?

The second point is: Suppose something is job threatening. Is that all that matters? How about our ability to live whatever’s left of our lives?

The third point is: Don’t we, without even getting close to threatening our jobs and careers, leak position and emotion?

As a, perhaps, aside I’ve just spent three days on a course with friends (both old and new) on "Personal Eminence and Gravitas". (Technically it’s called "Technical Leadership Masterclass 4", for the IBMers amongst you.) One of the "take homes" from that intensive 3 days is how others perceive you. I won’t discuss what my "take home" was but going through the process has caused me to think on reputation a little more. I was pleased the word "leak" was uttered by one of the course facilitators: I’ve used it before and picked up on it in class. It encapsulates the idea that you can’t really know what you’re giving away – so you can’t totally control it. Thank goodness!

Another data point is watching youngsters squabbling on Facebook. You’re supposed to think that’ll come back and haunt you – which would be grossly unfair. It’s actually rather funny. 🙂

I like to joke that we’re all unemployable – based on our Social Networking footprint. The serious point is that there is a definite need for forbearance on Social Networking. Otherwise the genie has to be put back in the bottle and the world has to return to being a really dull place. I for one don’t want to return to that. So much so that I’m prepared to take reputational risks: And we’re back to "authentic voice" and "authentic living".

So, in short, I hope we can all be authentic in what we say and do. That’s more a hope than an expectation – as I know there are parts of the world where it’s impossible to be authentic. 🙂

And, to recap on something I alluded to above: Who’s to say what emotion and position I’m leaking to you by writing this post?

(Originally posted 2011-06-12.)

Here’s the first-pass abstract slide for "I Know What You Did Last Summer". I’d be interested in your thoughts on it…

---

What’s The Point Of This Presentation?

So, it’s got a "tongue-in-cheek" title but what’s it all about?

I think one of the least appreciated aspects of z/OS and its middleware is the richness of instrumentation it gives you. Here I describe it and just some of the ways you can get value from SMF.

---

(Originally posted 2011-05-29.)

At one level Performance and Capacity Management and Systems Investigation are clearly linked: They share the same data. Or much of it at least.

But I think they’re linked in another way, too.

Over the past few years I’ve gradually shifted emphasis towards Systems Investigation. But this has only been a slight shift, a “non modo sed etiam” and still only really mainframe. So I’m still me but I’ve slowly realised I look at things a little differently – in addition to the old foci. And this thinking will feed into the “I Know What You Did Last Summer” presentation I’ve got started on.

But this post isn’t about that. It’s making a different point. And maybe a fairly obvious one:

When looking at Systems (and Application) performance it really pays to know what the landscape looks like. The same is true of Capacity, of course.

The received wisdom (and I certainly buy into it) is that Performance is about top down decomposition. For example, find the big CPU use categories and break them down: LPAR -> WLM Workload -> WLM Service Class -> Address Space -> Transaction -> Plan -> Package -> Statement.

We’re doing fine on this – particularly if we’re desirous of a technologically-neutral technique – until we hit the address space level. A CICS workload looks the same as a SAP one and as a Batch one: They’re just gobs of CPU. Even at this point I’m a little nervous as I like to be able to pronounce the names in the frames. (That’s why as a customer you might hear me ask “how do you pronounce this?”)

But when we get to the address space level that approach begins to unravel: For a start there might not be an address space. For instance, a WLM Service Class that manages DDF work won’t have address spaces in it: It’ll have enclaves. So how on earth can we decompose those two engines of CPU into actors – below the Service Class level? We certainly can’t do it with SMF Type 30 records.

Similarly, when I sketched out “Transaction -> Plan -> Package -> Statement” that only really works for CICS or IMS transactions accessing DB2.

To be fair you can do decomposition of things like DDF and Batch jobs with the right instrumentation and techniques. With that comment the point is hoving into view:

Sooner or later you have to know what this stuff is. The technologically neutral approach gives out after a certain point – and it’s different for different environments.

Another example is memory: With caveats on data clarity you can decompose memory usage in a similar way. But it behaves differently to CPU, and different from one user to another. While you might expect CPU to grow linearly – more or less – with workload, you wouldn’t really expect that of memory. Or at least you shouldn’t:

• Some workloads do use memory in a linear way: Double the workload (by whatever metric you choose) and the memory usage doubles. The classic example is TSO users: Go from 200 of them to 400 and the memory usage goes from 1GB to 2GB (at least in 1990 terms).
• Many workloads are sub-linear: Double the number of CICS users and they memory usage may go up by only 50%.

Indeed the latter case is an example of where it’s not clear at all: When you say “double the number of CICS users” are you expecting to double the number of regions? Or do you mean add the users into existing regions?

So the conclusion is you need to know about the applications to get very far. And you probably need to know a lot about things like LPAR setup. Indeed, as I’ve often said, just keeping track of all those LPARs is a major headache for many customers these days.

So, I’d encourage you to get curious about your systems. Take a Systems Investigative perspective when you can. It’s also a great way to build common understanding with those that actually run the systems.

But this is not the same as the school of tuning which says “find sins of omission or commission and comment on them”. These kinds of sins are important – but only in the context of a top-down approach. Who cares if a parameter is not set correctly – in a classical sense – if it affects nothing you care about?

So, as I say the linkage between Capacity / Performance and Understanding Systems is twofold: The raw data and the need to know what’s really happening on systems and with applications.