(Originally posted 2011-07-05.)

As you probably know Hardware Data Compression has been supported by MVS and IBM mainframes for around 20 years. In several recent batch studies I’ve conducted it’s been evident in a widespread way.

(In this post I’m not talking about DB2 compression of either flavour or VSAM compression – though some of the information here applies to these functions as well.)

It’s not as simple a deployment strategy as "turn it on for everything" though I think "turn it on for all sequential data sets large enough to stripe" may well be the flavour. There’s a certain logic to this: If striping can make sequential data access go faster then compressing the data will make it go even fasterer.

There’s a lot of truth in that. But consider one thing: Compression and (decompression) takes CPU cycles. In most software licencing schemes that translates into an increased software bill. For small-scale use that might not matter much. But used wholesale it could make a significant difference.

There’s another thing: If you cause a job that’s running in a CPU-constrained environment to burn more CPU through Compression the speed up may be disappointing. I’m not saying it’s worse than useless – I’ve no evidence of that – but it is a consideration.

My suspicion that the CPU cost can be quite high comes from seeing "low CPU" cases like DFSORT COPY operations and other simple reformatters burning far more CPU than I would have expected. (Quite often this includes a significant chunk of SRB time but not always. This is consistent with the way DFSMS/MVS does (de)compression.) There is no direct metric for the CPU cost of Compression: You have to use the overall CPU consumption and draw your own conclusions.

It sounds like I’m against using Hardware Data Compression. Actually I’m not: I’ve seen evidence of very good compression ratios (for example in SMF 14 and 15 records for non-VSAM data sets). And I know that in many of the cases I’ve seen the alternative might have been to write the data to tape, with all the handling that entails. Which brings us on to something else:

Quite a few of the scenarios I’ve seen have been one-writer-one-reader cases. You tend to think "Pipes" in those cases. You were expecting that, weren’t you. 🙂 but, seriously, the advantages are fairly obvious. One of the objections to Pipes has always been that it takes more CPU than writing data to disk or tape. But if you compress it* it’s not nearly so obvious that’s the case. It would be interesting to conduct an experiment.

This is pretty much an "it depends" situation. Aren’t they all? Actually, there’s quite possibly more speed advantage in getting the buffering and number of stripes right – as described in SG24-2557 Parallel Sysplex Batch Performance. (This book has lots of good advice on access method tuning – written by my residency teammates in 1995.)

In short, if you’re implementing Hardware Data Compression measure the impact and effectiveness and be open to the idea there may be other ways of achieving the speed up.

Finally, I said this wasn’t VSAM but note that SMF 64 also has a Compressed Data Statistics Section, just like SMF 14 and 15. My expectation is, however, that the access pattern to compressed VSAM is not sequential. (For QSAM and BSAM I expect it to be largely sequential.)

---

* Tape compression is a different matter – with the hard work being done in the control unit.

(Originally posted 2011-07-02.)

Following on from The Best Sort Is The One You Don’t Do here’s another reason for eliminating sorts. I think it’s worth a post in its own right.

(In this post, again, I’m talking about resequencing passes over data – not copying or merging.)

With a sort it’s possible the last record read in might be the first record written out. So you can never overlap input and output phases. (There might even be a phase between the end of the input phase and the beginning of the output phase – particularly if there are intermediate merges.)

A common pattern in a job is a processing step, then a sort step, then another processing step, and so on. Often the first processing step writes a data set the sort reads and the second processing step reads the sorted version of that. (It’s a separate question whether either processing step could have been performed by the sort, of course.) Let’s call the first processing step "W", the sort "S" and the second processing step "R".

Now consider what happens with BatchPipes/MVS. With Pipes you overlap the reader and the writer. That’s part of the benefit (along with I/O time reductions and, perhaps, the elimination of tape mounts). In the above scenario you can overlap W with the input phase of S – with a pipe. Likewise you can overlap the output phase of S with R. What you can’t do is overlap W with R – because you can’t overlap the two phases of S.

A pity but perhaps not a huge one. Let’s illustrate this with some numbers. Suppose:

• W runs for 10 minutes, writing all the while.
• S has an input phase of 5 minutes and an output phase of 5 minutes.
• R runs for 10 minutes, reading all the while.

The three steps between them take 10 + 5 + 5 +10 = 30 minutes.

Overlapping W with S’ input phase saves 5 minutes (as S’ input phase gets stretched to 10 minutes). Similarly overlapping S’ output phase with R saves 5 minutes. So we save 10 minutes overall.* We like to say "the sort gets done for free".

If we can eliminate the sort completely we can overlap S and W completely with a total saving of 20 minutes – 10 for the overlap and 10 for the sort removal. (Without Pipes we’d still see a 10 minute reduction just by removing the sort.)

So, this is another case where removing a sort is a valuable thing to achieve. But as I said in the referenced post that’s easier said than done.

---

* In this calculation you’ll notice I’ve assumed the I/O time reduction is zero and that there are no tape mounts eliminated. Generally there is some I/O reduction, of course. But it’s still a fair comparison. It’s just the benefits of Pipes are understated.

(Originally posted 2011-07-02.)

Have you ever had the suspicion a sort was unnecessary in your batch? I bet you have.

In recent Batch Performance studies I’ve had the suspicion that many of the sorts are unnecessary: Either they should be merges or not done at all. But how do you prove it?
 

But first, what do I mean by a sort not needing to be done at all?

Clearly if the data is reformatted then something has to be done to it – just maybe not a sort. It could be a copy or maybe a merge. So I’m really talking about the need to reorder the records. Reordering them is more expensive in terms of disk space (for sort work data sets), CPU and run time. So a sort is best avoided.

So how do you figure out if a sort needs doing? One way is to remove the sort and see what happens. Probably best not done in Production. 🙂 That’s a form of "destructive testing". But there is another way: Essentially running the test I’m about to outline in the time frame of the sort.
 

In a moment "choreography" but first the mechanics:

Testing If Data Is Already Sorted

For a DFSORT MERGE operation to be successful all the data sets being merged must already be sorted on the merge key(s). Otherwise you get a return code of 16 and a ICE068A message such as:

ICE068A 0 OUT OF SEQUENCE SORTIN01

We can use this to our advantage by attempting to merge the supposedly-already-sorted data set with a dummy file. (DD DUMMY will do just fine.) If the data set is already sorted the step will complete with a 0 return code. Otherwise, as I say, it will be RC=16.
 

It’s worth noting that the merge will fail immediately it detects a record out of sequence. This means a badly unsorted data set will fail fast. However a largely well sorted data set may not fail the test until it has been almost completely read: Perhaps the last record is the only one out of sequence.

How To Incorporate This Test

If the input data set is persistent you could either add the test step in just before the sort or at the end of the job. If it’s transient (perhaps temporary) it has to be tested before it goes away. In short you run the test when it’s still there but not otherwise updated.

One important aspect is the intrusiveness of the test: Breaking a BatchPipes/MVS pipe to do it is a particularly bad idea. Holding up processing for tape mounts is probably bad. And maybe running the test alongside the actual sort slows it down.

If you were tuning your batch window and suspected a sort was unnecessary you might allow the test to run a few dozen times to gain confidence that it’s not needed. But you might never be sufficiently confident.

One other thing: You might decide to always run the test and run the sort only if the test step ends with RC=16. But then a downstream step would have to access the right data set – presumably the SORTOUT from the test step or the SORTOUT from the sort, depending.

Concluding Thoughts

 

At the end of the day it’s better to know your batch well enough to avoid having to rely on tests like this. But in reality people don’t understand their batch that well, in my experience. Not a criticism, just an observation. And a situation caused by the complexity and longevity of the typical batchscape.

Maybe this technique is best used during development or in a test environment. As I said on Twitter yesterday I was thinking about ways of "injecting dye into the water" in a data flow sort of way. Maybe I’ll think of some additional "dye test" or "smoke test" (if you prefer) techniques. One other wrinkle might be to inject record sequence numbers into the data to see whether they’re preserved.

(Originally posted 2011-06-29.)

This may be stating the obvious – but I wonder to whom it actually is obvious…

I’ve been doing quite a lot of work with batch job timings and CPU recently. (Everything I’m about to say is equally true of steps.) It’s interesting to think about the effects of faster engines versus more engines (a question I haven’t been asked recently) and whether a customer needs more capacity or just faster engines (a question that has come into play).
 
There’s nothing very new about this but it is worth thinking about. And in particular what we can glean from SMF 30 Step- and Job-end records…
 
We know lots of things about a job’s timing and related stuff, most notably:

• When it started and ended, and hence the elapsed time.
• Where it ran.
  
• When it was read in and initiator delays.
• How much CPU it used – whether TCB or SRB.
• Whether it used a zIIP or zAAP or was eligible to but didn’t.
• How many disk or tape EXCPs it did (and how many tape mounts).
• Step condition codes.
  

But one thing we can’t know is whether a job met significant CPU queuing or not. At least not from Type 30. We might be able to infer something from DB2 Accounting Trace (SMF 101) but (to rehearse previous arguments):

• We can’t know if the Unaccounted For time (the most likely bucket) really is queuing time. (Generally it is but there are lots of edge cases where it’s something else.)
• We can encounter difficulties in tying up the 101 with the 30 – particularly for IMS.
• We have to take into account CP Query Parallelism.
• The job might not be DB2 at all.

Given we can’t readily establish the amount of time a job queued for CPU for it becomes difficult to establish whether more capacity would help the job. But there is some hope: "I have no idea" is probably not the best we can do:

• The job will be part of a WLM Service Class and so will – if the Service Class period has a velocity goal – have Delay For CPU samples. This is extremely broad brush but can tell you if a high CPU job is likely to suffer from queuing.
• If the EXCP count is high we can infer that a big chunk of the job’s time is for I/O.
• Variability of run time for similar CPU times and EXCP counts suggests the job sometimes gets held up for something.

So, suppose we kept capacity the same and, say, doubled the engine speed. Consider the case of a job that was 30% CPU. And ignore the n-way queuing effects of going from 2n engines to n. Then we might hazard the CPU time would halve and the elapsed time go down therefore by 15%.

Suppose we eliminated queuing (by and large). We can’t really say what the effect would be – except in our example job’s case it’s some fraction of the 70% that isn’t CPU. With the EXCP count we can "hand wave" a number, but it’s precisely that: a hand wave.

Some of the above is part of why I don’t like to give elapsed time speed up estimates. And why I’m not overly keen on answering the "faster versus more engines" question for batch. It’s actually worse than I’ve stated because individual job speed is hard to factor into the overall window’s outcome when migrating to a newer processor. (This same problem applies no matter which speed up you apply to individual jobs and steps, of course.)

But the "unknowable CPU queuing" fact plays into how to interpret other facts like (as in our example) the job is "only" 30% CPU. We don’t know whether it would’ve been 100% CPU without queuing or 30%. (Probably somewhere in between.) But we can use EXCP count, as I said, to help us guess.

For what it’s worth I rarely see jobs much above 30% or much below 10% CPU. I’d  say it was a bell curve around 20%. If true, this means most of the leverage is either in I/O time or CPU queuing. Though as a contrary data point I hear quite frequently of customers upgrading to faster engines and seeing their overall batch gets faster.

Welcome to my world of "it depends".:-)

(Originally posted 2011-07-19.)

Here’s another neat piece of algebra: A technique for summing series.

You know what b – a + c – b + d – c is. Right?

Suppose I were to write the same sum as:

(b – a) +

(c – b) +

(d – c)

The answer is still d – a. Right?

Now, suppose I re-label with s₀ = a, s₁ = b, s₂ = c and s₃ = d. We end up with:

(s₁ – s₀) +

(s₂ – s₁) +

(s₃ – s₂)

This is actually pretty scalable terminology as you can write s_r for any arbitrary value of r. And that’s one of the strengths of algebra: generalisation.

So let’s do that up to n:

(s₁ – s₀) +

(s₂ – s₁) +

…

(s_n – s_n-1)

which is, of course, s_n – s₀.

But what has that got to do with summing series?

If we can replace each (s_r – s_r-1) by a single term u_r you may see the relevance…

u₁ + u₂ + … + u_n = s_n – s₀.

The series summation boils down to a "simple" subtraction. The trick is to find these s terms, given the u terms. Let’s try it with an example.

 

---

Summing The Integers

 

This is the series 1, 2, 3, … , n.

The r’th u term is just r. u_r = r. So we now have to find the s_r term. Remember s_r – s_r-1 has to equal r.

Try s_r = r (r + 1).

Then s_r-1 = (r – 1) r   or   r (r – 1).

So s_r – s_r-1 = [(r + 1) – (r – 1)] r   or   2r. Not quite what we wanted. But we know – dividing by the factor of 2 – we should’ve guessed s_r = ½ r (r + 1).

So s_n – s₀ = ½ n (n + 1) – ½ 0 (0 + 1) = ½ n (n + 1) – 0 = ½ n (n + 1).

The sum of the first n integers being ½ n (n + 1) is a well-known result. Admittedly it could’ve been done another way. But it’s simple enough to show the method.

 

---

Another Example – Summing The Squares Of The Integers

 

This is the series 1, 4, 9, … , n² .

In this case we need to do something that will appear slightly perverse:

Rewrite r² as r ( r + 1) – r.

If you can split each of the terms in a series into two terms you can sum these sub terms. I just did the split. We already know how to sum the "r" portion. It’s ½ n (n + 1). So we need to sum the r (r+1) portion and subtract ½ n (n+1) from the result.

Try s_r = r (r + 1) (r + 2).

Again we need to find s_r-1.

It’s:

(r – 1) r (r + 1)

or, rearranging,

r (r+1) (r-1)

So

s_r – s_r-1 = [(r + 2) – (r – 1)] r (r + 1) or 3r (r + 1).

This is 3 times what we want so we should’ve guessed s_r = 1/3 r (r + 1) (r + 2).

So this portion of the sum is 1/3 n (n + 1) (n + 2) – 1/3 0 (0 + 1)(0 + 2) or 1/3 n (n + 1) (n + 2).

But we need to subtract ½ n (n + 1) from this:

1/3 n (n + 1) (n +2) – ½ n (n + 1) = 2/6 n (n + 1) (n +2) – 3/6 n (n + 1)

or

1/6 n (n + 1) [ 2 (n + 2) – 3] = 1/6 n (n + 1)( 2 n + 1) .

If you try it for a few values you’ll see it’s right. This isn’t such a well known result as for the sum of the integers.

 

---

 

I’m conscious there’s been some fiddliness here – which is where I normally fall down. 😦

But I think the "sum a series by converting it to a single subtraction" trick is a neat one – which is why I share it with you.

(Originally posted 2011-06-22.)

A need has arisen for pretty printing XML. This post includes some Python code to do it.

I’ve been working with the OpenOffice.org ODP presentation file format recently: I want to generate presentations from application code.

An ODP file is a structured zip file. Among other items it contains an XML file – content.xml – which describes the pages in the presentation. content.xml concatenates everything as one long line of text. It’s therefore very hard to read – without specialist XML parsing tools. And I’d like to – so I can teach my code to navigate and update it.

So the following is a short Python program to pretty print an XML file. It takes the XML file as standard input (stdin) and writes the formatted XML to standard output (stdout). If it can’t parse the XML it writes an appropriate error message to standard error (stderr).

While this isn’t a Python tutorial I’d like to outline how it works:

• I’m using the built-in xml.dom.minidom package to parse the XML and to pretty print it.
• I’m using the built in re regular expression package to remove trailing spaces on a line that minidom’s toprettyxml() seems to create.
• I’m using standard Python replace() to do some other substitutions to further clean up the XML.
• If the read-in XML isn’t valid minidom throws a ExpatError. In fact it uses xml.parsers.expat and this is what throws the error.

I’ll admit to a little disappointment that toprettyxml() doesn’t create quite as pretty an output file as I’d like. Hence the post-processing. Feel free to experiment with different values for the parameters to it. Also you might not feel the need for some of the post-processing.

Here’s the code. It’s more white space, comments and error handling than actual straight-through processing.

#!/usr/bin/python
from xml.dom import minidom
from xml.parsers.expat import ExpatError
import sys,re

# Edit the following to control pretty printing
indent="  "
newl=""
encoding="UTF-8"

# Regular expression to find trailing spaces before a newline
trails=re.compile(' *\n')

try:
  # Parse the XML - from stdin
  dom=minidom.parse(sys.stdin)
  
  # First-pass Pretty Print of the XML
  prettyXML=dom.toprettyxml(indent,newl,encoding)
  
  # Further clean ups
  prettyXML=prettyXML.replace("\t","")
  prettyXML=prettyXML.replace('"?><','"?>\n<')
  prettyXML=re.sub(trails,"\n",prettyXML)
  
  # Write XML to stdout
  sys.stdout.write(prettyXML)

except ExpatError as (expatError):
  sys.stderr.write("Bad XML: line "+str(expatError.lineno)+" offset "+str(expatError.offset)+"\n")

One question: Can this be run on z/OS? I believe it can: minidom is a core part of Jython and Jython certainly runs under z/OS. In fact, being a java jar I’d expect it to run on a zAAP (or zIIP with zAAP-on-zIIP). If you try it and get it working let us know.

(Originally posted 2011-06-21.)

Reading Now Your Embarrassing/Job-Threatening Facebook Photos Could Haunt You For Seven Years I think they’ve missed a point or two. Points which may be incidental to the thrust of the article but nonetheless are important ones:

Who’s to say what’s embarrassing or job-threatening?

Some things are obvious: In the post they talk about weapons ownership and racism as two things you wouldn’t want on your "record". Actually, the former example illustrates my point:

While racism is an obvious "no no" weapons ownership is in a grey area. Though personally I abjure weapons ownership there are many among you who feel differently. (And by espousing my position I’m exposing it – in the very previous sentence.) 🙂 But it shouldn’t be job threatening.

To repeat the first point: "Who’s to say what’s acceptable or not, embarrassing or not, job threatening or not?

The second point is: Suppose something is job threatening. Is that all that matters? How about our ability to live whatever’s left of our lives?

The third point is: Don’t we, without even getting close to threatening our jobs and careers, leak position and emotion?

As a, perhaps, aside I’ve just spent three days on a course with friends (both old and new) on "Personal Eminence and Gravitas". (Technically it’s called "Technical Leadership Masterclass 4", for the IBMers amongst you.) One of the "take homes" from that intensive 3 days is how others perceive you. I won’t discuss what my "take home" was but going through the process has caused me to think on reputation a little more. I was pleased the word "leak" was uttered by one of the course facilitators: I’ve used it before and picked up on it in class. It encapsulates the idea that you can’t really know what you’re giving away – so you can’t totally control it. Thank goodness!

Another data point is watching youngsters squabbling on Facebook. You’re supposed to think that’ll come back and haunt you – which would be grossly unfair. It’s actually rather funny. 🙂

I like to joke that we’re all unemployable – based on our Social Networking footprint. The serious point is that there is a definite need for forbearance on Social Networking. Otherwise the genie has to be put back in the bottle and the world has to return to being a really dull place. I for one don’t want to return to that. So much so that I’m prepared to take reputational risks: And we’re back to "authentic voice" and "authentic living".

So, in short, I hope we can all be authentic in what we say and do. That’s more a hope than an expectation – as I know there are parts of the world where it’s impossible to be authentic. 🙂

And, to recap on something I alluded to above: Who’s to say what emotion and position I’m leaking to you by writing this post?

(Originally posted 2011-06-12.)

Here’s the first-pass abstract slide for "I Know What You Did Last Summer". I’d be interested in your thoughts on it…

---

What’s The Point Of This Presentation?

So, it’s got a "tongue-in-cheek" title but what’s it all about?

I think one of the least appreciated aspects of z/OS and its middleware is the richness of instrumentation it gives you. Here I describe it and just some of the ways you can get value from SMF.

---

(Originally posted 2011-05-29.)

At one level Performance and Capacity Management and Systems Investigation are clearly linked: They share the same data. Or much of it at least.

But I think they’re linked in another way, too.

Over the past few years I’ve gradually shifted emphasis towards Systems Investigation. But this has only been a slight shift, a “non modo sed etiam” and still only really mainframe. So I’m still me but I’ve slowly realised I look at things a little differently – in addition to the old foci. And this thinking will feed into the “I Know What You Did Last Summer” presentation I’ve got started on.

But this post isn’t about that. It’s making a different point. And maybe a fairly obvious one:

When looking at Systems (and Application) performance it really pays to know what the landscape looks like. The same is true of Capacity, of course.

The received wisdom (and I certainly buy into it) is that Performance is about top down decomposition. For example, find the big CPU use categories and break them down: LPAR -> WLM Workload -> WLM Service Class -> Address Space -> Transaction -> Plan -> Package -> Statement.

We’re doing fine on this – particularly if we’re desirous of a technologically-neutral technique – until we hit the address space level. A CICS workload looks the same as a SAP one and as a Batch one: They’re just gobs of CPU. Even at this point I’m a little nervous as I like to be able to pronounce the names in the frames. (That’s why as a customer you might hear me ask “how do you pronounce this?”)

But when we get to the address space level that approach begins to unravel: For a start there might not be an address space. For instance, a WLM Service Class that manages DDF work won’t have address spaces in it: It’ll have enclaves. So how on earth can we decompose those two engines of CPU into actors – below the Service Class level? We certainly can’t do it with SMF Type 30 records.

Similarly, when I sketched out “Transaction -> Plan -> Package -> Statement” that only really works for CICS or IMS transactions accessing DB2.

To be fair you can do decomposition of things like DDF and Batch jobs with the right instrumentation and techniques. With that comment the point is hoving into view:

Sooner or later you have to know what this stuff is. The technologically neutral approach gives out after a certain point – and it’s different for different environments.

Another example is memory: With caveats on data clarity you can decompose memory usage in a similar way. But it behaves differently to CPU, and different from one user to another. While you might expect CPU to grow linearly – more or less – with workload, you wouldn’t really expect that of memory. Or at least you shouldn’t:

• Some workloads do use memory in a linear way: Double the workload (by whatever metric you choose) and the memory usage doubles. The classic example is TSO users: Go from 200 of them to 400 and the memory usage goes from 1GB to 2GB (at least in 1990 terms).
• Many workloads are sub-linear: Double the number of CICS users and they memory usage may go up by only 50%.

Indeed the latter case is an example of where it’s not clear at all: When you say “double the number of CICS users” are you expecting to double the number of regions? Or do you mean add the users into existing regions?

So the conclusion is you need to know about the applications to get very far. And you probably need to know a lot about things like LPAR setup. Indeed, as I’ve often said, just keeping track of all those LPARs is a major headache for many customers these days.

So, I’d encourage you to get curious about your systems. Take a Systems Investigative perspective when you can. It’s also a great way to build common understanding with those that actually run the systems.

But this is not the same as the school of tuning which says “find sins of omission or commission and comment on them”. These kinds of sins are important – but only in the context of a top-down approach. Who cares if a parameter is not set correctly – in a classical sense – if it affects nothing you care about?

So, as I say the linkage between Capacity / Performance and Understanding Systems is twofold: The raw data and the need to know what’s really happening on systems and with applications.

(Originally posted 2011-05-24.)

When tuning DB2 batch it’s important to know which SMF 101 Accounting Trace record corresponds to which job step.

A few years ago I wrote code to do this. It works fine for all z/OS DB2 Batch except that originated by IMS. Here’s how it works:

1. Find all the Type 30 Step-End records for a given job name.
2. Find the Type 101 Accounting Trace records for which the Correlation ID in the 101 record matches the job name. (First 8 bytes of Correlation ID is the job name for most DB2 Batch.)
3. You match up the records from steps 1 and 2: The SMF ID must match and the start timestamp (QWACBSC) and stop timestamp (QWACESC) in the 101 slot into the time range the Type 30 is for. This gives you the running of the job and the step. The latter is nice because SMF 101 doesn’t know anything about job steps.

But there’s a small problem with this: QWACBSC and QWACESC use GMT and the Type 30 timings use local time. (In the experiment below the local time is 2 hours ahead of GMT: Mainland Europe on Summer Time.) So you have to adjust QWACBSC using the timezone offset. But SMF 101 doesn’t have the timezone offset in it. Except it does. If you compare the SMF record timestamp to QWACESC you get the timezone offset (as QWACESC is GMT).

All the above has worked fine. But now I have a customer with IMS Batch. And for IMS Batch the Correlation ID isn’t the job name. So what can you do?

It turns out that for long-running IMS/DB2 Batch you can still do the timestamp comparison trick – so long as you don’t attempt to match on Correlation ID. For short-running jobs it’s not likely to work as well – because there will be too many potential matches. Fortunately I care rather less about these cases.

Here’s an example match:

• SMF 30: The job name is AFRF032B and it runs on SYSA for 4904 seconds, consuming 289 seconds of CPU.
• SMF 101: There was an IMS batch job (PSB name "AFRFD2D") which ran on SYSA for 4899 seconds, consuming 288 seconds of CPU.

So far so good but they might be different jobs. Let’s examine their timestamps (adjusted for timezones). First start times:

• SMF 30: 11 APR 06 00:31:12.39
• SMF 101: 11 APR 06 00:31:15.93

So we took 3 seconds to get started. Sounds reasonable.

Now end times:

• SMF 30: 11 APR 06 01:52:56.63
• SMF 101: 11 APR 06 01:52:55.40

So we cut the 101 record about a second before we ended the step. Again reasonable.

The bracketing is nice here and I think we have a match.

So now we can proceed to figure out why the step took as long as it did by using the other numbers in SMF 101 to explain the time:

• The Class 1 time not in Class 2 is 4899 – 4608 seconds = 5 minutes. So not much leverage in working on non-DB2 stuff. (For CPU it’s 288 – 273 = 15 seconds.)
• Of the 4608 seconds 2699 are accounted for – 59%.
• The majority of that is in Write To Log time (1450 seconds) but also Lock/Latch Wait time (651) seconds. So clearly some issues here to sort out.
• There’s very little database I/O time, surprisingly (about 250 seconds).

The unaccounted for time (the other 41%) is actually not surprising to me: It turns out this job runs in a low-velocity WLM service class on a busy system so the usual theory (that unaccounted for time is CPU Queuing) seems reasonable. It’s not definitive as there are lots of other components of unaccounted for time, but it’s a reasonable theory.

The last piece -where the time is going – is meant to give you some idea of why you’d want to find the 101 Accounting Trace records for an IMS/DB2 job step you’re interested in tuning. Of course I haven’t delved down into which DB2 package is the one where the Lock/Latch Wait time is most prevalent – but that can be done from the Class 7 and 8 Accounting traces. Nor have I gone anywhere near the SQL.

But this shows you how you can make a reasonable start – even for IMS/DB2. A nice result – having thought for a number of years it couldn’t be done.