Memories of Batch LSR

(Originally posted 2007-11-18.)

Another trip down memory lane – prompted by the thread on IBM-MAIN of Batch LSR vs Hiperbatch.

Batch LSR started out as a prototype by Paul Dorn of the IBM Washington Systems Center. He was contributing to a book on writing a subsystem but also trying to solve a problem with Batch VSAM tuning. So Batch LSR started out as an example of a subsystem.

(I don’t recall a rash of subsystems written by users or vendors after that book was published. 😦 But BatchPipes/MVS would later be built as one.)

Now to the problem Batch LSR (hereafter referred to as BLSR was designed to solve, back in the 1980s…

If a batch job accessed a VSAM file it would almost always use VSAM NSR, which was (or rather could be) optimised for sequential access. If, however, the access was random^[1] NSR would be pretty hopeless. To use the random-oriented VSAM LSR would require the program to create its own VSAM LSR buffer pool(s) using the BLDVRP macro. Yes, that’s right, Assembler. 🙂 Most batch jobs were never going to be written that way.

So Paul wrote (and Poughkeepsie rewrote) the subsystem. Here’s what it does…

The subsystem creates the VSAM LSR buffer pools for you. And it allows you to specify a number of parameters, such as the location of the buffer pools (above or below the 24-bit line) and their size. It’s very easy to set up and easy to code the JCL changes needed to use it. (In a large number of places in our VSAM-based (SLR) analysis code we’ve done this.)

Here’s a case Roger Fowler and I worked on in the mid-1990s:

A UK customer had a batch job that did 2 million I/Os to an 850 CI data set. Roger said “let’s turn on BLSR”. They did and the job went down to 0.5 million I/Os. So I said “let’s turn on the Deferred Write option”. Roger said “the what?” 🙂 Anyhow we turned it on and the job went down to around 1,000 I/Os. From 2 million down to 1 thousand is pretty good, I’d say. 🙂

There was a tool called BLSRAID which was SAS-based. We didn’t have SAS so we wrote our own VSAM analysis code in 1993. Another piece in the jigsaw that is the PMIO toolset. (Roger and I used this report to make that saving.)

Another tool – for when you have enabled BLSR for a data set / job – is the User F61 GTF trace. This fed into a popular tool – VLBPAA – but you wouldn’t want to run the trace for all that long. In the back of the SG24-2557 “Parallel Sysplex Batch Performance” Red Book I wrote an appendix on playing with this trace. You can use this trace (and could use VLBPAA) to model the effects of big buffer pools.

In principle, as my and Roger’s example showed, you could fit the whole data set into memory. In fact we recommended BUFND=1024 to this customer. Slightly wasteful but a nice round number. (1024 4KB buffers is, of course, 4MB.)

There were many cases down the years where BLSR was a good fit. But quite often we made the recommendation to stick with NSR and tune for sequential access instead. The basic VSAM instrumentation – SMF 62 and 64 – would lead us to robust conclusions. My good friend Dave Betten (now Performance Lead for DFSORT) put together the Type 30, 62 and 64 records in one SLR table. And we regularly used the SMF 42-6 records (designed by Jeff Berger and eulogised by our good friend John Burg) to round out the VSAM data set picture.

Of course in 1997, with OS/390 Release 4 DFSMS, System-Managed Buffering (SMB) came along. This made it much easier to optimise for sequential or random access, using DFSMS constructs. I’m not sure how widely this is used – but it’s worth taking a look at.

Of course DB2, suitably managed, has many I/O strategies. But for VSAM you have to do it yourself. And the Batch LSR Subsystem made it possible for “random” VSAM I/O.

^[1] When I say “random” I really mean “direct”. I’m not sure I believe in randomness, particularly in data processing. The point is that there is “dotting about” rather than sequential access.