Two Kinds Of Pipes

(Originally posted 2012-06-24.)

It’s been a while since I last posted here. And that’s been more a matter of being incredibly busy than anything. Some of you will have seen me "on stage" – there having been several conferences and user groups recently. Some of you were with me at the zChampions meeting and some of you I’m actively working on customer situations with.

(By the way one of the rules of zChampions isn’t "nobody mentions zChampions". 🙂 But it’s true to an extent that "what happens at zChampions stays in zChampions”. 🙂 But only to an extent.)

I won’t signal a let up in activity by suggesting I’ll post more frequently: That’s probably unrealistic.

But in this post I want to talk about z/OS Unix Pipes in JCL vs BatchPipes/MVS. The reason for doing this is that in some circumstances z/OS Unix Pipes in JCL can play a similar role to BatchPipes. But there are caveats. In what follows I’ll generally use the term “Unix Pipes” ¹ as shorthand for “Unix Pipes in JCL” (except for the next heading where I think it’s clearer to write it in full).

Unix Pipes in JCL

Here are two sample pieces of JCL you can play with:

First a writer job. Let’s call it "W":

//WRITE1  EXEC PGM=IEBGENER 
//SYSPRINT DD SYSOUT=K,HOLD=YES 
//SYSIN   DD DUMMY 
//SYSUT1  DD * 
ABCD 
/* 
//SYSUT2  DD PATH='/u/userhfs/pmmpac/pipe1',DSNTYPE=PIPE,
//        LRECL=80,BLKSIZE=3200,RECFM=FB, 
//        PATHOPTS=(OWRONLY,OCREAT), 
//        PATHMODE=(SIWUSR,SIRUSR), 
//        PATHDISP=(DELETE,DELETE) 

And now a reader job ("R"):

//READ1   EXEC PGM=IEBGENER 
//SYSPRINT DD SYSOUT=K,HOLD=YES 
//SYSIN   DD DUMMY 
//SYSUT1  DD PATH='/u/userhfs/pmmpac/pipe1',DSNTYPE=PIPE,
//        LRECL=80,BLKSIZE=3200,RECFM=FB, 
//        PATHOPTS=(ORDONLY,OCREAT), 
//        PATHMODE=(SIWUSR,SIRUSR), 
//        PATHDISP=(DELETE,DELETE) 
/* 
//SYSUT2  DD SYSOUT=K,HOLD=YES 

The pipe here is given by the PATH name “/u/userhfs/pmmpac/pipe1” (and you’ll have to figure out what it should be in your environment).

In this example I’m just copying from instream data (the “ABCD” line) to the pipe and copying from the pipe to the SPOOL. Simple stuff you can get working and build on.

I experimented with PATHOPTS, PATHMODE and PATHDISP and the ones in the sample JCL decks are the ones that worked best. In particular PATHDISP=(DELETE,DELETE) seemed to be necessary to get the pipe deleted after use.

When I run these they appear to perform as a BatchPipes/MVS pipe would: Overlapped and apparently without real I/O² – which are the key benefits.

To those who know BatchPipes/MVS this should all look quite familiar…

Unix Pipes vs BatchPipes/MVS

As I said, they’re quite similar. But there are differences that are worth noting. The most notable ones are:

• While BatchPipes can “add value” with fittings in the pipe-end definitions Unix Pipes can’t – you’d have to have a separate intermediary job (I tend to call “F” for “Filter” / “Fitting”. You can write it in whatever you like. Suggestions?
• You don’t get the equivalent of SMF 91 – which has handy things to enable you to e.g, balance the pipe.
• You don’t have a subsystem – and that’s probably a good thing. Instead you’re using the Unix file system – as the pipe name suggests.
• To convert a non-piping usage to a piping one requires significantly more JCL “surgery”. In particular you can’t use the data set name for the pipe name. And you can see things like DISP are completely different. So it’s more work.
• BatchPipes/MVS has the BatchPipePlex capability to pipe between Parallel Sysplex members using Coupling Facility structures. I don’t think you can do anything like that with Unix Pipes.

There are probably other differences but these are the most noteworthy to me – as someone who’s a lot of experience with BatchPipes/MVS. Some of them definitely can be worked around – such as the lack of Fittings support. Some of them are more difficult. But if you just want a simple inter-job piping capability have a go with Unix Pipes – perhaps starting with the two JCL samples above. And one key advantage of this approach might be that it’s free and available to all (I think) – so long as your jobs are allowed to use Unix System Services³.

And as to posting frequency: I feel under no pressure to publish – whether timing-wise or content-wise. If there’s stuff I think is interesting (and not damaging to a client) you’ll see it here – when I get the chance.

---

¹ Yes, I know Unix shells have their own built-in piping capabilities. Discussing those is beyond the scope of this post.

² I’m told no Unix piping implementation would perform physical I/O – so this isn’t a queue on disk (with the issues that would raise: I/O time and potential physical limits).

³ And I note the perpetual discussion on the IBM-MAIN Listserver group about whether “USS” is an acceptable abbreviation for “Unix System Services”… 🙂