Creating JSON with DFSORT

(Originally posted 2013-10-18.)

This post is yet another spin off from the residency I’m on in Poughkeepsie.

I mentioned in We Have Residents! I might do something with JSON (Javascript Object Notation) and indeed I have.

But why would a residency on Batch Performance concern itself with JSON (and indeed XML, which I’ve also written about in the Redbook)?

The reason lies in the word “modernisation”. This actually works two ways:

• Effective job cloning – where there is some kind of “printed” output – requires breaking the report data generation and report formatting into separate pieces. This is because there’s a need to fan in the reporting data. This re-engineered data flow provides the opportunity to publish the data to new consumers. If we’re going to do that it might as well be something nice and modern like JSON or XML. I’ve talked about XML before – so I won’t in this post.
• Modernising batch jobs means opening the code up anyway (and that might indeed be to produce new formats of output) so it would be good to consider whether it should be parallelised. And the most obvious way is by cloning it.

Now obviously not all batch jobs want modernising or cloning. But some in an installation probably do.

---

So below is a simple example of using DFSORT to create JSON from SYSIN. Consider the following JCL:

    //MAKEJSON EXEC PGM=ICEMAN 
    //SYSOUT   DD SYSOUT=* 
    //SYSPRINT DD SYSOUT=* 
    //SYMNOUT  DD SYSOUT=* 
    //SYMNAMES DD * 
    POSITION,1 
    NAME,*,8,CH 
    SKIP,1 
    NUMBER,*,8,CH 
    /* 
    //SORTIN   DD * 
    ALPHA    ONE 
    BRAVO    TWO 
    CHARLIE  THREE 
    DELTA    FOUR 
    /* 
    //SORTOUT  DD SYSOUT=* 
    //SYSIN    DD * 
    OPTION COPY 
    * 
    INREC IFTHEN=(WHEN=INIT,BUILD=(SEQNUM,4,BI, 
              C'{"name": "',NAME,C'","number": "',NUMBER,C'"}')),
            IFTHEN=(WHEN=(1,4,BI,GT,+1),BUILD=(2X,C',',5,70)), 
            IFTHEN=(WHEN=(1,4,BI,EQ,+1),BUILD=(2X,5,70)) 
    * 
    OUTFIL FNAMES=SORTOUT,REMOVECC, 
     HEADER1=('{'/, 
    '"inventory": ['), 
    TRAILER1=(']',/, 
    '}') 
     END 
    /*

On my system it produces:

{
"inventory": [
  {"name": "ALPHA   ","number": "ONE     "}
  ,{"name": "BRAVO   ","number": "TWO     "}
  ,{"name": "CHARLIE ","number": "THREE   "}
  ,{"name": "DELTA   ","number": "FOUR    "}
]
}

If you put that through a JSON validator, such as JSONLint it is reported as clean JSON. (This particular service reformats it prettily as well.)

There’s a trick here, though, that’s worth describing:

JSON is picky in that – for elements or arrays – you have commas in between but you can’t have a leading or a trailing comma separator. (This actually isn’t always true of Javascript but was enforced for (the derivative) JSON.

All the interesting action is in the INREC (could’ve been OUTREC or even OUTFIL OUTREC) statement. This has three IFTHEN clauses (or stages if you prefer):

1. Always fires. Produces the formatted line with a 4-byte sequence number on the front. The sequence number starts at 1 and is in binary format.

2. Fires if the sequence number is greater than 1. Places a comma (and two indenting spaces) in front of the formatted line.

3. Fires if the sequence number is 1. Just places the two spaces (and no comma) in front of the formatted line.

I say “trick” but this is just the standard “treat each record according to its characteristics and through multiple stages” approach you can take with DFSORT IFTHEN.

In any case it produces 1 line without a comma and the following ones with a comma. And JSON rules are satisfied.

---

By the way you might be wondering why the values are capitalised or have trailing spaces. This is actually preserving what was in the original records (in in-stream SORTIN). You can certainly take trailing spaces out but it needs a little more work. Semantically both belong in the output data, of course.

And if you insist on taking trailing spaces off the lines – as opposed to out of the items – you can always use DFSORT’s VLTRIM.

I think it looks a little odd to have the separator commas at the beginning of the lines. But I don’t know of a way in DFSORT to test for the last line – to avoid placing a comma on it. If you can think of a way let us know.

But for now we have valid JSON that any JSON reader – whether raw Javascript, a framework like jQuery or Dojo, or some other language – can process.

And yes it would be nice to breathe new life into old data.

And – below the line 🙂 – is a brief discussion on JSON itself.

---

JSON is a spin-off from javascript. As I mentioned above you can process it with javascript by assignment. The following is entirely valid:

var inv={
"inventory": [
  {"name": "ALPHA   ","number": "ONE     "}
  ,{"name": "BRAVO   ","number": "TWO     "}
  ,{"name": "CHARLIE ","number": "THREE   "}
  ,{"name": "DELTA   ","number": "FOUR    "}
]
}
alert(inv.inventory.name[0])

which would – in a browser pop up a message box with “ALPHA” in it.

You mightn’t want to do that as you can end up executing arbitrary code. So direct assignment should only be used where you can trust the JSON. Which is why people tend to use libraries to parse it – and the good ones don’t do direct assignment.

People like JSON because it’s easy to generate, less verbose than XML, and now has a lot of ways of processing it. What’s not so good is that there is no notion of things like namespaces and schemas – so perhaps not so good for “Enterprise”. And it doesn’t have transformational tooling like XSLT. But it’s very popular.