Search and Replace with the new DFSORT functional PTFs

(Originally posted 2006-04-20.)

Here’s my first tip with the new UK90006 / UK90007 DFSORT PTFs. It’s an apparently minor enhancement that I think people will like…

• It leads to a more elegant syntactical expression.
• It is more maintainable – in the face of changing record layouts.

Suppose you want to search a file for a string in a particular position in each record and replace all matching instances with a different string. (I might do that to scrub an SMFID in SMF records to produce sample reports.) Without DFSORT’s Symbols support (which is ancient history now) you might have to code something like…

OUTREC IFTHEN=(WHEN=(4,5,CH,EQ,C'84484'),OVERLAY=(4:C'ABCDE'))

In this example the field you are searching in is a 5-character string beginning in position 4. Hence 4 appears twice in the statement.

NOTE: I’ve already used two functions that are much more recent than the introduction of Symbols: IFTHEN and OVERLAY…

• IFTHEN applies a BUILD (also know as FIELDs) or an OVERLAY clause to only the records that meet the IFTHEN condition. In this case

  4,5,CH,EQ,C'84484'

  specifies which field to search in.
• OVERLAY changes only the specified fields. In this case

  4:C'ABCDE'

  overlays just the 4-byte field we previously searched in. Prior to OVERLAY you had to explicitly specify all the fields you wanted to appear in the output record, even if only one was changed – as is the case here.

Now, that works pretty well – because of these two new functions introduced with PTFs UQ95213 / UQ95214 at the end of 2004. But suppose the format of the record changes. You would have to recode all such utterances for the new field positions. It would be much better to do it in one place. And that, in a nutshell, is the case for Symbols…

Continuing with our example, you might map the record with a SYMNAMES DD that points to the following definitions…

F1,*,1,CHF2,*,1,CHSKIP,1   F3,*,5,CH

If you work through these definitions you’ll realise that F3 starts at position 4 and is of length 5 bytes. So it matches our original example. Prior to UK90006 / UK90007 you could code:

OUTREC IFTHEN=(WHEN=(F3,EQ,C'84484'),OVERLAY=(4:C'ABCDE'))

and the IFTHEN clause is fully symbolic (ignoring the literal comparison string – which could itself be converted to a symbol.)

But suppose the data was reorganised so that this field moved. You’ll notice that still requires you to change the OVERLAY clause to replace the 4 with a different position.

With UK90006 / UK90007 you can now code

OUTREC IFTHEN=(WHEN=(F3,EQ,C'84484'),OVERLAY=(F3:C'ABCDE'))

and a reorganisation of the data just means changing the Symbols deck pointed to by the SYMNAMES DD. If you’re sensible this will be in one permanent file and so it’s just one edit (plus, of course, some testing).

What has changed is the ability to specify a position (for example for OVERLAY) using a symbol.

So, I think I’ve made the maintainability point. I think this enhancement also allows more elegant expression: You now refer to the field consistently by its symbolic name, rather than in two different ways in two different places.

As I said, it’s a minor enhancement. But one I particularly like – as I mess around with DFSORT a lot and little annoys me more than clunkiness of expression and poor maintainability.

One final point: I mentioned above that you could replace the literal string values with symbols. Here’s how to do it (in this example). Add two symbol definitions to the SYMNAMES DD:

OLDVALUE,C'84484'NEWVALUE,C'ABCDE'

and replace the statement with

OUTREC IFTHEN=(WHEN=(F3,EQ,OLDVALUE),OVERLAY=(F3:NEWVALUE))

and now you can change the search and replacement string values without changing all the occurrences of these strings in all your code – just in the one place.