(Originally posted 2008-06-25.)

Many of you will have, by now, installed the Adobe AIR runtime. Most probably it will be to run something like Twhirl.

At this point many of you will be asking “what’s Twhirl?”

If I said it was a nice desktop application that makes using Twitter so much easier I hope you don’t ask “what’s Twitter?” 🙂

So, we’re beginning to see these desktop applications coded using Adobe AIR, which stands for “Adobe Integrated Runtime” (formerly “Apollo”).So, what’s special (if anything) about AIR?

In my experience, two things:

• AIR applications run across Mac, Windows (and in Alpha form) Linux.

  (In fact I installed the Linux AIR alpha plus Twhirl on my ASUS EEE PC some months ago. (Nice machine that EEE, by the way.) It worked fine within the limitations of the AIR alpha code.)

• It’s entirely possible to code up AIR applications using just a text editor. (in fact that’s precisely what I did.)

  My editor of choice, for whatever that’s worth, is Notepad++ on Windows. (And I’d rather not start an editor war here, thanks.) 🙂 And the EEE comes with the Kate editor, being built on Debian Linux.

  I like environments where there is little, if any, in the way of barriers to entry for programmers.

So what does an AIR application consist of?

In principle one can do fancy things with Adobe Flex Builder and .SWF files but that requires expensive tooling. But there is another way:

You can write an AIR application of arbitrary complexity with two files:

• A small XML file.

• Your HTML file which can also have javascript in, just like normal web pages. In fact your typical AIR HTML file could be thrown straight into a web browser and work just fine.(That is unless you use some of AIR’s special capabilities – which your javascript can detect the availability of and code around accordingly.) And I assume that when I say “your browser” I mean a standards-compliant one like Firefox. 🙂 Note: The flavour of AIR that uses HTML / Javascript is actually built around the Safari browser.

  So I really think the “this builds on what you already know” point makes it attractive to an awful lot of people.

This isn’t an AIR tutorial so I’m not going to give you samples of the small XML and html files. You can easily find those on the Web. I just want to leave you with the impression there isn’t much to it. And there is an O’Reilly pocket guide for AIR. (But there isn’t a “For Dummies” book for it – so that’s alright.) 🙂

There are alternatives – such as Mozilla’s XULRunner (whose user interface is based on XUL rather than HTML) and, I suppose, Microsoft’s Silverlight. But neither is as easy and “I just need a text editor and the SDK to do it” is pretty enticing.

So, I’m impressed at its simplicity, the fact it builds on HTML, XML and Javascript. And if you want to be able to build something that looks like a desktop application, complete with drag-and-drop and clipboard capabilities, this may well be a good fit for your needs.

I mentioned the SDK. It’s free (but it doesn’t run on Linux and I’ve not heard any suggestion that it will). It comprises two useful tools:

• ADL – which you use for testing
• ADT – which you use for packaging up your completed application. This involves signing the resulting .AIR file – and the ADT tool has the capability to enable you to be “self certifying” so that gets round the cost of getting a certificate from a more formal authority. But it does require users to trust you. 🙂

Both are simple to use.

So, I got a “Hello World” application up and running in about twenty minutes, and that even with typ(o)ing in the XML and HTML files from the O’Reilly book. Swiping from the Web would probably be quicker – but you’d learn less.

And now I’m proving remarkably productive at writing a full-scale application. What’s slowing me down – if anything – is my very basic CSS. And this application uses XMLHttpRequest and walks the DOM tree to build the window contents on the fly. All using standard javascript. Oh, and my CSS is rather more under control. 🙂 I just need to refactor it into multiple parts as all my CSS / HTML / Javascript is in the one file. Oh, and then there’s refactoring to use dojo. I’m told you can use dojo in an AIR application, so long as you ship the dojo runtime with it.

If you want to know more about AIR go to their blog. For Twhirl go here and, of course, Twitter is here.

Note: IBM doesn’t appear to have any commercial interest in AIR. And I certainly don’t. So I’m just telling you how I see it.

(Originally posted 2008-06-21.)

The same Kevin Keller I mentioned in this blog post is running a one day pilot of a “Web 2.0 and System z” workshop on July 8th, in IBM Böblingen. It’s designed for customers, though some of the audience will undoubtedly be IBMers. I’ve looked at the agenda and, while I’m on it, it’s a great agenda. 🙂 Actually it has the highly-acclaimed Luis Suarez on the agenda as well. (I’m looking forward to meeting Luis as, though we’ve talked many times using all the social media, we’ve never actually met before.)

If you want to attend the workshop – or would like more information – let me know and I’ll see what I can do. I don’t control the attendee list so I can’t promise you a place.

Though run by IBM Germany this workshop is in English.

I’m looking forward to this class tremendously: As my long-suffering audiences 🙂 know I’m keen that people and organisations realise there’s a lot of scope for doing Web 2.0 (or, more generally, modern) stuff on z/OS. So this workshop ought to be the proof of that.

(Originally posted 2008-06-20.)

Take a look at my German colleague Kevin Keller’s new blog in Web 2.0 and z/OS. I think you’ll be pleasantly surprised at what runs on z/OS that the rest of the world would regard as modern. 🙂

I say “modern” rather than “Web 2.0” because I feel the latter term is generally overused. What is really to the point is quite how much stuff could (and in many cases should) be implemented on z/OS, e.g. accessing DB2 (especially with the PureXML support in V9).

I’ve suggested to Kevin he might like to write additional posts on some of the products mention in his first mammoth post. I think some of them are going to prove remarkably easy to implement.

(Originally posted 2008-06-19.)

Hats off to Jonathan Feinberg (who’s created a number of neat sites inside IBM) for creating Wordle.

The above is a nice depiction of the keywords for my blog. Click on it for a bigger version.

Go to here to make your own from any sets of text you like.

(Originally posted 2008-06-16.)

I’m not sure if anyone’s done this before. Certainly I’ve not seen any results…

We’re beginning to write up some tests the “A Team” (Alain and Pierre) ran before their return to France. Which provides me with some test data. Fortunately I had a stab at mapping the new fields this data contains. One in particular is Stucture CPU (R744SETM)…

So, I got to plotting CPU per request against request rate (as well as elapsed time against request rate). I did this for two structures:

• LOCK1 – which has between 750 and 925 requests a second – based on 1 minute RMF intervals.
• ISGLOCK – which has between 5 and 65 requests a second.

The high request rate LOCK1 case is less interesting for the purposes of this discussion. The chart below is for ISGLOCK. I think it’s revealing.

The red line is the CPU time per request. Most importantly it goes down with the number of requests. That strongly suggests to me that there’s a “zero request rate” cost, which gets amortised over more requests as the rate increases. Now, some of this could just be Coupling Facility internal processes. Or it could be a cache effectiveness thing. And in any case maybe it’s RMF with its 1-minute intervals that’s driving the CPU consumption.

The blue line is the service time per request – which actually drops as the rate increases. This might be a “practice effect” or else some instrumentation effect. At any rate it drops rather less dramatically than the CPU time per request.

What’s also interesting is that the CPU time and the service time converge somewhat. You’d expect the service time to be more than the CPU time – for a request. But, as you can see from this blog entry that isn’t always the case.

So, the net result of this test is that there’s CPU to amortise over the requests and at lowish rates this can be enough to make it longer than the service time.

For ISGLOCK – locking with no record data payload – it’s pretty much all CPU time, so long as nothing else gets in the way. (Such as path / subchannel delays or extreme CPU queuing in the coupling facility. Or, more to the point, link latency.)

For LOCK1, the request rate was much too high to see the amortisation in action and CPU was always about 4μs vs service time of around 10μs per request. LOCK1 does have record data to manipulate.

There’s one other consideration, though: LOCK1 was in a coupling facility with a mixture of ISC and ICB links. ISGLOCK was in a coupling facility accessed using IC links. That probably accounts for some of the 10μs. for LOCK1 vs 3.5μs for ISGLOCK. In fact, the CPU time per request for LOCK1 was about 4μs out of the 10μs. Which suggests quite a bit of link latency.)

It’s a new-found frustration of mine that the instrumentation doesn’t tell me much about traffic by link or link type. (Earlier in the residency I messed around with channel activity and queuing record types. Perhaps I’ll have to mess with it again.)

In “real world” terms bear in mind, however, that the 50 requests per second and upwards cases are much more interesting and common. So typical behaviour is towards the right hand end of the graph and beyond.

(Originally posted 2008-06-12.)

It’s fair to say I’ve written very little in terms of book words these past two days. That’s because I’ve been “doing research” on how the cache structure counters really work. “Doing research” is a euphemism for “finding out how the heck the darned thing works”. Which is in itself a positive thing to do – but it does rather get in the way of writing stuff. 😦

There is a section in the SMF 74 Subtype 4 called the “Cache Data Section”. I thought you got one for each cache structure. It turns out you can get more than one. There are things inside cache structures called storage classes. So I trawled through all the customer data I have gathered over the past year and in only one> structure (not one customer) did I see a count of 2 sections. This for XBM. Even DB2, which uses multiple castout classes per structure, only has one storage class. So I don’t feel bad about my code assuming only one.

The other thing one needs to know – and it’s far more important – is that for every system connecting to a structure the Cache Data Section contains identical values. It’s rather like the Disk Cache statistics in SMF 74 Subtype 5: The controller / coupling facility maintains the stats. So it’s no use summing them.

When it comes to what’s in the section there are umpteen counters: 5 read counters, 5 write counters, 4 cross-invalidation counters and umpteen others. In my code we calculate read hit percentages, read-to-write ratios and lots of other things besides. Figuring out what all these counters mean and which ones are important is what’s euphemistically called “a work in progress”. 🙂

But the real learning point is that each exploiter does it differently. So, two examples:

• A store-through cache doesn’t do castouts. A store-in one does. And then there’s a “cross invalidation” cache that doesn’t either. But then it doesn’t actually store data to consider the fate of.
• DB2 Data Sharing with GBPCACHE(CHANGED) doesn’t write unchanged data to the cache structure.

And so it goes on.

I think it would be a noble aim to describe all these counters in terms that are related to exploiters. And also in terms we can all understand. Let’s hope it doesn’t remain just a noble aim. 🙂

(Originally posted 2008-06-10.)

I had to eat a little bit of humble pie today – because I made an elementary mistake. I’m going to share it with you – to save you making it, too. 🙂 And I’m very sure I’m not going to make it in a customer situation. (Residencies are great for making mistakes in a safe environment.) 🙂

In CFLEVEL 15 and z/OS Release 9 RMF you get a new field at the structure level: R744SETM.

This is the CPU used in the Coupling Facility to process requests to an individual structure. When you add the individual R744SETM values – for all the structures in the Coupling Facility – you get the same value as the sum of the R744PBSY times for all the processors. This is because R744PBSY is defined as the time spent processing requests. (You get this by individual coupling facility processor.) If you were to calculate a capture ratio using R744SETM and R744PBSY you’d always get 100% – so it’s probably not worth the bother. 🙂

Normally one calculates Coupling Facility busy using the formula 100*R744PBSY / (R744PBSY + R744PWAI) and summing over the processors. I’m beginning to think that 100*R744PBSY / SMF74INT is a more useful calculation (and it collapses down to the usual formula for dedicated processors).

So now to my mistake…

I wanted to compare the CPU time by structure to request service times. You can do this if you compare R744SETM to R744SSTM+R744SATM. (SSTM is the sum of Sync service times and SATM the same but for Async.) So I did this comparison and carefully noted that R744SETM is a 8-byte floating point number and the other two are 8-byte integers but that all three are in microseconds over the interval. What I failed to do is to realise that you need to add up the service times over all the z/OS systems connecting to the structure…

R744SETM is for all requests to the CF structure. The others are by z/OS system.

Why does this matter? Because I suffered the embarrassment 🙂 for several fraught hours of not knowing why the CPU time was longer than the elapsed time. 🙂

There are in fact a couple of cases where CPU time isn’t included in the service time. I intend to write those up in the book. But basically they are ones where z/OS has been given the “request complete” signal but there is still some processing to do. In these cases we continue to accumulate CPU but not service time. (And, yes, these cases are logically OK.)

So, I don’t feel too bad about my mistake, particularly as it led to some learning. And I look forward to making the same sorts of mistakes with Coupling Facility Cache Statistics – which are also collected at the “all systems” level. 🙂

(Originally posted 2008-06-04.)

Sometimes reading the SMF manual buys you less than you thought. I had completely ignored a nice little field – and the code I inherited to analyse SMF 74 Subtype 4 Coupling Facility data ignored it as well.

R744SLEC in the Request Data Section for a structure is described by the text “Lock structure only: lock table entry characteristic”. Can you guess what it is? 🙂 If I told you it was a one byte integer would that help?

If I had bolded the word “characteristic” would that have helped?

Probably only if you knew something about floating point numbers. I confess to knowing little about them other than the fact there is a “characteristic” and a “mantissa”. One’s the “power of n” thing, and the other’s whatever you multiply it by. But which is which? Anyhow…

So my first guess is that this field is the number of bytes in a lock table entry – plus one. I see values of 1 and 3 in it. So 2-byte entries and 4-byte entries. Right? Wrong! …

My friend “SuperMario” 🙂 points out to me that I’m looking at a GRS Star ISGLOCK) structure when I see the “3” and further that GRS Star always uses 8-byte lock table entries, no matter how few members (systems) connect to it.

The penny drops…

2³ is 8, right?

So the “characteristic” is really in this case the power of 2 that turns into the number of bytes in each lock table entry. So “3” means 8 bytes (as I just said) and “1” would mean 2 bytes (which figures for the plexes I see it for, being relatively small). So there’s that word “characteristic” again, and the field description (though terse) seems to make sense. 🙂

Now, why’s this important?

For two reasons:

• The installation has a choice of how many bytes to make each lock table entry, depending on the maximum number of lock table entries they want to cater for – without a rebuild. So, too wide a lock table entry and you waste lock structure space – potentially massively. Too narrow and you can’t connect all the systems (or e.g. DB2 members) you want to. Usually not a difficult trade off. But now I can wade into the discussion in a customer with evidence. 🙂
• The almost-neighbouring field R744SLTL gives the maximum number of lock table entries. Multiply the two together and you get the size of the lock table portion of the lock structure. The remainder – R744SSIZ minus the lock table entry size – is for record portion of the lock structure. There are various ways to make a hash (pun intended) of this. Which I won’t bore you with here. The point is you can do this calculation, armed with what the characteristic is.

Well, I thought this was a step forward in our understanding of how lock structures were actually allocated.

On the morrow (otherwise known as “presently”) I’ll write this up in the Redbook.

I’m taking the slightly unusual stance of assuming the readership can get the SMF manual out and follow along. Or at least wave around an SMF field name like it’s their “new best friend”. 🙂 Actually I’m using field names for clarity. There are – as I point out in the Redbook – at least 4 valid size fields for a structure. It’d be nice to be clear which one we’re talking about.

So we’re having fun in Poughkeepsie, going deep into the instrumentation. And we’re having nice discussions about what all the fields mean (and what they don’t mean). Here’s an example of what they don’t mean…

Field R744SSTA sounds like it ought to mean the number of Sync requests converted to Async. But in fact it’s almost always zero, despite the z/OS R.2 Dynamic Request Conversion – which we know goes on all the time. How do we know? I’m not sure really, except that in many situations we see plenty of Async requests where the exploiter is highly likely to have requested Sync. So this field turns out not to be the R.2 algorithm “in play”. It’s the older, possibly best dubbed “Static Conversion” algorithm. So, such things could easily trip you up. Or maybe it’s just me that stands to be tripped up. 🙂

Like I say, we’re having fun here in Poughkeepsie. 🙂 And if you happen to think this sort of thing is fun a residency could be right for you. Now, you might think as a customer (or other non-IBMer) you don’t have the possibility of being selected. In fact the “wall of fame” has photos of quite a few non-IBMers on it. No, I don’t know how it works with non-IBMer nominations. But it seems to be a non-zero-yield process.

(Originally posted 2008-05-27.)

The other three residents are busy doing extensive setup work – and we hope to have some nice measurements later on. (I’m not sure how much later on there actually is, mind.) 🙂

So I have no RMF or DB2 SMF to play with yet. 😦

>

Meanwhile I’m beginning to come up to speed again after the glorious experience of recovering from moving five time zones to the west. 🙂

So I actually got to writing today. And it’s the first actual content.

So I’ve concentrated on getting the structure of what I’m writing right. While my brain continues to accelerate. 🙂 I wonder whether I should use my brain in its present state to do the important work of defining the structure. After all that sets the tone and affects whether the material is accessible or not. Or whether to write some “disposable” content. After all it’s the content you want to read. But “disposable” in that I’ll probably consider today’s content trite and dull.

So I have written some stuff about the SMF records needed to do performance analysis for Data Sharing. This I consider to be “no brainer stuff”.

I’ve fretted a bit about comprehensibility and UK vs US English and whether I’m saying anything useful at all. :-)And how to get Framemaker to do my bidding (and whether that’s the right bidding). 🙂

In the end the best advice is to just get writing and to accept that much of the early stuff mightn’t survive.

And having recovered from almost hosing my Thinkpad last night I should just be grateful to be able to write at all. 🙂

And, finally, Twitter has proved today to be a useful resource for throwing out DB2 questions and getting them answered (with actual discussion). You can, as always, find me here. And if you do read Twitter you’ll discover the standard is pretty low – if my contributions are anything to go by.

(Originally posted 2008-05-23.)

So, I’m about to start writing. At last!

As a team we had a discussion yesterday about how to deal with performance numbers. One of the roles I’m playing on the team is “the guy who writes about performance numbers”. So we came to the following conclusions:

• We really can’t talk about products made by other people – such as SAS/MXG or TMON for DB2. That’s right out – because of our basic provenance and sponsorship.
• It’s not very helpful to customers to talk about eg Tivoli Performance Reporter (or whatever it’s called these days) much – as it’s not an assumed given. So it wouldn’t really help a SAS/MXG customer much if we talked about such stuff. Remember: This isn’t a Redbook about reporting tools. It is about how to manage Parallel Sysplex Performance. So we need to do that in a way that’s accessible to all customers.
• It seems to us reasonable to have one or two examples of RMF Postprocessor reports: We do assume access to RMF or at least the SMF record types that RMF produces – and CMF does do a pretty good job of creating the same records (though there may be some differences). So we don’t think we’re disenfranchising anyone with that.
• More than one or two RMF reports is unhelpful and makes the book unnecessarily turgid and besides it’s tough to format Postprocessor reports so they fit into a book page.
• SMF record fields are canonical in that they’re high up the data flow chain. So we think it’s fine to mention them – so long as we do it in a way that makes them comprehensible.

Taking all that into account we propose the following approach:

• Have a section that talks about performance numbers – including their sources and perhaps provenance. And in this section it’s OK to include a small number of RMF Postprocessor examples and maybe Omegamon XE for DB2 examples.
• For each test run that we document to use the field name or some alias for it. We’re assuming at this point that you’ve read and understood the “performance numbers” chapter.

The aim at the end of the day is to have a Redbook that works for you. So I hope this structure works for you. If you have thoughts on it feel free to comment below. Or to email me or comment at me via Twitter. My handle there is “MartinPacker”. But you’ll probably have to follow me first and thus nudge me into following you. As I think I’ve said before, I’d like to use the new media – eg Twitter and this blog to make the creation of this Redbook a more interactive and responsive affair.