The Right Curves?

(Originally posted 2015-08-17.)

In true IBM fashion this post features graphs without scales. [1] My aim is to share a rhetorical device I used in a recent customer workshop. I hope you find it useful. [2]

The customer was worried about a piece of hardware whose responsiveness had deteriorated over the past year, but got better in recent days, coinciding with some tuning changes they’d made. This graph shows that:

I’ve abstracted to “rate” and “response” to avoid getting specific. Specificity isn’t helpful here. You’ll see that the rate also dropped roughly concurrently with the response.

So could the rate drop have caused the response improvement all by itself?

Or did the tuning efforts actually make a difference?

It’s very hard to tell so I posited a different way of graphing it. I drew on their whiteboard a graph similar to the following:

Instead of drawing a pair of curves of response and rate (or load) against calendar time line:

1. Divide the timeline up into sections, with each section after the first being marked by a single tuning action.
2. Plot each section as a fresh line on the same graph, with the x axis being load / rate and the y axis being response.

In the (sketched) example there are three curves:

1. Baseline – before any tuning. (“Original”)
2. With one tuning action. (“Tune 1”)
3. With a second tuning action. (“Tune 2”)

If you can achieve a set of curves like this you can see the effect of tuning actions. In this case the first tuning action was clearly effective but the second had no effect (being essentially the same curve).

I think this is a handy technique, but there are a pair of issues that come readily to mind:

1. These sections might “go stale” after a while. For example, other changes that you didn’t take into account might happen within the life of the curve. An application change nobody told you about [3] could affect performance.
2. Getting enough data points could be tough. It’s tempting to go to 24 hours of 15-minute data points from, say, daily average data points. Care is required with this. Maybe restrict this to Prime Shift only, for example.

These issues need thinking about, but I don’t think they invalidate the idea. And more and more I’m plotting things like Response versus load / rate rather than against time. [4]