Pi As A Protocol Converter

I wrote about Automation with my Raspberry 3B in Raspberry Pi As An Automation Platform. It’s become a permanent fixture in my office and I’ve given it another task. This blog post is about that task.

Lots of things use JSON(Javascript Object Notation) for communication via HTTP. Unfortunately they don’t all speak the same dialect. Actually:

1. They do; It’s JSON pure and simple. Though some JSON processors are a little bit picky about things like quotes.
2. It’s just as well there is the flexibility to express a diverse range of semantics.

This post is about an experiment to convert one form of JSON to another. When I say “experiment” it’s actually something I have in Production – as this post was born from solving a practical problem. I would view it as more a template to borrow from and massively tailor.

The Overall Problem

I have a number of GitHub repositories. With GitHub you can raise an Issue – to ask a question, log a bug, or suggest an enhancement. When that happens to one of my repositories I want to create a task in my task manager – OmniFocus. And I want to do it as automatically as possible.

There isn’t an API to do this directly, so I have to do it via a Shortcuts shortcut (sic) on iOS. To cause the shortcut to fire I use the most excellent PushCut app. PushCut can kick off a shortcut on receipt of a webhook (a custom URL) invocation.

Originally I used an interface between GitHub and IFTTT to cause IFTTT to invoke this webhook. This proved unreliable.

The overall problem, then, is to cause a new GitHub issue to invoke a PushCut webhook with the correct parameters.

The Technical Solution

I emphasised “with the correct parameters” because that’s where this gets interesting:

You can set GitHub up – on a repository-by-repository basis – to invoke a webhook when a new issue is raised. This webhook delivers a comprehensive JSON object.

PushCut webhooks expect JSON – but in a different format to what GitHub provides. And neither of these is tweakable enough to get the job done.

The solution is to create a “protocol converter”, which transforms the JSON from the GitHub format into the PushCut format. This I did with a Raspberry Pi. (I have several already so this was completely free for me to do.)

Implementation consisted of several steps:

1. Install Apache web server and PHP on the Pi.
2. Make that web server accessible from the Internet. (I’m not keen on this but I think it’s OK in this case – and it is necessary.)
3. Write a script.
4. Install it in the /var/www/html/ directory on the Pi.
5. Set up the GitHub webhook to invoke the webhook at the address of the script on the Pi.

Only the PHP script is interesting. You can find how to do the rest on the web, so I won’t discuss them here.

PHP Sample Code

The following is just the PHP piece – with the eventual shortcut being a sideshow (so I haven’t included it).

<?PHP

$secret = "<mysecret>";
$json = file_get_contents('php://input');
$data = json_decode($json);

if($data->action == "opened"){
  $issue = $data->issue->number;
  $repository = $data->repository->name;
  $title = $data->issue->title;
  $url = $data->issue->html_url;

  $pushcutURL = "https://api.pushcut.io/" . $secret . "/notifications/New%20GitHub%20Issue%20Via%20Raspberry%20Pi";

  //The JSON data.
  $JSON = array(
      'title' => 'New Issue for ' . $repository,
      'text' => "$issue $title",
      'input' => "$repository $issue $url $title",
  );


  $context = stream_context_create(array(
    'http' => array(
      'method' => 'POST',
      'header' => "Content-Type: application/json\r\n",
      'content' => json_encode($JSON)
    )
  ));

  $response = file_get_contents($pushcutURL, FALSE, $context);
}

?>

But let me explain the more general pieces of the code.

• Before you could even use it for connecting GitHub to PushCut you would need to replace <mysecret> with your own personal PushCut secret, of course.
• $json = file_get_contents('php://input'); stores in a variable the JSON sent with the webhook. Let’s call this the “inbound JSON”.
• The JSON gets decoded into a PHP data structure with $data = json_decode($json);.
• The rest of the code only gets executed if $data->action is “opened” – as this code is only handling Open events for issues.
• The line $pushcutURL = "https://api.pushcut.io/" . $secret . "/notifications/New%20GitHub%20Issue%20Via%20Raspberry%20Pi"; is composing the URL for the PushCut webhook. In particular note the notification name “New GitHub Issue Via Raspberry Pi” is percent encoded.
• The outbound JSON has to be created using elements of the inbound JSON, plus some things PushCut wants – such as a title to display in a notification. In particular the value “input” is set to contain the repository name, the issue number, the original Issue’s URL, and the issue’s title. All except the last are single-word entities. If you are adapting this idea you need to make up your own convention.
• The $context = and $response = lines are where the PushCut webhook is actually invoked.

As I said, treat the above as a template, with the general idea being that the PHP code can translate the JSON it’s invoked with into a form another service can use, and then call that service.

Conclusion

It was very straightforward to write a JSON converter in PHP. You could do this for any JSON conversion – which is actually why I thought it worthwhile to write it up.

I would also note you could do exactly the same in other software stacks, in particular Node.js. I will leave that one as an exercise for the interested reader. I don’t know whether that would be faster or easier for most people.

On the question of “faster” my need was “low volume” so I didn’t much care about speed. It was plenty fast enough for my needs – being almost instant – and very reliable.

One other thought: My example is JSON but it needn’t be. There need not even be an inbound or outbound payload. The idea of using a web server on a Pi to do translation is what I wanted to get across – with a little, not terribly difficult, sample code.