Playing with Raspberry Pi and the Azure Event Hub

Azure Service Bus started with Relay services and then grew with Queues and Topics. The latest feature is the Event Hub, which is designed to handle million of devices sending and enormous amout of messages concurrently. This is known as the Internet of Things when little or big devices out there are connected and ingest data for telemetry purposes. The idea is naturally that you at the end increase value of you products somehow based on the insights you get from the data.

One “thing” that recently has caught much attention is Raspberry Pi. It’s InstagramCapture_8d1f9e9d-764a-41a3-bf79-9d2a6ea04560a ~35 USD computing device in the size of a credit card that you can attach USB-devices to, screens via HDMI and an ethernet cable. On a MicroSD card you can have the OS to boot the tiny machine. It is simply really cool and brings back memories of how companies like Microsoft and Apple got started and the stories about Altair 8800 and Homebrew Computer Club. What could be a more perfect combination than sending messages to Azure Event Hub from Raspberry Pi?

One common OS that runs on Raspberry Pi i the Linux distro Raspbian. You download Raspbian and copy it to your MicroSD card and boot the board. Included on the Linux distro is the java compiler, so writing an Azure Event Hub client is perfectly possible with that. In the AzureSDK for java, there is support for Service Bus if you are using Queues or Topics, but there is no support yet for the Event Hub. So, writing a client program would require using the raw REST API for the Event Hub.

AzureEventHubsOverview

In the Azure documentation there is sample code for the Event Hub with one eventhub processor (receiver) and two types of clients, where one is using the REST apis in C#. My plan was to convert that sample into java.

Authentication using SAS tokens

Service Bus authentication is nowadays about Shared Access Signatures which is a token you attach in the http header in the REST call. You create the SAS Key in the management portal, give it a name and specify what permissions it has on the event hub namespace. For an eventhub client, the send permission is all that is needed.

SAS-policy

 

In code, generating the SAS token is about Hash-based Message Authentication Code and Secure Hash Algorithm, where the url to the eventhub is encrypted with HMACSHA256 using the SAS key. Finding code samples to do this in C# on the internet isn’t hard. Translating it to java was a more difficult task to complete and after multiple attempts following suggestions on different java programming forums, I only got HTTP status code of 401 – Access Denied – from Azure Event Hub. My SAS token was malformed.

dotnet-SAS-token

Since SAS tokens are needed for Service Bus Queues and Topics, the guys as MS OpenTech had already solved the problem and the answer was somewhere inside AzureSDK. So with a little use of a java decompiler, my problem with the 401’s was solved. The missing piece was the use of the DatatypeConverter.printBase64Binary.

java-SAS-token

The token created then need to be set in the HTTP headers of the request with the name of Authorization. Content-type of the request must be set to application/atom+xml;type=entry;charset=utf-8.

java-headers

Sending messages to Azure Event Hub from the Raspberry Pi device

Besides following the installation instructions of NOOBS on Raspberry Pi, including getting it’s wlan to work, was really just the below, ie two simple steps

  • Downloading the Azure Java SDK using WGET and unzipping it to it’s own directory
  • Downloading my java source file and the accompanying config file (both which are available at the bottom of this page)

With that in place, I could compile and run the little java program and send a message to the event hub. It looked like this

java-running

On my laptop and in Visual Studio, I ran the Azure sample program doing the event processing. That program picks up the message sent from the little Raspberry Pi device from the event hub. As you can see I sent a few more messages.

dotnet-running

 

Summary

Using Raspberry Pi as an example, I showed you that you can write programs that only use the public REST APIs and integrate with Azure Event Hub. The program is written with “basic” java, that is, I used nothing from Microsoft and no jar-files from the AzureSDK. I can now add sensors or cameras to my Raspberry Pi device and start feeding in some real data. The architecture would scale up to million of devices ingesting whatever telemetry data they capture. Microsoft has a hands-on lab website where you can get ideas of all sorts of IoT projects you can build in the Pi.

Source Code

http://data.redbaronofazure.com/public/AzSbEvtHub.zip

References

Get Started with Event Hub – Microsoft Documentation and sample code
https://azure.microsoft.com/en-us/documentation/articles/service-bus-event-hubs-csharp-ephcs-getstarted/

MSDN – Event Hubs Overview
https://msdn.microsoft.com/en-us/library/azure/dn836025.aspx

TechEd 2014 – Clemens Vasters session on Hyper-Scale with Event Hub (a must see)
http://channel9.msdn.com/Events/TechEd/Europe/2014/CDP-B307

Weather station project on Raspberry Pi
https://microsoft.hackster.io/windowsiot/build-hands-on-lab-iot-weather-station-using-windows-10