Azure ServiceBus Message Payload Serialization using protobuf

| Comments

Choices are between ready made coffee maker and make it ourselves available in Windows Azure kitchen.  As long as we want cappuccino, Windows Azure .NET libraries are good to go in terms of productivity and maintainability.  Sometimes, we may need to prepare blended iced cappuccino.  REST API (the actual service interface to Windows Azure services) is the way for that.  Here, I am talking about Windows Azure ServiceBus queue and how to use custom serialization on message payload (or body).

My Blended Iced Cappuccino

One of the best and base element in WCF / .NET serialization is DataContractSerializer.  This is some time nightmare to the developers too.  Though there are various serializers in .NET stack, when SOAP or XML comes into the picture, DataContractSerializer is the natural option. In the ServiceBus brokered messaging, the managed library close the option to use serializer other than DataContractSerializer for message payload.

This is not the common case in the messaging world.  A queue may be designated for receiving document type message or command type message.  For command type messages, DataContractSerializer is fine.  When the message type is document, both sender and receiver can agreed upon specific content-type of the message payload.  For this, REST is the best friend.

Solution

To send a message, Windows Azure ServiceBus REST API requires the following:

  • URI – http{s}://{serviceNamespace}.servicebus.Windows.net/{queue path}/messages
  • Method – POST
  • Header – Authorization header with WRAP token as value
  • Request Body – could be anything

If everything going well, this web request returns 201.

To receive the message,

  • URI – https://{serviceNamespace}.servicebus.Windows.net/{queue path}/messages/head?timeout={seconds}
  • Method – POST (peek n lock) or DELETE (destructive)
  • Header – Authorization header with WRAP token as value

This would returns message properties (for destructive nothing will be returned) and payload with response code 200.

In this case, there is no restriction on which serialization to be used on message payload.  So, we can use protobuf.  The main reason is content size.  When a sender sends the message with HTTP content-type as application/protobuf,   the receive always gets the message with the same content-type.

In this demonstration, I have created a message payload as shown below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

[ProtoContract]
public class Person
{
[ProtoMember(1)]
public string Name { get; set; }
[ProtoMember(2)]
public int Age { get; set; }
[ProtoMember(3)]
public Address Address { get; set; }
}

[ProtoContract]
public class Address
{
[ProtoMember(1)]
public string City { get; set; }
[ProtoMember(2)]
public string Street { get; set; }
}

I’ve decorated “Person” class with proto-buf attributes (I have used http://code.google.com/p/protobuf-net/).

Let us see how we can use protobuf.

Start with two constant declaration,

1
2

const string ACS_HOST_URI_PART = "accesscontrol.windows.net";
const string SERVICE_BUS_URI_PART = "servicebus.windows.net";

WRAP access token is required to interact with Azure REST APIs.  Assume, the method CreateWrapToken(…) will generate based on issuer name and secret.

1
2

string sbAddress = "https://<namespace>." + SERVICE_BUS_URI_PART + "/";
string token = CreateWrapToken("<namespace>", "<issuer>", "<secret>");

To send a message to a queue,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

var request = HttpWebRequest.Create(sbAddress + "personq/messages");
request.Method = "POST";
request.Headers[HttpRequestHeader.Authorization] = token;

using (MemoryStream mem = new MemoryStream())
{
  var person = new Person
  {
    Name = "Sheik",
    Age = 30,
    Address = new Address
    {
      City = "Chennai",
      Street = "Mount Street"
    }
  };

  request.ContentType = "application/protobuf";
  var stream = request.GetRequestStream();
  Serializer.Serialize<Person>(stream, person);
  var response = request.GetResponse() as HttpWebResponse;
  Console.WriteLine(response.StatusCode);
}

The person instance is serialized on the request stream.  To receive a message (here I’m using destructive approach):

1
2
3
4
5
6
7
8

request = HttpWebRequest.Create(sbAddress + "personq/messages/head");
request.Method = "DELETE";
request.Headers[HttpRequestHeader.Authorization] = token;

var response2 = request.GetResponse() as HttpWebResponse;
Console.WriteLine(response2.StatusCode);
var stream2 = response2.GetResponseStream();
var person2 = Serializer.Deserialize<Person>(stream2);

The person2 contains the deserialized content of Person.

If we use data contract serializer, the overall message size 302 bytes.  If we use protobuf, it is just 171 bytes.

The code for CreateWrapToken(…) is

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

public static string CreateWrapToken(string serviceNamespace, string issuerName, string issuerSecret)
{
var acsUri = "https://" + serviceNamespace + "-sb." + ACS_HOST_URI_PART + "/WRAPv0.9/";
var realm = "http://" + serviceNamespace + "." + SERVICE_BUS_URI_PART + "/";

var values = new NameValueCollection();
values.Add("wrap_name", issuerName);
values.Add("wrap_password", issuerSecret);
values.Add("wrap_scope", realm);

var webClient = new WebClient();
byte[] response = webClient.UploadValues(acsUri, values);
string responseString = Encoding.UTF8.GetString(response);
var responseProperties = responseString.Split('&');
var tokenProperty = responseProperties[0].Split('=');
var token = Uri.UnescapeDataString(tokenProperty[1]);
return "WRAP access_token=\"" + token + "\"";
}