Getting to know MessagePack as an efficient alternative to JSON

JSON or JavaScript Object Notation is now one of the most popular formats of transmitting data objects, almost if not entirely replacing XML, because of its "very" human-readable, standard text format using the key-value pair structure.  The language-indepent data format was based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999 and was later popularized by Douglas Crockford on around 2001.

Today, JSON is already a standard and is used in many systems, in web, mobile, websockets, etc., even in Relation Database Management Systems and NoSQL databases.

But the innovation did not stop with JSON. Others have created alternatives to minimize the unnecessary characters and the size of the data object/s. One of them is MessagePack.

It's like JSON.
but fast and small.

MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it's faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.

MessagePack, the data is compact and therefore faster to transmit, in any way you want to transmit it, in bytes instead of string. And the beauty of it is that serialization and deserialization is fast.

Usages of MessagePack

You can use the MessagePack Specification in almost anywhere you want to use it. Some have used them in together with caching, rpc's, sockets, and data representation across multiple platforms.
Others have used MessagePack with Redis, with Memcached, and there is also an RPC project based on MessagePack.

Why use MessagePack?

• It is small in size, compact, and therefore efficient
• A little more efficient
• You can create application-specific type using its ext type family
• And what you can do with JSON you can do with MessagePack

Just for the record, MessagePack was not made mainly for consumer-facing API's.

What languages support it?

Basically, MessagePack specification is just encoding data into bytes, so most of, if not all, languages supports it. It's just the matter of implementing it. But don't worry! most of the popular languages already has MessagePack library, check this out.

MessagePack's specification is properly documented in here.

Usage Insight

Here is a little example of messagepack compared to json, serialization and deserialization, using the python msgpack and json library.

Install the library using pip:

$ pip install msgpack-python

#!/usr/bin/python

# msgpack-test.py

import msgpack
import timeit


result = msgpack.packb([1, 2, 3])
print 'output:', result
print "Avg time x 10e3 msgpack:", timeit.timeit("import msgpack; msgpack.packb([1, 2, 3])", number=10000)
print "Avg time x 10e3 json:", timeit.timeit("import json; json.dumps([1, 2, 3])", number=10000)
# output: 'x93x01x02x03'

result = msgpack.unpackb(b'x93x01x02x03')
print 'noutput:', result
print "Avg time x 10e3 msgpack:", timeit.timeit("import msgpack; msgpack.unpackb(b'x93x01x02x03')", number=10000)
print "Avg time x 10e3 json:", timeit.timeit("import json; json.loads('[1, 2, 3]')", number=10000)
# output: [1, 2, 3]

result = msgpack.unpackb(b'x93x01x02x03', use_list=False)
print 'noutput:', result
print "Avg time x 10e3 msgpack:", timeit.timeit("import msgpack; msgpack.unpackb(b'x93x01x02x03', use_list=False)", number=10000)
# output: (1, 2, 3)

$ python sample.py
output: ▒
Avg time x 10e3 msgpack: 0.129597902298
Avg time x 10e3 json: 0.0673019886017

output: [1, 2, 3]
Avg time x 10e3 msgpack: 0.17978978157
Avg time x 10e3 json: 0.0450839996338

output: (1, 2, 3)
Avg time x 10e3 msgpack: 0.19033408165

Conclusion

Use MessagePack depending on your needs and if your specification says that size should really matter. Remember that it was not created mainly for consumer-facing API's. For me, I would use MessagePack in sockets, rpc, caching mechanisms and data representation. There are other more ways to utilize MessagePack.

You would want to experiment first to get the grasp of it and decide on how to utilize it properly. Research more about it too.