N2O IoT

Protocols, Messages and Formatters

If you put regular JSON on the wire sooner or later you will discover your own wire protocol. Here is description of our N2O binary precise protocol. N2O also has ASN.1 formal description, however here we will speak on it freely.

The only thing I want to add is that N2O protocol was created naturally with niminal efforts, all modifications were applied targetly and atomically. The current version of the protocol 2.3 is totally compatible with earlier versions up to 1.5 the synrc/games version, which was very progressive at the time. Versions prior to 1.5 had very basic and naive but stable and fast implementation of the core. All N2O releases are known to be stable and supported half-year back. The evolution of the protocol is open for a deeper discussion with interested party or protocol users.

You may find N2O protocol similar to extensible XML-based XMPP, BINARY-based COM/CORBA and/or JSON-based WAMP; but most closer it is to BERT-RPC binary-based protocol introduced by the GitHub. N2O protocol is formatter agnostic and it doesn't strict you to use a particular encoder/decoder. N2O protocol has its own reference implementation in Erlang which is widely used in production applications. And experimental implementation in Haskell which is also using BERT formatter. N2O provides definition support for several unoverlapped sub-protocols. There is no single system shipped to support all of them but it could exist theoretically.

N2O should not define any particular set of formatters. Application developers could choose their own formatter per protocol. E.g. N2O uses TEXT formatting for <<"PING">> and <<"N2O,">> protocol messages, across versions N2O used to have IO message formatted with JSON and BERT both. All other protocol messages were BERT from origin. The only thing is that protocol formatter should be specified somehow.

TEXT
JSON
XML
BERT
MessagePack

All this formattes format message which comes directly to TCP WebSocket. However there are some endpoints which are not TCP sockets, even non-sockets, like gen_server endpoint, HTTP REST endpoint, which is non-WebSocket endpoint, and there is a room for other endpoint types. Here is a list of types of endpoints which arise in multi-protocol discussions:

gen_server
ws
http

The core protocols shipped with N2O provide very basic but yet powerful functionality such as reconnects, page initialization and common client/server messaging. The core is enough for binary upload/download, games construction and SPA applications.

HEART

The heart protocol defined client originated messages N2O, PING and server originated messages IO and NOP. IO message contains EVAL that contains UTF-8 JavaScript string and DATA reply contains any binary string. "PING" and "N2O," are defined as text 4-bytes messages and second could be followed by any text string. NOP is 0-byte acknowledging packet. This is heart essence protocol which is enough for any rpc and code transfering interface. Normally heart protocol is not for active client usage but for supporting active connection with notifications and possibly DOM updates.


1. IO tuple {io,EVAL,DATA}
2. NOP <<>>
3. N2O init marker <<"N2O,",Session/binary>>
4. PING message <<"PING">>

Heartbeat protocol is essential WebSocket application level protocol for PING and N2O initialization. It pings every 4-5 seconds from client-side to server thus allowing to determine client online presence. On reconnection or initial connect client sends N2O init marker telling to server to reinitialize the context.


ws.send('PING');
ws.send('N2O,');

IO events are containers for data and codata. Third element of a tuple will be directly evaluated in WebBrowser. IO events are normally not constructed on client.

Haskell


TupleTerm [AtomTerm "io", BinaryTerm eval, BinaryTerm data]

Erlang


{io,term(),term()}

SPA

The spa protocol defined client originated messages CLIENT, SERVER and server originated messages SERVER, IO and NOP. This protocol should be used for normal data transfer as enveloping protocol. Client and server DATA message should be specified for both server and client.


5. CLIENT tuple {client,DATA}
6. SERVER tuple {server,DATA}

Client messages usually originated at client and represent the Client API Requests:

JavaScript


ws.send(enc(tuple(
      atom('client'),
      tuple(atom('join_game'),1000001))));

Haskell


TupleTerm [AtomTerm "client", Term body]

Erlang


{client,_}

Extensions

N2O protocol is not something statically defined. It contains unintersected protocol sections, each defines its own complete part of functionality. You may extend the protocol sections by implementing you own protocol with its own envelop formatter.

NITRO

The nitrogen protocol defined client originated messages PICKLE/EV and server originated messages IO and NOP. This is something taken with historicaly reasons and roots in Rusty's Nitrogen Web Framework. This framework is transfering RPC events to page controllers though pickle envelop, which could also provide ciphering. N2O supports AES/CBC cypher.


9. PICKLE tuple {pickle,_,_,_}
10. EV tuple {ev,_,_,_,_}

You may think of nitrogen as a RPC secured protocol. If is very handy in enterprise to use FORMS with N2O application server, in those cases we use nitrogen as RPC protocol with nitogen-compatible pages.

JavaScript


JS> enc(tuple(atom('pickle'),
              atom('deposits'),
              tuple(atom('ev'),
                    atom('deposits'),
                    tuple(atom('error'),'6189'),
                    atom('#next'),
                    atom('event')),
              [tuple(atom('user'),'maxim'),
               tuple(atom('pass'),'token')]);

Erlang


{pickle,_,_,_}

    ERL> {pickle,deposits,
                 {ev,deposits,
                     {error,"6189"},
                     '#next',
                     event},
                 [{user,<<"maxim">>},
                  {pass,<<"token">>}].

PICKLE events are being sent pickled by server side picklers. N2O supports following picklers:

PLAIN binary BERT encoding of the event or
AES/CBC encrypted event for sensitive information about the API.

EV message contains RPC specification: target module, originated module, event arguments and name of controller function, for nitrogen the default type is event.

Haskell


TupleTerm [AtomTerm "ev", AtomTerm to, Term body,
                          AtomTerm from, AtomTerm type]

Erlang


{ev,_,_,_,_}

MQ

The mq protocol defined client originated messages SUB, UNSUB, PUB, SUSPEND, RESUME and server originated messages MSG.


11. SUB tuple {sub,NAME,OPTIONS}
12. UNSUB tuple {unsub,NAME}
13. PUB tuple {pub,NAME,DATA}
14. SUSPEND tuple {suspend,NAME}
15. RESUME tuple {suspend,NAME}
16. MSG tuple {msg,FROM,TO,DATA}

BPE

The bpe protocol defined client originated messages GET, COMPLETE, RUN, UNTIL, AMEND, EVENT and ANY server originated messages.


17. GET {get}
18. RUN {run}
19. UNTIL {until,Task}
20. EVENT {event,Event}
21. AMEND 2-tuple {amend,Form}
          3-tuple {amend,Form,true}
22. COMPLETE 1-tuple {complete}
             2-tuple {complete,Stage}
23. PROCESS 15-tuple
24. START

ROSTER

The roster protocol defined client originated messages AUTH, ADD, CONFIRM, CREATE, JOIN, MESSAGE, TYPING, and server originated messages PERSON, PRESENCE.


25. AUTH    {auth,USER,TOKEN,SERVICES}
26. PERSON  {person,ID,NAMES,SURNAMES,STATUS}
27. CONTACT {presence,SIZE,USERS}
32. ROSTER  {roster,PHONE,ID}
32. HISTORY {history,PHONE,ID}
30. MESSAGE {message,ID,AUTHOR,BODY,STATUS}
28. FRIEND  {friend,USER,STATUS}
29. CONFIRM {confirm,USER,TYPE}
31. TYPING  {typing,AUTHOR}

MUC

The multi-user chat protocol defined client originated messages JOIN, PUBLIC and server originated messages ROOM, PUBLIC.


32. ROOM   {room,ID,NAME}
33. PUBLIC {public,ID,AUTHOR,ROOM,BODY,STATUS}
34. JOIN   {join,USER,ROOM,ANSWER}

REST

The rest protocol defined client originated HTTP messages PUT, GET, OPTIONS, DELETE and server originated ANY messages formatted with JSON. This protocol is slightly differs and is not WebSocket protocol but HTTP endpoint protocol provided by rest library and kvs abstract database.


33. GET {get,Resource}
        {get,Resource,Id}
34. PUT {put,Resource,Object}
35. DELETE  {delete,Resource,Id}
35. OPTIONS {options,Resource}