Protobuf Protocol Intro

Overview

Protocol Buffers (protobuf) - Google's data interchange format. Protocol Buffers are a way of encoding structured data in an efficient yet extensible format.

Messages Structure

Network communication is done using only ProtoMessage objects (a protobuf message created by us). In order to deal with network fragmentation we will send messages to the network using the following frame structure:

+--------------------------+-----------------------------------------+
| Message Length (4 bytes) | Serialized ProtoMessage object (byte[]) |
+--------------------------+-----------------------------------------+
                           |<---------- Message Length ------------->|

ProtoMessage has the following structure:

+----------------------+
|  int32 payloadType   |
|  byte[] payload      |
|  string clientMsgId  |
+----------------------+

It contains 2 mandatory fields:

  • payloadType - contains id of ProtoPayloadType. This field tells us what is the type of the protobuf object which is serialized in the second field.
  • payload - serialized protobuf message that corresponds to payloadType.

And another optional field:

  • clientMsgId - request message id, assigned by the client that will be returned in the response.

Below you can see the actual ProtoMessage definition:

message ProtoMessage {
    required uint32 payloadType = 1; // contains id of ProtoPayloadType or other custom PayloadTypes (e.g. ProtoCHPayloadType)
    optional bytes payload = 2; // serialized protobuf message that corresponds to payloadType
    optional string clientMsgId = 3; // request message id, assigned by the client that will be returned in the response
}

Naming convention

File Naming Convention

  • All the files where protobuf messages are defined have .proto extension.
  • All protobuf generated classes are located in com.xtrader.protocol.proto.* package.
  • Always separate the domain messages from command messages.
  • All .proto files should start with the following lines:
option java_multiple_files = true;
option java_package = "com.xtrader.protocol.proto.XXX";
option java_outer_classname = "ContainerXXXMessages";

Message Naming Convention

  • All messages could be divided into command messages, event messages and domain messages.
  • Command messages are used for sending commands (requests) and results (responses) to the servers and back. Should be named in a format ProtoCommandName(Req|Res). Where “Req” means request command message, “Res” means response command message. Examples : ProtoAuthReq, ProtoAuthRes.
  • Event messages are used in notifying subscribers with asynchronous messages. The classic example is ping command, its proto class names is ProtoPingEvent. Naming convention of events is ProtoEventNameEvent
  • Domain messages describe entities that participates in Server domain model. Naming conventions is following ProtoEntityName. Examples : ProtoOrder, ProtoUser, ProtoPosition.

List of currently supported messages.

To find a list of already developed proto beans please check existing proto bean files.

Open API protobuf files can be find at the following location: open-api-proto-lib.zip.