Protocol Buffers

Protocol buffers or Protobuf is language- and platform-neutral, extensible mechanism for serializing structured data, a way of encoding structured data in an efficient yet extensible format. You define how you want your data to be structured once, then you use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

You specify how you want the information you're serializing to be structured by defining protocol buffer message types in .proto files. Each protocol buffer message is a small logical record of the information, containing the series of name-value pairs. Here's a very basic example of a .proto file that defines a message containing information about a person:

message Person {

    required string name = 1;  
    required int32 id = 2;  
    optional string email = 3;  


    enum PhoneType {  
        MOBILE = 0;  
        HOME = 1;  
        WORK = 2;  
    }  

    message PhoneNumber {  
       required string number = 1;  
       optional PhoneType type = 2 [default = HOME];  
    }  

    repeated PhoneNumber phone = 4;  
}

As you can see, the message format is simple – each message type has one or more uniquely numbered fields, and each field has a name and a value type, where value types can be numbers (integer or floating-point), booleans, strings, raw bytes, or even (as in the example above) other protocol buffer message types, allowing you to structure your data hierarchically. You can find more information about writing .proto files in the Protocol Buffer Language Guide.

Once you've defined your messages, you run the protocol buffer compiler for your application's language on your .proto file to generate data access classes. These provide simple accessors for each field (like name() and set_name()) as well as methods to serialize/parse the whole structure to/from raw bytes – so, for instance, if your chosen language is C++, running the compiler on the above example will generate a class called Person. You can then use this class in your application to populate, serialize, and retrieve Person protocol buffer messages.

Learn more about the Protocol Buffers here.

Generating Protobuf Classes

Protobuf is the flexible and automated solution designed to simplify the serializing and retrieving structured data. With protocol buffers, you write a .proto description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format.

Below we will focus on how to generate classes using protobuf tools for C# and Java languages.

Protobuf Basics for C#

  1. To generate C# classes from your .proto files use the C# classes generator that can be downloaded from here: Protocol Buffers Gen. Extract the downloaded files on your local drive.

  2. Download the latest version of Open API 2.0 .proto files from our GitHub repository and extract them on your local drive to the same location with your Protocol Buffers Generator from the step 1.

  3. Run the following command for all the protofiles:

ProtoGen.exe protofile.proto -output_directory=C:\output_folder --include_imports

where protofile.proto - is your .proto file name and output_folder is the output folder name.

  1. Copy newly generated .cs files to your project. Alternatively, you can create a class library project within your solution and copy the .cs files there.

  2. Link the .cs files to the project adding them to it and/or setting dependencies between your project and the proto library project (in case you have created the library). Add Reference of the Class Library project (if you have it) to your main project.

  3. Add Reference on Google.ProtocolBuffers.dll to the Class Library project (or your main project if you do not have the Class Library project).

Alternatively, if you use Visual Studio, then you can use the ProtobufGenerator for Visual Studio 2015/2017 from here. This extension runs protoc.exe for you after any change to the proto file. To use the extension, install it, then type in ProtobufGenerator in Custom tool in the properties for your proto file in the visual studio properties panel. The .cs file will be placed as a child file of the proto file in the project. If there is no .cs file produced it probably is because it did not find the protoc.exe file. You can compile it on your own, but it is also available in the nugget Google.Protobuf.Tools. But it is not mandatory to have that nugget in your project.

Also, you can find the detailed information on how to use Protobuf for C# here.

Protobuf Basics for Java

Please find detailed information on how to use Protobuf for Java here.

ProtoMessages

The network communication in Open API 2.0 is performed by means of ProtoMessage objects - the protobuf messages designed by Spotware. In order to deal with the network fragmentation we will send messages to the network using the following frame structure:

 +--------------------------+-----------------------------------------+  
 | Message Length (4 bytes) | Serialized ProtoMessage object (byte[]) |  
 +--------------------------+-----------------------------------------+  
                            |<---------- Message Length ------------->|

ProtoMessage has the following structure:

 +----------------------+  
 | int32 payloadType    |  
 | byte[] payload       |  
 | string clientMsgId   |  
 +----------------------+

It contains 2 mandatory fields:

  • payloadType - contains the ProtoPayloadType ID. This field tells us what is the type of the protobuf object serialized in the second field.

  • payload - serialized protobuf message that corresponds to payloadType.

And an optional field:

  • clientMsgId - request message ID, assigned by the client that will be returned in the response.

The actual ProtoMessage definition looks as follows:

message ProtoMessage {  
required uint32 payloadType = 1; //Contains ID of the ProtoPayloadType or other
custom PayloadTypes (e.g. ProtoCHPayloadType).  
optional bytes payload = 2; //Serialized protobuf message that corresponds to
the payloadType  
optional string clientMsgId = 3; //The request message ID, assigned by the
client that will be returned in the response.  
}

Naming convention

Message Naming Convention

All messages could be divided into the Request messages, Response messages, Event messages, and Model messages:

Request messages are used for sending requests to the server

Example:

ProtoAuthReq

where Req means request command message


Response messages are used to receive data back from the server.

Example:

ProtoAuthRes.

where Res means response command message.


Event messages are used for notifying subscribers with asynchronous messages.

The classic example is ping command, its proto class name is ProtoPingEvent.

The naming convention of events is ProtoEventNameEvent.


Model messages describe entities that participate in the Server domain model.

Naming conventions of the Model messages is ProtoEntityName.

Examples:

ProtoOrder

ProtoUser

ProtoPosition


Account Information

Use the following protobuf messages for retrieving or sending the account information:

ProtoOAApplicationAuthReq Request for authorizing an Application to work with cTrader platform Proxies.
ProtoOAApplicationAuthRes Response to ProtoOAApplicationAuthReq
ProtoOAAccountAuthReq Request for authorizing a Trading Account within authorized application connection. Required for starting work with the account.
ProtoOAAccountAuthRes Response on ProtoOAApplicationAuthRes.

Trading

Use the following protobuf messages for retrieving or sending the trading data:

ProtoOANewOrderReq Request for sending new trading order. Allowed only if the accessToken field has "trade" permissions for the Trading Account.
ProtoOAExecutionEvent Event that is returned following the successful order processing or order execution. Replaces responses for Trading requets ProtoOANewOrderReq, ProtoOACancelOrderReq, ProtoOAAmendOrderReq, ProtoOAAmendPositionSLTPReq, ProtoOAClosePositionReq. Also, event is sent when a Deposit/Withdrawal took place.
ProtoOACancelOrderReq Request for canceling an existing pending order. Allowed only if the access token has "trade" permissions for the Trading Account.
ProtoOAAmendOrderReq Request for amending existing pending order.
ProtoOAAmendPositionSLTPReq Request for amending StopLoss and TakeProfit for existing position.
ProtoOAClosePositionReq Request for closing the existing positions.
ProtoOATrailingSLChangedEvent Event that is returned when the level of a Trailing Stop Loss is changed.

Market data

Use the following protobuf messages for retrieving or sending the market data:

ProtoOASymbolsForConversionReq Request for getting a conversion chain between two assets that consists of several symbols.
ProtoOASymbolsForConversionRes Response to ProtoOASymbolsForConversionReq.
ProtoOADealListReq Request for getting the deals list.
ProtoOADealListRes Response to getting deals list.
ProtoOAAssetListReq Request for the list of assets available on the Server of the Trader's Account.
ProtoOAAssetListRes Response to the ProtoOAAssetListReq request.
ProtoOASymbolsListReq Request for a list of symbols available on the Server of the Trader's Account.
ProtoOASymbolByIdReq Request for getting a full Symbol entity.
ProtoOASymbolByIdRes Response to the ProtoOASymbolByIdReq request.
ProtoOASymbolsForConversionReq Request for getting a conversion chain between two assets that consists of several symbols.
ProtoOASymbolsForConversionRes Response to the ProtoOASymbolsForConversionReq request.
ProtoOAAssetClassListReq Request for a list of Asset Classes available on the Server of the Trader's Account.
ProtoOAAssetClassListRes Responce to the ProtoOAAssetListReq request.