Apache Ignite Documentation

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Binary Client Protocol

Communicate with your Ignite cluster without starting an Ignite node.

Overview

Ignite binary client protocol provides user applications the ability to communicate with an existing Ignite cluster without starting a full-fledged Ignite node. An application can connect to the cluster through a raw TCP socket. Once the connection is established, the application can communicate with the Ignite cluster and perform cache operations using the established format.

To communicate with the Ignite cluster, a client must obey the data format and communication details explained below.

Thin Clients

The protocol can also be considered as a foundation for Ignite "thin" (aka. lightweight) clients for programming languages of your choice. Ignite already supports .NET Thin Client built on top of the protocol, and plans to release thin clients for other major languages such as Java, C++, Python, Node.JS, PHP.

Data Format

Byte Ordering

Ignite binary client protocol has little-endian byte ordering.

Data Objects

User data, such as cache keys and values, are represented in the Ignite Binary Object format. A data object can be a standard (predefined) type or a complex object.

Standard (predefined) types

Standard data types are represented as a combination of type code and value.

Type
Description

byte

Type code

Standard type

Value

Supported standard types and their type codes are as follows:

Type
Type Code
Size (bytes)

byte

1

1

short

2

2

int

3

4

long

4

8

float

5

4

double

6

8

char

7

2

bool

8

1

String

9

4 bytes (for UTF-8 byte count) + UTF-8 bytes

UUID (Guid)

10

16

Date

11

8 (milliseconds from 1 Jan 1970)

byte array

12

4 bytes length + length

short array

13

4 bytes length + length * 2

int array

14

4 bytes length + length * 4

long array

15

4 bytes length + length * 8

float array

16

4 bytes length + length * 4

double array

17

4 bytes length + length * 8

char array

18

4 bytes length + length * 2

bool array

19

4 bytes length + length

String array

20

4 bytes length + variable length strings (see above)

UUID (Guid) array

21

4 bytes length + length * 16

Date array

22

4 bytes length + length * 8

Object array

23

4 bytes length + 4 bytes element type id + variable size data objects

Collection

24

4 bytes length + byte type (USER_SET = -1,
USER_COL = 0,
ARR_LIST = 1,
LINKED_LIST = 2,
HASH_SET = 3,
LINKED_HASH_SET = 4,
SINGLETON_LIST = 5) + length * (BinaryObject)

Map

25

4 bytes length + byte type (1 = HASH_MAP, 2 = LINKED_HASH_MAP) + length * (BinaryObject + BinaryObject)

Enum

28

4 bytes enum type id + 4 bytes ordinal

Enum Array

29

4 bytes enum type id + 4 bytes length + length * (Enum)

Decimal

30

4 bytes scale + 4 bytes length + byte array

Decimal Array

31

4 bytes length + length * 4

Timestamp

33

8 bytes time + 4 bytes nanos

Timestamp Array

34

4 bytes length + length * Timestamp

Time

36

8 bytes

Time Array

37

4 bytes length + length * 8

NULL

101

0

The following code snippet shows how to write and read a data object of type int, using a socket based output/input stream.

// Write int data object
DataOutputStream out = new DataOutputStream(socket.getOutputStream());

int val = 11;
writeByteLittleEndian(3, out);  // Integer type code
writeIntLittleEndian(val, out);

// Read int data object
DataInputStream in = new DataInputStream(socket.getInputStream());
int typeCode = readByteLittleEndian(in);
int val = readIntLittleEndian(in);

Refer to the example section for implementation of write...() and read..() methods shown above.

As another example, for String type, the structure would be:

Type
Description

byte

String type code, 9.

int

String length in UTF-8 bytes.

bytes

Actual string.

The code snippet below shows how to write and read a String value following this format:

private static void writeString (String str, DataOutputStream out) throws IOException {
  writeByteLittleEndian(9, out); // type code for String

  int strLen = str.getBytes("UTF-8").length; // length of the string
  writeIntLittleEndian(strLen, out);

  out.writeBytes(str);
}

private static String readString(DataInputStream in) throws IOException {
  int type = readByteLittleEndian(in); // type code

  int strLen = readIntLittleEndian(in); // length of the string

  byte[] buf = new byte[strLen];

  readFully(in, buf, 0, strLen);

  return new String(buf);
}

Complex Objects

Complex objects consist of a 24-byte header, set of fields (data objects), and a schema (field ids and positions). Depending on an operation and your data model, a data object can be of a primitive type or complex type (set of fields).

Type
Description

byte

Object type code, always 103.

byte

Version, always 1.

short

Flags-
USER_TYPE = 1
HAS_SCHEMA = 2

int

Type id, Java-style hash code of the type name.

int

Hash code, Java-style hash of contents without header, necessary for comparisons.

int

Length, including header.

int

Schema Id.

int

Schema offset from the header start, position where fields end.

Data Object

Fields.

Repeat for as many times as the number of fields.

Schema

Structure of Schema.

int field Id; Java-style hash code of field name.
int offset (from header start).

Repeat for as many times as the total number of schemas you have.

Objects Wrapped In​ Array

One or more binary objects can be wrapped in an array. This allows reading, storing, passing and writing objects efficiently without understanding their contents, performing simple byte copy.
All cache operations return complex objects inside a wrapper (but not primitives).

Type
Description

byte

Type code, always 27 .

int

Byte array size.

n bytes

Payload.

int

Offset of the object within an array (array can contain an object graph, this offset points to the root object).

A code template below shows how to read data of various types from a socket based input stream:

private static Object readDataObject(DataInputStream in) throws IOException {
  byte code = in.readByte();

  switch (code) {
    case 1:
      return in.readByte();
    case 2:
      return readShortLittleEndian(in);
    case 3:
      return readIntLittleEndian(in);
    case 4:
      return readLongLittleEndian(in);
    case 27: {
      int len = readIntLittleEndian(in);
      // Assume 0 offset for simplicity
      Object res = readDataObject(in);
      int offset = readIntLittleEndian(in);
      return res;
    }
    case 103:
      byte ver = in.readByte();
      assert ver == 1; // version
      short flags = readShortLittleEndian(in);
      int typeId = readIntLittleEndian(in);
      int hash = readIntLittleEndian(in);
      int len = readIntLittleEndian(in);
      int schemaId = readIntLittleEndian(in);
      int schemaOffset = readIntLittleEndian(in);
      byte[] data = new byte[len - 24];
      in.read(data);
      return "Binary Object: " + typeId;
    default:
      throw new Error("Unsupported type: " + code);
  }
}

Message Format

All messages- requests and responses, including handshake, start with an int type message length (excluding these first 4 bytes) followed by the payload (message body).

Handshake

The binary client protocol requires a connection handshake to ensure that client and server versions are compatible. The following tables show the structure of handshake message request and response. Refer to the example section on how to send and receive a handshake request and response respectively.

Request Type
Description

int

Length of handshake payload, always 8.

byte

Handshake code, always 1.

short

Version major.

short

Version minor.

short

Version patch.

byte

Client code, always 2.

Response Type (success)
Description

int

Success message length, 1.

byte

Success flag, 1.

Response Type (failure)
Description

int

Error message length.

byte

Success flag, 0.

short

Server version major.

short

Server version minor.

short

Server version patch.

String

Error message.

Standard Message Header

Client operation messages are composed of a header and operation-specific data. Each operation has it's own data request and response format, with a common header.

The following tables and examples show the request and response structure of a client operation message header:

Request Type
Description

int

Length of payload.

long

Request id, generated by client and returned as-is in response

private static void writeRequestHeader(int reqLength, short opCode, long reqId, DataOutputStream out) throws IOException {
  // Message length
  writeIntLittleEndian(10 + reqLength, out);

  // Op code
  writeShortLittleEndian(opCode, out);

  // Request id
  writeLongLittleEndian(reqId, out);
}
Response Type
Description

int

Length of response message.

long

Request id (see above)

int

Status code (0 for success, otherwise error code)

String

Error message (present only when status is not 0)

private static void readResponseHeader(DataInputStream in) throws IOException {
  // Response length
  final int len = readIntLittleEndian(in);

  // Request id
  long resReqId = readLongLittleEndian(in);

  // Success code
  int statusCode = readIntLittleEndian(in);
}

Connectivity

TCP Socket

Client applications should connect to server nodes with a TCP socket. By default, the connector is enabled at port 10800. You can configure the port number and other server-side​ connection parameters in the clientConnectorConfiguration property of IgniteConfiguration of your cluster, as shown below:

<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
    <!-- Thin client connection configuration. -->
    <property name="clientConnectorConfiguration">
        <bean class="org.apache.ignite.configuration.ClientConnectorConfiguration">
            <property name="host" value="127.0.0.1"/>
            <property name="port" value="10900"/>
            <property name="portRange" value="30"/>
        </bean>
    </property>
    
    <!-- Other Ignite Configurations. -->
    
</bean>
IgniteConfiguration cfg = new IgniteConfiguration();

ClientConnectorConfiguration ccfg = new ClientConnectorConfiguration();
ccfg.setHost("127.0.0.1");
ccfg.setPort(10900);
ccfg.setPortRange(30);

// Set client connection configuration in IgniteConfiguration
cfg.setClientConnectorConfiguration(ccfg);

// Start Ignite node
Ignition.start(cfg);

Connection Handshake

Besides socket connection, the thin client protocol requires a connection handshake to ensure that client and server versions are compatible. Note that handshake must be the first message after connection establishment.

For the handshake message request and response structure, see the handshake section above.

Example

Socket socket = new Socket();
socket.connect(new InetSocketAddress("127.0.0.1", 10800));

DataOutputStream out = new DataOutputStream(socket.getOutputStream());

// Message length
writeIntLittleEndian(8, out);

// Handshake operation
writeByteLittleEndian(1, out);

// Protocol version 1.0.0
writeShortLittleEndian(1, out);
writeShortLittleEndian(1, out);
writeShortLittleEndian(0, out);

// Client code
writeByteLittleEndian(2, out);

// send request
out.flush();

// Receive handshake response
DataInputStream in = new DataInputStream(socket.getInputStream());
int length = readIntLittleEndian(in);
int successFlag = readByteLittleEndian(in);

// Since Ignite binary protocol uses little-endian byte order, 
// we need to implement big-endian to little-endian 
// conversion methods for write and read.

// Write int in little-endian byte order
private static void writeIntLittleEndian(int v, DataOutputStream out) throws IOException {
  out.write((v >>> 0) & 0xFF);
  out.write((v >>> 8) & 0xFF);
  out.write((v >>> 16) & 0xFF);
  out.write((v >>> 24) & 0xFF);
}

// Write short in little-endian byte order
private static final void writeShortLittleEndian(int v, DataOutputStream out) throws IOException {
  out.write((v >>> 0) & 0xFF);
  out.write((v >>> 8) & 0xFF);
}

// Write byte in little-endian byte order
private static void writeByteLittleEndian(int v, DataOutputStream out) throws IOException {
  out.writeByte(v);
}

// Read int in little-endian byte order
private static int readIntLittleEndian(DataInputStream in) throws IOException {
  int ch1 = in.read();
  int ch2 = in.read();
  int ch3 = in.read();
  int ch4 = in.read();
  if ((ch1 | ch2 | ch3 | ch4) < 0)
    throw new EOFException();
  return ((ch4 << 24) + (ch3 << 16) + (ch2 << 8) + (ch1 << 0));
}


// Read byte in little-endian byte order
private static byte readByteLittleEndian(DataInputStream in) throws IOException {
  return in.readByte();
}

// Other write and read methods

Client Operations

Upon successful handshake, a client can start performing various cache operations:

Binary Client Protocol


Communicate with your Ignite cluster without starting an Ignite node.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.