Apache Ignite

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Load Balancing

Overview

Load balancing component balances job distribution among cluster nodes. In Ignite load balancing is achieved via LoadBalancingSpi which controls load on all nodes and makes sure that every node in the cluster is equally loaded. In homogeneous environments with homogeneous tasks load balancing is achieved by random or round-robin policies. However, in many other use cases, especially under uneven load, more complex adaptive load-balancing policies may be needed.

Data Affinity

Note that load balancing is triggered whenever your jobs are not collocated with data or have no real preference on which node to execute. If Collocation Of Compute and Data is used, then data affinity takes priority over load balancing.

Round-Robin Load Balancing

RoundRobinLoadBalancingSpi iterates through nodes in round-robin fashion and picks the next sequential node. Two modes of operation are supported: per-task and global. Global mode is used be default.

Per-Task Mode

When configured in per-task mode, implementation will pick a random node at the beginning of every task execution and then sequentially iterate through all nodes in topology starting from the picked node. For cases when split size is equal to the number of nodes, this mode guarantees that all nodes will participate in the split.

Global Mode

When configured in global mode, a single sequential queue of nodes is maintained for all tasks and the next node in the queue is picked every time. In this mode (unlike in per-task mode) it is possible that even if split size may be equal to the number of nodes, some jobs within the same task will be assigned to the same node whenever multiple tasks are executing concurrently.

<bean id="grid.custom.cfg" class="org.apache.ignite.IgniteConfiguration" singleton="true">
  ...
  <property name="loadBalancingSpi">
    <bean class="org.apache.ignite.spi.loadbalancing.roundrobin.RoundRobinLoadBalancingSpi">
      <!-- Set to per-task round-robin mode (this is default behavior). -->
      <property name="perTask" value="true"/>
    </bean>
  </property>
  ...
</bean>
RoundRobinLoadBalancingSpi = new RoundRobinLoadBalancingSpi();
 
// Configure SPI to use per-task mode (this is default behavior).
spi.setPerTask(true);
 
IgniteConfiguration cfg = new IgniteConfiguration();
 
// Override default load balancing SPI.
cfg.setLoadBalancingSpi(spi);
 
// Start Ignite node.
Ignition.start(cfg);

Random and Weighted Load Balancing

WeightedRandomLoadBalancingSpi picks a random node for job execution by default. You can also optionally assign weights to nodes, so nodes with larger weights will end up getting proportionally more jobs routed to them. By default all nodes get equal weight of 10.

<bean id="grid.custom.cfg" class="org.apache.ignite.IgniteConfiguration" singleton="true">
  ...
  <property name="loadBalancingSpi">
    <bean class="org.apache.ignite.spi.loadbalancing.weightedrandom.WeightedRandomLoadBalancingSpi">
      <property name="useWeights" value="true"/>
      <property name="nodeWeight" value="10"/>
    </bean>
  </property>
  ...
</bean>
WeightedRandomLoadBalancingSpi = new WeightedRandomLoadBalancingSpi();
 
// Configure SPI to used weighted random load balancing.
spi.setUseWeights(true);
 
// Set weight for the local node.
spi.setWeight(10);
 
IgniteConfiguration cfg = new IgniteConfiguration();
 
// Override default load balancing SPI.
cfg.setLoadBalancingSpi(spi);
 
// Start Ignite node.
Ignition.start(cfg);

Load Balancing