The Ignite Compute Grid allows you to take a computation i.e. a piece of business logic such as a calculation, optionally split it into multiple parts, and execute it on different grid nodes in parallel. This means that you can reduce the overall execution time of a collection of computations by using resources from all grid nodes in parallel. One of the most common design patterns for parallel execution is MapReduce. However, the Compute Grid can be useful even if you don't need to split and parallelize your computation — it can enable you improve overall scalability and fault-tolerance of your system by offloading computations onto the most available nodes.
The Ignite Compute Grid provides a set of simple APIs that allows users to distribute computations and data processing across multiple computers in the cluster and enables you to configure failover policies to control the behavior of a specific job during failure scenarios.
Key characteristics of the Ignite Compute Grid include:
Automatic Deployment - This allows for automatic distribution and deployment of classes and resources required to execute a specific job.
Topology Resolution - This allows the provisioning of nodes based on any node characteristic or user-specific configuration. For example, you can decide to only include Linux nodes for execution, or to only include a certain group of nodes within certain time window. You can do things like choose all nodes with CPU loaded, say, under 50% that have more than 2GB of available Heap memory.
Collision Resolution - This allows users to control which jobs get executed, which jobs get rejected, how many jobs can be executed in parallel, and the order of overall execution, etc.
Load Balancing - This allows users to properly balance load across the grid. Some of the most common ones are Round Robin, Random, or Adaptive. Ignite also offers Affinity Load Balancing where grid jobs always execute on the same node based on the job's affinity key. This allows the execution of code close to the data i.e. collocated processing.
Fail-over - Automatic and manual failover is configurable such that jobs automatically fail-over onto other nodes in case of node crash or some other exception.
IgniteCompute interface provides methods for running many types of computations over nodes in a cluster or a cluster group. These methods can be used to execute Tasks or Closures in a distributed fashion.
All jobs and closures are guaranteed to be executed as long as there is at least one node standing. If a job execution is rejected due to lack of resources, a failover mechanism is provided. In case of failover, the load balancer picks the next available node to execute the job. Here is how you can get an
Ignite ignite = Ignition.ignite(); // Get compute instance over all nodes in the cluster. IgniteCompute compute = ignite.compute();
You can also limit the scope of computations to a Cluster Group. In this case, computation will only execute on the nodes within the cluster group.
Ignite ignite = Ignition.ignite(); ClusterGroup remoteGroup = ignite.cluster().forRemotes(); // Limit computations only to remote nodes (exclude local node). IgniteCompute compute = ignite.compute(remoteGroup);