Apache Ignite

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Collocate Compute and Data

Collocate your computations with the data.

Overview

Collocation of computations with data allow for minimizing data serialization within network and can significantly improve performance and scalability of your application. Whenever possible, you should always make best effort to colocate your computations with the cluster nodes caching the data that needs to be processed.

Affinity Call and Run Methods

affinityCall(...) and affinityRun(...) methods co-locate jobs with nodes on which data is cached. In other words, knowing a cache name and affinity key these methods will be able to find the node that is the primary for the given key and will execute a job there.

Consistency Guarantee

Starting from Apache Ignite 1.8 it's guaranteed that the partition, which the affinity key belongs to, will not be evicted from a node while a job, triggered by affinityCall(...) or affinityRun(...), will be being executed there. The partition rebalancing usually happens due to the topology change event when a new node joins the cluster or the old one leaves it.

This guarantee makes it feasible to execute complex logic for which it's crucial that the data stays on the same node throughout the time the job is being executed there. For instance, this feature allows executing local SQL queries as a part of the job, triggered by affinityCall(...) or affinityRun(...), not worrying about the fact that the local query may return a partial result set due to the data rebalancing.

IgniteCache<Integer, String> cache = ignite.cache(CACHE_NAME);

IgniteCompute compute = ignite.compute();

for (int key = 0; key < KEY_CNT; key++) {
    // This closure will execute on the remote node where
    // data with the 'key' is located.
    compute.affinityRun(CACHE_NAME, key, () -> { 
        // Peek is a local memory lookup.
        System.out.println("Co-located [key= " + key + ", value= " + cache.localPeek(key) +']');
    });
}
IgniteCache<Integer, String> cache = ignite.cache(CACHE_NAME);

IgniteCompute asyncCompute = ignite.compute().withAsync();

List<IgniteFuture<?>> futs = new ArrayList<>();

for (int key = 0; key < KEY_CNT; key++) {
    // This closure will execute on the remote node where
    // data with the 'key' is located.
    asyncCompute.affinityRun(CACHE_NAME, key, () -> { 
        // Peek is a local memory lookup.
        System.out.println("Co-located [key= " + key + ", value= " + cache.peek(key) +']');
    });
  
    futs.add(asyncCompute.future());
}

// Wait for all futures to complete.
futs.stream().forEach(IgniteFuture::get);
final IgniteCache<Integer, String> cache = ignite.cache(CACHE_NAME);

IgniteCompute compute = ignite.compute();

for (int i = 0; i < KEY_CNT; i++) {
    final int key = i;
 
    // This closure will execute on the remote node where
    // data with the 'key' is located.
    compute.affinityRun(CACHE_NAME, key, new IgniteRunnable() {
        @Override public void run() {
            // Peek is a local memory lookup.
            System.out.println("Co-located [key= " + key + ", value= " + cache.peek(key) +']');
        }
    });
}

Both affinityCall(...) or affinityRun(...) have overloaded versions of the methods that allow locking a partition, avoiding its eviction during the job's execution, across several caches. All you need to do is to pass the names of the caches into the abovementioned methods.

Collocate Compute and Data

Collocate your computations with the data.