Apache Ignite Documentation

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Memory Configuration

Durable Memory Configuration

By default, Ignite nodes consume up to 20% of the RAM available locally, and in most cases, ‚Äčthis is the only parameter you might need to change. To do this, change the size of the default data region, as shown below:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">

<!-- Redefining maximum memory size for the cluster node usage. -->  
<property name="dataStorageConfiguration">
  <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
    <!-- Redefining the default region's settings -->
    <property name="defaultDataRegionConfiguration">
      <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
        <property name="name" value="Default_Region"/>
        <!-- Setting the size of the default region to 4GB. -->
        <property name="maxSize" value="#{4L * 1024 * 1024 * 1024}"/>
      </bean>
    </property>
  </bean>
</property>
  
<!-- The rest of the parameters. -->
</bean>
IgniteConfiguration cfg = new IgniteConfiguration();

// Changing total RAM size to be used by Ignite Node.
DataStorageConfiguration storageCfg = new DataStorageConfiguration();

// Setting the size of the default memory region to 4GB to achieve this.
storageCfg.getDefaultDataRegionConfiguration().setMaxSize(
    4L * 1024 * 1024 * 1024);

cfg.setDataStorageConfiguration(storageCfg);

// Starting the node.
Ignition.start(cfg);

For more advanced tuning of the durable memory, refer to the following documentation sections:

Global Configuration Parameters

To alter the main settings of the durable memory such as page size, use org.apache.ignite.configuration.DataStorageConfiguration that is passed via the IgniteConfiguration.setDataStorageConfiguration(...) method. Below you can see some of the parameters available:

Parameter Name
Description
Default Value

setPageSize(...)

Sets default page size.

Usually, you change the parameter when an application generates a lot of the objects that don't fit into a single page.

4 KB

setDefaultDataRegionConfiguration(...)

Sets the size of the default data region which is created automatically.

If the property is not set then the default region can consume up to 20% of RAM available on a local machine.

  • 20% of RAM
  • persistence is disabled

setDataRegionConfigurations(...)

Sets a list of all data regions configured on a cluster node.

Refer to data regions for more details.

An empty array. A configuration that is used to create the default region is not stored there.

setSystemRegionInitialSize(...)

Sets initial size of a data region reserved for system needs.

40 MB

setSystemRegionMaxSize(...)

Sets maximum size of a data region reserved for system needs.

The total size should not be less than 10 MB due to internal data structures overhead.

100 MB

setConcurrencyLevel(...)

Sets the number of concurrent segments in Apache Ignite internal page mapping tables.

A total number of available CPUs times 4.

Refer to Ignite javadoc to get a complete list of DataStorageConfiguration parameters.

The following example shows how to change the page size and concurrency level using DataStorageConfiguration:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  <property name="dataStorageConfiguration">
    <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
      <!-- Set concurrency level -->
      <property name="concurrencyLevel" value="4"/>

      <!-- Set the page size to 8 KB -->
      <property name="pageSize" value="8192"/>
    </bean>
  </property>
  
  <!--- Additional settings ---->
</bean>
// Ignite configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Durable memory configuration.
DataStorageConfiguration storageCfg = new DataStorageConfiguration();

// Altering the concurrency level.
storageCfg.setConcurrencyLevel(4);

// Changing the page size to 8 KB.
storageCfg.setPageSize(8192);

// Applying the new configuration.
cfg.setDataStorageConfiguration(storageCfg);

Data Regions

By default, the durable memory creates a single expandable data region that can take up to 20% of the RAM available on a local machine with persistence disabled. To define multiple data regions with various parameters such as region size, persistence, or eviction policy, use the org.apache.ignite.configuration.DataRegionConfiguration class.

Two or More Regions?

Refer to the memory region documentation section to discover why you may need to have more than one region configured.

For instance, to create a data region that will consume up to 500 MB of RAM and will have Ignite persistence enabled, we need to define the configuration as follows:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  <!-- Durable memory configuration. -->
  <property name="dataStorageConfiguration">
    <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
      <property name="dataRegionConfigurations">
        <list>
          <!--
              Defining a data region that will consume up to 500 MB of RAM and 
              will have eviction and persistence enabled.
          -->
          <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
            <!-- Custom region name. -->
            <property name="name" value="500MB_Region"/>

            <!-- 100 MB initial size. -->
            <property name="initialSize" value="#{100L * 1024 * 1024}"/>

            <!-- 500 MB maximum size. -->
            <property name="maxSize" value="#{500L * 1024 * 1024}"/>

            <!-- Enabling persistence for the region. -->
            <property name="persistenceEnabled" value="true"/>
          </bean>
        </list>
      </property>
    </bean>
  </property>
  
  <!-- Other configurations. -->
</bean>
// Ignite configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Durable Memory configuration.
DataStorageConfiguration storageCfg = new DataStorageConfiguration();

// Creating a new data region.
DataRegionConfiguration regionCfg = new DataRegionConfiguration();

// Region name.
regionCfg.setName("500MB_Region");

// Setting initial RAM size.
regionCfg.setInitialSize(100L * 1024 * 1024);

// Setting maximum RAM size.
regionCfg.setMaxSize(500L * 1024 * 1024);
        
// Enable persistence for the region.
regionCfg.setPersistenceEnabled(true);
        
// Setting the data region configuration.
storageCfg.setDataRegionConfigurations(regionCfg);
        
// Applying the new configuration.
cfg.setDataStorageConfiguration(storageCfg);

Then, to make use of the configured region, instruct Ignite caches to store data there by passing the region's name (500MB_Region) into the org.apache.ignite.configuration.CacheConfiguration.setDataRegionName(...) method:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
    <!-- Durable Memory and other configuration parameters. -->
    <!-- ....... -->
  
    <property name="cacheConfiguration">
       <list>
           <!-- Cache that is mapped to a specific data region. -->
           <bean class="org.apache.ignite.configuration.CacheConfiguration">
              <!--
                   Assigning the cache to the `500MB_Region` defined earlier.
               -->
               <property name="dataRegionName" value="500MB_Region"/>
               
             	 <!-- Cache name. -->
               <property name="name" value="SampleCache"/>
             
               <!-- Additional cache configuration parameters -->
           </bean>
       </list>
    </property>
    
    <!-- The rest of the configuration. -->
    <!-- ....... -->
</bean>  
// Ignite configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Durable Memory configuration and the rest of the configuration.
// ....

// Creating a cache configuration.
CacheConfiguration cacheCfg = new CacheConfiguration();

// Binding the cache to the earlier defined region.
cacheCfg.setDataRegionName("500MB_Region");
        
// Setting the cache name.
cacheCfg.setName("SampleCache");

// Applying the cache configuration.
cfg.setCacheConfiguration(cacheCfg);

Once a cluster node is started with this configuration, the durable memory will allocate a data region that will initially consume 100 MB of RAM and can grow up to 500 MB there. A superset of data will be always stored on disk ensuring that no data loss happens even if there is no more space left in RAM. Ignite automatically evicts the least recently used data from RAM if persistence enabled. The defined region will store data added to SampleCache only. The rest of the caches will be bound to the default data region unless you map them to a different one explicitly.

If Ignite persistence is disabled and the overall RAM usage goes beyond 500 MB, then you would get an out of memory exception. To avoid this:

  • Use Ignite persistence as shown in the example above.
  • Use one of the eviction algorithms available. Note that the eviction will be enabled by default only if the Ignite persistence is used.
  • Set region's maximum size to a bigger value.

Refer to the Ignite javadoc to get a complete list of DataRegionConfiguration parameters.

On-Heap Caching

Ignite Durable Memory is an off-heap memory that allocates all the memory regions outside of Java heap. However, you can enable on-heap caching by setting org.apache.ignite.configuration.CacheConfiguration.setOnheapCacheEnabled(...) to true.

On-heap caching is useful for scenarios when you do a lot of cache reads on server nodes that work with cache entries in the binary form or invoke cache entries' deserialization. For instance, this might happen when a distributed computation or deployed service gets some data from caches for further processing.

On-Heap Cache Size

To manage the on-heap cache size and avoid its constant growth, make sure to enable one of the available cache entries based eviction policies.

Performance Tips

See the Durable Memory Tuning documentation for performance suggestions.

Example

For more information on how to configure and use multiple data regions in your cluster, refer to the data regions example and its configuration file.

Memory Configuration

Durable Memory Configuration