Apache Ignite Documentation

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Swap Space

Overview

When using pure memory storage, it is possible for the data size to exceed the physical RAM size, leading to out of memory errors (OOMEs). To avoid this, the ideal approach would be to enable Ignite native persistence or use 3rd party persistence. However, if you do not want to use native or 3rd party persistence, you can enable swapping, in which case, Ignite in-memory data will be moved to the swap space located on disk. Please note that Ignite does not provide its own implementation of swap space. Instead, it takes advantage of the swapping functionality provided by the operating system (OS).

When swap space is enabled, Ignite stores data in memory mapped files (MMF) whose content will be swapped to disk by the OS depending on the current RAM consumption; however, in that scenario the data access time will be slower. Additionally, there are no data durability guarantees. Which means that the data from the swap space is available only as long as the node is alive. Once the node, where the swap space exists, shuts down, all data will be lost. Therefore, you should use swap space as an extension to RAM only to give yourself enough time to add more nodes to the cluster in order to redistribute data and avoid OOMEs which might happen if the cluster is not scaled in time.

We recommend using Ignite Native Persistence to go beyond​ RAM capacity and preserve system durability.

Configuration

Data Region maxSize defines the total maxSize of the region. You will get OOMEs if your data size exceeds the maxSize and neither Ignite persistence nor a 3rd party disk database is used. To avoid this situation with the swapping capabilities​, you need to :

  • Set maxSize = bigger_ than_RAM_size, in which case, the OS will take care of the swapping.
  • Enable swapping by setting the DataRegionConfiguration.swapPath property.
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  <!-- Durable memory configuration. -->
  <property name="dataStorageConfiguration">
    <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
      <property name="dataRegionConfigurations">
        <list>
          <!--
              Defining a data region that will consume up to 500 MB of RAM 
              with swap enabled.
          -->
          <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
            <!-- Custom region name. -->
            <property name="name" value="500MB_Region"/>

            <!-- 100 MB initial size. -->
            <property name="initialSize" value="#{100L * 1024 * 1024}"/>

            <!-- Setting region max size equal to physical RAM size(5 GB). -->
            <property name="maxSize" value="#{5L * 1024 * 1024 * 1024}"/>

            <!-- Enabling swap space for the region. -->
            <property name="swapPath" value="/path/to/some/directory"/>
          </bean>
        </list>
      </property>
    </bean>
  </property>
  
  <!-- Other configurations. -->
</bean>
// Ignite configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Durable Memory configuration.
DataStorageConfiguration storageCfg = new DataStorageConfiguration();

// Creating a new data region.
DataRegionConfiguration regionCfg = new DataRegionConfiguration();

// Region name.
regionCfg.setName("500MB_Region");

// Setting initial RAM size.
regionCfg.setInitialSize(100L * 1024 * 1024);

// Setting region max size equal to physical RAM size(5 GB)
regionCfg.setMaxSize(5L * 1024 * 1024 * 1024);
        
// Enable swap space.
regionCfg.setSwapPath("/path/to/some/directory");
        
// Setting the data region configuration.
storageCfg.setDataRegionConfigurations(regionCfg);
        
// Applying the new configuration.
cfg.setDataStorageConfiguration(storageCfg);

Possible data loss

While swap space is located on disk, it should not be considered as a replacement to Ignite persistence. Data from swap space is available as long as the node is active. Once the node shuts down, all data will be lost. To ensure that data is obtainable at all times, you should either enable Ignite persistence, or use a 3rd party persistence.

Swap Space


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.