Apache Ignite Documentation

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Write-Ahead Log

Overview

Ignite persistence maintains a dedicated file for every partition in a node. However, when a page is updated in RAM, the update is not directly written to the respective partition file because it can affect the performance dramatically. It's rather appended to the tail of a write-ahead log (WAL).

The purpose of the WAL is to provide a recovery mechanism for scenarios where a single node or the whole cluster goes down. It is worth mentioning that in case of a crash or restart, a cluster can always be recovered to the latest successfully committed transaction by relying on the contents of the WAL.

The whole WAL is split into several files, called segments, that are filled out sequentially. Once the 1st segment is full, its content is copied to the WAL archive (see the WAL Archive section below). While the 1st segment's content is being copied, the 2nd segment is treated as an active WAL file and accepts all the updates coming from the application side. By default, there are 10 such segments created and used. To change this number, use the DataStorageConfiguration.walSegments setting.

Each update is written to a buffer before being written to the WAL file. The size of this buffer is defined by setting the DataStorageConfiguration.walBuffSize property. By default, the WAL buffer size equals the WAL segment size if the memory mapped file is enabled, and (WAL segment size) / 4 if the memory-mapped file is disabled. Note that the memory mapped file is enabled by default. It can be turned off using the IGNITE_WAL_MMAP system property that can be passed as a JVM argument. For example: -DIGNITE_WAL_MMAP=false

WAL Modes

Depending on the WAL mode set, Ignite provides the following consistency guarantees:

WAL Mode
Description
Consistency Guarantees

FSYNC

The changes are guaranteed to be persisted to disk for every atomic write or transactional commit.

Data updates are never lost surviving any OS or process crashes, or power failure.

LOG_ONLY

That's the default mode.

The changes are guaranteed to be flushed to the OS buffer cache or a memory-mapped file for every atomic write or transactional commit.

The memory-mapped file approach is used by default and can be switched off by setting IGNITE_WAL_MMAP system property to false.

Data updates survive only process crash.

BACKGROUND

When IGNITE_WAL_MMAP property is enabled (default), this mode behaves like LOG_ONLY mode.

If the memory-mapped file approach is disabled then the changes stay in node's internal buffer. The frequency of flushing the buffer to disk is defined via the DataStorageConfiguration.setWalFlushFrequency parameter.

When IGNITE_WAL_MMAP property is enabled (default), the mode provides the same guarantees as LOG_ONLY mode has.

Otherwise, recent data updates may get lost in case of a process crash or other outages.

NONE

WAL is disabled. The changes are persisted only if you shutdown the node gracefully.
Use Ignite#active(false) to deactivate the cluster and shutdown the node.

Data loss might happen.

If an Ignite node is terminated abruptly, it is very likely that the data stored on the disk will be out-of-sync or corrupted and the persistence directory needs to be cleared for a node restart.

The following example shows how to set the WAL mode:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
   
  <property name="dataStorageConfiguration">
     <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
       <!-- Enabling Apache Ignite Persistent Store. -->     
       <property name="defaultDataRegionConfiguration">
         <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
           <property name="persistenceEnabled" value="true"/>
         </bean>
       </property>

       <!-- Changing WAL Mode. --> 
       <property name="walMode" value="FSYNC"/>
     </bean>
   </property>

  <!-- Additional setting. -->

</bean>
// Apache Ignite node configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Native Persistence configuration.
DataStorageConfiguration psCfg = new DataStorageConfiguration();

// Enabling the persistence.
psCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true);

// Set WAL Mode.
psCfg.setWalMode(WALMode.LOG_ONLY);

// Enabling the Persistent Store.
cfg.setDataStorageConfiguration(psCfg);

//Additional parameters.

WAL Activation and Deactivation

WAL is an essential component of Ignite persistence that ensures durability and consistency even during cluster failures.

However, there are situations when it's reasonable to have the WAL disabled to get better performance. For instance, it is useful to disable WAL during initial data loading and enable it after the pre-loading is complete.

You can enable or disable write-ahead logging for a specific cache via IgniteCluster.enableWal(cacheName) and IgniteCluster.disableWal(cachename) methods, respectively. To check if write-ahead logging is enabled, use the IgniteCluster.isWalEnabled(cacheName) method.

For SQL, enabling/disabling of WAL is available as part of the ALTER TABLE command.

WAL Archive

The WAL archive is used to keeps WAL segments that may be needed to recover the node after a crash. The number of segments kept in the archive is such that the total size of all segments does not exceed the established size of the WAL archive.

The size of the WAL archive can be set via the DataStorageConfiguration.maxWalArchiveSize property. If the property is not specified in the configuration, the archive size is defined as 4 times the size of the checkpointing buffer.

Note that setting the WAL archive size to a value lower than the default one may affect performance and should be tested before being used in production.

Tuning WAL Archive

Below are some tips on how to tune space usage and cluster performance by tweaking WAL archive settings.

WAL Archive Compaction

You can enable WAL Archive compaction to reduce the space occupied by the WAL Archive. By default, WAL Archive contains segments for the last 20 checkpoints (this number is configurable). If compaction is enabled, all archived segments that are 1 checkpoint old will be compressed in ZIP format. In case the segments are required (for example, to re-balance data between nodes), they will be uncompressed to RAW format.

To enable WAL Archive compaction, set the DataStorageConfiguration.walCompactionEnabled property to true. You can also specify the compression level (1 indicates the fastest speed, and 9 indicates the best compression).

<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
  <!-- Enabling Apache Ignite Persistent Store. -->
  <property name="defaultDataRegionConfiguration">
    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
      <property name="persistenceEnabled" value="true"/>
    </bean>
  </property>
  
  <property name="walCompactionEnabled" value="true" />
  <property name="walCompactionLevel" value="6" />
</bean>
 
DataStorageConfiguration dsCfg = new DataStorageConfiguration();

DataRegionConfiguration regCfg = new DataRegionConfiguration();
regCfg.setPersistenceEnabled(true);
dsCfg.setDefaultDataRegionConfiguration(regCfg);

dsCfg.setWalCompactionEnabled(true);

Disabling WAL Archiving

In some cases, you may want to disable WAL archiving, for example, to reduce the overhead associated with copying of WAL segments to the archive — There can be a situation where Ignite writes data to WAL segments faster than the segments are copied to the archive. This may create an I/O bottleneck that can freeze the operation of the node. If you experience such problems, try switching WAL archiving off.

To disable archiving, set the WAL path and the WAL archive path to the same value. In this case, Ignite will not copy segments to the archive; instead, it will overwrite the active segments in a cyclical order.

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
   
  <property name="dataStorageConfiguration">
     <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
  		<!-- Enabling Apache Ignite Persistent Store. -->     
       <property name="defaultDataRegionConfiguration">
         <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
           <property name="persistenceEnabled" value="true"/>

           <property name="walPath" value="/wal/path"/>
           
           <property name="walArchivePath" value="/wal/path"/>
         </bean>
       </property>
     
     </bean>
   </property>

  <!-- Additional setting. -->

</bean>
 
DataStorageConfiguration dsCfg = new DataStorageConfiguration();

DataRegionConfiguration regCfg = new DataRegionConfiguration();
regCfg.setPersistenceEnabled(true);
dsCfg.setDefaultDataRegionConfiguration(regCfg);

String walAbsPath = "/wal/path";

dsCfg.setWalPath(walAbsPath);

dsCfg.setWalArchivePath(walAbsPath);