Apache Ignite Documentation

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Memory Metrics

Durable Memory Metrics

Data Region Metrics

Ignite durable memory can be monitored via several parameters exposed through the DataRegionMetrics interface, as well as JMX bean. Having access to the data region metrics can help track overall memory utilization, measure its performance, and execute required optimizations.

The DataRegionMetrics interface is the main entry point that provides memory-related metrics of a specific Ignite node. Since there can be several regions configured on a node, metrics for every region are collected and obtained individually.

Enabling Data Region Metrics

To enable data region metrics, set DataRegionConfiguration.setMetricsEnabled(true) for every region you want to collect the metrics for.

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  <property name="dataStorageConfiguration">
    <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
      <property name="dataRegionConfigurations">
        <list>
          <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
            <!-- Custom region name. -->
            <property name="name" value="myDataRegion"/>

            <!-- Enable metrics for this data region  -->
            <property name="metricsEnabled" value="true"/>
            
            <!-- Other configurations -->
            ...
          </bean>
        </list>
      </property>
    </bean>
  </property>
  
  <!-- Other Ignite configurations -->
  ...
</bean>
// Ignite configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Durable Memory configuration.
DataStorageConfiguration storageCfg = new DataStorageConfiguration();

// Create a new data region.
DataRegionConfiguration regionCfg = new DataRegionConfiguration();

// Region name.
regionCfg.setName("myDataRegion");

// Enabe metrics for this region.
regionCfg.setMetricsEnabled(true);

// Set the data region configuration.
storageCfg.setDataRegionConfigurations(regionCfg);

// Other configurations
...

// Apply the new configuration.
cfg.setDataStorageConfiguration(storageCfg);

Getting Metrics

Use Ignite.dataRegionMetrics() interface method or a respective JMX bean to get the latest metrics snapshot and iterate over it, as shown in the example below:

// Get the metrics of all the data regions configured on a node.
Collection<DataRegionMetrics> regionsMetrics = ignite.dataRegionMetrics();
            
// Print out some of the metrics.
for (DataRegionMetrics metrics : regionsMetrics) {
    System.out.println(">>> Memory Region Name: " + metrics.getName());
    System.out.println(">>> Allocation Rate: " + metrics.getAllocationRate());
    System.out.println(">>> Fill Factor: " + metrics.getPagesFillFactor());
    System.out.println(">>> Allocated Size: " + metrics.getTotalAllocationSize());
    System.out.println(">>> Physical Memory Size: " + metrics.getPhysicalMemorySize());
}

Following is the list of metrics available for data region:

Method Name
Description

getName()

Returns the name of a data region the metrics belong to.

getTotalAllocatedPages()

Gets the total number of allocated pages related to the data region. When Ignite persistence is disabled, this metric shows the total number of pages in RAM. When Ignite persistence is enabled, this metric shows the total number of pages in memory and on disk.

getAllocationRate()

Gets pages allocation rate of this region.

getEvictionRate()

Gets pages eviction rate of this region.

getLargeEntriesPagesPercentage()

Gets the percentage of pages that are fully occupied by large entries that go beyond the page size. Large entities are split into fragments in a way that each fragment can fit into a single page.

getPagesFillFactor()

Gets the percentage of used space.

getDirtyPages()

Gets the number of dirty pages (pages which content varies from the content of the same pages on disk). This metric is used only when the Ignite persistence is enabled.

getPagesReplaceRate()

Gets the rate (pages per second) at which pages that are in RAM are replaced with other pages from disk. The metric effectively represents the rate at which pages get 'evicted' from RAM in favor of bringing other pages​ from disk. This metric is used only when the Ignite persistence is enabled.

getPhysicalMemoryPages()

Gets the number of pages currently loaded in RAM. When Ignite persistence is disabled, this metric is the same as getTotalAllocatedPages().

getTotalAllocatedSize()

Gets the total size (in bytes) of memory allocated to the data region. When Ignite persistence is disabled, this metric shows the total size of pages in RAM. When Ignite persistence is enabled, this metric shows the total size of pages in memory and on disk.

getPhysicalMemorySize()

Gets the total size (in bytes) of pages loaded in RAM. When persistence is disabled, this metric is same as getTotalAllocatedSize().

getCheckpointBufferPages()

Gets checkpoint buffer size in pages.

getCheckpointBufferSize()

Gets checkpoint buffer size in bytes.

getPageSize()

Gets memory page size.

Using JMX Bean

All DataRegionMetrics of a local Apache Ignite node are visible through JMX interface - DataRegionMetricsMXBean. You can connect to the bean from any JMX-compliant tool or API.

Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a special JMX bean to activate collecting of data region metrics.

The JMX beans expose the same set of metrics that DataRegionMetrics has, as well as a few additional ones. See DataRegionMetricsMXBean JavaDoc for more details.

Ignite Persistence Metrics

Ignite also provides a set of metrics for Ignite persistence when it's enabled for a specific data region covered above. These metrics are grouped under the DataStorageMetrics interface.

Enabling Data Storage Metrics

To enable Ignite presistence metrics, set DataStorageConfiguration.setMetricsEnabled(true), like so:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  <property name="dataStorageConfiguration">
    <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
      <!-- Enable metrics for Ignite persistence  -->
      <property name="metricsEnabled" value="true"/>
      
      <!-- Other configurations -->
      ...
    </bean>
  </property>
  
  <!-- Other Ignite configurations -->
  ...
</bean>
// Ignite configuration.
IgniteConfiguration cfg = new IgniteConfiguration();

// Durable Memory configuration.
DataStorageConfiguration storageCfg = new DataStorageConfiguration();

// Enable metrics for Ignite persistence.
storageCfg.setMetricsEnabled(true);

// Other configurations
...

// Apply the new configuration.
cfg.setDataStorageConfiguration(storageCfg);

Getting Metrics

Call Ignite.dataStorageMetrics() method to get the latest persistence metrics snapshot, as shown in the example below:

// Getting metrics.
DataStorageMetrics pm = ignite.dataStorageMetrics();

System.out.println("Fsync duration: " + pm.getLastCheckpointFsyncDuration());

System.out.println("Data pages: " + pm.getLastCheckpointDataPagesNumber());

System.out.println("Checkpoint duration:" + pm.getLastCheckpointDuration());

Following are some of the available metrics for data storage:

Method Name
Description

getWalWritingRate()

Gets an average number of WAL records per second written during the last configured time interval.

getWalArchiveSegments()

Gets a current number of WAL segments in the WAL archive.

getWalFsyncTimeAverage()

Gets an average WAL fsync duration in microseconds over the last configured time interval.

getLastCheckpointDuration()

Gets duration of the last checkpoint process in milliseconds.

getLastCheckpointTotalPagesNumber()

Gets a total number of pages written during the last checkpoint process

For a complete list, see DataStorageMetrics JavaDoc.

Using JMX Bean

Ignite persistence metrics can also be collected through the JMX interface - DataStorageMetricsMXBean. You can connect to the bean from any JMX-compliant tool or API.

Use the DataStorageMetricsMXBean.enableMetrics() method exposed by a special JMX bean to activate collecting of Ignite persistence related metrics.

The JMX beans expose the same set of metrics that DataStorageMetrics has, as well as a few additional ones. See DataStorageMetricsMXBean JavaDocs for more details.

Enabling Metrics Collection

Metrics collection is not a free operation and might affect the performance of an application. For this reason, the metrics is turned off by default.

To turn the metrics on, use one of the following approaches:

  • Set DataRegionConfiguration.setMetricsEnabled(true) for every region you want to collect the metrics for.
  • Set DataStorageConfiguration.setMetricsEnabled(true) to collect Ignite persistence metrics.
  • Use the DataRegionMetricsMXBean.enableMetrics() method exposed by a special JMX bean.
  • Use the DataStorageMetricsMXBean.enableMetrics() method exposed by a special JMX bean to activate collecting of Ignite persistence related metrics.

Memory Usage Calculation

You can also obtain metrics for caches associated with a particular CacheGroup. Currently, these metrics are available only through JMX interface - CacheGroupMetricsMXBean. Refer to CacheGroupMetricsMXBean JavaDoc for a complete list of metrics available.

Single Node Memory Usage

The following examples show how to calculate current node size - the total size of data on a node in MB/GB, and current cache size - the size of data in a cache in MB/GB.

  1. Current node size is DataStorageMetricsMXBean#getTotalAllocatedSize

  2. The current size of a specific cache on a node is CacheGroupMetricsMXBean#getTotalAllocatedSize. Note that there should be only one cache within a cache group (default behavior) to make use of the metric.

Cluster-Wide Memory Usage

  1. To calculate the total cluster size, you can sum the DataStorageMetricsMXBean#getTotalAllocatedSize of all nodes.

  2. Current cache size is the sum of CacheGroupMetricsMXBean#getTotalAllocatedSize of all nodes. Note that there should be only one cache within a cache group (default behavior) to make use of the metric.

Memory Metrics


Durable Memory Metrics

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.