Apache Ignite Documentation

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Memory Defragmentation

How Durable Memory Deals With Memory Fragmentation

📘

Automatic Defragmentation

Memory defragmentation in Ignite happens automatically and does not require any explicit action from a user.

Ignite Durable Memory keeps all the data in a special type of pages called data pages. Over time, as a result of CRUD operations, each individual page might be updated multiple times. This can lead to page and overall memory fragmentation.

To minimize the memory fragmentation, Ignite uses page compaction whenever a specific page gets too fragmented.

A compacted data page looks like the one in the picture below:

Compacted Data Page

The page has a header that stores information needed for internal usage. All key-value entries are always added right-to-left. In the picture, there are three entries (1, 2 and 3 respectively) stored in the page. These entries might be of different size.

The offsets (or references) to the entries' locations inside the page are stored left-to-right and are always of fixed size. The offsets are used as pointers to look up the key-value entries in a page.

Everything in the middle is a free space and will be filled in whenever more data is pushed into the cluster.

Next, let's assume that over the time entry 2 was removed which led to non-continuous free space in the page:

Fragmented Page

This is what a fragmented page looks like.

However, when the whole free space available in the page is needed or some fragmentation threshold is reached, the compaction process will defragment the page turning it into the state shown in the first picture above, where the free space is continuous. This process is automatic and doesn't require any action from the user side.

📘

Memory Use and Deleting Data from Caches

If you delete data from Ignite caches/tables, the cache size and memory use will not decrease because the memory is assigned to Ignite by the OS (and is reserved for later reuse). As for disk/physical memory, typically the Write Ahead Log (WAL) and the contents of Ignite database files on disk only grow in size. That disk space is only freed when caches are deleted. Therefore, if you want to release memory to the OS or delete data to save disk space, you should delete the cache.

Updated about a year ago

Memory Defragmentation


How Durable Memory Deals With Memory Fragmentation

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.