Apache Ignite memory-centric platform is based on the durable memory architecture that allows storing and processing data and indexes both in memory and on disk when the Ignite Native Persistence feature is enabled. The durable memory architecture helps achieve in-memory performance with the durability of disk using all the available resources of the cluster.
Ignite's durable memory operates in a way similar to the virtual memory of operating systems, such as Linux. However, one significant difference between these two types of architecture is that the durable memory, whenever persistence is enabled, will always treat the disk as the superset of the data, capable of surviving crashes and restarts, while the traditional virtual memory uses the disk only as a swap extension, which gets erased once the process stops.
Since this architecture is memory-centric, the RAM is always treated as the first memory tier, where all the processing happens. Following are the characteristics and benefits of the in-memory part of the architecture:
- Off-heap Based. All the data and indexes are stored outside of Java heap that allows easy processing petabytes of data located in the cluster.
- Removes Noticeable Garbage Collection Pauses. Since all the data and indexes are stored off-heap, application's code is the only source possible for long stop-the-world pauses.
- Predictable memory usage. By default, durable memory uses as much RAM and disk space as possible. However, it's easy to set up memory utilization to suit your application requirements.
- Automatic memory defragmentation. Apache Ignite uses the memory as efficiently as possible and executes defragmentation routines in the background avoiding fragmentation.
- Improved performance and memory utilization. All the data and indexes are stored in paged format with similar representation in memory and on disk, which removes any need for serializing or deserializing of data.
Ignite native persistence adds the following benefits:
- Persistence is optional. Choose which data sets are stored only in memory, only on disk, or when memory is used as a caching layer for disk.
- Data Resiliency. Since Ignite persistent tier stores the full data set, it can survive cluster crashes and restarts, without loosing any data and while preserving its strong transactional consistency.
- Cache only hot data in memory. Store the superset of data on disk and a subset of data in memory. With the Ignite Native Persistence feature enabled, there is no need to fit all the data in RAM. The durable memory will continue to retain the hot data in RAM, automatically purging the cold data from memory when there is no more space is left.
- Execute SQL over the whole data sets. Most in-memory systems can query data only when it is preloaded in memory, therefore limiting the data size to the size of memory only. Ignite SQL will always span the whole distributed persisted data set, utilizing the in-memory caches for better performance. Essentially, Ignite can serve as a fully featured distributed transactional SQL database.
- Instantaneous cluster restarts. If the whole cluster goes down, it can restart and become operational immediately. The in memory caches will warm up automatically, as the data starts getting accessed.