Apache Ignite

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples

Kubernetes Deployment

Kubernetes Deployment Getting Started Guide

Overview

Apache Ignite cluster can be easily deployed in and maintained by Kubernetes which is an open-source system for automating deployment, scaling, and management of containerized applications.

This getting started guide walks you through how to deploy an Apache Ignite cluster in Kubernetes environment.

Using minikube with Apple Macintosh Computers

A very useful tool for testing Kubernetes locally is to use Minikube. On Apple Macintosh Computers, it is necessary to use the xhyve driver to ensure that persistent volumes can be correctly accessed, as follows:

minikube start --vm-driver=xhyve

Furthermore, you might need to modify the directory path "/data/ignite" that is used in the examples below to "/Users/<username>/data/ignite", where <username> is substituted with the actual user name.

Kubernetes IP Finder

To enable Apache Ignite nodes auto-discovery in Kubernetes, you need to enable TcpDiscoveryKubernetesIpFinder in IgniteConfiguration. Let's create an example configuration file called example-kube.xml and define the IP finder configuration as follows:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">

<bean id="ignite.cfg"
    class="org.apache.ignite.configuration.IgniteConfiguration">

    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
        		<property name="ipFinder">
            		<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
            		</bean>
            </property>
        </bean>
    </property>
</bean>
</beans>

Kubernetes IP finder

To learn more about Kubernetes IP finder and Apache Ignite nodes auto-discovery in Kuberentes environment, refer to this documentation page.

Kubernetes Ignite Lookup Service

Ignite's KubernetesIPFinder requires users to configure and deploy a special Kubernetes service that maintains a list of the IP addresses of all the alive Ignite pods (nodes).

Every time a new Ignite pod is started, the IP finder will connect to the service via the Kubernetes API to obtain the list of the existing Ignite pods' addresses. Using these addresses, the new node will be able to discover the rest of the cluster nodes and finally join the Apache Ignite cluster.

Let's configure the service the following way:

apiVersion: v1
kind: Service
metadata:
  # Name of Ignite Service used by Kubernetes IP finder. 
  # The name must be equal to TcpDiscoveryKubernetesIpFinder.serviceName.
  name: ignite
spec:
  clusterIP: None # custom value.
  ports:
    - port: 9042 # custom value.
  selector:
    # Must be equal to one of the labels set in Ignite pods'
    # deployement configuration.
    app: ignite

and deploy it in Kubernetes using the command below:

kubectl create -f ignite-service.yaml

Make sure the service is up and running:

 kubectl get svc ignite

The output should be similar to the one below:

NAME      CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
ignite    None         <none>        9042/TCP   29s

Sharing Ignite Cluster Configuration

Before you start deploying Ignite pods in Kubernetes using Apache Ignite docker image, you need to find a way on how to pass example-kube.xml (prepared above) to that docker image.

There are several approaches you can use. Here, we show you how to share the Ignite cluster configuration via a shared Kubernetes PersistentVolume.

Sharing the configuration via an HTTP server

Deploying Apache Ignite in Kubernetes on Microsoft Azure blog post explains how to share an Apache Ignite configuration over an HTTP server.

Let's suppose that you have some shared directory named /data/ignite which can be accessed by any Ignite pod. Go to this directory and copy example-kube.xml there.

Create a PersistentVolume configuration that will be backed by your real storage and will refer to the /data/ignite directory.

kind: PersistentVolume
apiVersion: v1
metadata:
  name: ignite-volume
  labels:
    type: local
spec:
  storageClassName: manual
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/data/ignite"

Deploy the volume using the command below:

kubectl create -f ignite-volume.yaml

Check that the volume was deployed and available for usage:

kubectl get pv ignite-volume

The output should be similar to the one below:

NAME            CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM     REASON    AGE
ignite-volume   1Gi        RWO           Retain          Available                       3m

Finally, create a PersistentVolumeClaim that will be automatically bound to the PersistentVolume initiated above:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: ignite-volume-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Create the PersistentVolumeClaim using the configuration above:

kubectl create -f ignite-volume-claim.yaml 

Make sure that the PersistentVolumeClaim gets created and was bound to the PersistentVolume that stores example-kube.xml configuration.

kubectl get pvc ignite-volume-claim

NAME                  STATUS    VOLUME          CAPACITY   ACCESSMODES   AGE
ignite-volume-claim   Bound     ignite-volume   1Gi        RWO           2m

kubectl get pv ignite-volume

NAME            CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM                         REASON    AGE
ignite-volume   1Gi        RWO           Retain          Bound     default/ignite-volume-claim             13m

Kubernetes will look for a volume that matches the storageClassName of the PersistentVolumeClaim. If no PersistentVolume matches the storageClassName requested by the PersistentVolumeClaim, Kubernetes will dynamically create a new PersistentVolume. You can see all PersistentVolumes by running:

kubectl get pv

If the storageClassName did not match, you might see something like:

kubectl get pvc
NAME                  STATUS    VOLUME                                     CAPACITY   ACCESSMODES   STORAGECLASS   AGE
ignite-volume-claim   Bound     pvc-c3f0f289-9987-11e7-876b-525400ed8e86   1Gi        RWO           standard       9m

kubectl get pv
NAME                                       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM                         STORAGECLASS   REASON    AGE
ignite-volume                              1Gi        RWO           Retain          Available                                                          9m
pvc-c3f0f289-9987-11e7-876b-525400ed8e86   1Gi        RWO           Delete          Bound       default/ignite-volume-claim   standard                 9m

Now, it's time to prepare a Kubernetes deployment configuration for Ignite pods and deploy them

Ignite Pods Deployment

Finally, let's define a YAML configuration for Ignite pods:

# An example of a Kubernetes configuration for Ignite pods deployment.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  # Custom Ignite cluster's name.
  name: ignite-cluster
spec:
  # A number of Ignite pods to be started by Kubernetes initially.
  replicas: 2
  template:
    metadata:
      labels:
        # This label has to be added to the selector's section of 
        # ignite-service.yaml so that the Kubernetes Ignite lookup service
        # can easily track all Ignite pods available deployed so far.
        app: ignite
    spec:
      volumes:
        # Custom name for the storage that holds Ignite's configuration
        # which is example-kube.xml.
        - name: ignite-storage
          persistentVolumeClaim:
           # Must be equal to the PersistentVolumeClaim created before.
           claimName: ignite-volume-claim	
      containers:
        # Custom Ignite pod name.
      - name: ignite-node
        # Ignite Docker image. Kubernetes IP finder is supported starting from
        # Apache Ignite 1.9.0
        image: apacheignite/ignite:1.9.0
        env:
        # Ignite's Docker image parameter. Adding the jar file that
        # contain TcpDiscoveryKubernetesIpFinder implementation.
        - name: OPTION_LIBS
          value: ignite-kubernetes
        # Ignite's Docker image parameter. Passing the Ignite configuration
        # to use for an Ignite pod.
        - name: CONFIG_URI
          value: file:////data/ignite/example-kube.xml
        ports:
        # Ports to open.
        # Might be optional depending on your Kubernetes environment.
        - containerPort: 11211 # REST port number.
        - containerPort: 47100 # communication SPI port number.
        - containerPort: 47500 # discovery SPI port number.
        - containerPort: 49112 # JMX port number.
        volumeMounts:
        # Mounting the storage with the Ignite configuration.
        - mountPath: "/data/ignite"
          name: ignite-storage
          

As you can see, the configuration defines a couple of environment variables (OPTION_LIBS and CONFIG_URIL) that will be processed by a special shell script used by Ignite's docker image. The full list of docker image's configuration parameters is available on Docker Deployment page.

Ignite Docker Image Version

Kubernetes is supported in Apache Ignite 1.9 and later versions. Makes sure to use a docker image with a valid version. You can find a full list of tags here.

Ignite Configuration Shared over HTTP

If it's tricky for you to share the Apache Ignite configuration via PersistentVolumeClaim then you can upload it to an HTTP server and set the link as a value of CONFIG_URI parameter.

Next, go ahead and deploy Ignite pods in Kubernetes using the configuration​ above:

kubectl create -f ignite-deployment.yaml

Check that Ignite pods are up and running:

kubectl get pods

Pick a name of one of the pods available

NAME                              READY     STATUS    RESTARTS   AGE
ignite-cluster-3454482164-d4m6g   1/1       Running   0          25m
ignite-cluster-3454482164-w0xtx   1/1       Running   0          25m

and get the logs from it making sure that both Ignite pods were able to discover each other and from the cluster:

kubectl logs ignite-cluster-3454482164-d4m6g

Adjusting Ignite Cluster Size

You can adjust Apache Ignite cluster size on the fly using the standard Kubernetes API. For instance, if you want to scale out the cluster from 2 to 5 nodes then the command below can be used:

kubectl scale --replicas=5 -f ignite-deployment.yaml

Double check the cluster was scaled out successfully:

kubectl get pods

The output should show that you now have 5 Ignite pods up and running:

NAME                              READY     STATUS    RESTARTS   AGE
ignite-cluster-3454482164-d4m6g   1/1       Running   0          34m
ignite-cluster-3454482164-ktkrr   1/1       Running   0          58s
ignite-cluster-3454482164-r20f8   1/1       Running   0          58s
ignite-cluster-3454482164-vf8kh   1/1       Running   0          58s
ignite-cluster-3454482164-w0xtx   1/1       Running   0          34m

Deploying on Microsoft Azure

Deploying Apache Ignite in Kubernetes on Microsoft Azure article provides a step-by-step guide on how to run the cluster on Microsoft Azure.

Deploying on Amazon AWS

Kubernetes and Apache Ignite Deployment on AWS article provides a step-by-step guide on how to run the cluster on Amazon AWS.

Kubernetes Deployment

Kubernetes Deployment Getting Started Guide