Apache Ignite

GridGain Developer Hub - Apache Ignitetm

Welcome to the Apache Ignite developer hub run by GridGain. Here you'll find comprehensive guides and documentation to help you start working with Apache Ignite as quickly as possible, as well as support if you get stuck.

 

GridGain also provides Community Edition which is a distribution of Apache Ignite made available by GridGain. It is the fastest and easiest way to get started with Apache Ignite. The Community Edition is generally more stable than the Apache Ignite release available from the Apache Ignite website and may contain extra bug fixes and features that have not made it yet into the release on the Apache website.

 

Let's jump right in!

 

Documentation     Ask a Question     Download

 

Javadoc     Scaladoc     Examples
Ask A Question

Questions

1

Gridgain service on Docker

Hi, I am running my Gridgain server in docker container and have deployed the service. I am running my Intellij in host machine and trying to hit the service deployed in Gridgain docker container. I am always getting the error message as follows. Run command to start the container : docker run -it --net host -p 47500:47500 -p 47100:47100 apacheignite Error Message: Failed to connect to any address from IP finder (will retry to join topology every 2 secs): [/192.168.99.100:47500] Docker Machine default IP is : 192.168.99.100 My TCP configuration in both server and client : <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. --> <property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="localPort" value="47500"/> <property name="localPortRange" value="10"/> <!-- Setting up IP finder for this cluster --> <property name="ipFinder"> <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> <property name="addresses"> <list> <value>192.168.99.100:47500</value> </list> </property> </bean> </property> </bean> </property> My Client code from Intellij: Ignition.setClientMode(true); try (Ignite ignite = Ignition.start("sample.xml")) { GridGainService service = ignite.services().serviceProxy("GridGainService", GridGainService.class, true); return service.sayHello(); } But i am able to connect to the node using visor tool using ip and port like 192.168.99.100:11211. Can you help me to resolve this connectivity issue.

Posted by Saravanan Shanmugavel 9 days ago

1

class org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32

Hi Below is our architecture, Kafka version - 0.9, Ignite version - 1.9 1. Ignite receives data via Kafka Streamer. There are 27 Kafka partitions. We use 27 Ignite instances deployed on 3 physical nodes. 2. Tuple Extractor is implemented in ignite code 3. All the Caches are defined as partitioned 4. We also use streamvisitor for processing the data As part of failover testing, when we kill 1 instance (out of 27 instances) of ignite then we get the below error repeatedly on all the instances. Ultimately, all the data from Kafka is not loaded to Ignite cache. class org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32 Detailed log preceding the error: [17:27:18,633][INFO][sys-#136%null%][GridDhtPartitionDemander] <TOKEN_GRID_CACHE> Completed rebalancing [fromNode=51ef8616-b71d-4c4f-8387-a8119a9b4a1a, topology=AffinityTopologyVersion [topVer=520, minorTopVer=0], time=31 ms] [17:27:18,645][INFO][sys-#264%null%][GridDhtPartitionDemander] <TOKEN_GRID_CACHE> Completed rebalancing [fromNode=0774202a-fb4e-4698-8931-1c1d2ed89136, topology=AffinityTopologyVersion [topVer=520, minorTopVer=0], time=41 ms] [17:27:18,647][INFO][sys-#151%null%][GridDhtPartitionDemander] <TOKEN_GRID_CACHE> Completed rebalancing [fromNode=a1c7665f-5902-4ff3-8435-5c8cfb902d58, topology=AffinityTopologyVersion [topVer=520, minorTopVer=0], time=41 ms] [17:27:18,648][INFO][sys-#265%null%][GridDhtPartitionDemander] <TOKEN_GRID_CACHE> Completed rebalancing [fromNode=034dab7d-1929-4024-95c4-cb9c5e49395b, topology=AffinityTopologyVersion [topVer=520, minorTopVer=0], time=41 ms] [17:27:18,667][INFO][sys-#195%null%][GridDhtPartitionDemander] <TOKEN_GRID_CACHE> Completed rebalancing [fromNode=605f74e9-1c66-4698-9029-9b3ac12c5b74, topology=AffinityTopologyVersion [topVer=520, minorTopVer=0], time=62 ms] [17:27:18,687][INFO][sys-#154%null%][GridDhtPartitionDemander] <TOKEN_GRID_CACHE> Completed (final) rebalancing [fromNode=5cc1ae33-d9c3-4d19-b360-fa344a50babe, topology=AffinityTopologyVersion [topVer=520, minorTopVer=0], time=82 ms] [17:27:39,093][INFO][grid-timeout-worker-#63%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) [17:27:39,093][INFO][grid-timeout-worker-#63%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=a9bbc138, name=null, uptime=00:08:00:026] ^-- H/N/C [hosts=3, nodes=27, CPUs=168] ^-- CPU [cur=4.67%, avg=3.9%, GC=0.13%] ^-- Heap [used=2510MB, free=37.35%, comm=4007MB] ^-- Non heap [used=92MB, free=93.89%, comm=95MB] ^-- Public thread pool [active=0, idle=6, qSize=0] ^-- System thread pool [active=0, idle=56, qSize=0] ^-- Outbound messages queue [size=0] [17:28:39,094][INFO][grid-timeout-worker-#63%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=a9bbc138, name=null, uptime=00:09:00:028] ^-- H/N/C [hosts=3, nodes=27, CPUs=168] ^-- CPU [cur=0.07%, avg=3.46%, GC=0%] ^-- Heap [used=2627MB, free=34.43%, comm=4007MB] ^-- Non heap [used=92MB, free=93.9%, comm=95MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=6, qSize=0] ^-- Outbound messages queue [size=0] [17:29:39,101][INFO][grid-timeout-worker-#63%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=a9bbc138, name=null, uptime=00:10:00:035] ^-- H/N/C [hosts=3, nodes=27, CPUs=168] ^-- CPU [cur=0.07%, avg=3.09%, GC=0%] ^-- Heap [used=2660MB, free=33.61%, comm=4007MB] ^-- Non heap [used=92MB, free=93.9%, comm=95MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=7, qSize=0] ^-- Outbound messages queue [size=0] [17:30:39,106][INFO][grid-timeout-worker-#63%null%][IgniteKernal] [17:30:39,106][INFO][grid-timeout-worker-#63%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=a9bbc138, name=null, uptime=00:11:00:042] ^-- H/N/C [hosts=3, nodes=27, CPUs=168] ^-- CPU [cur=0.07%, avg=2.79%, GC=0%] ^-- Heap [used=2693MB, free=32.79%, comm=4007MB] ^-- Non heap [used=92MB, free=93.9%, comm=95MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=6, qSize=0] ^-- Outbound messages queue [size=0] [17:31:39,116][INFO][grid-timeout-worker-#63%null%][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=a9bbc138, name=null, uptime=00:12:00:050] ^-- H/N/C [hosts=3, nodes=27, CPUs=168] ^-- CPU [cur=0.13%, avg=2.55%, GC=0%] ^-- Heap [used=2726MB, free=31.96%, comm=4007MB] ^-- Non heap [used=92MB, free=93.88%, comm=95MB] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=0, idle=7, qSize=0] ^-- Outbound messages queue [size=0] [17:31:55,555][SEVERE][pub-#541%null%][DataStreamerImpl] DataStreamer operation failed. class org.apache.ignite.IgniteCheckedException: Failed to finish operation (too many remaps): 32 at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:863) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:828) at org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:456) at org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:439) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271) at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:259) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:389) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:355) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1787) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:335) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1222) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:850) at org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108) at org.apache.ignite.internal.managers.communication.GridIoManager$7.run(GridIoManager.java:790) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer request failed [node=187baa13-eb61-45d0-9c56-91ea81b9de55] at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1775) ... 8 more Caused by: java.lang.NullPointerException [17:31:55,559][SEVERE][pool-3-thread-1][] Message is ignored due to an error [msg=MessageAndMetadata(test18042017_884,2,Message(magic = 1, attributes = 0, CreateTime = -1, crc = 2732848622, key = java.nio.HeapByteBuffer[pos=0 lim=5 cap=506], payload = java.nio.HeapByteBuffer[pos=0 lim=497 cap=497]),4445657,kafka.serializer.DefaultDecoder@77a040f,kafka.serializer.DefaultDecoder@2b1fbd73,-1,CreateTime)] java.lang.IllegalStateException: Data streamer has been closed. at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667) at tcs.poc.surveillance.SurveillanceAlert$2.extract(SurveillanceAlert.java:481) at tcs.poc.surveillance.SurveillanceAlert$2.extract(SurveillanceAlert.java:1) at org.apache.ignite.stream.StreamAdapter.addMessage(StreamAdapter.java:181) at org.apache.ignite.stream.kafka.KafkaStreamer.access$100(KafkaStreamer.java:47) at org.apache.ignite.stream.kafka.KafkaStreamer$1.run(KafkaStreamer.java:156) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [17:31:55,562][SEVERE][pool-3-thread-1][] Message is ignored due to an error [msg=MessageAndMetadata(test18042017_884,2,Message(magic = 1, attributes = 0, CreateTime = -1, crc = 3750537118, key = java.nio.HeapByteBuffer[pos=0 lim=5 cap=517], payload = java.nio.HeapByteBuffer[pos=0 lim=508 cap=508]),4445767,kafka.serializer.DefaultDecoder@77a040f,kafka.serializer.DefaultDecoder@2b1fbd73,-1,CreateTime)]

Posted by Rishikesh Bansod 10 days ago