Endless warnings regarding disconnected connection after related durable notification member was removed due to long disconnection

Description

Durable notification replication target was removed at 9:47:
2017-03-28 09:47:01,202 ETFPROD10DubETF.1 [1] INFO [com.gigaspaces.replication.group.primary-backup-sync-1] - member [replication:NotifyDur:dubetfctx512[10832]_d403568b-993b-4ddc-829a-0f58e6d09dad] is disconnected for [300309ms] and is being dropped due to disconnection time limitations [300000ms], this member will be removed from the group

Few hours later 13:28 for same durable we still see that task of monitoring disconnected connection is active:
2017-03-28 13:28:18,303 ETFPROD10DubETF.1 [1] WARNING [com.gigaspaces.lrmi] - Connect Failed to [NIO://10.170.17.81:32101/pid[10832]/262470329817588_1_-1565097274208040573_details[class com.gigaspaces.internal.cluster.node.impl.router.AbstractConnectionProxyBasedReplicationRouter$ConnectionEndpoint(replication:NotifyDur:dubetfctx512[10832]_d403568b-993b-4ddc-829a-0f58e6d09dad)]]; Caused by: java.io.IOException: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
at com.gigaspaces.lrmi.nio.CPeer.createAsyncChannel(CPeer.java:275)
at com.gigaspaces.lrmi.nio.CPeer.connectAsync(CPeer.java:216)
at com.gigaspaces.lrmi.nio.CPeer.connect(CPeer.java:184)
at com.gigaspaces.lrmi.ConnectionPool.getConnection(ConnectionPool.java:111)
at com.gigaspaces.lrmi.ConnPoolInvocationHandler.invoke(ConnPoolInvocationHandler.java:53)
at com.gigaspaces.lrmi.MethodCachedInvocationHandler.invoke(MethodCachedInvocationHandler.java:71)
at com.gigaspaces.lrmi.DynamicSmartStub.invokeRemote(DynamicSmartStub.java:424)
at com.gigaspaces.lrmi.DynamicSmartStub.invoke(DynamicSmartStub.java:405)
at com.gigaspaces.reflect.$GSProxy12.dispatchAsync(Unknown Source)
at com.gigaspaces.internal.cluster.node.impl.router.AbstractConnectionProxyBasedReplicationRouter.pingStub(AbstractConnectionProxyBasedReplicationRouter.java:309)
at com.gigaspaces.internal.cluster.node.impl.router.DirectConnectionScheduledPoolConnectionMonitor$MonitorDisconnectedConnectionTask.run(DirectConnectionScheduledPoolConnectionMonitor.java:70)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused
at com.gigaspaces.async.SettableFuture.getResult(SettableFuture.java:43)
at com.gigaspaces.async.internal.AbstractFuture.get(AbstractFuture.java:64)
at com.gigaspaces.lrmi.nio.CPeer.createAsyncChannel(CPeer.java:269)
... 17 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at com.gigaspaces.lrmi.nio.selector.handler.client.Conversation.finishConnection(Conversation.java:110)
at com.gigaspaces.lrmi.nio.selector.handler.client.Conversation.handleKey(Conversation.java:70)
at com.gigaspaces.lrmi.nio.selector.handler.client.ClientConversationRunner.doSelect(ClientConversationRunner.java:60)
at com.gigaspaces.lrmi.nio.selector.handler.client.ClientConversationRunner.run(ClientConversationRunner.java:42)
... 1 more

Workaround

None

Acceptance Test

Tested manually

Assignee

Meron Avigdor

Reporter

Ester Atzmon

Labels

None

Priority

Medium

SalesForce Case ID

11052

Fix versions

Commitment Version/s

None

Due date

None

Product

XAP

Edition

Open Source

Platform

All

Sprint

None
Configure