Backup Space instance failed during a recovery attempt due to an IllegalStateException thrown from the Curator framework. This is caused by the closing of the attribute store done as part of bug fix GS-13502.
The fix for is to close the ZooKeeper attribute store, which in turn closes the Curator framework - shared by the attribute store and the leader selector. Due to this dependency, we got the following exception: "Failed while getting participants from zookeeper server: java.lang.IllegalStateException: instance must be started before calling this method".
Log extract:
2018-04-15 11:02:05,978 data-processor.31 [2] INFO [com.gigaspaces.space.space.31_1] - Space recovery failed - selecting primary
2018-04-15 11:02:05,979 data-processor.31 [2] WARNING [com.gigaspaces.space.active-election.space.31_1] - Failed while getting participants from zookeeper server: java.lang.IllegalStateException: instance must be started before calling this method
2018-04-15 11:02:06,979 data-processor.31 [2] WARNING [com.gigaspaces.space.active-election.space.31_1] - Failed while getting participants from zookeeper server: java.lang.IllegalStateException: instance must be started before calling this method
large cluster