Aggregation sum ignores task transactions

This issue was worked around by using one object that includes all related objects that should be committed in one transaction. Changing the object structures and making the queries and report code more complicated. and each object weight much more.

And solution can’t scale when more objects are involved

The customer is waiting for MVCC solution to have consistent query and aggregation results.

On Sep 30 Niv & I had a session with Wasim & Fahd. In this session Niv suggested 2 options to solve this issue.

1. Pessimistic concurrency - In the aggregation do read multiple with exclusive read lock.

2. Optimistic concurrency - scan the aggregate entries before and after the aggregation, verify that the entries haven't been modified by the change operations. This can be done by verifying that the entries version hasn't been changed. If it been changed, the query should be run again and again.

Wasim & Fahd prefered the second option and will try it out. The presentation and the code sample has been attache to FD4855

Attached slides and solution code.

Please close this issue.

The test is doing the following:

1. prepare 1K objects (properties: Integer id, Double risk) while the sum of all risk value is 0. This is done by setting 0 - 500 to half of them and -500 - 0 to the other half. so 1 + (-1) + 2 (-2) + ... = 0

2. write multiple the 1k objects.

3. start 5 threads. each run a loop from 0 to Integer MAX Value. In each iteration sum is performed: final Double sum = QueryExtension.sum(gigaSpace, new SQLQuery<>(TestBean.class, ""), "risk");

4. then a while true loop is executed. each iteration starts a transaction and commits at the end. Inside the transaction in each iteration, a transactional Task that received a Map with a changeSet: Map<Integer, ChangeSet> changeSetMap) is executed. The ChangeSet that passed to each Task is a change (by doubled with -1) of the risk values. So still the sun of the whole 1K risks is always 0.

5. the expectation is that the 5 thread (from point 3) will always receive sum = 0

6. worked with 1 partition.

7. running the test today I always got sum = 0 but it did reproduce a few months ago.

8. the test is attached.

9. can you think of a reason why the sum will not always return 0?

This is a WAD and the customer has a workaround.