hibernate/hibernate-search — pull request #700 by gustavonalle (2014-11-07 12:10:18+00:00)
Title: HSEARCH-1699 Batching IndexWriter commits for multiple synchronous works...

...ets

https://hibernate.atlassian.net/browse/HSEARCH-1699

There are still some minor issues to handle and more comprehensive performance tests to be done,
so this is to gather feedback only. All tests are passing and build time hasn't changed.

Some local perf data for:
* Infinispan + Hibernate Search indexing test [1]
* Single Cache Manager
* Writing 50k entities total, using variable number of threads. Infinispan generates many individual LuceneWork that are sent to the Hibernate Search engine

Threads | Docs per Thread | Commits Done | Time (sec)
---|---|---|---
1|50k|50k	|122
2|25k|~49k|120
5|10k|~20k|59
10|5k|~10k|36
20|2.5k|~4k|21
50|1k|~2k|11

For comparison, here are results for the current master sync:

Threads | Docs per Thread | Commits Done | Time (sec)
---|---|---|---
1|50k|50k	|112
2|25k|50k|118
5|10k|50k|109
10|5k|50k|111
20|2.5k|50k|108
50|1k|50k|111

As expected, under load the sync batching reduces the number of commits
considerably when compared to master, that will do one commit per LuceneWork.

[1] https://github.com/infinispan/infinispan/blob/master/query/src/test/java/org/infinispan/query/backend/MergeTest.java