elasticsearch/elasticsearch — pull request #8557 by bleskes (2014-11-19 17:37:48+00:00)
Title: Internal: IndexService - synchronize  close to prevent race condition with shard creation

During node shutdown we have a race condition between processing cluster state updates (creating shards) and closing down the index service. This may cause shards to leak and not be closed properly.

This commit removes the concurrency in shard closing as InternalIndexService.removeShard has been synchronized for a long time now.

On the other hand, the commit restores the parallel shutdown of indices lost in 7e1d8a6ca3b23c07cbaebe72831c7ef4a201bd2b

This is the cause of http://build-us-00.elasticsearch.org/job/es_core_1x_strong/1406/testReport/junit/org.elasticsearch.gateway.local/LocalGatewayIndexStateTests/testDanglingIndicesNoAutoImport/

```
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open locks: {write.lock=java.lang.RuntimeException: lock "write.lock" was not released}

Caused by: java.lang.RuntimeException: lock "write.lock" was not released
at org.apache.lucene.store.MockLockFactoryWrapper$MockLock.obtain(MockLockFactoryWrapper.java:74)
at org.apache.lucene.store.Lock.obtain(Lock.java:77)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:753)
at org.elasticsearch.index.engine.internal.InternalEngine.createWriter(InternalEngine.java:1453)
at org.elasticsearch.index.engine.internal.InternalEngine.start(InternalEngine.java:273)
at org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryPrepareForTranslog(InternalIndexShard.java:733)
```