elasticsearch/elasticsearch — Issue #1451: OverlappingFileLockException
Author: splix
Created: 2011-11-10 14:49:43+00:00
State: open

ElasticSearch sometimes fails with error like:

```
Error injecting constructor, java.nio.channels.OverlappingFileLockException
at org.elasticsearch.env.NodeEnvironment.(Unknown Source)
while locating org.elasticsearch.env.NodeEnvironment
for parameter 6 at org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData.(Unknown Source)
while locating org.elasticsearch.indices.store.TransportNodesListShardStoreMetaData
for parameter 2 at org.elasticsearch.gateway.local.LocalGatewayAllocator.(Unknown Source)
while locating org.elasticsearch.gateway.local.LocalGatewayAllocator
while locating org.elasticsearch.cluster.routing.allocation.allocator.GatewayAllocator
for parameter 1 at org.elasticsearch.cluster.routing.allocation.allocator.ShardsAllocators.(Unknown Source)
while locating org.elasticsearch.cluster.routing.allocation.allocator.ShardsAllocators
for parameter 2 at org.elasticsearch.cluster.routing.allocation.AllocationService.(Unknown Source)
while locating org.elasticsearch.cluster.routing.allocation.AllocationService
for parameter 5 at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService.(Unknown Source)
while locating org.elasticsearch.cluster.metadata.MetaDataCreateIndexService
for parameter 4 at org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.(Unknown Source)
while locating org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction
for parameter 6 at org.elasticsearch.action.index.TransportIndexAction.(Unknown Source)
while locating org.elasticsearch.action.index.TransportIndexAction
for parameter 3 at org.elasticsearch.client.node.NodeClient.(Unknown Source)
while locating org.elasticsearch.client.node.NodeClient
while locating org.elasticsearch.client.Client
for parameter 1 at org.elasticsearch.rest.action.admin.cluster.node.shutdown.RestNodesShutdownAction.(Unknown Source)
while locating org.elasticsearch.rest.action.admin.cluster.node.shutdown.RestNodesShutdownAction
Caused by: java.nio.channels.OverlappingFileLockException
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.checkList(FileChannelImpl.java:1166)
at sun.nio.ch.FileChannelImpl$SharedFileLockTable.add(FileChannelImpl.java:1068)
at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:868)
at java.nio.channels.FileChannel.tryLock(FileChannel.java:962)
at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:216)
at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:74)
at sun.reflect.GeneratedConstructorAccessor31.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:54)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:103)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:823)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:56)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:49)
at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:103)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:823)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:56)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:49)
at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:103)
at org.elasticsearch.common.inject.FactoryProxy.get(FactoryProxy.java:56)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:823)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:56)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:49)
at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:103)
at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:103)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:823)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:56)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:49)
at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:103)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:45)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:823)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:42)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:56)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:49)
at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
```

At this case it displays this stacktrace infinitely, eats all resources and can be killed only by kill -9.

Happens under Ubuntu 11.04, both 32 and 64 bit.

```
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)
```

and

```
java version "1.6.0_23"
OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b23~pre10-0ubuntu5)
OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
```