Repository: conda/conda-build

Issue:
Unable to acquire lock on Windows since 2.0
I'm currently unable to build on our build servers as conda-build just keeps timing out due to not being able to acquire the lock in time. I've changed down to only 1 build at a time and I'm still seeing the error. I run `conda clean --lock` before every build.

[Example 1](https://jenkins.menpo.org/job/menpo-pr/309/ARCH=x86,PYTHON_VERSION=3.4,label=windows/console)
[Example 2](https://jenkins.menpo.org/job/menpo-pr/309/ARCH=x86,PYTHON_VERSION=3.5,label=windows/console)

**Error**

```
File lock could on C:\miniconda\0\x86\pkgs\.conda_lock not be obtained.  You might need to try fewer builds at once.  Otherwise, run conda clean --lock
```

**System**

```
conda-build               2.0.1                    py35_0  
conda                     4.1.12                   py35_0
```

```
Windows 10 x64
NTFS File system - old 7200 RPM drive
```

This is likely because the machine is pretty slow. I'm wondering if it might be possible to have a flag for conda-build to ignore the locking mechanism as, in this case, I am absolutely certain only one build will be activated at a time. Or perhaps a flag to be able to customise the timeout interval?



Discussion:
I have seen issues with old lock files hanging around, but after I clear them (with conda clean --lock), these go away.  I think that these issues are new because conda-build just wasn't doing any locking before, so any cruft left over from failed conda runs was not affecting things.

If you clear the lock, does this problem persist?

OK - I deleted the miniconda installations entirely and recreated them and now everything seems to be OK - I'll keep an eye on it and if it doesn't resurface I'll close this.

I'm still seeing this [pretty often](https://jenkins.menpo.org/job/menpo-pr/314/ARCH=x64,PYTHON_VERSION=2.7,label=windows/console). Even after deleting the whole miniconda repository and starting again.

Dang.  Just so I can hope to narrow this down, are there any patterns to it?  For example, if a build fails for some other reason, then do subsequent builds fail because of the lock?

I'd really rather keep the locks in place and fix them than provide a hack to disable them.

The tricky thing right now is that the lock mechanism I'm using is not reentrant.  Thus, I can't use context managers in some places.  If I haven't handled all the possible exception paths, I might be missing cleanup of the lock file at some point.  I may be able to restructure things so that I can use context managers, though.

I always run `conda clean --lock` before each executor kicks off. This is happening through Jenkins. I've examined the failed build and it appears there is no lock file at all. I think that it's actually just that the machine is really slow and acquiring the lock is just taking forever for some reason. 

Yeah, I believe you, but I'm just really surprised.  The timeout is 90 seconds.  It seems to me like even an arduino should be faster than that.

I might have a related problem on Linux. My machine is quite fast though, so it should not be a hardware limitation. I made this [post](https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/xNp9oMi2vQE) in the anaconda mailing list about it. What does seem to take forever is the "updating index in:" during the build step.

Yes - it looks like it might be failing to grab the lock after some other operation that takes a long time like computing the dependencies? It seems to freeze for a long time on `BUILD START` and then fail.

Updating the index in the pkgs folder seems the likely culprit.  I will cut that out, but I'm away from coding for this week on paternity leave.  Sorry for the trouble in the meantime.  As a workaround, I suggest hacking your conda_build/config.py to set the default timeout to some longer value.  Default is 90 sec right now.


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -407,10 +407,10 @@ def create_env(prefix, specs, config, clear_cache=True):
                     if not os.path.isdir(folder):
                         os.makedirs(folder)
                     lock = filelock.SoftFileLock(join(folder, '.conda_lock'))
-                    update_index(folder, config=config, lock=lock, could_be_mirror=False)
-                    locks.append(lock)
-                for lock in locks:
                     lock.acquire(timeout=config.timeout)
+                    if not folder.endswith('pkgs'):
+                        update_index(folder, config=config, lock=lock, could_be_mirror=False)
+                    locks.append(lock)
 
                 index = get_build_index(config=config, clear_cache=True)
 


Test patch: