Repository: conda/conda-build

Issue:
--no-locking doesn't work + log is spammed
This is since updating conda. I was on conda 4.3.25 and conda-build 3.0.19 and this didn't happen. Updated to conda 4.3.27 and conda-build 3.0.23 and now more than half my log when building a package is just spam about acquiring/releasing locks over and over again, e.g. here's a small sample (not even all of them from a single build):

INFO:filelock:Lock 162828128 released on C:\conda\locks\RDpcaGJrYmluXGNvbmRh
INFO:filelock:Lock 162824656 released on C:\conda\locks\QzpcY29uZGEtb3BlcmF0
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 acquired on C:\conda\locks\QzpcVXNlcnNcanNleHRv
INFO:filelock:Lock 164179920 released on C:\conda\locks\QzpcVXNlcnNcanNleHRv

Also, it appears that the --no-locking flag is broken, as these messages are in the log just the same even with specify --no-locking.


Discussion:
Thanks for reporting. --no-locking is known to not affect absolutely everything. There are still some file operations that happen outside of a build configuration, and these are the ones causing problems.  The log output is a recurring problem due to how entangled conda and conda build are. I will try to figure out how conda has reintroduced this issue. No related code in conda build has changed recently.
Thanks for the info @msarahan. I've done some more digging and discovered that if I use the 4.3.21 miniconda installer (Windows 64-bit) and update conda and conda-build to the latest version, everything works fine! No filelock spam. However, if I use the 4.3.27 miniconda installer (Windows 64-bit) also with the latest version of conda-build, the spam is there. So it's some other dependency causing this. Later today I'll start with the 4.3.21 installer and update packages one-by-one to match 4.3.27 until I get the spam. Then I'll let you know the offending package.
Wow, that'll be very helpful information.  Much appreciated!
@msarahan and the winner is, filelock: https://github.com/benediktschmitt/py-filelock/commit/260be4d7df7fea377296a93d7407f037d2f50da0

That change actually seems fine though. The problems from my perspective are a few things on the conda side:

1) --no-locking doesn't actually do no locking
2) Conda is repeatedly acquiring and releasing the same lock over and over again?
3) All the logging messages are prefixed with "INFO:filelock". Not sure if this is expected or not, but thought I'd point it out.
OK, thanks. I'll dig in and take a look.

I'm not too concerned about 2 - there's a balance there between holding a lock long enough to get stuff done, vs. holding it too long and blocking other stuff.  Ultimately the locks are for two things:

1. preventing conda from running environment stuff in more than one process.
2. preventing builds from interfering with each other (e.g. preventing "purge" commands from nuking running builds)

Regarding your point 3: this is expected.  It's just python's logger output.  In the absence of any customization, it shows the message level and the module emitting the message.
Fair enough regarding #2, just a shame that there's so many lines in the log now. Makes it harder to find relevant information. Building a very small test package that I have, I get:

104 lines with "INFO:filelock:Lock ######## released on PATH_TO_CONDA\locks\abcxyz" by default
86 lines with "INFO:filelock:Lock ######## released on PATH_TO_CONDA\locks\abcxyz" with "--no-locking" option

So I guess --no-locking does do something ;)

I will also note that the logging messages in filelock are at the INFO level, and are being displayed even though I have conda config's verbosity set to 0 (i.e. WARN). So them getting through at verbosity 0 seems like a bug.

Resolving patch:
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -1067,6 +1067,9 @@ def filter(self, record):
 info_debug_stdout_filter = LessThanFilter(logging.WARNING)
 warning_error_stderr_filter = GreaterThanFilter(logging.INFO)
 
+# set filelock's logger to only show warnings by default
+logging.getLogger('filelock').setLevel(logging.WARN)
+
 
 def reset_deduplicator():
     """Most of the time, we want the deduplication.  There are some cases (tests especially)


Test patch: