astropy/astropy — pull request #2805 by mdboom (2014-07-31 17:03:27+00:00)
Title: Astropy 0.4 rewriting astropy.cfg all the time, leading to race conditions

Hi,

I have a default astropy.cfg file and currently astropy 0.4 rewrites it at every import, which leads to race conditions in multithreading scenarios. For example the following program
```python
import time,multiprocessing as mp
def func(x):
import astropy.table
time.sleep(0.01)

p=mp.Pool(16,maxtasksperchild=1)
p.map(func,range(10000000),chunksize=1)
```
fails with this message

```
Traceback (most recent call last):
File "xtest.py", line 10, in
p.map(func,range(10000000),chunksize=1)
File "/home/koposov/my_usr/pylib/lib/python2.7/multiprocessing/pool.py", line 250, in map
return self.map_async(func, iterable, chunksize).get()
File "/home/koposov/my_usr/pylib/lib/python2.7/multiprocessing/pool.py", line 554, in get
raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: 'ParseError('Invalid line at line "1".',)'. Reason: 'PicklingError("Can't pickle : import of module astropy.extern.configobj_py2.configobj failed",)'
```

To experience this bug, you must have either the default astropy 0.4 cfg file (or remove it first before running the program).

The cause of the problem is in astropy/config/configuration.py is_unedites_config_file() function. This function sees the cfg file written previously by astropy 0.4 and thinks that it "does not have any effective content" (see line 765) and decides to rewrite it. The problem is that this happens every time you import astropy....
And in multithreaded cases sometimes an incomplete file is being read. which leads to the exception seen above.

I think the easy fix is to compare the md5 with md5 of the default astropy0.4 config file fdf8a709588d9083d3ec0cbb3a0ac462 to avoid this problem, but I don't know whether that's satisfactory. I could provide a patch if needed.

Sergey

PS It is also easy to see that the rewriting happens all time by doing strace:

strace -e open python -c 'import astropy' 2>&1 |grep astropy.cfg
open("/home/koposov/.astropy/config/astropy.cfg", O_RDONLY) = 4
open("/home/koposov/.astropy/config/astropy.cfg", O_RDONLY) = 4
open("/home/koposov/my_usr/pylib/lib/python2.7/site-packages/astropy/astropy.cfg", O_RDONLY) = 4
open("/home/koposov/.astropy/config/astropy.cfg", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 4