Repository: conda/conda-build

Issue:
KeyError: "filename 'info/index.json' not found"
Trying to build tox using the recipe here: https://github.com/conda/conda-recipes/tree/master/tox

Getting the following err:

```
BUILD START: tox-1.5.0-py27_0
An unexpected error has occurred, please consider sending the
following traceback to the conda GitHub issue tracker at:

    https://github.com/conda/conda-build/issues

Include the output of the command 'conda info' in your report.


Traceback (most recent call last):
  File "C:\Anaconda\Scripts\conda-build-script.py", line 4, in <module>
    sys.exit(main())
  File "C:\Anaconda\lib\site-packages\conda_build\main_build.py", line 110, in m
ain
    args_func(args, p)
  File "C:\Anaconda\lib\site-packages\conda_build\main_build.py", line 312, in a
rgs_func
    args.func(args, p)
  File "C:\Anaconda\lib\site-packages\conda_build\main_build.py", line 272, in e
xecute
    build.build(m, verbose=not args.quiet, post=post)
  File "C:\Anaconda\lib\site-packages\conda_build\build.py", line 338, in build
    verbose=verbose)
  File "C:\Anaconda\lib\site-packages\conda_build\build.py", line 252, in create
_env
    update_index(config.bldpkgs_dir)
  File "C:\Anaconda\lib\site-packages\conda_build\index.py", line 65, in update_
index
    d = read_index_tar(path)
  File "C:\Anaconda\lib\site-packages\conda_build\index.py", line 20, in read_in
dex_tar
    return json.loads(t.extractfile('info/index.json').read().decode('utf-8'))
  File "C:\Anaconda\lib\tarfile.py", line 2115, in extractfile
    tarinfo = self.getmember(member)
  File "C:\Anaconda\lib\tarfile.py", line 1800, in getmember
    raise KeyError("filename %r not found" % name)
KeyError: "filename 'info/index.json' not found"

[D:\Users\108630\Programming\Python\tox]conda info
Current conda install:

             platform : win-32
        conda version : 3.9.0
  conda-build version : 1.10.2
       python version : 2.7.7.final.0
     requests version : 2.5.1
     root environment : C:\Anaconda  (writable)
  default environment : C:\Anaconda
     envs directories : C:\Anaconda\envs
        package cache : C:\Anaconda\pkgs
         channel URLs : https://conda.binstar.org/topper/win-32/
                        https://conda.binstar.org/topper/noarch/
                        http://repo.continuum.io/pkgs/free/win-32/
                        http://repo.continuum.io/pkgs/free/noarch/
                        http://repo.continuum.io/pkgs/pro/win-32/
                        http://repo.continuum.io/pkgs/pro/noarch/
          config file : D:\Users\108630\.condarc
    is foreign system : False
```



Discussion:
Something was corrupt in C:\Anaconda\conda-bld\win-32, so deleting that directory solved the problem.

We should make conda index more robust against this sort of thing. 

@kalefranz Customer trying to do a "conda index" on package ujson-1.35-py27-0.tar.bz2, got below error, although the repodata.json was created.Same thing happens with alpaca-static. 

<img width="1114" alt="screen shot 2018-01-25 at 9 46 47 am" src="https://user-images.githubusercontent.com/26414824/35397303-b8e8fbf4-01b4-11e8-8237-08e79cdfed03.png">

Similar issue here, https://github.com/conda/conda/issues/6397#issuecomment-349209596


@saraneem go ahead and open a new conda-build issue, and tag me in that.  Yes, I think it's similar to the conda issue you linked to.
@saraneem I take that back.  No need to open a new issue.  This one is fine.
I'll try to get to it today.  But if there's no progress in the next 48 hours, ping me on it again :)
I confirmed the issue on Windows 10 using conda-build 3.3 causes the error, rolling back to conda-build 3.2.2 resolves it for me.

```
C:\Users\builder\Downloads>conda index
updating: ujson-1.35-py36_0.tar.bz2
Error extracting info/paths.json in C:\Users\builder\Downloads\ujson-1.35-py36_0.tar.bz2: KeyError("filename 'info/paths.json' not found",)
Traceback (most recent call last):
  File "C:\Users\builder\Anaconda3\lib\site-packages\conda_build\index.py", line 306, in _read_index_tar
    paths_json = json.loads(t.extractfile('info/paths.json').read().decode('utf-8'))
  File "C:\Users\builder\Anaconda3\lib\tarfile.py", line 2073, in extractfile
    tarinfo = self.getmember(member)
  File "C:\Users\builder\Anaconda3\lib\tarfile.py", line 1749, in getmember
    raise KeyError("filename %r not found" % name)
KeyError: "filename 'info/paths.json' not found"
WARNING:conda_build.index:Error extracting info/paths.json in C:\Users\builder\Downloads\ujson-1.35-py36_0.tar.bz2: KeyError("filename 'info/paths.json' not found",)
Traceback (most recent call last):
  File "C:\Users\builder\Anaconda3\lib\site-packages\conda_build\index.py", line 306, in _read_index_tar
    paths_json = json.loads(t.extractfile('info/paths.json').read().decode('utf-8'))
  File "C:\Users\builder\Anaconda3\lib\tarfile.py", line 2073, in extractfile
    tarinfo = self.getmember(member)
  File "C:\Users\builder\Anaconda3\lib\tarfile.py", line 1749, in getmember
    raise KeyError("filename %r not found" % name)
KeyError: "filename 'info/paths.json' not found"

C:\Users\builder\Downloads>conda info

     active environment : None
       user config file : C:\Users\builder\.condarc
 populated config files :
          conda version : 4.4.6
    conda-build version : 0+unknown
         python version : 3.6.1.final.0
       base environment : C:\Users\builder\Anaconda3  (writable)
           channel URLs : https://repo.continuum.io/pkgs/main/win-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/win-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/win-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/win-64
                          https://repo.continuum.io/pkgs/pro/noarch
                          https://repo.continuum.io/pkgs/msys2/win-64
                          https://repo.continuum.io/pkgs/msys2/noarch
          package cache : C:\Users\builder\Anaconda3\pkgs
                          C:\Users\builder\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\builder\Anaconda3\envs
                          C:\Users\builder\AppData\Local\conda\conda\envs
                          C:\Users\builder\.conda\envs
               platform : win-64
             user-agent : conda/4.4.6 requests/2.14.2 CPython/3.6.1 Windows/10 Windows/10.0.15063
          administrator : False
             netrc file : None
           offline mode : False


C:\Users\builder\Downloads>conda list conda-build
# packages in environment at C:\Users\builder\Anaconda3:
#
conda-build               3.3.0                    py36_0

C:\Users\builder\Downloads>conda install conda-build=3.2
Solving environment: done

## Package Plan ##

  environment location: C:\Users\builder\Anaconda3

  added / updated specs:
    - conda-build=3.2


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    conda-build-3.2.2          |           py36_0         416 KB
    setuptools-38.4.0          |           py36_0         544 KB
    ------------------------------------------------------------
                                           Total:         961 KB

The following NEW packages will be INSTALLED:

    certifi:      2017.11.5-py36hb8ac631_0
    wincertstore: 0.2-py36h7fe50ca_0

The following packages will be UPDATED:

    conda:        4.4.6-py36_0             --> 4.4.7-py36_0
    setuptools:   27.2.0-py36_1            --> 38.4.0-py36_0

The following packages will be DOWNGRADED:

    conda-build:  3.3.0-py36_0             --> 3.2.2-py36_0

Proceed ([y]/n)? y


Downloading and Extracting Packages
conda-build 3.2.2: ############################################################################################ | 100%
setuptools 38.4.0: ############################################################################################ | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

C:\Users\builder\Downloads>conda index
C:\Users\builder\Downloads>type repodata.json
{
  "info": {
    "subdir": "Downloads"
  },
  "packages": {
    "ujson-1.35-py36_0.tar.bz2": {
      "build": "py36_0",
      "build_number": 0,
      "depends": [
        "python 3.6*"
      ],
      "license": "BSD",
      "md5": "8c821105f1ef8e4d834b541bec238e0e",
      "name": "ujson",
      "sha256": "c75c4c4c403d1e9945606bd423002ea21a3547b07919dd8687bd3169d3e7655f",
      "size": 24069,
      "subdir": "osx-64",
      "timestamp": 1516898381,
      "version": "1.35"
    }
  }
}

```
Ok maybe this is a red herring, I then updated conda-build back to 3.3 and conda index is still working, I am not entirely sure what changed.
nothing about this code has changed recently.  This issue can occur when packages have download issues - incomplete downloads or otherwise.  I see it in my test suite reasonably often - perhaps 1 in 10 test runs show this.

There may be better error handling that we can do - perhaps a retry.
All of the new `WARNING` stuff on this issue isn't actually a real error, even though it looks like it.  I'm pushing a PR to quiet that down.  There's no reason to have it.

Resolving patch:
diff --git a/conda_build/index.py b/conda_build/index.py
--- a/conda_build/index.py
+++ b/conda_build/index.py
@@ -298,22 +298,18 @@ def _read_index_tar(tar_path, lock, locking=True, timeout=90):
                                 "File probably corrupt." % tar_path)
             try:
                 about_json = json.loads(t.extractfile('info/about.json').read().decode('utf-8'))
-            except Exception as e:
-                log.warn('Error extracting info/about.json in %s: %r', tar_path, e, exc_info=True)
+            except KeyError:
                 about_json = {}
 
             try:
                 paths_json = json.loads(t.extractfile('info/paths.json').read().decode('utf-8'))
-            except Exception as e:
-                log.warn('Error extracting info/paths.json in %s: %r', tar_path, e, exc_info=True)
+            except KeyError:
                 paths_json = {}
 
             try:
                 recipe_text = t.extractfile('info/recipe/meta.yaml').read().decode('utf-8')
                 recipe_json = yaml.load(recipe_text)
-            except KeyError as e:
-                # log.warn('Error extracting info/recipe/meta.yaml in %s: %r',
-                #          tar_path, e, exc_info=True)
+            except KeyError:
                 recipe_json = {}
 
             # If a conda package contains an icon, also extract and cache that in an .icon/


Test patch: