Repository: conda/conda-build

Issue:
When testing a package, the folder containing the package needs be an indexed channel
When testing packages using: `conda-build /data/builds/linux-64/eigen3-3.2.9-he32ef9e_5.tar.bz2 --test` and the artifact is just in a folder without repodata.json, the above command would produce: 

```
conda.exceptions.ResolvePackageNotFound: 
  - eigen3 3.2.9 he32ef9e_5
```


after running `conda index` on that folder the above test is run successfully...

Going back to our discussion of #2260, where we concluded that testing recipes is unintuitive: I'd say that its also unintuitive to have to put a package-under-test (that we even specify by full path) into a valid conda channel... 

It should be: *CB, please test that package there!*, not *CB, please test that package in that channel there*.



Discussion:
You're right.  That's an unfortunate implementation detail.  In order to have conda respect dependencies, we have to fudge it into conda thinking that it's installing a package from a channel, not a path.  Explicit installs from paths do not install dependencies.  It seems like I should be able to generate an index on the fly if one isn't already present, and clean it up afterwards if necessary.
I understand that, maybe generating the index (like its done when testing recipes) would work...
@msarahan I'm sorry mike, this bug still exists... I looked at the code and i see and experience two problems... my setup for testing is an unindexed channel given by relative path:

```
builds/linux-64/mypackage.tar.bz2
```

then I call: `conda-build --test builds/linux-64/mypackage.tar.bz2`. Then conda indexes the folder `builds` and fails with error message "can't install mypackage".

I looked at the code and found two problems. The first problem can I see is that the wrong path gets indexed because of:

https://github.com/conda/conda-build/blob/70b6c3c4b1ad88e34cdafc17f7cf716044369493/conda_build/build.py#L1351-L1354

I don't know why the platform folders get stripped? It's right to detect that its a "channel/repository", but   in this case I'd index all subfolders of `builds/*` by globbing... and for sure not the folder `builds` itself. As a result we have `builds/repodata.json` but not `builds/linux-64/repodata.json`.


The second problem I see is that relative paths are not converted to absolute paths as expected, imho its an obvious error in these lines:

https://github.com/conda/conda-build/blob/70b6c3c4b1ad88e34cdafc17f7cf716044369493/conda_build/build.py#L1356-L1358

I think it should read `local_location = os.path.normpath(os.path.abspath(local_location))`, which is essentially the same as wanted.


What about the first problem? Do u want me to PR my suggestion using globbing?
Sure, go ahead and PR.  Much appreciated.  I think I forgot the fact that conda needs specific "subdir" named folders.  You have the correct folder name there - linux-64, but if you point the test at a package not in a known subdir folder, it probably will not work, and that's my fault.
>  I think I forgot the fact that conda needs specific "subdir" named folders. You have the correct folder name there - linux-64, but if you point the test at a package not in a known subdir folder, it probably will not work, and that's my fault.

sorry, can u explain that?
if you have your tarballs in a folder, say somefolder, conda will fail to see that as a channel.  Conda wants to instead see somefolder/linux-64/yourpackage.tar.bz2
ok.. then i had the right thought.. does that in turn mean that testing a package _only_ works in a channel environment? then I'd check that case and throw an error
yeah, or you could make conda-build create the right structure (make the subdir folder and move/copy the package into it).
ok thanks for this information.. I'll implement a fix..

before going into creating our own workaround I'd probably teach conda how to install artifacts directly, seems more like a workaround imho
conda can install artifacts directly, but when doing that, it does not install dependencies.  That's the reason for this dance.  You could instead inspect the dependencies, install those, and then install the artifact directly.

Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -1347,24 +1347,39 @@ def _construct_metadata_for_test_from_package(package, config):
             config.filename_hashing = False
             hash_input = {}
 
-    local_location = os.path.dirname(package)
-    # strip off extra subdir folders
-    for platform in ('win', 'linux', 'osx'):
-        if os.path.basename(local_location).startswith(platform + "-"):
-            local_location = os.path.dirname(local_location)
+    log = utils.get_logger(__name__)
 
-    if not os.path.abspath(local_location):
-        local_location = os.path.normpath(os.path.abspath(
-            os.path.join(os.getcwd(), local_location)))
+    # get absolute file location
+    local_pkg_location = os.path.normpath(os.path.abspath(os.path.dirname(package)))
+
+    # get last part of the path
+    last_element = os.path.basename(local_pkg_location)
+    is_channel = False
+    for platform in ('win-', 'linux-', 'osx-', 'noarch'):
+        if last_element.startswith(platform):
+            is_channel = True
+
+    if not is_channel:
+        raise ValueError("Your package must reside in a channel structure with "
+                         "platform-subfolders.  See more info at what a valid channel is "
+                         "at https://conda.io/docs/user-guide/tasks/create-custom-channels.html")
+
+    # get channel url
+    local_channel = os.path.dirname(local_pkg_location)
+
+    # update indices in the channel
+    for pattern in ('win-*', 'linux-*', 'osx-*', 'noarch'):
+        for folder in glob(os.path.join(local_channel, pattern)):
+            log.info("Updating index at %s to make package installable with dependencies" % folder)
+            update_index(folder)
+
+    # make an url out of the channel and add to channel list
+    local_channel_url = url_path(local_channel)
 
-    log = utils.get_logger(__name__)
-    log.info("Updating index at %s to make package installable with dependencies" % local_location)
-    update_index(local_location)
-    local_url = url_path(local_location)
     # channel_urls is an iterable, but we don't know if it's a tuple or list.  Don't know
     #    how to add elements.
     config.channel_urls = list(config.channel_urls)
-    config.channel_urls.insert(0, local_url)
+    config.channel_urls.insert(0, local_channel_url)
 
     try:
         metadata = render_recipe(os.path.join(info_dir, 'recipe'), config=config,


Test patch: