Repository: conan-io/conan

Issue:
[suggestion] Search custom sub-settings
To help us debug your issue please explain:

From slack discussion: https://cpplang.slack.com/archives/C41CWV9HA/p1560980070212900

For now `conan search` is only able to look regular settings, as `os`, `arch`, `build_type` and `compiler`, but it's limited in terms of customized settings, for example:

```
$ conan search openssl/1.0.2s@ennor/stable -q "os=iOS"
Existing packages for recipe openssl/1.0.2s@ennor/stable:
    Package_ID: 08859e5949c46fd78bb8b4d045f854329c141a56
        [settings]
            build_type: Release
            compiler: apple-clang
            compiler.version: 10.0
            os: iOS
            os.version: 10.0
        Outdated from recipe: False
    Package_ID: 5d2c60a906ecd568347527895bc32e4890264eea
        [settings]
            build_type: Debug
            compiler: apple-clang
            compiler.version: 10.0
            os: iOS
            os.version: 10.0
        Outdated from recipe: False

$ conan search openssl/1.0.2s@ennor/stable -q "os.version=10.0"
Existing packages for recipe openssl/1.0.2s@ennor/stable:

There are no packages for reference 'openssl/1.0.2s@ennor/stable' matching the query 'os.version=10.0'

conan search openssl/1.0.2s@ennor/stable -q "os=iOS AND os.version=10.0"
Existing packages for recipe openssl/1.0.2s@ennor/stable:

There are no packages for reference 'openssl/1.0.2s@ennor/stable' matching the query 'os=iOS AND os.version=10.0'
```

The command `search` is limited because the rule:

https://github.com/conan-io/conan/blob/df8fd41b63ee3054fe1160ded5e52e19c0a933c3/conans/search/search.py#L85

The suggestion is supporting custom settings, or sub-settings at least.

Regards!

/cc @Minimonium @Paultergeist

- [x] I've read the [CONTRIBUTING guide](https://github.com/conan-io/conan/blob/develop/.github/CONTRIBUTING.md).
- [ ] I've specified the Conan version, operating system version and any tool that can be relevant.
- [x] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.




Discussion:
It would be a nice addtion. It is already working for `compiler.xxxx`, so it should be easy to extend it to subsettings at least.
@jgsogo the question here is:
Would check for `info_settings.get(prop_name, None)` being `None` and only then fall back to options be a breaking change?
The query feature is very convenient and users do expect that it would work with their custom settings. 
Uff, the border between breaking and not-breaking changes sometimes is too thin... We would break things like:
 * custom settings matching the name of an option, now we would be looking for the setting instead of the option
 * and option matching a subsetting pattern (`os.version`), now we would be looking for the subsetting instead of looking for the option
 * people with invalid queries that now are finding nothing, and they will start to obtain results

Not sure if any of these use-cases is strong enough to block the feature, wdyt?
> Custom settings named the same as options
> and option matching a subsetting pattern (os.version)

That's a "funny" case indeed. Though we have a problem even now if the user makes an option named the same as the setting from the blessed list, right? Should we have a special syntax for for-sure options? `project:name`? Not sure.

> people with invalid queries that now are finding nothing, and they will start to obtain results

Are these "invalid" queries though? The query syntax doesn't differentiate options and settings, so it's very questionable what's the expected behaviour here. It may as well be a bug from the user perspective. 
And again, making a list of blessed settings in that part of the code makes an unfortunate dependency on the implementation of settings and punishes users with custom ones. 


Resolving patch:
diff --git a/conans/search/search.py b/conans/search/search.py
--- a/conans/search/search.py
+++ b/conans/search/search.py
@@ -81,9 +81,16 @@ def compatible_prop(setting_value, prop_value):
 
     info_settings = conan_vars_info.get("settings", [])
     info_options = conan_vars_info.get("options", [])
+    properties = ["os", "os_build", "compiler", "arch", "arch_build", "build_type"]
 
-    if (prop_name in ["os", "os_build", "compiler", "arch", "arch_build", "build_type"] or
-            prop_name.startswith("compiler.")):
+    def starts_with_common_settings(prop_name):
+        for setting in properties:
+            if prop_name.startswith(setting + '.'):
+                return True
+        return False
+
+    if (prop_name in properties or
+            starts_with_common_settings(prop_name)):
         return compatible_prop(info_settings.get(prop_name, None), prop_value)
     else:
         return compatible_prop(info_options.get(prop_name, None), prop_value)


Test patch:
diff --git a/conans/test/functional/command/search_test.py b/conans/test/functional/command/search_test.py
--- a/conans/test/functional/command/search_test.py
+++ b/conans/test/functional/command/search_test.py
@@ -61,6 +61,22 @@
   d41d8cd98f00b204e9800998ecf8427e
 '''  # The recipe_hash correspond to the faked conanmanifests in export
 
+conan_vars1d = '''
+[settings]
+    arch=x86
+    arch.vendor=intel
+    os=Linux
+    os.distro=Ubuntu
+    os.distro.version=19.04
+    compiler=gcc
+    compiler.version=4.3
+    compiler.libcxx=libstdc++
+    libc=glibc
+    libc.version=2.29
+[options]
+    use_Qt=True
+'''
+
 conan_vars2 = '''
 [options]
     use_OpenGL=True
@@ -131,6 +147,7 @@ def setUp(self):
         root_folder3 = 'Bye/0.14/myuser/testing'
         root_folder4 = 'NodeInfo/1.0.2/myuser/stable'
         root_folder5 = 'MissFile/1.0.2/myuser/stable'
+        root_folder6 = 'Hello/1.5.10/myuser/testing'
         root_folder11 = 'Hello/1.4.11/myuser/testing'
         root_folder12 = 'Hello/1.4.12/myuser/testing'
         root_folder_tool = 'Tool/1.0.0/myuser/testing'
@@ -151,6 +168,9 @@ def setUp(self):
                           "%s/%s/LinuxPackageSHA/%s" % (root_folder1,
                                                         PACKAGES_FOLDER,
                                                         CONANINFO): conan_vars1c,
+                          "%s/%s/LinuxPackageCustom/%s" % (root_folder6,
+                                                           PACKAGES_FOLDER,
+                                                           CONANINFO): conan_vars1d,
                           "%s/%s/a44f541cd44w57/%s" % (root_folder2,
                                                        PACKAGES_FOLDER,
                                                        CONANINFO): conan_vars2,
@@ -193,6 +213,7 @@ def create_metadata(folder, pids):
         create_metadata(root_folder3, ["e4f7vdwcv4w55d"])
         create_metadata(root_folder4, ["e4f7vdwcv4w55d"])
         create_metadata(root_folder5, ["e4f7vdwcv4w55d"])
+        create_metadata(root_folder6, ["LinuxPackageCustom"])
         create_metadata(root_folder_tool, ["winx86", "winx64", "linx86", "linx64"])
 
         # Fake some manifests to be able to calculate recipe hash
@@ -202,6 +223,7 @@ def create_metadata(folder, pids):
         fake_manifest.save(os.path.join(self.client.cache.store, root_folder3, EXPORT_FOLDER))
         fake_manifest.save(os.path.join(self.client.cache.store, root_folder4, EXPORT_FOLDER))
         fake_manifest.save(os.path.join(self.client.cache.store, root_folder5, EXPORT_FOLDER))
+        fake_manifest.save(os.path.join(self.client.cache.store, root_folder6, EXPORT_FOLDER))
         fake_manifest.save(os.path.join(self.client.cache.store, root_folder11, EXPORT_FOLDER))
         fake_manifest.save(os.path.join(self.client.cache.store, root_folder12, EXPORT_FOLDER))
         fake_manifest.save(os.path.join(self.client.cache.store, root_folder_tool, EXPORT_FOLDER))
@@ -218,6 +240,7 @@ def check():
 Hello/1.4.10@myuser/testing
 Hello/1.4.11@myuser/testing
 Hello/1.4.12@myuser/testing
+Hello/1.5.10@myuser/testing
 helloTest/1.4.10@myuser/stable""".format(remote)
                 self.assertIn(expected, self.client.out)
 
@@ -233,13 +256,15 @@ def recipe_search_test(self):
                           "Hello/1.4.10@myuser/testing\n"
                           "Hello/1.4.11@myuser/testing\n"
                           "Hello/1.4.12@myuser/testing\n"
+                          "Hello/1.5.10@myuser/testing\n"
                           "helloTest/1.4.10@myuser/stable\n", self.client.out)
 
         self.client.run("search Hello* --case-sensitive")
         self.assertEqual("Existing package recipes:\n\n"
                           "Hello/1.4.10@myuser/testing\n"
                           "Hello/1.4.11@myuser/testing\n"
-                          "Hello/1.4.12@myuser/testing\n", self.client.out)
+                          "Hello/1.4.12@myuser/testing\n"
+                          "Hello/1.5.10@myuser/testing\n", self.client.out)
 
         self.client.run("search *myuser* --case-sensitive")
         self.assertEqual("Existing package recipes:\n\n"
@@ -247,6 +272,7 @@ def recipe_search_test(self):
                           "Hello/1.4.10@myuser/testing\n"
                           "Hello/1.4.11@myuser/testing\n"
                           "Hello/1.4.12@myuser/testing\n"
+                          "Hello/1.5.10@myuser/testing\n"
                           "MissFile/1.0.2@myuser/stable\n"
                           "NodeInfo/1.0.2@myuser/stable\n"
                           "Tool/1.0.0@myuser/testing\n"
@@ -255,26 +281,30 @@ def recipe_search_test(self):
         self.client.run("search Hello/*@myuser/testing")
         self.assertIn("Hello/1.4.10@myuser/testing\n"
                       "Hello/1.4.11@myuser/testing\n"
-                      "Hello/1.4.12@myuser/testing\n", self.client.out)
+                      "Hello/1.4.12@myuser/testing\n"
+                      "Hello/1.5.10@myuser/testing\n", self.client.out)
 
     def search_partial_match_test(self):
         self.client.run("search Hello")
         self.assertEqual("Existing package recipes:\n\n"
                          "Hello/1.4.10@myuser/testing\n"
                          "Hello/1.4.11@myuser/testing\n"
-                         "Hello/1.4.12@myuser/testing\n", self.client.out)
+                         "Hello/1.4.12@myuser/testing\n"
+                         "Hello/1.5.10@myuser/testing\n", self.client.out)
 
         self.client.run("search hello")
         self.assertEqual("Existing package recipes:\n\n"
                          "Hello/1.4.10@myuser/testing\n"
                          "Hello/1.4.11@myuser/testing\n"
-                         "Hello/1.4.12@myuser/testing\n", self.client.out)
+                         "Hello/1.4.12@myuser/testing\n"
+                         "Hello/1.5.10@myuser/testing\n", self.client.out)
 
         self.client.run("search Hello --case-sensitive")
         self.assertEqual("Existing package recipes:\n\n"
                          "Hello/1.4.10@myuser/testing\n"
                          "Hello/1.4.11@myuser/testing\n"
-                         "Hello/1.4.12@myuser/testing\n", self.client.out)
+                         "Hello/1.4.12@myuser/testing\n"
+                         "Hello/1.5.10@myuser/testing\n", self.client.out)
 
         self.client.run("search Hel")
         self.assertEqual("There are no packages matching the 'Hel' pattern\n", self.client.out)
@@ -283,7 +313,8 @@ def search_partial_match_test(self):
         self.assertEqual("Existing package recipes:\n\n"
                          "Hello/1.4.10@myuser/testing\n"
                          "Hello/1.4.11@myuser/testing\n"
-                         "Hello/1.4.12@myuser/testing\n", self.client.out)
+                         "Hello/1.4.12@myuser/testing\n"
+                         "Hello/1.5.10@myuser/testing\n", self.client.out)
 
         self.client.run("search Hello/1.4.10")
         self.assertEqual("Existing package recipes:\n\n"
@@ -317,6 +348,7 @@ def search_raw_test(self):
         self.assertEqual("Hello/1.4.10@myuser/testing\n"
                          "Hello/1.4.11@myuser/testing\n"
                          "Hello/1.4.12@myuser/testing\n"
+                         "Hello/1.5.10@myuser/testing\n"
                          "helloTest/1.4.10@myuser/stable\n", self.client.out)
 
     def search_html_table_test(self):
@@ -404,16 +436,17 @@ def package_search_nonescaped_characters_test(self):
         self.assertEqual(str(self.client.out).count("PlatformIndependantSHA"), 2)
         self.assertNotIn("WindowsPackageSHA", self.client.out)
 
-    def _assert_pkg_q(self, query, packages_found, remote):
+    def _assert_pkg_q(self, query, packages_found, remote, ref="Hello/1.4.10@myuser/testing"):
 
-        command = 'search Hello/1.4.10@myuser/testing -q \'%s\'' % query
+        command = 'search %s -q \'%s\'' % (ref, query)
         if remote:
             command += " -r %s" % remote
         self.client.run(command)
 
-        for pack_name in ["LinuxPackageSHA", "PlatformIndependantSHA", "WindowsPackageSHA"]:
+        for pack_name in ["LinuxPackageSHA", "PlatformIndependantSHA", "WindowsPackageSHA",
+                          "LinuxPackageCustom"]:
             self.assertEqual(pack_name in self.client.out,
-                              pack_name in packages_found, "%s fail" % pack_name)
+                             pack_name in packages_found, "%s fail" % pack_name)
 
     def _assert_pkg_query_tool(self, query, packages_found, remote):
         command = 'search Tool/1.0.0@myuser/testing -q \'%s\'' % query
@@ -483,6 +516,11 @@ def test_cases(remote=None):
             q = '(os="Linux" OR os=Windows) AND use_Qt=True OR nonexistant_option=3'
             self._assert_pkg_q(q, ["WindowsPackageSHA", "LinuxPackageSHA"], remote)
 
+            q = 'compiler="gcc" AND compiler.libcxx="libstdc++" AND ' \
+                'arch="x86" AND arch.vendor="intel" AND os.distro="Ubuntu" AND ' \
+                'os.distro.version="19.04" AND compiler.version="4.3" AND os="Linux"'
+            self._assert_pkg_q(q, ["LinuxPackageCustom"], remote, ref="Hello/1.5.10@myuser/testing")
+
             q = 'os_build="Windows"'
             self._assert_pkg_query_tool(q, ["winx86", "winx64"], remote)
 
@@ -606,6 +644,10 @@ def package_search_properties_filter_test(self):
         self.assertIn("There are no packages for reference 'Hello/1.4.10@myuser/testing' "
                       "matching the query 'compiler.version=1.0'", self.client.out)
 
+        self.client.run("search Hello/1.5.10@myuser/testing -q 'glib=glibc AND glib.version=2.29'")
+        self.assertIn("There are no packages for reference 'Hello/1.5.10@myuser/testing' matching "
+                      "the query 'glib=glibc AND glib.version=2.29'", self.client.out)
+
         self.client.run('search Hello/1.4.10@myuser/testing '
                         '-q "compiler=gcc AND compiler.version=4.5"')
         self.assertNotIn("WindowsPackageSHA", self.client.out)
@@ -710,6 +752,10 @@ def search_json_test(self):
                             'recipe': {
                                 'id': 'Hello/1.4.12@myuser/testing'}
                         },
+                        {
+                            'recipe': {
+                                'id': 'Hello/1.5.10@myuser/testing'}
+                        },
                         {
                             'recipe': {
                                 'id': 'helloTest/1.4.10@myuser/stable'}
@@ -748,6 +794,10 @@ def search_json_test(self):
                              'recipe': {
                                  'id': 'Hello/1.4.12@myuser/testing'}
                         },
+                        {
+                            'recipe': {
+                                'id': 'Hello/1.5.10@myuser/testing'}
+                        },
                         {
                              'recipe': {
                                  'id': 'helloTest/1.4.10@myuser/stable'}
@@ -769,6 +819,10 @@ def search_json_test(self):
                             'recipe': {
                                 'id': 'Hello/1.4.12@myuser/testing'}
                         },
+                        {
+                            'recipe': {
+                                'id': 'Hello/1.5.10@myuser/testing'}
+                        },
                         {
                             'recipe': {
                                 'id': 'helloTest/1.4.10@myuser/stable'}
@@ -802,6 +856,10 @@ def search_json_test(self):
                             'recipe': {
                                 'id': 'Hello/1.4.12@myuser/testing'}
                         },
+                        {
+                            'recipe': {
+                                'id': 'Hello/1.5.10@myuser/testing'}
+                        },
                         {
                             'recipe': {
                                 'id': 'helloTest/1.4.10@myuser/stable'}
@@ -1064,6 +1122,8 @@ def search_json_test(self):
         }
         self.assertEqual(expected_output, output)
 
+
+
     def search_packages_with_reference_not_exported_test(self):
         self.client.run("search my_pkg/1.0@conan/stable", assert_error=True)
         self.assertIn("ERROR: Recipe not found: 'my_pkg/1.0@conan/stable'", self.client.out)