Repository: conda/conda-build

Issue:
conda skeleton cpan's detection of Perl core modules reports false-positives
### What happened?

`conda skeleton cpan` by default prevents the creation of recipes for [Perl core modules](https://perldoc.perl.org/modules).  Unfortunately, the detection for core modules reports many false-positives, i.e. modules are classified as core modules even though they are not. As a results, recipes for these modules cannot be created. 

As a very prominent example, consider `Module::Build`, a popular module for creating Perl distributions. Running `conda skeleton cpan Module::Build` yields the following error message:

```
WARNING: Module::Build is a Perl core module that is not developed outside of Perl, so we are skipping creating a recipe for it.
```
However, according to the [documentation](https://metacpan.org/pod/Module::Build#COMPARISON), `Module::Build` "was added to core in Perl 5.10 and removed from core in Perl 5.20". Thus, it needs to be provided as a conda package for recent Perl versions. `conda skeleton` should *not* refuse to create a recipe for it.


### Conda Details

</details>

<details>
<summary><code>conda info</code></summary>

```shell
active environment : conda-build
    active env location : /home/felix/miniconda3/envs/conda-build
            shell level : 2
       user config file : /home/felix/.condarc
 populated config files : /home/felix/.condarc
          conda version : 22.9.0
    conda-build version : not installed
         python version : 3.7.12.final.0
       virtual packages : __linux=5.15.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/felix/miniconda3  (writable)
      conda av data dir : /home/felix/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/felix/miniconda3/pkgs
                          /home/felix/.conda/pkgs
       envs directories : /home/felix/miniconda3/envs
                          /home/felix/.conda/envs
               platform : linux-64
             user-agent : conda/22.9.0 requests/2.28.1 CPython/3.7.12 Linux/5.15.0-48-generic neon/20.04 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

# Not sure why this says I have not installed conda-build...
# $ conda-build --version
# conda-build 3.22.0
# $type -p conda-build
# /home/felix/miniconda3/envs/conda-build/bin/conda-build
```

</details>

<details>
<summary><code>conda config</code></summary>

```shell
==> /home/felix/.condarc <==
auto_activate_base: False
channels:
  - conda-forge
  - bioconda
  - defaults
```

</details>

<details>
<summary><code>conda list</code></summary>

```shell
# packages in environment at /home/felix/miniconda3/envs/conda-build:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
beautifulsoup4            4.11.1             pyha770c72_0    conda-forge
brotlipy                  0.7.0           py39hb9d737c_1004    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2022.9.24            ha878542_0    conda-forge
certifi                   2022.9.24          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1           py39he91dace_0    conda-forge
chardet                   5.0.0            py39hf3d152e_0    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
click                     8.1.3            py39hf3d152e_0    conda-forge
colorama                  0.4.5              pyhd8ed1ab_0    conda-forge
conda                     22.9.0           py39hf3d152e_1    conda-forge
conda-build               3.22.0           py39hf3d152e_2    conda-forge
conda-package-handling    1.9.0            py39hb9d737c_0    conda-forge
conda-verify              3.1.1           py39hf3d152e_1004    conda-forge
cryptography              37.0.1           py39h9ce1e76_0    defaults
filelock                  3.8.0              pyhd8ed1ab_0    conda-forge
future                    0.18.2           py39hf3d152e_5    conda-forge
glob2                     0.7                        py_0    conda-forge
greenlet                  1.1.3            py39h5a03fae_0    conda-forge
icu                       70.1                 h27087fc_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
libarchive                3.5.2                hada088e_3    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
liblief                   0.12.2               h27087fc_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libsqlite                 3.39.3               h753d276_0    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libxml2                   2.10.2               h4c7fe37_1    conda-forge
libzlib                   1.2.12               h166bdaf_3    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
markupsafe                2.1.1            py39hb9d737c_1    conda-forge
msgpack-python            1.0.4            py39hf939315_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
openssl                   3.0.5                h166bdaf_2    conda-forge
patch                     2.7.6             h7f98852_1002    conda-forge
patchelf                  0.15.0               h58526e2_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
pkginfo                   1.8.3              pyhd8ed1ab_0    conda-forge
psutil                    5.9.2            py39hb9d737c_0    conda-forge
py-lief                   0.12.2           py39h5a03fae_0    conda-forge
pycosat                   0.6.3           py39hb9d737c_1010    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pynvim                    0.4.3            py39hf3d152e_0    conda-forge
pyopenssl                 22.0.0             pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.9.13          h2660328_0_cpython    conda-forge
python-libarchive-c       4.0              py39hf3d152e_1    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytz                      2022.2.1           pyhd8ed1ab_0    conda-forge
pyyaml                    6.0              py39hb9d737c_4    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
requests                  2.28.1             pyhd8ed1ab_1    conda-forge
ripgrep                   13.0.0               h2f28480_2    conda-forge
ruamel_yaml               0.15.80         py39hb9d737c_1007    conda-forge
setuptools                65.4.0             pyhd8ed1ab_0    conda-forge
shyaml                    0.6.2              pyhd3deb0d_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
sqlite                    3.39.3               h4ff8645_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tqdm                      4.64.1             pyhd8ed1ab_0    conda-forge
tzdata                    2022d                h191b570_0    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
zlib                      1.2.12               h166bdaf_3    conda-forge
zstd                      1.5.2                h6239696_4    conda-forge
```

</details>

### Additional Context

To prevent the creation of recipes for core modules, `conda skeleton` needs to detect if a given Perl module is a core module. To this end, `conda skeleton` gathers a list of all core modules. Unfortunately, this list contains many false-positives, i.e. modules that are not core modules (anymore).

The list of core modules is determined by function `install_perl_get_core_modules(version)` in `conda_build/skeletons/cpan.py` using the following code:
```python
args = [f'{join(tmpdir, *subdirs[1:])}', '-e',
                    'use Module::CoreList; print join "\n", Module::CoreList->find_modules(qr/.*/);']
```

The issue lies in the fact that `find_modules()` from [`Module::CoreList`](https://metacpan.org/pod/Module::CoreList) returns *all* modules that have been core modules at any time. It thus also reports modules which have been removed from core. `Module::CoreList` also provides the command line utility `corelist`, which confirms the removal of a given module:
```
$ corelist Module::Build

Module::Build was first released with perl v5.9.4, deprecated (will be CPAN-only) in v5.19.0 and removed from v5.21.0
```

To only return the modules that are in the core of the *current* Perl version, I suggest the following change in the Perl code:
```perl
use Module::CoreList; print join "\n", grep {Module::CoreList::is_core($_)} Module::CoreList->find_modules(qr/.*/);'
```
The `is_core` function returns a true value only if the given module is currently in the core. For Perl 5.32.1, this reduces the number of reported core modules from 911 to 643 (i.e., 1/3 of the reported modules were false-positives).


Discussion:

Resolving patch:
diff --git a/conda_build/skeletons/cpan.py b/conda_build/skeletons/cpan.py
--- a/conda_build/skeletons/cpan.py
+++ b/conda_build/skeletons/cpan.py
@@ -325,7 +325,9 @@ def install_perl_get_core_modules(version):
         with TemporaryDirectory() as tmpdir:
             environ.create_env(tmpdir, [f'perl={version}'], env='host', config=config, subdir=subdirs[0])
             args = [f'{join(tmpdir, *subdirs[1:])}', '-e',
-                    'use Module::CoreList; print join "\n", Module::CoreList->find_modules(qr/.*/);']
+                    'use Module::CoreList; '
+                    'my @modules = grep {Module::CoreList::is_core($_)} Module::CoreList->find_modules(qr/.*/); '
+                    'print join "\n", @modules;']
             from subprocess import check_output
             all_core_modules = check_output(args, shell=False).decode('utf-8').replace('\r\n', '\n').split('\n')
             return all_core_modules


Test patch:
diff --git a/tests/test_cpan_skeleton.py b/tests/test_cpan_skeleton.py
new file mode 100644
--- /dev/null
+++ b/tests/test_cpan_skeleton.py
@@ -0,0 +1,28 @@
+# Copyright (C) 2014 Anaconda, Inc
+# SPDX-License-Identifier: BSD-3-Clause
+"""
+Unit tests of the CPAN skeleton utility functions
+"""
+
+
+from pathlib import Path
+
+import pytest
+
+from conda_build.variants import get_default_variant
+from conda_build.skeletons.cpan import get_core_modules_for_this_perl_version
+
+
+@pytest.mark.slow
+def test_core_modules(testing_config):
+    """
+    Check expected core modules are recognized
+    (excluding known removed ones, e.g., Module::Build)
+    """
+    cache_dir = Path(testing_config.src_cache_root, ".conda-build", "pickled.cb")
+    perl_version = testing_config.variant.get(
+        "perl", get_default_variant(testing_config)["perl"]
+    )
+    core_modules = get_core_modules_for_this_perl_version(perl_version, str(cache_dir))
+    assert "Config" in core_modules
+    assert "Module::Build" not in core_modules