Using cpanm to Install Perl Modules in a Conda Environment
Tags: conda, perl, linux, troubleshooting
Published: 2020-07-01 01:30:11

---
title: Using cpanm to Install Perl Modules in a Conda Environment
published: true
description: This post explains how you can use cpanm (App::cpanminus) to install Perl modules in a conda environment when they are not available in a conda channel. It also addresses issues that can arise from certain configurations, or when you need to install modules using XS.
tags: conda,perl,linux,troubleshooting
//cover_image: https://direct_url_to_image.jpg
---

As a devoted Perl programmer, I find myself making heavy use of the fantastic Perl module repository, [CPAN](https://metacpan.org/). And I do my best to keep up with all the latest goings-on with Perl.

I also have recently gotten hooked on [conda](https://docs.conda.io/en/latest/), a neat tool which combines package management and environment management. If you haven't checked that out yet, I suggest you give it a look to see if it suits your needs.

## TL;DR

(Full post after this tl;dr)

```bash
# envname is the name of your (existing) conda env
conda activate envname
# cpanm is available on either conda-forge or
# bioconda, so add conda-forge (or look up
# how to set up bioconda if you need tools
# for bioinformatics)
conda config --add channels conda-forge
# Install cpanm
conda install perl-app-cpanminus
# packagename is the name of the Perl package you want
env PERL5LIB="" PERL_LOCAL_LIB_ROOT="" PERL_MM_OPT="" PERL_MB_OPT="" $CONDA_PREFIX/bin/cpanm packagename
```

If that didn't work:

```bash
# Make sure gcc is installed
# Replace gcc with gxx if you need to compile C++
conda install -n envname gcc_impl_linux-64
conda activate envname
declare perlarchloc=$($CONDA_PREFIX/bin/perl -V::installarchlib: | sed "s/[' ]//g")
for conffile in "Config_heavy.pl" "CORE/config.h" "Config.pm"
do
    sed -i -E 's%/tmp/build/[a-zA-Z0-9]+/perl_[0-9]+/_build_env%'"$CONDA_PREFIX"'%g' "$perlarchloc"/"$conffile"
done
unset perlarchloc
```

If that *still* didn't work, jump to [the last section](#compilerroot).

/tl;dr

## Installing Perl and Python Packages

Conda is written in Python, and I think because of that there is a slight bias towards things in Python working well. Case in point, installing a python module which hasn't already been packaged in a conda channel is relatively simple:

```bash
# envname is the name of your (existing) conda env
conda activate envname
# Make sure pip is installed
conda install pip
# packagename is the name of the python package you want
pip install packagename
```

Cool. Now, how about a Perl package?

```bash
# envname is the name of your (existing) conda env
conda activate envname
# cpanm is available on either conda-forge or
# bioconda, so add conda-forge (or look up
# how to set up bioconda if you need tools
# for bioinformatics)
conda config --add channels conda-forge
# Install cpanm
conda install perl-app-cpanminus
# packagename is the name of the Perl package you want
cpanm packagename
```

If this worked for you, great! If it didn't, well let's see what could have gone wrong.

## Problem 1: local::lib or other environment variables

A lot of us like to use [local::lib](https://metacpan.org/pod/local::lib) for installing Perl packages locally, without needing to mess with system-level Perl. This works great in general. Not so much when you're letting conda take care of things.

There are a few reasons you want to use conda, and if you're reading this, you probably already know why (go ahead and look up more about conda if you really don't know how useful it can be). And as such, it's preferable for you to install your Perl modules for an environment only in that environment.

Maybe your environment uses a different version of perl than your system (or another environment). Maybe you share the conda environment with many users, and you want any user of it to have access to some specific module(s).

Unfortunately, having local::lib set up might mean that the `cpanm` command from earlier installed the module only for you, outside of the conda env:

```bash
$ perldoc -l packagename
$HOME/perl5/lib/perl5/packagename/packagename.pm
```

Ugh.

To avoid this, and any other environment variables that affect Perl modules from interfering, you might try this:

```bash
env PERL5LIB="" PERL_LOCAL_LIB_ROOT="" PERL_MM_OPT="" PERL_MB_OPT="" $CONDA_PREFIX/bin/cpanm packagename
```

The above worked for me, and its a bit ugly but I needed a solution in the moment. If you know of a better way, let me know in the comments and I'll update the post.

## Problem 2: Modules using XS

It turns out that installing a module that uses [XS](https://perldoc.perl.org/perlxs.html) (as many modules that need the speed of C, or access to Perl internals do) in this way does not work. It's [a known bug in conda, too](https://github.com/conda/conda/issues/8425).

What happens is that the the configuration for cpanm retains some paths relative to the directory in which it was built, rather than to the conda environment (as you'll see in [the last comment](https://github.com/conda/conda/issues/8425#issuecomment-509452017) in that bug report).

That comment also explains how to fix the issue, but I will reproduce that solution here, a bit more generalized (this assumes the version of Perl you installed is 5.26.2, so make sure you check that first):

```bash
# Make sure gcc is installed
# Replace gcc with gxx if you need to compile C++
conda install -n envname gcc_impl_linux-64
conda activate envname
declare perlarchloc=$($CONDA_PREFIX/bin/perl -V::installarchlib: | sed "s/[' ]//g")
for conffile in "Config_heavy.pl" "CORE/config.h" "Config.pm"
do
    sed -i -E 's%/tmp/build/[a-zA-Z0-9]+/perl_[0-9]+/_build_env%'"$CONDA_PREFIX"'%g' "$perlarchloc"/"$conffile"
done
unset perlarchloc
```

Now just rerun your `cpanm` command(s) and that should work! Unless...

### Problem 2a: compilerroot?! <a name="compilerroot"></a>

Turns out that on some configurations, a variable, `$compilerroot`, is used to set the location of the compiler in the files above. It's not easy to automate (for me), so I suggest you manually open the "Config_heavy.pl" and "Config.pm" files under `"$CONDA_PREFIX"/lib/5.26.2/x86_64-linux-thread-multi/` and look for the following line:

```perl
my $compilerroot = "";
# Some other code is here
```

Change that assignment, and remove the if/else code following that line that conditionally sets `$compilerroot` too. Make sure you use the correct path for your environment's conda prefix (ie, the value of `$CONDA_PREFIX` when your environment is active):

```perl
my $compilerroot = "/your/conda/env/prefix";

# Don't change this code below
local *_ = \my $a;
```

*Now* try using cpanm :).

I hope this helped *someone*. It took me a few hours before I figured this out. In retrospect, it might have been easier to just package the missing perl modules for conda myself!

_Edit: Feb 20, 2021. Updated the code block showing how to change the paths in the conda perl headers. Now instead of the perl version being hardcoded, use the location of the arch lib location given by the conda perl installation. Also updated post so that call to conda `cpanm` use the `$CONDA_PREFIX` environment variable._