Better Perl: Using map and grep
Tags: programming, grep, lists, map
Published: 2021-03-16 14:11:44

---
title: Better Perl: Using map and grep
published: true
date: 2021-03-16 14:00:00 UTC
tags: Programming,grep,lists,map
canonical_url: https://phoenixtrap.com/2021/03/16/perl_map_grep/?utm_source=rss&utm_medium=rss&utm_campaign=perl_map_grep
series: List processing in Perl
---

As a Perl developer, you’re probably aware of the language’s strengths as a text-processing language and how many computing tasks can be broken down into those types of tasks. You might not realize, though, that Perl is also a world-class **list processing** language and that **many problems can be expressed in terms of lists** and their transformations.

Chief among Perl’s tools for list processing are the functions `map` and `grep`. I can’t count how many times in my twenty-five years as a developer I’ve run into code that could’ve been simplified if only the author was familiar with these two functions. Once you understand `map` and `grep`, **you’ll start seeing lists everywhere** and the opportunity to make your code more succinct and expressive at the same time.

## What are lists?

Before we get into functions that manipulate lists, we need to understand what they are. **A list is an ordered group of elements**, and those elements can be any kind of data you can represent in the language: numbers, strings, objects, regular expressions, references, etc., as long as they’re stored as scalars. You might think of a list as the thing that an array stores, and in fact Perl is fine with using an array where a list can go.

```perl
my @foo = (1, 2, 3);
```

Here we’re assigning the list of numbers from `1` to `3` to the array `@foo`. The difference between the array and the list is that the list is a fixed collection, while arrays and their elements can be modified by various operations. [`perlfaq4` has a great discussion](https://perldoc.pl/perlfaq4#What-is-the-difference-between-a-list-and-an-array%3F) on the differences between the two.

## Lists are everywhere, man!

Ever wanted to `sort` some data? You were using a list.

`join` a bunch of things together into a string? List again.

`split` a string into pieces? You got a list back (in list [context](https://perldoc.pl/perldata#Context); in scalar context, you got the size of the list.)

Heck, even the humble `print` function and its cousin `say` take a list (and an optional filehandle) as arguments; it’s why you can treat Perl as an upscale [AWK](https://en.wikipedia.org/wiki/AWK) and feed it scalars to output with a [field separator](https://perldoc.pl/variables/%24,).

**You’re using lists all the time and may not even know it.**

## `map`: The list transformer

The `map` function is devious in its simplicity: It takes two inputs, an expression or block of code, and a list to run it on. For every item in the list, it will alias `$_` to it, and then return none, one, or many items in a list based on what happens in the expression or code block. You can call it like this:

```perl
my @foo = map bar($_), @list;
```

Or like this:

```perl
my @foo = map { bar($_) } @list;
```

We’re going to ignore the first way, though because [Conway (_Perl Best Practices_, 2005)](https://www.oreilly.com/library/view/perl-best-practices/0596001738/) tells us that when you specify the first argument as an expression, it’s harder to tell it apart from the remaining arguments, especially if that expression uses a built-in function where the parentheses are optional. **So always use a code block!**

You should always turn to `map` (and not, say, a [`for` or `foreach`](https://perldoc.pl/perlsyn#Foreach-Loops) loop) **when generating a new list from an old list**. For example:

```perl
my @lowercased = map { lc } @mixed_case;
```

When paired with a lookup table, `map` is also the most efficient way to tell if a member of a list equals a string, especially if that list is static:

```perl
use Const::Fast;

const my %IS_EXIT_WORD => map { ($_ => 1) }
  qw(q quit bye exit stop done last finish aurevoir);

...

die if $IS_EXIT_WORD{$command};
```

Here we’re using `map`’s ability to return multiple items per source element to generate a constant hash, and then testing membership in that hash.

## `grep`: The list filter

You may recognize the word “grep” from the [Unix command of the same name](https://en.wikipedia.org/wiki/Grep). It’s a tool for finding lines of text inside of other text using a [regular expression](https://en.wikipedia.org/wiki/Regular_expression) describing the desired result.

Perl, of course, is [really good at regular expressions](https://perldoc.pl/perlre), but its `grep` function goes beyond and enables you to match using any expression or code block. Think of it as a partner to `map`; **where `map` uses a code block to transform a list, `grep` uses one to filter it down**. In fact, other languages typically call this function `filter`.

You can, of course, use regular expressions with `grep`, especially because a regexp match in Perl defaults to matching on the `$_` variable and `grep` happens to provide that to its code block argument. So:

```perl
my @months_with_a = grep { /[Aa]/ } qw(
  January February March
  April May June
  July August September
  October November December
);
```

But **`grep` really comes into its own when used for its general filtering capabilities**; for instance, making sure that you don’t accidentally try to compare an undefined value:

```perl
say $_ > 5
  ? "$_ is bigger"
  : "$_ is equal or smaller"
  for grep { defined } @numbers;
```

Or when executing a complicated function that returns _true_ or _false_ depending on its arguments:

```perl
my @results = grep { really_large_database_query($_) }
              @foo;
```

You might even consider **chaining `map` and `grep` together**. Here’s an example for getting the JPEG images out of a file list and then lowercasing the results:

```perl
my @jpeg_files = map { lc }
                 grep { /\.jpe?g$/i } @files;
```

## “Side effects may include…” *(updated)*

When introducing `map` above I noted that it **aliased** `$_` for every element in the list. I used that term deliberately because **modifications to `$_` will modify the original element itself**, and that is usually an error. Programmers call that a “side effect,” and they can lead to unexpected behavior or at least difficult-to-maintain code. Consider:

```perl
my @needs_docs = grep { s/\.pm$/.pod/ && !-e }
                 @pm_files;
```

The intent may have been to find files ending in `.pm` that don’t have a corresponding `.pod` file, but the actual behavior is replacing the `.pm` suffix with `.pod`, then checking whether that filename exists. If it doesn’t, it’s passed through to `@needs_docs`; regardless, `@pm_files` has had its contents modified.

If you really do need to modify a copy of each element, assign a variable within your code block like this:

```perl
my @needs_docs = grep {
                   my $file = $_;
                   $file =~ s/\.pm$/.pod/;
                   !-e $file
                 } @pm_files;
```

But at that point you should probably **refactor your multi-line block** as a separate function:

```perl
my @needs_docs = grep { file_without_docs($_) }
                 @pm_files;

sub file_without_docs {
    my $file = shift;
    $file =~ s/\.pm$/.pod/;
    return !-e $file;
}
```

In this case of using the substitution operator [`s///`](https://perldoc.pl/perlop#s/PATTERN/REPLACEMENT/msixpodualngcer), you could also do this when using Perl 5.14 or above to get [non-destructive substitution](https://perldoc.pl/perl5140delta#Non-destructive-substitution):

```perl
use v5.14;

my @needs_docs = grep { !-e s/\.pm$/.pod/r }
                 @pm_files;
```

And **if you do need side effects, just use a [`for` or `foreach`](https://perldoc.pl/perlsyn#Foreach-Loops) loop**; future code maintainers (i.e., you in six months) will thank you.

## Taking you higher

`map` and `grep` are examples of [higher-order functions](https://en.wikipedia.org/wiki/Higher-order_function), since they take a function (in the form of a code block) as an argument. So **congratulations, you just significantly leveled up your knowledge of Perl and computer science**. If you’re interested in more such programming techniques, I recommend [Mark Jason Dominus’ _Higher Order Perl_ (2005)](https://hop.perl.plover.com/), available for free online.