Better Perl: Four list processing best practices with map, grep, and more
Tags: perl, grep, lists, map
Published: 2021-10-26 16:05:04

---
title: Better Perl: Four list processing best practices with map, grep, and more
published: true
date: 2021-10-26 14:00:00 UTC
tags: Perl,grep,lists,map
canonical_url: https://phoenixtrap.com/2021/10/26/better-perl-four-list-processing-best-practices-with-map-grep-and-more/
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4hln3r5o2edh4t5tiwzx.jpeg
series: List processing in Perl
---

Six months ago I gave [an overview of Perl’s list processing fundamentals](https://phoenixtrap.com/2021/03/16/perl_map_grep/), briefly describing what lists are and then introducing the built-in [`map`](https://perldoc.perl.org/functions/map) and [`grep`](https://perldoc.perl.org/functions/grep) functions for transforming and filtering them. Later on, I compiled a list (how appropriate) of [list processing modules available via CPAN](https://phoenixtrap.com/2021/05/18/a-list-of-perl-list-processing-modules/), noting there’s some confusing duplication of effort. But you’re a busy developer, and you just want to know the Right Thing To Do™️ when faced with a list processing challenge.

First, some credit is due: these are all restatements of several [Perl::Critic policies](https://metacpan.org/dist/Perl-Critic/view/lib/Perl/Critic/PolicySummary.pod) which in turn codify standards described in [Damian Conway’s _Perl Best Practices_](https://www.oreilly.com/library/view/perl-best-practices/0596001738/) (2005). I’ve [repeatedly recommended](https://phoenixtrap.com/tag/pbp/) the latter as a starting point for higher-quality Perl development. Over the years these practices continue to be re-evaluated (including [by the author himself](https://www.oreilly.com/videos/modern-perl-best/9781491910719/)) and various authors release new [policy modules](https://metacpan.org/search?q=module%3APerl%3A%3ACritic%3A%3APolicy), but [`perlcritic`](https://metacpan.org/dist/Perl-Critic/view/bin/perlcritic) remains a great tool for ensuring you (and your team or other contributors) maintain a consistent high standard in your code.

With that said, on to the recommendations!

## Don’t use `grep` to check if _any_ list elements match

It might sound weird to lead off by recommending _not_ to use `grep`, but sometimes it’s not the right tool for the job. If you’ve got a list and want to determine if a condition matches any item in it, you might try:

```perl
if (grep { some_condition($_) } @my_list) {
    ... # don't do this!
}
```

Yes, this works because (in scalar [context](https://perldoc.perl.org/perldata#Context)) `grep` returns the number of matches found, but it’s wasteful, checking every element of `@my_list` (which could be lengthy) before finally providing a result. Use the standard [List::Util](https://perldoc.perl.org/List::Util) module’s [`any`](https://perldoc.perl.org/List::Util#any) function, which immediately returns (“short-circuits”) on the first match:

```perl
use List::Util 1.33 qw(any);

if (any { some_condition($_) } @my_list) {
    ... # do something
}
```

Perl has included the requisite version of this module [since version 5.20](https://perldoc.perl.org/perl5200delta#Updated-Modules-and-Pragmata) in 2014; for earlier releases, you’ll need to [update from CPAN](https://metacpan.org/dist/Scalar-List-Utils). List::Util has many other great [list-reduction](https://perldoc.perl.org/List::Util#LIST-REDUCTION-FUNCTIONS), [key/value pair](https://perldoc.perl.org/List::Util#KEY/VALUE-PAIR-LIST-FUNCTIONS), and [other related functions](https://perldoc.perl.org/List::Util#OTHER-FUNCTIONS) you can import into your code, so check it out before you attempt to re-invent any wheels.

As a side note for web developers, the [Perl Dancer framework](https://www.perldancer.org/) also includes an [`any` keyword](https://metacpan.org/dist/Dancer2/view/lib/Dancer2/Manual/Keywords.pod#any) for declaring multiple HTTP routes, so if you’re mixing List::Util in there don’t import it. Instead, call it explicitly like this or you’ll get an error about a redefined function:

```perl
use List::Util 1.33;

if (List::Util::any { some_condition($_) } @my_list) {
    ... # do something
}
```

This recommendation is codified in the [BuiltinFunctions::ProhibitBooleanGrep](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::ProhibitBooleanGrep) Perl::Critic policy, comes directly from _Perl Best Practices_, and is [recommended](https://wiki.sei.cmu.edu/confluence/display/perl/EXP06-PL.+Do+not+use+an+array+in+an+implicit+scalar+context) by the Software Engineering Institute Computer Emergency Response Team (SEI CERT)’s [Perl Coding Standard](https://wiki.sei.cmu.edu/confluence/display/perl/SEI+CERT+Perl+Coding+Standard).

## Don’t change `$_` in `map` or `grep`

I [mentioned this back in March](https://phoenixtrap.com/2021/03/16/perl_map_grep/#side_effects), but it bears repeating: `map` and `grep` are intended as [pure functions](https://en.wikipedia.org/wiki/Pure_function), not [mutators](https://en.wikipedia.org/wiki/Mutator_method) with side effects. This means that the original list should remain unchanged. Yes, each element aliases in turn to the [`$_` special variable](https://perldoc.perl.org/variables/%24_), but that’s for speed and can have surprising results if changed even if it’s technically allowed. If you need to modify an array in-place use something like:

```perl
for (@my_array) {
    $_ = ...; # make your changes here
}
```

If you want something that _looks_ like `map` but won’t change the original list (and don’t mind a few CPAN dependencies), consider [List::SomeUtils](https://metacpan.org/pod/List::SomeUtils)’ [`apply`](https://metacpan.org/pod/List::MoreUtils#apply-BLOCK-LIST) function:

```perl
use List::SomeUtils qw(apply);

my @doubled_array = apply {$_ *= 2} @old_array;
```

Lastly, side effects also include things like manipulating other variables or doing input and output. Don’t use `map` or `grep` in a void context (i.e., without a resulting array or list); do something with the results or use a [`for` or `foreach`](https://perldoc.perl.org/perlsyn#Foreach-Loops) loop:

```perl
map { print foo($_) } @my_array; # don't do this
print map { foo($_) } @my_array; # do this instead

map { push @new_array, foo($_) } @my_array; # don't do this
@new_array = map { foo($_) } @my_array; # do this instead
```

This recommendation is codified by the [BuiltinFunctions::ProhibitVoidGrep](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::ProhibitVoidGrep), [BuiltinFunctions::ProhibitVoidMap](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::ProhibitVoidMap), and [ControlStructures::ProhibitMutatingListFunctions](https://metacpan.org/pod/Perl::Critic::Policy::ControlStructures::ProhibitMutatingListFunctions) Perl::Critic policies. The latter comes from _Perl Best Practices_ and is an SEI CERT Perl Coding Standard [rule](https://wiki.sei.cmu.edu/confluence/display/perl/EXP34-PL.+Do+not+modify+%24_+in+list+or+sorting+functions).

## Use blocks with `map` and `grep`, not expressions

You can call `map` or `grep` like this ([parentheses are optional](https://perldoc.perl.org/perlfunc#DESCRIPTION) around built-in functions):

```perl
my @new_array = map foo($_), @old_array; # don't do this
my @new_array2 = grep !/^#/, @old_array; # don't do this
```

Or like this:

```perl
my @new_array = map { foo($_) } @old_array;
my @new_array2 = grep {!/^#/} @old_array;
```

Do it the second way. It’s easier to read, especially if you’re passing in a literal list or multiple arrays, and the expression forms can conceal bugs. This recommendation is codified by the [BuiltinFunctions::RequireBlockGrep](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::RequireBlockGrep) and [BuiltinFunctions::RequireBlockMap](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::RequireBlockMap) Perl::Critic policies and comes from _Perl Best Practices_.

## Refactor multi-statement `map`s, `grep`s, and other list functions

`map`, `grep`, and friends should follow the [Unix philosophy of “Do One Thing and Do It Well.”](https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_and_Do_It_Well) Your readability and maintainability drop with every statement you place inside one of their blocks. Consider junior developers and future maintainers (this includes you!) and refactor anything with more than one statement into a separate subroutine or at least a `for` loop. This goes for list processing functions (like the aforementioned `any`) imported from other modules, too.

This recommendation is codified by the _Perl Best Practices_-inspired [BuiltinFunctions::ProhibitComplexMappings](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::ProhibitComplexMappings) and [BuiltinFunctions::RequireSimpleSortBlock](https://metacpan.org/pod/Perl::Critic::Policy::BuiltinFunctions::RequireSimpleSortBlock) Perl::Critic policies, although those only cover `map` and `sort` functions, respectively.

* * *

Do you have any other suggestions for list processing best practices? Feel free to leave them in the comments or better yet, consider [creating new Perl::Critic policies](https://metacpan.org/pod/Perl::Critic#EXTENDING-THE-CRITIC) for them or [contacting the Perl::Critic team](mailto:team@perlcritic.com) to develop them for your organization.