How To Grep With PowerShell
Tags: powershell, bash, productivity
Published: 2020-09-30 13:27:39

Filtering data, from log or config files to data returned by an api,
is an important operation to remove noise from it and make further analysis possible.

In Bash most would gravitate towards `grep`, `egrep` or `awk`.

Classic examples would be

- looking at your history

  ```bash
  $ history | grep ssh
  ssh marco@0.0.0.0
  ssh deploy@0.0.0.0
  ssh admin@0.0.0.0
  ```

- looking for errors in a log

  ```bash
  $ cat /var/log/app.log | grep error
  2020-09-30 09:01:17 error: twitter api responded with 403
  2020-09-30 09:16:05 error: secret plan to take over the world failed
  ```

## grep

```bash
$ grep api app.conf
api_key_twitter: '***'
api_key_digitalocean: '***'
```

This will list all settings containing `api` in the configuration of my fictitious app.[^appconf]

With PowerShell we will use `Select-String` and define our search term in the `-Pattern` parameter.

```powershell
PS> Select-String -Path "app.conf" -Pattern "api"
app.conf:2:api_key_twitter: '***'
app.conf:4:api_key_digitalocean: '***'
```

{% details Important! %}
PowerShell includes the file and line of the match by default, if you want to disable this add `-Raw` to your command.
{% enddetails %}

## grep -v

```bash
$ grep -v api app.conf
# Super awesome app config!
super_secret_setting: 'the cake is a lie!'
```

This will list all settings not containing `api` in the configuration of my fictitious app.[^appconf]

PowerShell can replicate this with the `-NotMatch` parameter

```powershell
PS> Select-String -Path "app.conf" -Pattern "api" -NotMatch
app.conf:1:# Super awesome app config!
app.conf:3:super_secret_setting: 'the cake is a lie!'
```

## RegEx Patterns

PowerShell natively supports regular expressions, so writing more complicated filters is no problem.

```powershell
PS> Select-String -Path "app.conf" -Pattern "^\w*:"
app.conf:2:api_key_twitter: '***'
app.conf:3:super_secret_setting: 'the cake is a lie!'
app.conf:4:api_key_digitalocean: '***'
```

## Multiple Files

```bash
$ grep super_secret_setting *.py
debug.py:print(conf.super_secret_setting)
server.py:# TODO: Maybe we should not print super_secret_setting
server.py:#print(conf.super_secret_setting)
```

This will list all python code using my `super_secret_setting`.

PowerShell can replicate this with a similar syntax.

```powershell
PS> Select-String -Path "*.py" -Pattern "super_secret_setting"
debug.py:7:print(conf.super_secret_setting)
server.py:14:# TODO: Maybe we should not print super_secret_setting
server.py:15:#print(conf.super_secret_setting)
```

Searching through files recursively is really simple in Bash too.

```bash
$ grep -r super_secret_setting *.py
debug.py:print(conf.super_secret_setting)
server.py:# TODO: Maybe we should not print super_secret_setting
server.py:#print(conf.super_secret_setting)
utils.secrets.py:if conf.super_secret_setting:
```

Unfortunately this is not as simple with PowerShell as you need to chain multiple commands together.

```powershell
Get-ChildItem -Recurse -Include "*.py" | Select-String -Pattern "super_secret_setting"
debug.py:7:print(conf.super_secret_setting)
server.py:14:# TODO: Maybe we should not print super_secret_setting
server.py:15:#print(conf.super_secret_setting)
utils/secrets.py:4:if conf.super_secret_setting:
```

## Tipps & Tricks

This is a new section I wanted to introduce to give light to some minor features not worth a full example,
but still interesting and useful from time to time.

- Sometimes you do not require all matches for something but rather a quick scan of files containing at least one match.
  This can be achieved with the<br> `-List` switch
- Other times you may require getting all matches - even multiple ones per line.
  `-AllMatches` will provide you with just that
- Search case sensitive by using `-CaseSensitive`

## Shortcuts

Writing out `Select-String` every time is nothing to be desired, thankfully we can take some shortcuts and even bring in our Bash muscle memory.

- Shorten `Select-String` by using it's alias `sls`
- Just like grep
  - `-Pattern` can be provided as positional argument 0
  - `-Path` can be provided as positional argument 1
- Omit `""` around your strings as long as they have no spaces inside
- Commands are always case insensitive, this is more of a personal preference

With this shortcuts our commands are way shorter.

```powershell
Select-String -Path "app.conf" -Pattern "api"
sls api app.conf

Select-String -Path "app.conf" -Pattern "api" -NotMatch
sls api app.conf -NotMatch

Select-String -Path "app.conf" -Pattern "^\w*:"
sls "^\w*:" app.conf

Select-String -Path "*.py" -Pattern "super_secret_setting"
sls super_secret_setting *.py

Get-ChildItem -Recurse -Include "*.py" | Select-String -Pattern "super_secret_setting"
dir -Recurse -Include *.py | sls super_secret_setting

```

## Taking It Further

This covers basic examples for replacing grep in PowerShell.

Thanks to PowerShells object oriented piping system the information we see on screen is not everything we get to work with.

Lets take a look with one on the previous examples
and plug it into the fictional pipeline of creating a report for all usages of `super_secret_setting`.

```powershell
Select-String -Path "*.py" -Pattern "super_secret_setting" | Select-Object Path,LineNumber | Export-Csv -Path api_conf.csv -NoTypeInformation
```

Obviously this is just a example demonstrating the many possibilities in having a rich object describing the match.
You could write them to a variable for later use, loop over them or any other thing a real programming language allows you to do.

For quick reference this are all values accessible inside every MatchInfo object returned.

```powershell
IgnoreCase : True
LineNumber : 2
Line       : api_key_twitter: '***'
Filename   : app.conf
Path       : /Users/mkamner/projects/mkamner/blog/app.conf
Pattern    : ^\w*:
Context    :
Matches    : {0}
```

The `Context` value can be interesting to use, as it provides us with the possibility to get X lines before and after our matching line in the resulting object.

```powershell
# 3 lines before and after
-Context 3

# 1 line before, 3 after
-Context 1,3
```

A great ressource to dive in even deeper is the [official documentation](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/select-string?view=powershell-7) from microsoft.

Feel free to ask me about your PowerShell problem over on Twitter!

[^appconf]: [app.conf](https://ps1.guru/2020/09/powershell-grep/#fn:1)