Linux: 3 ways to search patterns in files
Tags: linux, beginners, bitesizelearning, tutorial
Published: 2023-09-22 20:15:57


# Introduction:
This is the first post in my bite-sized learning series. In this series we will be taking simple every day use cases and walking through multiple ways to achieve them. These posts are perfect for anyone who does not have a lot of experience working on Linux but wants to up their game, without spending hours at a stretch learning.

# Task:
Often during troubleshooting issues, we are required to search the presence of certain keywords in regular files. These can be words like: `error`, `warning`, `timeout`, `success` etc.
In today's post we will be discussing various ways to search simple patterns in regular files on Linux. 
_Note: this is the first post of the series and targeted at basic pattern search. Searching regular expressions is a more advanced topic and will be covered in a later post._

## Sample file we will be running our commands on: 
![](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5ap1zfunstm4io7bcem4.png)

## 1. grep

`grep` searches input files for matches to the patterns. When it finds a match in a line, it copies the line to standard output. `grep` searches the pattern in the file, line-by-line.
Since it reads one line at a time, `grep` may come across as a bit _slow_ when working with extremely large files. (say file size in GBs)

### 1.1 Matching the exact pattern

#### Syntax: `grep "<pattern>" <filename>`

Example: `grep "test" testfile`

![basic-grep](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q4qmeaarm0uhqok34y2x.png)

Compare the output to the actual input file `testfile`.
You will notice that `grep` only displays those lines from the file which have the pattern **test**. It has not included the lines with the words **Test** or **TEST**.
This is because, by default all grep pattern searches are case-sensitive, i.e. it matches the _word + defined case_ while comparing.
Let's try this again:

`grep "Test" testfile`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/puvim0hn3h70mf03if3h.png)

`grep "TEST" testfile`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bia5t5hwaee7zs7qjafa.png)

So far we have included the _pattern_ in double quotation marks(" ").
`grep` will still work if we do not encapsulate the pattern in quotes.
Example:

![grep_without_quote](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hn4rrz5c7u2pq619met2.png)

However, this will be an issue if the pattern contains _blank spaces_ or _special characters_
![grep_fails](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uyd4csuzhubt5tesxzni.png)


Same pattern works when placed in quotation marks:
![grep_works](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t2ylbikhlmmku6updphb.png)

> It is good practice to always place the search pattern in quotation marks.


### 1.2 Matching pattern while ignoring it case

We now understand how to match the exact pattern. How about situations where we are unsure about the case in which the pattern exists, or we want all occurrences of the word, irrespective of their case. 

This can be achieved by using the `grep -i` flag.
Here `-i` indicates that we wish to **ignore the case of the pattern**

#### Syntax: `grep -i "<pattern>" <filename>`

Example: `grep -i "test" testfile`
![grep-ignore-case](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tupcoeddovtreq0kchd1.png)



## 2. AWK
AWK is a powerful data extraction and processing language. Yes! you read it correctly! AWK is a programming language! It enables programmers to write powerful functionality while write just a few lines of code. It reads the entire file (or multiple files) at a time and then performs the requested action. This makes it super quick and the preferred choice when working with large file(s).

### 2.1 Matching the exact pattern
#### Syntax: `aws '/pattern/{ print $0 }' <filename>`

Example: `awk '/test/{ print $0 }' testfile`

![basic-pattern-match](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/18llc9l1v4jcoupqppys.png)

where:
- **{ print $0 }** : is used to print all contents of the file.
- since we are incorporating a search pattern "test" in the command, the command outputs _all lines_ from the file _testfile_, which contains the pattern _test_.
- this by default matches the _case_


### 2.2 Matching pattern while ignoring it case
#### Syntax: `awk 'BEGIN{IGNORECASE=1} /pattern/{print $0}' <filename>`

`awk 'BEGIN{IGNORECASE=1} /test/{print $0}' testfile`

![aws-ignore-case](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9ubsfvsn73seea5iwd74.png)

- **BEGIN** is a special pattern used to label actions that awk performs before reading any input records. It executes only once i.e. before all other rules. It is commonly used to initialize and setup tasks in an awk program.

## 3. SED
Short for **Stream Editor** is used for performing basic test transformations on an input. It scans the entire input (a file named `testfile` in our case) making it a very efficient way to transform data in large files.

_Note: since its a stream editor, it is not commonly used for basic pattern searching._

### 3.1 Matching the exact pattern
#### Syntax: `sed -n '/pattern/p' <filename>`

`sed -n '/test/p' testfile`

![sed](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p0t9ouk1mpgrwsvb46q6.png)

### 3.2 Matching pattern while ignoring it case
`sed -n '/[Tt][Ee][Ss][Tt]/p' testfile`

![sed-ignore-case](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ec2eocrbvbz047uyi22i.png)


Whoow! **What is that**?!? What's with these braces []? What exactly are we using as the _search pattern_?

Relax! That is the use of a **regular expression**.
I know! I know! This was supposed to be a basic tutorial and not contain scary looking expressions!
Since `sed` does not have a straightforward way to ignore case for basic pattern searches. When it comes to features like **find and replace** its a whole different story!
So if you do not understand _regular expressions_, ignore this one for now. We will discuss this is a later blog.


## Conclusion:

This was a basic tutorial on pattern searching in Linux. We learnt three different ways to search patterns in files, with or without matching cases.
I hope this was useful and did not take too much time to grasp. That would defeat the purpose of a **bite-sized learning** series!

Thank you for stopping by!
Drop a comment if you would like to see a specific Linux command covered in this bite-sized learning series! Suggestions or improvements are always welcome!