📂 Merge common words of two files in Python, practice file handling
Tags: tutorial, python, files, algorithms
Published: 2021-03-21 23:15:25


# Hi there 😁

In this tutorial you will learn to select words from files, compare them and merge the common words in a new file.

## You will practice 🔁

* File handling in Python
* Sets in Python
* Using function to re use code
* Some basics command line instruction

## Let's start 🐍

**Code:** Remember that all the code shown during these tutorials will be pushed to github.

{% github Developer-road/merge-common-words %}

First of all create a Python file, I'll use a long and descriptive file name `merge.py` 😆

Let's create that file with the terminal:

**Note:** if you are using windows, none of these commands will run, so just do all the things with a graphical interface, like a file manager of your code editor. If you are using Mac 🍎, or Linux 🐧 stay relaxed, and learn the terminal as best as you can.

Remember that Python is mainly a CLI programming language, so you will be using a lot the terminal, so my advice is to use the terminal whenever you can 😀

```shell
touch merge.py
```

The touch command will create an empty file `merge.py`, in the directory you are in.

Now I'm going to call a graphical editor, from the terminal 😱, in this case I'll use vscode.

```shell
code ./
```

Remember that in UNIX (don't be scared, it refers to Unix based OS, like Linux and Mac), `./` the dot refers to the current directory.

Now that I have Vscode set up and going let's start with the code.

## Open a file in Python

First of all , we will use some text files in this tutorial, you can download them easily on github, by cloning the repository or just following the links below.

[text_one.txt](https://github.com/Developer-road/merge-common-words/blob/main/text_one.txt)
[text_two.txt](https://github.com/Developer-road/merge-common-words/blob/main/text_two.txt)

In Python you open a file with the function, `open` which takes as parameter the path of the file.

Make sure you've downloaded the text files, and type in the `merge.py` file the following code.

```python
file1 = open("text_one.txt)
```

This will create a variable `file1`, which is a file object, with the mode of "read" (by default).

If you try to print that variable you will get something like this:

```python

print(file1)

# Result
# <_io.TextIOWrapper name='text_one.txt' mode='r' encoding='UTF-8'>
``` 

That's because that variable is just a file object of the text file named, "text_one.txt"

But be carefully if the file doesn't exist, you will get an error 😱.

```python
file1 = open("text_oness.txt")

# FileNotFoundError: [Errno 2] No such file or directory: 'text_oness.txt'
```

So each time we're working with files, we should use some defensive programming, to avoid the error in case the file doesn't exist

```python
try:
    file1 = open("text_oness.txt")
except FileNotFoundError:
    print("Sorry that file doesn't exist")
    exit()
``` 

We're using the `exit()` function because if the file doesn't exist we won't have the variable to work with, so in that case we will terminate the execution.

If you want to get the content of a file in Python, you would use the `{file_object}.read()` function, that as it name says read for us the content of that file.

```python
print(file1.read())

# Results
# Why should you learn to write programs?

# Writing programs (or programming) is a very creative 
# and rewarding activity. MORE TEXT .....
```

Now that you know how to open files, and read them you should be able to create that comparison functionality.

First make sure we are able to open the two files.

```python
try:
    file1 = open("text_one.txt")
    file2 = open("text_two.txt")
except FileNotFoundError:
    print("Sorry that file doesn't exist")
    exit()
```

Now we will use the content of the files and the power of sets to get all the words of each file.

## Sets in Python

A set in Python, is an unordered data structure, with a really special characteristic (Among others), it doesn't allow repeated elements 🤫.

We will use them, to be able to get the words, without repeating.

### Creating a set in Python

To create a set in Python we will use the `set()` function.

```python
empty_set = set()
```

#### A set VS a list:

```python
mylist = list("122333")
myset = set("122333")

print(mylist)
print(myset)

# List:
# ['1', '2', '2', '3', '3', '3']

# Set
# {'3', '2', '1'}
```

As you see the list printed all the elements in order, but the set only printed the elements that were not repeated, and unordered.

## Getting the words of a file

We are going to iterate through the words of the file with a for loop, and add them in a set for each file.

```python
file1_words = set()

file2_words = set()


for word in file1.read().split():
    file1_words.add(word.lower())

for word in file2.read().split():
    file2_words.add(word.lower())

print(file1_words)
print(file2_words)
```

Maybe the above code is a little bit confusing, but let's take a quick look on it.

First we initialize, two sets, one for the first file, the another for the second file.

Then we iterate with a for loop, each word of the file by calling `file1.read().split()`. 

In that part we use the method `read()`, that gives us the content of the file as a string, and then we use the string method `split()`, which gives us a list of the words in the file by splitting the string within each space.

So basically we are iterating over:

```python
for word in list_of_words_of_the_file:
    code...
```

Then we get the word, we make it lowercase to avoid word repeating, and add it to the `file1_words` set. Remember that in a set there can't be repeated elements, if a word is already in the set, then it won't be added.

We do the same thing for both files.

Running that piece of code, returns two sets with all the words of both files. 

### Challengue
**But there is a problem** and I challenge you 🔥 to solve it. 
There are some words that have special punctuation, for example `(on` and `on`, and your task is to figure out how to replace all the punctuation characters, so there  won't be repeated words, but with special punctuation.

Reach me out on [Twitter](https://twitter.com/RoadDeveloper), or [Instagram](https://www.instagram.com/developerroad/), if you achieve it.

## Using sets to intersect common words

We are looking for common words in the sets we just created, and for that we will use **Intersection**.

Yeah that word may look scary since probably you've seen it in math, but don't worry. Intersection are just the common parts of two sets. 

![Intersection](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/17ftvwbzd0qtc3c16wro.png)
 
Intersecting elements could be tedious 🙄, since with most data structures you would have to iterate through the variables that contain the elements, compare them select those that are repeated and append them in a new variable.

But with sets in Python, we have a special function that does all of that for us, `{set1}.intersection({set2})`

```python
common_words = file1_words.intersection(file2_words)

print(common_words)
```
There it is, now we can access the common words of the two files, in an one liner.

## Writing files in Python

Writing files in Python, is not that hard. We open a file in write mode by using `open("file/path", mode = "w")`.

This allow us to write to an existing file or in the case the file didn't exist, it creates a new one with the determinate file path.

So let's open our merge file.

```python
merge_file = open("merge.txt", mode = "w")

# code_here ....

merge_file.close()
```
Remember that every time  we open a file in write mode, we must close it, after making the desired operations.

Now we're going to write all the common words to the new `merge.txt` file.

```python
merge_file = open("merge.txt", mode = "w")

for word in common_words:
   word = word + ", "
   merge_file.write(word)

merge_file.close()
```
We need to append a comma string at the end of each word, so we can differentiate the words in the file.

If you run this code, you will get the desired result.

```python
try:
    file1 = open("text_one.txt")
    file2 = open("text_two.txt")
except FileNotFoundError:
    print("Sorry that file doesn't exist")
    exit()


file1_words = set()

file2_words = set()


for word in file1.read().split():

    file1_words.add(word.lower())

for word in file2.read().split():

    file2_words.add(word.lower())


common_words = file1_words.intersection(file2_words)

merge_file = open("merge.txt", mode="w")

for word in common_words:
    word = word + ", "
    merge_file.write(word)

merge_file.close()

```

If you check the new `"merge.txt"` file, you will see all the common words between the file `text_one.txt` and `text_two.txt`

Congratulations 🎉, you just created a merge algorithm, and in base of a merge algorithm (Much more complex for sure), is how git works, to compare code files.

But wait a minute, this code is clunky and there are many parts where we repeated the same process.

So let's use the power of functions, to make our code reusable and more escalable code.

First let's make a function to open a file, and handling exceptions. That function will take 2 arguments, the first will be the file path, and the second an optional argument with the open mode of the file.

```python
# Python function that handle exception while opening files

def open_file(file_path, open_mode="r"):

    try:
        file_handler = open(file_path, mode=open_mode)

    except FileNotFoundError:
        print(f"Sorry the file {file_path} doesn't exist")
        exit()

    except ValueError:
        print(f"Sorry the file {file_path} can't be opened with mode {open_mode}")
        exit()

    return file_handler            
```

Then a function that let us catch the words of a file.

```python
def get_file_words(file_path):

    file_words = set()

    read_file = open_file(file_path)

    for word in read_file.read().split():
        
        file_words.add(word.lower())
    return file_words

```
Here as you may notice, we take as parameter the path of the file we're going to get the words from, and get the file handler through the `open_file()` function, we just created.

Lastly let's create a function `merge`, which will make the operations of getting and intersecting the common words, and writing those words in a file.

```python
def merge(*filenames, merge_file="merge.txt"):

    list_of_file_words = []

    for filename in filenames:

        file_words = get_file_words(filename)
        list_of_file_words.append(file_words)

    common_words = set.intersection(*list_of_file_words)

    merge_write_file = open_file(merge_file, "w")

    for word in common_words:

        word = word + ", "

        merge_write_file.write(word)
    
    merge_write_file.close()
        
```

Here we use the power of `*args` in python functions, that allow us to pass multiple filenames to the function if we want to merge more than two.

You can notice that I used a for loop to iterate over the `*filename` argument. That's because we can receive any number of filenames now, and thus our merge algorithm is more powerful now.

## Compacting our code in a main() function

As a best practice in python, you can use a `main()` function, that will call and perform any operation your script does.

```python
def main():
    
    file1 = "text_one.txt"

    file2 = "text_two.txt"

    merge(file1, file2, merge_file = "main_merge.txt")
```

This main function calls the merge function and pass as parameters the file1 and file2 variables. Also we specified the `merge_file` parameter which tells the merge function which file it has to write in.


But as you may noticed we haven't call any function yet, so let's call the main function.

```python
if __name__ == "__main__":
    main()
```

The `__name__` variable deserves another blog post, but basically here you're telling to python:

- Hey dude 🥸, if this is the python file you're executing from terminal, run the `main()` function.

So the final code of this cool algorithm is this:

```python
def open_file(file_path, open_mode="r"):

    try:
        file_handler = open(file_path, mode=open_mode)

    except FileNotFoundError:
        print(f"Sorry the file {file_path} doesn't exist")
        exit()

    except ValueError:
        print(f"Sorry the file {file_path} can't be opened with mode {open_mode}")
        exit()

    return file_handler            


def get_file_words(file_path):

    file_words = set()

    read_file = open_file(file_path)

    for word in read_file.read().split():
        
        file_words.add(word.lower())

    return file_words

def merge(*filenames, merge_file="merge.txt"):

    list_of_file_words = []

    for filename in filenames:

        file_words = get_file_words(filename)
        list_of_file_words.append(file_words)

    common_words = set.intersection(*list_of_file_words)

    merge_write_file = open_file(merge_file, "w")

    for word in common_words:

        word = word + ", "

        merge_write_file.write(word)
    
    merge_write_file.close()



def main():
    
    file1 = "text_one.txt"

    file2 = "text_two.txt"

    merge(file1, file2, merge_file="merge_main.txt")


if __name__ == "__main__":
    main()
```

## Conclusions:

In this tutorial you practiced:

* File handling in Python
* Sets and lists in Python
* Iteration with for loops
* Function definition and usage
* Python main function best practices

If you found any error in this tutorial don't hesitate in contact me, or make a pull request in the [Github repo](https://github.com/Developer-road/merge-common-words)

>Follow me in [My blog](https://developerroad.herokuapp.com/blog/), 
 to get more awesome tutorials like this one.
 Please consider supporting me on [Ko-fi](https://ko-fi.com/daniel1404) you help me a lot to 
 continue building this tutorials!.
[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/Y8Y03P8P1)