Using pysimilar to compute the similarity between texts
Tags: python, nlp, computerscience, machinelearning
Published: 2021-04-07 19:48:42

Hi guys, 
I recently wrote an article titled [How to detect plagiarism in text using python](https://kalebujordan.dev/how-to-detect-plagiarism-in-text-using-python/) whereby I show how you can easily detect plagiarism between documents as the title says manually using cosine similarity.

I republished that article on multiple platforms including here on **dev.to** and **Hackernoon**, and it's one of my most viewed articles plus the most starred GitHub repository among articles repositories.

Which gave me a second thought to refactor the code/article to make it more easily and friendly to get started with even for absolute beginners leading me to build a python library **pysimilar** which I can say simplify it to the maximum;

## Getting started with Pysimilar

To get started with pysimilar for comparing text documents, you just need to install first of which you can either install directly from GitHub or using pip.

Here how to install pysimilar using pip

```bash
$ pip install pysimilar
```

Here how to install directly from github 
```bash
$ git clone https://github.com/Kalebu/pysimilar
$ cd pysimilar
$ pysimilar -> python setup.py install
```

With Pysimilar you can either compare text documents  as strings or specify the path to the file containing the textual documents.
 
### Comparing strings directly 
You can easily compare strings using pysimilar using *compare()* method just as illustrated below;

```python
>>> from pysimilar import compare
>>> compare('very light indeed', 'how fast is light')
0.17077611319011649
```

### Comparing strings contained files
To compare strings contained in the files, you just need to explicitly specify the **isfile** parameter to True just as illustrated below;

```python
>>> compare('README.md', 'LICENSE', isfile=True)
0.25545580376557886
```

You can also compare documents with particular **extension** in a given directory, for instance, let's say I want to compare all the documents with **.txt** in **documents** directory here is what I will do;

The directory for documents used by the example below looks like this

```bash
documents/
├── anomalie.zeta
├── hello.txt
├── hi.txt
└── welcome.txt
```

Here how to compare files of a particular extension

```python
>>> import pysimilar
>>> from pprint import pprint
>>> pysimilar.extensions = '.txt'
>>> comparison_result = pysimilar.compare_documents('documents')
>>> [['welcome.txt vs hi.txt', 0.6053485081062917],
    ['welcome.txt vs hello.txt', 0.0],
    ['hi.txt vs hello.txt', 0.0]]
```

You can also sort the comparison score based on their score by changing the **ascending** parameter, just as shown below;

```python
>>> comparison_result = pysimilar.compare_documents('documents', ascending=True)
>>> pprint(comparison_result)
[['welcome.txt vs hello.txt', 0.0],
 ['hi.txt vs hello.txt', 0.0],
 ['welcome.txt vs hi.txt', 0.6053485081062917]]
```

You can also set pysimilar to include files with multiple extensions

```python
>>> import pysimilar
>>> from pprint import pprint
>>> pysimilar.extensions = ['.txt', '.zeta']
>>> comparison_result = pysimilar.compare_documents('documents', ascending=True)
>>> pprint(comparison_result)
[['welcome.txt vs hello.txt', 0.0],
 ['hi.txt vs hello.txt', 0.0],
 ['anomalie.zeta vs hi.txt', 0.4968161174826459],
 ['welcome.txt vs hi.txt', 0.6292275146695526],
 ['welcome.txt vs anomalie.zeta', 0.7895651507603823]]

```


Well that's all for this article, Excited to see what you will build with it 

{% github https://github.com/Kalebu/pysimilar %}