semvec/pypi-packages huggingface dataset:
sitecrawl 1.0.5
Simple Python3 module to crawl a website and extract URLs
License: MIT

sitecrawl
=========

|Pypi| |Build Status| |codecov| |MIT licensed|

Simple Python module to crawl a website and extract URLs.

Installation
------------

Using pip:

.. code:: bash

   pip3 install sitecrawl

   sitecrawl --help

Or build from sources:

.. code:: bash

   # Clone project
   git clone https://github.com/gabfl/sitecrawl && cd sitecrawl

   # Installation
   pip3 install .

Usage
-----

CLI
~~~

.. code:: bash

   sitecrawl --url https://www.yahoo.com/ --depth 2 --max 4 --verbose

->

::

   * Found 4 internal URLs
     https://www.yahoo.com
     https://www.yahoo.com/entertainment
     https://www.yahoo.com/lifestyle
     https://www.yahoo.com/plus

   * Found 5 external URLs
     https://mail.yahoo.com/
     https://news.yahoo.com/
     https://finance.yahoo.com/
     https://sports.yahoo.com/
     https://shopping.yahoo.com/

   * Skipped 0 URLs

As a module
~~~~~~~~~~~

Basic example:

.. code:: py

   from sitecrawl import crawl

   crawl.base_url = 'https://www.yahoo.com'
   crawl.deep_crawl(depth=2)

   print('Internal URLs:', crawl.get_internal_urls())
   print('External URLs:', crawl.get_external_urls())
   print('Skipped URLs:', crawl.get_skipped_urls())

A more detailed example is available in
`example.py <https://github.com/gabfl/sitecrawl/blob/main/example.py>`__.

.. |Pypi| image:: https://img.shields.io/pypi/v/sitecrawl.svg
   :target: https://pypi.org/project/sitecrawl
.. |Build Status| image:: https://github.com/gabfl/sitecrawl/actions/workflows/ci.yml/badge.svg?branch=main
   :target: https://github.com/gabfl/sitecrawl/actions
.. |codecov| image:: https://codecov.io/gh/gabfl/sitecrawl/branch/main/graph/badge.svg
   :target: https://codecov.io/gh/gabfl/sitecrawl
.. |MIT licensed| image:: https://img.shields.io/badge/license-MIT-green.svg
   :target: https://raw.githubusercontent.com/gabfl/sitecrawl/main/LICENSE
pycounts-gp 0.1.3
Calculate word counts in a text file!
License: MIT
Requires Python: >=3.9,<4.0

# pycounts_gp

Calculate word counts in a text file!

## Installation

```bash
$ pip install pycounts_gp
```

## Usage

`pycounts_gp` can be used to count words in a text file and plot results
as follows:

```python
from pycounts_gp.pycounts_gp import count_words
from pycounts_gp.plotting import plot_words
import matplotlib.pyplot as plt

file_path = "test.txt"  # path to your file
counts = count_words(file_path)
fig = plot_words(counts, n=10)
plt.show()
```

## Contributing

Interested in contributing? Check out the contributing guidelines. 
Please note that this project is released with a Code of Conduct. 
By contributing to this project, you agree to abide by its terms.

## License

`pycounts_gp` was created by Gautham Pughazhendhi. It is licensed under the terms
of the MIT license.

## Credits

`pycounts_gp` was created with 
[`cookiecutter`](https://cookiecutter.readthedocs.io/en/latest/) and 
the `py-pkgs-cookiecutter` 
[template](https://github.com/py-pkgs/py-pkgs-cookiecutter).