YorkUScraper 0.1.1
YorkU Website Scraper in Selenium
License: MIT
Keywords: yorku,york university,python3,api,unofficial

YorkUScraper
==================
A Selenium scraper for various York University web-pages allowing for easy data exporting.

[![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white&style=flat-square)](https://www.linkedin.com/in/isaac-kogan-5a45b9193/ )
![HitCount](https://hits.dwyl.com/isaackogan/YorkUScraper.svg?style=flat&cachebust=4)
![Issues](https://img.shields.io/github/issues/isaackogan/YorkUScraper)
[![Support Server](https://img.shields.io/discord/977648006063091742.svg?color=7289da&logo=discord&style=flat-square)](https://discord.gg/e2XwPNTBBr)

A Python library using Selenium, Socket.IO to scrape the York University website using a Chrome browser asynchronously. By sending commands over a websocket to and from the browser, selenium becomes usable in an
asynchronous fashion.

This library includes a wrapper that connects to the YorkU website and indexes it. No credentials are required to use YorkUScraper as it only accesses publicly available endpoints on the YorkU website.

This is what my [YorkU API](https://github.com/isaackogan/YorkUAPI) project uses to pull course data. This project was created for that purpose.

Join my general [community discord](https://discord.gg/e2XwPNTBBr) for questions, contributions and ideas. Feel free to make pull requests with missing/new features, fixes, etc.

**NOTE:** This is **not** an official product. It's a reverse engineering and research project.

## Table of Contents

- [Documentation](https://isaackogan.github.io/YorkUScraper/)
- [Contributors](#contributors)
- [License](#license)
- [Examples](https://github.com/isaackogan/YorkUScraper/tree/master/examples)
- [Getting Started](#getting-started)

## Getting Started

1. Install the module via pip

```
pip install YorkUScraper
```

2. Create your first chat connection

```python
from YorkUScraper import YorkUScraperClient
from YorkUScraper.types.events import LaunchEvent, CourseSiteIndexEvent
from YorkUScraper.client.config import YORKU_COURSES_URL

# Instantiate the client with the user's username
client: YorkUScraperClient = YorkUScraperClient()


# Define how you want to handle specific events via decorator
@client.on("launch")
async def on_launch(_: LaunchEvent):
    print("Successfully launched the scraper")


# Notice no decorator?
async def on_course_site_index(event: CourseSiteIndexEvent):
    print(f"The YorkU course site {YORKU_COURSES_URL} was indexed:", event.data)


# Define handling an event via "callback"
client.add_listener("launch", on_course_site_index)

if __name__ == '__main__':
    # Run the client and block the main thread
    # await client.start() to run non-blocking
    client.run()
```

For more examples, [see the examples folder](https://github.com/isaackogan/YorkUScraper/tree/master/examples) provided in the tree.

### Handling Errors

Errors can be handled by listening to the `error` event.

If this handler is not present in the code, an internal default handler will log errors in the console. If a handler is added, all error handling (including logging) is up to the individual.

**Warning:** If you listen for the error event and do not log errors, you will not see when an error occurs. This is because listening to the error event causes the default one to be overriden/turned off.

```python

@client.on("error")
async def on_connect(error: Exception):
    # Handle the error
    if isinstance(error, SomeRandomError):
        print("Handle Some Error")
        return

```

## Contributors

* **Isaac Kogan** - *Initial work & primary maintainer* - [isaackogan](https://github.com/isaackogan)

See also the full list of [contributors](https://github.com/isaackogan/YorkUScraper/contributors) who have participated in this project.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.