semvec/pypi-packages huggingface dataset:
RAGScraper 11.5.2023
RAGScraper is a Python library designed for efficient and intelligent scraping of web documentation and content. Tailored for Retrieval-Augmented Generation systems, RAGScraper extracts and preprocesses text into structured, machine-learning-ready formats. It emphasizes precision, context preservation, and ease of integration with RAG models, making it an ideal tool for developers looking to enhance AI-driven applications with rich, web-sourced knowledge.
License: MIT
Requires Python: >=3.10,<4.0

# RAGScraper

RAGScraper is a simple Python package that scrapes webpages and converts them to markdown format for RAG usage.

## Installation

To install RAGScraper, simply run:

```bash
pip install ragscraper
```

## Usage

To use RAGScraper as a command-line tool:

```bash
rag-scraper <URL>
```

To use RAGScraper in a Python script:

```python
from rag_scraper.scraper import Scraper
from rag_scraper.converter import Converter

# Fetch HTML content
url = "https://example.com"
html_content = Scraper.fetch_html(url)

# Convert to Markdown
markdown_content = Converter.html_to_markdown(
    html=html_content, 
    base_url=base_url,
    parser_features='html.parser', 
    ignore_links=True
)
print(markdown_content)
```

## Development

To run the tests for RAGScraper, navigate to the package directory and run:

```bash
python -m unittest discover tests
```

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.
slackhistory 0.3.7
A slack helper library to get history (messages and replies) of a channel
License: MIT
Requires Python: >=3.11
Keywords: slackhistory,slack history,slack helper

# Slack History

A Slack helper library to get the history (messages and replies) of a channel


## Installation
```bash
pip install slackhistory
```

## Quick start
```python
import asyncio
from slackhistory import SlackHelper
from slackhistory.type.slack_types import SlackMessageAndReply
from os import environ


async def main():
    slack_helper = SlackHelper(token=environ.get("TOKEN", ""))

    # end_date is optional
    message_and_replies: SlackMessageAndReply = await slack_helper.get_message_and_replies(
        "alex-playground", start_date="2023-10-28 08:00:00", end_date="2023-11-4 22:00:00")

    # the result can be converted to dict or json
    print(message_and_replies.to_dict())
    print(message_and_replies.to_json())


if __name__ == "__main__":
    asyncio.run(main())
```
Example Result: [example_result.json](./examples/example_result.json)