semvec/pypi-packages huggingface dataset:
mcp-aim-crawler 0.1.0
Web crawler for datatist.com
Requires Python: >=3.12.9
Keywords: datatist, mcp, web crawler

# MCP Aim Crawler

这是一个基于MCP协议的网站爬虫项目。

## 项目结构

```
mcp_aim_crawler/
├── tests/                # 测试用例目录
└── mcp_aim_crawler/     # 主要源代码目录
    ├── crawler.py       # 爬虫核心逻辑
    └── logger.py        # 日志配置
```

## 环境要求

- Python 3.12.9
- Poetry
- pyenv

## 安装方法

1. 使用pyenv安装Python 3.12.9:
```bash
pyenv install 3.12.9
pyenv local 3.12.9
```

2. 安装项目依赖:
```bash
poetry install
```

## 运行方法

```bash
poetry run python -m mcp_aim_crawler
```

## 环境变量

在项目根目录创建 `.env` 文件，包含以下配置:

- `WEBSITE_URL`: 目标网站URL
- `USERNAME`: 登录用户名
- `PASSWORD`: 登录密码
- `MCP_TOKEN`: MCP令牌
ouseful-fsa-datasupply 0.2.0
A simple tool for bulk downloading of etsablishment details and ratings data from food.gov.uk

# food\_gov\_uk
Tools for grabbing data from food.gov.uk

`pip install --upgrade git+https://github.com/ouseful-datasupply/food_gov_uk.git`


[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/ouseful-datasupply/food_gov_uk/master)

## Command Line Interface

The CLI will scrape the [UK Food Standards Agency ratings data](https://ratings.food.gov.uk/open-data/en-GB) XML files and construct a simple SQLite database from them containing two tables:

- `fsa_ratings_metadata`: containing the name and publication date of each scraped file
- `USER-DEFINED` (default: `ratingstable`) containing the ratings data from the XML files.

The created database can then be used with tools such as [`datasette`](https://github.com/simonw/datasette/tree/master/datasette).

```
Usage: fsa_data [OPTIONS] COMMAND

Commands:
  collect              Collect data from FSA website

Options:
  --dbname TEXT        SQLite database name (default: fsa_ratings_all.db)
  --ratingstable TEXT  FSA Ratings table name (default: ratingstable)
  --area TEXT          Specfiy an area to download data for
  --authority TEXT     Specfiy a local authority to download data for
  --help               Show this message and exit.
```