semvec/pypi-packages huggingface dataset:
yahoo-finance-symbols 0.1.4
A Library for scraping all Ticker Symbols from Yahoo Finance
License: MIT
Requires Python: >=3.7

# YAHOO FINANCE SYMBOLS

This Library helps in scraping 450,000+ symbols from yahoo finance. The symbols are saved in a local sqlite database which can be used directly or accessed with the rust or python library functions.

## Installation

### Python

``` bash
pip install yahoo_finance_symbols
```

### Rust

``` bash
cargo install yahoo_finance_symbols
```


## Examples

### Python

``` python
import yahoo_finance_symbols as ys

# Fetch All Symbols
all_symbols = ys.get_symbols()
print(all_symbols)

# Search for Symbols With a Keyword
symbols = ys.search_symbols("Bitcoin", "ETF")
print(symbols)

# Update the Database
ys.update_database()
```

### Rust

``` rust
use yahoo_finance_symbols::{get_symbols_df, search_symbols, update_database};
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {

    // Fetch All Symbols
    let all_symbols = get_symbols_df().await?;
    println!("{:?}", all_symbols);

    // Search for Symbols with a Keyword
    let symbols = search_symbols("Apple", "Equity").await?;
    println!("{:?}", symbols);

    // Update the Database
    update_database().await()?;
}
```
scrapy_huo_utilities 0.0.4
scrapy_huo_utilities是自用的一些scrapy工具代码合集。
License: MIT
Requires Python: <4.0,>=3.5

# scrapy_huo_utilities

#### 介绍

包含一个随机UserAgent和基于leveldb的DuplicateUrls中间件、几个清洗html的函数,自用的几个常用功能。

#### 安装教程

`pip install scrapy_huo_utilities`

#### 使用说明

1. RandomUserAgent_DOWNLOADER_MIDDLEWARES:

   在`settings.py`的`DOWNLOADER_MIDDLEWARES`中修改添加：
   `"scrapy_huo_utilities.MIDDLEWARES.Downloader_Middleware_Utils.RandomUserAgent_DOWNLOADER_MIDDLEWARES": 543,`


2. DuplicateUrls_SPIDER_MIDDLEWARES:
   在`settings.py`的`SPIDER_MIDDLEWARES`中修改添加：
   `"scrapy_huo_utilities.MIDDLEWARES.Spider_Middleware_Utils.DuplicateUrls_SPIDER_MIDDLEWARES": 543,`

3. html_clean:
   `from scrapy_huo_utilities.PROCESSORS import clean_html_attributes,clean_html_tags,clean_empty_img_tags`

   `clean_empty_img_tags`:清除没有src属性或src属性为空字符串的img标签。

   `clean_empty_a_tags`:清除没有href属性或href属性为空字符串的a标签。

   `clean_html_tags`:用于清除指定列表中的HTML标签。
   ```
   :param html_data:  传入html
   :param remove_tags:删除标签列表
   :param reserve_content:是否保留删除标签的文本内容，默认保留。
   :return: 处理后的html
   ```

   `clean_html_attributes`：用于清除HTML标签中的属性，除了白名单中包含的属性。
   ```
   :param html_data:传入html
   :param whitelist:白名单
   :return: html_data处理后的html
   ```