semvec/pypi-packages huggingface dataset:
cfx-middleware 3.0
Provide a Middleware between the system and CFX
License: MIT
Requires Python: >=3

# CFX Middleware

## Description
CFX Middleware is a middleware platform designed to facilitate integration and communication between parser results systems and CM

## Author
- Victor Dominguez

## Installation

To install the package run

```bash
pip install cfx_middleware
```

if you want a specific version use

```bash
pip install cfx_middleware==version
```

## Usage
In your py file you must call the package `from cfx_middleware import parser_result` and provide the parser result dict or JSON as parameter

## Example
```python
from cfx_middleware import parser_result

success = parser_result({parser_result})

if success:

    do something
```

## License
This project is licensed under the PCD License.

## Contributing
Contributions are welcome! If you'd like to contribute to CFX Middleware, please follow these steps:
1. Fork the repository.
2. Create a branch for your new feature (`git checkout -b feature/new-feature`).
3. Make your changes and commit them (`git commit -am 'Add new feature'`).
4. Push the branch (`git push origin feature/new-feature`).
5. Create a new pull request.
pader 0.3
轻量级的爬虫框架，支持中间件、检验等功能
License: MIT
Keywords: Python, Spider

# 项目说明

- 轻量框架，支持中间件、检验等功能。用法与Scrapy、Feapder类似。

# Python解释器

- python3

# 如何使用pader？

#### 使用SlowSpider

- 单线程爬虫

```python
from loguru import logger

import pader


class TestSpider(pader.SlowSpider):
    start_urls = ['https://www.baidu.com']

    def when_spider_start(self):
        print('爬虫开始了...')

    def when_spider_close(self):
        print('...爬虫结束了')

    def parse(self, request, response):
        lis = response.xpath('//ul[@id="hotsearch-content-wrapper"]/li')
        for li in lis:
            url = li.xpath('./a/@href').get()
            title = li.xpath('./a/span[last()]/text()').get()
            logger.success(title)
            logger.success(url)
            logger.info('\r')
            yield pader.Request(url, callback=self.parse_detail)

    def parse_detail(self, request, response):
        nodes = response.xpath('//div[@class="c-container"]//h3')
        for node in nodes:
            some = node.xpath('./a//text()').getall()
            title = ''.join(some)
            url = node.xpath('./a/@href').get()
            logger.success(title)
            logger.success(url)

    def middleware(self, request):
        request.mark = '百度首页' if request.callback.__name__ == 'parse' else '百度搜索页'
        logger.info('进入了中间件，已设置记号为{}'.format(request.mark))

    def validate(self, request, response):
        logger.warning('进入了校验，记号={}'.format(request.mark))


if __name__ == '__main__':
    TestSpider().crawl()

```

#### 使用PaderSpider

- 多线程爬虫

```python
import threading
import time

from loguru import logger

import pader


def t_name():
    return threading.current_thread().name


def show(request):
    logger.success("回调: {}  =>  线程: {}".format(request.callback.__name__, t_name()))


URL = "https://www.baidu.com/s?&wd=python3"


class TestSpider(pader.PaderSpider):
    def start_requests(self):
        for i in range(5):
            yield pader.Request(URL)

    def when_spider_start(self):
        logger.info('爬虫开始了...')

    def when_spider_close(self):
        logger.info('...爬虫结束了')

    def parse(self, request, response):
        show(request)
        for i in range(2):
            mark = 'parse-{}'.format(i + 1)
            yield pader.Request(URL, mark=mark, callback=self.parse_list)

    def parse_list(self, request, response):
        show(request)
        for i in range(3):
            mark = 'parse_list-{}'.format(i + 1)
            yield pader.Request(URL, mark=mark, callback=self.parse_detail)

    def parse_detail(self, request, response):
        show(request)

    def middleware(self, request):
        time.sleep(1)  # 睡眠1S方便看出并发效果


if __name__ == '__main__':
    TestSpider(speed=5, qsize=10).crawl()

```
cli-support 2.0.2
Support component license information (CLI) files
License: MIT
Requires Python: <4.0,>=3.8

# CLI Support for Python

[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/sw360/clipython/blob/master/License.md)
[![Python Version](https://img.shields.io/badge/python-3.8%2C3.9%2C3.10%2C3.11-yellow?logo=python)](https://www.python.org/doc/versions/)
[![PyPI](https://shields.io/pypi/v/cli-support)](https://pypi.org/project/cli-support)
[![Static checks](https://github.com/sw360/clipython/actions/workflows/python-package.yml/badge.svg)](https://github.com/sw360/clipython/actions/workflows/python-package.yml)
[![Unit tests](https://github.com/sw360/clipython/actions/workflows/unit-test.yml/badge.svg)](https://github.com/sw360/clipython/actions/workflows/unit-test.yml)

Python library to read and write Component License Information (CLI) files. They can be
created by [FOSSology](https://www.fossology.org) and stored in
[SW360](https://www.eclipse.org/sw360/).

For more information about the CLI file format, please have a look at
[ComponentLicenseInformation.md](ComponentLicenseInformation.md).

## Usage

### Installation

This project is available as [Python package on PyPi.org](https://pypi.org/project/cli-support/).  
Install cli_support and required dependencies:

  ```shell
  pip install cli_support
  ```

### Required Packages

* none

## Using the API

* Start using the API:

  ```python
  import cli_support
  clifile = cli_support.CLI.CliFile()
  clifile.read_from_file("cli_filename")
  ```

## Contribute

* All contributions in form of bug reports, feature requests or merge requests are welcome!
* Please use proper [docstrings](https://realpython.com/documenting-python-code/) to document
  functions and classes.
* Extend the testsuite **poetry run pytest** with the new functions/classes

## Build

### Building Python package

For building the library, you need [Poetry](https://python-poetry.org/). Build is then
simply triggered using

```shell
poetry build
```

This creates the source and wheel files in ```dist/``` subdirectory -- which can then
be uploaded or installed locally using ```pip```.

## Test

Start the complete test suite or a specific test case (and generate coverage report):

```shell
poetry run pytest
```

or

```shell
poetry run coverage run -m pytest
poetry run coverage report -m --omit "*/site-packages/*.py"
poetry run coverage html --omit "*/site-packages/*.py"
```

## Demo

The script ``show_licenses.py`` shows how to use the library to retrieve some information
of a given CLI file.

```shell
python ./show_licenses.py ./test/testfiles/CLIXML_MIT_simple.xml
```

## License

Copyright 2019-2024 Siemens

The project is licensed under the MIT license.  
SPDX-License-Identifier: MIT