semvec/pypi-packages huggingface dataset:
ctmc 0.1.3
Continous Time Markov Chain
License: MIT
Requires Python: >=3.6

[![Build Status](https://travis-ci.org/kmedian/ctmc.svg?branch=master)](https://travis-ci.org/kmedian/ctmc)
[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/kmedian/ctmc/master?urlpath=lab)

# ctmc


## Table of Contents
* [Installation](#installation)
* [Usage](#usage)
* [Commands](#commands)
* [Support](#support)
* [Contributing](#contributing)


## Installation
The `ctmc` [git repo](http://github.com/kmedian/ctmc) is available as [PyPi package](https://pypi.org/project/ctmc)

```
pip install ctmc
```


## Usage
Check the [examples](https://github.com/kmedian/ctmc/tree/master/examples) folder for notebooks.


## Commands
* Check syntax: `flake8 --ignore=F401`
* Run Unit Tests: `python -W ignore -m unittest discover`
* Remove `.pyc` files: `find . -type f -name "*.pyc" | xargs rm`
* Remove `__pycache__` folders: `find . -type d -name "__pycache__" | xargs rm -rf`
* Upload to PyPi with twine: `python setup.py sdist && twine upload -r pypi dist/*`


## Debugging
* Notebooks to profile python code are in the [profile](https://github.com/kmedian/ctmc/tree/master/profile) folder


## Support
Please [open an issue](https://github.com/kmedian/ctmc/issues/new) for support.


## Contributing
Please contribute using [Github Flow](https://guides.github.com/introduction/flow/). Create a branch, add commits, and [open a pull request](https://github.com/kmedian/ctmc/compare/).
foliantcontrib.glossary 1.0.0
Preprocessor for Foliant for collecting document glossary.
License: MIT

# Glossary collector for Foliant

Glossary preprocessor collects terms and definitions from the files stated and inserts them to specified places of the document.


## Installation

```shell
$ pip install foliantcontrib.glossary
```


## Config

To enable the preprocessor, add `glossary` to `preprocessors` section in the project config.

```yaml
preprocessors:
  - glossary
```

The preprocessor has a number of options (default values stated below):

```yaml
preprocessors:
  - glossary:
        term_definitions: 'term_definitions.md'
        definition_mark: ':   '
        files_to_process: ''
```

`term_definitions`
:   Local or remote file with terms and definitions in Pandoc [definition_lists](https://pandoc.org/MANUAL.html#definition-lists) notation (by default this file stored in foliant project folder, but you can place it other folder). Also you can use [includes](https://foliant-docs.github.io/docs/preprocessors/includes/) in this file to join several glossary files. In this case `includes` preprocessor should be stated before `glossary` in `foliant.yml` preprocessors section. Note that if several definitions of one term are found, only first will be used.

`definition_mark`
:   Preprocessor uses this string to determine, if the definition should be inserted here. `":   "` for Pandoc [definition_lists](https://pandoc.org/MANUAL.html#definition-lists) notation.

`files_to_process`
:   You can set certain files to process by preprocessor.


## Usage

Just add the preprocessor to the project config, set it up and enjoy the automatically collected glossary in your document.


## Example

**foliant.yml**

```yaml
...
chapters:
    - text.md

preprocessors:
...
    - includes
    - glossary
...
```

**term_definitions.md**

```
# Glossary

<include nohead="true">
    $https://git.repo/repo_name_1$src/glossary_1.md
</include>

<include nohead="true">
    $https://git.repo/repo_name_2$src/glossary_2.md
</include>
```

**glossary_1.md** from **repo_1**

```
# Glossary

Term 1

:   Definition 1

Term 2

:   Definition 2

Term 3

:   Definition 3

```

**glossary_2.md** from **repo_2**

```
# Glossary

Term 4

:   Definition 4

        { some code, part of Definition 4 }

    Third paragraph of definition 4.

Term 5

:   Definition 5

```

**text.md**

```
# Main chapter

Some text.

# Glossary

:   Term 1

:   Term 4

:   Term 2

```

**\_\_all\_\_.md**

```
# Main chapter

Some text.

# Glossary

Term 1

:   Definition 1


Term 4

:   Definition 4

        { some code, part of Definition 4 }

    Third paragraph of definition 4.


Term 2

:   Definition 2

```
ethextended 0.2.1
Library with extra tools for ethereum, provided by Etherscan, Eth Gas Station, etc.
License: MIT

ethextended
===========
Library with extra tools for ethereum, provided by Etherscan, Eth Gas Station, etc.

.. image:: https://travis-ci.org/AlexSSD7/ethextended.svg?branch=master
    :target: https://travis-ci.org/AlexSSD7/ethextended

.. image:: https://readthedocs.org/projects/ethextended/badge/?version=latest
	:target: https://ethextended.readthedocs.io/en/latest/?badge=latest


`�� Read The Docs �� <https://ethextended.readthedocs.io/>`_

Installation
------------
Install using **pip**:

.. code-block:: shell

	$ pip3 install ethextended

Or build from **source**:

.. code-block:: shell

	$ python3 setup.py install
structure-spider 1.3.5
multi requests to combine a structure item.
License: MIT
Keywords: spider

# 结构化爬虫
通过组建Item请求树抓取结构化数据

![](https://github.com/ShichaoMa/structure_spider/blob/master/resources/item-collector.jpg)
# USAGE
### 安装structure_spider
```
dev@ubuntu:~$ pip install structure-spider
```
### 生成项目
```
dev@ubuntu:~$ structure-spider create project -n myapp
New structure-spider project 'myapp', using template directory '/home/dev/.pyenv/versions/3.6.0/lib/python3.6/site-packages/structor/templates/project', created in:
    /home/dev/myapp

You can start the spider with:
    cd myapp
    custom-redis-server -ll INFO -lf
    scrapy crawl douban
```
### 开始简单redis，可以使用正式版redis，只需把settings.py中的`CUSTOM_REDIS=True`注释掉即可
```
dev@ubuntu:~$ custom-redis-server -ll INFO -lf
```
### 生成自定义spider及item
使用createspider可以生成直接可用的spider，-s指定spider名称，随后创建要抓取的字段及其规则
，使用=连接。规则可以是正则表达式，xpath, css。

如需进一步增加复杂规则或进行数据清洗，请参考wiki。
```
dev@ubuntu:~$ cd myapp/myapp/
dev@ubuntu:~/myapp/myapp$ ls
items  settings.py  spiders
dev@ubuntu:~/myapp/myapp$ structure-spider create spider -n zhaopin "product_id=/(\d+)\\.htm" "job=//h1/text()" "salary=//a/../../strong/text()" 'city=//ul[@class="terminal-ul clearfix"]//strong/a/text()' 'education=//span[contains(text(), "学历")]/following-sibling::strong/text()' "company=h2 > a" -ip '//td[@class="zwmc"]/div/a[1]/@href' -pp '//li[@class="pagesDown-pos"]/a/@href'
ZhaopinSpdier and ZhaopinItem have been created.
dev@ubuntu:~/myapp/myapp$
```

参考资料：[使用structure_spider多请求组合抓取结构化数据](https://zhuanlan.zhihu.com/p/28636195)
### 启动爬虫
```
dev@ubuntu:~/myapp/myapp$ scrapy crawl zhaopin
```
### 投入任务
```
dev@ubuntu:~/myapp$ structure-spider feed -s zhaopin -u "https://sou.zhaopin.com/jobs/searchresult.ashx?jl=%E6%B5%8E%E5%8D%97&kw=%E9%94%80%E5%94%AE&sm=0&p=1" -c zhaopin --custom # --custom代表使用的是简单redis
```
### 查看任务状态
```
dev@ubuntu:~/myapp$ structure-spider check zhaopin --custom
```
更多资源:

[[structure_spider每周一练]：一键下载百度mp3](https://zhuanlan.zhihu.com/p/29076630)

[个性化爬虫一键生成，想抓哪里点哪里！](https://zhuanlan.zhihu.com/p/33561576)

[scrapy进阶，组合多请求抓取Item利器ItemCollector详解！](https://zhuanlan.zhihu.com/p/33699058)