semvec/pypi-packages huggingface dataset:
sshcracker 1.2.1
ssh弱密码破解。支持自定义目标ip和弱口令。每个ip都使用一个单独的线程破解。

# sshcracker
ssh弱密码破解。支持自定义目标ip和弱口令。每个ip都使用一个单独的线程破解。

## 安装依赖

执行 `pip install paramiko`

## 操作步骤：

1） 在 `IpText.txt` 中配置目标ip

2） 在 `Accounts.txt` 中配置要检测的用户名和密码。（忽略这步，则使用默认的弱口令）

3） 执行 `python run.py`

如果成功，则会打印 `<ip> user is: <username>, password is: <password>`
sphinx-quant 1.2.5
This project is sphinx-quant web technologies.

# sphinx-quant

## 开发进度

- [x] Circle Ci 自动化
- [x] 官方网站
- [x] 策略回测模块
- [ ] 交易管理模块
- [ ] 账户管理模块
- [ ] 网关管理模块
- [ ] 风险管理模块
- [ ] 分布式 Worker
- [ ] 数据统计模块
- [ ] 官方教程

## Document

请前往: [https://www.sphinxquant.com](https://www.sphinxquant.com)

## Screen Shot

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s1.png?v=1.0.1" alt="drawing" width="100%"/>

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s2.png?v=1.0.1" alt="drawing" width="100%"/>

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s3.png?v=1.0.1" alt="drawing" width="100%"/>

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s4.png?v=1.0.1" alt="drawing" width="100%"/>

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s5.png?v=1.0.1" alt="drawing" width="100%"/>

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s6.png?v=1.0.1" alt="drawing" width="100%"/>

<img src="https://sphinx-1253762454.file.myqcloud.com/screenshot/s7.png?v=1.0.1" alt="drawing" width="100%"/>

## 感谢支持

<center><img src="https://sphinx-1253762454.file.myqcloud.com/qrcode.jpg?v=1.0.1" alt="drawing" width="300"/></center>

## License

Icon made by [geotatah](https://www.flaticon.com/authors/geotatah) from www.flaticon.com
sogaQuant 2.1.2
沪深A股量化.

# 沪深A股量化

---
## 0.概述


## 1.运行环境

* python2.7
* php5.5
* mysql5
* mango
* Elasticsearch
+ pycrul
* shell
    * a

## 2.``交易系统``和``心智系统``
* 选股系统
    * 宏观-产业-个股-策略-交易-风控的投资分析逻辑
    * 基本-财务-地位-空间-估值-风险的分析框架

* 估值系统
    * 利润增长率再结合净资产收益率
    * 

* 指标系统
    * MACD
    * DMA

* 大盘系统
    * 盯盘
    * 扫单，吃单，对敲，出货，托盘，压盘，捂盘，洗盘，拉高
    * 建仓、减持、对倒、拉升、派发的特征
    * 大单意图定性、放量和缩量定性、换手率、K线建仓和减仓、对倒

* 风控系统
    * 资金管理

* 卖出系统
    * 给单
    * 止损

* 格局规划贪婪和认知战胜恐惧

## 3.系统
| 序号 | U | I | T |
|:-------------:|:-------------|:-------------|:-------------|
|1| [名称](a/b.md) |I|T|

## 4.
sinalv2 1.1.5
该软件需要自行购买新浪的level2标准版.

# Sina Level2 Quote Python API
该软件需要自行购买新浪的level2标准版 (168每月, 998每年)

暂时不支持新浪的level2普及版 (60每月, 298每年) (楼主正在研究中...)

购买地址 http://stock.finance.sina.com.cn/stock/buy.php

标准版网页 http://stock.finance.sina.com.cn/lv2/sh603001.html

普及版网页 http://finance.sina.com.cn/realstock/company/sh603001/l2.shtml

版本要求 Python3.4+

# 安装
```
git clone https://github.com/flycandy/sinalv2
cd sinalv2
pip install -r requirements.txt
vim account.example.json # 修改account.example.json 为自己的用户名密码
python sinalv2.py
```



# 新浪level2行情接口 说明

## Step 1 登陆新浪

登陆脚本参考该页面 Reference https://gist.github.com/mrluanma/3621775

## Step 2 请求一个token


请求地址: https://current.sina.com.cn/auth/api/jsonp.php/varxxxl/AuthSign_Service.getSignCode
方法: Get

字段       | 说明 
---------|-------
query_type | A_hq (A股行情)
 ip  | 当前机器的公网IP
list | 请求的查看股票的列表 
kick | 设置为1, 否则可能出现取不到token的情况




## 使用token用websocket获取行情

获取token是一个websocket协议

ws://ff.sinajs.cn/wskt?token={token}&list={list}

token必须是使用该{list}得到的token. 不然会报错

token需要定时更新. 每一个websocket只会持续大概3-5分钟. 当websocket disconnect之后, 需要重新请求一个token. 然后再次链接websocket
shadowProxy-pro 1.2.1
幽灵模式下动态评估代理质量，并进行优化选择

# shadowProxy


## 使用方法

- 查看帮助
```bash
 python3 shadowProxy.py -h
        .--.
       |o_o |    ------------------
       |:_/ |   < Author: Mr.Bingo >
      //   \ \   ------------------
     (|     | ) <    oddboy.cn     >
    /'\_   _/`\  ------------------
    \___)=(___/

usage: shadowProxy.py [-h] [--bind BIND] [--port PORT]
                      [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
                      [--proxyListFile PROXYLISTFILE] [-t MULTIPLETIMES]

optional arguments:
  -h, --help            show this help message and exit
  --bind BIND           Default: 127.0.0.1
  --port PORT           Default: 8088
  --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        Default: WARNING
  --proxyListFile PROXYLISTFILE
                        代理列表文件
  -t MULTIPLETIMES      单一代理可被使用的次数,默认为2^31-1
```

- 运行
```bash
python3 shadowProxy.py --proxyListFile proxylist.txt
        .--.
       |o_o |    ------------------
       |:_/ |   < Author: Mr.Bingo >
      //   \ \   ------------------
     (|     | ) <    oddboy.cn     >
    /'\_   _/`\  ------------------
    \___)=(___/

初始化代理池  本地IP :: 111.199.186.1
导入代理池:::	proxylist.txt
成功导入 55 个代理
Serving HTTP on 127.0.0.1 port 8088 (http://127.0.0.1:8088/) ...
```

- 安装SSL证书(访问HTTPS需要)

【方式一】

Unix-Like系统下，直接运行`setup_https_intercept.sh`生成证书。
```bash
./setup_https_intercept.sh
Generating RSA private key, 2048 bit long modulus
........................................................+++
.........+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
....................................................................+++
..+++
e is 65537 (0x10001)
```
【方式二】
*nix及windows主机均可使用。 
```bash
$ python3 certs.py CREATECA
生成CA根证书：shadowProxy/certs/shadowproxyCA.crt
根证书私钥：shadowProxy/certs/shadowproxyCA.key
Proxy私钥：shadowProxy/certs/cert.key
生成测试站点证书：shadowProxy/certs/sites/testcert.crt
```

然后在代理到端口的浏览器中访问[http://shadow.proxy/](http://shadow.proxy/) 即可下载，导入系统/浏览器即可。

## todo list
- 运行过程中动态更新代理库

- 明确打一枪换一炮模式与幽灵模式

- 幽灵模式下动态评估代理质量，并进行优化选择

- 完善日志记录
sdufeQuant 1.1.1
RQAlpha 从数据获取、算法交易、回测引擎，实盘模拟，实盘交易到数据分析，为程序化交易者提供了全套解决方案。


# Web Reverse Engineering Project

## Introduction

This project is dedicated to sharing and teaching the fundamentals and techniques of web reverse engineering. Web reverse engineering involves analyzing how websites and web applications work, and cracking or modifying existing code to achieve specific objectives. This project includes a series of tutorials, practical tools, and case studies aimed at helping developers, security researchers, and enthusiasts understand how to effectively reverse engineer web technologies.

## Features

- **Educational Material**: Provides tutorials on web reverse engineering techniques from beginner to advanced levels.
- **Tools and Resources**: Shares effective reverse engineering tools and resource links.
- **Case Studies**: Delve into actual web reverse cases, analyzing techniques and strategies.

## Installation

To use the resources of this project, you need to set up your development environment first. Below are the basic installation instructions:

```bash
# Clone the repository
git clone https://github.com/yanjlee/reverse_life.git

# Enter the project directory
cd your-repository-name

# Install dependencies (if any)
pip install -r requirements.txt
```

## Usage

This project provides examples and tools for various web reverse engineering techniques. Here is how to use them:

```python
# Example: Using a tool to reverse a website
python reverse_example.py
```

## Contribution

Contributions of any kind are welcome, be it new cases, documentation improvements, or code updates. Please read `CONTRIBUTING.md` for information on how to start contributing.

## License

This project is licensed under the MIT License. For more details, please check the `LICENSE` file.

## Contact

If you have any questions or suggestions, please contact me through Issues or directly via email.
scrapy-zhihu-github 1.2.5
用于爬取zhihu和github的代码，数据存储于mongodb。.

scrapy-zhihu-github
===================

用于爬取zhihu和github的代码，数据存储于mongodb。


# Install

Scrapy安装见[使用Scrapy抓取数据](http://blog.javachen.com/2014/05/24/using-scrapy-to-cralw-data.html)。

Mongodb安装在本机，数据库为`zhihu`，端口默认，存在以下collection：
              
   - `zh_user`：知乎用户
   - `zh_ask`：知乎问题
   - `zh_answer`：知乎回答
   - `zh_followee`：知乎关注列表
   - `zh_follower`：知乎粉丝列表
   - `gh_user`：github 用户
   - `gh_repo`：github 仓库

# zhihu

Scrapy爬取知乎数据，说明见[使用Scrapy爬取知乎网站](http://blog.javachen.com/2014/06/08/using-scrapy-to-cralw-zhihu.html)。

zhihu 用户表结构（db.zhihu.zh_user）为：

```
_id int, # 用户id
url string,
username string, # 用户名，如 zhouyuan
nickname string, # 昵称，如 周源
location string, # 居住地
industry string, # 行业，如 互联网
sex int, # 性别，1：男， 2：女， 0：未知
jobs [],
educations [],
description string, # 自我简介
sinaweibo string, # 新浪微博账号
tencentweibo string, # 腾讯微博账号
# qq string, # QQ号
ask_num int, # 提问数， 如 590
answer_num int, # 回答数，如 340
post_num int, # 专栏文章数， 如 3
collection_num int, # 收藏数，如 9
log_num int, # 编辑数，如14980
agree_num int, # 收到的赞同，如 15316
thank_num int, # 收到的感谢，如 3500
fav_num int, # 被收藏次数，如 9424
share_num int, # 被分享次数，如 922
followee_num int, # 关注数，如 1515
follower_num int, # 被关注数（粉丝），如 319529
update_time datetime # 信息更新时间，如 2014-05-17 11:15:00
```

先运行下面代码，采集用户信息以及用户的关注和粉丝列表：

```python
scrapy crawl zhihu_user
```

再来采集问题和答案：

```python
scrapy crawl zhihu_ask

scrapy crawl zhihu_answer
```



# github

github 用户表结构（db.zhihu.gh_user）为：

```
_id, #用户id
url, #主页url
username,#用户名
nickname,#昵称 
user_id,#用户id
type,#类型：1,组织；0,个人 

company,#公司
location,#位置 
website,#网站 
email,#邮箱 
update_time,#爬虫更新时间

join_date,#加入时间
followee_num,#关注数
follower_num,#粉丝数 
star_num,#星数 
organizations,#加入的组织

member_num,#组织成员数
```

先运行下面代码，采集用户信息：

```python
scrapy crawl github_user
```

爬取用户信息以及粉丝用户:

```python
scrapy crawl github_follower
```

查看爬取的结果:

```
> use zhihu
switched to db zhihu
> db.gh_user.count()
126135
```
scrapy-taobao 1.1.5
scrapy模拟淘宝登陆，未加代理ip的处理。希望有好的代理处理方法分享出来。.

# scrapy-taobao
scrapy模拟淘宝登陆，未加代理ip的处理。希望有好的代理处理方法分享出来。

# 确保安装了scrapy。
    self.http_user = 'xxxxxxxx'   # taobao username
    self.http_pass = 'xxxxxxxx'   # taobao password
记得修改taobao_spider.py中的用户名username和密码password。\<br>

# 运行命令
    scrapy crawl taobao
  如果用户登陆需要输入验证码，则会自动打开验证码的图片链接让客户手动输入，输入错误会重新打开验证码的图片链接供用户再次输入。

# 登陆成功的提示
    login-success, get user nick: ["user nick"]
用户看到这句代表登陆成功，可以进行一些其他数据的提取。
scrapy-jingdong 1.2.1
用于爬取JD和github的代码，数据存储于mongodb。.

scrapy crawl test -s JOBDIR=data -0 items.csv

watch -n 1 -d 'wc -l items.csv;du -h items.csv'

附数据 items.csv

![](img.png)

http://hackerxu.com/2014/09/07/scrapy-jd.html
rqalpha-minsec-trade-realtime 1.2.1
基于开源rqalpha4.0版本，支持分钟和tick级别的回测.

#本项目做了几个更改：
## 1 基于开源rqalpha4.0版本，支持分钟和tick级别的回测
	增加了两个mod，：rqalpha_mod_minute_trade（\rqalpha4_min_tick_trade_mod\minute_trade\rqalpha_mod_minute_trade）和rqalpha_mod_tick_trade（\rqalpha4_min_tick_trade_mod\tick_trade\rqalpha_mod_tick_trade）。当你按照rqalpha的h5格式存放你的分钟和tick数据时，就可以支持分钟和tick级别的回测
## 2 基于开源rqalpha3.4版本支持模拟交易的功能
###	2.1需要用rqalpha3.4_patch下面的文件覆盖原有的rqalpha3.4的文件
cp -rd ./rqalpha3.4_patch/disk_persist_provider.py /rqalpha_home/rqalpha/utils/
cp -rd ./rqalpha3.4_patch/asset_position.py /rqalpha_home/rqalpha/mod/rqalpha_mod_sys_accounts/position_model/
###    2.2 改写rqalpha_mod_stock_realtime,并且需要你有本地的历史指数和股票数据。因为rqalpha做模拟的时候有些条件下会需要历史history_bar,你可以不使用dataapi，但是你需要找到rqalpha相应的函数。
python-twitter-pro 0.2.5
A simple Python wrapper for Twitter API v2 :sparkles: :cake: :sparkles:


# Web Reverse Engineering Project

## Introduction

This project is dedicated to sharing and teaching the fundamentals and techniques of web reverse engineering. Web reverse engineering involves analyzing how websites and web applications work, and cracking or modifying existing code to achieve specific objectives. This project includes a series of tutorials, practical tools, and case studies aimed at helping developers, security researchers, and enthusiasts understand how to effectively reverse engineer web technologies.

## Features

- **Educational Material**: Provides tutorials on web reverse engineering techniques from beginner to advanced levels.
- **Tools and Resources**: Shares effective reverse engineering tools and resource links.
- **Case Studies**: Delve into actual web reverse cases, analyzing techniques and strategies.

## Installation

To use the resources of this project, you need to set up your development environment first. Below are the basic installation instructions:

```bash
# Clone the repository
git clone https://github.com/yanjlee/reverse_life.git

# Enter the project directory
cd your-repository-name

# Install dependencies (if any)
pip install -r requirements.txt
```

## Usage

This project provides examples and tools for various web reverse engineering techniques. Here is how to use them:

```python
# Example: Using a tool to reverse a website
python reverse_example.py
```

## Contribution

Contributions of any kind are welcome, be it new cases, documentation improvements, or code updates. Please read `CONTRIBUTING.md` for information on how to start contributing.

## License

This project is licensed under the MIT License. For more details, please check the `LICENSE` file.

## Contact

If you have any questions or suggestions, please contact me through Issues or directly via email.
python-telegram-bot-pro 2.1.1
This project is dedicated to sharing and teaching the fundamentals and techniques of web reverse engineering. Web reverse engineering involves analyzing how websites and web applications work, and cracking or modifying existing code to achieve specific objectives. This project includes a series of tutorials, practical tools, and case studies aimed at helping developers, security researchers, and enthusiasts understand how to effectively reverse engineer web technologies.


# Web Reverse Engineering Project

## Introduction

This project is dedicated to sharing and teaching the fundamentals and techniques of web reverse engineering. Web reverse engineering involves analyzing how websites and web applications work, and cracking or modifying existing code to achieve specific objectives. This project includes a series of tutorials, practical tools, and case studies aimed at helping developers, security researchers, and enthusiasts understand how to effectively reverse engineer web technologies.

## Features

- **Educational Material**: Provides tutorials on web reverse engineering techniques from beginner to advanced levels.
- **Tools and Resources**: Shares effective reverse engineering tools and resource links.
- **Case Studies**: Delve into actual web reverse cases, analyzing techniques and strategies.

## Installation

To use the resources of this project, you need to set up your development environment first. Below are the basic installation instructions:

```bash
# Clone the repository
git clone https://github.com/yanjlee/reverse_life.git

# Enter the project directory
cd your-repository-name

# Install dependencies (if any)
pip install -r requirements.txt
```

## Usage

This project provides examples and tools for various web reverse engineering techniques. Here is how to use them:

```python
# Example: Using a tool to reverse a website
python reverse_example.py
```

## Contribution

Contributions of any kind are welcome, be it new cases, documentation improvements, or code updates. Please read `CONTRIBUTING.md` for information on how to start contributing.

## License

This project is licensed under the MIT License. For more details, please check the `LICENSE` file.

## Contact

If you have any questions or suggestions, please contact me through Issues or directly via email.
Python-stock 1.2.5
”龙头量化交易策略.py”是基于龙头周期开发的量化交易策略，运行在聚宽平台上面

# “龙头周期.py”是制作龙头周期图的代码；”个股龙虎榜.py”和”每日龙虎榜.py”是制作每日龙虎榜的代码；”龙头量化交易策略.py”是基于龙头周期开发的量化交易策略，运行在聚宽平台上面
# 龙头周期日报![九阳真经日报](https://user-images.githubusercontent.com/6970847/184476348-5a01ee4f-81d3-4aed-9088-603d658cbb60.png)
# 龙虎榜日报![微信截图_20220813193136](https://user-images.githubusercontent.com/6970847/184489889-172df5a1-5b98-45c2-a7fb-78e46bcf8812.png)
Python-JiYanPingtu 1.12.15
jiyan拼图验证码，采用mitmdump脚本获取中间图片，用两张乱码图片合成一张新的图片，在用selenium+chrome滑动滑块

jiyan拼图验证码，采用mitmdump脚本获取中间图片，用两张乱码图片合成一张新的图片，在用selenium+chrome滑动滑块
portia-pro 1.2.5
Portia is a tool that allows you to visually scrape websites without any programming knowledge required. With Portia you can annotate a web page to identify the data you wish to extract, and Portia will understand based on these annotations how to scrape data from similar pages..

Portia
======

Portia is a tool that allows you to visually scrape websites without any programming knowledge required. With Portia you can annotate a web page to identify the data you wish to extract, and Portia will understand based on these annotations how to scrape data from similar pages.

# Running Portia

The easiest way to run Portia is using [Docker]:

You can run Portia using Docker & official Portia-image by running:

    docker run -v ~/portia_projects:/app/data/projects:rw -p 9001:9001 scrapinghub/portia

You can also set up a local instance with [Docker-compose] by cloning this repo & running from the root of the folder:

    docker-compose up

For more detailed instructions, and alternatives to using Docker, see the [Installation] docs.

# Documentation

Documentation can be found from [Read the docs]. Source files can be found in the ``docs`` directory.

[Docker]: https://www.docker.com/
[Docker-compose]:https://docs.docker.com/compose
[Installation]: http://portia.readthedocs.org/en/latest/installation.html
[Read the docs]: http://portia.readthedocs.org/en/latest/index.html
[Scrapinghub]: https://portia.scrapinghub.com/
OnePy-pro 1.1.5
Onepy is an event-driven algorithmic trading Python library.

# Onepy 2.51

Onepy is an event-driven algorithmic trading Python library.

更新日志：[Change Log](CHANGE_LOG.md)

## Install

Onepy is developed using Anaconda Python 3.6.x. 

```{python}
pip install plotly
pip install funcy
pip install arrow
pip install pymongo
pip install retry
pip install TA-Lib
pip install dataclasses # python 3.7不用安装
```

将该项目解压后，参照[这里](https://www.jianshu.com/p/cb6447e1cf88)将OnePy项目文件夹添加到Python搜索目录


## Getting Started

- **Tutorial Series**
  - [Tutorial-1-Introduction](./examples/Tutorial-1-Introduction.ipynb)
  - [Tutorial-2-Strategy-and-Cleaner](./examples/Tutorial-2-Strategy-and-Cleaner.ipynb)
  - [Tutorial-3-Parameters-Optimization](./examples/Tutorial-3-Parameters-Optimization.ipynb)
  - [Tutorial-4-Forward-Analysis](./examples/Tutorial-4-Fordward-Analysis.ipynb)
  - [Tutorial-5-Multi-Strategies](./examples/Tutorial-5-Multi-Strategies.ipynb)
  - [Tutorial-6-Forex-Backtesting](./examples/Tutorial-6-Forex-Backtesting.ipynb)
  - [Tutorial-7-Oanda-Live-Trading](./examples/Tutorial-7-Onada-Live-Trading.ipynb)

- 知乎专栏：[OnePy](https://zhuanlan.zhihu.com/onepy)
  - [OnePy--构建属于自己的量化回测框架](https://zhuanlan.zhihu.com/p/27730907)
  - [OnePy 1.x--量化回测框架初探](https://zhuanlan.zhihu.com/p/28085149)
  - [OnePy 1.x--量化回测框架计算问题反省](https://zhuanlan.zhihu.com/p/28333715)
  - [近况与OnePy 2.00初探](https://zhuanlan.zhihu.com/p/35623695)
  - [OnePy 2.00--整体脉络解读](https://zhuanlan.zhihu.com/p/35860348)
  - [OnePy 2.1 -- 添加单元测试](https://zhuanlan.zhihu.com/p/36707021)
  - [从Python到C++，对量化回测的一点思考](https://zhuanlan.zhihu.com/p/41108459)
  - [OnePy 2.5--自省](https://zhuanlan.zhihu.com/p/49385769)

## Main Features

#### OnePy 综合方面:

*   事件驱动回测设计 ✓
*   股票、外汇回测 ✓
*   多品种回测 ✓
*   多策略回测 ✓
*   设置手续费, 保证金/手, 杠杆大小 ✓
*   设置成交价格为 close 或者第二天 open ✓
*   设置是否打印交易日志 ✓
*   Plot 画图模块 ✓
*   多进程参数优化 ✓
*   多进程Fordward analysis ✓

#### Tools 工具方面:

*   To_MongoDB:自定义数据统一格式后存入数据库 ✓
*   To_MongoDB:tushare 股票数据 CSV 存入数据库 ✓
*   Tushare API 数据多线程存入 MongoDB ✓
*   Oanda API 数据导入MongoDB ✓

#### DataHandler 数据方面:

*   CSV 数据读取 ✓
*   MongoDB 数据读取 ✓

#### Strategy 策略方面:

*   做多 buy & sell, 做空 sell & cover 指令 ✓
*   撤单 cancel_pennding, cancel_tst 指令 ✓
*   按百分比或盈亏多少钱, 设置止盈 Limit、止损 Stop 和 Trailingstop 移动止损 ✓
*   按百分比或者指定价格, 建立挂单。 ✓
*   技术指标 Cleaner 模块 ✓
*   整合 Talib 模块 ✓

#### Broker 执行方面:

*   模拟发送指令 ✓
*   模拟检查指令是否发送成功 ✓
*   打印交易日志 ✓
*   手续费 commission, 百分比类型和固定类型 ✓

#### Recorder 日志方面:

*   计算保证金, 仓位, 浮动利润, 已平仓利润, 总资金, 剩余现金, 收益率, 市值, 时间序列化 ✓
*   输出交易记录, 包括出场时间, 入场时间, 盈亏点数, 盈亏利润等 ✓
*   详细盘后交易结果分析 ✓

#### 延展性方面:

*   在 Environment 中存有原始所有信号信息, 生成的订单信息, 便于分析 ✓
*   自定义扩展事件源 ✓
*   自定义数据源, 返回迭代器给 OnePy 即可 ✓
*   自定义策略模块 ✓
*   自定义技术指标模块 ✓
*   自定义风控模块 ✓
*   自定义经纪商模块 ✓
*   自定义日志记录模块 ✓

## Quick Look

![Plot](./docs/Summary-Plot.png)
![Plot](./docs/Trade-Analysis.png)
![Plot](./docs/Matplotlib-Plot.png)


## Road Map

![执行过程](docs/OnePy_执行过程.png)

## Contact

I'm very interested in your experience with **Onepy**.Please feel free to
contact me via **chenjiayicjy@gmail.com**

**Chandler_Chan**
ofoSpider 1.1.5
ofo共享单车地图爬虫.

#### 接口现在返回的单车id都是6666...
#### 360验证码平台被封，注册机暂时也不能用了。

ofo共享单车地图爬虫
====================

```
感谢您的支持
朋友要做城市数据的一些研究，拜托我写ofo地图的小爬虫
最初的看到了derekhe大神写的单车地图，觉得非常的优秀。
但是单车地图由于官方不断封杀爬虫的行为，被迫关闭了。
derekhe写的爬虫也没办法使用了，于是参考了derekhe的代码。
对现在ofo的api进行了分析，甚至还写了个ofo的token注册机，
实现了对ofo附近单车位置的爬取。

如果这个程序帮助到了你，请动动手指，给个Star！
```

该爬虫为ofo附近单车爬虫
* 新增ofo注册机
* 目前只支持ofo
* 多线程爬取
* 自动去重
* 输出csv文件，存放在db/【日期】/【日期】-【时间】-【品牌】.csv文件内
* gzip压缩存储

# 运行环境
* Python3
* Linux/Mac/Windows

请根据你的需要修改配置文件config.ini，请查看内置说明。

## Linux/Mac
* 下载[最新代码](https://github.com/SilverBooker/ofoSpider/archive/master.zip)并解压
* 修改config.ini确保坐标和区域等参数正确
* 运行：
```
pip3 install -r requirements.txt
python3 run.py
```

## Windows
* 下载[最新代码](https://github.com/SilverBooker/ofoSpider/archive/master.zip)并解压
* 安装好python3.5.3
* 在cmd中执行pip install -r requirements.txt
* 修改config.ini确保坐标和区域等参数正确
* 在cmd执行python run.py

# 输出格式

输出格式：CSV

每行格式：时间戳，单车编号，纬度，经度

# 关于token
```
很多人来问我token的取法，问得人多了，我就认识到了写好文档的重要性
token是发出获取附近单车请求的必要字段，保存在ofo账号登录后cookie，
具体获取方法如下所示
```
* 使用chrome浏览器访问https://common.ofo.so/newdist/?Login&~next=%22%3FJourney%22
* 输入你的账号验证码成功登陆后（强烈建议使用没有押金的小号，被封后果自负）
* 点击地址栏左边“安全”查看当前网站cookie信息
* 然后我们就可以在ofo.so域下发现我们当前账号的token

![图1](/image/1.png)
![图2](/image/2.png)

# 关于批量获取token——ofo注册机
```
这个注册机写的不是很好，毕竟用到了个人不喜欢的selenium，
打码，接码，记录token能够全部自动化，
日后可能会再修改。
```
* 确保你安装了chrome浏览器，之后打开设置->关于chrome查看chrome的版本
* 访问http://blog.csdn.net/huilan_same/article/details/51896672 查看版本对应的chromedriver
* 访问http://chromedriver.storage.googleapis.com/index.html 下载对应的chromedriver
* 修改ofoRegister/login.py下start()函数中chromedriver你所放置的路径
* 访问http://www.360yzm.com/ 360验证码平台注册账号，进行充值（1个账号1毛钱），并在login.py对应的位置填入你的账号密码
* 访问http://www.ruokuai.com/ 若快打码平台注册账号，进行充值（识别一条验证码好像是一分钱），并在login.py对应的位置填入账号密码
* 按需修改ofoRegister/login.py下start()函数中for循环的次数，每次循环生成一个账号
* 最后，在ofoRegister目录下执行python login.py(提示缺少哪个包就用pip安装哪个包)
* 最后就会在ofoRegister目录下生成一个写有token的txt文件
nebula-pro 1.2.1
Nebula 星云业务风控系统.

# Nebula "星云"业务风控系统

## 系统说明
#### 该系统目前定位为解决以下业务风控核心问题。

  恶意注册（Account Abuse）  
  账号被盗（Account Takeover）  
  内容欺诈（Content Abuse）  
#### 系统架构：
  ![](image/system.png  )
#### 系统组件：
  ![](image/componement.png  )
#### 工作方式：
  ![](image/workflow.png  )

## 项目说明
#### 目录结构
```
  db/                     #mysql初始化表  
  fakedata/               #伪造的用户登录日志（测试用）  
  go/  
    src/  
      /malicious_prevent  
        /automate         #风险决策控制模块  
        /mpevents         #业务接口分发模块  
        /rulengine        #数据计算模块：风险规则+风险评分  
        ...               #数据处理后台主逻辑  
  management/             #运营管理系统（flask+nginx，前后端分离）  
    app/                  #flask接口  
    config/               #nginx配置示例  
    loginapp/             #登录系统  
    static/               #html、js、css
    debug.py              
```

#### 项目部署
* 部署mysql、mongodb、redis
* mysql db初始化（db模块）， monodb日志数据生成（fakedata模块）
* gulp打包界面模块
* 部署nginx
* 启动运营管理平台python management/debug.py，配置风控决策
* \malicious_prevent：go test

## TODO
* 风险规则因子提取、风险规则制定
* 风险等级评分逻辑
* 管理平台完善