semvec/pypi-packages huggingface dataset:
pyquicksql 2.2
A light-weight (out-of-the box) tool for pushing SQL (MySQL) queries, a markup-language for structured txt files and running data loggers in python.
License: MIT
Keywords: MYSQL,SQLITE,LOGGING

# Python Quick SQL 
*A light-weight (out-of-the box) tool for pushing SQL (MySQL and SQLite) queries, markup-language for structured txt files and running data loggers in python.*

	pip install pyquicksql==1.1.1

## SQL Queries
Designed as a purely light-weight and object-oriented aproach to sending MySQL queries to a database server from python. Current modules offer partialy oo aproaches, resulting in over-complicated syntax for simple sql queries. The goal is that this module will offer ease-of-use in comparison to other modules while providing faster development time. 

### Supported Databases
	
1. __MySQL__ (v.7.0)

### Functionality

#### Constructors
	server = Server('0.0.0.0.0', 800, username, password)
	database = Database('users', 'credentials', True)
	connector = Connect(server, database)

Lookup
> result = connector.lookup(unknown_column, known_colum, known_element)

Push
> result = connector.push(columns, elements)

Swap
> result = connector.swap(unknown_data, known_data)

Remove
> result = connector.remove(column, element)

## Logger
The logger offers developers who wish to log all sql traffic localy within their project OR for those who do not wish to overcomplicate their projects with SQL queries and use standard txt files. The goal is to offer a basic but efficient markup language which mimics relational tables found within MySQL for faster lookup times for values. This is (yet another) out-of-the-box module the package promises, to speed up development time by offering individuals who do not know how to make their own relational markup language.

### Functionality

#### Constructor

	l = Logger(directory) <- directory must be an active file path
	
Retreive Logs
> backup = l.getLogs()

Log
> l.log(Server, Database, Message)

Index
> value = l.index(0)

Lookup
> value = l.lookup(column, element, starting)

#### Data Markup
The 'Data Markup' is a markup-language designed with Rust and interface with python to offer the speed that it cannot. The language offers relational column-element formating for standard txt files for faster data lookup and retreival.

Syntax
> <Server<>Database>?Time?!Message!

Parse
> value = l.parse(line, Request)

Request Types
1. Column.SERVER
2. Column.DATABASE
3. Column.TIMESTAMP
4. Column.MESSAGE
	
## Credits

pyquickdb
> Gabriel Cordovado

	Functionality of all classes are not limited to this README, I encourage your to view the source
airspider 1.0.0
A Fast Python distributed async web crawler framework based on Redis
License: MIT
Requires Python: >=3.6

<p align="center">
    <img src="https://picreso.oss-cn-beijing.aliyuncs.com/air.png" width="150px">
 </p>
<p align="center">
    <img src="https://picreso.oss-cn-beijing.aliyuncs.com/fonts.png" />
</p>
<p align="center" style="font-weight:bold">
    AirSpider��️, a Python distributed async web crawler framework based on redis��️
  <br>
 		Distributed��  -  Asynchronous��  -  Light☁️  -  Fast⚡️ - Easy��
  <br>
	</p>




## Introduction

> + AirSpider是一款面向开发者的一个高性能异步爬虫框架
> + AirSpider模块之间耦合性低，内聚性高，方便扩展，并且工作流有条不紊
> + 基于Redis进行任务分发，任务去重，并且实现分布式。
> + 作为一个定制化爬虫框架，用户只需要编写指定模块，便可以开始高性能的爬虫任务
>

## Document

> ### Unfinished yet

### Basic Architecture：

![](https://picreso.oss-cn-beijing.aliyuncs.com/basis.png)

#### AirSpider共有6大模块，三小模块：

#### 6大模块：

1. 核心模块：Core：
   + 作为核心引擎，连接了其他几大模块，进行工作，工作流以它为中枢进行
2. 解析模块：Spider：
   + 作为解析模块，负责对Response进行解析，即对爬取回来的页面，进行指定数据的解析
3. 下载模块：Downloader
   + 作为下载模块，负责将Request，进行网络请求，返回指定URL的HTML内容
4. 管道模块：Piplines
   + 作为管道模块，负责将解析模块返回的指定数据进行封装，保存，连接数据库
5. 调度模块：Scheduler
   + 作为调度模块，负责与Redis交互，发送给Redis带下载的Request，Redis将调度好的Request返回给调度模块
6. NLP模块：NLP processor
   + 作为附加的模块，主要是对下游数据进行分析，挖掘

#### 3小模块：

##### 三个中间件：

+ Spider MiddleWare： 负责对解析模块进行额外处理
+ Downloader MiddleWare： 负责下载模块的反爬虫策略处理
+ NLP MiddleWare：负责NLP模块的策略处理

### Distributed Architecture：

### Redis作为Master，进行任务的分发，任务去重，各个AirSpider作为Slaver，分别工作如下

![](https://picreso.oss-cn-beijing.aliyuncs.com/cloud.png)

## WorkFlow：
![](https://picreso.oss-cn-beijing.aliyuncs.com/flow.png)

1. Spiders将初始URL指定，封装成Request

2. Core接受到Request，转发给Scheduler

3. Scheduler接收到Request，发给Redis

4. Redis收到Request，转发给Scheduler

5. Scheduler收到Request转发给Core

6. Core接收到Request，转发给Downloader

7. Downloader将Request进行网络爬取指定URL，返回数据封装成Response

8. Core接受到Response，转发给Spiders

9. Spiders将Response里的内容进行解析获取指定内容，封装成Items转发给Piplines 并且将里面符合规定的URLs，封装成Requests，发给Core

10. Core接受到Requests转发给Scheduler

11. Scheduler将Requests转发给Redis

12. Redis进行去重处理，分发任务Request 发给Core

13. Core接收到Request转发给Downloader

    然后重复到第7步

    一直到Redis所有任务都爬取完

14. 对Piplines里面的数据进行持久化处理

## Announcement

> AirSpider V 1.0 Relesed����������������

### Lot of things need to do, sorry about everything!
nluanalizer 1.0.2
Natural Language Tool to Extract Features from Intents Dataset.
License: MIT
Requires Python: >=3
Keywords: nlu natural language spanish

============
NLU Analyzer
============
---------------------------------------------------------------
Natural Language Tool to Extract Features from Intents Dataset.
---------------------------------------------------------------

Introduction
============

This package will help the user to extract ngrams, nouns, verbs,
adverbs and more features from an Intent Dataset in Spanish. This
will help the user to balance the dataset by comparing number of
ocurrences of each feature in each intent.

The entry format is an excel (.xlsx). Each column would be an intent
and in the first row it has to be informed the intent name, then
in the next rows the sentences for each intent training.

The output format is a json. It will have one dictionary per intent
including all the features extracted.

How to use it?
==============

Installation
------------
.. sourcecode::

    pip install nluanalyzer

Usage
-----
Usage

.. sourcecode::

    nluanalyzer -h|--help
    nluanalyzer -v|--version
    nluanalyzer <excel_path>


Options

.. sourcecode::

    -h --help                    Show help screen.
    -v --version                 Show version.

Example
-------
.. sourcecode::

    $ nlu_analyzer  intents.xlsx
    'Successfully saved intents analyzed file in /home/project/location/Formatted_Metrics.xlsx'

Package Requirements
====================

- nltk==3.4.5
- spacy==2.2.4
- docopt==0.6.2
- numpy==1.17.4
- docopt==0.6.2
- pandas==0.25.3
- textacy==0.10.0
- scikit_learn==0.23.1



History
-------

1.0.0 (2020-05-26)
------------------

* Spanish Analysis Implemented.
jsonobj 0.1.0
Wrapper module to provide a simple property accessor for JSON objects.
License: MIT
Requires Python: >=2.7, !=3.0.*
Keywords: json

# JSONobj

JSONobj is a Python module
for providing attribute-like access to the properties of JSON objects.


## Requirement

* Python 2.7 or above


## Installation

```sh
pip install jsonobj
```


## Basic Usage

```python
>>> import jsonobj
>>> obj = jsonobj.loads('{"prop1": "value"}')
>>> obj.prop1
'value'
>>> obj.prop1 = 'changed'
>>> obj.prop2 = ['foo', 'bar', {'fiz': 'buz'}]
>>> 'prop2' in obj
True
>>> obj.dict
{'prop1': 'changed', 'prop2': ['foo', 'bar', {'fiz': 'buz'}]}
>>> obj.prop2[2].fiz
'buz'
>>> del obj.prop1
>>> jsonobj.dumps(obj)
'{"prop2": ["foo", "bar", {"fiz": "buz"}]}'
```


## Documentation

Documentation is [available online](https://pyjsonobj.readthedocs.io/).
pdlog 0.1.0.post0
Logging for pandas dataframes
License: MIT
Requires Python: >=3.6.1

# pdlog

`pdlog` provides logging for [`pandas`](https://pandas.pydata.org/) dataframes, to better enable you to monitor and debug your data pipelines.

For example:

```pycon
>>> import pdlog
>>> df = df.log.dropna()
2020-05-26 20:55:30,049 INFO <pdlog> dropna: dropped 1 row (17%), 5 rows remaining
```

## Example

The above assumes that the [`logging`](https://docs.python.org/3/library/logging.html) module has been configured and that data has been loaded into a `pandas` `DataFrame`. Let's walk through those steps with a simple example.

1. Configure `logging`:

   ```pycon
   >>> import logging
   >>> fmt = "{asctime} {levelname} <{name}> {message}"
   >>> logging.basicConfig(format=fmt, style="{", level=logging.INFO)
   ```

2. Load data into a `pandas.DataFrame`:

   ```pycon
   >>> import pandas as pd
   >>> df = pd.DataFrame([0, 1, 2, None, 4])
   >>> df.head()
        0
   0  0.0
   1  1.0
   2  2.0
   3  NaN
   4  4.0
   ```

3. Importing `pdlog` and call a method under the `log` accessor:

   ```pycon
   >>> import pdlog
   >>> df = df.log.dropna()
   2020-05-26 20:55:30,049 INFO <pdlog> dropna: dropped 1 row (17%), 5 rows remaining
   ```

## Supported methods

`pdlog` currently supports the following `pandas.DataFrame` methods:

- Filter rows and select columns:
  - `drop_duplicates`
  - `drop`
  - `dropna`
  - `head`
  - `query`
  - `sample`
  - `tail`
- (Re-)set indexes:
  - `reset_index`
  - `set_index`
- Rename indexes:
  - `rename`
- Reshape:
  - `melt`
  - `pivot`
- Impute:
  - `bfill`
  - `ffill`
  - `fillna`

## Related Work

### [`pandas-log`](https://github.com/eyaltrabelsi/pandas-log)

`pandas-log` is aimed at interactive usage. Its messages are friendlier and more verbose than `pdlog` aims to be.
Ideally, each `pdlog` message should be a single line of dense information to help you understand whether your production code is doing what you think it is, while not overflowing your logs.
These don't tend to make particularly friendly messages.

### [`tidylog`](https://github.com/elbersb/tidylog)

`pdlog` can be considered a port of `tidylog` (R package) to `pandas`.
Their goals align with ours, and we think they've done a great job at reaching those goals.
jinjasql 0.1.8
Generate SQL Queries and Corresponding Bind Parameters using a Jinja2 Template
License: MIT
Keywords: Jinja2,SQL,Python,Template


Generate SQL Queries using a Jinja Template, without worrying about SQL Injection

JinjaSQL automatically binds parameters that are inserted into the template.
After JinjaSQL evaluates the template, you get 1) Query with placeholders
for parameters, and 2) List of values that need to be bound to the query. 

JinjaSQL doesn't actually execute the query - it only prepares the 
query and the bind parameters. You can execute the query using any 
database engine / driver you are working with.