semvec/pypi-packages huggingface dataset:
semanticmatcher 0.0.2
Semantic search of text and semanic matching of tables using FAISS
Requires Python: >=3.8

# semantic-matcher

[![Unit Tests](https://github.com/Aayushchou/semantic-matcher/actions/workflows/unit_tests.yml/badge.svg)](https://github.com/Aayushchou/semantic-matcher/actions/workflows/unit_tests.yml)

This library is built to handle anything related to semantic matching.

In its current state, it has two main uses:

* Find the closest matches of a user query to a text corpus, using sentence transformer encodings and FAISS for optimization.
* Measure the semantic similarity between two tables and determine common columns. Useful for detecting duplicates and determining which columns to join on.


## 1. Installation

This package can be installed after cloning by running "make install". Alternatively it can be installed using pip: 

```
pip install semanticmatcher
```

## 2. Usage: 

This package can be used to simply search for a query within a text corpus. 

```python
from semanticmatcher.search import semantic_search

query = ["boy jumping"]
corpus = ["There was a young man running around town", "The mayor is looking for a new house", "I had pasta for dinner"]

scores, indices = semantic_search(query, corpus)

```

Additionally, the package can also be used to determine the similarity between two tables. It returns a matrix that compares each column.

```python
df1 = pd.DataFrame(
        {
            "col1": ["hello", "world"], 
            "col2": ["how", "are"], 
            "col3": ["you", "doing"]
         }
    )
    
df2 = pd.DataFrame(
        {
            "col1": ["hola", "mundo"],
            "col2": ["como", "estas"],
            "col3": ["tu", "haciendo"],
        }
    )

similarity_matrix_df = similarity_matrix(df1, df2)
```

By default, the following sentence transformer model is used to encode the text: "all-MiniLM-L6-v2".

## 3. Next Steps

Add functionality to allow users to join two tables on a column, depending on the similarity match between the columns.
pyldcn 0.4
Python-native implementation of LdcnLib
Requires Python: >=3.7

# Python-native implementation of LdcnLib
This is a basic python-native implementation of LdcnLib. It implements some Ldcn and stepper methods. It solely relies on the package `pyserial`.

## Ldcn methods
The following Ldcn methods were implemented.


| Method             | Description                                     |
|--------------------|-------------------------------------------------|
| LdcnHardReset      | Resets all devices (broadcast)                  |
| LdcnSendCmd        | Send command to device                          |
| LdcnInit           | Initialize network using default group 0xFF     |
| LdcnShutdown       | Stop motor, delete devices (reinit necessary)   |
| LdcnGetDeviceId    | Returns `device, devid` for `addr`              |
| LdcnGetDevice      | Returns `device` for `addr`                     |
| LdcnGetStat        | Returns device status                           |
| LdcnGetStatItems   | Returns configured statusitems                  |
| LdcnReadStatus     | Reads local device status                       |


## Stepper methods
The following Stepper methods were implemented.


| Method             | Description                                     |
|--------------------|-------------------------------------------------|
| GetSteppers        | Returns addresses of discovered stepper devices |
| StepStopMotor      | Setup stop behaviour                            |
| StepSetParam       | Sets motor operation parameters                 |
| StepLoadTraj       | Set-up and start movement                       |
| StepResetPos       | Reset motor position to 0                       |

## Usage
To find out how the methods are used, either check LdcnLib documentation or example given in `demo.py`.

## Disclaimer
Usage at your *own risk*, the implementation is *neither complete nor well tested*.
span-to-ibo 0.1.0
This is a script to convert the output file of doccano to a format that is easy to handle with sklearn-crfsuite.
Requires Python: >=3.9,<4.0

# Span to IBO

This is a script to convert the output file of `doccano` 
to a format that is easy to handle with `sklearn-crfsuite`.

## Usage

```bash
python doccano.py --input_path <path to doccano exported jsonl file> --output_path <path to output file>
```

## Input file format

The input file is a jsonl file exported from `doccano`.

```json
{"text": "東京都渋谷区渋谷 ２丁目２−８ 渋谷マークシティ", "labels": [[0, 9, "LOC"]]}
{"text": "東京都渋谷区神南 １丁目１−１", "labels": [[0, 7, "LOC"]]}
...
```

## Output file format

The output file is a json file of the following format:

```json
[
    [
        {"word": "東京都", "label": "B-LOC", "pos_tag": "名詞", "pos_tag[:2]": "名詞,固有名詞", "pos_tag_all": "名詞,固有名詞,地域,一般,*,*,東京都,トウキョウト,トーキョート", "BOS": true, "EOS": false},
        {"word": "渋谷区", "label": "I-LOC", "pos_tag": "名詞", "pos_tag[:2]": "名詞,固有名詞", "pos_tag_all": "名詞,固有名詞,地域,一般,*,*,渋谷区,シブヤク,シブヤク", "BOS": false, "EOS": false},
        ...
    ],
    ...,
]
```

## Reference

This program is mainly based on the following repository.
https://github.com/ToshihikoSakai/jsontoconll

All mistakes in this script are mine.
auto-subtitles-generator 0.2.0
Auto subtitles generator for videos
Requires Python: >=3.11,<4.0

# Auto-subtitles video tool using AssemblyAI API and ffmpeg

## Requirements

You need an AssemblyAI API key. You can get one [here](https://app.assemblyai.com/login).
API key can be set using the `--api_key` option or by setting the `AAI_API_KEY` environment variable.

## Install

### Pip

```bash
pip3 install auto-subtitles-generator
```

### From source

```bash
git clone 
cd auto-subtitles-generator
poetry install
```

## Usage

```bash
Usage: auto_subtitles [OPTIONS]

Options:
  -i, --input PATH   Path to video file  [required]
  -o, --output PATH  Output video file  [required]
  --api_key TEXT     AssemblyAI API key
  --help             Show this message and exit.
```

## Build

```bash
poetry build
```

## Coverage

```bash
coverage run -m unittest && coverage report -m
```