semvec/pypi-packages huggingface dataset:
cjeReportingTool 2.0.1
This Tool made for CJE(Chishiki Joho Enshu 3) class at Univ. of Tsukuba, KLIS. This package converts source code comment to markdown text using split words.
License: MIT
Keywords: cjeReportingTool comment converter CJE University of Tsukuba KLIS

# cjeReportingTool

![example](./img/image.png "サンプル")

This tool made for CJE(Chishiki Joho Enshu 3) class at Univ. Tsukuba, klis.
This package converts source code comment to markdown text using split words.

> ja:
> 筑波大学 KLISのCJE3のマークダウン形式の小レポート用ライブラリです。
> 特定の文字列`##`や`#$`等をコメントに用いると，その記号を用いたコード行末に書いたコメントのみを行番号と合わせて出力します。

## Requirement

This package does not require on other any packages.

## Installation

This package runs on Python 3.6 or higher version. You can install it from PyPI via pip:

> ja:
> このパッケージはPython 3.6以上で動作します。
> またpipコマンドで以下のようにインストールできます。

```bash
pip install cjeReportingTool
```

to install.

## Usage

To use this tool as a package, load as a function.

```python
import cjeReportingTool.cjeReportingTool as c

c.cjeReportingTool(path, outpath, split_str, prefix)
```

Each args:

- `path` Read file path
- `outpath` Export file path
- `split_str` A symbol or string that separates comments from source code
- `prefix` Prefixes to be written out

To use this command as a command,

```bash
$ cjerep -h
usage: cjeReportingTool [-h] path outpath split_str prefix

This Tool made for CJE(Chishiki Joho Enshu 3) class at Univ. Tsukuba, klis.
This package converts source code comment to markdown text using split words.

positional arguments:
  path        Read file path
  outpath     Export file path
  split_str   A symbol or string that separates comments from source code
  prefix      A Prefix to be written out

optional arguments:
  -h, --help  show this help message and exit

$ cjerep sample.py out.md
```

## Example

- File

```text
.
├── sample.py
├── out.md
└── main.py
```

---

- Read file `sample.py`

```python
# num
i = 1 ##number
n = i * 2
# output
print(n) ##2
```

- Main program `main.py`

```text
import cjeReportingTool.cjeReportingTool as c

c.cjeReportingTool('sample.py', 'out.md', '##', '>')
```

---

- Output file  `out.md`

```text
1   > 2: number
2   
3   > 5: 2
```

![example](./img/preview_ex.png "サンプル")

### Note

`split_str` uses a string other than the spelling of the symbols used your python code.

- Good: `##`, `#&`, `#$`, `#%`
- Bad: `#` (No difference from other comments), `$` (That's not comment)

## License

This software is released under the MIT License, see LICENSE.
show all <https://github.com/murataka9/cjeReportingTool/blob/main/LICENSE>
watch-The-Marksman-2021-online-free-123movies-HD 0.2
Watch The Marksman Online Full Movie For HD
License: MIT
Keywords: Full Movie,Watch online,Putlocker

The Marksman 2020 Full Online Free : Click Homepage To Watch Or Download The Marksman 2021 full HD

- Below To Watch The Marksman 2020 Full Movie HD

- HERE FULL NOW-IN-HD >> https://t.co/Oydw70Ojvs

- HERE FULL NOW-IN-720p >> https://t.co/Oydw70Ojvs

- HERE FULL NOW-IN-1080p >> https://t.co/Oydw70Ojvs

- HERE FULL NOW-IN-FULL VERSION >> https://t.co/Oydw70Ojvs
dupecheck 0.7.1
Check for duplicate word series between generated text and a dataset
License: MIT

# Dupecheck

Analyze your generated text for plagiarizing your dataset

Use this to prevent ["Thomas Riker" situations](https://memory-alpha.fandom.com/wiki/Thomas_Riker), when generating text with machine learning systems.

At [Eclectic Beams](https://eclecticbeams.com/), when generating stories from finetuned GPT-2 models, I kept finding nice stories, but whole segments of them were just copied verbatim from the dataset text. I started battling this by using my editor's "find" feature on each line of generated text, or random subsequences against the dataset. It was time-consuming, and caused much eye pain and frustration, each time I'd read a story I liked, only to find out it had large sections copied verbatim.

Time to automate the tedious away.

Dupecheck will search for substrings of given length in a larger text dataset for you.
Use this to validate uniqueness before saving generated text, and you'll save yourself a huge headache.

It compares word series without punctuation.

```sh
pip install dupecheck
```

```py
from dupecheck import chunks, sliding_window, sliding_window2
```

## Chunking method
Find word series at least 5 words long in dataset. 
(Prone to mid-pattern window-splitting false negatives, since the window doesn't slide. Your pattern might get cut and split between two chunks, then not found even if it should match)
```py
duplicates = chunks.dupecheck(
              min=5,
              max=10, 
              text=my_gen_text, 
              dataset=my_dataset, 
              verbose=False
            )
```

## Sliding window `find`, try 1
- find word series at least 5 words long of your text string in dataset string
```py
duplicates = sliding_window.dupecheck(
              min=5, 
              max=10, 
              text=my_gen_text, 
              dataset=my_dataset,
            )
```

## Sliding window `find` try 2 
- find word series at least 5 words long of your text string in dataset string.
- cleans text and dataset for you if you don't provide cleaned and specify `cleaned=True`. 
- Cleaning is currently VERY slow.
```py
duplicates = sliding_window2.dupecheck(
              min=5, 
              max=10, 
              text=my_gen_text, 
              dataset=my_dataset,
              cleaned=False,
              verbose=False
            )
```

## Preprocessing helpers
```py
from dupecheck.preprocess import pre_process_text

dataset = pre_process_text(dataset)
text = pre_process_text(text)
```