Saravanan666/pypi-org huggingface dataset:
Package: docset
DocSet (Document Dataset)The project contains writer and reader utilities to store and extract documents (python dict) by a single ".ds" file as follow:graph LR
classDef nobg fill:none,stroke:none;
classDef box fill:none,stroke:#000;
ds[(person.ds)]:::box
writer[DocSetWriter]:::box
reader[DocSetReader]:::box
doc1("{'name': 'Alice', 'age': 17, ...}<br>{'name': 'Bob', 'age': 18, ...}<br>...<br>{'name': 'Tony', 'age': 20, ...}"):::nobg
doc1_("{'name': 'Alice', 'age': 17, ...}<br>{'name': 'Bob', 'age': 18, ...}<br>...<br>{'name': 'Tony', 'age': 20, ...}"):::nobg
doc1 --> writer --> ds
ds --> reader --> doc1_InstallationInstall from PyPI repository.pip3installdocsetTutorialHere, we give an example on a very simple machine learning task. Suppose we want to train a model to perform an image classification task, one important step is to construct a train dataset. To achieve this, we usually use a text file (e.g., csv, json) to store the "image path" and "label" information, and we store all the actual image files in another folder. This "text+folder" dataset can be fine in most of the situations, while it will suffer poor performance when the total number of samples is very large. The reason is that the file system is not good at reading/writing huge numbers of tiny files which are separately stored on the disk. So, storing the whole dataset in a single file is one possible way to solve this problem.The following examples first show how to make a DocSet file based on the "text+folder" storage, and then show how to read or iterate this dataset.Make a Datasetimportcsvimportosimportcv2ascvimportnumpyasnpfromdocsetimportDocSetcsv_path='train.csv'image_dir_path='images'ds_path='train.ds'withcsv.DictReader(csv_path)asreader,DocSet(ds_path,'w')asds:forrowinreader:image_path=os.path.join(image_dir_path,row['filename'])image=cv.imread(image_path,cv.IMREAD_COLOR)# load the image as ndarraydoc={'feature':image.astype(np.float32)/255.0,# convert the image into [0, 1] range'label':row['label']}# a data sample is represented by dict, ndarray can be used directlyds.write(doc)Read the DatasetfromdocsetimportDocSetds_path='train.ds'withDocSet(ds_path,'r')asds:print(len(ds),'samples')doc=ds[0]# the sample can be access by subscript, a dict will be returnedprint('feature shape',doc['feature'].shape)print('feature dtype',doc['feature'].dtype)print('label',doc['label'])View the Dataset in Consoledocset/path/to/the/file.dsIntegrate with Pytorchfromtorch.utils.dataimportDataset,DataLoaderfromdocsetimportDocSetclassMyDataset(Dataset):def__init__(self,path):self._impl=DocSet(path,'r')def__len__(self):returnlen(self._impl)def__getitem__(self,i:int):doc=self._impl[i]# perform any data processingreturndocds_path='train.ds'train_loader=DataLoader(MyDataset(ds_path),batch_size=256,shuffle=True,num_workers=5,pin_memory=True)forepochinrange(50):fordocintrain_loader:feature=doc['feature']label=doc['label']# invoke the train function of the model# model.train(feature, label)Technical DetailsTo be added...
Package: docs-from-tests
docs-from-testsThis python package contains functions to instrument your code / tests to create sequence diagrams (in markdown / mermaid) as the tests run.The sequence diagrams show the runtime call hierachy of the code. They are saved as files in the repository, which you should check in.You can choose to ignore specific functions, and private functions, to make diagrams readable in each context you are interested in.The diagrams are useful documentation, and will stay up to date as they are automatically generated. Hopefully it will encourage us to have useful diagrams in the repo, with the appropriate levels of detail, which in turn will encourage us to structure the code well so it produces good diagrams, and to  have tests at these levels of detail.You can see an example of this package in use atdocs-from-tests-example.Build StatusUsageYou can see an example of instrumenting a test attests\test_hello_world.pyThis test creates the following diagramdoc/end-to-end-sequence-diagram.mdThe diagram is reproduced here for convenience. It is a mermaid diagram, you may need to install aBrowser PluginorVsCode extensionor similar to view it.View full screen as SVGsequenceDiagram
  start->>HelloWorldCombiner.combine: calls x1
  HelloWorldCombiner.combine->>hello: calls x1
  hello-->>HelloWorldCombiner.combine: returns str
  HelloWorldCombiner.combine->>world: calls x1
  world-->>HelloWorldCombiner.combine: returns str
  HelloWorldCombiner.combine-->>start: returns strRunning testsuse python>=3.7pip install requirements.txtset pythonpath=<path to root of this repo>pytestContributingUsePEP-0008Please open up an Issue for new work, where any discussion can take place, and then submit a pull request to fix the issue.Project ChampionThe champion for this a repo isCedd Burgewho is accountable for:The usability of the repo, and ensuring user guides and READMEs are up to dateThe code quality of the repo (CODEOWNERS, pull requests, tests, and builds all help here)The usability of the code for other developers (retrospectives, code quality, architecture, and documentation all help here)
Package: docshund
d o c s h u n dsimple code documentation generationNOTE: Docshund is in alpha and may not yet be suitable for production workloads. Please exercise caution when using. (We'll try not to break everything if you promise not to expect it not to break everything.)WhyWe were tired of complex, long-running doc-generators. In some cases, we wanted to generate documentation for our source but didn't want to installallof our libraries' dependencies just to make a small change to a README. Docshund is the answer to this problem. Docshund doesn't require pip-installation of your package and, even for large projects, runs in just a handful of milliseconds.On the flip-side, Docshund does not have the same featurelist as many other libraries, and Docshund also has a hard time jumping up onto your couch without help.Installationpip3installdocshundUsagedocshundcode.py>documentation.mdOr document an entire module with:docshund.docshund/path/to/moduleWhat she do:Go from this:classFoo:"""This is a foo.Do not cross the foo streams!"""def__init__(self):"""Create a new foo."""defcross_streams(self,other_foo:Foo=None):"""Cross the streams of this foo with another foo.If you do this, it will throw an error.Arguments:other_foo (Foo: None): The foo with which you'd like to cross streamsReturns:NoneRaises:FooError: Raised if you cross the streams."""ifother_foo:raiseFooError("What did we JUST tell you?!")To this markdown:## *Class* `Foo`


This is a foo.

Do not cross the foo streams!


## *Function* `__init__(self)`


Create a new foo.


## *Function* `cross_streams(self, other_foo: Foo = None)`


Cross the streams of this foo with another foo.

If you do this, it will throw an error.

### Arguments
> - **other_foo** (`Foo`: `None`): The foo with which you'd like to cross streams

### Returns
    None

### Raises
> - **FooError** (`None`: `None`): Raised if you cross the streams.ClassFooThis is a foo.Do not cross the foo streams!Function__init__(self)Create a new foo.Functioncross_streams(self, other_foo: Foo = None)Cross the streams of this foo with another foo.If you do this, it will throw an error.Argumentsother_foo(Foo:None): The foo with which you'd like to cross streamsReturnsNoneRaisesFooError(None:None): Raised if you cross the streams.Generating documentation for this repositoryThe reference documentation for this repository was generated using the following command:docshunddocshund/__init__.py>docs/Reference.mdYou might say we... 😎 ate our own dogfood.LegalLicensed under Apache 2.0. Reach out toopensource@fitmango.comwith questions.Copyright 2019 FitMango.Licensed under the Apache License, Version 2.0 (the "License");
you may not use this codebase except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.(_______________()'`;
/,               /`
\\"-------------\\Made with ❤️ at🥭
Package: docSilhouette
# docSilhouette
:tada: docSilhouette## What is it?This library wraps pytesseract and adds some useful features for text processing. Objevtively it takes information from the bouding boxes issued by tesseract and it exctracts some coherent information from the text aesthetic, like page and document position for each text block.We also applied a greedy algorithm to organize the words in blocks, firstly processing lines, after that processing the groups of words as exposed by tesseract dataframe.## How to useYou’d rather install the library using pip:`shell pip install docSilhouette `Then you can use it:`python from docSilhouette.docSilhouette import docSilhouette doc =docSilhouette('./tests/assets/single_page.pdf')doc.setup() print(doc.get_text(True)) `You might find output like the following`shell xxP001 xxQ00_00 xxbob Universal Language ModelFine-tuningfor Text Classification xxeob xxQ00_03 `## Special TokensxxP001: Page numberxxbob: Begin of blockxxeob: End of blockxxQ01_00: Block number, where 01 refers to the first line of the page matrix and the 00 refers to the first column of the page. Check out the image bellow with a page with the matrix plotted on it. When set to issue quadrants, every block will have axxQfor the beginning of the block and another for the end of the block. The following example highlights the quadrant of the block1 Introduction, which starts at line 3 and column 0 and ends at line 3 and column 1. Refer to the image bellow for a more detailed example.`shell xxQ03_00 xxbob 1 Introduction xxeob xxQ03_01 `xxbcet: centralized text linexxecet: end of centralized text line![](imgs/2022-04-23-15-08-27.png)## License
MIT
Package: docsix
DocSixis a tool which helps you run yourdoctestsunder both
Python 2 and Python 3 (specifically 3.3 and later).DocSix works by stripping unicode indicators from expected test output
before running the test on Python 3.To use DocSix, simply pass it the list of files to execute tests on:>>> import unittest
>>> from docsix import get_doctest_suite
>>> suite = get_doctest_suite(['testdoc.rst'])
>>> unittest.TextTestRunner(verbosity=2).run(suite)
<unittest.runner.TextTestResult run=1 errors=0 failures=0>News0.1Release date: 17-Feb-2014Support for SphinxtestcodeblockSupport for stripping unicode indicators from output strings in
doctests.
Package: docspec
docspecThis Python packages providesA library to (de-) serialize Docspec conformat JSON payloadsA CLI to validate and introspect such payloadsExample:importdocspec,sysformoduleindocspec.load_modules(sys.stdin):module.members=[memberformemberinmodule.membersifmember.docstring]docspec.dump_module(sys.stdout)$ docspec module.json --dump-tree
module docspec
| class Location
| | data filename
| | data lineno
| class Decoration
| | data name
# ...ThedocspecPython module requires Python 3.5 or newer.Copyright © 2020, Niklas Rosenstein
Package: docspec-python
docspec-pythonA parser based onlib2to3procudingdocspecdata from Python source code.Example:from docspec_python import parse_python_module
import docspec, sys
docspec.dump_module(parse_python_module(sys.stdin, print_function=False), sys.stdout)$ docspec-python -p docspec | docspec --dump-tree --multiple | head
module __init__
| data __author__
| data __version__
| data __all__
| data _ClassProxy
| data _mapper
| class Location
| | data filename
| | data lineno
| class DecorationCopyright © 2020, Niklas Rosenstein
Package: doc-spell
doc spellCode documentation spell checker.Retrieve human text from the code and pipe it to spell checkerThe flow:collect all comments and doc stringsstrip technical text from commentsfenced block - multi-line text between 3 back ticks, similar to<pre>tagin-line code - text between back ticks, similar to<code>special comment - e.g.noqa,pylintfeed it to spell checkerInstallationpipinstalldoc-spell
Package: docsplice
DocSpliceDocstring splicing for python functions 🧬This project allows you to splice docstrings for python functions in a
convenient way using a functional decorator. This enables you to more easily
create consistent documentation for your API without needing to duplicate
docstring sections for functions that have the same parameter descriptions or
other related content. This will ease the maintenance burden for your package
documentation since the content of your function or class documentation reside
in a single location in the source code, instead of being duplicated within each
function body.InstallpipinstalldocspliceUsePulling parameters from multiple sourcesimportdocspliceasdocdeffun1(a):"""Do something with a number.Parameters----------a : intThe number."""deffun2(b=0):"""Another function that does something with an integer.Parameters----------b : int, optionalAnother number! By default 0."""@doc.splice({'Parameters[a]':fun1,'Parameters[b] as n=7':fun2})defcombined(a,n=7):"""Some profound computation. Parameter descriptions follow:"""TheParameterssection in the docstring forcombinedhas been created by thesplicedecorator. Inspecting the object in an interactive terminal reveals
that the docstring has been filled:combined??Signature: combined(a, n=7)
Docstring:
Some profound computation. Parameter descriptions follow:

Parameters
----------
a : int
    The number.
n : int, optional
    Another number! By default 7.
Source:   
@doc.splice({'Parameters[a]': fun1,
             'Parameters[b] as n=7': fun2}) 
def combined(a, n=7):
    """
    Some profound computation. Parameter descriptions follow:
    """TestThetest suitecontains further examples of howDocSplicecan be used.  Testing is done withpytest:pytestdocspliceContributeContributions are welcome!Fork it!Create your feature branchgit checkout -b feature/radCommit your changesgit commit -am 'Add something rad'Push to the branchgit push origin feature/radCreate a new Pull RequestContacte-mail:hannes@saao.ac.zaLicenseseeLICENSE
Package: docs-ranking-metrics
ОписаниеДанный репозиторий содержит реализацию алгоритмов ранжирования Bm25, LaBSE, MsMarcoST, MsMarcoCE
с подсчетом метрик:Top@1;Top@3;Top@5;Средняя позиция в выдачах (AverageLoc);Cредняя относительная позиция в выдачах (AverageRelLoc);Оценка как часто фейковый документ выше всех релевантных (FDARO@v1);Оценка как часто фейковый документ выше хотя бы одного релевантного (FDARO@v2);Частота попадания фейкового документа в топ 25% (UpQuartile).УстановкаДля установки пакета воспользуйтесь командойpip install docs-ranking-metricsПример использованияПример использования представлен вexamples/using_metrics.py# Объявление метрик
metrics = [LaBSE(), Bm25()]
# Объявление класса агрегирующего обновление метрик
rank_metrics = RankingMetrics(metrics)

...

'''
Обновление значений метрик, где 
query - запрос по которому сгенерирован документ, 
sentences - массив документов,
labels - метки документов
'''
rank_metrics.update(query, sentences, labels)

...
# Получение значений подсчитанных метрик ввиде словаря
rank_metrics.get()
# Получение значений метрик при помощи функции show_metrics
rank_metrics.show_metrics()Возможный вывод метода get:{
    'LaBSE_AverageLoc': 10.5, 
    'Bm25_AverageLoc': 1.13513, 
    'LaBSE_Top@1': 0.0, 
    'LaBSE_Top@3': 0.013513, 
    'LaBSE_Top@5': 0.013513, 
    'Bm25_Top@1': 0.91891, 
    'Bm25_Top@3': 1.0, 
    'Bm25_Top@5': 1.0, 
    'LaBSE_FDARO': 0.6216, 
    'Bm25_FDARO': 1.0
}Возможный вывод метода show_metrics():LaBSE_AverageLoc: 4.5   Bm25_AverageLoc: 3.0   
-----------------------------
LaBSE_AverageRelLoc: 0.75   Bm25_AverageRelLoc: 0.5   
-----------------------------
LaBSE_Top@1: 0.0   Bm25_Top@1: 0.5   
LaBSE_Top@3: 0.5   Bm25_Top@3: 0.5   
LaBSE_Top@5: 0.5   Bm25_Top@5: 1.0   
-----------------------------
LaBSE_FDARO@v1: 0.5   Bm25_FDARO@v1: 0.5   
LaBSE_FDARO@v2: 0.5   Bm25_FDARO@v2: 0.5   
-----------------------------
LaBSE_UpQuartile: 0.5   Bm25_UpQuartile: 0.5
Package: docstamp
docstampInitially it was a conference badge creator based on SVG templates,
but we thought it could be more generic and have many other applications.DocStamp is a generic template renderer which takes the data from a
.CSV file and creates one rendered template file for each row of the data.It is PDF centric, however it can also export in some cases to PNG.It needs:Inkscape or rsvg-convert for .SVG templates, andPDFLateX or XeLateX for LateX templates.CLI UsageThe CSV header fields must match the ones in the template file.docstampcreate-ibadge.csv-tbadge_template.svg-obadgesInstallationTo install the development version:python-mpipinstallgit+https://www.github.com/PythonSanSebastian/docstamp.gitTo install the latest release:python-mpipinstalldocstampRequirementsSeesetup.cfgfile. Also you will need Inkscape, XeLatex, or PDFLatex
installed in your system.For unicode support in SVG exports, you need to installrsvg-convert,
which is available inlibrsvg-bin.CompatibilityDocStamp is compatible with Python 2 (we wish to believe) and 3.We could not test it on Windows.AuthorsAlexandre M. Savio @alexsavioContributorsOier Etxaniz @oechanizLuis Javier Salvatierra @ljsalvatierraHaseeb Majid
Package: docs-translate
Docs TranslateCLI tool to translate.mdand.rstfiles from English to Korea and back.Can use Yandex Translation API and Google Cloud translation.InstallationInstall project:$pipinstalldocs-translateSettings fileYou can store your default settings in.jsonfile.Settings file content example:{
  "target_dir":"{D:\\language_convert_folder}",
  "reserved_words":"{D:\\reserved_words_json_file}",
  "source_lang": "ko",
  "target_lang": "en",
  "service_name": "Google_v2",
}If you set config file, you should specify it with-c CONFIG_PATHargument!Usage$docs-translate[-h][-cCONFIG_PATH][-s{Google,Google_v2}][-S][-T][-path{source_path}][-d{target_dir}]If you set config file, you can override any of settings by argumentsPositional arguments:pathPath to folder to process. If not set, uses current folder-d TARGET_DIR, --target_dir TARGET_DIR, If not set, the target_dir in config.json is setOptional arguments:-h, --help, show this help message and exit-c CONFIG_PATH, --config_path CONFIG_PATH, Path to config_file-s {Yandex,Google,Bing,Deepl}, --service_name {Yandex,Google,Bing,Deepl}, Translating service-S SOURCE_LANG, --source_lang SOURCE_LANG, Source language code-T TARGET_LANG, --target_lang TARGET_LANG, Target language codeTranslation services:Now usedYandex,Google,Bing,Deepl,Google_v2Some of them can be not working, try it and find the most suitable for youGoogle --> It's free, but the translation isn't good.Google_v2 --> need to google Cloud Translation API key.
Package: docstringargs
Python module to conveniently build an argparse config based on function docstrings.License: MITCopyright (c) 2011-2014, Chris AngelicoPermission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the “Software”), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Package: docstring-extractor
docstring-extractorGet Python docstrings from files or Python source code.Example usage:>>>fromdocstring_extractorimportget_docstrings>>>>>>withopen("example.py")asfile:...get_docstrings(file)...{'module':'example','content':[{'type':'Function','name':'my_fuction','line':4,'docstring':'Long description spanning multiple lines\n- First line\n- Second line\n- Third line\n\n:param name: description 1\n:param int priority: description 2\n:param str sender: description 3\n:raises ValueError: if name is invalid'}]}ContributingThis project usesBlack.
Package: docstring-parser
docstring_parserParse Python docstrings. Currently support ReST, Google, Numpydoc-style and
Epydoc docstrings.Example usage:>>>fromdocstring_parserimportparse>>>>>>>>>docstring=parse(...'''...     Short description......     Long description spanning multiple lines...     - First line...     - Second line...     - Third line......     :param name: description 1...     :param int priority: description 2...     :param str sender: description 3...     :raises ValueError: if name is invalid...     ''')>>>>>>docstring.long_description'Long description spanning multiple lines\n- First line\n- Second line\n- Third line'>>>docstring.params[1].arg_name'priority'>>>docstring.raises[0].type_name'ValueError'ReadAPI Documentation.ContributingTo set up the project:pipinstall--userpoetry

gitclonehttps://github.com/rr-/docstring_parser.gitcddocstring_parser

poetryinstall
poetryrunpre-commitinstallTo run tests:poetry run pytest
Package: docstring-parser-fork
docstring_parser_forkThis is a fork ofdocstring_parser. The reason I'm forking that is to quickly get some bug fixes out for users ofpydoclint.Parse Python docstrings. Currently support ReST, Google, Numpydoc-style and
Epydoc docstrings.Example usage:>>>fromdocstring_parserimportparse>>>>>>>>>docstring=parse(...'''...     Short description......     Long description spanning multiple lines...     - First line...     - Second line...     - Third line......     :param name: description 1...     :param int priority: description 2...     :param str sender: description 3...     :raises ValueError: if name is invalid...     ''')>>>>>>docstring.long_description'Long description spanning multiple lines\n- First line\n- Second line\n- Third line'>>>docstring.params[1].arg_name'priority'>>>docstring.raises[0].type_name'ValueError'ReadAPI Documentation.InstallationInstallation using pippipinstalldocstring_parser_fork# or if you want to install it in a virtual environmentpython-mvenvvenv# create environmentsourcevenv/bin/activate# activate environmentpython-mpipinstalldocstring_parser_forkInstallation using condaDownload and install miniconda or anacondaInstall the package from the conda-forge channel via:conda install -c conda-forge docstring_parseror create a new conda environment viaconda create -n my-new-environment -c conda-forge docstring_parserContributingTo set up the project:pipinstall--userpoetry

gitclonehttps://github.com/rr-/docstring_parser.gitcddocstring_parser

poetryinstall
poetryrunpre-commitinstallTo run tests:poetry run pytest
Package: docstringtest
docstringtest is a small package for regression testing of docstrings in Python code. It checks that all appropriate methods and functions are using ReST docstrings with parameter information so that nice documentation can be generated. This format looks like:defexample_function(varA,varB):"""
   This function does nothing

   :param varA: The first variable
   :param varB: The second variable
   :type varA: The type of the first variable
   :type varB: The type of the second variable
   """passIt makes it easy to check that the docstrings match with the current parameters so that documentation doesn’t become out-of-step with the code.If you wanted to add docstringtest to a standard test algorithm, it would simply be something like thisimportmymoduleimportdocstringtestdeftest_docstringtest():docstringtest.testModule(mymodule)InstallationYou can install “docstringtest” viapipfromPyPI:$ pip install docstringtestContributingContributions are very welcome.LicenseDistributed under the terms of theMITlicense, “docstringtest” is free and open source softwareIssuesIf you encounter any problems, pleasefile an issuealong with a detailed description.