Saravanan666/pypi-org huggingface dataset:
Package: compressedlogger
CompressedLoggerThis is a logging handler to be used with the standardpython loggingmodule.
The handler creates a log file with the current logs and automatically moves old log files into zipped archives.
You can set how many uncompressed logfiles you want to hold and what timespan they should cover.
For example you can set each log file to cover three hours and keep four uncompressed logfiles.
When the limit of uncompressed logfiles is reached, the oldest uncompressed file will be
moved into a zipped archive file for that day. This way, all logentries for a single day are combined into one zipped file.
You can also set a maximum of days to keep the archive files as well as a general size limit for archives.
If either one of them is reached, the oldest archives are deleted.Behaviour:Using the following configuration:
.. code-block:: pythoncompressed_handler = compressedlogger.CompressedLogger(log_path=”logs/”,filename=”mylog”,
header=”- - version: 1.2.34 - - -“,
live_log_minutes=300,
live_log_count=3,
max_archive_size_mb=3,
archive_days=2)This will rotate the live log every 300 minutes. When started at 21.9.2020 - 10:33h, the first live log file will be named
mylog-10_33.log. The first rotation will not be at 15:33, but at 15:00. Log rotation timestamp is calculated from 0:00h and
not from the start of the application. Therefore, the next live log will be named mylog-15_00.log.
So there will be the log files:mylog-10_33.log,mylog-15:00.log,mylog-20:00.log. Because the live logs rotate on
day change, the next rotation will happen at 00:00h and not at 01:00h.
Once there are more livelogs than in themax_live_logsconfiguration, the latest log will be moved into a compressed archive for that day.
Theheaderwill be written to the top of every archived log. So in this example, there will be the log archivemylog2020-09-21.log.gzwhich will contain one logfilemylog2020-09-21.logwith the content of [header + mylog-10_33.log +  mylog-15_00.log + mylog-20_00.log].Usage:Parameters:log_path: where your logs will be storedfilename: name of you logfilelive_log_minutes: timespan that is covered by a single live-loglive_log_count: maximum number of live logs to keep uncompressedmax_archive_size_mb: maximum combined size of archived logs, in megabytearchive_days: maximum of days to keep log archivesheader: this header will be written on top of every archived logCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.3.4 (2020-10-23)when rotating a log file, read and write old logs chunkwise to lower RAM usage0.3.3 (2020-10-23)check for log rotation everytime a new log is opened0.3.2 (2020-10-08)timestamps as “10_00” instead of “10:00” for windows compatibility0.3.1 (2020-09-21)rework whole behaviour0.2.9 (2020-09-10)add new parameter ‘maximum_days’update readme0.2.8 (2020-07-28)add missing support of logging formattersfix the naming of logfiles if more than one handler writes into a log path0.2.5 (2020-07-15)updated project information and readme0.2.0 (2020-07-15)some improvements concerning the log paths and log file names0.1.0 (2020-07-14)First release on PyPI.
Package: compressed-segmentation
Compress SegLibrary for compressing and decompressing image segmentation (adapted fromneuroglancer)importcompressed_segmentationascsegsx,sy,sz=(128,128,128)dtype=np.uint64order='C'labels=np.arange(0,sx*sy*sz,dtype=dtype).reshape((sx,sy,sz),order=order)compressed=cseg.compress(labels,order=order)recovered=cseg.decompress(compressed,(sx,sy,sz)dtype=dtype,order=order)arr=CompressedSegmentationArray(compressed,shape=(sx,sy,sz),dtype=dtype)label=arr[54,32,103]# random access to single voxels w/o decompressinguniq_labels=arr.labels()# get all distinct values w/o decompressingbinary2=arr.remap({1:2},preserve_missing_labels=False)# remap labels in segmentation w/o decompressingrecovered=arr.numpy()# decompress to a numpy array, same as decompress124213inarr# test if a value is in the arraycsegcompressconnectomics.npy
csegdecompressconnectomics.npy.cseg--volume-size512,512,512--bytes4NOTE: This repository is the PyPI distribution repo but is based on work done by Jeremy Maitin-Shepard (Google), Stephen Plaza (Janelia Research Campus), and William Silversmith (Princeton) here:https://github.com/janelia-flyem/compressedsegThis library contains routined to decompress and compress segmentation and to manipulate compressed segmentation data defined by theneuroglancer project. compressed_segmentation essentially renumbers large bit width labels to smaller ones in chunks. This provides for large reductions in memory usage and higher compression.Note that limitations in the compressed_segmentation format restrict the size of the chunk that can be compressed. As this limitation is data dependent, for example a random array with 1024 labels passes testing at 256x256x128, but 256x256x256 often does not.FeaturesCompression and decompressionRandom access to voxels without decompressionRead out unique values without decompressionRemap labels without decompressionCommand line interface for numpy files(TBD) Interface to relabel and manipulate segmentation from the compressed dataC++, Python, and Go interface (see original repo for Golang)C++ CompilationCompiling as a shared library. Feel free to subsititute e.g. clang for the C++ compiler.g++-std=c++11-O3-fPIC-shared-I./includesrc/compress_segmentation.ccsrc/decompress_segmentation.cc-ocompress_segmentation.soPython InstallationpipBinary Installation$pipinstallcompressed-segmentation

$python
>>>importcompressed_segmentationascseg
>>>help(cseg)If there are pre-built binaries available for your architecture this should just work.pipSource InstallationIf you need to build from source, you will need to have a C++ compiler installed:$sudoapt-getinstallg++python3-dev$pipinstallnumpy
$pipinstallcompressed-segmentation

$python
>>>importcompressed_segmentationascseg
>>>help(cseg)Direct InstallationRequires a C++ compiler such as g++ or clang.Works with both Python 2 and 3. Encodes from / decodes to 3D or 4D numpy ndarrays.$sudoapt-getinstallg++python3-dev$pipinstall-rrequirements.txt
$pythonsetup.pyinstall

$python
>>>importcompressed_segmentationascseg
>>>help(cseg)LicensePlease see the licenses in this repo.
Package: compressfiles
GZip Compression UtilityThis is a Python package that provides a simple utility for compressing and decompressing files using the GZip format. The package uses thegzipmodule from the Python standard library.InstallationYou can install the package using pip:pip installcompressfilesUsageThe package provides two main functions:compress()anddecompress().Compressing a txt fileTo compress a file, use thecompress()function. For example:from compressfiles import compresscompress('my_file.txt')This will compressmy_file.txtand save the compressed file asmy_file.gz.You can also specify the output file path using theoutput_file_pathparameter:compress('my_file.txt','my_file.gz')
You can customize the compression level (0-9), default is '6'
compress('my_file.txt','my_file.gz', '8')Decompressing a fileTo decompress a file, use thedecompress()function. For example:from compressfiles import decompressdecompress('my_file.gz')This will decompressmy_file.gzand save the decompressed file asmy_file.txt.You can also specify the output file path using theoutput_file_pathparameter:decompress('my_file.gz', 'my_file.txt')LicenseThis project is licensed under the terms of the MIT license. See theLICENSEfile for details.
Package: compress-gpt
CompressGPTSelf-extracting GPT prompts for ~70% token savingsCheck out the accompanying blog posthere.Installation$pipinstallcompress-gptUsageSimply change your existing imports oflangchain.PromptTemplatetocompress_gpt.langchain.CompressTemplate(to compress prompts before populating variables) orcompress_gpt.langchain.CompressPrompt(to compress prompts after populating variables).-from langchain import PromptTemplate+from compress_gpt.langchain import CompressPrompt as PromptTemplateFor very simple prompts, useCompressSimplePromptandCompressSimpleTemplateinstead.If compression ever fails or results in extra tokens, the original prompt will be used. Each compression result is aggressively cached, but the first run can take a hot sec.Clearing the cacheimportcompress_gptcompress_gpt.clear_cache()DemoHow CompressGPT WorksMyblog posthelps explain the below image.
Package: compress-json
The missing Python utility to read and write large compressed JSONs.The library is loosely based on thecompress_picklelibrary.How do I install this package?As usual, just download it using pip:pipinstallcompress_jsonAvailable compression modesThe compression modes, detected automatically by the file name, aregzip,bz2andlzma,
with the notable exception ofzipwhich seems difficult to integrate in the JSON pipeline.Usage exampleThe library is extremely easy to use:importcompress_jsonD={"A":{"B":"C"}}compress_json.dump(D,"filepath.json.gz")# for a gzip filecompress_json.dump(D,"filepath.json.bz")# for a bz2 filecompress_json.dump(D,"filepath.json.lzma")# for a lzma fileD1=compress_json.load("filepath.json.gz")# for loading a gzip fileD2=compress_json.load("filepath.json.bz")# for loading a bz2 fileD3=compress_json.load("filepath.json.lzma")# for loading a lzma fileSome extra perks: local loading and dumpingThe library makes available, other than the usual load and dump from the JSON library, the methods local_load and local_dump, which let you load and dump file in the same directory of wherever you are calling them, by using the call stack.This can get useful, especially when loading files within packages.importcompress_jsonD={"A":{"B":"C"}}compress_json.local_dump(D,"filepath.json.gz")# for a gzip filecompress_json.local_dump(D,"filepath.json.bz")# for a bz2 filecompress_json.local_dump(D,"filepath.json.lzma")# for a lzma fileD1=compress_json.local_load("filepath.json.gz")# for loading a gzip fileD2=compress_json.local_load("filepath.json.bz")# for loading a bz2 fileD3=compress_json.local_load("filepath.json.lzma")# for loading a lzma fileLoading with RAM cacheSometimes you need to load a compressed JSON file a LOT of times, and you may want to
put this document in a cache or something of the sorts. Fortunately, we already provide
this option for you:importcompress_jsonD1=compress_json.load("filepath.json.gz",use_cache=True)D1=compress_json.local_load("filepath.json.gz",use_cache=True)Advanced usageYou can pass parameters to either the chosen compression mode or the JSON library.With thejson_kwargsparameter you can specify any of the kwargs that should
be forwarded to the JSON library method, which you can obtain for your Python version
by runninghelp(json.dump)andhelp(json.load), depending whether you are
dumping or loading the json object.Analogously, with thecompression_kwargsparameter you can specify any parameter that
has to be forwarded to the compression library that you intend to use, whether that islzma,gziporbz2, and as per JSON will depend on which version you have installed.Whether you are dumping or loading a compressed JSON object, you can get the list of parameters you
have available to forward to the compression method by runninghelp(lzma.open),help(gzip.open)orhelp(bz2.open), respectively.importcompress_jsonD={"A":{"B":"C"}}compress_json.dump(D,"filepath.json.gz",compression_kwargs={kwargsgohere},json_kwargs={kwargsgohere})D4=compress_json.load("filepath.json.gz",compression_kwargs={kwargsgohere},json_kwargs={kwargsgohere})
Package: compressor
PyCompressor======================PyCompressor collect all css and js references in html files, merge and compress then, puting all static files in an specific bucket, filtering all js,css and images references in html to apply your bucket url.Installation-----------------Installing: `pip install compressor`Usage=======compressor -c compressor.yaml --colorcompressor.yaml-----------------# your files pathpath:template: templatecss: media/cssjs: media/js# compressor results pathoutput:template: compressor/templatecss: compressor/media/cssjs: compressor/media/js# pattern to find tags in html codepattern:css: ../media/cssjs: ../media/jsimg: ../media/img# filter url tagsfilter:url:css: http://demo.pycompressor.com/media/cssjs: http://demo.pycompressor.com/media/jsimg: http://demo.pycompressor.com/media/imgRequirements------------The following python libraries are required* PyYAMLIssues------Please report any issues via [github issues](https://github.com/marcelnicolay/pycompressor/issues)
Package: compress-pickle
compress_pickleStandard python pickle, thinly wrapped with standard compression librariesThe standardpickle packageprovides an excellent default tool for serializing arbitrary python objects and storing them to disk. Standard python also includes broad set ofdata compression packages.compress_pickleprovides an interface to the standardpickle.dump,pickle.load,pickle.dumpsandpickle.loadsfunctions, but wraps them in order to direct the serialized data through one of the standard compression packages. This way you can seemlessly serialize data to disk or to any file-like object in a compressed way.compress_picklesupports python >= 3.6. If you must support python 3.5, installcompress_pickle==v1.1.1.Supported compression protocols:gzipbz2lzmazipfileFurthermore,compress_picklesupports thelz4compression protocol, that isn't part of the standard python compression packages. This is provided as an optional extra requirement that can be installed as:pipinstallcompress_pickle[lz4]Please refer to thepackage's documentationfor more information
Package: compress_pyscripts
Examples:python scripts/yui_compress_files.py –compressor path/to/yuicompressor-2.4.8.jar –root path/to/stylesheets –files cssfile1.css stylesheetlib/cssfile2.css cssfile3.css# creates files: cssfile1.min.css, stylesheetlib/cssfile2.min.css, cssfile3.min.css using yuicompressor jar filepython scripts/compile_single_file.py –root path/to/javascripts –files jsfile1.js jsfile2.js jsfile3.js
–filename single_optimized_file.js# creates one file single_optimized_file.js using unix concat
Package: comprise
CompriseFeaturesTODORequirementsTODOInstallationYou can installCompriseviapipfromPyPI:$pipinstallcompriseUsagePlease see theCommand-line Referencefor details.ContributingContributions are very welcome.
To learn more, see theContributor Guide.LicenseDistributed under the terms of theMIT license,Compriseis free and open source software.IssuesIf you encounter any problems,
pleasefile an issuealong with a detailed description.CreditsThis project was generated from@cjolowicz'sHypermodern Python Cookiecuttertemplate.
Package: compromise-marian
What is this repoOne encoder and two decoders training on a parallel corpura of English and France from HuggingFace dataset.One decoder for translatingOne decoder for reconstructing the original sentenceCredit: This project will not be success if not for the API of HuggingFace team.How to installInstall the environment using environment.yml from this repomarian. We use miniconda3condacreate-fenvironment.ymlInstall this packagepipinstallcompromise-marian
Package: comps
Comps brings together many machine learning and statistical methods for
finding look alike entities to ease the selection and management of comparison
groups in observational studies and experimental design to support causal
analysis. It also provides a number of standardized aggregations that are
helpful for comparing different types of metrics between groups.Comps provides:Systematic API for applying and matching methods and assessing the similarity
of different groups of entities based on common characteristicsMethods for easily managing groups of entities across time to maintain the
similarity between groupsAll Comps package wheels distributed on PyPI are BSD 3-clause licensed.
Package: compsheet
Compare Microsoft Spreadsheet FilesLook for instances of plagiarism.Compare file meta data [default]File creation time stampFile modification time stampAuthor of file creationAuthor of last modification of fileCell values [optional]Non-formula stringsCell-by-cell valuesCell layout [optional]Check locations of filled/unfilled cellsSpreadsheet files must be of type.xlsxResults are saved to spreadsheet.Installation and SetupTo install runpip install --user compsheetfrom the terminal.To run typepython3-mcompsheetfollowed by the various inputs and flags. It is recommended that the user establish the aliascompsheetfor easy usage.
To do this, edit the file.bashrcfrom the home directory and add the linealiascompsheet='python3 -m compsheet'to the end. One can do this in their favourite editor or simply open a terminal and typeecho"alias compsheet='python3 -m compsheet'">>.bashrcinto the prompt and press enter.Some examples of usage:# basic usagepython3-mcompsheet# show help message with no aliascompsheet-h# show help message with alias set to "compsheet"compsheet--explain# print description of table headerscompsheet# compare all files in current directory# example compbining various switchescompsheet-v-n4-t-o"meta,exact,string"./dir# [-v]      verbose output: print names of files as they are compared (default: quiet)# [-n]      use x processors in multiprocessing                       (default: 1)# [-t]      print table summary to stdout                             (default: off)# [-o]      do additional optional comparisons                        (default: "meta")# [./dir]   path to directory                                         (default: ".")
Package: compspec
Compspec PythonA compspec (Composition spec) is a specification and model for comparing things. This means
that we take an abstract and simple approach to model complex systems as graphs (nodes
and relaionshps) and then can compare between graphs. Note that the library has been developed over time
to handle different kinds of speifications, including:Answer Set Programming (asp): read about the background, concepts, and design of this simple specification.Compatibility: of container images and applications to a host environment. This is supported by way of generating compatibility artifacts and other assets.The second is under development!See the ⭐️Documentation⭐️ for basic usage
and getting started.TODOsupport for packaging / pushing artifacts with orasLicenseHPCIC DevTools is distributed under the terms of the MIT license.
All new contributions must be made under this license.SeeLICENSE,COPYRIGHT, andNOTICEfor details.SPDX-License-Identifier: (MIT)LLNL-CODE- 842614
Package: comptests
[![CircleCI](https://circleci.com/gh/AndreaCensi/comptests.svg?style=shield)](https://circleci.com/gh/AndreaCensi/comptests)comptests=========Testing utilities built on top of [ConfTools][conftools], [CompMake][compmake]and [QuickApp][quickapp].Basic Usage-----------This is the usage for packages that register objects using [ConfTools][conftools].# Defining testsCall the function ``comptests_for_all`` to create a decorator:# get the library (ObjSpec)library_robots = get_conftools_robots()# Create a test decoratorfor_all_robots = comptests_for_all(library_robots)# Use the decorator to specify tests. Test functions# must take two arguments: id object and object itself@for_all_robotsdef check_robot_type(id_robot, robot):assert isinstance(robot, RobotInterface)You can also register tests for pairs:library_nuisances = get_conftools_nuisances()for_all_robot_nuisance_pairs = comptests_for_all_pairs(library_robots, library_nuisances)@for_all_robot_nuisance_pairsdef check_nuisances_obs(id_robot, robot, id_nuisance, nuisance):check_conversions(robot.get_spec().get_observations(), nuisance)# Running testsUse the command line:comptests <module>This looks for the ``get_comptests()`` function in ``<module>``:def get_comptests():get_comptests():# get testing configuration directoryfrom pkg_resources import resource_filenamedirname = resource_filename("boot_agents", "configs")# load unittestsfrom . import unittestsfrom comptests import get_comptests_app# Get the Quickapp for the boot_configapp = get_comptests_app(get_boot_config())return [app]Finding coverage information============================It's very usuful to use the ``coverage`` tool together with comptests.Install the ``coverage`` tool:pip install coverageIf ``coverage`` is installed then comptests automatically computesthe coverage information for nosetests execution.To do the coverage information for the comptests, run like this:coverage run =comptests -c "make recurse=1" <package>Note that you cannot use parallel testing (using ``parmake``) otherwisecoverage gets confused.This displays the results:coverage report -mThen create the HTML report:coverage html -d outdir
Package: compton
python-comptonAn abstract data-flow framework for quantitative trading, which decouples data initialization, data composition and data processing.InstallpipinstallcomptonUsagefromcomptonimport(Orchestrator,Provider,Reducer,Consumer)VectorWe call a tuple of hashable parameters as a vector which is used to identify a certain kind of data.fromenumimportEnumclassDataType(Enum):KLINE=1ORDER_BOOK=2classTimeSpan(Enum):DAY=1WEEK=2vector=(DataType.KLINE,TimeSpan.DAY)Orchestrator(reducers, loop=None)reducersList[Reducer]reducers to compose dataloop?Optional[EventLoop]The event loop object to use. In most cases, you shouldNOTpass this argument, unless you exact know what you are doing.Orchestrator(reducers).connect(provider).subscribe(consumer).add('TSLA')orchestrator.connect(provider: Provider) -> selfConnects to a data providerorchestrator.subscribe(consumer: Consumer) -> selfSubscribes the consumer to orchestrator.orchestrator.add(symbol: str) -> selfAdds a new symbol to orchestrator, and start the data flow forsymbolProviderProvideris an abstract class which provides initial data and data updates.A provider should be implemented to support many symbolsWe must inherit classProviderand implement some abstract method before use.@property vectorreturns anVectorasync def init()method returns the initial dataThere is an protected methodself.dispatch(symbol, payload)to set the payload updated, which should only be called in a coroutine, or aRuntimeErroris raised.classMyProvider(Provider):@propertydefvector(self):return(DataType.KLINE,TimeSpan.DAY)asyncdefinit(self,symbol):return{}ReducerAnother abstract class which handles data composition.Thereducer.vectorcould be a generic vector which applies partial match to other vectorsclassMyReducer(Reducer):@propertydefvector(self):# So, MyReducer support both# - (DataType.KLINE, TimeSpan.DAY)# - and (DataType.KLINE, TimeSpan.WEEK)return(DataType.KLINE,)defmerge(self,old,new):# `old` might be `None`, if `new` is the initial dataifoldisNone:# We could clean the initial datareturnclean(new)return{**old,**new}ConsumerA consumer could subscribes to more than one kind of data typesclassMyConsumer(Consumer):@propertydefvectors(self):# Subscribe to two kinds of data typesreturn[(DataType.KLINE,TimeSpan.DAY),(DataType.KLINE,TimeSpan.WEEK)]@propertydefall(self)->bool:"""`True` indicates that the consumer will only go processingif both of the data corresponds with the two vectors have changesAnd by default, `Consumer::all` is False"""returnTrue@propertydefconcurrency(self)->int:"""Concurrency limit for method `process()`By default, `Consumer::concurrency` is `0` which means no limit"""return1defshould_process(self,*payloads)->bool:"""If this method returns `False`, then the data update will not be processed"""returnTrue# Then there will be# both `kline_day` and `kline_week` passed into method `process`asyncdefprocess(self,symbol,kline_day,kline_week):awaitdoSomething(symbol,kline_day,kline_week)LicenseMIT