Saravanan666/pypi-org huggingface dataset:
Package: prepextend
prepextendA toolset to expand the functionality for the Tableau PrepWhat are the Key Features?Run Tableau Prep file through Python API.Read the Tableau Prep file as a dict, which has an overview of all inputs and outputs.Produce a flows execution sequential list from a group of the Tableau prep files with dependence.Add-on features (require setting the config file):4.a. On top of feature 1, add retry when there is an error in running Tableau Prep file4.b. On top of feature 1, notify by Slack when running error.4.c. On top of feature 1, assign a specific version of Tableau Prep in a specific file to run.4.d. On top of feature 1, save running log.4.e. On top of feature 1, no more need to locate the credential file4.f. On top of feature 3, can assign a folder as a pool to the group of the Tableau prep files located.4.g. Can mark csv outputs as checkpoints and verify them after running file to secure the data quality.4.h. On top of feature 4-g, notify by Slack when getting issues in checkpoints.Please Note!Featurs 2 & 3 are in the limited scope of testing:Local file type: excel, csv, hyperNon-Local: Postgres DB, Snowflake, Tableau ServerIf your Tableau Prep files contain some connection out of the above scope, might get error or incorrect result.What's the Restriction?Currently, Windows OS support only.How to Install?Use the package managerpipto install.pipinstallprepextendHow to Use?1. Run Tableau Prep file# returns runnning logfromprepextendimportflow_runrunning_log=flow_run(prep_script="[Tableau Prep Builder install location]\Tableau Prep Builder <version>\scripts",flow_file="path\to\[your flow file name].tfl",credential_file="path\to\[your credential file name].json")# eg. prep_script = "...\Tableau\Tableau Prep Builder 2020.2\scripts"2. Read the Tableau Prep file# returns flow_infofromprepextendimportflow_readflow_info=flow_read("path\to\[your flow file name].tfl")3. Produce a flows execution sequential list with organized dependence.fromprepextendimportflows_roadmapdepend_flows=['...\flow_3.tfl','...\flow_1.tfl','...\flow_2.tfl']# dependency = flow 1 -> flow 2 -> flow 3target_flows=['...\flow_3.tfl',]flow_map=flows_roadmap(target_flows,depend_flows)# flow_map will be ['...\flow_1.tfl', '...\flow_2.tfl', '...\flow_3.tfl']target_flows=['...\flow_2.tfl',]flow_map=flows_roadmap(target_flows,depend_flows)#  flow_map will be ['...\flow_1.tfl', '...\flow_2.tfl']4. Set config for add-on featuresfromprepextendimportflow_manageflow_management=flow_manage(general_config,version_assigned)4.a ~ 4.e, 4.h Run flow with add-on featuresrunning_log=flow_management.run_flow("path\to\[your flow file name].tfl")4.f When produce flow_map, apply the assign a folder as a pool to the group of the Tableau prep files located without input depend_flowstarget_flows=['...\flow_3.tfl',]flow_map=flow_management.flows_roadmap(target_flows)
Package: pre-pip
Run some python code just before your pip commands.Use casesBefore installing a package, check it against a list of known malicious packages.Upgrade pip automatically before installing a package.Inject pip proxy settings into the environment before installing a package.You can use it to run any custom python code before a pip command is executed.Supported shellsThe following shells are currently supported:zshbashI'm currently working on adding support forpowershelland will thereafter look atfish.Contributions for any other shells are welcome.Installationpipinstallpre-pipThere is potential to make thispipxinstallable.UsageInstallpre-pipinto your.*rcfile using:pre-pipinstallRegister a custom demo hookCreate a new file calledhook.pyin your current directory with the following content:# hook.pyfromrichimportprintasrprintdefmain(args):rprint(f"This [italic green]pre-pip[/italic green] hook received: [italic cyan]{args}[/italic cyan]",)Register the hook using:pre-pipaddhook.pyYou can view the list of registered hooks using:pre-piplistUninstallUninstallpre-pipusing:pre-pipuninstallThis will remove thepre-piphook from your.*rcfile as well as all registered hooks.To remove the pre-pip package, use:pipuninstallpre-pipLicenseMIT
Package: prepl
Command line tool for rerunning a command when file dependencies change.Example usageprepl pytestwill automatically run your pytest tests as source files are saved.prepl -c 'make && ./a.out'will build and run an executable as you save source files.Descriptionprepl monitors files that were accessed by the command process and reruns it when any of the files change. prepl can be run  with either a single command or with a command string that is executed in a shell. The second form is useful when you want to run multiple commands (like build a project, then run some tests).usage: prepl [-h] [-c COMMAND_STRING] ...

Autorun command on file change.

positional arguments:
  COMMAND ...        the command to run and any arguments

optional arguments:
  -h, --help         show this help message and exit
  -c COMMAND_STRING  command string to run in shell (alternative to COMMAND ...)Installationprepl current supports Linux. macOS may be supported in the future. prepl requires Python 3.6+.You can install the latest release withpip3 install preplMotivationWhile developing software and writing tests it is typically advantageous to immediately see the result of code changes while they are being made. Some testing frameworks like jest rerun tests when source files are changed but they are limited to the languages that the testing framework supports. Other tools like inotifywait, fswatch, entr, etc. provide a generic interface for rerunning a command when files change, but these tools require the files to monitor to be explicitly specified. This can be cumbersome because in addition to specifying all the files to monitor, it is also typically necessary to exclude build directories and temporary backup files created by editors, for instance, that may inadvertently trigger the command to be rerun. prepl provides its functionality for any programming language without requiring an explicit list to files to monitor.Example Development Setup# Note: the repository's submodules must be cloned
git clone --recurse-submodules git@github.com:9pt7/prepl.git

# Setup a virtual environment...
python3 -m venv prepl
cd prepl
source ./bin/activate

# Install the package in editable mode with development dependencies
pip3 install --editable .[dev]

# Run tests
pytest prepl

# Test packaging with tox
tox
Package: prepnet
PrepnetReconstructable preprocessor library.There are concepts of this library.All pre-processes can save as a pickle.Reconstructable pre-processes for feature analysisExampleA simple example is see examples/01_iris.ipynb
There is pre-process using prepnet for iris dataset in a part of example.importprepnetfromsklearnimportdatasets# Load dataset.iris=datasets.load_iris()df=pd.DataFrame(iris.data,columns=iris.feature_names)df['target']=iris.target_names[iris.target]# Scale by std and mean, and split 5 folds.context=prepnet.FunctionalContext()withcontext.enter('normalize'):# All pre-process method allow method chain.context['sepal length (cm)','sepal width (cm)','petal length (cm)','petal width (cm)',].standardize()# context.post is execute always after other preprocesses.withcontext.enter('post'):context.split()# convert python list object from prepnet.DataFrameArray.preprocessed_df_list=list(context.encode(df))# Concat first 4 element for train dataset.train_df=pd.concat(preprocessed_df_list[:4],axis=0)# Use last element for test dataset.test_df=preprocessed_df_list[-1]And above preprocessor context can disable normalize easilynew_context=context.disable()preprocessed_df_list=list(context.encode(df))# Concat first 4 element for train datasetnonnorm_train_df=pd.concat(preprocessed_df_list[:4],axis=0)# Use last element for test datasetnonnorm_test_df=preprocessed_df_list[-1]Do you ever remember this?Boss: Hey, what's the difference between the new results and the old ones?Someone: Well, some preprocesses are different.Boss: Okay. Let me see the dataset.Someone: Yes, sir. It's this and this.Boss: What's the difference two datasets? The value that comes out of the difference is slightly, what's the difference in the preprocess?Someone: Well, I just don't know.Boss: Why? The dataset contains a commit idand you're managing source codes with git.Someone: Even if I knew what version of the dataset it was created from. I would have commented out the details and preprocessed it...Boss: Hey you...Installpipinstallprepnetorgitclonehttps://github.com/elda27/prepnetcdprepnet
pythonsetup.pyinstallTestpython-mpytest--cov=prepnet
Package: preposterous
Estimate the impact of an intervention, with simplicity and humilityPre/Posterous is under active developmentand the current release
can be considered a ‘proof of concept’. The largest restriction is that
it only imports data from theReporter
app, which is my current default
recommendation for anyone who is trying to track something for
Quantified Self.InstallationYou can install withpip install preposterousIf you want to install from source, then clone this repository and runpip install-e.from the project root.TestingTests can be seen in thetests/directory and run withpytestUse casesQuantified Self dataThe primary use case is for quantified self, where you have periodic
measurements of the target metric (weight, categorical sleep quality,
ect) and potential interventions (medications, diet shifts, ect). This
library can organize these into ‘natural experiments’ that point the way
towards a causal relationshipWarningPython is pretty great, but nothing can replace a well poweredDouble Blind Randomized Controlled
Trialfor
establishing causality. That said, many (most?) situations do not lend
themselves to RCTs, and yet we’re still forced to make decisions. That’s
where tools like this, used with an appreciation for non-binary modes of
belief, can be helpful.Exampleimport preposterous.preposterous as ppl
pdf = ppl.PrePostDF()
pdf.add_outcome(filename='data/sample_reporter_output.csv')
pdf.add_intervention(filename='data/sample_reporter_output.csv')

# Sanity check the data
pdf.basic_info()

# Basic statistical test of difference between periods pre and post intervention
pdf.fisher_test(intervention='Exercise')

# Bayesian comparison of relative impact of multiple interventions
# (note that the sample data only contains one)
# Output is written to an image file named 'relative_effectiveness_YYYYMMDD.png'
pdf.outcomes(
    positive_outcomes=['Totally fine'],
    negative_outcomes=['Noticeable', 'Distracting'],
    window=3
)
pdf.calculate_relative_effectiveness()
_ = pdf.plot_relative_effectiveness()
Package: preppipe
语涵编译器 (preppipe compiler)语涵编译器是一个读取 odt 文档并生成视觉小说引擎剧本的程序，目前仅支持以下导出形式：RenPy 工程目录纯文本发言信息，可以与 GameCreator 的对话批量导入工具衔接（目前只接受*.odt格式的文档，docx 等格式请先另存为 odt）语涵编译器本体是一个命令行程序，图形界面在这。请以以下命令形式运行本程序：python3 -m preppipe.pipeline_cmd <commands...>文档在写了在写了。（新建文件夹）QQ群：732421719
Package: preprocess
There are millions of templating systems out there (most of them
developed for the web). This isn’t one of those, though it does share
some basics: a markup syntax for templates that are processed to give
resultant text output.  The main difference withpreprocess.pyis
that its syntax is hidden in comments (whatever the syntax for comments
maybe in the target filetype) so that the file can still have valid
syntax. A comparison with the C preprocessor is more apt.preprocess.pyis targetted at build systems that deal with many
types of files. Languages for which it works include: C++, Python,
Perl, Tcl, XML, JavaScript, CSS, IDL, TeX, Fortran, PHP, Java, Shell
scripts (Bash, CSH, etc.) and C#. Preprocess is usable both as a
command line app and as a Python module.
Package: preprocess1
PreProcess1Super Easy Way of PreProcessing your Data!Good News!Preprocess1 simplifies the preprocessing steps that are some time essential for ML/modelling, such as imputations,
one hot encoding. There are over 20 preprocessing steps available. A summary of the options is below:Auto infer data typesImpute (simple or with surrogate columns)Ordinal EncoderDrop categorical variables that have zero variance or near-zero varianceClub categorical variables levels together as a new level (other_infrequent) that are rare / at the bottom 5% of the variabledistributionClub unseen levels in test dataset with most/least frequent levels in train datasetReduce high cardinality in categorical features using clustering or countsGenerate sub-features from time feature such as 'month','weekday',is_month_end','is_month_start' & 'hour'Group features by calculating min, max, mean, median & sd of similar featuresMake nonlinear features (polynomial, sin, cos & tan)Scales & Power Transform (zscore,minmax,yeo-johnson,quantile,maxabs,robust) , including option to transform target variableApply binning to variables when numeric features are provided as a listDetect & remove outliers using isolation forest, KNN and PCAApply clusters to segment entire dataOne Hot / Dummy encodingRemove special characters from column names such as commas, square brackets etc. to make it compatible with Jason dependent modelsFeature Selection through Random Forest, LightGBM and Pearson CorrelationFix multicollinearityFeature Interaction (DFS), multiply, divided, add and subtract featuresApply dimension reduction techniques such as pca_liner, pca_kernal, incremental or  Tsne.
except for pca_liner, all other methods only take the number of components (as integer)
i.e no variance explanation method availableYou can install the library aspipinstallpreprocess1frompreprocess1importtoolkitastAlthough one can use the methods individually (by calling the respective class) , such as:binn=t.Binning(['feature_tobin'])binned_data=binn.fit_transform(training_data)binned_new_data=binn.transform(test_data)However, there is more power to it. We have made pre-built complete pipelines to deploy all sorts of preprocessing transformers. Path1 is for supervised ML, and Path2 is for unsupervised ML problems. Below is how you use it:# apply the path to the training dataset while clubbing rare categorical levels & scaling numerical features# Imputation & One Hot Encoding is automatically applieddata_training_transformed=t.Preprocess_Path_One(training_data,'target_column',club_rare_levels=True,scale_data=True)# apply the pipeline to the test data setdata_test_transformed=pipe.fit_transform(test_data)You can find more information under the docstring of each class/function. Enjoy coding!
Please share your ideas, suggestions and critique with me.LicenseCopyright 2019-2020 Fahad Akbarfahad.akbar@gmail.com
Package: preprocess_cancellation
Klipper GCode Preprocessor for Object CancellationThis preprocessor modifies GCode files to add klipper's exclude object gcode.The following slicers are supported:SuperSlicerPrusaSlicerSlic3rCuraIdeaMakerInstallation and usageSuperSlicer, PrusaSlicer, and Slic3rDownload the provided binary for your platform, and place it in with in your slicer's folder.In your Print Settings, under Output Options, addpreprocess_cancellation.exe;to the
"Post-Processing Scripts". For mac or linux, you should just usepreprocess_cancellation;Then, all generated gcode should be automatically processed and rewritten to support cancellation.G-Codes for Object CancelationThere are 3 gcodes inserted in the files automatically, and 4 more used to control the
object cancellation.EXCLUDE_OBJECT_DEFINE NAME=<object name> [CENTER=x,y] [POLYGON=[[x,y],...]]The NAME must be unique and consistent throughout the file. CENTER is the center location
for the mesh, used to show on interfaces where and object being canceled is on the bed.
POLYGON is a series of points, used to represent the bounds of the object. It can be just
a bounding box, a simplified outline, or another useful shape.EXCLUDE_OBJECT_START NAME=<object name>andEXCLUDE_OBJECT_END [NAME=<object name>]The beginning and end markers for the gcode for a single object. When an object is excluded,
anything between these markers is ignored.For a full breakdown, seethe klipper G-Code ReferenceKnown LimitationsCura and Ideamaker sliced files have all support material as a single non-mesh entity.
This means that when canceling an object, it's support will still print. Including
support that is inside or built onto the canceled mesh. The Slic3r family (including
PrusaSlicer and SuperSlicer) treat support as part of the individual mesh's object,
so canceling a mesh cancels it's support as well.How does it workThis looks for known markers inside the GCode, specific to each slicer. It uses those
to figure out the printing object's name, and track's all extrusion moves within it's
print movements. Those are used to calculate a minimal bounding box for each mesh.
A series ofEXCLUDE_OBJECT_DEFINEgcodes are placed in a header, including the bounding boxes
and objects centers. Then, these markers are used to placeEXCLUDE_OBJECT_STARTandEXCLUDE_OBJECT_ENDgcodes in the file surrounding each set of extrusions for that object.
Package: preprocess-corpora
preprocess-corporaThis repository contains Python scripts to preprocess and sentence-align parallel (or monolingual) corpora.
The repository heavily relies uponUplugand (in lesser respect)TreeTaggerto work.InstallationFirst, make sure to have installedUplugandTreeTagger.Then, install the requirements via:$ pip install -r requirements.txtFinally, create the executablespreprocessandalignvia:$ pip install --editable .UsagePreprocessingThepreprocessscript allows to preprocess raw text and then to tokenize and tag the text in theXML format used in OPUS.Runpreprocessto process all unformatted .txt-files in a folder.Usage:process [OPTIONS] FOLDER_IN FOLDER_OUT [de|en|es|fr|it|nl|ru|ca|sv|pt]Options:--from_wordto use .docx-files as input, rather than .txt-files.--tokenizeto tokenize the files (requires installation of Uplug (and language support in Uplug)).--tagto tag the files (requires installation of TreeTagger (and language support in TreeTagger))AlignmentRunalignto sentence-align .xml-files in a working directory. Requires installation of Uplug.Usage:align [OPTIONS] WORKING_DIR [[de|en|es|fr|it|nl|ru|ca|sv|pt]]...Supported languagesFull supportGerman (de)English (en)Spanish (es) (+ variants Rioplatense (ar) and Mexican (mx) Spanish)French (fr)Italian (it)Dutch (nl)Russian (ru)Portuguese (pt)Limited supportBreton (br) [not supported in Uplug/TreeTagger]Catalan (ca) [not supported in Uplug/TreeTagger]Swedish (sv) [not supported in Uplug/TreeTagger]TestsRun the tests viapython -m unittest discoverInpreprocess_corpora/tests/data/alice, you can find the example corpus used in the tests.
This corpus was compiled from Lewis Carroll's Alice in Wonderland and its translations into German, French, and Italian.
The source files are available throughProject Gutenberg.
Package: preprocessdf-yapbarry
Hello WorldThis is an project that contains preprocessing techniques for numpy arrays and panda dataframes.InstallationRun the following to install:pipinstallpreprocessdf-yapbarryUsagefrompreprocessdfimportpreprocess# Pass your dataframe (aka df) into the preprocess classp=preprocess(df)# Do standard scaling on your dataframe valuesp.standard_scale()For developerspipinstall-e.[dev]
Package: preprocessing
‘preprocessing’SummaryText pre-processing package to aid in NLP package development for Python3. With this package you
can order text cleaning functions in the order you prefer rather than relying on the order of an
arbitrary NLP package.Installationpip:pip install preprocessingPyPI - You can also download the source distribution from:https://pypi.python.org/pypi/preprocessing/You can then perform:pip install <path_to_tar_file>on the tar file, orpython setup.py installon/inside, respectively, the extracted package to installpreprocessing.ExampleOnce you have the package installed, implementing it with Python3 takes the following form:importpreprocessing.textasptextfrompreprocessing.textimportkeyword_tokenize,remove_unbound_punct,remove_urlstext_string="important string at: http://example.com"clean_string=ptext.preprocess_text(text_string,[remove_urls,remove_unbound_punct,keyword_tokenize])>>> print(clean_string)
"important string"Should the functions be performed in a different order (i.e. keyword_tokenize -> remove_urls ->
remove_non_bound_punct) :>>> print(clean_string)
"important string http example.com"OrganisationThis package is comprised of a single module with no intended subpackages currently. Thepreprocessingpackage is dependent on NLTK for tokenizers and stopwords. However, ignoring this,
the package only has built-in dependencies from Python 3.ContributingIf you feel like contributing:Check for open issuesor open a new issueFork the preprocessing repository to start making your changesWrite a test which shows the bug was fixed or that the feature works as expectedSend a pull request and remember to add yourself toCONTRIBUTORS.mdLicenseThis project is licensed under the MIT license (seeLICENSE)
Package: preProcessingGEP
Usage of PreProcessingServiceYou can access following functions:The provided package is a collection of functions for text preprocessing and cleaning. Here is a brief explanation of each function:_en_stopwords: Returns a list of English stopwords.strip_numeric: Removes numbers from a string.strip_punctuation: Removes all punctuation marks from a string.remove_stopwords: Removes stopwords from a string.strip_multiple_whitespaces: Removes excess whitespace from a string.strip_short: Removes words with a length less than a specified size from a string.remove_alpha_num: Removes alphanumeric characters from a string.lemmatize: Lemmatizes a list of tokens.nltk_pos: Performs Part-of-Speech (POS) tagging using NLTK.stanford_pos: Performs POS tagging using Stanford POS Tagger.filter_pos: Filters words based on specific POS tags.clean_text: Cleans the text by applying various cleaning functions.preprocess_string: Preprocesses a string by applying a list of cleaning functions.preprocess_postags: Applies a list of filtering functions to a list of strings.guided_buy_preprocess: Preprocesses a list of strings for guided buying purposes.dump_interim_steps: Creates a DataFrame with the intermediate steps of preprocessing.lemmatization: Performs lemmatization on a string.cold_start_preprocess: Preprocesses a list of words for cold start purposes.unique_value_col: Splits words based on '|'.convert_synonyms:A collection of synonym mappingsclean_description: Includes all above cleaning stepsExample usage:pip install preProcessingGEP==1.0.4import preProcessingGEPprint(preProcessingGEP.PreProcessingService.clean_description(inputText))General Syntax to use any functionpreProcessingGEP.PreProcessingService.function_name()