Saravanan666/pypi-org huggingface dataset:
Package: automessage
automessageis a library that helps you quickly createprotorpc-based web services that interact
withndbmodels (Google Cloud Datastore) by automatically generatingMessageclasses for yourndb.Modelsubclasses, along with easy serialization/deserialization.CaveatsThis is pretty rough, alpha-level code. There are lots of TODOs. Use at your own risk.This only works for Google App Engine standard environment + Python with theprotorpcandndblibraries.InstallationFollow this guideto install automessage as a third-party library for your App Engine Python app; thepipcommand
you want is:pip install -t lib/ automessageUsageFirst, use a@automessage.attachdecorator on yourndb.Modelsubclass:fromgoogle.appengine.extimportndbimportautomessage@automessage.attach()classBook(ndb.Model):title=ndb.StringProperty()author=ndb.StringProperty()publish_date=ndb.DateTimeProperty(indexed=True)This generates a classBookMessage(a subclass ofprotorpc.messages.Message) in the same module
as theBookclass, that you can then use in yourprotorpc-based services, like so:classBooksService(remote.Service):classFindRequest(messages.Message):title=messages.StringField(1,required=True)@remote.method(FindRequest,BookMessage)deffind(self,request):return(Book.query(Book.title==request.title).to_message())# to_message() added by automessage@remote.method(BookMessage,BookMessage)defcreate(self,request):book=Book.from_message(request)# from_message() added by automessagebook.put()returnbook.to_message()attachtakes several parameters (see the code for details) that lets you customize the name of the
generated message class, convert to camel case, add an ID field, blacklist/whitelist properties,
etc. You can decorate models with multipleattachcalls (with different parameters) to create
multiple message types for a given model. When doing so, you'll need to provide the message type
into_messagecalls, e.g.book.to_message(CustomBookMessage).Related workProtopigeonis an almost identical, older approach with a slightly different API and better testing.
Package: autometric
====autometric====``autometric`` is to help draw ROC,PR curves for image semantic segmentation problem.When you have groundtruth image andpredict probabilistic image,this package can draw ROC,PR curves and save them to your savepath.It can also calculate FPR(False Positive Rate) TPR(True Positive Rate) AUC(Area under Curve) MAP(Mean Average Precision) IoU(Intersection over Union)at the same time.Usage-----``autometric`` include the following function/class:- ``CollectData``: a ``class`` to calculate FPR,TPR,AUC,MAP,IOU- ``DrawCurve``: a ``class``  to draw and save ROC PR curves- ``fileList``: a ``function`` to collect file to a list- ``drawCurve``: a ``function`` to draw curves and calculate parameters.Read example.py to get an idea of how to use!.. image:: url(https://i.imgur.com/ZCNAIFw.png):width: 500px:alt: Sphinx Neo-Hittite:target:
Package: automig
automigrateAutomigrate is a command-line tool for SQL migrations.
Unlike other migration tools, it uses git history to do diffs oncreate tablestatements instead of storing the migration history in a folder somewhere.How is this differentUsageAdvanced featuresHow is this differentThis tool uses git history to infer database migrations, and uses git SHAs to version production databases.Other schema migration tools typically work by diffing ORM definitions against a live database (which can be your local DB).
Often these diffs are then checked into a folder in your project repo.Seriously, all you have to do is maintain a file like this:-- schema.sqlcreatetablewhatever(useriduuidprimarykey,age_at_birthintdefault0);And when you add a field to the create table statement, automig figures out the 'alter table'.UsageIf you're using postgres:pipinstallautomig[postgres]# set an environment var with postgres connection detailsexportAUTOMIG_CON=postgresql://postgres:$PGPASSWORD@host# initialize the postgres DB to schema/*.sql -- do this once to create a DBautomig_pg--glob'schema/*.sql'init# update your DB to whatever sha is at git HEAD -- do this whenever your schema changesautomig_pg--glob'schema/*.sql'update# do a dry-run, show the output without applying itautomig_pg--preview--glob'schema/*.sql'updateIf you're using another database, you can get the raw SQL for these actions by using theautomigtool instead ofautomig_pg. (docs coming soon).Postgres is the primary database I test on, with sqlite support secondary.Advanced featuresInstructions for doing kube-native migrationsare in thekubefolder.Lambda support: you'll need to bundle a binary version of git that's compatible with amazonlinux. Post an issue if you need help with this.You can skip over bad diffs using--opaquemode, docs coming soonYou can specify manual overrides for erroring diffs, or skip over whole shas, by using.manualmig.yml file. docs coming soon but in the meantime look inside that file for an example.Convert an existing DB to use automig -- please post a github issue if you have this issue and I'll add instructionsGenerate ORM definitions from SQLExperimental sqlalchemy generator in sa_harness.py. Try it out with:python-mautomig.lib.sa_harness'test/schema/*.sql'
Package: auto-mix-prep
Performs operations on wave files in preperation for mixing.Current featuresGain staging, i.e. adjusting peak dB to desired levelBackup stems before editingSplit stereo files into separate mono tracks (L/R)Planned featuresInteractive usageRenaming filesAbility to exclude filesRequirementspython (>=3)pip3pydubInstallation$sudopip3installauto-mix-prepOr$gitclonehttps://github.com/dashj/auto-mix-prep.git.$cdauto-mix-prep$sudosetup.pyinstallOptions-g,--gainstage, peak dB desired (-12 dB if not specified).-s,--splitstereo, splits stereo files into two mono files.-d,--directory, directory containing wave files (working directory
if not specified).-f,--file, select individual file instead of all wave files in
directory.-b,--backup, creates a backup before overwriting (creates folder in
working directory if not specified).Usage$amp<operation>For example:$amp-s-g-24-d/path/to/stems-b/path/to/backup_directoryFirst creates a backup in/path/to/backup_directorythen searches/path/to/stemsfor wave files and performs the selected operations.
Which in this case would be splitting the stereo files into mono and
gain staging with -24 dB as the peak amplitude.
Package: autoMKL
autoMKLauto kernel selection and fusionFree software: Apache Software License 2.0Documentation:https://autoMKL.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.0.1 (2018-12-21)First release on PyPI.
Package: automl-alex
AutoML AlexState-of-the art Automated Machine Learning python library for Tabular DataWorks with Tasks:Binary ClassificationRegressionMulticlass Classification (in progress...)Benchmark ResultsThe bigger, the betterFromAutoML-BenchmarkSchemeFeaturesAutomated Data Clean (Auto Clean)AutomatedFeature Engineering(Auto FE)Smart Hyperparameter Optimization (HPO)Feature GenerationFeature SelectionModels SelectionCross ValidationOptimization Timelimit and EarlyStopingSave and Load (Predict new data)Installationpipinstallautoml-alexDocsDocPage🚀 ExamplesClassifier:fromautoml_aleximportAutoMLClassifiermodel=AutoMLClassifier()model.fit(X_train,y_train,timeout=600)predicts=model.predict(X_test)Regression:fromautoml_aleximportAutoMLRegressormodel=AutoMLRegressor()model.fit(X_train,y_train,timeout=600)predicts=model.predict(X_test)DataPrepare:fromautoml_aleximportDataPreparede=DataPrepare()X_train=de.fit_transform(X_train)X_test=de.transform(X_test)Simple Models Wrapper:fromautoml_aleximportLightGBMClassifiermodel=LightGBMClassifier()model.fit(X_train,y_train)predicts=model.predict_proba(X_test)model.opt(X_train,y_train,timeout=600,# optimization time in seconds,)predicts=model.predict_proba(X_test)More examples in the folder ./examples:01_Quick_Start.ipynb02_Data_Cleaning_and_Encoding_(DataPrepare).ipynb03_Models.ipynb04_ModelsReview.ipynb05_BestSingleModel.ipynbProduction Docker templateWhat's insideIt integrates many popular frameworks:scikit-learnXGBoostLightGBMCatBoostOptuna...Works with FeaturesCategorical FeaturesNumerical FeaturesBinary FeaturesTextDatetimeTimeseriesImageNoteWith a large dataset, a lot of memory is required!Library creates many new features. If you have a large dataset with a large number of features (more than 100), you may need a lot of memory.Realtime DashboardWorks withoptuna-dashboardRun$optuna-dashboardsqlite:///db.sqlite3Road MapFeature GenerationSave/Load and Predict on New SamplesAdvanced LoggingAdd opt PrunersDocs SiteDL EncodersAdd More libs (NNs)Multiclass ClassificationBuild pipelinesContactTelegram Group
Package: automl_gs
Give an input CSV file and a target field you want to predict to automl-gs, and get a trained high-performing machine learning or deep learning model plus native code pipelines allowing you to integrate that model into any prediction workflow. No black box: you can seeexactlyhow the data is processed, how the model is constructed, and you can make tweaks as necessary.automl-gs is an AutoML tool which, unlike Microsoft’s [NNI](https://github.com/Microsoft/nni), Uber’s [Ludwig](https://github.com/uber/ludwig), and [TPOT](https://github.com/EpistasisLab/tpot), offers azero code/model definition interfaceto getting an optimized model and data transformation pipeline in multiple popular ML/DL frameworks, with minimal Python dependencies (pandas + scikit-learn + your framework of choice). automl-gs is designed for citizen data scientists and engineers without a deep statistical background under the philosophy that you don’t need to know any modern data preprocessing and machine learning engineering techniques to create a powerful prediction workflow.Nowadays, the cost of computing many different models and hyperparameters is much lower than the oppertunity cost of an data scientist’s time. automl-gs is a Python 3 module designed to abstract away the common approaches to transforming tabular data, architecting machine learning/deep learning models, and performing random hyperparameter searches to identify the best-performing model. This allows data scientists and researchers to better utilize their time on model performance optimization.Generates native Python code; no platform lock-in, and no need to use automl-gs after the model script is created.Train model configurations super-fastfor freeusing aTPUin Google Colaboratory.Handles messy datasets that normally require manual intervention, such as datetime/categorical encoding and spaced/parathesized column names.Each part of the generated model pipeline is its own function w/ docstrings, making it much easier to integrate into production workflows.Extremely detailed metrics reporting for every trial stored in a tidy CSV, allowing you to identify and visualize model strengths and weaknesses.Correct serialization of data pipeline encoders on disk (i.e. no pickled Python objects!)Retrain the generated model on new data without making any code/pipeline changes.Quit the hyperparameter search at any time, as the results are saved after each trial.The models generated by automl-gs are intended to give a very strongbaselinefor solving a given problem; they’re not the end-all-be-all that often accompanies the AutoML hype, but the resulting code is easily tweakable to improve from the baseline.
Package: automl-helpers
automl-helpersautoml-helpers is a Python module for machine learning built on top of scikit-learn, numpy, pandas, and tf/keras designed to train various modeling pipelines to see what algorithms and pre-processing approaches work well for a given data set. The pipelines can either be saved for prediction purposes or just used as baselines for further refinement.User installationIf you already have a working installation of scikit-learn, numpy and tf/keras, the easiest way to install automl-helpers is using pip:pip install automl-helpers
Package: automlkiller
automlkillerAutomated Machine LearningUsageStep 1: Load data and Preprocessingmodel=AUTOML(X,y,cleancolumnname={},datatype={"categorical_columns":[],"numeric_columns":[],"time_columns":[]},simpleimputer={"numeric_strategy":"mean","categorical_strategy":"most_frequent"},zeronearzerovariance={"threshold_first":0.1,"threshold_second":20},categoryencoder={"cols":[],"method":"targetencoder"},groupsimilarfeature={"group_name":[],"list_of_group_feature":[]},binning={"features_to_discretize":[]},maketimefeature={"time_columns":[],"list_of_feature":['month','dayofweek','weekday','is_month_end','is_month_start','hour']},scaling={"method":"zscore","numeric_columns":[]},# outlier = {"methods": ["pca", "iforest", "knn"], "contamination": 0.2},removeperfectmulticollinearity={},makenonlinearfeature={"polynomial_columns":[],"degree":2,"interaction_only":False,"include_bias":False,"other_nonlinear_feature":["sin","cos","tan"]},# rfe = {"estimator": None, "step": 1, "min_features_to_select": 3, "cv": 3},# reducedimension = {"method": "pca_linear", "n_components": 0.99})Step 2: Training Modelmodel.create_model(estimator=['classification-lgbmclassifier',# 'classification-kneighborsclassifier','classification-logisticregression',# 'classification-xgbclassifier',# 'classification-catboostclassifier',# 'classification-randomforestclassifier'],verbose=True,n_jobs=2,cv=2,estimator_params={'classification-lgbmclassifier':{'n_jobs':8},},scoring=['accuracy','roc_auc','recall','precision','f1'])model.ensemble_model(scoring=['accuracy'])model.voting_model(scoring=['accuracy'])model.stacking_model(scoring=['accuracy'])Step 3: Model Performancemodel.report_tensorboard()
Package: automl-lib
使用方法，对外提供两个方法build_framework，export_saved_model'''-- coding: utf-8 --@Author : Zip@Time   : 2020/12/18|12:53@Moto   : Knowledge comes from decompositionfromfutureimport absolute_import
fromfutureimport division
fromfutureimport print_functionimport warnings
from lib.estimator import build_framework
from lib.export import export_saved_model
import argparsewarnings.filterwarnings('ignore')def main():
parser = argparse.ArgumentParser()
parser.add_argument("--config_file", default="conf/job_sort_with_fm_with_two_feature.yaml")
args = parser.parse_args()build_framework(args.config_file)
# export_saved_model(args.config_file)ifname== 'main':
main()'''
Package: auto-ml-openai-sdk
auto-ml-openai-sdkWhen merged to master, this package will be updloaded to Artylab. The versioning and auto docs are taken care of during the build process after being merged to master as well. After the package is published for the first time a branch called "docs" will be created. Github Pages can be set up by going to settings and selecting the branch "docs" and the folder "docs" under the Github Pages section.Build and Dev locallybash scripts/build_local.sh --enterThis command allows you to build and enter the docker image. All core folders are mounted so any changes made to your code will be reflected inside the image.Running Tests and checksbash scripts/qc_check.shThis command is to be ran inside the image.
Package: auto-mlops
Deploy your ML pipelines effortlessly, scalably and reliablyDatarmada aims atremoving all the friction that comes with Machine Learning in production.
We understand that Data Scientists are not trained to do that, and sometimes they are
not even attracted by this Software Engineering / DevOps aspect.This package aims at deploying your machine learning pipeline on a server in one line.Your pipeline is deployed on an OVH server so that you own your data and it is compliant with European regulations.InstallationInstall the package python using pippip install auto-mlopsDeploy your pipelineImport theDeployerclass from the package.fromauto_mlopsimportDeployerdeployer=Deployer()Now, deploy your pipeline by passing to thedeploymethod a list containing all of its elements.
The pipeline elements (except for the last one) must be either :A function returning transformed data if your pipeline element doesn't need to be fittedAn instance of a class implementing atransformmethodThe last element of the pipeline must be an instance of a class apredictmethods, such as a
scikit-learn or a Keras model.fromsklearn.linear_modelimportLogisticRegressiondefpreprocess(raw_data):# preprocess the datareturnpreprocessed_dataclassFeaturizer:deftransform(self,preprocessed_data):# transform the datareturnfeaturized_datalog_reg=LogisticRegression()log_reg.fit(featurized_data,y)deployer.deploy([preprocess,featurizer,log_reg])Remember your elements must be fitted if they need to !You will be asked for your email addressso that we can keep track of the ownership of the pipelines deployed, and give you
access to monitoring functions in the future.deployer.deploy([preprocess,featurizer,log_reg])>>Pleaseenteryouremailaddresssothatwecankeeptrackofyourpipelines:you@example.com>>Yourpipelinehasbeendeployedtohttps://cloud.datarmada.com/idYou can access your route whenever you want throughdeployer.routeMake predictionsYou can now send data to the route by making a POST request as followingimportrequestsres=requests.post("https://cloud.datarmada.com/id",json={"data":your_raw_data})print(res.json())>>{"prediction":prediction}It may be possible that one of the package you are using is not available in the environment we are deploying your model.
If you receive an error saying so, please email us atcontact@datarmada.comso that we can fix it.
Package: automl-sphinx-theme
AutoML Sphinx ThemeThe goal of this repository is to set up a documentation as fast and as clean as possible. In your
source project you only need to installautoml_sphinx_themeto get all the features you need.
Those include automatic API generation, beautiful theme, numpy docstring interpretation, and
executing/plotting examples.NO COMPLICATED.confOR SPHINX KNOWLEDGE ARE NECESSARY!:sparkling_heart: You're welcome. :sparkling_heart:InstallationIncludename,version,author,copyrightin{PACKAGE_NAME}/__init__.py.Copy thedocsandexamplesdirectories to your root folder. You don't have to include all
files but you should at least includedocs/conf.pyanddocs/Makefile.Indocs/conf.pychangesrcto{PACKAGE_NAME}. Adapt the file if needed.Install this repo via pypi:pip install automl_sphinx_themeYou can generate the docs viamake htmlinside the docs directory. Have a look inside theMakefileto get more information and commands.We recommend using the repositoryautoml_templatewhich incorporatesautoml_sphinx_themeautomatically.GithubThe documentation can be pushed automatically to a branch, which is used to display a webpage.
Follow the steps to enable it:Copy.github/workflows/docs.ymlto you root folder.Create a new branch calledgh-pages.Go tosettings > pagesand selectgh-pagesas source.Version SwitchingAutoML Sphinx Theme provides the user to link older versions of documentations (must be done manually!). Specifyversionsin the options to enable it. We recommend setting up a Github workflow which automatically pushes the latest tag to the gh-pages. From there, you could link the version directly.CustomizationLogo: Add docs/images/logo.png.Favicon: Add docs/images/favicon.ico.
Package: automl-streams
AutoML StreamsAn AutoML framework for implementing automated machine learning on data streams
architectures in production environments.InstallationFrompippipinstall-Uautoml-streamsorconda:condainstallautoml-streamsUsagefromskmultiflow.treesimportHoeffdingTreefromskmultiflow.evaluationimportEvaluatePrequentialfromautomlstreams.streamsimportKafkaStreamstream=KafkaStream(topic,bootstrap_servers=broker)stream.prepare_for_use()ht=HoeffdingTree()evaluator=EvaluatePrequential(show_plot=True,pretrain_size=200,max_samples=3000)evaluator.evaluate(stream=stream,model=[ht],model_names=['HT'])More demonstrations available in thedemosdirectory.DevelopmentCreate and activate avirtualenvfor the project:$virtualenv.venv
$source.venv/bin/activateInstall thedevelopmentdependencies:$pipinstall-e.Install the app in "development" mode:$pythonsetup.pydevelop
Package: automl-tools
Automl_tools: automl binary classificationAutoml_tools is a Python library that implements Gradient BoostingInstallationThe code is packaged for PyPI, so that the installation consists in running:pipinstallautoml-toolsColabUsageProbabilistic binary example on the Boston housing dataset:importpandasaspdfromautoml_toolsimportautoml_runtrain=pd.read_csv("https://raw.githubusercontent.com/jonaqp/automl_tools/main/automl_tools/examples/train.csv?token=AAN2ZBDWF77QITK4ARSFIFDABUGAU")test=pd.read_csv("https://raw.githubusercontent.com/jonaqp/automl_tools/main/automl_tools/examples/test.csv?token=AAN2ZBD6TMUC5XSGRTJNVPDABUGCO")automl_run(train=train,test=test,id_col=None,target_col="Survived",imp_num="knn",imp_cat="knn",processing="binding",mutual_information=False,correlation_drop=False,model_feature_selection=None,model_run="LR",augmentation=True,Stratified=True,cv=5)Parameterimp_num:"gaussian","arbitrary","median","mean","random","knn"imp_cat:"frequent","constant","rare","knn"processing:"woe","binding"Support Binarymodel_feature_selection:default:["LR","RF","LGB"]LR:LogisticRegressionRF:RandomForestClassifierSVM:SVCLS:LASSORD:RIDGENET:ElasticnetDT:DecisionTreeClassifierET:ExtraTreesClassifierGB:GradientBoostingClassifierAB:AdaBoostClassifierXGB:XGBClassifierLGB:LGBMClassifierCTB:CatBoostClassifierNGB:NGBClassifier

model_run:default:"LR"LR:LogisticRegressionRF:RandomForestClassifierSVM:SVCLS:LASSORD:RIDGENET:ElasticnetDT:DecisionTreeClassifierET:ExtraTreesClassifierGB:GradientBoostingClassifierAB:AdaBoostClassifierXGB:XGBClassifierLGB:LGBMClassifierCTB:CatBoostClassifierNGB:NGBClassifierLicenseApache License 2.0.New features v1.0multi_classregressionintegrations GCP deploy model CI/CDintegrations AWS deploy model CI/CDBugFix0.1.5fix imputerfix space hyperparameterupdate catboost test0.1.4add parameter cvadd confusion Matrixadd comments readme.txt0.1.3add parameter id_coladd comments readme.txtReferenceJonathan Quizagithub.Jonathan QuizaRumiMLSpark.Jonathan Quizalinkedin.
Package: automl-utils
AutoML UtilsWhile machine learning is facing areproducibility crisis, the problem
is exacerbated in the subdiscipline of automated machine learning [1,2] where the number of potential hyperparameters and variations in search and
training regimens can be vast.When combined with the duplication of code across projects, often with subtle differences in implementation, it can be
challenging to resolve whether changes in performance stem from improved methodology or from changes in configuration.This repository aims to facilitate these comparisons by providing reference implementations of commonly used components.
It is designed to be minimal and unopinionated to ensure maximum flexibility for the researcher.Note: This work is intentionally being released in an early state of development to enable use in other projects and
guide future efforts based on feedback received.