Saravanan666/pypi-org huggingface dataset:
Package: mlscorer
ML ScorerML Scorer is the solution to your classification scores of ML algorithms.Installationpip install mlscorerPreperationMake a class mapping dictionary(map_class) usingMethod1orMethod2Method 1Make all the data categorical using following code snippetmap_class=dict(zip(df.classes.astype("category").cat.codes,df.classes))print(map_class)output:
{1: 'positive', 0: 'negative'}here, df is theDataframeandclassesis  a column which may have class values likepositivenegative[N.B.Don't change "category",  it's a datatype]orMethod 2Make the Dictionary manually according to your classesmap_class={1:'positive',0:'negative'}Usagefromsklearn.linear_modelimportLogisticRegressionimportmlscorerasmsclassifier=LogisticRegression()classifier.fit(X_train,y_train)y_pred=classifier.predict(X_test)ms.get_score_table(y_test,y_pred,map_class)Output:Parametersy_test: target values of test sety_pred: predicted target valuesmap_class:dict: your categoricl class mappingmetrics:list: use one or more evaluation metric from f1, precision, recall or accuracyeg:ms.get_score_table(y_test,y_pred,map_class,metrics=['precision','recall'])Output:
Package: mlscraper
Turn HTML intro structured data automatically with Machine Learning with mlscrapermlscraperallows you to extract structured data from HTML automatically with Machine Learning.
You train it by providing a few examples of your desired output.
It will then be able to extract this information from any new page you provide.Background StoryMany services for crawling and scraping automation allow you to select data in a browser and get JSON results in return.
No need to specify CSS selectors or anything else.I’ve been wondering for a long time why there’s no Open Source solution that does something like this.
So here’s my attempt at creating a python library to enable automatic scraping.All you have to do is define some examples of scraped data.autoscraperwill figure out everything else and return clean data.Currently, this is a proof of concept with a simplistic solution.How it worksAfter you’ve defined the data you want to scrape, mlscraper will:find your samples inside the HTML DOMdetermine which rules/methods to apply for extractionextract the data for you and return it in a dictionaryfrommlscraperimportMultiItemScraperfrommlscraper.trainingimportMultiItemPageSample# the items found on the training pageitems=[{"title":"One great result!","description":"Some description"},{"title":"Another great result!","description":"Another description"},{"title":"Result to be found","description":"Description to crawl"},]# training the scraper with the itemssample=MultiItemPageSample(html,items)scraper=MultiItemScraper.build([sample])scraper.scrape(html)# will produce the items abovescraper.scrape(new_html)# will apply the learned rules and extract new itemsGetting startedInstall the library locally viapip install -e ..
You can then import it viamlscraperand use it as shown in the examples.DevelopmentSeeCONTRIBUTING.rstRelated workIf you’re interested in the underlying research, I can highly recommend these publications:Learning to extract hierarchical information from semi-structured documents:http://ftp.cse.buffalo.edu/users/azhang/disc/disc01/cd1/out/papers/cikm/p250.pdfWHISK: Extraction of structured and unstructured information:https://www.cis.uni-muenchen.de/~yeong/Kurse/ws0809/WebDataMining/whisk.pdfI originally called this autoscraper but while working on it someone else released a library named exactly the same.
Check it out here: [autoscraper](https://github.com/alirezamika/autoscraper).History0.1.2 (2020-09-27)First release on PyPI.
Package: mlscratch
Machine Learning From ScratchAboutPractical implementations of common machine learning algorithms with test
cases coded from scratch in a similar style as Scikit-learn using
Object-Oriented Programming.This project is meant to increase our understanding of machine learning
algorithms and what is  'underneath the hood'. It is not meant for use
in actual data science work.InstallationTo use the classes and functions for testing purposes create a virtual
environment and pip install the project.$ pip install mlscratch==0.0.1To download all source code in a local repository from Github create a virtual
environment and run the following commands in your terminal.$ git clone https://github.com/lukenew2/mlscratch
    $ cd mlscratch
    $ python setup.py installLearning AlgorithmsSupervisedElastic NetLasso RegressionLinear RegressionLogistic RegressionPolynomial RegressionRidge RegressionSoftmax RegressionWrite-UpsML From Scratch: Linear, Polynomial, and Regularized Regression Models
Package: ml-scripts
Machine Learning Kit (mlkit)INSTALLpipinstallml_scriptsEXAMPLES1. Model Evaluate1.1 Classifier EvaluateAdd Datafromml_scripts.model_evaluateimportClassifierEvaluatepred_probs_vectors=[[1,3,2,7,0],[4,7,2,8,2],[1,3,7,4,9]]# normalization is not requiredreal_idxes=[2,1,3]ce=ClassifierEvaluate(pred_probs_vectors,real_idxes)more_pred_probs_vectors=[[7,5,3,6,1],[1,4,6,7,9]]more_real_idxes=[0,4]ce.extend_pred_and_real(more_pred_probs_vectors,more_real_idxes)Calculate Topk-Accuraciesce.get_topk_acc(1)# 0.4ce.get_topk_acc(2)# 0.6ce.get_topk_acc(3)# 1.0Calculate Topk-Precision/Recall/Accuracies for each classce.get_topk_tp_fn_fp_tn_for_idxes(1)# {0: (1, 0, 0, 4),#  1: (0, 1, 0, 4),#  2: (0, 1, 0, 4),#  3: (0, 1, 2, 2),#  4: (1, 0, 1, 3)}ce.get_topk_tp_fn_fp_tn_for_idxes(2)# {0: (1, 0, 0, 4),#  1: (1, 0, 1, 3),#  2: (0, 1, 1, 3),#  3: (0, 1, 4, 0),#  4: (1, 0, 1, 3)}Save All Results of Evaluationce.save_result(max_k=50,result_dp='[directory_path_to_save_result_of_evaluation]')
Package: mlserve
mlservemlserveturns your python models into RESTful API, serves web page with
form generated to match your input data.It may be useful if one wants to demonstrate created predictive model and
quickly integrate into existing application. Additionally UI is provided for
input data (based on training dataframe) and simple dashboard.Project is not complete but already usable, so no any guaranties on API or UI
backward compatibility.Online DemoSeveral models deployed online usingheroku.com/freefree dynos.
Free apps sleep automatically after 30 mins of inactivity so first request
may take some time.https://young-ridge-56019.herokuapp.comIdeasmlsserveis small using following design based on following ideas:Simplicity and ease of use is primary objective.Application consists of two processes: IO process that runs HTTP server
and responsible for fetching and sending data, as well as serve UI, other
process (worker) is doing CPU intensive work related to predictions
calculations.FeaturesModel predictions serving via RESTful API endpoint.Model predictions serving via generated UI.Web page to simplify models usage.Automatic UI generation to match your input data.Simple dashboard for monitoring purposes.InstallationInstallation process is simple, just:$ pip install git+https://github.com/jettify/mlserve.gitExampleTo deploy model just follow following simple steps:Save your model into pickle file:with open('boston_gbr.pkl', 'wb') as f:
    pickle.dump(clf, f)Usebuild_schemafunction to build UI representation of pandas dataframe,
and save it as json file file:import mlserve

data_schema = mlserve.build_schema(df)
with open('boston.json', 'wb') as f:
    json.dump(data_schema, f)Create configuration file with following format:models:
  - name: "boston_regressor"  # url friendly name
    description: "Boston GBR"  # optional model description
    model_path: "boston_gbr.pkl"  # path to your saved model
    data_schema_path: "boston.json"  # path to data representation
    target: "target"  # name of the target columnServe model:$ mlserve -c models.yamlThats it, model is available throw REST API, you can test is with curl command:$ curl --header "Content-Type: application/json" --request POST
--data '[{"feature1": 1, "feature2": 2}]'
http://127.0.0.1:9000/api/v1/models/boston_gradient_boosting_regressor/predictUI is available viahttp://127.0.0.1:9000Supported FrameworksScikit-LearnKeras (planning)PyTorch (planning)RequirementsPython3.6+aiohttpCHANGES
Package: mlserver-alibi-detect
Alibi-Detect runtime for MLServerThis package provides a MLServer runtime compatible withalibi-detectmodels.UsageYou can install themlserver-alibi-detectruntime, alongsidemlserver, as:pipinstallmlservermlserver-alibi-detectFor further information on how to use MLServer with Alibi-Detect, you can check
out thisworked out example.Content TypesIf nocontent typeis present on the
request or metadata, the Alibi-Detect runtime will try to decode the payload
as aNumPy Array.
To avoid this, either send a different content type explicitly, or define the
correct one as part of yourmodel's
metadata.SettingsThe Alibi Detect runtime exposes a couple setting flags which can be used to
customise how the runtime behaves.
These settings can be added under theparameters.extrasection of yourmodel-settings.jsonfile, e.g.---
emphasize-lines: 6-8
---
{
  "name": "drift-detector",
  "implementation": "mlserver_alibi_detect.AlibiDetectRuntime",
  "parameters": {
    "uri": "./alibi-detect-artifact/",
    "extra": {
      "batch_size": 5
    }
  }
}ReferenceYou can find the full reference of the accepted extra settings for the Alibi
Detect runtime below:.. autopydantic_settings:: mlserver_alibi_detect.runtime.AlibiDetectSettings
Package: mlserver-huggingface
HuggingFace runtime for MLServerThis package provides a MLServer runtime compatible with HuggingFace Transformers.UsageYou can install the runtime, alongsidemlserver, as:pipinstallmlservermlserver-huggingfaceFor further information on how to use MLServer with HuggingFace, you can check
out thisworked out example.Content TypesThe HuggingFace runtime will always decode the input request using its own
built-in codec.
Therefore,content type annotationsat
the request level willbe ignored.
Note that thisdoesn't includeinput-level content
typeannotations, which will be
respected as usual.SettingsThe HuggingFace runtime exposes a couple extra parameters which can be used to
customise how the runtime behaves.
These settings can be added under theparameters.extrasection of yourmodel-settings.jsonfile, e.g.---
emphasize-lines: 5-8
---
{
  "name": "qa",
  "implementation": "mlserver_huggingface.HuggingFaceRuntime",
  "parameters": {
    "extra": {
      "task": "question-answering",
      "optimum_model": true
    }
  }
}These settings can also be injected through environment variables prefixed with `MLSERVER_MODEL_HUGGINGFACE_`, e.g.

```bash
MLSERVER_MODEL_HUGGINGFACE_TASK="question-answering"
MLSERVER_MODEL_HUGGINGFACE_OPTIMUM_MODEL=true
```Loading modelsLocal modelsIt is possible to load a local model into a HuggingFace pipeline by specifying the model artefact folder path inparameters.uriinmodel-settings.json.HuggingFace modelsModels in the HuggingFace hub can be loaded by specifying their name inparameters.extra.pretrained_modelinmodel-settings.json.If `parameters.extra.pretrained_model` is specified, it takes precedence over `parameters.uri`.ReferenceYou can find the full reference of the accepted extra settings for the
HuggingFace runtime below:.. autopydantic_settings:: mlserver_huggingface.settings.HuggingFaceSettings
Package: mlserver-huggingface-striveworks
HuggingFace runtime for MLServerThis package provides a MLServer runtime compatible with HuggingFace Transformers.UsageYou can install the runtime, alongsidemlserver, as:pipinstallmlservermlserver-huggingfaceFor further information on how to use MLServer with HuggingFace, you can check
out thisworked out example.Content TypesThe HuggingFace runtime will always decode the input request using its own
built-in codec.
Therefore,content type annotationsat
the request level willbe ignored.
Not that thisdoesn't includeinput-level content
typeannotations, which will be
respected as usual.SettingsThe HuggingFace runtime exposes a couple extra parameters which can be used to
customise how the runtime behaves.
These settings can be added under theparameters.extrasection of yourmodel-settings.jsonfile, e.g.---
emphasize-lines: 5-8
---
{
  "name": "qa",
  "implementation": "mlserver_huggingface.HuggingFaceRuntime",
  "parameters": {
    "extra": {
      "task": "question-answering",
      "optimum_model": true
    }
  }
}These settings can also be injected through environment variables prefixed with `MLSERVER_MODEL_HUGGINGFACE_`, e.g.

```bash
MLSERVER_MODEL_HUGGINGFACE_TASK="question-answering"
MLSERVER_MODEL_HUGGINGFACE_OPTIMUM_MODEL=true
```Loading modelsLocal modelsIt is possible to load a local model into a HuggingFace pipeline by specifying the model artefact folder path inparameters.uriinmodel-settings.json.HuggingFace modelsModels in the HuggingFace hub can be loaded by specifying their name inparameters.extra.pretrained_modelinmodel-settings.json.If `parameters.extra.pretrained_model` is specified, it takes precedence over `parameters.uri`.Model InferenceModel inference is done by HuggingFace pipeline. It allows users to run inference on a batch of inputs. Extra inference kwargs can be kept inparameters.extra.{
    "inputs": [
        {
            "name": "text_inputs",
            "shape": [1],
            "datatype": "BYTES",
            "data": ["My kitten's name is JoJo,","Tell me a story:"],
        }
    ],
    "parameters": {
        "extra":{"max_new_tokens": 200,"return_full_text": false}
    }
}ReferenceYou can find the full reference of the accepted extra settings for the
HuggingFace runtime below:.. autopydantic_settings:: mlserver_huggingface.settings.HuggingFaceSettings
Package: mlserver-lightgbm
LightGBM runtime for MLServerThis package provides a MLServer runtime compatible with LightGBM.UsageYou can install the runtime, alongsidemlserver, as:pipinstallmlservermlserver-lightgbmFor further information on how to use MLServer with LightGBM, you can check out
thisworked out example.Content TypesIf nocontent typeis present on the
request or metadata, the LightGBM runtime will try to decode the payload as
aNumPy Array.
To avoid this, either send a different content type explicitly, or define the
correct one as part of yourmodel's
metadata.
Package: mlserver-mlflow
MLflow runtime for MLServerThis package provides a MLServer runtime compatible withMLflow
models.UsageYou can install the runtime, alongsidemlserver, as:pipinstallmlservermlserver-mlflowContent TypesThe MLflow inference runtime introduces a newdictcontent type, which
decodes an incoming V2 request as adictionary of
tensors.
This is useful for certain MLflow-serialised models, which will expect that the
model inputs are serialised in this format.The `dict` content type can be _stacked_ with other content types, like
[`np`](../../docs/user-guide/content-type).
This allows the user to use a different set of content types to decode each of
the dict entries.
Package: mlserver-openvino
OverviewThis package provides a MLServer runtime compatible with Openvino. This package has couple features:If server detect that model file is onnx format script will auto convert to openvino format (xml, bin) with dynamic batch size for openvino.Openvino dynamic batch sizeGrpc ReadyV2 Inference ProtocolModels metricsWhy MLserver?For serving Openvino I choose MLServer because this framework has V2 Inference Protocol (https://kserve.github.io/website/modelserving/inference_api/), grpc and metrics out of the box.Installpipinstallmlservermlserver-openvinoContent TypesIf no content type is present on the request or metadata,
the Openvino runtime will try to decode the payload as a NumPy Array.
To avoid this, either send a different content type explicitly, or define the correct one as part of your model’smetadata.Models repositoryYour models add to models folder.
Accepted files: ["model.xml", "model.onnx"]/example
/models/your-model-name/
/tests
setup.py
README.mdTraining and serve example:https://mlserver.readthedocs.io/en/latest/examples/sklearn/README.htmlMetricsFor download metrics (prometheus) use below linksGEThttp://<your-endpoint>/metrics
GEThttp://0.0.0.0:8080/metricsStart docker server# Build docker imagemlserverbuild.-ttest# Start server and pass mlserevr_models_dirdockerrun-it--rm-eMLSERVER_MODELS_DIR=/opt/mlserver/models/-p8080:8080-p8081:8081testExample queries:For example script see below files:/example/grpc-example.py
/example/rest-example.pyKserve usageFirst create one time kserve runtime from file: kserve/cluster-runtime.yamlCreate InferenceService from template:apiVersion:"serving.kserve.io/v1beta1"kind:"InferenceService"metadata:name:"my-openvino-model"spec:predictor:model:modelFormat:name:openvinoruntime:kserve-mlserver-openvino#storageUri: "gs://kfserving-examples/models/xgboost/iris"storageUri:https://github.com/myrepo/models/mymodel.joblib?raw=trueExample model-settings.json{"name":"mnist-onnx-openvino","implementation":"mlserver_openvino.OpenvinoRuntime","parameters":{"uri":"./model.onnx","version":"v0.1.0","extra":{"transform":[{"name":"Prepare Metadata","pipeline_file_path":"./pipeline.cloudpickle","input_index":0}]}},"inputs":[{"name":"input-0","datatype":"FP32","shape":[28,28,1]}],"outputs":[{"name":"output","datatype":"FP32","shape":[10]}]}TransformersIf you add transformer pipeline in extra properties you should dump code in same python version as execute mlserverTestsmaketest
Package: mlserver-sklearn
Scikit-Learn runtime for MLServerThis package provides a MLServer runtime compatible with Scikit-Learn.UsageYou can install the runtime, alongsidemlserver, as:pipinstallmlservermlserver-sklearnFor further information on how to use MLServer with Scikit-Learn, you can check
out thisworked out example.Content TypesIf nocontent typeis present on the
request or metadata, the Scikit-Learn runtime will try to decode the payload as
aNumPy Array.
To avoid this, either send a different content type explicitly, or define the
correct one as part of yourmodel's
metadata.Model OutputsThe Scikit-Learn inference runtime exposes a number of outputs depending on the
model type.
These outputs match to thepredict,predict_probaandtransformmethods
of the Scikit-Learn model.OutputReturned By DefaultAvailabilitypredict✅Available on most models, but not inScikit-Learn pipelines.predict_proba❌Only available on non-regressor models.transform❌Only available onScikit-Learn pipelines.By default, the runtime will only return the output ofpredict.
However, you are able to control which outputs you want back through theoutputsfield of your {class}InferenceRequest <mlserver.types.InferenceRequest>payload.For example, to only return the model'spredict_probaoutput, you could
define a payload such as:---
emphasize-lines: 10-12
---
{
  "inputs": [
    {
      "name": "my-input",
      "datatype": "INT32",
      "shape": [2, 2],
      "data": [1, 2, 3, 4]
    }
  ],
  "outputs": [
    { "name": "predict_proba" }
  ]
}
Package: mlserver-xgboost
XGBoost runtime for MLServerThis package provides a MLServer runtime compatible with XGBoost.UsageYou can install the runtime, alongsidemlserver, as:pipinstallmlservermlserver-xgboostFor further information on how to use MLServer with XGBoost, you can check out
thisworked out example.XGBoost Artifact TypeThe XGBoost inference runtime will expect that your model is serialised via one
of the following methods:ExtensionDocsExample*.jsonJSON Formatbooster.save_model("model.json")*.ubjBinary JSON Formatbooster.save_model("model.ubj")*.bst(Old) Binary Formatbooster.save_model("model.bst")By default, the runtime will look for a file called `model.[json | ubj | bst]`.
However, this can be modified through the `parameters.uri` field of your
{class}`ModelSettings <mlserver.settings.ModelSettings>` config (see the
section on [Model Settings](../../docs/reference/model-settings.md) for more
details).

```{code-block} json
---
emphasize-lines: 3-5
---
{
  "name": "foo",
  "parameters": {
    "uri": "./my-own-model-filename.json"
  }
}
```Content TypesIf nocontent typeis present on the
request or metadata, the XGBoost runtime will try to decode the payload as aNumPy Array.
To avoid this, either send a different content type explicitly, or define the
correct one as part of yourmodel's
metadata.Model OutputsThe XGBoost inference runtime exposes a number of outputs depending on the
model type.
These outputs match to thepredictandpredict_probamethods of the XGBoost
model.OutputReturned By DefaultAvailabilitypredict✅Available on all XGBoost models.predict_proba❌Only available on non-regressor models (i.e.XGBClassifiermodels).By default, the runtime will only return the output ofpredict.
However, you are able to control which outputs you want back through theoutputsfield of your {class}InferenceRequest <mlserver.types.InferenceRequest>payload.For example, to only return the model'spredict_probaoutput, you could
define a payload such as:---
emphasize-lines: 10-12
---
{
  "inputs": [
    {
      "name": "my-input",
      "datatype": "INT32",
      "shape": [2, 2],
      "data": [1, 2, 3, 4]
    }
  ],
  "outputs": [
    { "name": "predict_proba" }
  ]
}
Package: mlservicewrapper-host-http
Host anmlservicewrapper.core.services.Serviceas an HTTP API.Testing locallyUsing the development server is sufficient for testing, but should not be used in production.Command Line ArgumentsParameterRequired?DescriptionFormat--configYesPath to service configuration file.--config <path>--hostNoHost to bind to. Defaults to 127.0.0.1.--host <host name>--portNoPort to bind to. Defaults to 5000.--host <host name>DeploymentUsegunicornto run in production:exportSERVICE_CONFIG_PATH=/path/to/config.json
gunicorn-w4-kuvicorn.workers.UvicornWorkermlservicewrapper.host.http
Package: mlserving
ML Servingmlservingis a framework for developing a realtime model-inference service.Allows you to easily set-up an inference-endpoint for your ML Model.mlservingemphasizes on high performance and allows easy integration with other model servers such asTensorFlow ServingDocs can found here:https://mlserving.readthedocs.io/en/latest/MotivationData Scientists usually struggle with integrating their ML-models to production.mlservingis here to make the development of model-servers easy for everyone.Installation$pipinstallmlserving
Package: mlserving-utf-8
# ML Serving
[![PyPI version](https://badge.fury.io/py/mlserving.svg)](https://badge.fury.io/py/mlserving)
[![Build](https://github.com/orlevii/mlserving/workflows/build/badge.svg)]()mlservingis a framework for developing a realtime model-inference service.Allows you to easily set-up an inference-endpoint for your ML Model.mlservingemphasizes on high performance and allows easy integration with other model servers such asTensorFlow ServingDocs can found here:https://mlserving.readthedocs.io/en/latest/## Motivation
Data Scientists usually struggle with integrating their ML-models to production.mlservingis here to make the development of model-servers easy for everyone.## Installation`bash $ pip install mlserving `
Package: mlsetup
Project Structure for MLflow integrated ML ProjectsThis cli tool generates the following directory structure for quickstart ML projectsinstallaton:pip install mlsetupexample use:mlsetup create mytestprojectIt generates the project with following structureProject
|
+--- Input
|    |
|    +--- raw               Raw data here
|    |
|    +--- interim           Any intermediate data, to pause and continue experiments
|    |
|    +--- processed         Processed data ready for ML pipeline
|
+--- output
|    |
|    +--- models             Model pickle or model weights stored here
|    |
|    +--- artifacts         Serialized artifacts like LabelEncoder, Vectorizer etc
|    |
|    +--- images            All plots and visualizations goes here
|    |
|    +--- Results           If the results needs to be stored for review, save here
|
+--- notebooks              All notebooks and experiments resides here
|
+--- src                    Final program, with training and prediction pipeline
|
|
README.md                   Description and instruction about the project
mlsetup                   MLflow project file. If you want to use this directory as MLflow project
Requirements.txt            python dependencies
Config.yml                  configuration key values in yaml format