Saravanan666/pypi-org huggingface dataset:
Package: ml_utils_jme45
ml_utilsCode for training pytorch modulesInstallationpip install <directory of source code>Usage exampleimportml_utilsfromtorchimportnntrainer=ml_utils.ClassificationTrainer(model,train_dataloader,test_dataloader,"Adam",{"lr":1e-3},nn.CrossEntropyLoss(),n_epochs=2,output_path='run1',device='cpu',num_classes=2,)results=trainer.train()
Package: mlvajra
mlVajraA framework or best practices to develop end to end machine learning pipeline (also has some tips for ML-management people )
Aim :
To built robust depoyment pipeline strategies using Open source stack
planning to add as many strategies in this repo pertaining to ML-deploymentInstallation :pip install mlvajra- only installs mlvajra binariesTo install complete dependencies: (for time being)git clonehttps://github.com/rajagurunath/mlvajra.gitcreate virtualenv

virtualenv -p python3 vajra_env

source vajra_env\bin\activate

cd mlvajra
Install all required dependencies from repo

pip install -r requirements.txtTODO listDeployMlflowTensorflow servingmodel-Training /distribuitedmlflow -generic classification metrics (done)nnictl-automl -tensorflow /pytorchFeature Engineeringpandaspyspark-Flintpreprocessingcyclic features (done)lag featureswindow features
Package: mlvis
IntroductionThe mlvis library contains a suite a visualization components. This library is still under active development. The current version consists of several prebuilt visualization components:Stacked CalendarGraph BuilderManifoldFeature List ViewInstallationpip install mlvisThen install the nbextension into the Jupyter notebookjupyter nbextension install --py --symlink --sys-prefix mlvis
jupyter nbextension enable --py --sys-prefix mlvisDevelopmentInstructions for building this project locally can be found in thedevelopment documentation.
Package: mlvision
MLVISIONA pytorch train libraryIntroductionMost classification tasks do not require to rewrite a train loop in pytorch, using this library allows for fast iteration using any deep learning architecture.The library can be use for cpu, gpu, and multi gpu training, and implements an efficient train loop as well as logging in order to plot train history.InstallationYou can install it using poetry by running:poetry installUsageTrain a ModelTo train a model you will need to edit a config file with at least the path of the training and validation data.For more details on the config you can take a look at the next section.You can now run the train command using:# One GPUpoetryrunpythontrain.py--config/path/to/config--devicescuda:0The code support multi gpu training using DDP:poetryrunpythontrain.py--config/path/to/config--devicescuda:0cuda:1cuda:2...or cpu training:poetryrunpythontrain.py--config/path/to/config--devicescpuConfigThe config allows you to change multiple hyper parameter of the training:datapath to datasetdata augmentationworkers for data loadersoptimizationnumber of epochslearning ratefloat16 / float32modelArchitecture (defined inmlvision.models.__init__.py)args: given as kwargs to the model classAdd a new modelYou can easily define a new model and use it, to do so create a new file inmlvision/modelsand add the class in themlvision.models.__init__.pyfile.You should now be able to use it in a config file.
Package: mlvizlib
MLVizLib (Machine Learning Visualization Library)is a powerful
library for generating quick, insightful, and stylish visualizations for
machine learning (ML). Our goal is to enhance the ML workflow by
providing insightful visualizations with minimum effort.Documentation: (COMING SOON)https://mlvizlib.readthedocs.io.NOTEThis project is in early stage development, and can thus go trough major changes.InstallMLVizLib can be installed fromPyPI:pip install mlvizlibFeaturesConfusion Matrix VisualizationnoteMore coming soon.Confusion Matrix Visualization exampleimport matplotlib.pyplot as plt
from mlvizlib.classification import confusion_matrix

# example data
eg_y_true = [2,0,1,0,2,0,1,2,0,0,2,0,1,1,0,1,1,0,0,0,0,2,2]
eg_y_pred = [2,0,0,0,2,0,1,2,1,0,2,2,1,1,0,2,1,0,1,0,0,1,2]

confusion_matrix(eg_y_true, eg_y_pred)
plt.show()
Package: mlvtk
MLVTKA loss surface visualization toolSimple DNN trained on MNIST data set, using Adamax optimizerSimple DNN trained on MNIST, using SGD optimizerSimple DNN trained on MNIST, using Adam optimizerSimple DNN trained on MNIST, using SGD optimizerWhy?:shipit:Simple: A single line addition is all that is needed.:question:Informative: Gain insight into what your model is seeing.:notebook:Educational:Seehow your hyper parameters and architecture impact your
models perception.Quick StartRequiresversionpython>= 3.6.1tensorflow>= 2.3.1, <  2.4.2plotly>=4.9.0Install locally (Also works in google Colab!):pipinstallmlvtkOptionally for use with jupyter notebook/lab:Notebookpipinstall"notebook>=5.3""ipywidgets==7.5"Labpipinstalljupyterlab"ipywidgets==7.5"# Basic JupyterLab renderer supportjupyterlabextensioninstalljupyterlab-plotly@4.10.0# OPTIONAL: Jupyter widgets extension for FigureWidget supportjupyterlabextensioninstall@jupyter-widgets/jupyterlab-managerplotlywidget@4.10.0Basic Examplefrommlvtk.baseimportVmodelimporttensorflowastfimportnumpyasnp# NN with 1 hidden layerinputs=tf.keras.layers.Input(shape=(None,100))dense_1=tf.keras.layers.Dense(50,activation='relu')(inputs)outputs=tf.keras.layers.Dense(10,activation='softmax')(dense_1)_model=tf.keras.Model(inputs,outputs)# Wrap with Vmodelmodel=Vmodel(_model)model.compile(optimizer=tf.keras.optimizers.SGD(),loss=tf.keras.losses.CategoricalCrossentropy(),metrics=['accuracy'])# All tf.keras.(Model/Sequential/Functional) methods/properties are accessible# from Vmodelmodel.summary()model.get_config()model.get_weights()model.layers# Create random example datax=np.random.rand(3,10,100)y=np.random.randint(9,size=(3,10,10))xval=np.random.rand(1,10,100)yval=np.random.randint(9,size=(1,10,10))# Only difference, model.fit requires validation_data (tf.data.Dataset, or# other containerhistory=model.fit(x,y,validation_data=(xval,yval),epochs=10,verbose=0)# Calling model.surface_plot() returns a plotly.graph_objs.Figure# model.surface_plot() will attempt to display the figure inlinefig=model.surface_plot()# fig can save an interactive plot to an html file,fig.write_html("surface_plot.html")# or display the plot in jupyter notebook/lab or other compatible tool.fig.show()
Package: ml-wac
WAC: ML Web-based Attack ClassifierA Machine Learning Web-based Attack Classifier for the purpose of detecting and identifying LFI, RFI, SQLI, and XSS attacks based on request paths. This initiative is part of a research project at the University of Amsterdam conducted byJordandIsaacunder the supervision ofEvgeniia.Getting startedInstallation using PyPIpip install ml-wacPerformancePerformance evaluations have been conducted on all models to determine their accuracy and inference time on the 3579-item test set.Model NameAccuracyInference Time (ms)XGBoost98.80%82.71Decision Tree98.21%1.95Logistic Regression98.30%5.58Support Vector Machine99.25%19225.70ExamplesPredict a single pathfrom ml_wac.wac import WebAttackClassifier

# Create new instance
wac = WebAttackClassifier()

# Predict a single path. Optionally, a certainty threshold can be provided
prediction = wac.predict_single("/test?id=<script>alert(1)</script>", threshold=0.7)

print(prediction)Predict multiple pathsfrom ml_wac.wac import WebAttackClassifier
	
# Create new instance
wac = WebAttackClassifier()

# Predict a list of paths, returns a list of predicted attack types
predictions = wac.predict([
    "/status?message=<script>/*+Bad+stuff+here+*/</script>",
    "/?download=../include/connection.php",
    "/?file=../../uploads/evil.php",
    "/products?category=Gifts'+OR+1=1--"
])

print(predictions)Use other trained modelsUseone of the other pre-trained modelsfor inference. By default the logistic regression model is used.from ml_wac.types.model_type import ModelType
from ml_wac.wac import WebAttackClassifier

# Load the XG_BOOST model
wac = WebAttackClassifier(model_type=ModelType.XG_BOOST)

# Predict a single path. Optionally, a certainty threshold can be provided
prediction = wac.predict_single("/test?id=<script>alert(1)</script>", threshold=0.7)

print(prediction)
Package: mlwhatif
mlwhatifData-Centric What-If Analysis for Native Machine Learning PipelinesRun mlwhatif locallyPrerequisite: Python 3.9Clone this repositorySet up the environmentcd mlwhatifpython -m venv venvsource venv/bin/activateIf you want to use the visualisation functions we provide, install graphviz which can not be installed via pipLinux:apt-get install graphvizMAC OS:brew install graphvizInstall pip dependenciespip install -e ."[dev]"To ensure everything works, you can run the tests (without graphviz, the visualisation test will fail)python setup.py testNotesFor debugging in PyCharm, set the pytest flag--no-cov(Link)LicenseThis library is licensed under the Apache 2.0 License.
Package: mlwithmsb
mlwithmsb: A Versatile Python Package for Outlier Handling in Data AnalysisEffortlessly clean your datasets and ensure data quality with Outlier Cleaner! This user-friendly package empowers you to detect and address outliers using multiple effective methods, tailored to your specific needs.Key Features:Flexible Outlier Detection:Choose from IQR, Z-score, or visual inspection methods to identify outliers based on statistical principles or your expert judgment.Customizable Thresholds:Fine-tune the sensitivity of outlier detection to match your dataset's characteristics and analysis goals.Visual Exploration:Leverage informative histograms and box plots to visualize data distribution and guide outlier identification in the visual method.Streamlined NaN Handling:Automatically remove rows containing missing values to maintain data integrity.Easy Integration:Seamlessly incorporate outlier cleaning into your data analysis workflows with a simple function call.Benefits:Improve data quality and accuracy for reliable analysis outcomes.Gain deeper insights into your data by identifying and addressing potential anomalies.Enhance model performance by mitigating the impact of outliers.Foster reproducibility and transparency in your data analysis processes.Installation:pipinstallmlwithmsbUsage:importpandasaspdfrommlwithmsbimportremove_outliers# Load your datasetdata=pd.read_csv("your_data.csv")# Clean the data using your preferred method and thresholdcleaned_data=remove_outliers(data,method="iqr",threshold=1.5)# Example**Embracerobustdatacleaningforaccurateandreliableinsights—installmlwithmsbtoday!**
Package: ml-workflow
ML workflow contains our process of bringing a project to fruition as
efficiently as possible. This is subject to change as we iterate and improve.
This package implements tools and missing features to help bridge the gap
between frameworks and libraries that we utilize.The main packages and tools that we build around are:pytorchignitepytorch-datastreamguildSee thedocumentationfor more information.Install in existing projectpip install ml-workflowCreate new project with MNIST templatemkdir new-project
cd new-project
virtualenv venv -p python3.8
source venv/bin/activate
pip install ml-workflow
python -m workflow.setup_project

pip install -r requirements.txt
pip install -r dev_requirements.txt
pip freeze > dev_requirements.txt

# reactivate environment to find guild
deactivate
source venv/bin/activateYou can train a model and inspect the training with:guild run prepare
guild run train
guild tensorboardDevelopmentPrepare and run testsgit clone git@github.com:aiwizo/ml-workflow.git
cd ml-workflow
virtualenv venv --python python3.8
source venv/bin/activate
pip install -r requirements.txt
pip install -r dev_requirements.txt
pip install pytest
python -m pytestTest template./setup_template.py
./test_template.pyUse development version in projectThe following steps will create a link to the local directory and any changes
made to the package there will directly carry over to your project environment.cd path/to/my/project
source venv/bin/activate

cd path/to/work/area
git clone git@github.com:aiwizo/ml-workflow.git
cd ml-workflow
pip install -e .
Package: mlwrapper
MLWrapper v0.1MLwrapper is a context manager that helps you store experiment results.Context manager __enter__ creates mlflow run and store logged values inside. It allows logging following stuff:script argumentsimagesscalarsmetricsQuick start# data to logkwargs={"experiment parameter":42,}test_image_1=np.ones(shape=(3,40,40,1))test_image_1[0,:20,:,:]=0.test_image_1[1,:,:,:]=0.test_image_1[2,20:,:,:]=0.test_image_2=np.ones(shape=(3,1,40,40))test_image_2[0,:,20:,:]=0.test_image_2[1,:,:,:]=0.test_image_2[2,:,:20,:]=0.deftest(logger):logger.log_args(**{"run param":"value"})forstepinrange(0,50):logger.log_scalar("test_loss",value=100-step*1.5,step=step)logger.log_scalar("test_acc",value=0.00+step*0.01,step=step)logger.log_images("test_image",test_image_1,1)logger.log_images("test_image",test_image_2,2,channel_first=True)logger.log_metric("result",result)# approach 1Experiment=MLWrapper(mlflow_dir="/tmp/mlruns/",**kwargs)withExperimentaslogger:test(logger)# approach 2withMLWrapper(mlflow_dir="/tmp/mlruns/",**kwargs)aslogger:test(logger)# approach 3Experiment=MLWrapper(mlflow_dir="/tmp/mlruns/",**kwargs)wrapped_test=Experiment(test)# func needs to accept "logger" or "**kwarg"wrapped_test()TestingTesting will create files under /tmp directory. Those files are not deleted automatically.python3-munittestdiscover.Referencesmlflowtensorboard
Package: ml-wrappers
Machine Learning Wrappers SDK for PythonThis package has been tested with Python 3.8, 3.9 and 3.10The Machine Learning Wrappers SDK provides a unified wrapper for various ML frameworks - to have one uniform scikit-learn format predict and predict_proba functions.Highlights of the package include:A dataset wrapper to handle scipy sparse, pandas and numpy datasets in a uniform manner.A model wrapper to handle models from various frameworks uniformly, including scikit-learn, tensorflow, pytorch, lightgbm and xgboostPlease see the github website for the documentation and sample notebooks:https://github.com/microsoft/ml-wrappers
Package: mlx
MLXQuickstart|Installation|Documentation|ExamplesMLX is an array framework for machine learning research on Apple silicon,
brought to you by Apple machine learning research.Some key features of MLX include:Familiar APIs: MLX has a Python API that closely follows NumPy.  MLX
also has fully featured C++,C, andSwiftAPIs, which closely mirror
the Python API.  MLX has higher-level packages likemlx.nnandmlx.optimizerswith APIs that closely follow PyTorch to simplify building
more complex models.Composable function transformations: MLX supports composable function
transformations for automatic differentiation, automatic vectorization,
and computation graph optimization.Lazy computation: Computations in MLX are lazy. Arrays are only
materialized when needed.Dynamic graph construction: Computation graphs in MLX are constructed
dynamically. Changing the shapes of function arguments does not trigger
slow compilations, and debugging is simple and intuitive.Multi-device: Operations can run on any of the supported devices
(currently the CPU and the GPU).Unified memory: A notable difference from MLX and other frameworks
is theunified memory model. Arrays in MLX live in shared memory.
Operations on MLX arrays can be performed on any of the supported
device types without transferring data.MLX is designed by machine learning researchers for machine learning
researchers. The framework is intended to be user-friendly, but still efficient
to train and deploy models. The design of the framework itself is also
conceptually simple. We intend to make it easy for researchers to extend and
improve MLX with the goal of quickly exploring new ideas.The design of MLX is inspired by frameworks likeNumPy,PyTorch,Jax, andArrayFire.ExamplesTheMLX examples repohas a
variety of examples, including:Transformer language modeltraining.Large-scale text generation withLLaMAand
finetuning withLoRA.Generating images withStable Diffusion.Speech recognition withOpenAI's Whisper.QuickstartSee thequick start
guidein the documentation.InstallationMLX is available onPyPI. To install the Python API, run:Withpip:pip install mlxWithconda:conda install -c conda-forge mlxCheckout thedocumentationfor more information on building the C++ and Python APIs from source.ContributingCheck out thecontribution guidelinesfor more information
on contributing to MLX. See thedocsfor more
information on building from source, and running tests.We are grateful for all ofour
contributors. If you contribute
to MLX and wish to be acknowledged, please add your name to the list in your
pull request.Citing MLXThe MLX software suite was initially developed with equal contribution by Awni
Hannun, Jagrit Digani, Angelos Katharopoulos, and Ronan Collobert. If you find
MLX useful in your research and wish to cite it, please use the following
BibTex entry:@software{mlx2023,
  author = {Awni Hannun and Jagrit Digani and Angelos Katharopoulos and Ronan Collobert},
  title = {{MLX}: Efficient and flexible machine learning on Apple silicon},
  url = {https://github.com/ml-explore},
  version = {0.0},
  year = {2023},
}
Package: mlx90640-driver
MLX90640 driverIntroMLX90640 is a thermal camera (32x24 pixels) using Far InfraRed radiation from objects to measure the object temperature.https://www.melexis.com/mlx90640This python driver interfaces the MLX90640 and aims to facilitate rapid prototyping.Currently this driver supports 3 type of I2C interfaces:EVB90640-41 ==>https://www.melexis.com/en/product/EVB90640-41/Raspberry Pi with I2C on the 40-pin header.MCP2221 USB to I2C adaptor.And 4 operation systems platforms:Windows 10UbutuntuRasberry pi 4BJetson Nano (Nvidia)Getting startedInstallationpipinstallmlx90640-driverhttps://pypi.org/project/mlx90640-driver/https://pypistats.org/packages/mlx90640-driverNote This is a base package, therefore one of the below I2C drivers is still needed:https://github.com/melexis-fir/mlx90640-driver-evb9064x-pyhttps://github.com/melexis-fir/mlx90640-driver-devicetree-py(/dev/i2c-)https://github.com/melexis-fir/mlx90640-driver-mcp2221-pyor make your own.Running the driver demoConnect the EVB to your PC.Open a terminal and run following command:mlx90640-dumpautoThis program takes 1 optional argument.mlx90640-dump<communication-port>
Package: mlx90640-driver-devicetree
MLX90640 driver for device tree on a SBC.MLX90640 is a thermal camera (32x24 pixels) using Far InfraRed radiation from objects to measure the object temperature.https://www.melexis.com/mlx90640The python package "mlx90640-driver" driver interfaces the MLX90640 and aims to facilitate rapid prototyping.This package provide the I2C low level routines.
It uses the I2C from the device tree of a single board computer(SBC).Getting startedInstallationpipinstallmlx90640-driver-devicetreehttps://pypi.org/project/mlx90640-driver-devicetreehttps://pypistats.org/packages/mlx90640-driver-devicetreeNote:Make sure the user has access to/dev/i2c-<x>.And easy way to do this is by adding the user to the groupi2c.Running the driver demoConnect the MLX90640 on the I2C bus of the SBC.Open a terminal and run following command:mlx90640-devicetree-dump/dev/i2c-1This program takes 1 optional argument.mlx90640-devicetree-dump<communication-port>Note: this dump command is not yet available!
Package: mlx90640-driver-evb9064x
MLX90640 driver for EVB90640-41Warning: not stable yet.Update is commit, but only validated on windows OS.Currently only COM1 .. COM32 are supported.Linux based OSes are not checked/validated.MLX90640 is a thermal camera (32x24 pixels) using Far InfraRed radiation from objects to measure the object temperature.https://www.melexis.com/mlx90640The python package "mlx90640-driver" driver interfaces the MLX90640 and aims to facilitate rapid prototyping.This package provide the I2C low level routines.
It uses the I2C from the EVB90640-41 which is connected via the USB cable to the computer.https://www.melexis.com/en/product/EVB90640-41/Getting startedInstallationpipinstallmlx90640-driver-evb9064xhttps://pypi.org/project/mlx90640-driver-evb9064xhttps://pypistats.org/packages/mlx90640-driver-evb9064xNote:On linux OS, make sure the user has access to the comport/dev/ttyUSB<x>or/dev/ttyACM<x>.And easy way to do this is by adding the user to the groupdialout.Running the driver demoConnect the MLX90640 to the EVBConnect the EVB to your PC with the USB cable.Open a terminal and run following command:mlx90640-evb9064x-dumpautoThis program takes 1 optional argument.mlx90640-evb9064x-dump<communication-port>Note: this dump command is not yet available!
Package: mlx90640-driver-mcp2221
MLX90640 driver for MCP2221 USB I2C hubMLX90640 is a thermal camera (32x24 pixels) using Far InfraRed radiation from objects to measure the object temperature.https://www.melexis.com/mlx90640The python package "mlx90640-driver" driver interfaces the MLX90640 and aims to facilitate rapid prototyping.This package provide the I2C low level routines.
It uses the I2C hub from MCP2221 chip which is connected via the USB cable to the computer.https://www.microchip.com/wwwproducts/en/mcp2221https://www.adafruit.com/product/4471Getting startedInstallationpipinstallmlx90640-driver-mcp2221https://pypi.org/project/mlx90640-driver-mcp2221https://pypistats.org/packages/mlx90640-driver-mcp2221Extra installation for linux based OS.install udev, libusb and libhidapi:sudoaptupdate
sudoapt-getinstalllibudev-devlibusb-1.0-0-devlibhidapi-devConfigure such that non-root users have access.Place a file in/etc/udev/rules.d/20-microchip.rules:KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", MODE="0666"You might use this command line to create that file:echo'KERNEL=="hidraw*", ATTRS{idVendor}=="04d8", MODE="0666"'|sudotee/etc/udev/rules.d/20-microchip.rules>/dev/nullNote:Make sure to (re-)plug the MCP2221 after this file is written!Running the driver demoConnect the MLX90640 to the MCP2221 I2C portConnect the MCP2221 to your PC with the USB cable.Open a terminal and run following command:mlx90640-mcp2221-dumpmcp://mcp:2221/0This program takes 1 optional argument.mlx90640-mcp2221-dump<communication-port>Note: this dump command is not yet available!
Package: mlx90641-driver
MLX90641 driverIntroMLX90641 is a thermal camera (16x12 pixels) using Far InfraRed radiation from objects to measure the object temperature.https://www.melexis.com/mlx90641This python driver interfaces the MLX90641 and aims to facilitate rapid prototyping.Currently this driver supports 3 type of I2C interfaces:EVB90640-41 ==>https://www.melexis.com/en/product/EVB90640-41/Raspberry Pi with I2C on the 40-pin header.MCP2221 USB to I2C adaptor.And 4 operation systems platforms:Windows 10UbutuntuRasberry pi 4BJetson Nano (Nvidia)Getting startedInstallationpipinstallmlx90641-driverhttps://pypi.org/project/mlx90641-driver/https://pypistats.org/packages/mlx90641-driverNote This is a base package, therefore one of the below I2C drivers is still needed:https://github.com/melexis-fir/mlx90641-driver-evb9064x-pyhttps://github.com/melexis-fir/mlx90641-driver-devicetree-py(/dev/i2c-)https://github.com/melexis-fir/mlx90641-driver-mcp2221-pyor make your own.Running the driver demoConnect the EVB to your PC.Open a terminal and run following command:mlx90641-dumpautoThis program takes 1 optional argument.mlx90641-dump<communication-port>