Package: mldiag
MLDiagThis python library helps you diagnose machine learning models before deployment.Visit this introduction to understand aboutMLDiag.FeaturesGenerate synthetic data with adversarial attacks to evaluate model robustnessMake some interesting statistics on model behaviourSimple, easy-to-use and lightweight library. Diagnose data in 3 lines of codePlug and play to any neural network frameworks (e.g. PyTorch, TensorFlow) or standard machine learning framework (e.g. scikit-learn)Support textual, image, audio and structured dataCan be added in a CI workflowCan be used in command line or python scriptsQuick DemoExample of diagnosis of Text classification modelQuick startInstallationThe library supports python 3.7+ in linux and window platform.To install the library:pipinstallmldiagor install the latest version (include BETA features) from github directlypipinstallgit+https://github.com/AI-MEN/mldiag.gitRun a diagosticMethod 1:This method uses command lines only.
It requires a model running as a webservice.
We provide for a demo a complete example:create a text classification model:pythonexamples/text_classification/tf_text_classification.pytrain--save_model_path=./mldiaga tensorflow modelmodel.h5is created in themldiagdirectoryRun a text classification web service:pythonexamples/text_classification/flask_text_classification_service.py--model_path./mldiag/model.h5a local webservice is running underhttp://localhost:8080/querycreate the test set to diagnose the modelpythonexamples/text_classification/tf_text_classification.pysave_test_set--out_path=./mldiaga test settest.npyis saved inmldiag.
It contains a numpy array of text examples and their class labelsrun the diagnostic application calling the web servicepythonmldiag/cli.pydiagnose--eval_set"./mldiag/test.npy"--config_file"examples/text_classification/config_text_classification.yaml"--service_urlhttp://localhost:8080/query--report_path"./mldiag"--json_field"results"whereresultsis the key used to jsonify data from the webservice (see the web service script).![MLDiag](https://github.com/AI-MEN/MLDiag/tree/master/blog/capture.jpg=300x100)Method 2This method uses python scripts.
it supports a number of machine learning models and data formats through wrappers.
Ready to use wrappers can be found inmldiag/wrappers.pyIn the following, a complete example is proposed as demo.create a text classification model:pythonexamples/text_classification/tf_text_classification.pytrain--save_model_path=./mldiaga tensorflow modelmodel.h5is created in themldiagdirectorycall the  python scrip (the diagnose config file is available inexamples/text_classification/config_text_classification.yaml):pythonexamples/text_classification/tf_text_classification_diag.pyrun--model_path=./mldiag/model.h5--repor_path=./mldiagDiagnosticsDiagnosticTargetActionDescriptionTextualCharacterOCRErrorSimulate ocr errorRecent ChangesSeechangelogfor more details.Extension ReadingData Augmentation library for TextDoes your NLP model able to prevent adversarial attack?How does Data Noising Help to Improve your NLP Model?Data Augmentation library for Speech RecognitionData Augmentation library for AudioUnsupervied Data AugmentationA Visual Survey of Data Augmentation in NLPReferenceThis library uses:data (e.g. capturing from internet),research (e.g. following augmenter idea),model (e.g. using pre-trained model)TODO: update sourcesSeedata sourcefor more details.Citing@misc{shabou2020mldiag,
  title={Machine learning diagnosis},
  author={Aymen SHABOU},
  howpublished={https://github.com/AI-MEN/MLDiag},
  year={2020}}Contributions