Package: ml-things
Machine Learning ThingsMachine Learning Thingsis a lightweight python library that contains functions and code snippets that
I use in my everyday research with Machine Learning, Deep Learning, NLP.I created this repo because I was tired of always looking up same code from older projects and I wanted to gain some experience in building a Python library.
By making this available to everyone it gives me easy access to code I use frequently and it can help others in their machine learning work.
If you find any bugs or something doesn't make sense please feel free to open an issue.That is not all! This library also contains Python code snippets and notebooks that speed up my Machine Learning workflow.Note:If I reach 100 stars I will release the first official version and add it to the pip install modules!Update:July 16, 2021Thank you all for your support and kidness! As I promissed I will move this repo to pip install modules.Table of contentsML_things:InstallationDetails on how to installml_things.Array FunctionsDetails on theml_thingsarray related functions:pad_arraybatch_arrayPlot FunctionsDetails on theml_thingsplot related functions:plot_arrayplot_dictplot_confusion_matrixText FunctionsDetails on theml_thingstext related functions:clean_textWeb RelatedDetails on theml_thingsweb related functions:download_fromSnippets: Curated list of Python snippets I frequently use.Comments: Sample on how I like to comment my code. It is still a work in progress.Notebooks Tutorials: Machine learning projects that I converted to tutorials and posted online.Final Note: Being grateful.ML_thingsInstallationThis repo is tested with Python 3.6+.It's always good practice to installml_thingsin avirtual environment. If you guidance on using Python's virtual environments you can check out the user guidehere.You can installml_thingswith pip from GitHub:pipinstallgit+https://github.com/gmihaila/ml_thingsFunctionsAll function implemented in theml_thingsmodule.Array FunctionsArray manipulation related function that can be useful when working with machine learning.pad_array[source]Pad variable length array to a fixed numpy array. It can handle single arrays [1,2,3] or nested arrays [[1,2],[3]].By default it will padd zeros to the maximum length of row detected:>>>fromml_thingsimportpad_array>>>pad_array(variable_length_array=[[1,2],[3],[4,5,6]])array([[1.,2.,0.],[3.,0.,0.],[4.,5.,6.]])It can also pad to a custom size and with cusotm values:>>>pad_array(variable_length_array=[[1,2],[3],[4,5,6]],fixed_length=5,pad_value=99)array([[1.,2.,99.,99.,99.],[3.,99.,99.,99.,99.],[4.,5.,6.,99.,99.]])batch_array[source]Split a list into batches/chunks. Last batch size is remaining of list values.Note:This is also called chunking. I call it batches since I use it more in ML.The last batch will be the reamining values:>>>fromml_thingsimportbatch_array>>>batch_array(list_values=[1,2,3,4,5,6,7,8,8,9,8,6,5,4,6],batch_size=4)[[1,2,3,4],[5,6,7,8],[8,9,8,6],[5,4,6]]Plot FunctionsPlot related function that can be useful when working with machine learning.plot_array[source]Create plot from a single array of values.All arguments are optimized for quick plots. Change themagnifyarguments to vary the size of the plot:>>>fromml_thingsimportplot_array>>>plot_array([1,3,5,3,7,5,8,10],path='plot_array.png',magnify=0.1,use_title='A Random Plot',start_step=0.3,step_size=0.1,points_values=True,use_ylabel='Thid',use_xlabel='This')plot_dict[source]Create plot from a single array of values.All arguments are optimized for quick plots. Change themagnifyarguments to vary the size of the plot:>>>fromml_thingsimportplot_dict>>>plot_dict({'train_acc':[1,3,5,3,7,5,8,10],'valid_acc':[4,8,9]},use_linestyles=['-','--'],magnify=0.1,start_step=0.3,step_size=0.1,path='plot_dict.png',points_values=[True,False],use_title='Title')plot_confusion_matrix[source]This function prints and plots the confusion matrix. Normalization can be applied by settingnormalize=True.All arguments are optimized for quick plots. Change themagnifyarguments to vary the size of the plot:>>>fromml_thingsimportplot_confusion_matrix>>>plot_confusion_matrix(y_true=[1,0,1,1,0,1],y_pred=[0,1,1,1,0,1],magnify=0.1,use_title='My Confusion Matrix',path='plot_confusion_matrix.png');Confusionmatrix,withoutnormalizationarray([[1,1],[1,3]])Text FunctionsText related function that can be useful when working with machine learning.clean_text[source]Clean text using various techniques:>>>fromml_thingsimportclean_text>>>clean_text("ThIs is $$$%.\t\t\n\\so dirtyyy$$ text :'(.   omg!!!",full_clean=True)'this is so dirtyyy text omg'Web RelatedWeb related function that can be useful when working with machine learning.download_from[source]Download file from url. It will return the path of the downloaded file:>>>fromml_thingsimportdownload_from>>>download_from(url='https://raw.githubusercontent.com/gmihaila/ml_things/master/setup.py',path='.')'./setup.py'SnippetsThis is a very large variety of Python snippets without a certain theme. I put them in the most frequently used ones while keeping a logical order.
I like to have them as simple and as efficient as possible.NameDescriptionRead FIleOne liner to read any file.Write FileOne liner to write a string to a file.DebugStart debugging after this line.Pip Install GitHubInstall library directly from GitHub usingpip.Parse ArgumentParse arguments given when running a.pyfile.DoctestHow to run a simple unittesc using function documentaiton. Useful when need to do unittest inside notebook.Fix TextSince text data is always messy, I always use it. It is great in fixing any bad Unicode.Current DateHow to get current date in Python. I use this when need to name log files.Current TimeGet current time in Python.Remove PunctuationThe fastest way to remove punctuation in Python3.PyTorch-DatasetCode sample on how to create a PyTorch Dataset.PyTorch-DeviceHow to setup device in PyTorch to detect if GPU is available.CommentsThese are a few snippets of how I like to comment my code. I saw a lot of different ways of how people comment their code. One thing is for sure:any comment is better than no comment.I try to follow as much as I can thePEP 8 — the Style Guide for Python Code.When I comment a function or class:# required import for variables type declarationfromtypingimportList,Optional,Tuple,Dictdefmy_function(function_argument:str,another_argument:Optional[List[int]]=None,another_argument_:bool=True)->Dict[str,int]r"""Function/Class main comment.More details with enough spacing to make it easy to follow.Arguments:function_argument (:obj:`str`):A function argument description.another_argument (:obj:`List[int]`, `optional`):This argument is optional and it will have a None value attributed inside the function.another_argument_ (:obj:`bool`, `optional`, defaults to :obj:`True`):This argument is optional and it has a default value.The variable name has `_` to avoid conflict with similar name.Returns::obj:`Dict[str: int]`: The function returns a dicitonary with string keys and int values.A class will not have a return of course."""# make sure we keep out promise and return the variable type we described.return{'argument':function_argument}Notebooks TutorialsThis is where I keep notebooks of some previous projects which I turnned them into small tutorials. A lot of times I use them as basis for starting a new project.All of the notebooks are inGoogle Colab. Never heard of Google Colab? :scream_cat: You have to check out theOverview of Colaboratory,Introduction to Colab and Pythonand what I think is a great medium article about itto configure Google Colab Like a Pro.If you check the/ml_things/notebooks/a lot of them are not listed here because they are not in a 'polished' form yet. These are the notebooks that are good enough to share with everyone:NameDescriptionLinks:grapes: Better Batches with PyTorchText BucketIteratorHow to use PyTorchText BucketIterator to sort text data for better batching.:dog: Pretrain Transformers Models in PyTorch using Hugging Face TransformersPretrain 67 transformers models on your custom dataset.:violin: Fine-tune Transformers in PyTorch using Hugging Face TransformersComplete tutorial on how to fine-tune 73 transformer models for text classification — no code changes necessary!⚙️ Bert Inner Workings in PyTorch using Hugging Face TransformersComplete tutorial on how an input flows through Bert.🎱 GPT2 For Text Classification using Hugging Face 🤗 TransformersComplete tutorial on how to use GPT2 for text classification.Final NoteThank you for checking out my repo. I am a perfectionist so I will do a lot of changes when it comes to small details.If you see something wrong please let me know by opening anissue on my ml_thingsGitHub repository!A lot of tutorials out there are mostly a one-time thing and are not being maintained. I plan on keeping my
tutorials up to date as much as I can.Contact🎣🦊 GitHub:gmihaila🌐 Website:gmihaila.github.io👔 LinkedIn:mihailageorge📬 Email:georgemihaila@my.unt.edu.com