Package: ktrain
Welcome to ktraina "Swiss Army knife" for machine learningNews and Announcements2024-02-20ktrain 0.41.xis released and removes thektrain.text.qa.generative_qamodule.  OurOnPrem.LLMpackage should be used for Generative Question-Answering tasks. Seeexample notebook.Overviewktrainis a lightweight wrapper for the deep learning libraryTensorFlow Keras(and other libraries) to help build, train, and deploy neural networks and other machine learning models.  Inspired by ML framework extensions likefastaiandludwig,ktrainis designed to make deep learning and AI more accessible and easier to apply for both newcomers and experienced practitioners. With only a few lines of code,ktrainallows you to easily and quickly:employ fast, accurate, and easy-to-use pre-canned models fortext,vision,graph, andtabulardata:textdata:Text Classification:BERT,DistilBERT,NBSVM,fastText, and other models[example notebook]Text Regression:BERT,DistilBERT, Embedding-based linear text regression,fastText, and other models[example notebook]Sequence Labeling (NER):  Bidirectional LSTM with optionalCRF layerand various embedding schemes such as pretrainedBERTandfasttextword embeddings and character embeddings[example notebook]Ready-to-Use NER models for English, Chinese, and Russianwith no training required[example notebook]Sentence Pair Classificationfor tasks like paraphrase detection[example notebook]Unsupervised Topic ModelingwithLDA[example notebook]Document Similarity with One-Class Learning:  given some documents of interest, find and score new documents that are thematically similar to them usingOne-Class Text Classification[example notebook]Document Recommendation Engines and Semantic Searches:  given a text snippet from a sample document, recommend documents that are semantically-related from a larger corpus[example notebook]Text Summarization:  summarize long documents - no training required[example notebook]Extractive Question-Answering:  ask a large text corpus questions and receive exact answers using BERT[example notebook]Generative Question-Answering:  ask a large text corpus questions and receive answers with citations using local or OpenAI models[example notebook]Easy-to-Use Built-In Search Engine:  perform keyword searches on large collections of documents[example notebook]Zero-Shot Learning:  classify documents into user-provided topicswithouttraining examples[example notebook]Language Translation:  translate text from one language to another[example notebook]Text Extraction: Extract text from PDFs, Word documents, etc.[example notebook]Speech Transcription: Extract text from audio files[example notebook]Universal Information Extraction:  extract any kind of information from documents by simply phrasing it in the form of a question[example notebook]Keyphrase Extraction:  extract keywords from documents[example notebook]Sentiment Analysis: easy-to-use wrapper to pretrained sentiment analysis[example notebook]Generative AI with GPT: Provide instructions to a lightweight ChatGPT-like model running on your own own machine to solve various tasks.[example notebook]visiondata:image classification(e.g.,ResNet,Wide ResNet,Inception)[example notebook]image regressionfor predicting numerical targets from photos (e.g., age prediction)[example notebook]image captioningwith a pretrained model[example notebook]object detectionwith a pretrained model[example notebook]graphdata:node classificationwith graph neural networks (GraphSAGE)[example notebook]link predictionwith graph neural networks (GraphSAGE)[example notebook]tabulardata:tabular classification(e.g., Titanic survival prediction)[example notebook]tabular regression(e.g., predicting house prices)[example notebook]causal inferenceusing meta-learners[example notebook]estimate an optimal learning rate for your model given your data using a Learning Rate Finderutilize learning rate schedules such as thetriangular policy, the1cycle policy, andSGDRto effectively minimize loss and improve generalizationbuild text classifiers for any language (e.g.,Arabic Sentiment Analysis with BERT,Chinese Sentiment Analysis with NBSVM)easily train NER models for any language (e.g.,Dutch NER)load and preprocess text and image data from a variety of formatsinspect data points that were misclassified andprovide explanationsto help improve your modelleverage a simple prediction API for saving and deploying both models and data-preprocessing steps to make predictions on new raw databuilt-in support for exporting models toONNXandTensorFlow Lite(seeexample notebookfor more information)TutorialsPlease see the following tutorial notebooks for a guide on how to usektrainon your projects:Tutorial 1:IntroductionTutorial 2:Tuning Learning RatesTutorial 3:Image ClassificationTutorial 4:Text ClassificationTutorial 5:Learning from Unlabeled Text DataTutorial 6:Text Sequence Taggingfor Named Entity RecognitionTutorial 7:Graph Node Classificationwith Graph Neural NetworksTutorial 8:Tabular Classification and RegressionTutorial A1:Additional tricks, which covers topics such as previewing data augmentation schemes, inspecting intermediate output of Keras models for debugging, setting global weight decay, and use of built-in and custom callbacks.Tutorial A2:Explaining Predictions and MisclassificationsTutorial A3:Text Classification with Hugging Face TransformersTutorial A4:Using Custom Data Formats and Models: Text Regression with Extra RegressorsSome blog tutorials and other guides aboutktrainare shown below:ktrain: A Lightweight Wrapper for Keras to Help Train Neural NetworksBERT Text Classification in 3 Lines of CodeText Classification with Hugging Face Transformers in  TensorFlow 2 (Without Tears)Build an Open-Domain Question-Answering System With BERT in 3 Lines of CodeFinetuning BERT using ktrain for Disaster Tweets Classificationby Hamiz AhmedIndonesian NLP Examples with ktrainby Sandy KhosasiExamplesUsingktrainonGoogle Colab?  See these Colab examples:text classification:a simple demo of Multiclass Text Classification with BERTtext classification:a simple demo of Multiclass Text Classification with Hugging Face Transformerssequence-tagging (NER):NER example usingtransformerword embeddingsquestion-answering:End-to-End Question-Answeringusing the 20newsgroups dataset.image classification:image classification with Cats vs. DogsTasks such as text classification and image classification can be accomplished easily with
only a few lines of code.Example: Text Classification ofIMDb Movie ReviewsUsingBERT[see notebook]importktrainfromktrainimporttextastxt# load data(x_train,y_train),(x_test,y_test),preproc=txt.texts_from_folder('data/aclImdb',maxlen=500,preprocess_mode='bert',train_test_names=['train','test'],classes=['pos','neg'])# load modelmodel=txt.text_classifier('bert',(x_train,y_train),preproc=preproc)# wrap model and data in ktrain.Learner objectlearner=ktrain.get_learner(model,train_data=(x_train,y_train),val_data=(x_test,y_test),batch_size=6)# find good learning ratelearner.lr_find()# briefly simulate training to find good learning ratelearner.lr_plot()# visually identify best learning rate# train using 1cycle learning rate schedule for 3 epochslearner.fit_onecycle(2e-5,3)Example: Classifying Images ofDogs and CatsUsing a PretrainedResNet50model[see notebook]importktrainfromktrainimportvisionasvis# load data(train_data,val_data,preproc)=vis.images_from_folder(datadir='data/dogscats',data_aug=vis.get_data_aug(horizontal_flip=True),train_test_names=['train','valid'],target_size=(224,224),color_mode='rgb')# load modelmodel=vis.image_classifier('pretrained_resnet50',train_data,val_data,freeze_layers=80)# wrap model and data in ktrain.Learner objectlearner=ktrain.get_learner(model=model,train_data=train_data,val_data=val_data,workers=8,use_multiprocessing=False,batch_size=64)# find good learning ratelearner.lr_find()# briefly simulate training to find good learning ratelearner.lr_plot()# visually identify best learning rate# train using triangular policy with ModelCheckpoint and implicit ReduceLROnPlateau and EarlyStoppinglearner.autofit(1e-4,checkpoint_folder='/tmp/saved_weights')Example: Sequence Labeling forNamed Entity Recognitionusing a randomly initializedBidirectional LSTM CRFmodel[see notebook]importktrainfromktrainimporttextastxt# load data(trn,val,preproc)=txt.entities_from_txt('data/ner_dataset.csv',sentence_column='Sentence #',word_column='Word',tag_column='Tag',data_format='gmb',use_char=True)# enable character embeddings# load modelmodel=txt.sequence_tagger('bilstm-crf',preproc)# wrap model and data in ktrain.Learner objectlearner=ktrain.get_learner(model,train_data=trn,val_data=val)# conventional training for 1 epoch using a learning rate of 0.001 (Keras default for Adam optmizer)learner.fit(1e-3,1)Example: Node Classification onCora Citation Graphusing aGraphSAGEmodel[see notbook]importktrainfromktrainimportgraphasgr# load data with supervision ratio of 10%(trn,val,preproc)=gr.graph_nodes_from_csv('cora.content',# node attributes/labels'cora.cites',# edge listsample_size=20,holdout_pct=None,holdout_for_inductive=False,train_pct=0.1,sep='\t')# load modelmodel=gr.graph_node_classifier('graphsage',trn)# wrap model and data in ktrain.Learner objectlearner=ktrain.get_learner(model,train_data=trn,val_data=val,batch_size=64)# find good learning ratelearner.lr_find(max_epochs=100)# briefly simulate training to find good learning ratelearner.lr_plot()# visually identify best learning rate# train using triangular policy with ModelCheckpoint and implicit ReduceLROnPlateau and EarlyStoppinglearner.autofit(0.01,checkpoint_folder='/tmp/saved_weights')Example: Text Classification withHugging Face Transformerson20 Newsgroups DatasetUsingDistilBERT[see notebook]# load text datacategories=['alt.atheism','soc.religion.christian','comp.graphics','sci.med']fromsklearn.datasetsimportfetch_20newsgroupstrain_b=fetch_20newsgroups(subset='train',categories=categories,shuffle=True)test_b=fetch_20newsgroups(subset='test',categories=categories,shuffle=True)(x_train,y_train)=(train_b.data,train_b.target)(x_test,y_test)=(test_b.data,test_b.target)# build, train, and validate model (Transformer is wrapper around transformers library)importktrainfromktrainimporttextMODEL_NAME='distilbert-base-uncased't=text.Transformer(MODEL_NAME,maxlen=500,class_names=train_b.target_names)trn=t.preprocess_train(x_train,y_train)val=t.preprocess_test(x_test,y_test)model=t.get_classifier()learner=ktrain.get_learner(model,train_data=trn,val_data=val,batch_size=6)learner.fit_onecycle(5e-5,4)learner.validate(class_names=t.get_classes())# class_names must be string values# Output from learner.validate()#                        precision    recall  f1-score   support##           alt.atheism       0.92      0.93      0.93       319#         comp.graphics       0.97      0.97      0.97       389#               sci.med       0.97      0.95      0.96       396#soc.religion.christian       0.96      0.96      0.96       398##              accuracy                           0.96      1502#             macro avg       0.95      0.96      0.95      1502#          weighted avg       0.96      0.96      0.96      1502Example: Tabular Classification forTitanic Survival PredictionUsing an MLP[see notebook]importktrainfromktrainimporttabularimportpandasaspdtrain_df=pd.read_csv('train.csv',index_col=0)train_df=train_df.drop(['Name','Ticket','Cabin'],1)trn,val,preproc=tabular.tabular_from_df(train_df,label_columns=['Survived'],random_state=42)learner=ktrain.get_learner(tabular.tabular_classifier('mlp',trn),train_data=trn,val_data=val)learner.lr_find(show_plot=True,max_epochs=5)# estimate learning ratelearner.fit_onecycle(5e-3,10)# evaluate held-out labeled test settst=preproc.preprocess_test(pd.read_csv('heldout.csv',index_col=0))learner.evaluate(tst,class_names=preproc.get_classes())Additional examples can be foundhere.InstallationMake sure pip is up-to-date with:pip install -U pipInstall TensorFlow 2if it is not already installed (e.g.,pip install tensorflow)Installktrain:pip install ktrainThe above should be all you need on Linux systems and cloud computing environments like Google Colab and AWS EC2.  If you are usingktrainon aWindows computer, you can follow thesemore detailed instructionsthat include some extra steps.Supported TensorFlow Versions:ktrainshould currently support any version of TensorFlow at or above to v2.3: i.e.,pip install tensorflow>=2.3.  However, if usingtensorflow>=2.11, then you must only use legacy optimizers such astf.keras.optimizers.legacy.Adam.  The newertf.keras.optimizers.Optimizerbase class is not supported at this time.  For instance, when using TensorFlow 2.11 and above, please usetf.keras.optimzers.legacy.Adam()instead of the string"adam"inmodel.compile.ktraindoes this automatically when using out-of-the-box models (e.g., models from thetransformerslibrary).Additional Notes About InstallationSome optional, extra libraries used for some operations can be installed as needed. (Notice thatktrainis using forked versions of theeli5andstellargraphlibraries in order to support TensorFlow2.)# for graph module:pipinstallhttps://github.com/amaiya/stellargraph/archive/refs/heads/no_tf_dep_082.zip# for text.TextPredictor.explain and vision.ImagePredictor.explain:pipinstallhttps://github.com/amaiya/eli5-tf/archive/refs/heads/master.zip# for tabular.TabularPredictor.explain:pipinstallshap# for text.zsl (ZeroShotClassifier), text.summarization, text.translation, text.speech:pipinstalltorch# for text.speech:pipinstalllibrosa# for tabular.causal_inference_model:pipinstallcausalnlp# for text.summarization.core.LexRankSummarizer:pipinstallsumy# for text.kw.KeywordExtractorpipinstalltextblob# for text.qa.generative_qapipinstallpaper-qa==2.1.1langchain==0.0.240# for text.generative_aipipinstallonpremktrainpurposely pins to a lower version oftransformersto include support for older versions of TensorFlow.  If you need a newer version oftransformers, it is usually safe for you to upgradetransformers, as long as you do itafterinstallingktrain.As of v0.30.x, TensorFlow installation is optional and only required if training neural networks.  Althoughktrainuses TensorFlow for neural network training, it also includes a variety of useful pretrained PyTorch models and sklearn models, which
can be used out-of-the-boxwithouthaving TensorFlow installed, as summarized in this table:FeatureTensorFlowPyTorchSklearntrainingany neural network (e.g., text or image classification)✅❌❌End-to-End Question-Answering(pretrained)✅✅❌QA-Based Information Extraction(pretrained)✅✅❌Zero-Shot Classification(pretrained)❌✅❌Language Translation(pretrained)❌✅❌Summarization(pretrained)❌✅❌Speech Transcription(pretrained)❌✅❌Image Captioning(pretrained)❌✅❌Object Detection(pretrained)❌✅❌Sentiment Analysis(pretrained)❌✅❌GenerativeAI(sentence-transformers)❌✅❌Topic Modeling(sklearn)❌❌✅Keyphrase Extraction(textblob/nltk/sklearn)❌❌✅As noted above, end-to-end question-answering and information extraction inktraincan be used with either TensorFlow (usingframework='tf') or PyTorch (usingframework='pt').How to CitePlease cite thefollowing paperwhen usingktrain:@article{maiya2020ktrain,
    title={ktrain: A Low-Code Library for Augmented Machine Learning},
    author={Arun S. Maiya},
    year={2020},
    eprint={2004.10703},
    archivePrefix={arXiv},
    primaryClass={cs.LG},
    journal={arXiv preprint arXiv:2004.10703},
}Creator:Arun S. MaiyaEmail:arun [at] maiya [dot] net