Package: tomodapi
Topic Modeling APIThis API is built to dynamically perform training, inference, and evaluation for different topic modeling techniques.
The API grant common interfaces and command for accessing the different models, make easier to compare them.A demo is available athttp://hyperted.eurecom.fr/topic.ModelsIn this repository, we provide:Code to perform training, inference, and evaluation for 9 Topic Modeling packages:LDA from theMalletpackage.LFTM-paper.Doc2TopicGSDMM-paperNon-Negative Matrix factorization (NMF)Hierarchical Dirichlet Processing Model (HDP)Latent Semantic Indexing (LSI)Paragraph Vector Topic Model (PVTM)-paperContext Topic Model (CTM)-paperA set of pre-trained models, downloadable fromhere. **NOTE: Newly trained models are by default stored in.\models, replacing the old ones, unless a new model path is given **Data files containing pre-processed corpus:20ng.txtand20ng_labels.txt, with 11314 news from the20 NewsGroup datasetted.txtwith 51898 subtitles ofTED Talkstest.txtandtest_labels.txt, an extraction of 30 documents from20_ng.txt, used for testing reasonEach model expose the following functions:Training the modelm.train(data,num_topics,preprocessing)# => 'success'Print the list of computed topicsfori,xinenumerate(m.topics):print(f'Topic{i}')forword,weightinzip(x['words'],x['weights']):print(f'-{word}=>{weight}')Access to the info about a specific topicx=m.topic(0)words=x['words']weights=x['weights']Access to the predictions computed on the training corpusfori,pinenumerate(m.get_corpus_predictions(topn=3)):# predictions for each documentprint(f'Predictions on document{i}')fortopic,confidenceinp:print(f'- Topic{topic}with confidence{confidence}')# - Topic 21 with confidence 0.03927058187976461Predict the topic of a new textpred=m.predict(text,topn=3)fortopic,confidenceinpred:print(f'- Topic{topic}with confidence{confidence}')# - Topic 21 with confidence 0.03927058187976461Computing the coherence against a corpus# coherence: Type of coherence to compute, among <c_v, c_npmi, c_uci, u_mass>. See https://radimrehurek.com/gensim/models/coherencemodel.html#gensim.models.coherencemodel.CoherenceModelpred=m.coherence(mycorpus,metric='c_v')print(pred)#{#  "c_v": 0.5186710138972105,#  "c_v_std": 0.1810477961008996,#  "c_v_per_topic": [#    0.5845048872767505,#    0.30693460230781777,#    0.2611738203246824,#    ...#  ]#}Evaluating against a grount truth# metric: Metric for computing the evaluation, among <purity, homogeneity, completeness, v-measure, nmi>.res=m.get_corpus_predictions(topn=1)v=m.evaluate(res,ground_truth_labels,metric='purity')# 0.7825333630516738The possible parameters can differ depending on the model.Use in a Python enviromentInstall this packagepip install tomodapiUse it in a Python scriptfromtomodapiimportLdaModel# init the modelm=LdaModel(model_path=path_location)# train on a corpusm.train(my_corpus,preprocessing=False,num_topics=10)# infer topic of a sentencebest_topics=m.predict("In the time since the industrial revolution the climate has increasingly been affected by human activities that are causing global warming and climate change")topic,confidence=best_topics[0]# get top words for a given topicprint(m.topic(topic))#If themodel_pathis not specified, the library will load/save the model from/undermodels/<model_name>.Web APIA web API is provided for accessing to the library as a serviceInstall dependenciesYou should install 2 dependencies:mallet 2.0.8to be placed inapp\builtinglove.6B.50d.txtto be placed inapp\builtin\gloveUnder UNIX, you can use thedownload_dep.shscript.sh download_dep.shStart the serverpython server.pyDockerAlternatively, you can run a docker container withdocker-compose -f docker-compose.yml upThe container usesmounted volumesso that you can easily update/access to the computed models and the data files.Manual Docker installationdocker build -t hyperted/topic .
docker run -p 27020:5000 --env APP_BASE_PATH=http://hyperted.eurecom.fr/topic/api -d -v /home/semantic/hyperted/tomodapi/models:/models -v /home/semantic/hyperted/tomodapi/data:/data --name hyperted_topic hyperted/topic

# Uninstall
docker stop hyperted_topic
docker rm hyperted_topic
docker rmi hyperted/topicPublicationsIf you find this library or API useful in your research, please consider citing ourpaper:@inproceedings{Lisena:NLPOSS2020,
   author = {Pasquale Lisena and Ismail Harrando and Oussama Kandakji and Raphael Troncy},
   title =  {{ToModAPI: A Topic Modeling API to Train, Use and Compare Topic Models}},
   booktitle = {2$^{nd}$ International Workshop for Natural Language Processing Open Source Software (NLP-OSS)},
   year =   {2020}
}