Package: omesa
A small framework for reproducible Text Mining research that largely builds
on top ofscikit-learn. Its goal is to make common research procedures fully
automated, optimized, and well recorded. To this end it features:Exhaustive search over best features, pipeline options, to classifier optimization.Flexible wrappers to plug in your tools and features of choice.Completely sparse pipeline through hashing - from data to feature space.Record of all settings and fitted parts of the entire experiment, promoting reproducibility.Dump an easily deployable version of the final model for plug-and-play demos.Read the documentation atreadthedocs.Important NoteThis repository is currently in alpha development, so don’t expect any stable
functionality until this part is removed. Thedevbranch will usually have the
latest (not always stable) version.Front-end PreviewIn ‘front’ a web front-end is being developed that uses a standalone
database for storing models. This provides visualization and comparison of
model performance. Some extra dependencies are introduced, such asbottle,blitzdb,plotlyandlime. Currently only the ‘Results’ section works,
preview below:If you want to take a peek, install all above dependencies, do the following:$cd/dir/to/omesa/examples$python3n_gram.py$cd../front$python3./app.wsgiAnd follow thelocalhostlink that is shown to access the web app. Please
note that this part can be quite unstable. Bug reports are welcome.DependenciesOmesa currently heavily relies onnumpy,scipyandsklearn. To useFrogas a Dutch back-end, we strongly recommend usingLaMachine. For
English, there is aspaCywrapper available.Omesa Only - End-To-End In 2 MinutesWith the end-to-endExperimentpipeline and a configuration dictionary,
several experiments or set-ups can be ran and evaluated with a very minimal
piece of code. One of the test examples provided is that of n-gram
classification of Wikipedia documents. In this experiment, we are provided with
a toy set n_gram.csv that features 10 articles about Machine Learning, and 10
random other articles. To run the experiment, the following configuration is used:ExampleWith the end-to-endExperimentpipeline and a configuration dictionary,
several experiments or set-ups can be ran and evaluated with a very minimal
piece of code. One of the test examples provided is that ofn-gram classificationof Wikipedia documents. In this experiment, we are provided with a toy setn_gram.csvthat features 10 articles about Machine Learning, and 10 random
other articles. To run the experiment, the following configuration is used:fromomesa.experimentimportExperimentfromomesa.featurizerimportNgramsfromomesa.containersimportCSVfromsklearn.naive_bayesimportMultinomialNBExperiment(project="unit_tests",name="gram_experiment",train_data=CSV("n_gram.csv",data="gram",label="label"),lime_data=CSV("n_gram.csv",data="gram",label="label"),features=[Ngrams(level='char',n_list=[3])],classifiers=[{'clf':MultinomialNB()}],"save":("log"))This will cross validate performance on the.csv, selecting text
and label columns and indicating a header is present in the.csvdocument.
We provide theNgramsfunction and parameters to be used as features, and
store the log.OutputThe log file will be printed during run time, as well as stored in the
script’s directory. A sample from the output of the current experiment is as
follows:----Omesa----Config:feature:char_ngramn_list:[3]name:gram_experimentseed:42Sparsetrainshape:(20,1301)Performanceontestset:precisionrecallf1-scoresupportDF0.830.500.6210ML0.640.900.7510avg/total0.740.700.6920Experimenttook0.2seconds----------Adding own FeaturesHere’s an example of the most minimum word frequency feature class:classSomeFeaturizer(object):def__init__(self,some_params):"""Set parameters for SomeFeaturizer."""self.name='hookname'self.some_params=some_paramsdeftransform(self,raw,parse):"""Return a dictionary of feature values."""returnCounter([xforxinraw])This returns a{word: frequency}dict per instance that can easily be
transformed into a sparse matrix.AcknowledgementsPart of the work on Omesa was carried out in the context of theAMiCA(IWT SBO-project 120007) project, funded by the government agency for
Innovation by Science and Technology (IWT).