Package: ntap
ntap: Neural Text Analysis Pipelinentapis a python package built on top oftensorflow,sklearn,pandas,gensim,nltk, and other libraries to facilitate the core functionalities of text analysis using modern methods from NLP.Data loading and Text featurizationAllntapfunctionalities use the Dataset object class, which is responsible for loading datasets from file, cleaning text, transforming text into features, and saving results to file.ntap.data.DatasetDataset(source, tokenizer="wordpunct", vocab_size=5000, embed="glove",
		min_token=5, stopwords=None, stem=False, lower=True, max_len=100,
		include_nums=False, include_symbols=False, num_topics=100, 
		lda_max_iter=500)Parameterssource:str, path to single data file. Supported formats: newline-delimited.json,.csv,.tsv, saved Pandas DataFrame as.pklfiletokenizer:str, select which tokenizer to use. ifNone, will tokenize based on white-space. Options are based onnltkword tokenizers: "wordpunct", ... (others not currently supported)vocab_size:int, keep the topvocab_sizetypes, by frequency. Used in bag-of-words features, as well as neural methods. IfNone, use all of vocabulary.embed:str, select which word embedding to use for initialization of embedding layer. Currently onlygloveis supportedmin_token:int, indicates the minimum size, by number of tokens, for a document to be included after callingclean.stopwords:iterableorstr, set of words to exclude. Default isNone, which excludes no words. Options include lists/sets, as well as strings indicating the use of a saved list:nltkis the only currently supported option, and indicates the defaultnltkEnglish liststem:boolorstr, ifFalsethen do not stem/lemmatize, otherwise follow the stemming procedure named bystem. Options aresnowballlower:bool, ifTruethen cast all alpha characters to lowercasemax_len:int, maximum length, by number of valid tokens, for a document to be included during modeling.Nonewill result in the maximum length being calculated by the existing document setinclude_nums:bool, ifTrue, then do not discard tokens which contain numeric characters. Examples of this include dates, figures, and other numeric datatypes.include_symbols:bool, ifTrue, then do not discard tokens which contain non-alphanumeric symbolsnum_topics:int, sets default number of topics to use ifldamethod is called at a later point.lda_max_iter:int, sets default number of iterations of Gibbs sampling to run during LDA model fittingMethodsThe Dataset class has a number of methods for control over the internal functionality of the class, which are called by Method objects. The most important stand-alone methods are the following:Dataset.set_params(**kwargs):Can be called at any time to reset a subset of the parameters inDatasetTODO: call specific refitting (i.e.__learn_vocab)Dataset.clean(column, remove=["hashtags", "mentions", "links"], mode="remove"):Removes any tokens (before calling tokenizer) matching the descriptions in theremovelist. Then tokenizes documents incolumn, defines the vocabulary, the prunes documents from the Dataset instance that do not match the length criteria. All these are defined by the stored parameters in Datasetcolumn:str, indicates the column name of the text fileremove:listofstr, each item indicates a type of token to remove. IfNoneor list is empty, no tokens are removedmode:str, for later iterations, could potentially store hashtag or links. Currently only option isremoveThe Dataset object supports a number of feature methods (e.g. LDA, TFIDF), which can be called directly by the user, or implicitly during a Method construction (see Method documentation)Dataset.lda(column, method="mallet", save_model=None, load_model=None):Usesgensimwrapper ofMalletjava application. Currently only this is supported, though other implementations of LDA can be added.save_modelandload_modelare currently unsupportedcolumn:str, text columnmethod: only "mallet" is supportedsave_model:str, indicate path to save trained topic model. Not yet implementedload_model:str, indicate path to load trained topic model. Not yet implementedDataset.ddr(column, dictionary, **kwargs):Only method which must be called in advance (currently; advanced versions will store dictionary internallycolumn: column in Dataset containing text. Does not have to be tokenized.dictionary:str, path to dictionary file. Current supported types are.jsonand.csv..dicto be added in a later versionpossiblekwargsincludeembed, which can be used to set the embedding source (i.e.embed="word2vec", but this feature has not yet been added)Dataset.tfidf(column):usesgensimTFIDF implementation. Ifvocabhas been learned previously, uses that. If not, relearns and computes DocTerm matrixcolumn:str, text columnLater methods will include BERT, GLOVE embedding averagesExamplesBelow are a set of use-cases for the Dataset object. Methods likeSVMare covered elsewhere, and are included here only for illustrative purposes.from ntap.data import Dataset
from ntap.models import RNN, SVM

gab_data = Dataset("./my_data/gab.tsv")
other_gab_data = Dataset("./my_data/gab.tsv", vocab_size=20000, stem="snowball", max_len=1000)
gab_data.clean()
other_gab_data.clean() # using stored parameters
other_gab_data.set_params(include_nums=True) # reset parameter
other_gab_data.clean() # rerun using updated parameters

gab_data.set_params(num_topics=50, lda_max_iter=100)
base_gab = SVM("hate ~ lda(text)", data=gab_data)
base_gab2 = SVM("hate ~ lda(text)", data=other_gab_data)Base ModelsFor supervised learning tasks,ntapprovides two (currently) baseline methods,SVMandLM.SVMusessklearn's implementation of Support Vector Machine classifier, whileLMuses eitherElasticNet(supporting regularized linear regression) orLinearRegressionfromsklearn. Both models support the same type of core modeling functions:CV,train, andpredict, withCVoptionally supporting Grid Search.All methods are created using anR-like formula syntax. Base models likeSVMandLMonly support single target models, while other models support multiple targets.ntap.models.SVMSVM(formula, data, C=1.0, class_weight=None, dual=False, penalty='l2', loss='squared_hinge', tol=0.0001, max_iter=1000, random_state=None)

LM(formula, data, alpha=0.0, l1_ratio=0.5, max_iter=1000, tol=0.001, random_state=None)Parametersformula:str, contains a single~symbol, separating the left-hand side (the target/dependent variable) from the right-hand side (a series of+-delineated text tokens). The right hand side tokens can be either a column in Dataset object given to the constructor, or a feature call in the following form:<featurename>(<column>).data:Dataset, an existing Dataset instancetol:float, stopping criteria (difference in loss between epochs)max_iter:int, max iterations during trainingrandom_state:intSVM:C:float, corresponds to thesklearn"C" parameter in SVM Classifierdual:bool, corresponds to thesklearn"dual" parameter in SVM Classifierpenalty:string, regularization function to use, corresponds to thesklearn"penalty" parameterloss:string, loss function to use, corresponds to thesklearn"loss" parameterLM:alpha:float, controls regularization.alpha=0.0corresponds to Least Squares regression.alpha=1.0is the default ElasticNet settingl1_ratio:float, trade-off between L1 and L2 regularization. Ifl1_ratio=1.0then it is LASSO, ifl1_ratio=0.0it is RidgeFunctionsA number of functions are common to bothLMandSVMset_params(**kwargs)CV:Cross validation that implicitly support Grid Search. If a list of parameter values (instead of a single value) is given,CVruns grid search over all possible combinations of parametersLM:CV(data, num_folds=10, metric="r2", random_state=None)SVM:CV(data, num_epochs, num_folds=10, stratified=True, metric="accuracy")num_epochs: number of epochs/iterations to train. This should be revisednum_folds: number of cross foldsstratified: if true, split data using stratified folds (even split with reference to target variable)metric: metric on which to compare different CV results from different parameter grids (if no grid search is specified, no comparison is done andmetricis disregarded)Returns: An instance of ClassCV_ResultsContains information of all possible classification (or regression) metrics, for each CV fold and the mean across foldsContains saved parameter settrainNot currently advised for user application. UseCVinstead`predictNot currently advised for user application. UseCVinsteadExamplesfrom ntap.data import Dataset
from ntap.models import SVM

data = Dataset("./my_data.csv")
model = SVM("hate ~ tfidf(text)", data=data)
basic_cv_results = model.CV(num_folds=5)
basic_cv_results.summary()
model.set_params(C=[1., .8, .5, .2, .01]) # setting param
grid_searched = model.CV(num_folds=5)
basic_cv_results.summary()
basic_cv_results.paramsModelsOne basic model has been implemented forntap:RNN. Later models will includeCNNand other neural variants. All model classes (CNN,RNN, etc.) have the following methods:CV,predict, andtrain.Model formulas using text in a neural architecture should use the following syntax:"<dependent_variable> ~ seq(<text_column>)"ntap.models.RNNRNN(formula, data, hidden_size=128, cell="biLSTM", rnn_dropout=0.5, embedding_dropout=None,
	optimizer='adam', learning_rate=0.001, rnn_pooling='last', embedding_source='glove', 
	random_state=None)Parametersformulasimilar to base methods, but supports multiple targets (multi-task learning). The format for this would be:"hate + moral ~ seq(text)"data:Datasetobjecthidden_size:int, number of hidden units in the 1-layer RNN-type model\cell:str, type of RNN cell. Default is a bidirectional Long Short-Term Memory (LSTM) cell. Options includebiLSTM,LSTM,GRU, andbiGRU(bidirectional Gated Recurrent Unit)rnn_dropout:float, proportion of parameters in the network to randomly zero-out during dropout, in a layer applied to the outputs of the RNN. IfNone, no dropout is applied (not advised)embedding_dropout:str, not implementedoptimizer:str, optimizer to use during training. Options are:adam,sgd,momentum, andrmsproplearning_rate: learning rate during trainingrnn_pooling:strorint. Ifint, model has self-attention, and a Feed-Forward layer of sizernn_poolingis applied to the outputs of the RNN layer in order to produce the attention alphas. If string, possible options arelast(default RNN behavior, where the last hidden vector is taken as the sentence representation and prior states are removed)mean(average hidden states across the entire sequence) andmax(select the max hidden vector)embedding_source:str, eithergloveor (other not implemented)random_state:intFunctionsCV(data, num_folds, num_epochs, comp='accuracy', model_dir=None)Automatically performs grid search if multiple values are given for a particular parameterdata:Dataseton which to perform CVnum_folds:intcomp:str, metric on which to compare different parameter grids (does not apply if no grid search)model_dir: ifNone, trained models are saved in a temp directory and then discarded after script exits. Otherwise,CVattempts to save each model in the path given bymodel_dir.Returns:CV_resultsinstance with best model stats (if grid search), and best parameters (not supported)train(data, num_epochs=30, batch_size=256, indices=None, model_path=None)method called byCV, can be called independently. Can train on all data (indices=None) or a specified subset. Ifmodel_pathisNone, does not save model, otherwise attempt to save model atmodel_pathindices: eitherNone(train on all data) or list ofint, where each value is an index in the range(0, len(data) - 1)predict(data, model_path, indices=None, batch_size=256, retrieve=list())Predicts on new data. Requires a saved model to exist atmodel_path.indices: eitherNone(train on all data) or list ofint, where each value is an index in the range(0, len(data) - 1)retrieve: contains list of strings which indicate which model variables to retrieve during prediction. Includes:rnn_alpha(if attention model) andhidden_states(any model)Returns: dictionary with {variable_name: value_list}. Contents are predicted values for each target variable and any model variables that are given inretrieve.from ntap.data import Dataset
from ntap.models import RNN

data = Dataset("./my_data.csv")
base_lstm = RNN("hate ~ seq(text)", data=data)
attention_lstm = RNN("hate ~ seq(text)", data=data, rnn_pooling=100) # attention
context_lstm = RNN("hate ~ seq(text) + speaker_party", data=data) # categorical variable
base_model.set_params({"hidden"=[200, 50], lr=[0.01, 0.05]}) # enable grid search during CV

# Grid search and print results from best parameters
base_results = base_model.CV()
base_results.summary()

# Train model and save. 
attention_lstm.train(data, model_path="./trained_model")
# Generate preditions for a new dataset
new_data = Dataset("./new_data.csv")
predictions = attention_lstm.predict(new_data, data, column="text", model_path="./trained_model",
							indices=[0,1,2,3,4,5], retrieve=["rnn_alphas"])
for alphas in predictions["rnn_alphas"]:
	print(alphas)  # prints list of floats, each the weight of a word in the ith documentComing soon...MIL(formula, data, ...)not implementedHAN(formula, data, ...)not implementedCNN()not implementedNTAP.data.TagmeNot implementedTagme(token="system", p=0.15, tweet=False)token(str): PersonalTagmetoken. Users can retrieve token byCreating Account. Default behavior ("system") assumesTagmetoken has been set during installation ofNTAP.
Members:get_tags(list-like of strings)Storesabstractsandcategoriesas member variablesreset()abstracts: dictionary of {entity_id:abstract text ...}categories: dictionary of {entity_id:[category1, category2,}data = Dataset("path.csv")
data.tokenize(tokenizer='tweettokenize')
abstracts, categories = data.get_tagme(tagme_token=ntap.tagme_token, p=0.15, tweet=False)
# tagme saved as data object at data.entities
data.background_features(method='pointwise-mi', ...)  # assumes data.tagme is set; creates features
saves features at data.background

background_mod = RNN("purity ~ seq(words) + background", data=data)
background_mod.CV(kfolds=10)NTAP.data.TACITnot implemented. Wrapper around TACIT instanceTACIT(path_to_tacit_directory, params to create tacit session)