Package: sert
Sparse Encoder Representations from Transformerssertis a Python module for machine learning built on top of Tensorflow. It is designed for deep learning on sets, focused on predictive modeling tasks, including both regression and classification, on time series and non-time series data.Many datasets, including standard tabular ones with p columns and N rows, can be represented as a set of observations in the format [row id, column name, value]. This is equivalent to pivoting a wide Nxp table into a long format:Wide Dataframe:NameAppleBananaCherryAlice57NABob342Long Dataframe:NameFruitRatingAliceApple5AliceBanana7AliceCherryNABobApple3BobBanana4BobCherry2However, in some applications, storing data in a wide format is not feasible. For instance, with log data that's stored as [timestamp, user id, action], the data is inherently in a long format. Pivoting this data into a wide format isn't sensible due to the large number of unique values in the timestamp column.sertis designed to work with data in this long format, making it suitable for a variety of problems, including tabular, time series, and log data. The benefits of using long format data withsertare:It allows for the removal of individual observations with missing values without having to remove the entire row. As such,sertcan handle missing values without requiring imputation or the removal of observed data from the same row.It can be applied to datasets that are best represented in a long format, such as log data or multivariate non-aligned time series data.sertleverages the powerful transformer architecture to learn from sets by discerning how to focus on important observations. By utilizing the transformer architecture,sertis parallelizable and scalable to large multivariate time series datasets, unlike RNNs which are sequential and cannot be parallelized.InstallationTo installsert, simply run:pipinstallsertNote: Dependencies will be installed automatically.Dependenciessertrequires:NumPyPandastensorflow (>= 2.0.0)keras_nlp (>= 0.6.0)Quick StartBelow is an example of how to usesertto classify irregular time series. For more examples and details on how to use the package for different problems, please refer to thevignettes.fromsert.modelsimportTimeSERTfromsert.preprocessingimportSeqDataPreparerfromsert.datasetsimportts_classificationimporttensorflowastf# Load the datasetX_train,X_test,y_train,y_test=ts_classification()# Prepare the data for SERTtoken_cap=X_train.groupby('id').size().max()processor=SeqDataPreparer(token_capacity=token_cap)train_input=processor.fit_transform(X_train,index='id',times='time',names='var_name',values='value')test_input=processor.transform(X_test)# Instantiate the modelmodel=TimeSERT(num_var=1,emb_dim=15,num_head=3,ffn_dim=5,num_repeat=2,num_out=y_train.shape[1],task='classification')# Compile the modelcategorical_loss=tf.keras.losses.CategoricalCrossentropy()model.compile(optimizer='adam',loss=categorical_loss,metrics=['accuracy'])# Fit the modelmodel.fit(train_input,y_train,epochs=100,batch_size=250)# Predictionsy_pred=model.predict(test_input)# Evaluate the model_,accuracy=model.evaluate(test_input,y_test)print(f"Test Accuracy:{accuracy:.2f}")LicenseDistributed under the Apache License. SeeLICENSEfor more information.ContactName: Amin Shoari NejadEmail:amin.shoarinejad@gmail.com