Package: eventdetector-ts
Universal Event Detection in Time SeriesTable of ContentsIntroductionInstallationQuickstartMake PredictionDocumentationHow to credit our packageFutures WorksReferencesIntroductionWe present a newUniversaldeep-learning supervised method for detecting events in
multivariate time series data. This method combines4distinct
novelties compared to existing deep-learning supervised methods.
Firstly, it is based onregressioninstead of binary classification.
Secondly, itdoes not require labeled datasetswhere each point is
labeled; instead, it only requires reference events defined as time
points or intervals of time. Thirdly, it is designed to berobustthrough the use of a stacked ensemble learning metamodel that
combines deep learning models, from classic Feed-Forward
Neural Networks (FFNs) to the state-of-the-art architectures like
Transformers. By leveraging the collective strengths of multiple
models, this ensemble approach can mitigate individual model
weaknesses and biases, resulting in more robust predictions.
Finally, to facilitate practical implementation, we have developed
this package calledEventDetectorto accompany the proposed method.
It provides a rich support for event detection in time series. We establishmathematicallythat our method is universal, and capable of detecting any type of event with arbitrary precision under mild continuity assumptions on the time series. These events may encompass change points, frauds, anomalies, physical occurrences, and more. We substantiate our theoretical results using the universal approximation theorem. Additionally, we provide empirical validations that confirm our claims, demonstrating that our method, with a limited number of parameters, outperforms other deep learning approaches, particularly for rare events and imbalanced datasets from different domains.InstallationBefore installing this package, please ensure that you haveTensorFlowinstalled in your environment.This package relies onTensorFlowfor its functionality, but does not include it as a dependency to allow users to manage their own TensorFlow installations. You can install TensorFlow via pip withpip install tensorflow.Once TensorFlow is installed, you can proceed with the installation of this package.
Please follow the instructions below:PyPi installationpip install eventdetector-tsManual installationTo get started usingEvent Detector, simply follow the instructions below to install the required packages and
dependencies.Clone the repository:git clone https://github.com/menouarazib/eventdetector.git
cd eventdetectorCreate a virtual environment:python -m venv env
source env/bin/activate  # for Linux/MacOS
env\Scripts\activate.bat  # for WindowsInstall the required packages:pip install -r requirements.txtQuickstart ExamplesData setsTo quickly get started withEventDetector, follow the steps below:You can either download the datasets and event catalogs manually or use the built-in methods for the desired application:Bow Shock Crossings:eventdetector_ts.load_martian_bow_shock()bow_shock_datasetbow_shock_eventsCredit Card Frauds:eventdetector_ts.load_credit_card_fraud()credit_card_datasetcredit_card_eventsNLP:Keyword extraction and the identification of tags for part-of-speech tagging (POS) in textual data.Code Implementations:Credit Card Frauds:fromeventdetector_tsimportload_credit_card_fraud,FFNfromeventdetector_ts.metamodel.meta_modelimportMetaModeldataset,events=load_credit_card_fraud()meta_model=MetaModel(dataset=dataset,events=events,width=2,step=1,output_dir='credit_card_fraud',batch_size=3200,s_h=0.01,models=[(FFN,1)],hyperparams_ffn=(1,1,20,20,"sigmoid"))meta_model.fit()Martian Bow Shock:fromeventdetector_tsimportload_martian_bow_shock,FFNfromeventdetector_ts.metamodel.meta_modelimportMetaModeldataset,events=load_martian_bow_shock()meta_model=MetaModel(output_dir="mex_bow_shocks",dataset=dataset,events=events,width=76,step=1,time_window=5400.0,batch_size=3000,models=[(FFN,1)],hyperparams_ffn=(1,1,20,20,"sigmoid"))meta_model.fit()Performance Evaluation and OutputsComparison of Our Method with Deep Learning MethodsCredit Card FraudsMethodNumber of ParametersPrecisionRecallF1-ScoreCNN[1]119,4570.890.680.77FFN+SMOTE[2]5,5610.790.810.80FFN+SMOTE[3]N/A0.820.790.81Ours1,2010.980.740.85Bow Shock CrossingsMethodNumber of ParametersPrecisionRecallF1-ScoreResNat18[4]29,886,9790.99[0.83 , 0.88][0.91 , 0.94]Ours6,1210.950.960.95Training and Validation LossesThe Figure below showcases the training loss and validation loss of the FFNs on the Bow Shock Crossings and Credit Card Frauds.
The low losses observed in both cases indicate that the metamodel has successfully learned the underlying patterns,
justifying the obtained good metrics.Comparison of Predictedopand TrueopThe Figure below illustrates the comparison between the predicted $op$ values and the true $op$ values on the Bow Shock Crossings and Credit Card Frauds.Distribution of time differences δ(t) between predicted events and ground truth events for Bow Shock Crossings and Credit Card FraudsMake Predictionfromeventdetector_ts.prediction.predictionimportpredictfromeventdetector_ts.prediction.utilsimportplot_predictiondataset_for_prediction=...# Call the 'predict' methodpredicted_events,predicted_op,filtered_predicted_op=predict(dataset=dataset_for_prediction,path='path to output_dir')# Plot the predictionsplot_prediction(predicted_op=predicted_op,filtered_predicted_op=filtered_predicted_op)DocumentationFor a deeper understanding of the parameters presented below,
please refer to our paper available at thislink.Meta ModelThe first step is to instantiate theMetaModelobject with the required arguments:fromeventdetector_ts.metamodel.meta_modelimportMetaModelmeta_model=MetaModel(output_dir=...,dataset=...,events=...,width=...,step=...)For a complete description of the required and optional arguments, please refer to the following tables:Required ArgumentsArgumentTypeDescriptionDefault Valueoutput_dirstrThe name or path of the directory where all outputs will be saved. Ifoutput_diris a folder name, the full path in the current directory will be created.-datasetpd.DataFrameThe input dataset as a Pandas DataFrame.-eventsUnion[list, pd.DataFrame]The input events as either a list or a Pandas DataFrame.-widthintNumber of consecutive time steps in each partition (window) when creating overlapping partitions (sliding windows).-stepintNumber of time steps to advance the sliding window.1width_eventsUnion[int, float]The width of each event. If it's anìnt, it represents the number of time steps that constitute an event. If it's afloat, it represents the duration in seconds of each event. If not provided (None), it defaults to the value ofwidth -1.width -1Optional Arguments: KwargsArgumentTypeDescriptionDefault Valuet_maxfloatThe maximum total time is linked to thesigmavariable of the Gaussian filter. This time should be expressed in the same unit of time (seconds, minutes, etc.) as used in the dataset. The unit of time for the dataset is determined by its time sampling. In other words, thesigmavariable should align with the timescale used in your time series data.(3 x(width -1)x time_sampling) / 2deltaUnion[int, float]The maximum time tolerance used to determine the correspondence between a predicted event and its actual counterpart in the true events. If it's an integer, it represents the number of time steps. If it's a float, it represents the duration in seconds.width_eventsx time_samplings_hfloatA step parameter for adjusting the peak height thresholdhduring the peak detection process.0.05epsilonfloatA small constant used to control the size of set which contains the top models with the lowest MSE values.0.0002paintThe patience for the early stopping algorithm.5t_rfloatThe ratio threshold for the early stopping algorithm.0.97time_windowUnion[int, float]This parameter controls the amount of data within the dataset is used for the training process. If it's an integer, it represents a specific number time steps.  If it's a float, it represents a duration in seconds. By default, it is set to None, which means all available data will be used. However, if a value is provided, the dataset will include a specific interval of data surrounding each reference event. This interval includes data from both sides of each event, with a duration equal to the specifiedtime_window. Setting atime_windowin some situations can offer several advantages, such as accelerating the training process and enhancing the neural networks' understanding of rare events.NonemodelsList[Union[str, Tuple[str, int]]]Determines the type of deep learning models and the number of instances to use. Available models:LSTM,GRU,CNN,RNN_BIDIRECTIONAL,RNN_ENCODER_DECODER,CNN_RNN,FFN,CONV_LSTM1D,SELF_ATTENTION,TRANSFORMER.[(FFN, 2)]hyperparams_ffnTuple[int, int, int, int, str]Specify for the FFN the minimum and the maximum number of layers, the minimum and the maximum number of neurons per layer, and the activation function. The List of available activation functions are ["relu","sigmoid","tanh","softmax","leaky_relu","elu","selu","swish"]. If you passNone, no activation is applied (i.e. "linear" activation:a(x) = x).(1, 3, 64, 256, "sigmoid")hyperparams_cnnTuple[int, int, int, int, int, int, str]Specify for the CNN the minimum and maximum number of filters, the minimum and the maximum kernel size, the minimum and maximum number of pooling layers, and the activation function.(16, 64, 3, 8 , 1, 2, "relu")hyperparams_transformerTuple[int, int, int, bool, str]Specify for Transformer the Key dimension, number of heads, the number of the encoder blocks, a flag to indicate the use of the original architecture, and the activation function.(256, 8, 10, True, "relu")hyperparams_rnnTuple[int, int, int, int, str]Specify for the RNN the minimum and maximum number of recurrent layers,the minimum and the maximum number of hidden units, and the activation function.(1,2, 16, 128,"tanh")hyperparams_mm_networkTuple[int,int,str]Specify for the MetaModel network the number of layers,the number of neurons per layer, and the activation function.(1 ,32,"sigmoid")epochsintThe number of epochs to train different models.256batch_sizeintThe number of samples per gradient update.32fill_nanstrSpecifies the method to use for fillingNaNvalues in the dataset. Supported methods are 'zeros', 'ffill', 'bfill', and 'median'."zeros"type_trainingstrSpecifies the type of training technique to use for the MetaModel. Supported techniques are 'average' and 'ffn'."average"scalerstrThe type of scaler to use for preprocessing the data. Possible values are "MinMaxScaler", "StandardScaler", and "RobustScaler"."StandardScaler"use_kfoldboolWhether to use k-fold cross-validation technique or not.Falsetest_sizefloatThe proportion of the dataset to include in the test split. Should be a value between 0 and 1.0.2val_sizefloatThe proportion of the training set to use for validation. Should be a value between 0 and 1.0.2save_models_as_dot_formatboolWhether to save the models as a dot format file. If set to True, then you should have graphviz software installed on your machine.Falseremove_overlapping_eventsboolWhether to remove the overlapping events or not.TruedropoutfloatThe dropout rate, which determines the fraction of input units to drop during training.0.3last_act_funcstrActivation function for the final layer of each model. If set toNone, no activation will be applied (i.e., "linear" activation:a(x) = x)."sigmoid"The methodfitThe methodfitcalls automatically the following methods:Prepare data for computing the overlapping parameteropThe second thing to do is to prepare the events and the dataset for computingop:meta_model.prepare_data_and_computing_op()Stacking Ensemble Learning PipelineThe third thing to do is to build a stacking learning pipeline using the provided models and hyperparameters:meta_model.build_stacking_learning()Event Extraction OptimizationThe fourth thing to do is to run the Event Extraction Optimization process:meta_model.event_extraction_optimization()Get The Results and PlotsFinally, you can plot the results, which are saved automatically: losses, true/predicted ops, true/predicted events, and delta_t.meta_model.plot_save(show_plots=True)How to credit our packageIf you use our package, please cite the following papers:@misc{azib2023universal,title={UniversalEventDetectioninTimeSeries},author={MenouarAzibandBenjaminRenardandPhilippeGarnierandVincentGénotandNicolasAndré},year={2023},eprint={2311.15654},archivePrefix={arXiv},primaryClass={stat.ML}}@misc{azib2023comprehensive,title={AComprehensivePythonLibraryforDeepLearning-BasedEventDetectioninMultivariateTimeSeriesDataandInformationRetrievalinNLP},author={MenouarAzibandBenjaminRenardandPhilippeGarnierandVincentGénotandNicolasAndré},year={2023},eprint={2310.16485},archivePrefix={arXiv},primaryClass={cs.LG}}Future WorksIn our future works, we aim to enhance our model’s capabilities by predicting events of varying durations. This would be a significant improvement over our current approach, which only predicts the midpoint of events with a fixed duration.References[1] F. K. Alarfaj, I. Malik, H. U. Khan, N. Almusallam, M. Ramzan and M. Ahmed, “Credit Card Fraud Detection Using State-of-the-Art Machine Learning and Deep Learning Algorithms,” in IEEE Access, vol. 10, pp. 39700-39715, 2022, doi: 10.1109/ACCESS.2022.3166891.[2] D. Varmedja, M. Karanovic, S. Sladojevic, M. Arsenovic and A. Anderla, “Credit Card Fraud Detection - Machine Learning methods,” 2019 18th International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Bosnia and Herzegovina, 2019, pp. 1-5, doi: 10.1109/INFOTEH.2019.8717766.[3] E. Ileberi, Y. Sun and Z. Wang, “A machine learning based credit card fraud detection using the GA algorithm for feature selection,” in J Big Data, vol. 9, no. 24, 2022. [Online]. Available:https://doi.org/10.1186/s40537-022-00573-8.[4] I. K. Cheng, N. Achilleos and A. Smith, “Automated bow shock and magnetopause boundary detection with Cassini using threshold and deep learning methods,” Front. Astron. Space Sci., vol. 9, 2022, doi: 10.3389/fspas.2022.1016453.