## 2 Materials and Methods  
### General Concept of TOREAD  
Our approach is of the type of reinforcement learning that uses constant assessments of the detector performance and optimizes the parameters accordingly. For a general overview and full description of this type of machine learning, we refer to a few of the numerous publications available [5; 6; 7; 8; 9; 10], and here, we describe our specific methodology. We illustrate the overall process for the proposed adaptive classification of events in Figure 1.  
Figure 1: “Category” representation of the TOREAD approach to reinforcement.  
The generic description of our approach is illustrated in Figure 1. The green blocks represent the input and the output of an automated system. The blue block \"Decision Manifold\" represents a direct (without any adaptive reinforcement) reactive process. The reinforcement learning loop consists of an embedding (brown oval containing \"Embedding\") step where the decision criteria are relaxed and define the \"Observation Manifold\" (blue box) in order to account for a larger variety of input events (the training data). An assessment procedure (brown rectangle containing \"Assessment\") that can be either supervised or automated, labels the events as true or false (blue box containing \"Labelled Data\"). The labeled set of events provides an input for constructing an envelope of decision criteria (brown oval containing \"Envelope\") that optimizes the detection of the true events and the rejection of the false ones. This decision envelope forms the reinforced, adapted, new decision manifold. The assumption is that our system detects a certain class of events from an input stream of data. Upon a detection, the system will proceed with a predefined action, like raising an alert or simply logging the event. The detection algorithm is encapsulated as \(M_{d}\), the \"Detection Manifold\". The latter can be explicit or implicit (like in the case of trained classifiers), but in all cases, we assume that the event features fall into a certain area, or manifold, that defines them. The adaptive reinforcement involves on-the-move deforming or adapting \(M_{d}\) according to the performance of the detector. This process should lead to both decreasing the number of false detections and avoiding undetected events. As our training process includes only events that are detected and not on screening the whole data history, we introduce an embedding procedure:  
\[M_{d}\to M_{o}\
i M_{d} \tag{1}\]  
This step expands the decision manifold into a larger manifold that may include events missed by the initial detection. The extra events will enter the dynamic training set for the adaptive detector but will not affect directly the output (for example, they will be logged separately and no alerts will be generated).  
The assessment of the whole set of events is outside the scope of this work. In an earlier work [11], a clustering-based algorithm for separating the true from the false events was proposed. Alternatively, a trained operator can perform the assessment. Here, we assume that a labeled set of events is available for the next phase of the training process, the construction of the envelope manifold:  
\[M_{o}\to M^{\prime}{}_{d}\in M_{o} \tag{2}\]  
Finally, the reinforcement cycle closes by substituting the initial detection manifold with the envelope from (2):  
\[M_{d}\Rightarrow M^{\prime}{}_{d} \tag{3}\]  
The above procedure repeats with every new event or in blocks of predefined number of events. It is a learning during the operation scheme that does not require precollected datasets.  
Equations (1)-(3) are very abstract and do not reveal any specific algorithm of performing the process. In Sections 2.2 and 2.3, we provide the specific application to the task of alerting for convulsive epileptic seizures.