dataesr/scientific-paragraphs-categorization huggingface dataset:
On the other hand, audio events of interest, such as gun shots, screams and broken glasses are recognized by analyzing the audio stream acquired by a microphone (Audio Event Analysis). Finally, this large amount of data needs to be properly stored (Storing) in a way that the human operator can efficiently interact with the system by submitting different typologies of queries (Querying), whose parameters are defined only at query time (that is exactly in the moment the query is thought). For instance, the objects crossing a given area in a given time interval (dynamic spatio temporal query) or the objects following a particular trajectory, similar to the one manually drawn by the user (query by sketch). A brief description of the above mentioned modules is provided in the following: • Detection: this module is devoted to detect moving objects of interest inside the scene.
Furthermore, in real applications there is often the problem that sounds of interest are superimposed to significant background sounds; thus it might be difficult to separate noise to be ignored from the useful sounds to be recognized. In Section 4 a detailed description of the proposed method will be provided. • Storing: this module stores the large amount of information previously acquired, in terms of both events and trajectories. Its aim is to improve the overall performance of the system during the retrieval phase. This module needs to face with different problems: the former pertains the huge amount of data to be efficiently managed. As a matter of fact, in real scenarios it is required that millions of trajectories, extracted from different cameras, must be stored and that on this wide database a human operator can submit complex queries involving spatial and temporal data, as well as semantic one.
Moreover, it is worth noting that the events can be simply stored by using available relational datasets, while trajectories are much more complex data to deal with, thus customized and enhanced solutions need to be properly defined. The Storing module will be deeply described in Section 3. 3. • Querying: human operator can interact with the system by submitting different typologies of queries, whose parameters are defined only at query time, concerning both spatio-temporal and semantic information. Examples of queries that the user can submit involve the objects crossing a given area in a given time interval (dynamic spatio temporal query) or the objects following a particular trajectory, similar to the one hand drawn by the user (query by sketch). More details related to these interactions will be provided respectively in Sections 3.
3 details the strategy adopted for indexing and storing trajectories data. This chapter is concluded by Section 3. 4 , where the queries that the user is allowed to submit are defined. • Chapter 4 is devoted to analyze the method proposed for recognizing sound events of interest. • Chapter 5 analyzes the results obtained by the proposed modules. In particular, five different standard datasets, detailed in the following of this thesis, have been used in order to prove the effectiveness of the proposed algorithms. • Chapter 6 draws some conclusions and describes future works. Chapter 2.
Storing and Retrieval Another relevant consequence of the scenario we are working on is the possibility of storing a huge amount of trajectories extracted from the scene, each annotated with the properties of the associated object; this is related to the important perspective to efficiently extract synthetic data by suited queries using jointly geometric and temporal information: geometric information refers to the spatial area where the event of interest occurs, while temporal one pertains the time interval associated to the event of interest. For instance, if we are interested in investigated on the theft which took place in the line 4 of the train station at 9pm, we need to find all the objects, and then all the trajectories stored in our database, crossing the portion of the scene where line 4 lies in the selected temporal interval (8:50pm -9:10pm).
However, only a modest attention has been devoted to systems able to store and also retrieve off-line moving object trajectories, able to cope with very large amount of trajectory data and sufficiently general to deal with the needs of different application domains. This is an important and not negligible feature, especially when considering crowded real world scenarios. In these cases it is required that millions of trajectories must be stored and that, on this wide database, the user must be able to submit complex queries involving geometric and temporal data. A few tries have been given in this sense in the last years; for instance, the method proposed in  [71]  aims at counting the number of vehicles crossing the scene at a given time instant. It is achieved by partitioning the scene into a fixed number of zones and by indexing the trajectory data using a multimedia augmented transition network model and a novel multimedia input strings.
A similar approach has been used in  [72] , where the spatial structures of the monitored environment (like its division into regions corresponding to traffic lanes, curb, exit lane, etc. ) are directly stored into the database. The main drawback of the above mentioned methods is related to the fact that the reduction of the computational cost is obtained at the expense of the staticity of the query's schema. It is required that the parameters (both geometrical and temporal) included in queries are pre-determined and cannot be consequently chosen by the user at query time; this implies that the user is forced to use a set of pre-definite queries and the addition of another query typology requires heavy operations by the system designer, involving a system stop and a database recompiling. An example of query supported by the above mentioned systems can be given by asking to the system to retrieve all those vehicles flowing on the different lanes.
e. exactly when the query is thought) the area in which we are interested in is neglected. The main contributions to this problem can be obtained by browsing the literature coming from the database field: while objects' meta-informations can be easily stored using standard tables of well known relational DBMSs, spatially enabled DBMSs can be profitably used to store trajectories represented as a sequence of 3D points. In particular, indexing moving objects databases has been an active research area in the recent past and several solutions have been proposed. [18]  and  [19]  survey many accessing strategies, proposed in the last two decades, which are able to index the past and the current position, as well as methods supporting queries related to the future. A widely adopted solution for bi-dimensional spatial indexing is based on R-Trees  [73] , which hierarchically organize geometric bi-dimensional data representing each object by using its MBR (Minimum Bounding Rectangle), an expression of the object's maximum extents in its coordinate system.
As a consequence, there is a strong interest in those methods which, even using well established 2D solutions, allow to solve the indexing problem in a multi-dimensional space. In this thesis we propose to index 3D trajectories by turning the 3D representations into a set of 2D schemes, so as to make it possible the use of well established and optimized available 2D indexing solutions. In particular, the proposed solution is optimized to solve Range Spatial Queries assumed to be Dynamic (DRSQs), i. e. are formulated in any their part at query time, so allowing the user the important potentiality of extracting from the database non only predefined information but the data he is interest in at any moment of system's use. The contribution of this thesis also includes the definition of different typologies of 3D queries, so general to hopefully cover most of the applicative needs and that can be formulated as a combination of two or more DRSQs.
It is worth pointing out that there is no large consensus in the scientific community on which background subtraction method gives the best results; in  [102]  a comparison involving algorithms belonging to different categories have been performed and the obtained results confirm that  [101]  ranks at the top positions, attaining a high value of the performance indices on all the datasets used for the experimentations. For the above mentioned reason, in this thesis it is used the algorithm proposed in  [101]. A generic background subtraction algorithm is composed by the following steps: • pixel segmentation, which produces a foreground pixel mask from input frames, obtained by thresholding the absolute difference between the current image frame and the background image; • morphological filtering, which is applied to the foreground mask; • blob segmentation, which identifies semantically separated groups of pixels and localizes them by a connected components labeling algorithm.
Conclusion In this section I presented the novel tracking algorithm introduced in this thesis, able to overcome many of the problems induced by the object detection phase, as well as to deal with total or partial occlusions. In order to confirm these characteristics, the algorithm has been tested over two standard datasets, namely PETS and ISSIA Soccer. As I will show in Section 5. 1, the obtained results confirm the effectiveness of the proposed approach in very different environments. Furthermore, this algorithm participated to an international competition on surveillance applications, PETS 2013 Contest  [105] , and the obtained results confirms that it is a very promising algorithm. Furthermore its low computational cost makes this algorithm well suited for real-time applications for behavior analysis.
Visual Behavior Analysis Once trajectories have been extracted (by using the algorithm detailed in Section 3. 1), the proposed system has to analyze them in order to detect dangerous behaviors and then alert the human operator. This responsibility is assigned to the Visual Behavior Analysis module: this module is able to identify during a preliminary learning step typical behaviors occurring in the scene by a clustering algorithm, especially designed for this purpose. One of the main advantages lies in the fact that this step is performed into an unsupervised way, without requiring any knowledge of the human operator about the particular environment. The extracted clusters will be used during the operating phase; in particular, the following interactions with the system are allowed to the human operator: an abnormal behavior framework, which allows the human operator to be informed of abnormal behaviors occurring in the scene, and a query by sketch application, which makes possible an efficient search of the k most similar trajectories to the one hand drawn by the human operator without analyzing the entire dataset.
The scene partitioning algorithm introduced in this thesis is detailed in Subsection 3. 2. 3. • Trajectories preprocessing: each trajectory is represented as a sequence of symbols, namely a string. Each symbol contains those discriminant features (related to position, in terms of crossed zones, speed and shape in each zone) required for discriminate two different trajectories (see Section 3. 2. 4). The similarity between two trajectories is evaluated by using a kernel-based method. The main advantage in this choice lies in the fact that it is possible to combine the kernel with a large class of clustering and machine learning algorithms, which can be expressed using only scalar product between input data (see Subsection 3. 2. 5). • Clustering: given the kernel, a novel tree-based clustering algorithm is applied in order to extract clusters of trajectories inside the scene, each corresponding to a typical normal path.
In particular the following considerations have been considered: on the one hand, two clusters need to encode different behaviors (and then have to contain different trajectories). On the other hand, the risk to have clusters with a very low number of trajectories should be avoided. For these reasons, we choose to use as stop condition a lower bound on the mean squared error (MSE) made when assimilating one trajectory to its cluster: M SE(C t ) = SE(C t )/|C t |. In this way, the system does not need knowledge of the human operator about the environment, but is able to determine the optimum number of clusters starting from the distribution of trajectories. The output of the proposed clustering algorithm on the dataset shown in Figure  3. 25a is depicted in Figure  3. 29.
Conclusion In this section I analyzed the algorithm proposed in this thesis for learning typical trajectory patterns inside a scene by using a novel string-kernel based approach: each trajectory is represented as a string by combining different typologies of information concerning position, speed and shape; the similarity between trajectories is evaluated by defining a novel string kernel metric, which is used by the clustering algorithm proposed in this thesis. Each cluster encodes a kind of prototype of normal trajectories. The tree obtained by the proposed clustering algorithm will be used for a double aim: on the one hand, it allows to recover abnormal trajectories if not enough similar to the clusters obtained during the learning phase (see Section 3. 4. 1). On the other hand, the tree is used for solving into a very efficient way the queries by sketch submitted by the human operator; in fact, the tree allows to find the most k similar trajectories to the one hand drawn by the user without searching in the entire dataset (see Section 3.
Indexing and Storing Engine Once extracted, the raw trajectories need to be properly indexed and stored in order to be efficiently retrieved. The main contribution of the proposed approach is to index 3D trajectories by turning the 3D representations into a set of 2D schemes, so as to make it possible the use of well established and optimized available 2D indexing solutions. In particular, the proposed solution is optimized to solve Range Spatial Queries assumed to be Dynamic (DRSQs), i. e. are formulated in any their part at query time, so allowing the user the important potentiality of extracting from the database non only predefined information but the data he is interest in at any moment of system's use. The contribution of the method also includes the definition of different typologies of 3D queries, so general to hopefully cover most of the applicative needs and that can be formulated as a combination of two or more DRSQs.
Finally, we describe how the retrieving efficiency can be significantly increased using a properly designed segmentation algorithm aimed at improving the selectivity of the proposed indexing strategy. In this section I will focus on the storing engine: after a brief introduction over spatial databases in Subsection 3. 3. 1, some preliminary concepts will be provided in Subsection 3. 3. 2. The indexing engine and the segmentation strategy will be detailed respectively in Subsection 3. 3. 3 and 3. 3. 4. Finally, the physical representation scheme designed for this purpose will be discussed in Subsection 3. 3. 5. More information on the retrieval engine devoted to the interactions with the user will be discussed in Section 3. 4, and in particular in Subsection 3. 4. 3.
Spatial Databases A spatial database defines special data types for geometric objects and allows to store geometric data in regular database tables. In particular, the following three basic geometries, shown in Figure  3. 30, can be defined: points, linestrings and polygons. The main advantage in the use of such databases for managing the above defined spatial entities lies in the fact that special functions and indexes for querying and manipulating that data using something like Structured Query Language (SQL) are provided  [83]. Furthermore, such functions are optimized for answering spatial queries: how far two points differ, whether points fall within a spatial are of interest and so on. A few examples are shown in Figure  3. 31: for instance, function Overlaps verifies if two given polygons share space without being completely contained by each other, while function Crosses verifies if geometries (for instance a polygon and a linestring) have some, but not all, interior points in common.
On the other hand, the most efficient functions available in spatial databases are based on Minimum Bounding Rectangles (MBRs), which play a critical role when two geometries interact. As a matter of fact, it is possible to efficiently verify the interactions between geometries by checking the interactions between the corresponding bounding boxes. It is worth pointing out that this kind of comparison is made by taking advantage on the indices and then without the need of extracting geometries from the database, as in the cases shown in Figure  3. 31. On the other hand, it is clear that the geometries whose MBRs overlap do not necessarily overlap. In order to clarify this concept, an example is shown in Figure  3. 32: on the left two geometries are shown; on the right the corresponding MBRs are depicted.
34c). This clipping algorithm can be easily extended for dealing with 3D trajectories by considering 27 spatial regions, rather than 9. Although the aforementioned algorithm also provides the sub-segment lying inside the box, for simplicity in this discussion from now on we will indicate with the term clipping the boolean operation aimed at verifying if a segment intersects a box. Note that the use of this kind of brute-force approach to solve a DRSQ is a very expensive operation, since it requires to preliminary extract each trajectory from the database and successively verify if at least one segment of each trajectory intersects Q by applying the aforementioned strategies. In real applications the number of trajectories is too high to apply a similar approach, and more efficient approaches, coming from the spatial database field, are thus mandatory: it implies that suitable indexing strategies are necessary to reduce the number of trajectories to be extracted and clipped.
Indexing Engine Before entering into details of the proposed approach, it is worth observing that if a trajectory T k intersects a query box Q, then B k will intersect Q as well: Equation 3. 42 represents a necessary but not sufficient condition, as the opposite is clearly not true. It means that if the right part of the equation is satisfied, we can not assume that the trajectory intersects the query box. On the contrary, if the right part is not verified, we can confirm that the trajectory does not intersect the query box (see Figure  3. 35(d)). This is a very important and not negligible consideration, since it allows to diminish the computational burden of the extraction of trajectories from the database, assuming the set of trajectories satisfying the right part of Equation 3. 42 as the candidate set of trajectories to be extracted and clipped.
49) Consequently, we can state that:    B k | xy ⊂ Q| xy B k | xt ⊂ Q| xt B k | yt ⊂ Q| yt    ⇒ T k ∩ Q = ∅. (3. 50) The aforementioned consideration allows to select all those trajectories which surely satisfy the query, without the need to be clipped, and then without the need to be extracted from the database. According to the considerations stated until this moment and independently on how the projection of trajectories are represented and indexed in the database, the selection of the trajectories intersecting a given query box can be carried out by three distinct successive steps, as shown in Figure  4. 1: • Min-selection: the determination of all the trajectories T i contained into the database D, whose bounding box B i is completely contained inside the query box Q: P S = T i ∈ D|B i ⊂ Q.
(3. 51) It is worth noting that trajectories satisfying Equation  3. 50 can be obtained by only working on their bounding box and checking that the latter is included in Q; the computational burden deriving from the need of extracting the trajectories from the database and successively checking that each its segment is included in Q can be avoided, making this phase very effective from a computational point of view. • Max-selection: the determination of all the trajectories T i , not yet selected in the previous step, whose bounding box projections on the coordinate axes intersect the query box Q: • Clipping: the selection among the trajectories selected in the previous step of all those trajectories really intersecting the query box (since the condition about the bounding boxes is only a necessary condition): M S = T i ∈ D/P S|B i ∩ Q = ∅.
( 3 C = T i ∈ M S|T i ∩ Q = ∅. (3. 53) This is the most expensive step, since it necessarily requires the extraction of each trajectory T i from the dataset and the iteratively check of each segment. The final set of trajectories T satisfying our query will be composed both by the trajectories belonging to P S, selected during the min-selection step without the need to be extracted, and the trajectories C, selected by means of the max selection and the clipping steps: It is important to underline that a 3D indexing has been recently implemented in some of the available spatial databases (like, for instance, Postgis), so allowing to verify the intersection between two boxes, namely the query box and the bounding box trajectory, directly into the 3D space (B k ∩ Q = ∅), without the need to decompose the problem into three bi-dimensional spaces.
Trajectory Representation Scheme It is clear at this point that the retrieval performance of the proposed approach strongly depends on the capability of the system to reduce the number of trajectories to be clipped, and then to increase the number of trajectories satisfying Equations 3. 43 and 3. 51, namely the selectivity of the indexes. Furthermore, it is simple to imagine that the longer are the trajectories in the database the higher will be the size of the corresponding bounding boxes. The rationale of the proposed approach is to preliminarly segment each trajectory into smaller pieces, so as that, in the whole, the min-selection, max-selection and clipping steps are computationally more convenient. The segmentation algorithm recursively segments a trajectory into smaller units (called trajectory units); the generic trajectory unit of the trajectory T k is identified by the ordinal number of its starting point and end point respectively, say r and s, and denoted with T k r,s.
Physical Representation Scheme In the following, a detailed description of the physical schema of the database will be provided. For each trajectory, an instance in the table Object, depicted in Figure  5. 2, is associated. This table contains all the additional information associated to the appearance of an object: its class (person, animal, vehicle and so on) and its dominant color. Both these information are extracted during the tracking phase. It is worth pointing out that any other useful and available information extracted by the tracking algorithm can be very easily stored in the table Object: for instance, the plate of the vehicle or its brand if available and so on. According to the indexing method that we proposed, we need to store, for each acquired trajectory T k , all its component units: a generic unit T k r,s is represented by means of two typologies of information: the sequence of consecutive points and the bounding boxes on the coordinate planes B efficiently stored in PostGIS through a Well-known binary (WKB) representation  [118] ; it is a markup language standardized from the Open Geospatial Consortium (OGC) in the ISO/IEC 13249-3:2011 standard (Information technology -Database languages -SQL multimedia and application packages -Part 3: Spatial).