dataesr/scientific-paragraphs-categorization huggingface dataset:
(2013) 's works, laser scanner suffers from some drawbacks due to clutter and occlusion. • The combination of radio and vision sensors. Low cost radar are more and more available, and radial information provides accurate object identification. • Stereo vision was also investigated for its ability to capture object gesture and its resilience to illumination changes. As for single modal methods, the authors have pointed out the necessity to enhance the quality of information fusion and sensors and cues calibration. In the study, the authors presented a series of datasets that match with the previously introduced works. The authors discuss the performance measures used to measure the robustness and efficiency of several trackers. They mention two cases: the presence of Ground Truth (GT) data and its absence and how to deal with it.
• The second classification divides the behaviors into four levels depending on their duration only: Motion, Action, Activity and Behavior. In the second part of the review, the authors describe datasets used for group activity recognition such as: BEHAVE, CAVIAR, CVBASE, ETISEO, ETH, UHD, HMDB, SportsVU, PETS, ViF. Using the Group Activity Descriptor Vector (GADV), proposed by Azorin-Lopez et al. (  2016 ), the authors classify the group behaviors by taking into account the number of individuals involved in each of them. This descriptor is obtained after extracting features from trajectories using Neural Networks. Afterwards, the authors present the features used for several group behavior recognition tasks: anomaly detection when this anomaly is caused by a small group within a crowd; the distinction between a normal and an abnormal behavior based on the assumption that normal behaviors last longer; group's characteristics such as the distance between its members, and the velocity of each member.
If in a trajectory the authors face missed locations, they infer them through linear interpolation. From a set of trajectories, they combine the sliding window strategy to hierarchical clustering to find small groups of people. After that, to measure the proximity between small groups, they use the symmetric Hausdorff distance (a distance already used for trajectory analysis  (Wang, Tieu, & Grimson, 2006). The authors undertook their experiments on a self-made dataset of 5 video sequences: SU1, SU2, Artefest, Stadium1, Stadium2. In their experiments, they outperform, in terms of groups detection accuracy, the methods of Corner Clustering  (Rabaud & Belongie, 2006) , and another category of methods:  (Brostow & Cipolla, 2006; Sugimura, Kitani, Okabe, Sato, & Sugimoto, 2009).
After that, they use a Support Vector Machine (SVM) to classify the obtained features to identify pedestrians. The RCNN is trained and tested on the INRIA pedestrian dataset. During the training procedure, they fine-tuned the AlexNet model  (Krizhevsky et al. , 2012)  that was trained on the ILSVRC2012 datasets. To avoid over-fitting, the authors expanded the INRIA dataset with comparable pedestrian data from  Wang, Shi, Song, and Shen (2007) 's work. During the testing phase, they evaluated a model trained on the non-extended INRIA dataset, and a model relying on the Selective Search  (Uijlings, Van De Sande, Gevers, & Smeulders, 2013). The authors' model outperforms both of them. Moreover, it realizes better results than the Histogram of Diagrams (HOG) features and Viola Jones methods. In other words, the model achieves a performance of 10% of false detection rate and 23% of missing rate, which is 23% higher than the HOG features method.
Training and testing were held on the Caltech dataset. During experiments held on the same dataset, an overall model of DeepParts reached a miss rate score of 11. 89%. It was also tested on KITTI dataset, despite being trained on Caltech, and it achieved an Average Precision (AP) score of 58. 67%, on a moderate subset, but seems outperformed by Regionlets. As a perspective, the authors propose to treat DeepParts as a cascade stack over other detectors to increase performance. They also suggest model compression, by including all part detectors within a unique Convolutional Neural Network (CNN). Inspired by the successes brought by the development of Deep Learning-based accurate detectors  (Luo, Tian, Wang, & Tang, 2014)  and by the remarkable high speed of very fast cascades  (Benenson, Mathias, Timofte, & Van Gool, 2012)  when processing image patches,  Angelova et al.
(2015)  built DeepCascade, a combination of very fast cascades with deep neural networks. Specifically, they did not completely copy Benenson's cascades, but modified their constitution to keep only 10% of their stages. To obtain a good model they used a pretrained Deep Neural Network (DNN) on ImageNet dataset. Besides, they used data augmentation. Moreover, to select the best model, they trained their algorithm on three datasets producing three different models: the Caltech Pedestrian dataset, an independently collected dataset, and an extra dataset containing Caltech, ETH and Daimler. The most well performing model resulted from the training on the extra dataset. Tested on Nvidia Tesla K20 GPU, the runtime of their cascade is of 15 Frames per Second (FPS) on 640 × 480 pixels images. Furthermore, it achieved very good results regarding accuracy; they reached the average miss rate score of 26.
2% on the Caltech Dataset. To improve their model,  they encourage future research to include motion information from images and increase the depth of DeepCascade by adding small deep nets and investigating efficiency-accuracy compromises. We saw recently that although Faster RCNN reached excellent performances for object detection  (Ren, He, Girshick, & Sun, 2015) , it is not a perfect pedestrian detector  (Zhang, Lin, Liang, & He, 2016). When we choose a pedestrian detector, we have to make a difficult trade-off between speed and accuracy. Some tried to reduce the side-effects of this compromise to the extreme extent  (Liu et al. , 2016).
After that, they construct a SFM-based collision avoidance model  (Karamouzas, Heil, Van Beek, & Overmars, 2009). Then, they apply the K Nearest Neighbors algorithm to find the K neighbors of a pedestrian. Finally, they use the goal-based coherent filtering to cluster pedestrians sharing the same goal. They compare their method to that of  Solera, Calderara, and Cucchiara (2016) , and they achieve, most of the time, a better precision with frequently comparable recall scores in a reduced computation time. They tested their method on the following datasets: student003, GVEII, and MPT-20X100. Wang et al. (2018)  propose a top-down approach to detect groups of pedestrians. It consists in crowd segmentation. The method is broken down into two steps: detection of feature points using  Tomasi and Kanade (1991) 's method and the computation of their attribute value, and then, the segmentation of the crowd into many groups using the computed attribute values.
The authors tested their method on the CUHK Crowd dataset on various situations: such as structured/unstructured dense/less dense crowds. The authors compare their method to that of  Shao, Change Loy, and Wang (2014) , where they beat them in computation time. However, they did not compare their method to other detectors on CUHK Crowd. Voon et al. (2019)  propose Collective Interaction Filtering (CIF), a top-down clustering approach based on the Expectation-Maximization algorithm that uses trajectories to detect groups. First, they start by extracting pedestrians' tracklets using the Kanade-Lucas-Tomasi (KLT) tracker  (Tomasi & Kanade, 1991). Tracklets are used as inputs to CIF to find tracklets' clusters. Within a cluster, they select the pedestrian that has the longest duration and whose trajectory has a small variance to be the key person.
Second, they compute the degree of connectivity of each person with the key persons. To do so, distances like Distance Connectivity (DC), Occurrence connectivity (OC), Speed correlation (SC), are computed, from the first frame to the last frame, between the key person and the other persons and are stored within an adjacency matrix. After that, they use the EM algorithm to find the persons that are close to the key persons. Finally, they use a group refinement threshold to handle crowds with various densities, structures, and degrees of occlusion. The method is evaluated on the CUHK Crowd dataset. Compared to Collective Transition  (Shao, Loy, & Wang, 2017)  and Coherent Filtering  (Zhou, Tang, & Wang, 2012) , their method obtains better results in terms of the Normalized Mutual Information (NMI) and the Rand Index (RI).
The authors' purpose is to estimate the size of inhomogeneous crowds. The authors segment the crowd using the Dynamic Textures Mixture (DTM)  (Doretto, Chiuso, Wu, & Soatto, 2003)  They extract the following holistic features from the segmented regions: segment features, internal edge features (via the Canny edge detector  (Canny, 1986) , texture features via the Gray-level Co-occurrence Matrix (GLCM)  (Haralick, Shanmugam, & Dinstein, 1973). After that, the Gaussian process regression is used to find the number of pedestrians in each crowd segment. The authors validate their approach on the UCSD Anomaly dataset, which they created. The Receiver Operating Characteristic (ROC) curve had been used to validate the segmentation process of the DTM. During the experiments, the DTM outperforms the NCuts method  (Shi & Malik, 1998).
The Mean Squared Error (MSE) and the absolute error was used to test the Gaussian process that estimates the pedestrians count. In the UCSD Anomaly detection context, the authors show the superiority of their method and the features they extracted over those chosen by  Davies, Yin, and Velastin (1995)  and  Kong, Gray, and Tao (2005). Marsden et al. (2017)  propose ResnetCrowd, a Residual Network (ResNet) architecture to learn many tasks related to crowd analysis. The architecture is intended for Multi Task Learning purposes: learning simultaneously crowd counting and crowd density estimation as well as violent behavior classification. The architecture is based on ResNet18  (He, Zhang, Ren, & Sun, 2016) , which is originally pretrained on the ImageNet dataset. Globally, the architecture is the same.
Still, the authors modified it slightly to get accurate task-specific outcomes. The architecture is trained, validated and tested on the selfdeveloped dataset, Multi Task Crowd dataset, that we present later on in this review in Section 5. 2. The computations were undertaken using the Nvidia Gefore GTX 970 GPU. The architecture was tested, in terms of Mean Absolute Error (MAE), Mean Squared Error (MSE), and Area Under the Curve (AUC), and compared to state-of-the-art methods on other datasets, such as UCF Crowd 50, WWW Crowd test set  (Shao, Kang, Change Loy, & Wang, 2015)  and the UMN Anomaly dataset. Their experiments show that architectures that are trained to perform many close-related tasks, yield better results for violence recognition and density estimation than those trained solely on a single task.
However, single task trained architectures outperform ResnetCrowd on crowd counting. It is worth noting that the authors' method is massively outperformed by  Zhang, Zhou, Chen, Gao, and Ma (2016)  In the same line,  Sindagi and Patel (2017)  propose a Cascade of CNNs to jointly learn end-to-end crowd counting and density estimation. The purpose of this work is to yield models that are resilient to scale and appearance variations, and that can classify crowd counts based on the density estimation of the scene. To do so, the network learns relevant discriminative global features to estimate density maps. The architecture of their model consists into two stages. It contains shared convolutional layers, and a high-level prior stage. Training and evaluation were performed on Nvidia GTX TITAN-X GPU, on two publicly available datasets: ShanghaiTech  (Zhang, Zhou et al.
, 2016)  and UCF Crowd 50  (Idrees et al. , 2013). Compared to other methods on these datasets, the method outperforms state-of-the-art methods in terms of the Mean Squared Error (MSE) and the Mean Absolute Error (MAE) on ShanghaiTech. However, it is outperformed by Onoro-Rubio and López-Sastre (2016)'s and  Walach and Wolf (2016) 's in terms of MSE, on UCF CROWD 50. Liu et al. (2019)  propose an end-to-end multi-scale trainable deep architecture which relies on density estimation for crowd counting. Their method extracts features from different unit areas on the image using various receptive field sizes depending on the image perspective to take into account scale variations. Based on the context, they do not predefine image patches suiting every scale, but rather weight each extracted feature to anticipate scale variations, and then fuse the multi-scale information.
Their method works well on uncalibrated cameras, but leverages the presence of calibration information. They use a pretrained VGG-16 network to extract features. They obtain scaleaware features using Spatial Pyramid Pooling  (He, Zhang, Ren, & Sun, 2015)  by taking into consideration 4 different scales. The extracted features from the different scales are then concatenated. The performance of the network is reinforced when the calibration information is available. To leverage the existence of the calibration information, the authors use a supplementary branch, which consists in a truncated VGG network, dedicated to extract features from a perspective map that encodes the number of pixels per meter. They held their experiments on the following datasets: ShanghaiTech, 1  WorldExpo'10, UCF Crowd Counting 50, UCF QNRF, 2  and the ad hoc Venice dataset.
To evaluate their methods, they use the mean absolute error (MAE) and the root mean squared error (RMSE). Compared to state-of-the-art methods, they found that their method perform well on densely crowded scenes but has comparable results and is sometimes outperformed in less dense scenes. Wang et al. (2019)  propose a data collector and labeler that generates crowded scenes and automatically annotates them. This helps them to create a synthetic dataset named GTA 5 Crowd Counting (GCC). Second, they propose a crowd counting network, named Spatial Fully Convolutional Network (SFCN), which they pretrain on the GCC dataset and then finetune it on real data. They obtain their best results using the ResNet101 backbone for the crowd counter on UCF-QNRF, ShanghaiTech A and B, UCF Crowd Counting 50.
Finally, they propose a domain adaptive crowd counter to convert the synthetic data to real data adapted for each target real dataset. After that, they train the SFCN on the converted data and test it on the real data. The converter they use is SSIM Embedding Cycle Generative Adversarial Network, SE Cycle GAN. It is equipped with the Structural Similarity Index (SSIM) loss. This loss measures the similarity between two images by comparing their local patterns. They evaluated their method using the Mean Squared Error (MSE), Mean Absolute Error (MAE), SSIM measure, and the Peak Signal to Noise Ratio (PSNR) measure on the following datasets: UCF-QNRF, UCF_CC_50, ShanghaiTech A/B, and on WorldExpo'10. Their method outperforms Cycle GAN  (Zhu, Park, Isola, & Efros, 2017)  and SFCN (without domain adaptation) in every dataset.
They use compare different existing counter networks: MCNN  (Zhang, Zhou et al. , 2016) , FCN-7c  (Kang & Chan, 2018) , SFCN  (Wang et al. , 2019)  and CSRNet  (Li, Zhang, & Chen, 2018). A combined loss for refinement and counting is computed during the training process to jointly train the counter and the refiner. Experiments were undertaken on the following datasets: ShanghaiTech A and B, WorldExpo'10, UCF-QNRF, and the methods were evaluated using the Mean Absolute Error (MAE) and the Root Mean Squared Error (RMSE). As we can see from the above-mentioned works  (Marsden et al. , 2017; Sindagi & Patel, 2017) , Multi Task Learning is a good idea for learning/fine-tuning simultaneously several tasks of crowd statistics. However, the training should be undertaken on various types of datasets to yield a good-performing model.
The authors posit the action detection problem using Multiple Instance Learning (MIL) by considering clips containing a specific action as positive bags and clips not containing it as negative bags. They annotate only one positive clip with an action cuboid. They solve the MIL problem by using a greedy K Nearest Neighbor (KNN) approach. Their method is validated on the CMU  (Ke, Sukthankar, & Hebert, 2007)  and the i-LIDS  (Branch, 2006)  datasets. The evaluation metrics they work with are the precision-recall curve and the mean average precision (MAP). In their pioneering work,  Baccouche et al. (2011)  propose one of the first uses of 3D CNNs in the classification of video clips for human action recognition. They learn spatio-temporal features from action clips using 3D Convolutional Neural Networks (CNN), after extending the 2D convolutions to 3D.
After that, they use Recurrent Neural Networks (RNN) with Long-short-term memory (LSTM) units to classify each clip by making use of the temporal evolution of its features. The features extraction architecture (3D CNNs) contains 10 layers alternating convolutions, rectification, and sub-sampling in 8 layers and ending by 2 fully connected network. The architecture contains 17,169 trainable parameters. It is trained by online Backpropagation with momentum  (LeCun, Bottou, Bengio, & Haffner, 1998)  ''adapted to weight sharing''. The classifier architecture (RNN) contains 50 LSTM units. It contains 25,000 trainable parameters and is trained using online Backpropagation through time with momentum  (Gers, Schraudolph, & Schmidhuber, 2002). The approach had been tested on the KTH dataset  (Schuldt, Laptev, & Caputo, 2004)  which contains 6 classes of video clips and on which they obtain good results compared to existing methods.
They follow the evaluation protocol proposed by  Gao, Chen, Hauptmann, and Cai (2010)  and relied on cross-validation with 5 runs. Tran et al. (2018)  use a factorized version of 3D Residual Nets (ResNets) to model separately spatial and temporal components of a video clip. They use R(2+1)D blocks, which are convolutional spatiotemporal blocks, trained from scratch on two popular datasets intended for individual-scene action recognition: Sports-1 m and Kinetics. After that, they fine-tuned their blocks on two other well-known datasets: UCF-101 and HMDB-51. Amongst other models, the authors compared their model variants to several I3D variants  (Carreira & Zisserman, 2017). The best authors' model outperforms, in terms of accuracy, all the methods, except one, the I3D two-stream. This latter slightly outperforms R(2+1)D two-stream on UCF-101 and HMBD-51.
They trained their model on a self-made 4D Action dataset depicting a scene viewed from multiple angles. Although their method can be used in real-time in a multi-cameras setup, the use of RGBD cameras is not enough widespread to make their algorithm applicable in every situation. They tested their method in terms of accuracy, revised accuracy (RAcc), and the confusion matrix. Wei et al. (2020)  support that the crowd type stems from the crowd mood and the crowd behavior. Because of this, they propose a model representation of a crowd, summarized into the following triad : Behavior-Mood-Organized (BMO). They perceive three crowd types, each with a number of characteristics to which they associate a rule-based alert system: 1. heterogeneous crowds which originate for ordinary reasons in common crowded places such as railway stations and supermarkerts, should not be continuously controlled, 2.
homogeneous crowds which rise from demonstrations, urban parades or during sports events, should be under control but with a certain restraint, 3. violent crowds which are an exacerbation of a political or a social demonstration must be controlled. To learn this model representation they propose the Crowd Type Recognition Network (CTRN). The CTRN is a two stream network architecture. Each stream is a Visual Geometry Group network of 5 convolutional layers (VGG16) that was pretrained on the ImageNet dataset. The first stream is fed with a static red-greenblue (RGB) image of a scene, and the second stream is fed with a motion map of a scene that contains trajectory features. The CTRN method is included within an emergency alert system that send alerts when the BMO triad satisfies certain conditions.
The method is trained and tested onto the CUHK crowd and the normal-abnormal datasets. CUHK Crowd is divided into homogeneous and heterogeneous crowds, and the normal-abnormal crowds, is a local dataset, that contains homogeneous and heterogeneous crowds, but essentially violent crowds. The BMO representation of the crowd, which is a good start to differentiate multiple crowd types situations, had been reduced to a three classes classifier. Hence, the CTRN and the alert system will not be able to catch special cases scenarios where heterogeneous crowds would be endangered by the abnormal behavior of individual elements. The evaluation metrics that were used are the Receiver Operating Characteristic (ROC) curve, the Area Under the Curve (AUC), and the Mean Accuracy. Wang and O'Sullivan (  2016 ) propose a Spatio-temporal Hierarchical Dirichlet Process (STHDP) that is a non-parametric Bayesian method to learn spatial and temporal patterns in order to detect activities and anomalies in video data.
Time is modeled in a non-Markovian continuous fashion to handle the varying time duration of each activity. The STHDP is an unsupervised clustering method that does not require much prior knowledge about the crowd dynamics. In this situation, the number of clusters is not predetermined. The type of clusters are trajectory clusters. The authors do not group trajectories using a distance metric but cluster individual observations of trajectories from frame to frame. The method detects the occurrence of an activity, its duration, its fading, and its disappearance. The method had been trained and tested on a synthetic data and on real data. Three datasets were used for the real data: 1. Edinburgh dataset (Forum), 2. MIT Carpark (Carpark), 3. New York Central Terminal (Train Station). On these datasets, STHDP was compared to two non-recent methods: MOTIF  (Emonet, Varadarajan, & Odobez, 2011)  and Dual Hierarchical Dirichlet Processes (DHDP)  (Wang, Ma, Ng, & Grimson, 2011).
Although, it is slower than MOTIF and as fast as DHDP, it performs slightly better than DHDP and is highly more accurate than MOTIF. Yan et al. (2019)  propose a crowd video captioning approach applied on off-site spectators: a type of crowds that they consider as neglected by research studies. The purpose is to generate 8 different comments describing the density and the behavior of a crowd: someone walk in, someone run in, someone walk out, someone run out, many people walk in, many people walk out, many people run in, many people run out. As the possible classes suggest, these off-site spectators crowds occur at entrances or exits of stadiums, theaters, rallies, etc. They apply their methods on the WorldExpo'10 dataset, and they evaluate them using the following evaluation metrics for captioning: Cider, Meteor, Bleu, Rouge.
They propose a pipeline of an encoderdecoder made up of a feature extractor (the encoder) that feeds a sequence-to-sequence network that converts video to text (the decoder) S2VT. The feature extractor is either a 3D ConvNets (C3D) network pretrained on the Sports-1 m dataset, or ResNet-152, Inception V3/V4, each of them trained on ImageNet. The S2VT is a 2-layer Recurrent Neural Network (RNN) whose cells are either LSTM or GRU. They evaluate all the possible combinations, in terms of accuracy and the above-mentioned metrics, to select the best combination which ends up to be Inception V3 as feature extractor coupled with GRU as cells for the S2VT to generate comments. Ullah et al. (2016)  work on crowd behavior identification. They intend to identify 5 types of behaviors which are: lane, arch/ring, bottleneck, blocking, fountainhead.