varunr14/newsummary huggingface dataset:
Automatic lip reading is a challenging task because the visual speech signal is known to be missing some important information such as voicing . We propose an approach to ALR that acknowledges that this information is missing but assumes that it is substituted or deleted in a systematic way that can be modelled . We describe a system that learns such a model and then incorporates it into decoding which is realised as a cascade of weighted finite state transducers . Our results show a small but statistically significant improvement in recognition accuracy . We also investigate the issue of suitable visual units for ALR and show that visemes are sub optimal not but because they introduce lexical ambiguity but because the reduction in modelling units entailed by their use reduces accuracy . 


Summary: A novel technique for automatic lip reading is proposed. A weighted finite state transducer cascade is used incorporating a confusion model. Performance was slightly better than a standard HMM system. The issue of suitable units for automatic lip reading was also studied. It was found that visemes are sub optimal because of reduced contextual modelling.
This paper proposes a globally rotation invariant multi scale co occurrence local binary pattern feature for texture relevant tasks . In MCLBP we arrange all co occurrence patterns into groups according to properties of the co patterns and design three encoding functions to extract features from each group . The MCLBP can effectively capture the correlation information between different scales and is also globally rotation invariant . The MCLBP is substantially different from most existing LBP variants including the LBP the CLBP and the MSJ LBP that achieves rotation invariance by locally rotation invariant encoding . We fully evaluate the properties of the MCLBP and compare it with some powerful features on five challenging databases . Extensive experiments demonstrate the effectiveness of the MCLBP compared to the state of the art LBP variants including the CLBP and the LBPHF . Meanwhile the dimension and computational cost of the MCLBP is also lower than that of the CLBP S M C and LBPHF S M . 


Summary: This paper proposes a globally rotation invariant multi scale co occurrence of LBPs MCLBP . The proposed MCLBP can effectively capture the correlation between the LBPs in different scales. Three globally rotation invariant encoding methods are introduced for MCLBP. The proposed MCLBP performs very well on texture material and medical cell classification.
This paper proposed a new method based on spatial filter banks and discrete wavelet transform for invariant texture classification . The method used a multi resolution analysis method like DWT and applied the proposed filter bank on different resolutions . Then a simple fusion of features on different resolutions was used for invariant texture analysis . A comprehensive study was done to examine the effectiveness of the proposed method . Different datasets with different properties were used in this paper such as Brodatz Outex and KTH TIPS for the evaluation . Local binary pattern methods have been one of the powerful methods in recent years for invariant texture classification . A comparative study was performed with some state of the art LBP methods . This comparison indicated promising results for the proposed approach as compared with the LBP methods . 


Summary: A novel idea has been presented for invariant texture classification. The idea uses a combination of wavelet analysis and spatial filter bank method. The method has been tested on a variety of texture databases.
Learning based hashing methods are becoming the mainstream for approximate scalable multimedia retrieval . They consist of two main components hash codes learning for training data and hash functions learning for new data points . Tremendous efforts have been devoted to designing novel methods for these two components i.e . supervised and unsupervised methods for learning hash codes and different models for inferring hashing functions . However there is little work integrating supervised and unsupervised hash codes learning into a single framework . Moreover the hash function learning component is usually based on hand crafted visual features extracted from the training images . The performance of a content based image retrieval system crucially depends on the feature representation and such hand crafted visual features may degrade the accuracy of the hash functions . In this paper we propose a semi supervised deep learning hashing method for fast multimedia retrieval . More specifically in the first component we utilize both visual and label information to learn an optimal similarity graph that can more precisely encode the relationship among training data and then generate the hash codes based on the graph . In the second stage we apply a deep convolutional network to simultaneously learn a good multimedia representation and a set of hash functions . Extensive experiments on five popular datasets demonstrate the superiority of our DLH over both supervised and unsupervised hashing methods . 


Summary: A semi supervised graph approach is proposed to infer the hash codes. We simultaneously learn visual features and hash functions. Extensive experiments are conducted to exploit the performance of our method.
We digitized three years of Dutch election manifestos annotated by the Dutch political scientist Isaac Lipschits . We used these data to train a classifier that can automatically label new unseen election manifestos with themes . Having the manifestos in a uniform XML format with all paragraphs annotated with their themes has advantages for both electronic publishing of the data and diachronic comparative data analysis . The data that we created will be disclosed to the public through a search interface . This means that it will be possible to query the data and filter them on themes and parties . We optimized the Lipschits classifier on the task of classifying election manifestos using models trained on earlier years . We built a classifier that is suited for classifying election manifestos from 2002 onwards using the data from the 1980s and 1990s . We evaluated the results by having a domain expert manually assess a sample of the classified data . We found that our automatic classifier obtains the same precision as a human classifier on unseen data . Its recall could be improved by extending the set of themes with newly emerged themes . Thus when using old political texts to classify new texts work is needed to link and expand the set of themes to newer topics . 


Summary: We digitized political texts from the 1980s and 1990s. We used these data to learn a classifier that can label more recent political texts. Change of themes over the years affects recall of the learned classifier. But precision is comparable to the precision obtained by a human expert labeller. For political themes a high level of detail seems to be preferred by domain experts.
In this paper we present a new algorithm for the computation of the focus of expansion in a video sequence. Although several algorithms have been proposed in the literature for its computation almost all of them are based on the optical flow vectors between a pair of consecutive frames so being very sensitive to noise optical flow errors and camera vibrations. Our algorithm is based on the computation of the vanishing point of point trajectories thus integrating information for more than two consecutive frames. It can improve performance in the presence of erroneous correspondences and occlusions in the field of view of the camera. The algorithm has been tested with virtual sequences generated with Blender as well as some real sequences from both the public KITTI benchmark and a number of challenging video sequences also proposed in this paper. For comparison purposes some algorithms from the literature have also been implemented. The results show that the algorithm has proven to be very robust outperforming the compared algorithms specially in outdoor scenes where the lack of texture can make optical flow algorithms yield inaccurate results. Timing evaluation proves that the proposed algorithm can reach up to 15fps showing its suitability for real time applications. 


Summary: Development of an algorithm for the estimation of the focus of expansion Algorithm based on the cross ratio property applied to interest point trajectories The algorithm gives accurate focus of expansion localization even for low textured scenes.
Visual speech information plays an important role in automatic speech recognition ASR especially when audio is corrupted or even inaccessible. Despite the success of audio based ASR the problem of visual speech decoding remains widely open. This paper provides a detailed review of recent advances in this research area. In comparison with the previous survey 97 which covers the whole ASR system that uses visual speech information we focus on the important questions asked by researchers and summarize the recent studies that attempt to answer them. In particular there are three questions related to the extraction of visual features concerning speaker dependency pose variation and temporal information respectively. Another question is about audio visual speech fusion considering the dynamic changes of modality reliabilities encountered in practice. In addition the state of the art on facial landmark localization is briefly introduced in this paper. Those advanced techniques can be used to improve the region of interest detection but have been largely ignored when building a visual based ASR system. We also provide details of audio visual speech databases. Finally we discuss the remaining challenges and offer our insights into the future research on visual speech decoding. 


Summary: A detailed review of the recent advances in the area of visual speech decoding. Visual features tackling speaker dependency head poses and temporal information. Dynamic audio visual speech information fusion. Recent techniques of facial landmark localization. Summary of audio visual speech databases and ASR performance on them.
A trustworthy protocol is essential to evaluate a text detection algorithm in order to first measure its efficiency and adjust its parameters and second to compare its performances with those of other algorithms . However current protocols do not give precise enough evaluations because they use coarse evaluation metrics and deal with inconsistent matchings between the output of detection algorithms and the ground truth both often limited to rectangular shapes . In this paper we propose a new evaluation protocol named EvaLTex that solves some of the current problems associated with classical metrics and matching strategies . Our system deals with different kinds of annotations and detection shapes . It also considers different kinds of granularity between detections and ground truth objects and hence provides more realistic and accurate evaluation measures . We use this protocol to evaluate text detection algorithms and highlight some key examples that show that the provided scores are more relevant than those of currently used evaluation protocols . 


Summary: We propose a two level annotation evaluation protocol for text detection algorithms. Algorithms with different granularity outputs are equitably compared. All matching strategies between the ground truth and the detections are handled. Quantity and quality scores are given to describe a detector s behavior. The protocol can manage any irregular text representation.
With the increasing number of videos all over the Internet and the increasing number of cameras looking at people around the world one of the most interesting applications would be human activity recognition in videos . Many researches have been conducted in the literature for this purpose . But still recognizing activities in a video with unrestricted conditions is a challenging problem . Moreover finding the spatio temporal location of the activity in the video is another issue . In this paper we present a method based on a non negative matrix completion framework that learns to label videos with activity classes and localizes the activity of interest spatio temporally throughout the video . This approach has a multi label weakly supervised setting for activity detection with a convex optimization procedure . The experimental results show that the proposed approach is competitive with the state of the art methods . 


Summary: Developing a multi label classification framework with a convex optimization process for activity detection. Histogram correction for activity representation in each class to localize activities in a weakly supervised setting. Proposing a new formulation for matrix completion to deal with classification localization in video. Developing an activity recognition system in a totally weakly supervised multi label setting. Developing a non negative matrix completion framework based on Alternating Direction Method ADM .
The possibility of sharing multimedia contents in easy and ubiquitous way has brought to the creation of multiuser photo albums . Pictures and video sequences taken by different people attending common social events are gathered together into huge sets of heterogeneous multimedia data . These databases require effective compression strategies that exploit the common visual information related to the scene but compensate effectively the differences depending on the acquiring viewpoints camera models and acquisition time instants . The paper presents a predictive coding strategy for multi user photo gallery which initially localizes each picture in terms of viewpoint orientation time and acquired elements . This information permits ordering all the images in a prediction tree and associates to each of them a reference picture . From this structure it is possible to build a predictive coding strategy that exploits the redundant elements between the image to be coded and its reference . Experimental results show an average bit rate reduction up to 75 with respect to HEVC Intra low complexity coding . 


Summary: The paper presents a new coding scheme for photo collections of social events. The approach is intended for server side storage of image databases. Heterogeneous users could contribute to the dataset. The algorithm implements a predictive coder exploiting the visual similarity. The coding efficiency significantly improves with respect to existing solutions.
Recent studies witness the success of Bag of Features frameworks for video based human action recognition . The detection and description of local interest regions are two fundamental problems in BoF framework . In this paper we propose a motion boundary based sampling strategy and spatial temporal co occurrence descriptors for action video representation and recognition . Our sampling strategy is partly inspired by the recent success of dense trajectory based features for action recognition . Compared with DT we densely sample spatial temporal cuboids along a motion boundary which can greatly reduce the number of valid trajectories and preserve the discriminative power . Moreover we develop a set of 3D co occurrence descriptors which take account of the spatial temporal context within local cuboids and deliver rich information for recognition . Furthermore we decompose each 3D co occurrence descriptor at pixel level and bin level and integrate the decomposed components with a multi channel framework which can improve the performance significantly . To evaluate the proposed methods we conduct extensive experiments on three benchmarks including KTH YouTube and HMDB51 . The results show that our sampling strategy significantly reduces the computational cost of point tracking without degrading performance . Meanwhile we achieve superior performance than the state of the art methods . We report 95.6 on KTH 87.6 on YouTube and 51.8 on HMDB51 . 


Summary: A motion boundary based sampling strategy is proposed for dense trajectory. A set of 3D co occurrence descriptors is developed to describe cuboids. Two decomposition strategies are presented to further improve performance. We achieve state of the art results on several human action datasets.
This study aims to compare representations of Japanese personal and corporate name authority data in Japan South Korea China and the Library of Congress in order to identify differences and to bring to light issues affecting name authority data sharing projects such as the Virtual International Authority File . For this purpose actual data manuals formats and case reports of organizations as research objects were collected . Supplemental e mail and face to face interviews were also conducted . Subsequently five check points considered to be important in creating Japanese name authority data were set and the data of each organization were compared from these five perspectives . Before the comparison an overview of authority control in Chinese Japanese Korean speaking countries was also provided . The findings of the study are as follows the databases of China and South Korea have mixed headings in Kanji and other Chinese characters few organizations display the correspondence between Kanji and their yomi romanization is not mandatory in some organizations and is different among organizations some organizations adopt representations in their local language and some names in hiragana are not linked with their local forms and might elude a search . 


Summary: China Korea and LC have mixed headings in Kanji and other Chinese characters. Few organizations display correspondences between Kanji and their yomi. Romanized descriptions are different between organizations. Some names in hiragana might escape a search.
This article focuses on the usability evaluation of biometric recognition systems in mobile devices. In particular a behavioural modality has been used the dynamic handwritten signature. Testing usability in behavioural modalities involves a big challenge due to the number of degrees of freedom that users have in interacting with sensors as well as the variety of capture devices to be used. In this context we propose a usability evaluation that allows users to interact freely with the system while minimizing errors at the same time. The participants signed in a smartphone with a stylus through the different phases in the use of a biometric system training enrolment and verification. In addition a profound study on the automation of the evaluation processes has been done so as to reduce the resources employed. The influence of the users stress has also been studied to obtain conclusions on its impact on both the usability systems in scenarios where the user may suffer a certain level of stress such as in courts banks or even shopping. In brief the results shown in this paper prove not only that a dynamic handwritten signature is a trustable solution for a large number of applications in the real world but also that the evaluation of the usability of biometric systems can be carried out at lower costs and shorter duration. 


Summary: We made usability stress tests on mobile biometrics successfully. We optimized resources through automation. Stress is not a major drawback for handwritten signature recognition. The use of colours as a feedback of the recognition benefits usability and performance.
Enabling process changes constitutes a major challenge for any process aware information system . This not only holds for processes running within a single enterprise but also for collaborative scenarios involving distributed and autonomous partners . In particular if one partner adapts its private process the change might affect the processes of the other partners as well . Accordingly it might have to be propagated to concerned partners in a transitive way . A fundamental challenge in this context is to find ways of propagating the changes in a decentralized manner . Existing approaches are limited with respect to the change operations considered as well as their dependency on a particular process specification language . This paper presents a generic change propagation approach that is based on the Refined Process Structure Tree i.e . the approach is independent of a specific process specification language . Further it considers a comprehensive set of change patterns . For all these change patterns it is shown that the provided change propagation algorithms preserve consistency and compatibility of the process choreography . Finally a proof of concept prototype of a change propagation framework for process choreographies is presented . Overall comprehensive change support in process choreographies will foster the implementation and operational support of agile collaborative process scenarios . 


Summary: Enabling flexibility and change in process choreographies. Proposing change propagation algorithms for process choreographies. Handling transitive change propagation across several partners. Ensuring consistency and compatibility when propagating changes. First comprehensive proof of concept implementation of change propagation framework for process choreographies.
3D face recognition and emotion analysis play important roles in many fields of communication and edutainment . An effective facial descriptor with higher discriminating capability for face recognition and higher descriptiveness for facial emotion analysis is a challenging issue . However in the practical applications the descriptiveness and discrimination are independent and contradictory to each other . 3D facial data provide a promising way to balance these two aspects . In this paper a robust regional bounding spherical descriptor is proposed to facilitate 3D face recognition and emotion analysis . In our framework we first segment a group of regions on each 3D facial point cloud by shape index and spherical bands on the human face . Then the corresponding facial areas are projected to regional bounding spheres to obtain our regional descriptor . Finally a regional and global regression mapping technique is employed to the weighted regional descriptor for boosting the classification accuracy . Three largest available databases FRGC v2 CASIA and BU 3DFE are contributed to the performance comparison and the experimental results show a consistently better performance for 3D face recognition and emotion analysis . 


Summary: A regional bounding spherical descriptor is used for 3D face and emotion analysis. Region segmentation is based on shape index and spherical band on 3D face. The regional descriptors are obtained by projecting regional bounding spheres. A regional and global regression is proposed for the weighted regional descriptor.