Reviewerly/exHarmony huggingface dataset:
An Empirical Investigation of Relevant Changes and Automation Needs in Modern Code Review

Abstract Recent research has shown that available tools for Modern Code Review (MCR) are still far from meeting the current expectations of developers. The objective of this paper is to investigate the approaches and tools that, from a developer’s point of view , are still needed to facilitate MCR activities. To that end, we first empirically elicited a taxonomy of recurrent review change types that characterize MCR. The taxonomy was designed by performing three steps: (i) we generated an initial version of the taxonomy by qualitatively and quantitatively analyzing 211 review changes/commits and 648 review comments of ten open-source projects; then (ii) we integrated into this initial taxonomy, topics, and MCR change types of an existing taxonomy available from the literature; finally, (iii) we surveyed 52 developers to integrate eventually missing change types in the taxonomy. Results of our study highlight that the availability of new emerging development technologies (e.g., Cloud-based technologies) and practices (e.g., Continuous delivery) has pushed developers to perform additional activities during MCR and that additional types of feedback are expected by reviewers. Our participants provided recommendations, specified techniques to employ, and highlighted the data to analyze for building recommender systems able to automate the code review activities composing our taxonomy. We surveyed 14 additional participants (12 developers and 2 researchers), not involved in the previous survey, to qualitatively assess the relevance and completeness of the identified MCR change types as well as assess how critical and feasible to implement are some of the identified techniques to support MCR activities. Thus, with a study involving 21 additional developers, we qualitatively assess the feasibility and usefulness of leveraging natural language feedback (automation considered critical/feasible to implement) in supporting developers during MCR activities. In summary, this study sheds some more light on the approaches and tools that are still needed to facilitate MCR activities, confirming the feasibility and usefulness of using summarization techniques during MCR activities. We believe that the results of our work represent an essential step for meeting the expectations of developers and supporting the vision of full or partial automation in MCR.
A Secure Video Steganography Based on the Intra-Prediction Mode (IPM) for H264

Recently, many video steganography algorithms based on the intra-prediction mode (IPM) have been adaptive steganography algorithms. These algorithms usually focus on the research about mapping rules and distortion functions while ignoring the fact that adaptive steganography may not be suitable for video steganography based on the intra-prediction mode; this is because the adaptive steganography algorithm must first calculate the loss of all cover before the first secret message is embedded. However, the modification of an IPM may change the pixel values of the current block and adjacent blocks, which will lead to the change of the loss of the following blocks. In order to avoid this problem, a new secure video steganography based on a novel embedding strategy is proposed in this paper. Video steganography is combined with video encoding. Firstly, the frame is encoded by an original encoder and all the relevant information is saved. The candidate block is found according to the relevant information and mapping rules. Then every qualified block is analyzed, and a one-bit message is embedded during intra-prediction encoding. At last, if the IPM of this block is changed, the values of the residual are modified in order to keep the optimality of the modified IPM. Experimental results indicate that our algorithm has good security performance and little impact on video quality.
Dialogue Relation Extraction with Document-Level Heterogeneous Graph Attention Networks

We propose a heterogeneous graph attention network to address the problem of dialogue relation extraction. Compared with several popular sequence-based and graph-based models, our method shows superior performance on the benchmark dataset DialogRE. The implementation of this work can be found at https://github.com/declare-lab/dialog-HGAT Dialogue relation extraction aims to detect the relation between pairs of entities mentioned in a multi-party dialogue. It plays an essential role in understanding the deep logic of dialogues and facilitating the development of intelligent dialogue systems. We introduce a heterogeneous graph attention network to model the cross-sentence relations in a conversation. This heterogeneous graph attention network has modeled multi-type features of the conversation, such as utterance, word, speaker, argument, and entity type information. We compare our method with several popular baselines such as convolutional neural networks and long short-term memory, experimental results show our model outperforms the state-of-the-art method by 9.4%/7.8% F1 scores, and 6.6%/3.9% $$F1_c$$ scores in both validation and test sets with only 4.0M parameters. In this work, we present an attention-based heterogeneous graph network to deal with the dialogue relation extraction task in an inductive manner. Experimental results on the dataset DialogRE confirm the effectiveness of our method.
Bodyprint—A Meta-Feature Based LSTM Hashing Model for Person Re-Identification

Person re-identification is concerned with matching people across disjointed camera views at different places and different time instants. This task results of great interest in computer vision, especially in video surveillance applications where the re-identification and tracking of persons are required on uncontrolled crowded spaces and after long time periods. The latter aspects are responsible for most of the current unsolved problems of person re-identification, in fact, the presence of many people in a location as well as the passing of hours or days give arise to important visual appearance changes of people, for example, clothes, lighting, and occlusions; thus making person re-identification a very hard task. In this paper, for the first time in the state-of-the-art, a meta-feature based Long Short-Term Memory (LSTM) hashing model for person re-identification is presented. Starting from 2D skeletons extracted from RGB video streams, the proposed method computes a set of novel meta-features based on movement, gait, and bone proportions. These features are analysed by a network composed of a single LSTM layer and two dense layers. The first layer is used to create a pattern of the person’s identity, then, the seconds are used to generate a bodyprint hash through binary coding. The effectiveness of the proposed method is tested on three challenging datasets, that is, iLIDS-VID, PRID 2011, and MARS. In particular, the reported results show that the proposed method, which is not based on visual appearance of people, is fully competitive with respect to other methods based on visual features. In addition, thanks to its skeleton model abstraction, the method results to be a concrete contribute to address open problems, such as long-term re-identification and severe illumination changes, which tend to heavily influence the visual appearance of persons.
Attention Meets Perturbations: Robust and Interpretable Attention With Adversarial Training

Although attention mechanisms have been applied to a variety of deep learning models and have been shown to improve the prediction performance, it has been reported to be vulnerable to perturbations to the mechanism. To overcome the vulnerability to perturbations in the mechanism, we are inspired by adversarial training (AT), which is a powerful regularization technique for enhancing the robustness of the models. In this paper, we propose a general training technique for natural language processing tasks, including AT for attention (Attention AT) and more interpretable AT for attention (Attention iAT). The proposed techniques improved the prediction performance and the model interpretability by exploiting the mechanisms with AT. In particular, Attention iAT boosts those advantages by introducing adversarial perturbation, which enhances the difference in the attention of the sentences. Evaluation experiments with ten open datasets revealed that AT for attention mechanisms, especially Attention iAT, demonstrated (1) the best performance in nine out of ten tasks and (2) more interpretable attention (i.e., the resulting attention correlated more strongly with gradient-based word importance) for all tasks. Additionally, the proposed techniques are (3) much less dependent on perturbation size in AT. Our code is available at https://github.com/shunk031/attention-meets-perturbation
Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

Abstract Purpose Mixed reality (MR) is being evaluated as a visual tool for surgical navigation. Current literature presents unclear results on intraoperative accuracy using the Microsoft HoloLens 1 ® . This study aims to assess the impact of the surgeon’s sightline in an inside-out marker-based MR navigation system for open surgery. Methods Surgeons at Akershus University Hospital tested this system. A custom-made phantom was used, containing 18 wire target crosses within its inner walls. A CT scan was obtained in order to segment all wire targets into a single 3D-model (hologram). An in-house software application (CTrue), developed for the Microsoft HoloLens 1, uploaded 3D-models and automatically registered the 3D-model with the phantom. Based on the surgeon’s sightline while registering and targeting (free sightline /F/or a strictly perpendicular sightline /P/), 4 scenarios were developed (FF-PF-FP-PP). Target error distance (TED) was obtained in three different working axes-( XYZ ). Results Six surgeons (5 males, age 29–62) were enrolled. A total of 864 measurements were collected in 4 scenarios, twice. Scenario PP showed the smallest TED in XYZ -axes mean = 2.98 mm ± SD 1.33; 2.28 mm ± SD 1.45; 2.78 mm ± SD 1.91, respectively. Scenario FF showed the largest TED in XYZ -axes with mean = 10.03 mm ± SD 3.19; 6.36 mm ± SD 3.36; 16.11 mm ± SD 8.91, respectively. Multiple comparison tests, grouped in scenarios and axes, showed that the majority of scenario comparisons had significantly different TED values ( p &lt; 0.05). Y -axis always presented the smallest TED regardless of scenario tested. Conclusion A strictly perpendicular working sightline in relation to the 3D-model achieves the best accuracy results. Shortcomings in this technology, as an intraoperative visual cue, can be overcome by sightline correction. Incidentally, this is the preferred working angle for open surgery.
Graph Convolution Over Multiple Latent Context-Aware Graph Structures for Event Detection

Event detection is a particularly challenging problem in information extraction.The current neural network models have proved that dependency tree can better capture the correlation between candidate trigger words and related context in the sentence.However, syntactic information conveyed by the original dependency tree is insufficient for detecting trigger since the dependency tree obtained from natural language processing toolkits ignores semantic context information.Existing approaches employ a static graph structure based on original dependency tree which is incompetent in terms of distinguishing interrelations among trigger words and contextual words.So how to effectively make use of relevant information while ignoring irrelevant information from the dependency trees remains a challenging research question.To address this problem, we investigate a graph convolutional network over multiple latent context-aware graph structures to perform event detection.We exploit a multi-head attention mechanism on BERT representation and original adjacency matrix to generate multiple latent context-aware graph structures (a ''dynamic cutting'' strategy), which can automatically learn how to select the useful dependency information.Furthermore, we investigate graph convolutional networks with residual connections to combine the local and non-local contextual information.Experimental results on ACE2005 dataset show that our model achieves competitive performances compared with the methods based on dependency tree for event detection.
Exploring deep learning for the analysis of emotional reactions to terrorist events on Twitter

Terrorist events have a substantial emotional impact on the population, and understanding these effects is very important to design effective assistance programs. However, investigating community-wide traumas is a complex and costly task, where most challenges are related to the data collection process. Social media has been used as a relevant source of data to investigate people’s sentiments and ideas. In this article, we study the emotional reactions of Twitter users regarding two terrorist events that occurred in the United Kingdom. The contributions are twofold: a) we experiment two deep learning architectures to develop an emotion classifier, and b) we develop an analysis on tweets related to terrorist events to underst and whether there is an emotional shift due to a terrorist attack andwhether the emotional reactions are dependent on the event, or on the demographics of the users. Both models, based on convolutional and recurrent neural architectures, presented very similar performances. The analyses revealed an emotion shift due to the events and a difference in the reactions to each specific event, where gender is the most significant factor.
Reinforcement Learning for Variable Selection in a Branch and Bound Algorithm

Mixed integer linear programs are commonly solved by Branch and Bound algorithms. A key factor of the efficiency of the most successful commercial solvers is their fine-tuned heuristics. In this paper, we leverage patterns in real-world instances to learn from scratch a new branching strategy optimised for a given problem and compare it with a commercial solver. We propose FMSTS, a novel Reinforcement Learning approach specifically designed for this task. The strength of our method lies in the consistency between a local value function and a global metric of interest. In addition, we provide insights for adapting known RL techniques to the Branch and Bound setting, and present a new neural network architecture inspired from the literature. To our knowledge, it is the first time Reinforcement Learning has been used to fully optimise the branching strategy. Computational experiments show that our method is appropriate and able to generalise well to new instances.
A Unified Hardware Architecture for Convolutions and Deconvolutions in CNN

Deconvolution plays an important role in the state-of-the-art convolutional neural networks (CNNs) for the tasks like semantic segmentation, image super resolution, etc. In this paper, a scalable neural network hardware architecture for image segmentation is proposed. By sharing the same computing resources, both convolution and deconvolution operations are handled by the same process element array. In addition, access to on-chip and off-chip memories is optimized to alleviate the burden introduced by partial sum. As an example, SegNet-Basic has been implemented using the proposed unified architecture by targeting on Xilinx ZC706 FPGA, which achieves the performance of 151.5 GOPS and 94.3 GOPS for convolution and deconvolution respectively. This unified convolution/deconvolution design is applicable to other CNNs with deconvolution.
Smart glasses display device for fluoroscopically guided minimally invasive spinal instrumentation surgery: a preliminary study

Most surgeons are forced to turn their heads away from the surgical field to see various intraoperative support monitors. These movements may result in inconvenience to surgeons and lead to technical difficulties and potential errors. Wearable devices that can be attached to smart glasses or any glasses are novel visualization tools providing an alternative screen in front of the user's eyes, allowing surgeons to keep their attention focused on the operative task without taking their eyes off the surgical field. The aim of the present study was to examine the feasibility of using glasses equipped with a wearable display device that transmits display monitor data during fluoroscopically guided minimally invasive spinal instrumentation surgery.In this pilot prospective randomized study, 20 consecutively enrolled patients who underwent single-segment posterior lumbar interbody fusion (PLIF) at L5-S1 performed using the percutaneous pedicle screw technique were randomly divided into two groups, a group for which the surgeon used a wearable display device attached to regular glasses while performing surgery (smart glasses group) and a group for which the surgeon did not use such a device (nonglasses group). Real-time intraoperative fluoroscopic images were wirelessly transmitted to the display device attached to the surgeon's glasses. The number of head turns performed by the surgeon to view the standard fluoroscopic monitor during procedures and the operative time, estimated blood loss, radiation exposure time, screw placement accuracy, and intraoperative complication rate were evaluated for comparison between the two groups.The number of surgeon head turns to view the fluoroscopic monitor in the smart glasses group was 0.10 ± 0.31 times, which was significantly fewer than the head turns in the nonglasses group (82.4 ± 32.5 times; p < 0.001). The operative and radiation exposure times in the smart glasses group were shorter than those in the nonglasses group (operative time 100.2 ± 10.4 vs 105.5 ± 14.6 minutes, radiation exposure time 38.6 ± 6.6 vs 41.8 ± 16.1 seconds, respectively), although the differences were not significant. Postoperative CT showed one screw perforation in the nonglasses group, and no intraoperative complications were observed in either group.This is, to the authors' knowledge, the first report on the feasibility of using this wearable display device attached to glasses for fluoroscopically guided minimally invasive spinal instrumentation surgery. Smart glasses display devices such as this one may be a valid option to facilitate better concentration on operative tasks by improving ergonomic efficiency during surgery.
Learning Actionness via Long-Range Temporal Order Verification

Current methods for action recognition typically rely on supervision provided by manual labeling. Such methods, however, do not scale well given the high burden of manual video annotation and a very large number of possible actions. The annotation is particularly difficult for temporal action localization where large parts of the video present no action, or background. To address these challenges, we here propose a self-supervised and generic method to isolate actions from their background. We build on the observation that actions often follow a particular temporal order and, hence, can be predicted by other actions in the same video. As consecutive actions might be separated by minutes, differently to prior work on the arrow of time, we here exploit long-range temporal relations in 10–20 min long videos. To this end, we propose a new model that learns actionness via a self-supervised proxy task of order verification. The model assigns high actionness scores to clips which order is easy to predict from other clips in the video. To obtain a powerful and action-agnostic model, we train it on the large-scale unlabeled HowTo100M dataset with highly diverse actions from instructional videos. We validate our method on the task of action localization and demonstrate consistent improvements when combined with other recent weakly-supervised methods.
One-shot Text Field labeling using Attention and Belief Propagation for Structure Information Extraction

Structured information extraction from document images usually consists of three steps: text detection, text recognition, and text field labeling. While text detection and text recognition have been heavily studied and improved a lot in literature, text field labeling is less explored and still faces many challenges. Existing learning based methods for text labeling task usually require a large amount of labeled examples to train a specific model for each type of document. However, collecting large amounts of document images and labeling them is difficult and sometimes impossible due to privacy issues. Deploying separate models for each type of document also consumes a lot of resources. Facing these challenges, we explore one-shot learning for the text field labeling task. Existing one-shot learning methods for the task are mostly rule-based and have difficulty in labeling fields in crowded regions with few landmarks and fields consisting of multiple separate text regions. To alleviate these problems, we proposed a novel deep end-to-end trainable approach for one-shot text field labeling, which makes use of attention mechanism to transfer the layout information between document images. We further applied conditional random field on the transferred layout information for the refinement of field labeling. We collected and annotated a real-world one-shot field labeling dataset with a large variety of document types and conducted extensive experiments to examine the effectiveness of the proposed model. To stimulate research in this direction, the collected dataset and the one-shot model will be released (https://github.com/AlibabaPAI/one_shot_text_labeling).
Random-telegraph-noise-enabled true random number generator for hardware security

The future security of Internet of Things is a key concern in the cyber-security field. One of the key issues is the ability to generate random numbers with strict power and area constrains. "True Random Number Generators" have been presented as a potential solution to this problem but improvements in output bit rate, power consumption, and design complexity must be made. In this work we present a novel and experimentally verified "True Random Number Generator" that uses exclusively conventional CMOS technology as well as offering key improvements over previous designs in complexity, output bitrate, and power consumption. It uses the inherent randomness of telegraph noise in the channel current of a single CMOS transistor as an entropy source. For the first time multi-level and abnormal telegraph noise can be utilised, which greatly reduces device selectivity and offers much greater bitrates. The design is verified using a breadboard and FPGA proof of concept circuit and passes all 15 of the NIST randomness tests without any need for post-processing of the generated bitstream. The design also shows resilience against machine learning attacks performed by the LSTM neural network.
Meta-User2Vec model for addressing the user and item cold-start problem in recommender systems

Abstract The cold-start scenario is a critical problem for recommendation systems, especially in dynamically changing domains such as online news services. In this research, we aim at addressing the cold-start situation by adapting an unsupervised neural User2Vec method to represent new users and articles in a multidimensional space. Toward this goal, we propose an extension of the Doc2Vec model that is capable of representing users with unknown history by building embeddings of their metadata labels along with item representations. We evaluate our proposed approach with respect to different parameter configurations on three real-world recommendation datasets with different characteristics. Our results show that this approach may be applied as an efficient alternative to the factorization machine-based method when the user and item metadata are used and hence can be applied in the cold-start scenario for both new users and new items. Additionally, as our solution represents the user and item labels in the same vector space, we can analyze the spatial relations among these labels to reveal latent interest features of the audience groups as well as possible data biases and disparities.