Reviewerly/exHarmony huggingface dataset:
Topic Scene Graph Generation by Attention Distillation from Caption

If an image tells a story, the image caption is the briefest narrator. Generally, a scene graph prefers to be an omniscient "generalist", while the image caption is more willing to be a "specialist", which outlines the gist. Lots of previous studies have found that a scene graph is not as practical as expected unless it can reduce the trivial contents and noises. In this respect, the image caption is a good tutor. To this end, we let the scene graph borrow the ability from the image caption so that it can be a specialist on the basis of remaining all-around, resulting in the socalled Topic Scene Graph. What an image caption pays attention to is distilled and passed to the scene graph for estimating the importance of partial objects, relationships, and events. Specifically, during the caption generation, the attention about individual objects in each time step is collected, pooled, and assembled to obtain the attention about relationships, which serves as weak supervision for regularizing the estimated importance scores of relationships. In addition, as this attention distillation process provides an opportunity for combining the generation of image caption and scene graph together, we further transform the scene graph into linguistic form with rich and free-form expressions by sharing a single generation model with image caption. Experiments show that attention distillation brings significant improvements in mining important relationships without strong supervision, and the topic scene graph shows great potential in subsequent applications.
Breaking the Information Cocoon: When Do People Actively Seek Conflicting Information?

Abstract Influenced by existing beliefs, people are often biased in favor of information that supports their attitudes, intensifying the information cocoon effect. To break information cocoon and promote open‐minded searches, it is important to guide seekers to approach to conflicting information and formulate critical thinking. This study aimed to examine when and why people actively seek conflicting information using grounded theory. Specifically, we interviewed 13 people who have experiences of actively seeking conflicting information. All the transcript data were coded for analysis. The results demonstrated that context, user characteristics, and motivation were the three main factors that led users to seek conflicting information actively and influenced seekers' preferences of information. Finally, we define this type of search tasks as “attitude tasks,” and call for more attention on this type of tasks.
Practical Non-interactive Publicly Verifiable Secret Sharing with Thousands of Parties

Non-interactive publicly verifiable secret sharing (PVSS) schemes enables (re-)sharing of secrets in a decentralized setting in the presence of malicious parties. A recently proposed application of PVSS schemes is to enable permissionless proof-of-stake blockchains to "keep a secret" via a sequence of committees that share that secret. These committees can use the secret to produce signatures on the blockchain's behalf, or to disclose hidden data conditioned on consensus that some event has occurred. That application needs very large committees with thousands of parties, so the PVSS scheme in use must be efficient enough to support such large committees, in terms of both computation and communication. Yet, previous PVSS schemes have large proofs and/or require many exponentiations over large groups. We present a non-interactive PVSS scheme in which the underlying encryption scheme is based on the learning with errors (LWE) problem. While lattice-based encryption schemes are very fast, they often have long ciphertexts and public keys. We use the following two techniques to conserve bandwidth: First, we adapt the Peikert-Vaikuntanathan-Waters (PVW) encryption scheme to the multi-receiver setting, so that the bulk of the parties' keys is a common random string. The resulting scheme yields $$\varOmega (1)$$ amortized plaintext/ciphertext rate, where concretely the rate is $$\approx 1/60$$ for 100 parties, $$\approx 1/8$$ for 1000 parties, and approaching 1/2 as the number of parties grows. Second, we use bulletproofs over a DL-group of order about 256 bits to get compact proofs of correct encryption/decryption of shares. Alternating between the lattice and DL settings is relatively painless, as we equate the LWE modulus with the order of the group. We also show how to reduce the the number of exponentiations in the bulletproofs by applying Johnson-Lindenstrauss-like compression to reduce the dimension of the vectors whose properties must be verified. An implementation of our PVSS with 1000 parties showed that it is feasible even at that size, and should remain so even with one or two order of magnitude increase in the committee size.
Comprehensive improvement of camera calibration based on mutation particle swarm optimization

In order to meet the requirements of high-precision measurement, the method of improving camera calibration is studied. In the calibration process, the quality of the calibration image, the extraction accuracy of the calibration image corner and the nonlinear optimization effect of the camera linear parameters directly affect the calibration accuracy. First of all, in order to solve the problems in image acquisition, especially in the case of over exposure, an adaptive gamma correction method is designed to automatically adjust the image brightness, and enhance the contrast of black and white grid to improve the image acquisition quality. Secondly, a sub-pixel corner extraction algorithm based on homography matrix mapping is designed, which overcomes the error and omission of Harris corner extraction algorithm, and improves the accuracy of corner extraction. At last, adaptive weight and mutation particle swarm optimization algorithm are studied to optimize the camera parameters. Compared with other optimization algorithms, this optimization algorithm needs less parameter settings, fast convergence speed, and can obtain more accurate camera parameters. The average calibration error is 0.038 pixels.
The KIT Bimanual Manipulation Dataset

Learning models of bimanual manipulation tasks from human demonstration requires capturing human body and hand motions, as well as the objects involved in the demonstration, to provide all the information needed for learning manipulation task models on symbolic and subsymbolic level. We provide a new multi-modal dataset of bimanual manipulation actions consisting of accurate human whole-body motion data, full configuration of both hands, and the 6D pose and trajectories of all objects involved in the task. The data is collected using five different sensor systems: a motion capture system, two data gloves, three RGB-D cameras, a headmounted egocentric camera and three inertial measurement units (IMUs). The dataset includes 12 actions of bimanual daily household activities performed by two healthy subjects with a large number of intra-action variations and three repetitions of each action variation, resulting in 588 recorded demonstrations. A total of 21 household items are used to perform the various actions. In addition to the data collection, we developed tools and methods for the standardized representation and organization of multi-modal sensor data in large-scale human motion databases. We extended our Master Motor Map (MMM) framework to allow the mapping of collected demonstrations to a reference model of the human body as well as the segmentation and annotation of recorded manipulation tasks. The dataset includes raw sensor data, normalized data in the MMM format and annotations, and is made publicly available in the KIT Whole-Body Human Motion Database.
Occlusion-aware Bi-directional Guided Network for Light Field Salient Object Detection

Existing light field based works utilize either views or focal stacks for saliency detection. However, since depth information exists implicitly in adjacent views or different focal slices, it is difficult to exploit scene depth information from both. By comparison, Epipolar Plane Images (EPIs) provide explicit accurate scene depth and occlusion information by projected pixel lines. Due to the fact that the depth of an object is often continuous, the distribution of occlusion edges concentrates more on object boundaries compared with traditional color edges, which is more beneficial for improving accuracy and completeness of saliency detection. In this paper, we propose a learning-based network to exploit occlusion features from EPIs and integrate high-level features from the central view for accurate salient object detection. Specifically, a novel Occlusion Extraction Module is proposed to extract occlusion boundary features from horizontal and vertical EPIs. In order to naturally combine occlusion features in EPIs and high-level features in central view, we design a concise Bi-directional Guiding Flow based on cascaded decoders. The flow leverages generated salient edge predictions and salient object predictions to refine features in mutual encoding processes. Experimental results demonstrate that our approach achieves state-of-the-art performance in both segmentation accuracy and edge clarity.
College Physical Education Teaching Aided by Virtual Reality Technology

In recent years, College PE has received more and more attention. Traditional PE and teaching cannot meet the demand of students in today’s society and cannot attract students’ interest in sports. Under the current situation that the country actively promotes information science education, there is an urgent need to innovate PE teaching methods. VRT is very important to support college PE. In order to improve the quality of university PE and increase students’ interest in sports, this article uses the Internet of Things as a research background and 5G technology as a research topic. It provides technical support for the Internet of Things and research sports colleges on the Internet of Things. This study builds a virtual sports teaching environmental education through VRT. Experimental data were collected by sensors and other types of infrastructure. The experiment counted the number of sports safety incidents that occurred before and after the application of VRT in five universities of A, B, C, D, and E, collected and analyzed the teaching data of PE teachers in five universities, and studied the changes in student learning and interest in sports. These experimental data are used to analyze the impact of the application of VRT in PE on sports safety, PE teachers, and students. The final result shows that, after applying VRT in PE, the five schools have reduced the number of safety incidents in sports by more than 65%. VRT has the greatest impact on the teaching philosophy of PE teachers and the least impact on teachers’ teaching management. After the application of VRT, the number of students who independently engage in PE and sports has increased by more than 20%, and the students’ interest in sports learning and sports has increased. Finally, the application of VRT in the PE of the five schools accounted for 25%, 21%, 11%, 35%, and 30%, respectively. The application of VRT will also indirectly affect the quality of PE in universities.
Horizontal Side-Channel Vulnerabilities of Post-Quantum Key Exchange and Encapsulation Protocols

Key exchange protocols and key encapsulation mechanisms establish secret keys to communicate digital information confidentially over public channels. Lattice-based cryptography variants of these protocols are promising alternatives given their quantum-cryptanalysis resistance and implementation efficiency. Although lattice cryptosystems can be mathematically secure, their implementations have shown side-channel vulnerabilities. But such attacks largely presume collecting multiple measurements under a fixed key, leaving the more dangerous single-trace attacks unexplored. This article demonstrates successful single-trace power side-channel attacks on lattice-based key exchange and encapsulation protocols. Our attack targets both hardware and software implementations of matrix multiplications used in lattice cryptosystems. The crux of our idea is to apply a horizontal attack that makes hypotheses on several intermediate values within a single execution all relating to the same secret, and to combine their correlations for accurately estimating the secret key. We illustrate that the design of protocols combined with the nature of lattice arithmetic enables our attack. Since a straightforward attack suffers from false positives, we demonstrate a novel extend-and-prune procedure to recover the key by following the sequence of intermediate updates during multiplication. We analyzed two protocols, Frodo and FrodoKEM , and reveal that they are vulnerable to our attack. We implement both stand-alone hardware and RISC-V based software realizations and test the effectiveness of the proposed attack by using concrete parameters of these protocols on physical platforms with real measurements. We show that the proposed attack can estimate secret keys from a single power measurement with over 99% success rate.
Shadowed Type-2 Fuzzy Systems for Dynamic Parameter Adaptation in Harmony Search and Differential Evolution for Optimal Design of Fuzzy Controllers

This article mainly focuses on the utilization of shadowed type-2 fuzzy systems used to achieve the goal of dynamically adapting the parameters of two already known algorithms in the literature: the harmony search and the differential evolution algorithms. It has already been established that type-2 fuzzy logic enhances the performance of metaheuristics by enabling parameter adaptation; however, the utilization of fuzzy logic results in an increased execution time. For this reason, in this article, the shadowed type-2 fuzzy approach is put forward as a way of reducing execution time, while maintaining the good results that the complete type-2 fuzzy model produces. The harmony search and differential evolution algorithms with shadowed type-2 parameter adaptations were applied to the problem of optimally designing fuzzy controllers. The simulations were performed with the controllers working in an ideal situation, and then with a real situation under different noise levels in order to reach a conclusion regarding the performance of each of the algorithms that were applied.
A Theory of Quantum Subspace Diagonalization

Quantum subspace diagonalization methods are an exciting new class of algorithms for solving large\rev{-}scale eigenvalue problems using quantum computers. Unfortunately, these methods require the solution of an ill-conditioned generalized eigenvalue problem, with a matrix pair corrupted by a non-negligible amount of noise that is far above the machine precision. Despite pessimistic predictions from classical \rev{worst-case} perturbation theories, these methods can perform reliably well if the generalized eigenvalue problem is solved using a standard truncation strategy. By leveraging and advancing classical results in matrix perturbation theory, we provide a theoretical analysis of this surprising phenomenon, proving that under certain natural conditions, a quantum subspace diagonalization algorithm can accurately compute the smallest eigenvalue of a large Hermitian matrix. We give numerical experiments demonstrating the effectiveness of the theory and providing practical guidance for the choice of truncation level. Our new results can also be of independent interest to solving eigenvalue problems outside the context of quantum computation.
Teachers’ agency during the Covid-19 lockdown: A new materialist perspective

Teachers' agency is an essential factor in understanding and developing pedagogics. The study adds to the previous research by employing a new materialist perspective, highlighting the notion that teachers' agency is not merely a matter of humans, but results from assemblages of both human and nonhuman elements in teaching. The context of the study is the school lockdown period of the Covid-19 pandemic. Twenty Finnish primary teachers were interviewed to explore how teachers verbalised the rapid transition to a distance teaching environment and to discern what kind of agency that transition unfolds. The findings illustrate lost agency, but simultaneously, new forms of agency emerging from the entanglement of humans and materiality in the changed assemblages. This understanding helps to support both preservice and in-service teachers' agency in ways that acknowledge the complexity of teacher learning beyond individualistic and controllable views in increasingly multifaceted teaching environments.
Spread-Transform Dither Modulation Watermarking of Deep Neural Network

DNN watermarking is receiving an increasing attention as a suitable mean to protect the Intellectual Property Rights associated to DNN models. Several methods proposed so far are inspired to the popular Spread Spectrum (SS) paradigm according to which the watermark bits are embedded into the projection of the weights of the DNN model onto a pseudorandom sequence. In this paper, we propose a new DNN watermarking algorithm that leverages on the watermarking with side information paradigm to decrease the obtrusiveness of the watermark and increase its payload. In particular, the new scheme exploits the main ideas of ST-DM (Spread Transform Dither Modulation) watermarking to improve the performance of a recently proposed algorithm based on conventional SS. The experiments we carried out by applying the proposed scheme to watermark different models, demonstrate its capability to provide a higher payload with a lower impact on network accuracy than a baseline method based on conventional SS, while retaining a satisfactory level of robustness.
iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering

Generating "bullet-time" effects of human free-viewpoint videos is critical for immersive visual effects and VR/AR experience. Recent neural advances still lack the controllable and interactive bullet-time design ability for human free-viewpoint rendering, especially under the real-time, dynamic and general setting for our trajectory-aware task. To fill this gap, in this paper we propose a neural interactive bullet-time generator (iButter) for photo-realistic human free-viewpoint rendering from dense RGB streams, which enables flexible and interactive design for human bullet-time visual effects. Our iButter approach consists of a real-time preview and design stage as well as a trajectory-aware refinement stage. During preview, we propose an interactive bullet-time design approach by extending the NeRF rendering to a real-time and dynamic setting and getting rid of the tedious per-scene training. To this end, our bullet-time design stage utilizes a hybrid training set, light-weight network design and an efficient silhouette-based sampling strategy. During refinement, we introduce an efficient trajectory-aware scheme within 20 minutes, which jointly encodes the spatial, temporal consistency and semantic cues along the designed trajectory, achieving photo-realistic bullet-time viewing experience of human activities. Extensive experiments demonstrate the effectiveness of our approach for convenient interactive bullet-time design and photo-realistic human free-viewpoint video generation.
Video based human crowd analysis using machine learning: a survey

World population has increased manifolds in the last ten years. With the increase in population at this alarming rate, studying and understanding crowd patterns and their collective behaviour is very important. Researchers from various domains like artificial intelligence, machine learning, social science have shown their interest in understanding crowd phenomena from the social, psychological, and technical points of view. Computer vision techniques play a vital role in developing methods that help in understanding and analysing crowd behaviour automatically. In this article, we have surveyed many models related to crowd analysis developed and employed in computer vision. We aim to provide a comprehensive overview of the research from different aspects of crowd analysis like crowd count, human detection, anomaly detection, human behaviour, the importance of crowd analysis, and recent developments in this field. Major contributions have been included, along with their strengths and limitations.
Fine-Grained Style Control In Transformer-Based Text-To-Speech Synthesis

In this paper, we present a novel architecture to realize fine-grained style control on the transformer-based text-to-speech synthesis (TransformerTTS). Specifically, we model the speaking style by extracting a time sequence of local style tokens (LST) from the reference speech. The existing content encoder in TransformerTTS is then replaced by our designed cross-attention blocks for fusion and alignment between content and style. As the fusion is performed along with the skip connection, our cross-attention block provides a good inductive bias to gradually infuse the phoneme representation with a given style. Additionally, we prevent the style embedding from encoding linguistic content by randomly truncating LST during training and using wav2vec 2.0 features. Experiments show that with fine-grained style control, our system performs better in terms of naturalness, intelligibility, and style transferability. Our code and samples are publicly available. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Group-based Distinctive Image Captioning with Memory Attention

Describing images using natural language is widely known as image captioning, which has made consistent progress due to the development of computer vision and natural language generation techniques. Though conventional captioning models achieve high accuracy based on popular metrics, i.e., BLEU, CIDEr, and SPICE, the ability of captions to distinguish the target image from other similar images is under-explored. To generate distinctive captions, a few pioneers employ contrastive learning or re-weighted the ground-truth captions, which focuses on one single input image. However, the relationships between objects in a similar image group (e.g., items or properties within the same album or fine-grained events) are neglected. In this paper, we improve the distinctiveness of image captions using a Group-based Distinctive Captioning Model (GdisCap), which compares each image with other images in one similar group and highlights the uniqueness of each image. In particular, we propose a group-based memory attention (GMA) module, which stores object features that are unique among the image group (i.e., with low similarity to objects in other images). These unique object features are highlighted when generating captions, resulting in more distinctive captions. Furthermore, the distinctive words in the ground-truth captions are selected to supervise the language decoder and GMA. Finally, we propose a new evaluation metric, distinctive word rate (DisWordRate) to measure the distinctiveness of captions. Quantitative results indicate that the proposed method significantly improves the distinctiveness of several baseline models, and achieves the state-of-the-art performance on both accuracy and distinctiveness. Results of a user study agree with the quantitative evaluation and demonstrate the rationality of the new metric DisWordRate.