Reviewerly/exHarmony huggingface dataset:
Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition

Prior studies use one attention mechanism to improve contextual semantic representation learning for implicit discourse relation recognition (IDRR).However, diverse relation senses may benefit from different attention mechanisms.We also argue that some linguistic relation in between two words can be further exploited for IDRR.This paper proposes a Multi-Attentive Neural Fusion (MANF) model to encode and fuse both semantic connection and linguistic evidence for IDRR.In MANF, we design a Dual Attention Network (DAN) to learn and fuse two kinds of attentive representation for arguments as its semantic connection.We also propose an Offset Matrix Network (OMN) to encode the linguistic relations of word-pairs as linguistic evidence.Our MANF model achieves the state-of-the-art results on the PDTB 3.0 corpus.
CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022

In this paper, we describe our submission to the Simultaneous Speech Translation at IWSLT 2022.We explore strategies to utilize an offline model in a simultaneous setting without the need to modify the original model.In our experiments, we show that our onlinization algorithm is almost on par with the offline setting while being 3× faster than offline in terms of latency on the test set.We also show that the onlinized offline model outperforms the best IWSLT2021 simultaneous system in medium and high latency regimes and is almost on par in the low latency regime.We make our system publicly available. 1
Bootstrapping for Approximate Homomorphic Encryption with Negligible Failure-Probability by Using Sparse-Secret Encapsulation

Bootstrapping parameters for the approximate homomorphic-encryption scheme of Cheon et al., CKKS (Asiacrypt 17), are usually instantiated using sparse secrets to be efficient. However, using sparse secrets constrains the range of practical parameters within a tight interval, as they must support a large enough depth for the bootstrapping circuit but also be secure with respect to the sparsity of their secret. We present a bootstrapping procedure for the CKKS scheme that combines both dense and sparse secrets. Our construction enables the use of parameters for which the homomorphic capacity is based on a dense secret, yet with a bootstrapping complexity that remains the one of a sparse secret and with a large security margin. Moreover, this also enables us to easily parameterize the bootstrapping circuit so that it has a negligible failure probability that, to the best of our knowledge, has never been achieved for the CKKS scheme. When using the parameters of previous works, our bootstrapping procedures enable a faster execution with an increased precision and lower failure probability. For example, we are able to bootstrap a plaintext of $$\mathbb {C}^{32768}$$ in 20.2 s, with 32.11 bits of precision, 285 remaining modulus bits, a failure probability of $$2^{-138.7}$$ , and 128 bit security.
A Neural Network Approach for High-Dimensional Optimal Control Applied to Multiagent Path Finding

We propose a neural network (NN) approach that yields approximate solutions for high-dimensional optimal control (OC) problems and demonstrate its effectiveness using examples from multiagent path finding. Our approach yields control in a feedback form, where the policy function is given by an NN. In particular, we fuse the Hamilton–Jacobi–Bellman (HJB) and Pontryagin maximum principle (PMP) approaches by parameterizing the value function with an NN. Our approach enables us to obtain approximately OCs in real time without having to solve an optimization problem. Once the policy function is trained, generating a control at a given space–time location takes milliseconds; in contrast, efficient nonlinear programming methods typically perform the same task in seconds. We train the NN offline using the objective function of the control problem and penalty terms that enforce the HJB equations. Therefore, our training algorithm does not involve data generated by another algorithm. By training on a distribution of initial states, we ensure the controls’ optimality on a large portion of the state space. Our grid-free approach scales efficiently to dimensions where grids become impractical or infeasible. We apply our approach to several multiagent collision-avoidance problems in up to 150 dimensions. Furthermore, we empirically observe that the number of parameters in our approach scales linearly with the dimension of the control problem, thereby mitigating the curse of dimensionality.
Robust Hand Gesture Recognition Based on RGB-D Data for Natural Human–Computer Interaction

To naturally interact with virtual environment by hand gesture, this paper presents a robust RGB-D data based recognition method of static and dynamic hand gesture.Firstly, for static hand gesture recognition, starting from the hand gesture contour extraction, the palm center is identified by Distance Transform (DT) algorithm.The fingertips are localized by employing the K-Curvature-Convex Defects Detection algorithm (K-CCD).On the basis, the distances of the pixels on hand gesture contour to palm center and the angle between fingertips are considered as the auxiliary features to construct a multimodal feature vector, and then recognition algorithm is presented to robustly recognize the static hand gestures.Secondly, combining Euclidean distance between hand joints and shoulder center joint with the modulus ratios of skeleton features, this paper generates a unifying feature descriptor for each dynamic hand gesture and proposes an improved dynamic time warping (IDTW) algorithm to obtain recognition results of dynamic hand gestures.Finally, we conduct extensive experiments to test and verify the static and dynamic hand gesture recognition algorithm and realize a low-cost real-time application of natural interaction with virtual environment by hand gestures.
Context Matters: A Pragmatic Study of PLMs’ Negation Understanding

In linguistics, there are two main perspectives on negation: a semantic and a pragmatic view.So far, research in NLP on negation has almost exclusively adhered to the semantic view.In this article, we adopt the pragmatic paradigm to conduct a study of negation understanding focusing on transformer-based PLMs.Our results differ from previous, semantics-based studies and therefore help to contribute a more comprehensive -and, given the results, much more optimistic -picture of the PLMs' negation understanding.
It is AI’s Turn to Ask Humans a Question: Question-Answer Pair Generation for Children’s Story Books

Existing question answering (QA) techniques are created mainly to answer questions asked by humans.But in educational applications, teachers often need to decide what questions they should ask, in order to help students to improve their narrative understanding capabilities.We design an automated question-answer generation (QAG) system for this education scenario: given a story book at the kindergarten to eighth-grade level as input, our system can automatically generate QA pairs that are capable of testing a variety of dimensions of a student's comprehension skills.Our proposed QAG model architecture is demonstrated using a new expert-annotated FairytaleQA dataset, which has 278 child-friendly storybooks with 10,580 QA pairs.Automatic and human evaluations show that our model outperforms stateof-the-art QAG baseline systems.On top of our QAG system, we also start to build an interactive story-telling application for the future real-world deployment in this educational scenario.
Pemanfaatan Media Audio Visual dan Media Gambar Pada Siswa Kelas V

This research was conducted with the aim of knowing the average test result score as a reflection of the effectiveness of audio visual media and image media. This is based on learning outcomes that are considered less than optimal and many are still incomplete because teachers are accustomed to the use of image media in learning. The selection of audio-visual media as a way of teaching data presentation material is because it has better features than just ordinary images. This study determined the subject, namely the fifth grade students at one of the private elementary schools in Surabaya which had 34 students. The results of this study presented an average of learning outcomes in a class using audio visual media was 87.68. In another class, the average result of student learning in the class using image media obtained an average result of 79.59. The conclusion from these results is that the class that uses audio-visual media is more effective than the class that uses image media.
Zipf’s law revisited: Spoken dialog, linguistic units, parameters, and the principle of least effort

Abstract The ubiquitous inverse relationship between word frequency and word rank is commonly known as Zipf’s law. The theoretical underpinning of this law states that the inverse relationship yields decreased effort in both the speaker and hearer, the so-called principle of least effort. Most research has focused on showing an inverse relationship only for written monolog, only for frequencies and ranks of one linguistic unit, generally word unigrams, with strong correlations of the power law to the observed frequency distributions, with limited to no attention to psychological mechanisms such as the principle of least effort. The current paper extends the existing findings, by not focusing on written monolog but on a more fundamental form of communication, spoken dialog, by not only investigating word unigrams but also units quantified on syntactic, pragmatic, utterance, and nonverbal communicative levels by showing that the adequacy of Zipf’s formula seems ubiquitous, but the exponent of the power law curve is not, and by placing these findings in the context of Zipf’s principle of least effort through redefining effort in terms of cognitive resources available for communication. Our findings show that Zipf’s law also applies to a more natural form of communication—that of spoken dialog, that it applies to a range of linguistic units beyond word unigrams, that the general good fit of Zipf’s law needs to be revisited in light of the parameters of the formula, and that the principle of least effort is a useful theoretical framework for the findings of Zipf’s law.
Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolation

Expanding an existing tourist photo from a partially captured scene to a full scene is one of the desired experiences for photography applications. Although photo extrapolation has been well studied, it is much more challenging to extrapolate a photo (i.e., selfie) from a narrow field of view to a wider one while maintaining a similar visual style. In this paper, we propose a factorized neural re-rendering model to produce photorealistic novel views from cluttered outdoor Internet photo collections, which enables the applications including controllable scene re-rendering, photo extrapolation and even extrapolated 3D photo generation. Specifically, we first develop a novel factorized re-rendering pipeline to handle the ambiguity in the decomposition of geometry, appearance and illumination. We also propose a composited training strategy to tackle the unexpected occlusion in Internet images. Moreover, to enhance photo-realism when extrapolating tourist photographs, we propose a novel realism augmentation process to complement appearance details, which automatically propagates the texture details from a narrow captured photo to the extrapolated neural rendered image. The experiments and photo editing examples on outdoor scenes demonstrate the superior performance of our proposed method in both photo-realism and downstream applications. Code and the supplementary material are available on the project webpage: https://zju3dv.github.io/neural_outdoor_rerender/.
Targeted Multimodal Sentiment Classification based on Coarse-to-Fine Grained Image-Target Matching

Targeted Multimodal Sentiment Classification (TMSC) aims to identify the sentiment polarities over each target mentioned in a pair of sentence and image. Existing methods to TMSC failed to explicitly capture both coarse-grained and fine-grained image-target matching, including 1) the relevance between the image and the target and 2) the alignment between visual objects and the target. To tackle this issue, we propose a new multi-task learning architecture named coarse-to-fine grained Image-Target Matching network (ITM), which jointly performs image-target relevance classification, object-target alignment, and targeted sentiment classification. We further construct an Image-Target Matching dataset by manually annotating the image-target relevance and the visual object aligned with the input target. Experiments on two benchmark TMSC datasets show that our model consistently outperforms the baselines, achieves state-of-the-art results, and presents interpretable visualizations.
Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering

Video question answering (VideoQA) is challenging given its multimodal combination of visual understanding and natural language processing. While most existing approaches ignore the visual appearance-motion information at different temporal scales, it is unknown how to incorporate the multilevel processing capacity of a deep learning model with such multiscale information. Targeting these issues, this paper proposes a novel Multilevel Hierarchical Network (MHN) with multiscale sampling for VideoQA. MHN comprises two modules, namely Recurrent Multimodal Interaction (RMI) and Parallel Visual Reasoning (PVR). With a multiscale sampling, RMI iterates the interaction of appearance-motion information at each scale and the question embeddings to build the multilevel question-guided visual representations. Thereon, with a shared transformer encoder, PVR infers the visual cues at each level in parallel to fit with answering different question types that may rely on the visual information at relevant levels. Through extensive experiments on three VideoQA datasets, we demonstrate improved performances than previous state-of-the-arts and justify the effectiveness of each part of our method.
Cat Swarm Optimization Algorithm - A Survey and Performance Evaluation

This paper presents an in-depth survey and performance evaluation of the Cat Swarm Optimization (CSO) Algorithm. CSO is a robust and powerful metaheuristic swarm-based optimization approach that has received very positive feedback since its emergence. It has been tackling many optimization problems and many variants of it have been introduced. However, the literature lacks a detailed survey or a performance evaluation in this regard. Therefore, this paper is an attempt to review all these works, including its developments and applications, and group them accordingly. In addition, CSO is tested on 23 classical benchmark functions and 10 modern benchmark functions (CEC 2019). The results are then compared against three novel and powerful optimization algorithms, namely Dragonfly algorithm (DA), Butterfly optimization algorithm (BOA) and Fitness Dependent Optimizer (FDO). These algorithms are then ranked according to Friedman test and the results show that CSO ranks first on the whole. Finally, statistical approaches are employed to further confirm the outperformance of CSO algorithm.
Overview of Deep Neural Networks

Deep Neural Networks (DNNs) come in a wide variety of shapes and sizes depending on the application.1 The popular shapes and sizes are also evolving rapidly to improve accuracy and efficiency. In all cases, the input to a DNN is a set of values representing the information to be analyzed by the network. For instance, these values can be pixels of an image, sampled amplitudes of an audio wave, or the numerical representation of the state of some system or game.
Lightweight Architecture for Elliptic Curve Scalar Multiplication over Prime Field

In this paper, we present a novel lightweight elliptic curve scalar multiplication architecture for random Weierstrass curves over prime field Fp. The elliptic curve scalar multiplication is executed in Jacobian coordinates based on the Montgomery ladder algorithm with (X,Y)-only common Z coordinate arithmetic. At the finite field operation level, the adder-based modular multiplier and modular divider are optimized by the pre-calculation method to reduce the critical path while maintaining low resource consumption. At the group operation level, the point addition and point doubling methods in (X,Y)-only common Z coordinate arithmetic are modified to improve computation parallelism. A compact scheduling method is presented to improve the architecture’s performance, which includes appropriate scheduling of finite field operations and specific register connections. Compared with existing works, our design is implemented on the FPGA platform without using DSPs or BRAMs for higher portability. It utilizes 6.4~6.5k slices in Kintex-7, Virtex-7, and ZYNQ FPGA and executes an elliptic curve scalar multiplication for a field size of 256-bit in 1.73 ms, 1.70 ms, and 1.80 ms, respectively. Additionally, our design is resistant to timing attacks, simple power analysis attacks, and safe-error attacks. This architecture outperforms most state-of-the-art lightweight designs in terms of area-time products.
An Adaptive Multi-Level Quantization-Based Reinforcement Learning Model for Enhancing UAV Landing on Moving Targets

The autonomous landing of an unmanned aerial vehicle (UAV) on a moving platform is an essential functionality in various UAV-based applications. It can be added to a teleoperation UAV system or part of an autonomous UAV control system. Various robust and predictive control systems based on the traditional control theory are used for operating a UAV. Recently, some attempts were made to land a UAV on a moving target using reinforcement learning (RL). Vision is used as a typical way of sensing and detecting the moving target. Mainly, the related works have deployed a deep-neural network (DNN) for RL, which takes the image as input and provides the optimal navigation action as output. However, the delay of the multi-layer topology of the deep neural network affects the real-time aspect of such control. This paper proposes an adaptive multi-level quantization-based reinforcement learning (AMLQ) model. The AMLQ model quantizes the continuous actions and states to directly incorporate simple Q-learning to resolve the delay issue. This solution makes the training faster and enables simple knowledge representation without needing the DNN. For evaluation, the AMLQ model was compared with state-of-art approaches and was found to be superior in terms of root mean square error (RMSE), which was 8.7052 compared with the proportional–integral–derivative (PID) controller, which achieved an RMSE of 10.0592.
Personalized Recommendation Method of Sports Online Video Teaching Resources Based on Multiuser Characteristics

Aiming at the problems of poor precision, low recall rate, and large recommendation time overhead in the personalized recommendation of sports online video teaching resources, this paper designs a personalized recommendation method for sports online video teaching resources based on multi-user characteristics. The area where the collected sports online video teaching resources are collected is fixed, and the confidence space for data collection is determined. The components of each clustering point are determined by the k-means clustering algorithm, and the data collection is completed continuously iteratively. The similar data in the data segments is removed with the help of cosine similarity, and the video segment data in the sports online video teaching resources with high similarity is removed for further data normalization. According to the multi-user feature analysis, the video teaching resource data that the user is interested in is matched, and the matching weight is calculated. In this paper, a personalized recommendation model of sports online video teaching resources is constructed by the recurrent convolutional neural networks (RCNN) method, and the personalized recommendation of video teaching resources is realized. The experimental results show that the mining accuracy of this method is about 96%, the highest recall rate can reach 98%, and its data recommendation time overhead is less than 1.3 s. This method effectively improves the effect of sports online video teaching resources.
Dynamic Segmentation of Sensor Events for Real-Time Human Activity Recognition in a Smart Home Context

Human activity recognition (HAR) is fundamental to many services in smart buildings. However, providing sufficiently robust activity recognition systems that could be confidently deployed in an ordinary real environment remains a major challenge. Much of the research done in this area has mainly focused on recognition through pre-segmented sensor data. In this paper, real-time human activity recognition based on streaming sensors is investigated. The proposed methodology incorporates dynamic event windowing based on spatio-temporal correlation and the knowledge of activity trigger sensor to recognize activities and record new events. The objective is to determine whether the last event that just happened belongs to the current activity, or if it is the sign of the start of a new activity. For this, we consider the correlation between sensors in view of what can be seen in the history of past events. The proposed algorithm contains three steps: verification of sensor correlation (SC), verification of temporal correlation (TC), and determination of the activity triggering the sensor. The proposed approach is applied to a real case study: the "Aruba" dataset from the CASAS database. F1 score is used to assess the quality of the segmentation. The results show that the proposed approach segments several activities (sleeping, bed to toilet, meal preparation, eating, housekeeping, working, entering home, and leaving home) with an F1 score of 0.63-0.99.
The virtual worlds of the metaverse <b>The Metaverse: And How It Will Revolutionize Everything</b> <i>Matthew Ball</i> Liveright, 2022. 352 pp.

An immersive internet is just around the corner, for better or worse
Investigating the Visual Behavior Characteristics of Architectural Heritage Using Eye-Tracking

The visual quality of architectural heritage is crucial to the preservation of architectural features, enhancement of the environmental quality, and conservation of the sustainable development and adaptive use of architectural heritage. Few studies have explored the visual behavior characteristics of rural architectural heritage and which elements influence visual perception. Our study used eye-tracking technology to explore this issue. The results indicate that participants have different visual behavior characteristics for architectural heritage in different scenarios, with five eye movement metrics showing statistical differences. Featured elements attracted more visual attention. The visual behavior characteristics were related to the area, relative area, distance from center, and perimeter. Based on the results, decision-makers can target the sustainable and virtuous development of architectural heritage and enhance environmental quality.
Dealing with imbalanced data for interpretable defect prediction

Interpretation has been considered as a key factor to apply defect prediction in practice. As interpretation from rule-based interpretable models can provide insights about past defects with high quality, many prior studies attempt to construct interpretable models for both accurate prediction and comprehensible interpretation. However, class imbalance is usually ignored, which may bring huge negative impact on interpretation.