Reviewerly/exHarmony huggingface dataset:
Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answering

We study visual question answering in a setting where the answer has to be mined from a pool of relevant and irrelevant images given as a context. For such a setting, a model must first retrieve relevant images from the pool and answer the question from these retrieved images. We refer to this problem as retrieval-based visual question answering (or RETVQA in short). The RETVQA is distinctively different and more challenging than the traditionally-studied Visual Question Answering (VQA), where a given question has to be answered with a single relevant image in context. Towards solving the RETVQA task, we propose a unified Multi Image BART (MI-BART) that takes a question and retrieved images using our relevance encoder for free-form fluent answer generation. Further, we introduce the largest dataset in this space, namely RETVQA, which has the following salient features: multi-image and retrieval requirement for VQA, metadata-independent questions over a pool of heterogeneous images, expecting a mix of classification-oriented and open-ended generative answers. Our proposed framework achieves an accuracy of 76.5% and a fluency of 79.3% on the proposed dataset, namely RETVQA and also outperforms state-of-the-art methods by 4.9% and 11.8% on the image segment of the publicly available WebQA dataset on the accuracy and fluency metrics, respectively.
Federated Probabilistic Preference Distribution Modelling with Compactness Co-Clustering for Privacy-Preserving Multi-Domain Recommendation

With the development of modern internet techniques, Cross-Domain Recommendation (CDR) systems have been widely exploited for tackling the data-sparsity problem. Meanwhile most current CDR models assume that user-item interactions are accessible across different domains. However, such knowledge sharing process will break the privacy protection policy. In this paper, we focus on the Privacy-Preserving Multi-Domain Recommendation problem (PPMDR). The problem is challenging since different domains are sparse and heterogeneous with the privacy protection. To tackle the above issues, we propose Federated Probabilistic Preference Distribution Modelling (FPPDM). FPPDM includes two main components, i.e., local domain modelling component and global server aggregation component with federated learning strategy. The local domain modelling component aims to exploit user/item preference distributions using the rating information in the corresponding domain. The global server aggregation component is set to combine user characteristics across domains. To better extract semantic neighbors information among the users, we further provide compactness co-clustering strategy in FPPDM ++ to cluster the users with similar characteristics. Our empirical studies on benchmark datasets demonstrate that FPPDM/ FPPDM ++ significantly outperforms the state-of-the-art models.
Privacy-Preserving End-to-End Spoken Language Understanding

Spoken language understanding (SLU), one of the key enabling technologies for human-computer interaction in IoT devices, provides an easy-to-use user interface. Human speech can contain a lot of user-sensitive information, such as gender, identity, and sensitive content. New types of security and privacy breaches have thus emerged. Users do not want to expose their personal sensitive information to malicious attacks by untrusted third parties. Thus, the SLU system needs to ensure that a potential malicious attacker cannot deduce the sensitive attributes of the users, while it should avoid greatly compromising the SLU accuracy. To address the above challenge, this paper proposes a novel SLU multi-task privacy-preserving model to prevent both the speech recognition (ASR) and identity recognition (IR) attacks. The model uses the hidden layer separation technique so that SLU information is distributed only in a specific portion of the hidden layer, and the other two types of information are removed to obtain a privacy-secure hidden layer. In order to achieve good balance between efficiency and privacy, we introduce a new mechanism of model pre-training, namely joint adversarial training, to further enhance the user privacy. Experiments over two SLU datasets show that the proposed method can reduce the accuracy of both the ASR and IR attacks close to that of a random guess, while leaving the SLU performance largely unaffected.
Probabilistic Temporal Logic for Reasoning about Bounded Policies

To build a theory of intention revision for agents operating in stochastic environments, we need a logic in which we can explicitly reason about their decision-making policies and those policies' uncertain outcomes. Towards this end, we propose PLBP, a novel probabilistic temporal logic for Markov Decision Processes that allows us to reason about policies of bounded size. The logic is designed so that its expressive power is sufficient for the intended applications, whilst at the same time possessing strong computational properties. We prove that the satisfiability problem for our logic is decidable, and that its model checking problem is PSPACE-complete. This allows us to e.g. algorithmically verify whether an agent's intentions are coherent, or whether a specific policy satisfies safety and/or liveness properties.
Singularformer: Learning to Decompose Self-Attention to Linearize the Complexity of Transformer

Transformers achieve excellent performance in a variety of domains since they can capture long-distance dependencies through the self-attention mechanism. However, self-attention is computationally costly due to its quadratic complexity and high memory consumption. In this paper, we propose a novel Transformer variant (Singularformer) that uses neural networks to learn the singular value decomposition process of the attention matrix to design a linear-complexity and memory-efficient global self-attention mechanism. Specifically, we decompose the attention matrix into the product of three matrix factors based on singular value decomposition and design neural networks to learn these matrix factors, then the associative law of matrix multiplication is used to linearize the calculation of self-attention. The above procedure allows us to compute self-attention as two-dimensional reduction processes in the first and second token dimensional spaces, followed by a multi-head self-attention computational process on the first dimensional reduced token features. Experimental results on 8 real-world datasets demonstrate that Singularformer performs favorably against the other Transformer variants with lower time and space complexity. Our source code is publicly available at https://github.com/CSUBioGroup/Singularformer.
FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment

Different from conventional federated learning, personalized federated learning (PFL) is able to train a customized model for each individual client according to its unique requirement. The mainstream approach is to adopt a kind of weighted aggregation method to generate personalized models, in which weights are determined by the loss value or model parameters among different clients. However, such kinds of methods require clients to download others' models. It not only sheer increases communication traffic but also potentially infringes data privacy. In this paper, we propose a new PFL algorithm called FedDWA (Federated Learning with Dynamic Weight Adjustment) to address the above problem, which leverages the parameter server (PS) to compute personalized aggregation weights based on collected models from clients. In this way, FedDWA can capture similarities between clients with much less communication overhead. More specifically, we formulate the PFL problem as an optimization problem by minimizing the distance between personalized models and guidance models, so as to customize aggregation weights for each client. Guidance models are obtained by the local one-step ahead adaptation on individual clients. Finally, we conduct extensive experiments using five real datasets and the results demonstrate that FedDWA can significantly reduce the communication traffic and achieve much higher model accuracy than the state-of-the-art approaches.
Temporal Constrained Feasible Subspace Learning for Human Pose Forecasting

Human pose forecasting is a sequential modeling task that aims to predict future poses from historical motions. Most existing approaches focus on the spatial-temporal neural network model design for learning movement patterns to reduce prediction errors. However, they usually do not strictly follow the temporal constraints in the inference stage. Even though a small Mean Per Joint Position Error (MPJPE) is achieved, some of the predicted poses are not temporal feasible solutions, which disobeys the continuity of the body movement. In this paper, we consider the temporal constrained feasible solutions for human pose forecasting, where the predicted poses of input historical poses are guaranteed to obey the temporal constraints strictly in the inference stage. Rather than direct supervision of the prediction in the original pose space, a temporal constrained subspace is explicitly learned and then followed by an inverse transformation to obtain the final predictions. We evaluate the proposed method on large-scale benchmarks, including Human3.6M, AMASS, and 3DPW. State-of-the-art performance has been achieved with the temporal constrained feasible solutions.
Towards Accurate Video Text Spotting with Text-wise Semantic Reasoning

Video text spotting (VTS) aims at extracting texts from videos, where text detection, tracking and recognition are conducted simultaneously. There have been some works that can tackle VTS; however, they may ignore the underlying semantic relationships among texts within a frame. We observe that the texts within a frame usually share similar semantics, which suggests that, if one text is predicted incorrectly by a text recognizer, it still has a chance to be corrected via semantic reasoning. In this paper, we propose an accurate video text spotter, VLSpotter, that reads texts visually, linguistically, and semantically. For ‘visually’, we propose a plug-and-play text-focused super-resolution module to alleviate motion blur and enhance video quality. For ‘linguistically’, a language model is employed to capture intra-text context to mitigate wrongly spelled text predictions. For ‘semantically’, we propose a text-wise semantic reasoning module to model inter-text semantic relationships and reason for better results. The experimental results on multiple VTS benchmarks demonstrate that the proposed VLSpotter outperforms the existing state-of-the-art methods in end-to-end video text spotting.
Cross-Domain Facial Expression Recognition via Disentangling Identity Representation

Most existing cross-domain facial expression recognition (FER) works require target domain data to assist the model in analyzing distribution shifts to overcome negative effects. However, it is often hard to obtain expression images of the target domain in practical applications. Moreover, existing methods suffer from the interference of identity information, thus limiting the discriminative ability of the expression features. We exploit the idea of domain generalization (DG) and propose a representation disentanglement model to address the above problems. Specifically, we learn three independent potential subspaces corresponding to the domain, expression, and identity information from facial images. Meanwhile, the extracted expression and identity features are recovered as Fourier phase information reconstructed images, thereby ensuring that the high-level semantics of images remain unchanged after disentangling the domain information. Our proposed method can disentangle expression features from expression-irrelevant ones (i.e., identity and domain features). Therefore, the learned expression features exhibit sufficient domain invariance and discriminative ability. We conduct experiments with different settings on multiple benchmark datasets, and the results show that our method achieves superior performance compared with state-of-the-art methods.
An Empirical Study on the Language Modal in Visual Question Answering

Generalization beyond in-domain experience to out-of-distribution data is of paramount significance in the AI domain. Of late, state-of-the-art Visual Question Answering (VQA) models have shown impressive performance on in-domain data, partially due to the language prior bias which, however, hinders the generalization ability in practice. This paper attempts to provide new insights into the influence of language modality on VQA performance from an empirical study perspective. To achieve this, we conducted a series of experiments on six models. The results of these experiments revealed that, 1) apart from prior bias caused by question types, there is a notable influence of postfix-related bias in inducing biases, and 2) training VQA models with word-sequence-related variant questions demonstrated improved performance on the out-of-distribution benchmark, and the LXMERT even achieved a 10-point gain without adopting any debiasing methods. We delved into the underlying reasons behind these experimental results and put forward some simple proposals to reduce the models' dependency on language priors. The experimental results demonstrated the effectiveness of our proposed method in improving performance on the out-of-distribution benchmark, VQA-CPv2. We hope this study can inspire novel insights for future research on designing bias-reduction approaches.
Hierarchical Transformer for Scalable Graph Learning

Graph Transformer is gaining increasing attention in the field of machine learning and has demonstrated state-of-the-art performance on benchmarks for graph representation learning. However, as current implementations of Graph Transformer primarily focus on learning representations of small-scale graphs, the quadratic complexity of the global self-attention mechanism presents a challenge for full-batch training when applied to larger graphs. Additionally, conventional sampling-based methods fail to capture necessary high-level contextual information, resulting in a significant loss of performance. In this paper, we introduce the Hierarchical Scalable Graph Transformer (HSGT) as a solution to these challenges. HSGT successfully scales the Transformer architecture to node representation learning tasks on large-scale graphs, while maintaining high performance. By utilizing graph hierarchies constructed through coarsening techniques, HSGT efficiently updates and stores multi-scale information in node embeddings at different levels. Together with sampling-based training methods, HSGT effectively captures and aggregates multi-level information on the hierarchical graph using only Transformer blocks. Empirical evaluations demonstrate that HSGT achieves state-of-the-art performance on large-scale benchmarks with graphs containing millions of nodes with high efficiency.
Physics-Guided Human Motion Capture with Pose Probability Modeling

Incorporating physics in human motion capture to avoid artifacts like floating, foot sliding, and ground penetration is a promising direction. Existing solutions always adopt kinematic results as reference motions, and the physics is treated as a post-processing module. However, due to the depth ambiguity, monocular motion capture inevitably suffers from noises, and the noisy reference often leads to failure for physics-based tracking. To address the obstacles, our key-idea is to employ physics as denoising guidance in the reverse diffusion process to reconstruct physically plausible human motion from a modeled pose probability distribution. Specifically, we first train a latent gaussian model that encodes the uncertainty of 2D-to-3D lifting to facilitate reverse diffusion. Then, a physics module is constructed to track the motion sampled from the distribution. The discrepancies between the tracked motion and image observation are used to provide explicit guidance for the reverse diffusion model to refine the motion. With several iterations, the physics-based tracking and kinematic denoising promote each other to generate a physically plausible human motion. Experimental results show that our method outperforms previous physics-based methods in both joint accuracy and success rate. More information can be found at https://github.com/Me-Ditto/Physics-Guided-Mocap.
ARTIFICIAL INTELLIGENCE IN THE CONTEXT OF COPYRIGHT

Generative artificial intelligence is rapidly entering our lives. Chatbots like ChatGPT are already mass-producing newsletters, writing essays, summaries, criticism and even works of art, solving various tasks and creating all kinds of verification tests, skilfully imitating styles and forms. Image generators like Stable Diffusion, Midjourney, and more can create outstanding visuals in a variety of styles: from aged photos, graphics, watercolours, and paintings to street graffiti look-alikes. The resulting products are in most cases remarkable not only for their quality, but also for the speed of their creation. And while these practices are becoming more common in the creative industries, the volumes of text, images and music generated by artificial intelligence, as well as the process used to create them, raises many complex ethical and legal questions. Some of them are largely related to today’s understanding of authorship as a legal doctrine and its protection by current laws, as well as the permitted possibility of using parts of one work or the whole of it for the purpose of creating another work, without prejudice to the legal copyright of the original author. In this article, we will provide some clarification on how copyright norms relate to generative artificial intelligence and reflect on the ethical dilemmas associated with this, which clearly pose a huge challenge to our understanding of property and justice, and also to the very nature of creativity.
ASPECTS OF THE QUALITY OF EDUCATION IN KINDERGARTEN

Key aspects of quality of education are considered in the context of teacher qualifications in the process of education, upbringing and socialization of children, communication in the kindergarten community and the children’s portfolio as a tool to reflect their progress in relation to quality assurance. A survey among teachers from kindergartens in Varna, Shumen, Targovishte and Razgrad is presented, based on which criteria and indicators for measuring quality in the kindergarten are derived. The analysis of the results of the study leads to conclusions about the importance of the professional competences of teachers for the quality of education, upbringing and socialization of children and reflecting their achievements for assessing progress.
ARTIFICIAL INTELLIGENCE – NORMATIVE FRAMEWORK IN AN EDUCATIONAL CONTEXT

The purpose of the article is to provide definitions and to review the normative framework related to artificial intelligence and its use into the educational field. Concepts, strategic documents of the European Union, UNESCO, the United Nations, documents at the national level are presented. Summaries and analyses of the challenges that artificial intelligence poses to the educational system have been made.
GAMIFICATION AND OTHER APPROACHES TO INCREASE THE INTEREST OF PYTHON LEARNERS

This paper describes the challenges of teaching technological disciplines at high school and higher education levels. In particular four challenges are identified – participation motivation, motivation for cooperation, the need for feedback and the need for recognition. The strategies for answering these challenges are described as they were identified when teaching at university level, particularly taking into account the psychological peculiarities of the newer generation and using gamification techniques. A system for achievements and badges for unofficial channel feedback channel is described.
CODE: SUCCESSFUL ONE. STEP-BY-STEP TO ALGORITHMICAL THINKING AND PROGRAMMING SKILLS

The article presents examples of the content of a model for individualized and differentiated instruction in computer modelling in grades 5. – 7. Two example topics are presented. The first one is from the 5th grade Computer Modelling and Information Technology curriculum – Creating a computer character in Scratch. The second topic is from the 7th grade syllabus – Using strings in Python. Students work on the same task, structured in three levels of complexity – beginner, advanced and master level. Each level challenges students to develop the next better version of their computer project. The presented model is implemented in students’ book „Code – successful one“. The aim is to support the process of effective building of digital competences at school by applying individualization and differentiation of training.
The Relationship of the Learning Environment with the Activity of the Participants of Taklim Assembly Activities in Maransi Aia Pacah Padang

ABSTRACTThis research is motivated by the fact that the attendance of participants in the Nurul Haq Mosque Taklim activity in Maransi, Aia Pacah Village, Padang City 2021, is decreasing over time. This is presumably due to the less conducive learning environment. The purpose of this study is to see if there is a relevant relationship between the learning environment and the activeness of participants in the Nurul Haq Taklim Council activities in Maransi, Aia Pacah Village, Padang City 2021. This type of research is quantitative correlation. The population in this study were all 46 participants of the Taklim Mosque Nurul Haq Mosque in Maransi, Aia Pacah Village, Padang City 2021, with a sample size of 36 people. Swallowing data using a questionnaire or questionnaire. The data analysis technique uses the proportion formula and the Product Moment correlation formula. The results of the study 1) the learning environment of activities is not conducive; 2) Low participant learning activity; 3) There is a significant relationship between the learning environment and the active learning of participants. The statistical test obtained rcount = 0.826, it can be found a significant relationship between the learning environment of the participants in the Taklim Assembly activity in Maransi, Aia Pacah Village, Padang City.Keywords: Learning Environment, Activity, Taklim Assembly