Reviewerly/exHarmony huggingface dataset:
Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions

Deep neural networks have been critical in the task of Visual Question Answering (VQA), with research traditionally focused on improving model accuracy. Recently, however, there has been a trend towards evaluating the robustness of these models against adversarial attacks. This involves assessing the accuracy of VQA models under increasing levels of noise in the input, which can target either the image or the proposed query question, dubbed the main question. However, there is currently a lack of proper analysis of this aspect of VQA. This work proposes a new method that utilizes semantically related questions, referred to as basic questions, acting as noise to evaluate the robustness of VQA models. It is hypothesized that as the similarity of a basic question to the main question decreases, the level of noise increases. To generate a reasonable noise level for a given main question, a pool of basic questions is ranked based on their similarity to the main question, and this ranking problem is cast as a LASSO optimization problem. Additionally, this work proposes a novel robustness measure, R_score, and two basic question datasets to standardize the analysis of VQA model robustness. The experimental results demonstrate that the proposed evaluation method effectively analyzes the robustness of VQA models. Moreover, the experiments show that in-context learning with a chain of basic questions can enhance model accuracy.
Synthesizing Anyone, Anywhere, in Any Pose

We address the task of in-the-wild human figure synthesis, where the primary goal is to synthesize a full body given any region in any image. In-the-wild human figure synthesis has long been a challenging and under-explored task, where current methods struggle to handle extreme poses, occluding objects, and complex backgrounds. Our main contribution is TriA-GAN, a keypoint-guided GAN that can synthesize Anyone, Anywhere, in Any given pose. Key to our method is projected GANs combined with a well-crafted training strategy, where our simple generator architecture can successfully handle the challenges of in-the-wild full-body synthesis. We show that TriA-GAN significantly improves over previous in-the-wild full-body synthesis methods, all while requiring less conditional information for synthesis (keypoints \vs DensePose). Finally, we show that the latent space of TriA-GAN is compatible with standard unconditional editing techniques, enabling text-guided editing of generated human figures.
Selective Data Augmentation for Robust Speech Translation

Speech translation (ST) systems translate speech in one language to text in another language. End-to-end ST systems (e2e-ST) have gained popularity over cascade systems because of their enhanced performance due to reduced latency and computational cost. Though resource intensive, e2e-ST systems have the inherent ability to retain para and non-linguistic characteristics of the speech unlike cascade systems. In this paper, we propose to use an e2e architecture for English-Hindi (en-hi) ST. We use two imperfect machine translation (MT) services to translate Libri-trans en text into hi text. While each service gives MT data individually to generate parallel ST data, we propose a data augmentation strategy of noisy MT data to aid robust ST. The main contribution of this paper is the proposal of a data augmentation strategy. We show that this results in better ST (BLEU score) compared to brute force augmentation of MT data. We observed an absolute improvement of 1.59 BLEU score with our approach.
CoT-MAE v2: Contextual Masked Auto-Encoder with Multi-view Modeling for Passage Retrieval

Growing techniques have been emerging to improve the performance of passage retrieval. As an effective representation bottleneck pretraining technique, the contextual masked auto-encoder utilizes contextual embedding to assist in the reconstruction of passages. However, it only uses a single auto-encoding pre-task for dense representation pre-training. This study brings multi-view modeling to the contextual masked auto-encoder. Firstly, multi-view representation utilizes both dense and sparse vectors as multi-view representations, aiming to capture sentence semantics from different aspects. Moreover, multiview decoding paradigm utilizes both autoencoding and auto-regressive decoders in representation bottleneck pre-training, aiming to provide both reconstructive and generative signals for better contextual representation pretraining. We refer to this multi-view pretraining method as CoT-MAE v2. Through extensive experiments, we show that CoT-MAE v2 is effective and robust on large-scale passage retrieval benchmarks and out-of-domain zero-shot benchmarks.
Face Animation with an Attribute-Guided Diffusion Model

Face animation has achieved much progress in computer vision. However, prevailing GAN-based methods suffer from unnatural distortions and artifacts due to sophisticated motion deformation. In this paper, we propose a Face Animation framework with an attribute-guided Diffusion Model (FADM), which is the first work to exploit the superior modeling capacity of diffusion models for photo-realistic talking-head generation. To mitigate the uncontrollable synthesis effect of the diffusion model, we design an Attribute-Guided Conditioning Network (AGCN) to adaptively combine the coarse animation features and 3D face reconstruction results, which can incorporate appearance and motion conditions into the diffusion process. These specific designs help FADM rectify unnatural artifacts and distortions, and also enrich high-fidelity facial details through iterative diffusion refinements with accurate animation attributes. FADM can flexibly and effectively improve existing animation videos. Extensive experiments on widely used talking-head benchmarks validate the effectiveness of FADM over prior arts.
The Concept of Forward-Forward Learning Applied to a Multi Output Perceptron

The concept of a recently proposed Forward-Forward learning algorithm for fully connected artificial neural networks is applied to a single multi output perceptron for classification. The parameters of the system are trained with respect to increased (decreased) "goodness" for correctly (incorrectly) labelled input samples. Basic numerical tests demonstrate that the trained perceptron effectively deals with data sets that have non-linear decision boundaries. Moreover, the overall performance is comparable to more complex neural networks with hidden layers. The benefit of the approach presented here is that it only involves a single matrix multiplication.
Diffusion Models as Masked Autoencoders

There has been a longstanding belief that generation can facilitate a true understanding of visual data. In line with this, we revisit generatively pre-training visual representations in light of recent interest in denoising diffusion models. While directly pre-training with diffusion models does not produce strong representations, we condition diffusion models on masked input and formulate diffusion models as masked autoencoders (DiffMAE). Our approach is capable of (i) serving as a strong initialization for downstream recognition tasks, (ii) conducting high-quality image inpainting, and (iii) being effortlessly extended to video where it produces state-of-the-art classification accuracy. We further perform a comprehensive study on the pros and cons of design choices and build connections between diffusion models and masked autoencoders.
Large language models effectively leverage document-level context for literary translation, but critical errors persist

Large language models (LLMs) are competitive with the state of the art on a wide range of sentence-level translation datasets. However, their ability to translate paragraphs and documents remains unexplored because evaluation in these settings is costly and difficult. We show through a rigorous human evaluation that asking the Gpt-3.5 (text-davinci-003) LLM to translate an entire literary paragraph (e.g., from a novel) at once results in higher-quality translations than standard sentence-by-sentence translation across 18 linguistically-diverse language pairs (e.g., translating into and out of Japanese, Polish, and English). Our evaluation, which took approximately 350 hours of effort for annotation and analysis, is conducted by hiring translators fluent in both the source and target language and asking them to provide both span-level error annotations as well as preference judgments of which system's translations are better. We observe that discourse-level LLM translators commit fewer mistranslations, grammar errors, and stylistic inconsistencies than sentence-level approaches. With that said, critical errors still abound, including occasional content omissions, and a human translator's intervention remains necessary to ensure that the author's voice remains intact. We publicly release our dataset and error annotations to spur future research on evaluation of document-level literary translation.
When do you need Chain-of-Thought Prompting for ChatGPT?

Chain-of-Thought (CoT) prompting can effectively elicit complex multi-step reasoning from Large Language Models~(LLMs). For example, by simply adding CoT instruction ``Let's think step-by-step'' to each input query of MultiArith dataset, GPT-3's accuracy can be improved from 17.7\% to 78.7\%. However, it is not clear whether CoT is still effective on more recent instruction finetuned (IFT) LLMs such as ChatGPT. Surprisingly, on ChatGPT, CoT is no longer effective for certain tasks such as arithmetic reasoning while still keeping effective on other reasoning tasks. Moreover, on the former tasks, ChatGPT usually achieves the best performance and can generate CoT even without being instructed to do so. Hence, it is plausible that ChatGPT has already been trained on these tasks with CoT and thus memorized the instruction so it implicitly follows such an instruction when applied to the same queries, even without CoT. Our analysis reflects a potential risk of overfitting/bias toward instructions introduced in IFT, which becomes more common in training LLMs. In addition, it indicates possible leakage of the pretraining recipe, e.g., one can verify whether a dataset and instruction were used in training ChatGPT. Our experiments report new baseline results of ChatGPT on a variety of reasoning tasks and shed novel insights into LLM's profiling, instruction memorization, and pretraining dataset leakage.
I2I: Initializing Adapters with Improvised Knowledge

Adapters present a promising solution to the catastrophic forgetting problem in continual learning. However, training independent Adapter modules for every new task misses an opportunity for cross-task knowledge transfer. We propose Improvise to Initialize (I2I), a continual learning algorithm that initializes Adapters for incoming tasks by distilling knowledge from previously-learned tasks' Adapters. We evaluate I2I on CLiMB, a multimodal continual learning benchmark, by conducting experiments on sequences of visual question answering tasks. Adapters trained with I2I consistently achieve better task accuracy than independently-trained Adapters, demonstrating that our algorithm facilitates knowledge transfer between task Adapters. I2I also results in better cross-task knowledge transfer than the state-of-the-art AdapterFusion without incurring the associated parametric cost.
What Do Children and Parents Want and Perceive in Conversational Agents? Towards Transparent, Trustworthy, Democratized Agents

Historically, researchers have focused on analyzing Western, Educated, Industrialized Rich and Democratic (WEIRD), adult perspectives on technology. This means we may not have technology developed appropriately for children and those from non-WEIRD countries. In this paper, we analyze children and parents from various countries’ perspectives on an emerging technology: conversational agents. We aim to better understand participants’ trust of agents, partner models, and their ideas of “ideal future agents” such that researchers can better design for these users—for instance, by ensuring children do not overtrust agents. Additionally, we empower children and parents to program their own agents through educational workshops, and present changes in perceptions as participants create and learn about agents. Results from the study (n=49) included how children felt agents were significantly more human-like, warm, and dependable than parents did, how overall participants trusted agents more than parents or friends for correct information, how children described their ideal agents as being more artificial than human-like than parents did, and how children tended to focus more on fun features, approachable/friendly features and addressing concerns through agent design than parents did, among other results. We also discuss potential agent design implications of the results, including how designers may be able to best foster appropriate levels of trust towards agents by focusing on designing agents’ competence and predictability indicators, as well as increasing transparency in terms of agents’ information sources.
An End-to-End Model for Multi-View Scene Text Recognition

Due to the increasing applications of surveillance and monitoring such as person re-identification, vehicle re-identification and sports events tracking, the necessity of text detection and end-to-end recognition is also growing. Although the past deep learning-based models have addressed several challenges, such as arbitrary-shaped text, multiple scripts, and variation in the geometric structure of characters, the scope of the models is limited to a single view. This paper presents an end-to-end model for text recognition through refining the multi-views of the same scene, which is called E2EMVSTR (End-to-End Model for Multi-View Scene Text Recognition). Considering the common characteristics shared in multi-view texts, we propose a cycle consistency pairwise similarity-based deep learning model to find texts more efficiently in three input views. Further, the extracted texts are supplied to a Siamese network and semi-supervised attention embedding combinational network for obtaining recognition results. The proposed model combines natural language processing and genetic algorithm models to restore missing character information and correct wrong recognition results. In experiments on our multi-view dataset and several benchmark datasets, the proposed method is proven effective compared to state-of-the-art methods. The dataset and codes will be made available to the public upon acceptance.
A controllable sharing scheme of logistics data based on blockchain

With the development of cooperation and communication between different enterprises in the field of logistics, a challenge is that few effective ways can solve the contradiction between data sharing and privacy protection. In this paper, we propose a novel scheme against the contradiction. Benefitting from the performance of blockchain and smart contract technology, we realize the identity authentication, data exchange and transaction process among different stakeholders. At the same time, the shared data is encrypted with Ciphertext-Policy Attribute-Based Encryption (CP-ABE) technology, so as to achieve the confidentiality protection and access control during the process of data sharing. In addition, by introducing the Handle system, we participate in data sharing through data identifiers to keep the data on the local server and enhance the security of the data. In summary, the scheme combines Blockchain, CP-ABE and Handle technologies to provide an efficient and secure new approach to data sharing among collaborators with data sharing needs in the logistics field.
Power helmet detection based on improved YOLOX

For power safety operations, the environmental background is complex, and the construction workers have different forms of small targets, thus bringing great difficulties to the helmet wearing situation detection. This paper proposes a deep learning-based method for detecting the wearing condition of electric safety helmets. The method is based on the YOLOX target detection method, and its network structure has been improved. Firstly, for the complex background, the CA-Net(Coordinate Attention) attention mechanism is integrated in the backbone feature output layer to effectively retain the location information of the target by increasing the proportion of the weight of the important feature channels in the coordinate space to achieve the effective differentiation between the target and the background. The feature fusion layer effectively preserves small target features by adding large scale output feature maps to improve the detection accuracy of small targets. The effectiveness of the improved YOLOX algorithm is verified in a real image dataset, and the results show that the M-AP (mean-average accuracy) of the improved algorithm is 96.39%, which is about 2% improvement compared to the original network, and the detection speed FPS (Frames Per Second) is up to 23.83, which meets the real-time and accuracy requirements of engineering applications.
Research on Simulation Method of Ship Equipment maintenance Support process

Due to the long mission cycle and continuity of tasks performed by ships at present, there are high requirements for the maintenance of ship equipment during missions. In order to improve the working efficiency of ship equipment during missions, an algorithm model is proposed to simulate the maintenance and guarantee process of ship equipment. Firstly, from the event-driven point of view, we study the generation of events and the model of ship equipment maintenance and guarantee process and specify its invocation trigger mechanism; then we establish the event scheduling simulation process to divide the task phases, realize the simulation of the maintenance and guarantee process and output the maintenance and guarantee evaluation results; finally, the functional modules are designed and validated by example. The results show that the algorithm is feasible and effective, and can be used for equipment maintenance security evaluation.
Yolov5-based rotating target pose grasping

The manufacturing industry is the backbone of the country and supports the development of the national economy. Due to the wide variety of industrial parts and the extreme complexity and variability of the industrial environment, artificially intelligent grasping and sorting technology has raised higher requirements. This has made machine learning-based, computer vision neural networks for visual grasping technology the focus of research in the field of industrial intelligent manufacturing. However, the focus of traditional common computer vision neural networks currently lies mainly on target detection and recognition classification, while the industrial work environment is cluttered with industrial parts placement and the current grasping pose problem has to be implemented with the help of non-neural network methods. This leads to a bloated and complex grasping and sorting system and is highly prone to problems such as missed grasping and misgrasping under problematic conditions such as mutual occlusion, which hinders the application and development of intelligent industrial production. In this paper, from the perspective of improving the algorithmic function of deep neural networks, we propose a target recognition and pose localization algorithm based on the Yolov5 model to achieve continuous and effective classification recognition localization and pose grasping of industrial parts moving on a conveyor belt using only one neural network.
The influence of axiological sphere of a person on the psychological readiness for entrepreneurial activity

The paper studies the influence of values and socio-psychological attitudes towards the psychological readiness for entrepreneurship of the students of humanitarian training programs. The study is caused by the demand of the government for the search, development, and support of initiative young people who are focused on entrepreneurial activity, who want to carry on individual business and benefit society. The empirical study sample involves the Humanities students. The research was carried out on the base of the laboratory of V.G. Timiryasov Kazan Innovative University. The M. Rokeach Value Survey, O.F. Potemkina’s “Social-Psychological Attitudes of a Personality in the Motivation-Need Sphere” test, and E.K. Klimova’s “The Diagnosis of Psychological Readiness for Entrepreneurial Activity” questionnaire were the diagnostic tools. The authors compiled the ranking of values and attitudes of students in the motivation-need sphere. It was determined that for the surveyed students, the concrete values – “health” and “self-confidence” are the most significant, and the abstract values, not related to their ego – “the beauty of nature” and “happiness of others” are the least significant. Among values-means, action and communication values and ethic values are significant. The students manifested a dominant attitude towards freedom and a low level of general psychological readiness for entrepreneurship combined with the pronounced intention to become entrepreneurs and high enterprise self-esteem. The authors established the influence of values and socio-psychological attitudes towards the psychological readiness for entrepreneurial activity of the Humanities students. The values of active life and life wisdom, the focus on money, freedom, altruism, and selfishness play an activating role in it.
Psycho-emotional peculiarities of workers from legal field

The activity in legal field is varied, taking into account that this large sphere is related to the defense of fundamental rights and freedom of a person, duet to the responsibility provided, emanated by the workers from justice field, the responsibility to protect fundamental norms in force. Workers of legal, justice field, daily confront situations of infringements of civil, criminal and administrative laws, as well as the violation of procedural-criminal order. In the given study we are going to determine the psycho-emotional peculiarities of specialists from justice/legal field, namely to research the level of manifestation of anxiety, aggressiveness, rigidity and frustration. The sample of the given study is formed by jurists, attorneys, as well as prosecuting officers – the majority of subjects are men with different age and professional experience in the field.