pwc-archive/papers-with-abstracts huggingface dataset:
FAMO: Fast Adaptive Multitask Optimization
Authors: Bo Liu, Yihao Feng, Peter Stone, Qiang Liu
Date: 2023-06-06 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Computational Efficiency

One of the grand enduring goals of AI is to create generalist agents that can learn multiple different tasks from diverse data via multitask learning (MTL). However, in practice, applying gradient descent (GD) on the average loss across all tasks may yield poor multitask performance due to severe under-optimization of certain tasks. Previous approaches that manipulate task gradients for a more balanced loss decrease require storing and computing all task gradients ($\mathcal{O}(k)$ space and time where $k$ is the number of tasks), limiting their use in large-scale scenarios. In this work, we introduce Fast Adaptive Multitask Optimization FAMO, a dynamic weighting method that decreases task losses in a balanced way using $\mathcal{O}(1)$ space and time. We conduct an extensive set of experiments covering multi-task supervised and reinforcement learning problems. Our results indicate that FAMO achieves comparable or superior performance to state-of-the-art gradient manipulation techniques while offering significant improvements in space and computational efficiency. Code is available at \url{https://github.com/Cranial-XIX/FAMO}.
Collapsed Inference for Bayesian Deep Learning
Authors: Zhe Zeng, Guy Van Den Broeck
Date: 2023-06-16 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Deep Learning

Bayesian neural networks (BNNs) provide a formalism to quantify and calibrate uncertainty in deep learning. Current inference approaches for BNNs often resort to few-sample estimation for scalability, which can harm predictive performance, while its alternatives tend to be computationally prohibitively expensive. We tackle this challenge by revealing a previously unseen connection between inference on BNNs and volume computation problems. With this observation, we introduce a novel collapsed inference scheme that performs Bayesian model averaging using collapsed samples. It improves over a Monte-Carlo sample by limiting sampling to a subset of the network weights while pairing it with some closed-form conditional distribution over the rest. A collapsed sample represents uncountably many models drawn from the approximate posterior and thus yields higher sample efficiency. Further, we show that the marginalization of a collapsed sample can be solved analytically and efficiently despite the non-linearity of neural networks by leveraging existing volume computation solvers. Our proposed use of collapsed samples achieves a balance between scalability and accuracy. On various regression and classification tasks, our collapsed Bayesian deep learning approach demonstrates significant improvements over existing methods and sets a new state of the art in terms of uncertainty estimation as well as predictive performance.
OpenMedCalc: Augmentation of ChatGPT with Clinician-Informed Tools Improves Performance on Medical Calculation Tasks
Authors: Alex J Goodell, Simon N Chu, Dara Rouholiman, Larry F Chu
Date: 2023-12-15 00:00:00
Proceeding: medRxiv 2023 12
Tasks: Clinical Information Retreival, Clinical Language Translation, Medical Diagnosis, Professional Medicine

Prior work has shown that large language models (LLMs) have the ability to answer expert-level multiple choice questions in medicine, but are limited by both their tendency to hallucinate knowledge and their inherent inadequacy in performing basic mathematical operations. Unsurprisingly, early evidence suggests that LLMs perform poorly when asked to execute common clinical calculations. Recently, it has been demonstrated that LLMs have the capability of interacting with external programs and tools, presenting a possible remedy for this limitation. In this study, we explore the ability of ChatGPT (GPT-4, November 2023) to perform medical calculations, evaluating its performance across 48 diverse clinical calculation tasks. Our findings indicate that ChatGPT is an unreliable clinical calculator, delivering inaccurate responses in one-third of trials (n=212). To address this, we developed an open-source clinical calculation API (openmedcalc.org), which we then integrated with ChatGPT. We subsequently evaluated the performance of this augmented model by comparing it against standard ChatGPT using 75 clinical vignettes in three common clinical calculation tasks: Caprini VTE Risk, Wells DVT Criteria, and MELD-Na. The augmented model demonstrated a marked improvement in accuracy over unimproved ChatGPT. Our findings suggest that integration of machine-usable, clinician-informed tools can help alleviate the reliability limitations observed in medical LLMs.
High Performance on Atari Games Using Perceptual Control Architecture Without Training
Authors: Tauseef Gulrez, Warren Mansell
Date: 2022-10-08 00:00:00
Proceeding: Journal of Intelligent and Robotic Systems 2022 10
Tasks: Deep Reinforcement Learning

Deep reinforcement learning (DRL) requires large samples and a long training time to operate optimally. Yet humans rarely
require long periods of training to perform well on novel tasks, such as computer games, once they are provided with an
accurate program of instructions. We used perceptual control theory (PCT) to construct a simple closed-loop model which
requires no training samples and training time within a video game study using the Arcade Learning Environment (ALE).
The model was programmed to parse inputs from the environment into hierarchically organised perceptual signals, and it
computed a dynamic error signal by subtracting the incoming signal for each perceptual variable from a reference signal to
drive output signals to reduce this error. We tested the same model across three different Atari games Breakout, Pong and
Video Pinball to achieve performance at least as high as DRL paradigms, and close to good human performance. Our study
shows that perceptual control models, based on simple assumptions, can perform well without learning. We conclude by
specifying a parsimonious role of learning that may be more similar to psychological functioning.
FaraPy: An Augmented Reality Feedback System for Facial Paralysis using Action Unit Intensity Estimation
Authors: Giuliana Barrios Dell’Olio;Misha Sra
Date: 2021-10-10 00:00:00
Proceeding: Barrios Dell'Olio G, Sra M. FaraPy: An Augmented Reality Feedback System for Facial Paralysis using Action Unit Intensity Estimation. InThe 34th Annual ACM Symposium on User Interface Software and Technology 2021 10

Facial paralysis is the most common facial nerve disorder. It causes
functional and aesthetic defcitsthat often lead to emotional distress
and afect psychosocial well-being. One form of treatment is mirror
therapy, which has shown potential but has several mirror-related
drawbacks that limit its efectiveness. We propose FaraPy, the frst
mobile augmented reality mirror therapy system for facial paralysis
that provides real-time feedback and tracks user progress over time.
We developed an efcient convolutional neural network to detect
muscle activations and intensities as users perform facial exercises
in front of a mobile device camera. Our model outperformsthe stateof-the-art model on benchmark data for detecting facial action unit
intensity. Our user study (n=20) shows high user satisfaction and
greater preference for our interactive system over traditional mirror
therapy
Context-Aware Online Adaptation of Mixed Reality Interfaces
Authors: David Lindlbauer;Anna Maria Feit;Otmar Hilliges
Date: 2019-10-10 00:00:00
Proceeding: The 32th Annual ACM Symposium on User Interface Software and Technology 2019 10
Tasks: Combinatorial Optimization, Decision Making, Mixed Reality

We present an optimization-based approach for Mixed Reality
(MR) systems to automatically control when and where applications are shown, and how much information they display.
Currently, content creators design applications, and users then
manually adjust which applications are visible and how much
information they show. This choice has to be adjusted every
time users switch context, i.e., whenever they switch their task
or environment. Since context switches happen many times
a day, we believe that MR interfaces require automation to
alleviate this problem. We propose a real-time approach to
automate this process based on users’ current cognitive load
and knowledge about their task and environment. Our system
adapts which applications are displayed, how much information they show, and where they are placed. We formulate this
problem as a mix of rule-based decision making and combinatorial optimization which can be solved effciently in real-time.
We present a set of proof-of-concept applications showing
that our approach is applicable in a wide range of scenarios.
Finally, we show in a dual-task evaluation that our approach
decreased secondary tasks interactions by 36%.
Constraint-Induced Symmetric Nonnegative Matrix Factorization for Accurate Community Detection
Authors: ZhiGang Liu, Xin Luo, Zidong Wang, Xiaohui Liu
Date: 2023-01-01 00:00:00
Proceeding: journal 2023 1
Tasks: Community Detection, Representation Learning

As a fundamental characteristic of an undirected network, community reveals its networking organization and
functional mechanisms, making community detection be a highly-interesting issue in network representation
learning. With great interpretability, a symmetric and nonnegative matrix factorization (SNMF)-based approach
is frequently adopted to tackle this issue. However, it only adopts a unique feature matrix for describing the
symmetry of an undirected network, which unfortunately results in a reduced feature space that evidently impairs its representation learning ability. Motivated by this discovery, this paper proposes a novel Constraintinduced Symmetric Nonnegative Matrix Factorization (C-SNMF) model that adopts three-fold ideas: a) Representing a target undirected network with multiple latent feature matrices, thus preserving its representation
learning capacity; b) Incorporating a symmetry-regularizer into its objective function, which preserves the
symmetry of the learnt low-rank approximation to the adjacency matrix, thereby making the resultant detector
precisely illustrate the target network’s symmetry; and c) Introducing a graph-regularizer that preserves local
invariance of the network’s intrinsic geometry into its learning objective, thus making the achieved detector
well-aware of community structure within the target network. Note that the regularization coefficients are
selected according to the modularity of the learnt community structure on the training data only, thereby greatly
improving the achieved model’s practical significance for real applications. Experimental results on six realworld networks demonstrate that the proposed C-SNMF model significantly outperforms the benchmarks and
state-of-the-art models in achieving highly-accurate community detection results.
A Differential Evolution-Enhanced Position-Transitional Approach to Latent Factor Analysis
Authors: Jia Chen; Renfang Wang; Di Wu; Xin Luo
Date: 2023-07-22 00:00:00
Proceeding: journal 2023 7
Tasks: Position

High-dimensional and sparse (HiDS) matrices are frequently adopted to describe the complex relationships in various big data-related systems and applications. A Position-transitional Latent Factor Analysis (PLFA) model can accurately and efficiently represent an HiDS matrix. However, its involved latent factors are optimized by particle-swarm-optimization-incorporated stochastic gradient descent with the specific gradient direction step-by-step, which may cause a suboptimal solution. To address this issue, this paper proposes a S equential- G roup- D ifferential- E volution (SGDE) algorithm to refine the latent factors optimized by a PLFA model, thereby achieving a highly-accurate SGDE-PLFA model to HiDS matrices. The main idea of SGDE-PLFA is two-fold. First, it divides the obtained latent factors by PLFA into row and column sub-groups to ensure the efficient low-dimensional optimization. Second, it initializes and optimizes these sub-group latent factors with our improved Differential Evolution algorithm to search more accurate solutions nearby the suboptimum. As demonstrated by the extensive experiments on four HiDS matrices, an SGDE-PLFA model outperforms the state-of-the-art models in representing HiDS matrices.
Towards Food Image Retrieval via Generalization-oriented Sampling and Loss Function Design
Authors: Jiajun Song, Zhuo Li, Weiqing Min, Shuqiang Jiang
Date: 2023-08-25 00:00:00
Proceeding: journal 2023 8
Tasks: Image Retrieval, Metric Learning, Retrieval

Food computing has increasingly received widespread attention in the multimedia field. As a basic task of food computing, food image retrieval has wide applications, that is, food image retrieval can help users to find the desired food from a large number of food images. Besides, the retrieved information can be applied to establish a richer database for the subsequent food content-related recommendation. Food image retrieval aims to achieve better performance on novel categories. Thus, it is worth studying to transfer the embedding ability from the training set to the unseen test set, that is, the generalization of the model. Food is influenced by various factors, such as culture and geography, leading to great differences between domains, such as Asian food and western food. Therefore, it is challenging to study the generalization of the model in food image retrieval. In this article, we improve the classical metric learning framework and propose a generalization-oriented sampling strategy, which boosts the generalization of the model by maximizing the intra-class distance from a proportion of positive pairs to avoid the excessive distance compression in the embedding space. Considering that the existing optimization process is in an opposite direction to our proposed sampling strategy, we further propose an adaptive gradient assignment policy named gradient-adaptive optimization, which can alleviate the intra-class distance compression during optimization by assigning different gradients to different samples. Extensive evaluation on three popular food image datasets demonstrates the effectiveness of the proposed method. We also experiment on three popular general datasets to prove that solving the problem from the generalization can also improve the performance of general image retrieval.
Vision-Based Fruit Recognition via Multi-Scale Attention CNN
Authors: Weiqing Min, Zhiling Wang, Jiahao Yang, Chunlin Liu, Shuqiang Jiang
Date: 2023-07-01 00:00:00
Proceeding: journal 2023 7

Fruit quality assessment, grading and sorting are of vital importance to fruit processing, and all these involve fruit recognition. Vision-based fruit recognition can recognize fruit automatically and further support more applications such as fruit-picking robots, self-checkout service, and dietary guidance. Recent works resort to Convolutional Neural Networks (CNNs) for vision-based fruit recognition because of their powerful expressive capacity. However, most works simply utilize existing CNNs and ignore characteristics of fruit images, resulting in sub-optimal recognition performance. To solve this problem, we adopt a Multi-Scale Attention Network (MSANet), which explores attention from different layers of CNNs and aggregates various visual attentional features from different levels into final comprehensive representations. Extensive evaluations on four fruit benchmark datasets demonstrate that the method achieves state-of-the-art recognition performance. In addition, we report various experimental results from different deep networks as baselines. Codes and models will be released upon publication. Models and code are available at http://123.57.42.89/codes/msanet.html
Vision-based Food Nutrition Estimation via RGB-D Fusion Network
Authors: Wenjing Shao, Weiqing Min, Sujuan Hou, Mengjiang Luo, TianHao Li, Yuanjie Zheng, Shuqiang Jiang
Date: 2023-10-25 00:00:00
Proceeding: journal 2023 10
Tasks: Nutrition

With the development of deep learning technology, vision-based food nutrition estimation is gradually entering the public view for its advantage in accuracy and efficiency. In this paper, we designed one RGB-D fusion network, which integrated multimodal feature fusion (MMFF) and multi-scale fusion for visioin-based nutrition assessment. MMFF performed effective feature fusion by a balanced feature pyramid and convolutional block attention module. Multi-scale fusion fused different resolution features through feature pyramid network. Both enhanced feature representation to improve the performance of the model. Compared with state-of-the-art methods, the mean value of the percentage mean absolute error (PMAE) for our method reached 18.5%. The PMAE of calories and mass reached 15.0% and 10.8% via the RGB-D fusion network, improved by 3.8% and 8.1%, respectively. Furthermore, this study visualized the estimation results of four nutrients and verified the validity of the method. This research contributed to the development of automated food nutrient analysis (Code and models can be found at http://123.57.42.89/codes/RGB-DNet/nutrition.html).
Using Fuzzy Logic to Learn Abstract Policies in Large-Scale Multi-Agent Reinforcement Learning
Authors: Jingchen Li; Haobin Shi; Kao-Shing Hwang
Date: 2022-04-27 00:00:00
Proceeding: journal 2022 4
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning

Large-scale multiagent reinforcement learning requires huge computation and space costs, and the too-long execution process makes it hard to train policies for agents. This work proposes a concept of fuzzy agent, which is a new paradigm for training homogeneous agents. Aiming at a lightweight and affordable reinforcement learning mechanism for large-scale homogeneous multiagent systems, we break the one-to-one correspondence between agent and policy, designing abstract agents as the substitute for the multiagent to interact with the environment. The Markov decision process models for these abstract agents are conducted by fuzzy logic, which also acts on the behavior mapping from abstract agent to entity. Specifically, just the abstract agents execute their policy at a time step, and the concrete behaviors are generated by simple matrix operations. The proposal has lower space and computation complexities because the number of abstract agents is far less than that of entities, and the coupling among agents is retained implicitly. Compared with other approximation and simplification methods, the proposed fuzzy agent not only greatly reduces required computing resources but also ensures the effectiveness of the learned policies. Several experiments are conducted to validate our method. The results show that the proposal outperforms the baseline methods, while it has satisfactory zero-shot and few-shot transfer abilities.
A collaboration of multi-agent model using an interactive interface
Authors: Jingchen Li, Fan Wu, Haobin Shi, Kao-Shing Hwang
Date: 2022-09-01 00:00:00
Proceeding: journal 2022 9
Tasks: Multi-agent Reinforcement Learning

Multi-agent reinforcement learning algorithms scarcely attend to noisy environments, in which agents are inhibited from achieving optimal policy training and making correct decisions. This work investigates the effect of noises in multi-agent environments and proposes a multi-agent actor-critic with collaboration (MACC) model. The model uses lightweight communication to overcome the interference from noises. There are two policies for each agent in MACC: collaboration policy and behavior policy. The behavior of an agent not only depends on its own state but also be influenced by each other agent through a scalar, collaboration value. The collaboration value is generated by the collaboration policy for each individual agent, and it ensures a succinct consensus about the environment. This paper elaborates on the training of the collaboration policy and specifies how it coordinates the behavior policy in a manner of temporal abstraction mechanism, while the observation sequence is considered for more accurate perception. Several experiments on multi-agent collaboration simulation platforms demonstrate that the MACC performs better than baselines in noisy environments, especially in partially observable environments.
Contrastive Latent Variable Models for Neural Text Generation
Authors: Zhiyang Teng, Chenhua Chen, Yan Zhang, Yue Zhang
Date: 2023-03-05 00:00:00
Proceeding: Conference 2023 3
Tasks: Contrastive Learning, Text Generation

Deep latent variable models such as variational autoencoders and energy-based models are widely used for neural text generation. Most of them focus on matching the prior distribution with the posterior distribution of the latent variable for text reconstruction. In addition to instance-level reconstruction, this paper aims to integrate contrastive learning in the latent space, forcing the latent variables to learn high-level semantics by exploring inter-instance relationships. Experiments on various text generation benchmarks show the effectiveness of our proposed method. We also empirically show that our method can mitigate the posterior collapse issue for latent variable based text generation models.
LogiQA 2.0—An Improved Dataset for Logical Reasoning in Natural Language Understanding
Authors: Hanmeng Liu; Jian Liu; Leyang Cui; Zhiyang Teng; Nan Duan; Ming Zhou; Yue Zhang
Date: 2023-06-06 00:00:00
Proceeding: journal 2023 6
Tasks: Logical Reasoning, Logical Reasoning Reading Comprehension, Natural Language Inference, Natural Language Understanding, Question Answering, Reading Comprehension

NLP research on logical reasoning regains momentum with the recent releases of a handful of datasets, notably LogiQA and Reclor. Logical reasoning is exploited in many probing tasks over large Pre-trained Language Models (PLMs) and downstream tasks like question-answering and dialogue systems. In this article, we release LogiQA 2.0. The dataset is an amendment and re-annotation of LogiQA in 2020, a large-scale logical reasoning reading comprehension dataset adapted from the Chinese Civil Service Examination. We increase the data size, refine the texts with manual translation by professionals, and improve the quality by removing items with distinctive cultural features like Chinese idioms. Furthermore, we conduct a fine-grained annotation on the dataset and turn it into a two-way natural language inference (NLI) task, resulting in 35 k premise-hypothesis pairs with gold labels, making it the first large-scale NLI dataset for complex logical reasoning. Compared to Question Answering, Natural Language Inference excels in generalizability and helps downstream tasks better. We establish a baseline for logical reasoning in NLI and incite further research.