OpenRaiser/Intern-Atlas huggingface dataset:
Who breaks early, looses: goal oriented training of deep neural networks based on port Hamiltonian dynamics
Venue: arXiv cs.LG 2023
Year: 2023

The highly structured energy landscape of the loss as a function of parameters for deep neural networks makes it necessary to use sophisticated optimization strategies in order to discover (local) minima that guarantee reasonable performance. Overcoming less suitable local minima is an important prerequisite and often momentum methods are employed to achieve this. As in other non local optimization procedures, this however creates the necessity to balance between exploration and exploitation. In this work, we suggest an event based control mechanism for switching from exploration to exploitation based on reaching a predefined reduction of the loss function. As we give the momentum method a port Hamiltonian interpretation, we apply the 'heavy ball with friction' interpretation and trigger breaking (or friction) when achieving certain goals. We benchmark our method against standard stochastic gradient descent and provide experimental evidence for improved performance of deep neural networks when our strategy is applied.
BCE-Net: Reliable Building Footprints Change Extraction based on Historical Map and Up-to-Date Images using Contrastive Learning
Venue: arXiv cs.CV 2023
Year: 2023

Automatic and periodic recompiling of building databases with up-to-date high-resolution images has become a critical requirement for rapidly developing urban environments. However, the architecture of most existing approaches for change extraction attempts to learn features related to changes but ignores objectives related to buildings. This inevitably leads to the generation of significant pseudo-changes, due to factors such as seasonal changes in images and the inclination of building façades. To alleviate the above-mentioned problems, we developed a contrastive learning approach by validating historical building footprints against single up-to-date remotely sensed images. This contrastive learning strategy allowed us to inject the semantics of buildings into a pipeline for the detection of changes, which is achieved by increasing the distinguishability of features of buildings from those of non-buildings. In addition, to reduce the effects of inconsistencies between historical building polygons and buildings in up-to-date images, we employed a deformable convolutional neural network to learn offsets intuitively. In summary, we formulated a multi-branch building extraction method that identifies newly constructed and removed buildings, respectively. To validate our method, we conducted comparative experiments using the public Wuhan University building change detection dataset and a more practical dataset named SI-BU that we established. Our method achieved F1 scores of 93.99% and 70.74% on the above datasets, respectively. Moreover, when the data of the public dataset were divided in the same manner as in previous related studies, our method achieved an F1 score of 94.63%, which surpasses that of the state-of-the-art method.
Near Tight Shortest Paths in the Hybrid Model
Venue: arXiv cs.DS 2023
Year: 2023

Shortest paths problems are subject to extensive studies in classic distributed models such as the CONGEST or Congested Clique. These models dictate how nodes may communicate in order to determine shortest paths in a distributed input graph. This article focuses on shortest paths problems in the HYBRID model, which combines local communication along edges of the input graph with global communication between arbitrary pairs of nodes that is restricted in terms of bandwidth. Previous work showed that it takes $\tilde Ω(\!\sqrt{k})$ rounds in the \hybrid model for each node to learn its distance to $k$ dedicated source nodes (aka the $k$-SSP problem), even for crude approximations. This lower bound was also matched with algorithmic solutions for $k \geq n^{2/3}$. However, as $k$ gets smaller, the gap between the known upper and lower bounds diverges and even becomes exponential for a single source. In this work we close this gap for the whole range of $k$ (up to terms that are polylogarithmic in $n$), by giving algorithmic solutions for $k$-SSP in $\tilde O\big(\!\sqrt k\big)$ rounds for any $k$.
Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering
Venue: arXiv cs.CL 2023
Year: 2023

Having an intelligent dialogue agent that can engage in conversational question answering (ConvQA) is now no longer limited to Sci-Fi movies only and has, in fact, turned into a reality. These intelligent agents are required to understand and correctly interpret the sequential turns provided as the context of the given question. However, these sequential questions are sometimes left implicit and thus require the resolution of some natural language phenomena such as anaphora and ellipsis. The task of question rewriting has the potential to address the challenges of resolving dependencies amongst the contextual turns by transforming them into intent-explicit questions. Nonetheless, the solution of rewriting the implicit questions comes with some potential challenges such as resulting in verbose questions and taking conversational aspect out of the scenario by generating self-contained questions. In this paper, we propose a novel framework, CONVSR (CONVQA using Structured Representations) for capturing and generating intermediate representations as conversational cues to enhance the capability of the QA model to better interpret the incomplete questions. We also deliberate how the strengths of this task could be leveraged in a bid to design more engaging and eloquent conversational agents. We test our model on the QuAC and CANARD datasets and illustrate by experimental results that our proposed framework achieves a better F1 score than the standard question rewriting model.
Uncoordinated Interference Avoidance Between Terrestrial and Non-Terrestrial Communications
Venue: arXiv cs.NI 2023
Year: 2023

This paper proposes an algorithm that uses geospatial analytics and the muting of physical resources in next-generation base stations (BSs) to avoid interference between cellular (or terrestrial) and satellite communication (non-terrestrial) systems. The information exchange between satellite and terrestrial stations is minimal, but a hybrid edge cloud node with access to estimated satellite trajectories can enable these BSs to take proactive steps to avoid interference. To validate the superiority of our proposed algorithm over a conventional method, we show the performance of the algorithm using two measures: number of concurrent uses of Doppler corrected radio frequency resources and the sum-rate capacity of the BSs. Our algorithm not only provides significant sum-rate capacity gains in both directions enabling better use of the spectrum, but also runs in polynomial time, making it suitable for real-time interference avoidance.
LoRe: A Programming Model for Verifiably Safe Local-First Software
Venue: arXiv cs.PL 2023
Year: 2023

Local-first software manages and processes private data locally while still enabling collaboration between multiple parties connected via partially unreliable networks. Such software typically involves interactions with users and the execution environment (the outside world). The unpredictability of such interactions paired with their decentralized nature make reasoning about the correctness of local-first software a challenging endeavor. Yet, existing solutions to develop local-first software do not provide support for automated safety guarantees and instead expect developers to reason about concurrent interactions in an environment with unreliable network conditions. We propose LoRe, a programming model and compiler that automatically verifies developer-supplied safety properties for local-first applications. LoRe combines the declarative data flow of reactive programming with static analysis and verification techniques to precisely determine concurrent interactions that violate safety invariants and to selectively employ strong consistency through coordination where required. We propose a formalized proof principle and demonstrate how to automate the process in a prototype implementation that outputs verified executable code. Our evaluation shows that LoRe simplifies the development of safe local-first software when compared to state-of-the-art approaches and that verification times are acceptable.
TUM-FAÇADE: Reviewing and enriching point cloud benchmarks for façade segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Point clouds are widely regarded as one of the best dataset types for urban mapping purposes. Hence, point cloud datasets are commonly investigated as benchmark types for various urban interpretation methods. Yet, few researchers have addressed the use of point cloud benchmarks for façade segmentation. Robust façade segmentation is becoming a key factor in various applications ranging from simulating autonomous driving functions to preserving cultural heritage. In this work, we present a method of enriching existing point cloud datasets with façade-related classes that have been designed to facilitate façade segmentation testing. We propose how to efficiently extend existing datasets and comprehensively assess their potential for façade segmentation. We use the method to create the TUM-FAÇADE dataset, which extends the capabilities of TUM-MLS-2016. Not only can TUM-FAÇADE facilitate the development of point-cloud-based façade segmentation tasks, but our procedure can also be applied to enrich further datasets.
The Night Shift: Understanding Performance Variability of Cloud Serverless Platforms
Venue: arXiv cs.DC 2023
Year: 2023

Function-as-a-Service is a popular cloud programming model that supports developers by abstracting away most operational concerns with automatic deployment and scaling of applications. Due to the high level of abstraction, developers rely on the cloud platform to offer a consistent service level, as decreased performance leads to higher latency and higher cost given the pay-per-use model. In this paper, we measure performance variability of Google Cloud Functions over multiple months. Our results show that diurnal patterns can lead to performance differences of up to 15%, and that the frequency of unexpected cold starts increases threefold during the start of the week. This behavior can negatively impact researchers that conduct performance studies on cloud platforms and practitioners that run cloud applications.
Machine Learning-Based Multi-Objective Design Exploration Of Flexible Disc Elements
Venue: arXiv cs.NE 2023
Year: 2023

Design exploration is an important step in the engineering design process. This involves the search for design/s that meet the specified design criteria and accomplishes the predefined objective/s. In recent years, machine learning-based approaches have been widely used in engineering design problems. This paper showcases Artificial Neural Network (ANN) architecture applied to an engineering design problem to explore and identify improved design solutions. The case problem of this study is the design of flexible disc elements used in disc couplings. We are required to improve the design of the disc elements by lowering the mass and stress without lowering the torque transmission and misalignment capability. To accomplish this objective, we employ ANN coupled with genetic algorithm in the design exploration step to identify designs that meet the specified criteria (torque and misalignment) while having minimum mass and stress. The results are comparable to the optimized results obtained from the traditional response surface method. This can have huge advantage when we are evaluating conceptual designs against multiple conflicting requirements.
Learn What Is Possible, Then Choose What Is Best: Disentangling One-To-Many Relations in Language Through Text-based Games
Venue: arXiv cs.CL 2023
Year: 2023

Language models pre-trained on large self-supervised corpora, followed by task-specific fine-tuning has become the dominant paradigm in NLP. These pre-training datasets often have a one-to-many structure--e.g. in dialogue there are many valid responses for a given context. However, only some of these responses will be desirable in our downstream task. This raises the question of how we should train the model such that it can emulate the desirable behaviours, but not the undesirable ones. Current approaches train in a one-to-one setup--only a single target response is given for a single dialogue context--leading to models only learning to predict the average response, while ignoring the full range of possible responses. Using text-based games as a testbed, our approach, PASA, uses discrete latent variables to capture the range of different behaviours represented in our larger pre-training dataset. We then use knowledge distillation to distil the posterior probability distribution into a student model. This probability distribution is far richer than learning from only the hard targets of the dataset, and thus allows the student model to benefit from the richer range of actions the teacher model has learned. Results show up to 49% empirical improvement over the previous state-of-the-art model on the Jericho Walkthroughs dataset.
Planar and Minor-Free Metrics Embed into Metrics of Polylogarithmic Treewidth with Expected Multiplicative Distortion Arbitrarily Close to 1
Venue: arXiv cs.DS 2023
Year: 2023

We prove that there is a randomized polynomial-time algorithm that given an edge-weighted graph $G$ excluding a fixed-minor $Q$ on $n$ vertices and an accuracy parameter $\varepsilon>0$, constructs an edge-weighted graph~$H$ and an embedding $η\colon V(G)\to V(H)$ with the following properties: * For any constant size $Q$, the treewidth of $H$ is polynomial in $\varepsilon^{-1}$, $\log n$, and the logarithm of the stretch of the distance metric in $G$. * The expected multiplicative distortion is $(1+\varepsilon)$: for every pair of vertices $u,v$ of $G$, we have $\mathrm{dist}_H(η(u),η(v))\geq \mathrm{dist}_G(u,v)$ always and $\mathrm{Exp}[\mathrm{dist}_H(η(u),η(v))]\leq (1+\varepsilon)\mathrm{dist}_G(u,v)$. Our embedding is the first to achieve polylogarithmic treewidth of the host graph and comes close to the lower bound by Carroll and Goel, who showed that any embedding of a planar graph with $\mathcal{O}(1)$ expected distortion requires the host graph to have treewidth $Ω(\log n)$. It also provides a unified framework for obtaining randomized quasi-polynomial-time approximation schemes for a variety of problems including network design, clustering or routing problems, in minor-free metrics where the optimization goal is the sum of selected distances. Applications include the capacitated vehicle routing problem, and capacitated clustering problems.
Designing Fair, Cost-optimal Auctions based on Deep Learning for Procuring Agricultural Inputs through Farmer Collectives
Venue: arXiv cs.GT 2023
Year: 2023

Procuring agricultural inputs (agri-inputs for short) such as seeds, fertilizers, and pesticides, at desired quality levels and at affordable cost, forms a critical component of agricultural input operations. This is a particularly challenging problem being faced by small and marginal farmers in any emerging economy. Farmer collectives (FCs), which are cooperative societies of farmers, offer an excellent prospect for enabling cost-effective procurement of inputs with assured quality to the farmers. In this paper, our objective is to design sound, explainable mechanisms by which an FC will be able to procure agri-inputs in bulk and distribute the inputs procured to the individual farmers who are members of the FC. In the methodology proposed here, an FC engages qualified suppliers in a competitive, volume discount procurement auction in which the suppliers specify price discounts based on volumes supplied. The desiderata of properties for such an auction include: minimization of the total cost of procurement; incentive compatibility; individual rationality; fairness; and other business constraints. An auction satisfying all these properties is analytically infeasible and a key contribution of this paper is to develop a deep learning based approach to design such an auction. We use two realistic, stylized case studies from chili seeds procurement and a popular pesticide procurement to demonstrate the efficacy of these auctions.
Cross-domain Food Image-to-Recipe Retrieval by Weighted Adversarial Learning
Venue: arXiv cs.MM 2023
Year: 2023

Food image-to-recipe aims to learn an embedded space linking the rich semantics in recipes with the visual content in food image for cross-modal retrieval. The existing research works carry out the learning of such space by assuming that all the image-recipe training example pairs belong to the same cuisine. As a result, despite the excellent performance reported in the literature, such space is not transferable for retrieving recipes of different cuisine. In this paper, we aim to address this issue by cross-domain food image-to-recipe retrieval, such that by leveraging abundant image-recipe pairs in source domain (one cuisine), the embedding space is generalizable to a target domain (the other cuisine) that does not have images to pair with recipes for training. With the intuition that the importance of different source samples should vary, this paper proposes two novel mechanisms for cross-domain food image-to-recipe retrieval, i.e., source data selector and weighted cross-modal adversarial learning. The former aims to select source samples similar to the target data and filter out distinctive ones for training. The latter is capable to assign higher weights to the source samples more similar to the target data and lower weights to suppress the distinctive ones for both cross-modal and adversarial learning. The weights are computed from the recipe features extracted from a pre-trained source model. Experiments on three different cuisines (Chuan, Yue and Washoku) demonstrate that the proposed method manages to achieve state-of-the-art performances in all the transfers.
Repeated Principal-Agent Games with Unobserved Agent Rewards and Perfect-Knowledge Agents
Venue: arXiv cs.LG 2023
Year: 2023

Motivated by a number of real-world applications from domains like healthcare and sustainable transportation, in this paper we study a scenario of repeated principal-agent games within a multi-armed bandit (MAB) framework, where: the principal gives a different incentive for each bandit arm, the agent picks a bandit arm to maximize its own expected reward plus incentive, and the principal observes which arm is chosen and receives a reward (different than that of the agent) for the chosen arm. Designing policies for the principal is challenging because the principal cannot directly observe the reward that the agent receives for their chosen actions, and so the principal cannot directly learn the expected reward using existing estimation techniques. As a result, the problem of designing policies for this scenario, as well as similar ones, remains mostly unexplored. In this paper, we construct a policy that achieves a low regret (i.e., square-root regret up to a log factor) in this scenario for the case where the agent has perfect-knowledge about its own expected rewards for each bandit arm. We design our policy by first constructing an estimator for the agent's expected reward for each bandit arm. Since our estimator uses as data the sequence of incentives offered and subsequently chosen arms, the principal's estimation can be regarded as an analogy of online inverse optimization in MAB's. Next we construct a policy that we prove achieves a low regret by deriving finite-sample concentration bounds for our estimator. We conclude with numerical simulations demonstrating the applicability of our policy to real-life setting from collaborative transportation planning.
Masked Pre-Training of Transformers for Histology Image Analysis
Venue: arXiv cs.CV 2023
Year: 2023

In digital pathology, whole slide images (WSIs) are widely used for applications such as cancer diagnosis and prognosis prediction. Visual transformer models have recently emerged as a promising method for encoding large regions of WSIs while preserving spatial relationships among patches. However, due to the large number of model parameters and limited labeled data, applying transformer models to WSIs remains challenging. Inspired by masked language models, we propose a pretext task for training the transformer model without labeled data to address this problem. Our model, MaskHIT, uses the transformer output to reconstruct masked patches and learn representative histological features based on their positions and visual features. The experimental results demonstrate that MaskHIT surpasses various multiple instance learning approaches by 3% and 2% on survival prediction and cancer subtype classification tasks, respectively. Furthermore, MaskHIT also outperforms two of the most recent state-of-the-art transformer-based methods. Finally, a comparison between the attention maps generated by the MaskHIT model with pathologist's annotations indicates that the model can accurately identify clinically relevant histological structures in each task.
Video Generation Beyond a Single Clip
Venue: arXiv cs.CV 2023
Year: 2023

We tackle the long video generation problem, i.e.~generating videos beyond the output length of video generation models. Due to the computation resource constraints, video generation models can only generate video clips that are relatively short compared with the length of real videos. Existing works apply a sliding window approach to generate long videos at inference time, which is often limited to generating recurrent events or homogeneous content. To generate long videos covering diverse content and multiple events, we propose to use additional guidance to control the video generation process. We further present a two-stage approach to the problem, which allows us to utilize existing video generation models to generate high-quality videos within a small time window while modeling the video holistically based on the input guidance. The proposed approach is complementary to existing efforts on video generation, which focus on generating realistic video within a fixed time window. Extensive experiments on challenging real-world videos validate the benefit of the proposed method, which improves over state-of-the-art by up to 9.5% in objective metrics and is preferred by users more than 80% of time.
Virtual Reality Training of Social Skills in Autism Spectrum Disorder: An Examination of Acceptability, Usability, User Experience, Social Skills, and Executive Functions
Venue: arXiv cs.HC 2023
Year: 2023

Poor social skills in autism spectrum disorder (ASD) are associated with reduced independence in daily life. Current interventions for improving the social skills of individuals with ASD fail to represent the complexity of real-life social settings and situations. Virtual reality (VR) may facilitate social skills training in social environments and situations proximal to real life, however, more research is needed for elucidating aspects such as the acceptability, usability, and user experience of VR systems in ASD. Twenty-five participants with ASD attended a neuropsychological evaluation and three sessions of VR social skills training, incorporating five (5) social scenarios with three difficulty levels for each. Participants reported high acceptability, system usability, and user experience. Significant correlations were observed between performance in social scenarios, self-reports, and executive functions. Working memory and planning ability were significant predictors of functionality level in ASD and the VR system's perceived usability respectively. Yet, performance in social scenarios was the best predictor of usability, acceptability, and functionality level in ASD. Planning ability substantially predicted performance in social scenarios, postulating an implication in social skills. Immersive VR social skills training appears effective in individuals with ASD, yet an error-less approach, which is adaptive to the individual's needs, should be preferred.