OpenRaiser/Intern-Atlas huggingface dataset:
Vehicle-to-Grid and ancillary services:a profitability analysis under uncertainty
Venue: arXiv cs.RO 2023
Year: 2023

The rapid and massive diffusion of electric vehicles poses new challenges to the electric system, which must be able to supply these new loads, but at the same time opens up new opportunities thanks to the possible provision of ancillary services. Indeed, in the so-called Vehicle-to-Grid (V2G) set-up, the charging power can be modulated throughout the day so that a fleet of vehicles can absorb an excess of power from the grid or provide extra power during a shortage.To this end, many works in the literature focus on the optimization of each vehicle daily charging profiles to offer the requested ancillary services while guaranteeing a charged battery for each vehicle at the end of the day. However, the size of the economic benefits related to the provision of ancillary services varies significantly with the modeling approaches, different assumptions, and considered scenarios. In this paper we propose a profitability analysis with reference to a recently proposed framework for V2G optimal operation in presence of uncertainty. We provide necessary and sufficient conditions for profitability in a simplified case and we show via simulation that they also hold for the general case.
Traffic-Aware Hierarchical Beam Selection for Cell-Free Massive MIMO
Venue: arXiv cs.IT 2023
Year: 2023

Beam selection for joint transmission in cell-free massive multi-input multi-output systems faces the problem of extremely high training overhead and computational complexity. The traffic-aware quality of service additionally complicates the beam selection problem. To address this issue, we propose a traffic-aware hierarchical beam selection scheme performed in a dual timescale. In the long-timescale, the central processing unit collects wide beam responses from base stations (BSs) to predict the power profile in the narrow beam space with a convolutional neural network, based on which the cascaded multiple-BS beam space is carefully pruned. In the short-timescale, we introduce a centralized reinforcement learning (RL) algorithm to maximize the satisfaction rate of delay w.r.t. beam selection within multiple consecutive time slots. Moreover, we put forward three scalable distributed algorithms including hierarchical distributed Lyapunov optimization, fully distributed RL, and centralized training with decentralized execution of RL to achieve better scalability and better tradeoff between the performance and the execution signal overhead. Numerical results demonstrate that the proposed schemes significantly reduce both model training cost and beam training overhead and are easier to meet the user-specific delay requirement, compared to existing methods.
Are Large Language Models Really Robust to Word-Level Perturbations?
Venue: arXiv cs.CL 2023
Year: 2023

The swift advancement in the scales and capabilities of Large Language Models (LLMs) positions them as promising tools for a variety of downstream tasks. In addition to the pursuit of better performance and the avoidance of violent feedback on a certain prompt, to ensure the responsibility of the LLM, much attention is drawn to the robustness of LLMs. However, existing evaluation methods mostly rely on traditional question answering datasets with predefined supervised labels, which do not align with the superior generation capabilities of contemporary LLMs. To address this issue, we propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools to evaluate the longer conversation generated from more challenging open questions by LLMs, which we refer to as the Reward Model for Reasonable Robustness Evaluation (TREvaL). Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions, a capability not entirely encompassed by individual words or letters, which may exhibit oversimplification and inherent biases. Our extensive empirical experiments demonstrate that TREvaL provides an innovative method for evaluating the robustness of an LLM. Furthermore, our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage. Notably, we are surprised to discover that robustness tends to decrease as fine-tuning (SFT and RLHF) is conducted. The code of TREval is available in https://github.com/Harry-mic/TREvaL.
Automatic Bat Call Classification using Transformer Networks
Venue: arXiv cs.CV 2023
Year: 2023

Automatically identifying bat species from their echolocation calls is a difficult but important task for monitoring bats and the ecosystem they live in. Major challenges in automatic bat call identification are high call variability, similarities between species, interfering calls and lack of annotated data. Many currently available models suffer from relatively poor performance on real-life data due to being trained on single call datasets and, moreover, are often too slow for real-time classification. Here, we propose a Transformer architecture for multi-label classification with potential applications in real-time classification scenarios. We train our model on synthetically generated multi-species recordings by merging multiple bats calls into a single recording with multiple simultaneous calls. Our approach achieves a single species accuracy of 88.92% (F1-score of 84.23%) and a multi species macro F1-score of 74.40% on our test set. In comparison to three other tools on the independent and publicly available dataset ChiroVox, our model achieves at least 25.82% better accuracy for single species classification and at least 6.9% better macro F1-score for multi species classification.
Towards Robust Few-shot Point Cloud Semantic Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Few-shot point cloud semantic segmentation aims to train a model to quickly adapt to new unseen classes with only a handful of support set samples. However, the noise-free assumption in the support set can be easily violated in many practical real-world settings. In this paper, we focus on improving the robustness of few-shot point cloud segmentation under the detrimental influence of noisy support sets during testing time. To this end, we first propose a Component-level Clean Noise Separation (CCNS) representation learning to learn discriminative feature representations that separates the clean samples of the target classes from the noisy samples. Leveraging the well separated clean and noisy support samples from our CCNS, we further propose a Multi-scale Degree-based Noise Suppression (MDNS) scheme to remove the noisy shots from the support set. We conduct extensive experiments on various noise settings on two benchmark datasets. Our results show that the combination of CCNS and MDNS significantly improves the performance. Our code is available at https://github.com/Pixie8888/R3DFSSeg.
Data-Driven Analysis of Gender Fairness in the Software Engineering Academic Landscape
Venue: arXiv cs.CY 2023
Year: 2023

Gender bias in education gained considerable relevance in the literature over the years. However, while the problem of gender bias in education has been widely addressed from a student perspective, it is still not fully analysed from an academic point of view. In this work, we study the problem of gender bias in academic promotions (i.e., from Researcher to Associated Professor and from Associated to Full Professor) in the informatics (INF) and software engineering (SE) Italian communities. In particular, we first conduct a literature review to assess how the problem of gender bias in academia has been addressed so far. Next, we describe a process to collect and preprocess the INF and SE data needed to analyse gender bias in Italian academic promotions. Subsequently, we apply a formal bias metric to these data to assess the amount of bias and look at its variation over time. From the conducted analysis, we observe how the SE community presents a higher bias in promotions to Associate Professors and a smaller bias in promotions to Full Professors compared to the overall INF community.
Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Information
Venue: arXiv cs.CV 2023
Year: 2023

The state-of-the-art neural video codecs have outperformed the most sophisticated traditional codecs in terms of RD performance in certain cases. However, utilizing them for practical applications is still challenging for two major reasons. 1) Cross-platform computational errors resulting from floating point operations can lead to inaccurate decoding of the bitstream. 2) The high computational complexity of the encoding and decoding process poses a challenge in achieving real-time performance. In this paper, we propose a real-time cross-platform neural video codec, which is capable of efficiently decoding of 720P video bitstream from other encoding platforms on a consumer-grade GPU. First, to solve the problem of inconsistency of codec caused by the uncertainty of floating point calculations across platforms, we design a calibration transmitting system to guarantee the consistent quantization of entropy parameters between the encoding and decoding stages. The parameters that may have transboundary quantization between encoding and decoding are identified in the encoding stage, and their coordinates will be delivered by auxiliary transmitted bitstream. By doing so, these inconsistent parameters can be processed properly in the decoding stage. Furthermore, to reduce the bitrate of the auxiliary bitstream, we rectify the distribution of entropy parameters using a piecewise Gaussian constraint. Second, to match the computational limitations on the decoding side for real-time video codec, we design a lightweight model. A series of efficiency techniques enable our model to achieve 25 FPS decoding speed on NVIDIA RTX 2080 GPU. Experimental results demonstrate that our model can achieve real-time decoding of 720P videos while encoding on another platform. Furthermore, the real-time model brings up to a maximum of 24.2\% BD-rate improvement from the perspective of PSNR with the anchor H.265.
FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion
Venue: arXiv cs.CV 2023
Year: 2023

Speech-driven 3D facial animation synthesis has been a challenging task both in industry and research. Recent methods mostly focus on deterministic deep learning methods meaning that given a speech input, the output is always the same. However, in reality, the non-verbal facial cues that reside throughout the face are non-deterministic in nature. In addition, majority of the approaches focus on 3D vertex based datasets and methods that are compatible with existing facial animation pipelines with rigged characters is scarce. To eliminate these issues, we present FaceDiffuser, a non-deterministic deep learning model to generate speech-driven facial animations that is trained with both 3D vertex and blendshape based datasets. Our method is based on the diffusion technique and uses the pre-trained large speech representation model HuBERT to encode the audio input. To the best of our knowledge, we are the first to employ the diffusion method for the task of speech-driven 3D facial animation synthesis. We have run extensive objective and subjective analyses and show that our approach achieves better or comparable results in comparison to the state-of-the-art methods. We also introduce a new in-house dataset that is based on a blendshape based rigged character. We recommend watching the accompanying supplementary video. The code and the dataset will be publicly available.
Lazy Contracts: Alleviating High Gas Costs by Secure and Trustless Off-chain Execution of Smart Contracts
Venue: arXiv cs.CR 2023
Year: 2023

Smart contracts are programs that are executed on the blockchain and can hold, manage and transfer assets in the form of cryptocurrencies. The contract's execution is then performed on-chain and is subject to consensus, i.e. every node on the blockchain network has to run the function calls and keep track of their side-effects. In most programmable blockchains, such as Ethereum, the notion of gas is introduced to prevent DoS attacks by malicious parties who might try to slow down the network by performing heavy computations. A fixed cost to each atomic operation, and the initiator of a function call pays the total gas cost as a transaction fee. This helps prevent DoS attacks, but the resulting fees are extremely high. For example, in 2022, on Ethereum alone, there has been a total gas usage of 1.77 Million ETH ~ 4.3 Billion USD. This thesis proposes "lazy contracts" as a solution to alleviate these costs. Our solution moves most of the computation off-chain, ensuring that each function call incurs only a tiny amount of gas usage, while preserving enough data on-chain to guarantee an implicit consensus about the state of the contract variables and ownership of funds. A complete on-chain execution of the functions will only be triggered in case two parties to the contract are in disagreement about the current state, which in turn can only happen if at least one party is dishonest. In such cases, our protocol can identify the dishonest party and penalize them by having them pay for the entire gas usage. Hence, no rational party has an incentive to act dishonestly. Finally, we perform extensive experiments over 160,735 real-world Solidity contracts that were involved in 9,055,492 transactions in January 2022--January 2023 on Ethereum and show that our approach reduces the overall gas usage by 55.4%, which amounts to an astounding saving of 109.9 Million USD in gas fees.
How to turn your camera into a perfect pinhole model
Venue: arXiv cs.CV 2023
Year: 2023

Camera calibration is a first and fundamental step in various computer vision applications. Despite being an active field of research, Zhang's method remains widely used for camera calibration due to its implementation in popular toolboxes. However, this method initially assumes a pinhole model with oversimplified distortion models. In this work, we propose a novel approach that involves a pre-processing step to remove distortions from images by means of Gaussian processes. Our method does not need to assume any distortion model and can be applied to severely warped images, even in the case of multiple distortion sources, e.g., a fisheye image of a curved mirror reflection. The Gaussian processes capture all distortions and camera imperfections, resulting in virtual images as though taken by an ideal pinhole camera with square pixels. Furthermore, this ideal GP-camera only needs one image of a square grid calibration pattern. This model allows for a serious upgrade of many algorithms and applications that are designed in a pure projective geometry setting but with a performance that is very sensitive to nonlinear lens distortions. We demonstrate the effectiveness of our method by simplifying Zhang's calibration method, reducing the number of parameters and getting rid of the distortion parameters and iterative optimization. We validate by means of synthetic data and real world images. The contributions of this work include the construction of a virtual ideal pinhole camera using Gaussian processes, a simplified calibration method and lens distortion removal.
TRAVID: An End-to-End Video Translation Framework
Venue: arXiv cs.CL 2023
Year: 2023

In today's globalized world, effective communication with people from diverse linguistic backgrounds has become increasingly crucial. While traditional methods of language translation, such as written text or voice-only translations, can accomplish the task, they often fail to capture the complete context and nuanced information conveyed through nonverbal cues like facial expressions and lip movements. In this paper, we present an end-to-end video translation system that not only translates spoken language but also synchronizes the translated speech with the lip movements of the speaker. Our system focuses on translating educational lectures in various Indian languages, and it is designed to be effective even in low-resource system settings. By incorporating lip movements that align with the target language and matching them with the speaker's voice using voice cloning techniques, our application offers an enhanced experience for students and users. This additional feature creates a more immersive and realistic learning environment, ultimately making the learning process more effective and engaging.
Single-Exponential FPT Algorithms for Enumerating Secluded $\mathcal{F}$-Free Subgraphs and Deleting to Scattered Graph Classes
Venue: arXiv cs.DS 2023
Year: 2023

The celebrated notion of important separators bounds the number of small $(S,T)$-separators in a graph which are 'farthest from $S$' in a technical sense. In this paper, we introduce a generalization of this powerful algorithmic primitive that is phrased in terms of $k$-secluded vertex sets: sets with an open neighborhood of size at most $k$. In this terminology, the bound on important separators says that there are at most $4^k$ maximal $k$-secluded connected vertex sets $C$ containing $S$ but disjoint from $T$. We generalize this statement significantly: even when we demand that $G[C]$ avoids a finite set $\mathcal{F}$ of forbidden induced subgraphs, the number of such maximal subgraphs is $2^{O(k)}$ and they can be enumerated efficiently. This allows us to make significant improvements for two problems from the literature. Our first application concerns the 'Connected $k$-Secluded $\mathcal{F}$-free subgraph' problem, where $\mathcal{F}$ is a finite set of forbidden induced subgraphs. Given a graph in which each vertex has a positive integer weight, the problem asks to find a maximum-weight connected $k$-secluded vertex set $C \subseteq V(G)$ such that $G[C]$ does not contain an induced subgraph isomorphic to any $F \in \mathcal{F}$. The parameterization by $k$ is known to be solvable in triple-exponential time via the technique of recursive understanding, which we improve to single-exponential. Our second application concerns the deletion problem to scattered graph classes. Here, the task is to find a vertex set of size at most $k$ whose removal yields a graph whose each connected component belongs to one of the prescribed graph classes $Π_1, \ldots, Π_d$. We obtain a single-exponential algorithm whenever each class $Π_i$ is characterized by a finite number of forbidden induced subgraphs. This generalizes and improves upon earlier results in the literature.
Learning and DiSentangling Patient Static Information from Time-series Electronic HEalth Record (STEER)
Venue: arXiv cs.LG 2023
Year: 2023

Recent work in machine learning for healthcare has raised concerns about patient privacy and algorithmic fairness. For example, previous work has shown that patient self-reported race can be predicted from medical data that does not explicitly contain racial information. However, the extent of data identification is unknown, and we lack ways to develop models whose outcomes are minimally affected by such information. Here we systematically investigated the ability of time-series electronic health record data to predict patient static information. We found that not only the raw time-series data, but also learned representations from machine learning models, can be trained to predict a variety of static information with area under the receiver operating characteristic curve as high as 0.851 for biological sex, 0.869 for binarized age and 0.810 for self-reported race. Such high predictive performance can be extended to a wide range of comorbidity factors and exists even when the model was trained for different tasks, using different cohorts, using different model architectures and databases. Given the privacy and fairness concerns these findings pose, we develop a variational autoencoder-based approach that learns a structured latent space to disentangle patient-sensitive attributes from time-series data. Our work thoroughly investigates the ability of machine learning models to encode patient static information from time-series electronic health records and introduces a general approach to protect patient-sensitive attribute information for downstream tasks.
Long-Form End-to-End Speech Translation via Latent Alignment Segmentation
Venue: arXiv cs.CL 2023
Year: 2023

Current simultaneous speech translation models can process audio only up to a few seconds long. Contemporary datasets provide an oracle segmentation into sentences based on human-annotated transcripts and translations. However, the segmentation into sentences is not available in the real world. Current speech segmentation approaches either offer poor segmentation quality or have to trade latency for quality. In this paper, we propose a novel segmentation approach for a low-latency end-to-end speech translation. We leverage the existing speech translation encoder-decoder architecture with ST CTC and show that it can perform the segmentation task without supervision or additional parameters. To the best of our knowledge, our method is the first that allows an actual end-to-end simultaneous speech translation, as the same model is used for translation and segmentation at the same time. On a diverse set of language pairs and in- and out-of-domain data, we show that the proposed approach achieves state-of-the-art quality at no additional computational cost.
SkeleTR: Towrads Skeleton-based Action Recognition in the Wild
Venue: arXiv cs.CV 2023
Year: 2023

We present SkeleTR, a new framework for skeleton-based action recognition. In contrast to prior work, which focuses mainly on controlled environments, we target more general scenarios that typically involve a variable number of people and various forms of interaction between people. SkeleTR works with a two-stage paradigm. It first models the intra-person skeleton dynamics for each skeleton sequence with graph convolutions, and then uses stacked Transformer encoders to capture person interactions that are important for action recognition in general scenarios. To mitigate the negative impact of inaccurate skeleton associations, SkeleTR takes relative short skeleton sequences as input and increases the number of sequences. As a unified solution, SkeleTR can be directly applied to multiple skeleton-based action tasks, including video-level action classification, instance-level action detection, and group-level activity recognition. It also enables transfer learning and joint training across different action tasks and datasets, which result in performance improvement. When evaluated on various skeleton-based action recognition benchmarks, SkeleTR achieves the state-of-the-art performance.
Digital twins of nonlinear dynamical systems: A perspective
Venue: arXiv cs.LG 2023
Year: 2023

Digital twins have attracted a great deal of recent attention from a wide range of fields. A basic requirement for digital twins of nonlinear dynamical systems is the ability to generate the system evolution and predict potentially catastrophic emergent behaviors so as to providing early warnings. The digital twin can then be used for system "health" monitoring in real time and for predictive problem solving. In particular, if the digital twin forecasts a possible system collapse in the future due to parameter drifting as caused by environmental changes or perturbations, an optimal control strategy can be devised and executed as early intervention to prevent the collapse. Two approaches exist for constructing digital twins of nonlinear dynamical systems: sparse optimization and machine learning. The basics of these two approaches are described and their advantages and caveats are discussed.