OpenRaiser/Intern-Atlas huggingface dataset:
Robust Online Video Instance Segmentation with Track Queries
Venue: arXiv cs.CV 2022
Year: 2022

Recently, transformer-based methods have achieved impressive results on Video Instance Segmentation (VIS). However, most of these top-performing methods run in an offline manner by processing the entire video clip at once to predict instance mask volumes. This makes them incapable of handling the long videos that appear in challenging new video instance segmentation datasets like UVO and OVIS. We propose a fully online transformer-based video instance segmentation model that performs comparably to top offline methods on the YouTube-VIS 2019 benchmark and considerably outperforms them on UVO and OVIS. This method, called Robust Online Video Segmentation (ROVIS), augments the Mask2Former image instance segmentation model with track queries, a lightweight mechanism for carrying track information from frame to frame, originally introduced by the TrackFormer method for multi-object tracking. We show that, when combined with a strong enough image segmentation architecture, track queries can exhibit impressive accuracy while not being constrained to short videos.
Improving Effectiveness of Seamless Redundancy in Real Industrial Wi-Fi Networks
Venue: arXiv cs.NI 2022
Year: 2022

Reliability and determinism of Wi-Fi can be tangibly improved by means of seamless redundancy, to the point of making this technology suitable for industrial environments. As pointed out in recent papers, the most benefits can be achieved when no phenomena can simultaneously affect transmissions on all channels of a redundant link. In this paper, several aspects are analyzed which, if not properly counteracted, may worsen seamless redundancy effectiveness. Effects they cause on communication have been experimentally evaluated in real testbeds, which rely on commercial Wi-Fi devices. Then, practical guidelines are provided, which aim at preventing joint interference through a careful system design. Results show that measured communication quality can be made as good as expected in theory.
Engineering Monosemanticity in Toy Models
Venue: arXiv cs.LG 2022
Year: 2022

In some neural networks, individual neurons correspond to natural ``features'' in the input. Such \emph{monosemantic} neurons are of great help in interpretability studies, as they can be cleanly understood. In this work we report preliminary attempts to engineer monosemanticity in toy models. We find that models can be made more monosemantic without increasing the loss by just changing which local minimum the training process finds. More monosemantic loss minima have moderate negative biases, and we are able to use this fact to engineer highly monosemantic models. We are able to mechanistically interpret these models, including the residual polysemantic neurons, and uncover a simple yet surprising algorithm. Finally, we find that providing models with more neurons per layer makes the models more monosemantic, albeit at increased computational cost. These findings point to a number of new questions and avenues for engineering monosemanticity, which we intend to study these in future work.
Which tweets 'deserve' to be included in news stories? Chronemics of tweet embedding
Venue: arXiv cs.SI 2022
Year: 2022

The use and selection of user-generated social media content, specifically tweets, as a news source has become an integral part of news production practice. Yet, the mapping and the extent of the nature of the practices in which news outlets integrate social media use are still lacking. This study focuses on the pressures of immediacy on the media ecosystems, i.e., as organizational practices of news outlets that make choices related to social media content integration. By analyzing a large corpora of news outlets that have embedded tweets, this study analyzes tweet embedding practices by specifically focusing on the concept of chronemics, conceptualized here as the time needed to embed tweets. Temporal constraints are particularly pressing for journalistic practices, given the continuous pressures of the 24/7 news cycle. We ask two main questions: which types of outlets are quicker to embed tweets, and which types of users' tweets are more likely to be embedded quickly?
A Graph-Based Context-Aware Model to Understand Online Conversations
Venue: arXiv cs.CL 2022
Year: 2022

Online forums that allow for participatory engagement between users have been transformative for the public discussion of many important issues. However, such conversations can sometimes escalate into full-blown exchanges of hate and misinformation. Existing approaches in natural language processing (NLP), such as deep learning models for classification tasks, use as inputs only a single comment or a pair of comments depending upon whether the task concerns the inference of properties of the individual comments or the replies between pairs of comments, respectively. But in online conversations, comments and replies may be based on external context beyond the immediately relevant information that is input to the model. Therefore, being aware of the conversations' surrounding contexts should improve the model's performance for the inference task at hand. We propose GraphNLI, a novel graph-based deep learning architecture that uses graph walks to incorporate the wider context of a conversation in a principled manner. Specifically, a graph walk starts from a given comment and samples "nearby" comments in the same or parallel conversation threads, which results in additional embeddings that are aggregated together with the initial comment's embedding. We then use these enriched embeddings for downstream NLP prediction tasks that are important for online conversations. We evaluate GraphNLI on two such tasks - polarity prediction and misogynistic hate speech detection - and found that our model consistently outperforms all relevant baselines for both tasks. Specifically, GraphNLI with a biased root-seeking random walk performs with a macro-F1 score of 3 and 6 percentage points better than the best-performing BERT-based baselines for the polarity prediction and hate speech detection tasks, respectively.
edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms
Venue: arXiv cs.HC 2022
Year: 2022

We present edBB-Demo, a demonstrator of an AI-powered research platform for student monitoring in remote education. The edBB platform aims to study the challenges associated to user recognition and behavior understanding in digital platforms. This platform has been developed for data collection, acquiring signals from a variety of sensors including keyboard, mouse, webcam, microphone, smartwatch, and an Electroencephalography band. The information captured from the sensors during the student sessions is modelled in a multimodal learning framework. The demonstrator includes: i) Biometric user authentication in an unsupervised environment; ii) Human action recognition based on remote video analysis; iii) Heart rate estimation from webcam video; and iv) Attention level estimation from facial expression analysis.
Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses
Venue: arXiv cs.CL 2022
Year: 2022

Existing approaches for disfluency detection typically require the existence of large annotated datasets. However, current datasets for this task are limited, suffer from class imbalance, and lack some types of disfluencies that can be encountered in real-world scenarios. This work proposes LARD, a method for automatically generating artificial disfluencies from fluent text. LARD can simulate all the different types of disfluencies (repetitions, replacements and restarts) based on the reparandum/interregnum annotation scheme. In addition, it incorporates contextual embeddings into the disfluency generation to produce realistic context-aware artificial disfluencies. Since the proposed method requires only fluent text, it can be used directly for training, bypassing the requirement of annotated disfluent data. Our empirical evaluation demonstrates that LARD can indeed be effectively used when no or only a few data are available. Furthermore, our detailed analysis suggests that the proposed method generates realistic disfluencies and increases the accuracy of existing disfluency detectors.
Optimizing Function Layout for Mobile Applications
Venue: arXiv cs.PL 2022
Year: 2022

Function layout, also referred to as function reordering or function placement, is one of the most effective profile-guided compiler optimizations. By reordering functions in a binary, compilers are able to greatly improve the performance of large-scale applications or reduce the compressed size of mobile applications. Although the technique has been studied in the context of large-scale binaries, no recent study has investigated the impact of function layout on mobile applications. In this paper we develop the first principled solution for optimizing function layouts in the mobile space. To this end, we identify two important optimization goals, the compressed code size and the cold start-up time of a mobile application. Then we propose a formal model for the layout problem, whose objective closely matches the goals. Our novel algorithm to optimize the layout is inspired by the classic balanced graph partitioning problem. We carefully engineer and implement the algorithm in an open source compiler, LLVM. An extensive evaluation of the new method on large commercial mobile applications indicates up to 2% compressed size reduction and up to 3% start-up time improvement on top of the state-of-the-art approach.
Asymptotic behaviour of a conservative reaction-diffusion system associated with a Markov process algebra model
Venue: arXiv cs.PF 2022
Year: 2022

This paper demonstrates a lower and upper solution method to investigate the asymptotic behaviour of the conservative reaction-diffusion systems associated with Markovian process algebra models. In particular, we have proved the uniform convergence of the solution to its constant equilibrium for a case study as time tends to infinity, together with experimental results illustrations.
Estimation of physical activities of people in offices from time-series point-cloud data
Venue: arXiv cs.HC 2022
Year: 2022

This paper proposes an edge computing system that enables estimating physical activities of people in offices from time-series point-cloud data, obtained by using a light-detection-and-ranging (LIDAR) sensor network. The paper presents that the proposed system successfully constructs the model for estimating the number of typed characters from time-series point-cloud data, through an experiment using real LIDAR sensors.
Continuous Prediction of Web User Visual Attention on short span Windows based on Gaze Data Analytics
Venue: arXiv cs.HC 2022
Year: 2022

The existing approaches to identify personalized salience zones of a Web page do not consider the dynamic behavior in time of the Web user's gaze or the alterations of its content. For this reason, this paper proposes the concept of visit intention, an indicator of the visual attention of a Web user in a certain period of time, short span time windows, in different areas of interest. This indicator gives information on the areas of interest of a website that will be visited by a user over a time window, without requiring to know the structure of the site in each window. Our approach leverages the population-level general gaze patterns and the user's visual kinetics. We show experimentally that it is possible to conduct such a prediction through multilabel classification models using a small number of users, obtaining an average area under curve of 84.3 %, an average accuracy of 79 %, and an individual area of interest accuracy of 77 %. Furthermore, the user's visual kinetics features are consistently selected in every set of a cross-validation evaluation.
Generalizable Deepfake Detection with Phase-Based Motion Analysis
Venue: arXiv cs.CV 2022
Year: 2022

We propose PhaseForensics, a DeepFake (DF) video detection method that leverages a phase-based motion representation of facial temporal dynamics. Existing methods relying on temporal inconsistencies for DF detection present many advantages over the typical frame-based methods. However, they still show limited cross-dataset generalization and robustness to common distortions. These shortcomings are partially due to error-prone motion estimation and landmark tracking, or the susceptibility of the pixel intensity-based features to spatial distortions and the cross-dataset domain shifts. Our key insight to overcome these issues is to leverage the temporal phase variations in the band-pass components of the Complex Steerable Pyramid on face sub-regions. This not only enables a robust estimate of the temporal dynamics in these regions, but is also less prone to cross-dataset variations. Furthermore, the band-pass filters used to compute the local per-frame phase form an effective defense against the perturbations commonly seen in gradient-based adversarial attacks. Overall, with PhaseForensics, we show improved distortion and adversarial robustness, and state-of-the-art cross-dataset generalization, with 91.2% video-level AUC on the challenging CelebDFv2 (a recent state-of-the-art compares at 86.9%).
Self-Training with Purpose Preserving Augmentation Improves Few-shot Generative Dialogue State Tracking
Venue: arXiv cs.CL 2022
Year: 2022

In dialogue state tracking (DST), labeling the dataset involves considerable human labor. We propose a new self-training framework for few-shot generative DST that utilize unlabeled data. Our self-training method iteratively improves the model by pseudo labeling and employs Purpose Preserving Augmentation (PPAug) to prevent overfitting. We increaese the few-shot 10% performance by approximately 4% on MultiWOZ 2.1 and enhances the slot-recall 8.34% for unseen values compared to baseline.
Planning Irregular Object Packing via Hierarchical Reinforcement Learning
Venue: arXiv cs.RO 2022
Year: 2022

Object packing by autonomous robots is an im-portant challenge in warehouses and logistics industry. Most conventional data-driven packing planning approaches focus on regular cuboid packing, which are usually heuristic and limit the practical use in realistic applications with everyday objects. In this paper, we propose a deep hierarchical reinforcement learning approach to simultaneously plan packing sequence and placement for irregular object packing. Specifically, the top manager network infers packing sequence from six principal view heightmaps of all objects, and then the bottom worker network receives heightmaps of the next object to predict the placement position and orientation. The two networks are trained hierarchically in a self-supervised Q-Learning framework, where the rewards are provided by the packing results based on the top height , object volume and placement stability in the box. The framework repeats sequence and placement planning iteratively until all objects have been packed into the box or no space is remained for unpacked items. We compare our approach with existing robotic packing methods for irregular objects in a physics simulator. Experiments show that our approach can pack more objects with less time cost than the state-of-the-art packing methods of irregular objects. We also implement our packing plan with a robotic manipulator to show the generalization ability in the real world.
Feature-augmented Machine Reading Comprehension with Auxiliary Tasks
Venue: arXiv cs.CL 2022
Year: 2022

While most successful approaches for machine reading comprehension rely on single training objective, it is assumed that the encoder layer can learn great representation through the loss function we define in the predict layer, which is cross entropy in most of time, in the case that we first use neural networks to encode the question and paragraph, then directly fuse the encoding result of them. However, due to the distantly loss backpropagating in reading comprehension, the encoder layer cannot learn effectively and be directly supervised. Thus, the encoder layer can not learn the representation well at any time. Base on this, we propose to inject multi granularity information to the encoding layer. Experiments demonstrate the effect of adding multi granularity information to the encoding layer can boost the performance of machine reading comprehension system. Finally, empirical study shows that our approach can be applied to many existing MRC models.
Unique-Neighbor-Like Expansion and Group-Independent Cosystolic Expansion
Venue: arXiv cs.CC 2022
Year: 2022

In recent years, high dimensional expanders have been found to have a variety of applications in theoretical computer science, such as efficient CSPs approximations, improved sampling and list-decoding algorithms, and more. Within that, an important high dimensional expansion notion is \emph{cosystolic expansion}, which has found applications in the construction of efficiently decodable quantum codes and in proving lower bounds for CSPs. Cosystolic expansion is considered with systems of equations over a group where the variables and equations correspond to faces of the complex. Previous works that studied cosystolic expansion were tailored to the specific group $\mathbb{F}_2$. In particular, Kaufman, Kazhdan and Lubotzky (FOCS 2014), and Evra and Kaufman (STOC 2016) in their breakthrough works, who solved a famous open question of Gromov, have studied a notion which we term ``parity'' expansion for small sets. They showed that small sets of $k$-faces have proportionally many $(k+1)$-faces that contain \emph{an odd number} of $k$-faces from the set. Parity expansion for small sets could be used to imply cosystolic expansion only over $\mathbb{F}_2$. In this work we introduce a stronger \emph{unique-neighbor-like} expansion for small sets. We show that small sets of $k$-faces have proportionally many $(k+1)$-faces that contain \emph{exactly one} $k$-face from the set. This notion is fundamentally stronger than parity expansion and cannot be implied by previous works. We then show, utilizing the new unique-neighbor-like expansion notion introduced in this work, that cosystolic expansion can be made \emph{group-independent}, i.e., unique-neighbor-like expansion for small sets implies cosystolic expansion \emph{over any group}.
Double Balanced Sets in High Dimensional Expanders
Venue: arXiv cs.CC 2022
Year: 2022

Recent works have shown that expansion of pseudorandom sets is of great importance. However, all current works on pseudorandom sets are limited only to product (or approximate product) spaces, where Fourier Analysis methods could be applied. In this work we ask the natural question whether pseudorandom sets are relevant in domains where Fourier Analysis methods cannot be applied, e.g., one-sided local spectral expanders. We take the first step in the path of answering this question. We put forward a new definition for pseudorandom sets, which we call ``double balanced sets''. We demonstrate the strength of our new definition by showing that small double balanced sets in one-sided local spectral expanders have very strong expansion properties, such as unique-neighbor-like expansion. We further show that cohomologies in cosystolic expanders are double balanced, and use the newly derived strong expansion properties of double balanced sets in order to obtain an exponential improvement over the current state of the art lower bound on their minimal distance.
Systematic Literature Review of Gender and Software Engineering in Asia
Venue: arXiv cs.SE 2022
Year: 2022

It is essential to discuss the role, difficulties, and opportunities concerning people of different gender in the field of software engineering research, education, and industry. Although some literature reviews address software engineering and gender, it is still unclear how research and practices in Asia exist for handling gender aspects in software development and engineering. We conducted a systematic literature review to grasp the comprehensive view of gender research and practices in Asia. We analyzed the 32 identified papers concerning countries and publication years among 463 publications. Researchers and practitioners from various organizations actively work on gender research and practices in some countries, including China, India, and Turkey. We identified topics and classified them into seven categories varying from personal mental health and team building to organization. Future research directions include investigating the synergy between (regional) gender aspects and cultural concerns and considering possible contributions and dependency among different topics to have a solid foundation for accelerating further research and getting actionable practices.
Convolutional neural networks for medical image segmentation
Venue: arXiv cs.CV 2022
Year: 2022

In this article, we look into some essential aspects of convolutional neural networks (CNNs) with the focus on medical image segmentation. First, we discuss the CNN architecture, thereby highlighting the spatial origin of the data, voxel-wise classification and the receptive field. Second, we discuss the sampling of input-output pairs, thereby highlighting the interaction between voxel-wise classification, patch size and the receptive field. Finally, we give a historical overview of crucial changes to CNN architectures for classification and segmentation, giving insights in the relation between three pivotal CNN architectures: FCN, U-Net and DeepMedic.
Dynamic Pricing with Volume Discounts in Online Settings
Venue: arXiv cs.LG 2022
Year: 2022

According to the main international reports, more pervasive industrial and business-process automation, thanks to machine learning and advanced analytic tools, will unlock more than 14 trillion USD worldwide annually by 2030. In the specific case of pricing problems-which constitute the class of problems we investigate in this paper-, the estimated unlocked value will be about 0.5 trillion USD per year. In particular, this paper focuses on pricing in e-commerce when the objective function is profit maximization and only transaction data are available. This setting is one of the most common in real-world applications. Our work aims to find a pricing strategy that allows defining optimal prices at different volume thresholds to serve different classes of users. Furthermore, we face the major challenge, common in real-world settings, of dealing with limited data available. We design a two-phase online learning algorithm, namely PVD-B, capable of exploiting the data incrementally in an online fashion. The algorithm first estimates the demand curve and retrieves the optimal average price, and subsequently it offers discounts to differentiate the prices for each volume threshold. We ran a real-world 4-month-long A/B testing experiment in collaboration with an Italian e-commerce company, in which our algorithm PVD-B-corresponding to A configuration-has been compared with human pricing specialists-corresponding to B configuration. At the end of the experiment, our algorithm produced a total turnover of about 300 KEuros, outperforming the B configuration performance by about 55%. The Italian company we collaborated with decided to adopt our algorithm for more than 1,200 products since January 2022.
Analyse der Entwicklungstreiber militärischer Schwarmdrohnen durch Natural Language Processing
Venue: arXiv cs.CL 2022
Year: 2022

Military drones are taking an increasingly prominent role in armed conflict, and the use of multiple drones in a swarm can be useful. Who the drivers of the research are and what sub-domains exist is analyzed and visually presented in this research using NLP techniques based on 946 studies. Most research is conducted in the Western world, led by the United States, the United Kingdom, and Germany. Through Tf-idf scoring, it is shown that countries have significant differences in the subdomains studied. Overall, 2019 and 2020 saw the most works published, with significant interest in military swarm drones as early as 2008. This study provides a first glimpse into research in this area and prompts further investigation.