OpenRaiser/Intern-Atlas huggingface dataset:
Metaverse Native Communication: A Blockchain and Spectrum Prospective
Venue: arXiv cs.DC 2022
Year: 2022

Metaverse depicts a vista of constructing a virtual environment parallel to the real world so people can communicate with others and objects through digital entities. In the real world, communication relies on identities and addresses that are recognized by authorities, no matter the link is established via post, email, mobile phone, or landline. Metaverse, however, is different from the real world, which requires a single identity belongs to the individual. This identity can be an encrypted virtual address in the metaverse but no one can trace or verify it. In order to achieve such addresses to hide individuals in the metaverse, re-mapping the virtual address to the individual's identity and a specific spectrum to support the address-based communication for the metaverse are needed. Therefore, metaverse native or meta-native communications based on blockchain could be a promising solution to directly connect entities with their native encrypted addresses that gets rid of the existing network services based on IP, cellular, HTTP, etc. This paper proposes a vision of blockchain, encrypted address and address-based access model for all users, devices, services, etc. to contribute to the metaverse. Furthermore, the allocation architecture of a designated spectrum for the metaverse is proposed to remove the barrier to access to the metaverse/blockchain in response to the initiatives of metaverse and decentralized Internet.
GOMP-ST: Grasp Optimized Motion Planning for Suction Transport
Venue: arXiv cs.RO 2022
Year: 2022

Suction cup grasping is very common in industry, but moving too quickly can cause suction cups to detach, causing drops or damage. Maintaining a suction grasp throughout a high-speed motion requires balancing suction forces against inertial forces while the suction cups deform under strain. In this paper, we consider Grasp Optimized Motion Planning for Suction Transport (GOMP-ST), an algorithm that combines deep learning with optimization to decrease transport time while avoiding suction cup failure. GOMP-ST first repeatedly moves a physical robot, vacuum gripper, and a sample object, while measuring pressure with a solid-state sensor to learn critical failure conditions. Then, these are integrated as constraints on the accelerations at the end-effector into a time-optimizing motion planner. The resulting plans incorporate real-world effects such as suction cup deformation that are difficult to model analytically. In GOMP-ST, the learned constraint, modeled with a neural network, is linearized using Autograd and integrated into a sequential quadratic program optimization. In 420 experiments with a physical UR5 transporting objects ranging from 1.3 to 1.7 kg, we compare GOMP-ST to baseline optimizing motion planners. Results suggest that GOMP-ST can avoid suction cup failure while decreasing transport times from 16% to 58%. For code, video, and datasets, see https://sites.google.com/view/gomp-st.
Concurrent CPU-GPU Task Programming using Modern C++
Venue: arXiv cs.DC 2022
Year: 2022

In this paper, we introduce Heteroflow, a new C++ library to help developers quickly write parallel CPU-GPU programs using task dependency graphs. Heteroflow leverages the power of modern C++ and task-based approaches to enable efficient implementations of heterogeneous decomposition strategies. Our new CPU-GPU programming model allows users to express a problem in a way that adapts to effective separation of concerns and expertise encapsulation. Compared with existing libraries, Heteroflow is more cost-efficient in performance scaling, programming productivity, and solution generality. We have evaluated Heteroflow on two real applications in VLSI design automation and demonstrated the performance scalability across different CPU-GPU numbers and problem sizes. At a particular example of VLSI timing analysis with million-scale tasking, Heteroflow achieved 7.7x runtime speed-up (99 vs 13 minutes) over a baseline on a machine of 40 CPU cores and 4 GPUs.
On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models
Venue: arXiv cs.SE 2022
Year: 2022

Many studies have developed Machine Learning (ML) approaches to detect Software Vulnerabilities (SVs) in functions and fine-grained code statements that cause such SVs. However, there is little work on leveraging such detection outputs for data-driven SV assessment to give information about exploitability, impact, and severity of SVs. The information is important to understand SVs and prioritize their fixing. Using large-scale data from 1,782 functions of 429 SVs in 200 real-world projects, we investigate ML models for automating function-level SV assessment tasks, i.e., predicting seven Common Vulnerability Scoring System (CVSS) metrics. We particularly study the value and use of vulnerable statements as inputs for developing the assessment models because SVs in functions are originated in these statements. We show that vulnerable statements are 5.8 times smaller in size, yet exhibit 7.5-114.5% stronger assessment performance (Matthews Correlation Coefficient (MCC)) than non-vulnerable statements. Incorporating context of vulnerable statements further increases the performance by up to 8.9% (0.64 MCC and 0.75 F1-Score). Overall, we provide the initial yet promising ML-based baselines for function-level SV assessment, paving the way for further research in this direction.
How do you Converse with an Analytical Chatbot? Revisiting Gricean Maxims for Designing Analytical Conversational Behavior
Venue: arXiv cs.HC 2022
Year: 2022

Chatbots have garnered interest as conversational interfaces for a variety of tasks. While general design guidelines exist for chatbot interfaces, little work explores analytical chatbots that support conversing with data. We explore Gricean Maxims to help inform the basic design of effective conversational interaction. We also draw inspiration from natural language interfaces for data exploration to support ambiguity and intent handling. We ran Wizard of Oz studies with 30 participants to evaluate user expectations for text and voice chatbot design variants. Results identified preferences for intent interpretation and revealed variations in user expectations based on the interface affordances. We subsequently conducted an exploratory analysis of three analytical chatbot systems (text + chart, voice + chart, voice-only) that implement these preferred design variants. Empirical evidence from a second 30-participant study informs implications specific to data-driven conversation such as interpreting intent, data orientation, and establishing trust through appropriate system responses.
PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression
Venue: arXiv cs.CV 2022
Year: 2022

We push forward neural network compression research by exploiting a novel challenging task of large-scale conditional generative adversarial networks (GANs) compression. To this end, we propose a gradually shrinking GAN (PPCD-GAN) by introducing progressive pruning residual block (PP-Res) and class-aware distillation. The PP-Res is an extension of the conventional residual block where each convolutional layer is followed by a learnable mask layer to progressively prune network parameters as training proceeds. The class-aware distillation, on the other hand, enhances the stability of training by transferring immense knowledge from a well-trained teacher model through instructive attention maps. We train the pruning and distillation processes simultaneously on a well-known GAN architecture in an end-to-end manner. After training, all redundant parameters as well as the mask layers are discarded, yielding a lighter network while retaining the performance. We comprehensively illustrate, on ImageNet 128x128 dataset, PPCD-GAN reduces up to 5.2x (81%) parameters against state-of-the-arts while keeping better performance.
KinyaBERT: a Morphology-aware Kinyarwanda Language Model
Venue: arXiv cs.CL 2022
Year: 2022

Pre-trained language models such as BERT have been successful at tackling many natural language processing tasks. However, the unsupervised sub-word tokenization methods commonly used in these models (e.g., byte-pair encoding - BPE) are sub-optimal at handling morphologically rich languages. Even given a morphological analyzer, naive sequencing of morphemes into a standard BERT architecture is inefficient at capturing morphological compositionality and expressing word-relative syntactic regularities. We address these challenges by proposing a simple yet effective two-tier BERT architecture that leverages a morphological analyzer and explicitly represents morphological compositionality. Despite the success of BERT, most of its evaluations have been conducted on high-resource languages, obscuring its applicability on low-resource languages. We evaluate our proposed method on the low-resource morphologically rich Kinyarwanda language, naming the proposed model architecture KinyaBERT. A robust set of experimental results reveal that KinyaBERT outperforms solid baselines by 2% in F1 score on a named entity recognition task and by 4.3% in average score of a machine-translated GLUE benchmark. KinyaBERT fine-tuning has better convergence and achieves more robust results on multiple tasks even in the presence of translation noise.
Building AI Innovation Labs together with Companies
Venue: arXiv cs.OH 2022
Year: 2022

In the future, most companies will be confronted with the topic of Artificial Intelligence (AI) and will have to decide on their strategy in this regards. Currently, a lot of companies are thinking about whether and how AI and the usage of data will impact their business model and what potential use cases could look like. One of the biggest challenges lies in coming up with innovative solution ideas with a clear business value. This requires business competencies on the one hand and technical competencies in AI and data analytics on the other hand. In this article, we present the concept of AI innovation labs and demonstrate a comprehensive framework, from coming up with the right ideas to incrementally implementing and evaluating them regarding their business value and their feasibility based on a company's capabilities. The concept is the result of nine years of working on data-driven innovations with companies from various domains. Furthermore, we share some lessons learned from its practical applications. Even though a lot of technical publications can be found in the literature regarding the development of AI models and many consultancy companies provide corresponding services for building AI innovations, we found very few publications sharing details about what an end-to-end framework could look like.
Upper Bounds on the Mismatched Reliability Function and Capacity Using a Genie Receiver
Venue: arXiv cs.IT 2022
Year: 2022

We develop a novel framework for proving converse theorems for channel coding, which is based on the analysis technique of multicast transmission with an additional auxiliary receiver, which serves as a genie to the original receiver. The genie provides the original receiver a certain narrowed list of codewords to choose from that includes the transmitted one. This technique is used to derive upper bounds on the mismatch capacity of discrete memoryless channels as well as the reliability function with a mismatched decoding metric. Unlike previous works, our bounding technique exploits also the inherent symmetric requirement from the codewords, leading to these new upper bounds. Since the computations of most of the known bounds on the mismatch capacity are rather complicated, we further present a method to obtain relaxed bounds that are easier to compute. As an example, we analyze the obtained bounds in the binary-input channels case. We conclude by presenting simpler bounds on the reliability function, and provide sufficient conditions for their tightness in certain ranges of rates.
Whither the Priors for (Vocal) Interactivity?
Venue: arXiv cs.RO 2022
Year: 2022

Voice-based communication is often cited as one of the most `natural' ways in which humans and robots might interact, and the recent availability of accurate automatic speech recognition and intelligible speech synthesis has enabled researchers to integrate advanced off-the-shelf spoken language technology components into their robot platforms. Despite this, the resulting interactions are anything but `natural'. It transpires that simply giving a robot a voice doesn't mean that a user will know how (or when) to talk to it, and the resulting `conversations' tend to be stilted, one-sided and short. On the surface, these difficulties might appear to be fairly trivial consequences of users' unfamiliarity with robots (and \emph{vice versa}), and that any problems would be mitigated by long-term use by the human, coupled with `deep learning' by the robot. However, it is argued here that such communication failures are indicative of a deeper malaise: a fundamental lack of basic principles -- \emph{priors} -- underpinning not only speech-based interaction in particular, but (vocal) interactivity in general. This is evidenced not only by the fact that contemporary spoken language systems already require training data sets that are orders-of-magnitude greater than that experienced by a young child, but also by the lack of design principles for creating effective communicative human-robot interaction. This short position paper identifies some of the key areas where theoretical insights might help overcome these shortfalls.
The Structured Abstain Problem and the Lovász Hinge
Venue: arXiv cs.LG 2022
Year: 2022

The Lovász hinge is a convex surrogate recently proposed for structured binary classification, in which $k$ binary predictions are made simultaneously and the error is judged by a submodular set function. Despite its wide usage in image segmentation and related problems, its consistency has remained open. We resolve this open question, showing that the Lovász hinge is inconsistent for its desired target unless the set function is modular. Leveraging a recent embedding framework, we instead derive the target loss for which the Lovász hinge is consistent. This target, which we call the structured abstain problem, allows one to abstain on any subset of the $k$ predictions. We derive two link functions, each of which are consistent for all submodular set functions simultaneously.
Learning video retrieval models with relevance-aware online mining
Venue: arXiv cs.CV 2022
Year: 2022

Due to the amount of videos and related captions uploaded every hour, deep learning-based solutions for cross-modal video retrieval are attracting more and more attention. A typical approach consists in learning a joint text-video embedding space, where the similarity of a video and its associated caption is maximized, whereas a lower similarity is enforced with all the other captions, called negatives. This approach assumes that only the video and caption pairs in the dataset are valid, but different captions - positives - may also describe its visual contents, hence some of them may be wrongly penalized. To address this shortcoming, we propose the Relevance-Aware Negatives and Positives mining (RANP) which, based on the semantics of the negatives, improves their selection while also increasing the similarity of other valid positives. We explore the influence of these techniques on two video-text datasets: EPIC-Kitchens-100 and MSR-VTT. By using the proposed techniques, we achieve considerable improvements in terms of nDCG and mAP, leading to state-of-the-art results, e.g. +5.3% nDCG and +3.0% mAP on EPIC-Kitchens-100. We share code and pretrained models at \url{https://github.com/aranciokov/ranp}.
Tangles and Hierarchical Clustering
Venue: arXiv cs.DM 2022
Year: 2022

We establish a connection between tangles, a concept from structural graph theory that plays a central role in Robertson and Seymour's graph minor project, and hierarchical clustering. Tangles cannot only be defined for graphs, but in fact for arbitrary connectivity functions, which are functions defined on the subsets of some finite universe. In typical clustering applications these universes consist of points in some metric space. Connectivity functions are usually required to be submodular. It is our first contribution to show that the central duality theorem connecting tangles with hierarchical decompositions (so-called branch decompositions) also holds if submodularity is replaced by a different property that we call maximum-submodular. We then define a connectivity function on finite data sets in an arbitrary metric space and prove that its tangles are in one-to-one correspondence with the clusters obtained by applying the well-known single linkage clustering algorithms to the same data set. Lastly we generalize this correspondence for any hierarchical clustering. We show that the data structure that represents hierarchical clustering results, called dendograms, are equivalent to maximum-submodular connectivity functions and their tangles. The idea of viewing tangles as clusters has first been proposed by Diestel and Whittle in 2016 as an approach to image segmentation. To the best of our knowledge, our result is the first that establishes a precise technical connection between tangles and clusters.
Behaviour in social media for floods and heat waves in disaster response via Artificial Intelligence
Venue: arXiv cs.SI 2022
Year: 2022

This paper analyses social media data in multiple disaster-related collections of floods and heat waves in the UK. The proposed method uses machine learning classifiers based on deep bidirectional neural networks trained on benchmark datasets of disaster responses and extreme events. The resulting models are applied to perform sentiment and qualitative analysis of inferred topics in text data. We further analyse a set of behavioural indicators and match them with climate variables via decoding synoptical records to analyse thermal comfort. We highlight the advantages of aligning behavioural indicators along with climate variables to provide with additional valuable information to be considered especially in different phases of a disaster and applicable to extreme weather periods. The positiveness of messages is around 8% for disaster, 1% for disaster and medical response, 7% for disaster and humanitarian related messages. This shows the reliability of such data for our case studies. We show the transferability of this approach to be applied to any social media data collection.
Adaptive Objective Configuration in Bi-Objective Evolutionary Optimization for Cervical Cancer Brachytherapy Treatment Planning
Venue: arXiv cs.NE 2022
Year: 2022

The Multi-Objective Real-Valued Gene-pool Optimal Mixing Evolutionary Algorithm (MO-RV-GOMEA) has been proven effective and efficient in solving real-world problems. A prime example is optimizing treatment plans for prostate cancer brachytherapy, an internal form of radiation treatment, for which equally important clinical aims from a base protocol are grouped into two objectives and bi-objectively optimized. This use of MO-RV-GOMEA was recently successfully introduced into clinical practice. Brachytherapy can also play an important role in treating cervical cancer. However, using the same approach to optimize treatment plans often does not immediately lead to clinically desirable results. Concordantly, medical experts indicate that they use additional aims beyond the cervix base protocol. Moreover, these aims have different priorities and can be patient-specifically adjusted. For this reason, we propose a novel adaptive objective configuration method to use with MO-RV-GOMEA so that we can accommodate additional aims of this nature. Based on results using only the base protocol, in consultation with medical experts, we configured key additional aims. We show how, for 10 patient cases, the new approach achieves the intended result, properly taking into account the additional aims. Consequently, plans resulting from the new approach are preferred by medical specialists in 8/10 cases.
Finite-Length Scaling of SC-LDPC Codes With a Limited Number of Decoding Iterations
Venue: arXiv cs.IT 2022
Year: 2022

We propose four finite-length scaling laws to predict the frame error rate (FER) performance of spatially-coupled low-density parity-check codes under full belief propagation (BP) decoding with a limit on the number of decoding iterations and a scaling law for sliding window decoding, also with limited iterations. The laws for full BP decoding provide a choice between accuracy and computational complexity; a good balance between them is achieved by the law that models the number of decoded bits after a certain number of BP iterations by a time-integrated Ornstein-Uhlenbeck process. This framework is developed further to model sliding window decoding as a race between the integrated Ornstein-Uhlenbeck process and an absorbing barrier that corresponds to the left boundary of the sliding window. The proposed scaling laws yield accurate FER predictions.
Social-Cultural Factors in the Design of Technology for Hispanic People with Stroke
Venue: arXiv cs.RO 2022
Year: 2022

Stroke is a leading cause of serious, long-term disability in the United States. There exist disparities in both stroke prevalence and outcomes between people with stroke in Hispanic and Latinx communities and the general stroke population. Current stroke technology - which aims to improve quality of life and bring people with stroke to the most functional, independent state possible - has shown promising results for the general stroke population, but has failed to close the recovery outcome gap for underserved Hispanic and Latinx people with stroke. Previous work in health education, digital health, and HRI has improved human health outcomes by incorporating social-cultural factors, though not for stroke. In this position paper, we aim to justify accounting for unique cultural factors in stroke technology design for the Hispanic and Latinx community. We review examples of successful culturally appropriate interventions and suggest design considerations (mutually beneficial community consultation, accommodating for barriers beforehand, building on culture, and incorporating education of the family) to provide more culturally appropriate design of Hispanic and Latinx stroke technology and reduce the disparity gap.
Creating Multimedia Summaries Using Tweets and Videos
Venue: arXiv cs.CL 2022
Year: 2022

While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time. In this paper, we propose a simple yet effective approach to combine social media commentary and videos to create a multimedia summary of televised events. Our approach identifies scenes from these events based on spikes of mentions of people involved in the event and automatically selects tweets and frames from the videos that occur during the time period of the spike that talk about and show the people being discussed.