OpenRaiser/Intern-Atlas huggingface dataset:
American == White in Multimodal Language-and-Image AI
Venue: arXiv cs.CY 2022
Year: 2022

Three state-of-the-art language-and-image AI models, CLIP, SLIP, and BLIP, are evaluated for evidence of a bias previously observed in social and experimental psychology: equating American identity with being White. Embedding association tests (EATs) using standardized images of self-identified Asian, Black, Latina/o, and White individuals from the Chicago Face Database (CFD) reveal that White individuals are more associated with collective in-group words than are Asian, Black, or Latina/o individuals. In assessments of three core aspects of American identity reported by social psychologists, single-category EATs reveal that images of White individuals are more associated with patriotism and with being born in America, but that, consistent with prior findings in psychology, White individuals are associated with being less likely to treat people of all races and backgrounds equally. Three downstream machine learning tasks demonstrate biases associating American with White. In a visual question answering task using BLIP, 97% of White individuals are identified as American, compared to only 3% of Asian individuals. When asked in what state the individual depicted lives in, the model responds China 53% of the time for Asian individuals, but always with an American state for White individuals. In an image captioning task, BLIP remarks upon the race of Asian individuals as much as 36% of the time, but never remarks upon race for White individuals. Finally, provided with an initialization image from the CFD and the text "an American person," a synthetic image generator (VQGAN) using the text-based guidance of CLIP lightens the skin tone of individuals of all races (by 35% for Black individuals, based on pixel brightness). The results indicate that biases equating American identity with being White are learned by language-and-image AI, and propagate to downstream applications of such models.
Few-shot incremental learning in the context of solar cell quality inspection
Venue: arXiv cs.CV 2022
Year: 2022

In industry, Deep Neural Networks have shown high defect detection rates surpassing other more traditional manual feature engineering based proposals. This has been achieved mainly through supervised training where a great amount of data is required in order to learn good classification models. However, such amount of data is sometimes hard to obtain in industrial scenarios, as few defective pieces are produced normally. In addition, certain kinds of defects are very rare and usually just appear from time to time, which makes the generation of a proper dataset for training a classification model even harder. Moreover, the lack of available data limits the adaptation of inspection models to new defect types that appear in production as it might require a model retraining in order to incorporate the detects and detect them. In this work, we have explored the technique of weight imprinting in the context of solar cell quality inspection where we have trained a network on three base defect classes, and then we have incorporated new defect classes using few samples. The results have shown that this technique allows the network to extend its knowledge with regard to defect classes with few samples, which can be interesting for industrial practitioners.
Triangle-oriented Community Detection considering Node Features and Network Topology
Venue: arXiv cs.SI 2022
Year: 2022

The joint use of node features and network topology to detect communities is called community detection in attributed networks. Most of the existing work along this line has been carried out through objective function optimization and has proposed numerous approaches. However, they tend to focus only on lower-order details, i.e., capture node features and network topology from node and edge views, and purely seek a higher degree of optimization to guarantee the quality of the found communities, which exacerbates unbalanced communities and free-rider effect. To further clarify and reveal the intrinsic nature of networks, we conduct triangle-oriented community detection considering node features and network topology. Specifically, we first introduce a triangle-based quality metric to preserve higher-order details of node features and network topology, and then formulate so-called two-level constraints to encode lower-order details of node features and network topology. Finally, we develop a local search framework based on optimizing our objective function consisting of the proposed quality metric and two-level constraints to achieve both non-overlapping and overlapping community detection in attributed networks. Extensive experiments demonstrate the effectiveness and efficiency of our framework and its potential in alleviating unbalanced communities and free-rider effect.
ReCoAt: A Deep Learning-based Framework for Multi-Modal Motion Prediction in Autonomous Driving Application
Venue: arXiv cs.RO 2022
Year: 2022

This paper proposes a novel deep learning framework for multi-modal motion prediction. The framework consists of three parts: recurrent neural networks to process the target agent's motion process, convolutional neural networks to process the rasterized environment representation, and a distance-based attention mechanism to process the interactions among different agents. We validate the proposed framework on a large-scale real-world driving dataset, Waymo open motion dataset, and compare its performance against other methods on the standard testing benchmark. The qualitative results manifest that the predicted trajectories given by our model are accurate, diverse, and in accordance with the road structure. The quantitative results on the standard benchmark reveal that our model outperforms other baseline methods in terms of prediction accuracy and other evaluation metrics. The proposed framework is the second-place winner of the 2021 Waymo open dataset motion prediction challenge.
Brief Industry Paper: The Necessity of Adaptive Data Fusion in Infrastructure-Augmented Autonomous Driving System
Venue: arXiv cs.DC 2022
Year: 2022

This paper is the first to provide a thorough system design overview along with the fusion methods selection criteria of a real-world cooperative autonomous driving system, named Infrastructure-Augmented Autonomous Driving or IAAD. We present an in-depth introduction of the IAAD hardware and software on both road-side and vehicle-side computing and communication platforms. We extensively characterize the IAAD system in the context of real-world deployment scenarios and observe that the network condition that fluctuates along the road is currently the main technical roadblock for cooperative autonomous driving. To address this challenge, we propose new fusion methods, dubbed "inter-frame fusion" and "planning fusion" to complement the current state-of-the-art "intra-frame fusion". We demonstrate that each fusion method has its own benefit and constraint.
Golfer: Trajectory Prediction with Masked Goal Conditioning MnM Network
Venue: arXiv cs.CV 2022
Year: 2022

Transformers have enabled breakthroughs in NLP and computer vision, and have recently began to show promising performance in trajectory prediction for Autonomous Vehicle (AV). How to efficiently model the interactive relationships between the ego agent and other road and dynamic objects remains challenging for the standard attention module. In this work we propose a general Transformer-like architectural module MnM network equipped with novel masked goal conditioning training procedures for AV trajectory prediction. The resulted model, named golfer, achieves state-of-the-art performance, winning the 2nd place in the 2022 Waymo Open Dataset Motion Prediction Challenge and ranked 1st place according to minADE.
Rationale-Augmented Ensembles in Language Models
Venue: arXiv cs.CL 2022
Year: 2022

Recent research has shown that rationales, or step-by-step chains of thought, can be used to improve performance in multi-step reasoning tasks. We reconsider rationale-augmented prompting for few-shot in-context learning, where (input -> output) prompts are expanded to (input, rationale -> output) prompts. For rationale-augmented prompting we demonstrate how existing approaches, which rely on manual prompt engineering, are subject to sub-optimal rationales that may harm performance. To mitigate this brittleness, we propose a unified framework of rationale-augmented ensembles, where we identify rationale sampling in the output space as the key component to robustly improve performance. This framework is general and can easily be extended to common natural language processing tasks, even those that do not traditionally leverage intermediate steps, such as question answering, word sense disambiguation, and sentiment analysis. We demonstrate that rationale-augmented ensembles achieve more accurate and interpretable results than existing prompting approaches--including standard prompting without rationales and rationale-based chain-of-thought prompting--while simultaneously improving interpretability of model predictions through the associated rationales.
GUIM -- General User and Item Embedding with Mixture of Representation in E-commerce
Venue: arXiv cs.AI 2022
Year: 2022

Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites. The representation of users and items has been playing a critical role in various downstream applications, including recommendation system, search, marketing, demand forecasting and so on. Inspired from the BERT model in natural language processing (NLP) domain, we propose a GUIM (General User Item embedding with Mixture of representation) model to achieve the goal with massive, structured, multi-modal data including the interactions among hundreds of millions of users and items. We utilize mixture of representation (MoR) as a novel representation form to model the diverse interests of each user. In addition, we use the InfoNCE from contrastive learning to avoid intractable computational costs due to the numerous size of item (token) vocabulary. Finally, we propose a set of representative downstream tasks to serve as a standard benchmark to evaluate the quality of the learned user and/or item embeddings, analogous to the GLUE benchmark in NLP domain. Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model.
MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages
Venue: arXiv cs.CL 2022
Year: 2022

We present the results of the Workshop on Multilingual Information Access (MIA) 2022 Shared Task, evaluating cross-lingual open-retrieval question answering (QA) systems in 16 typologically diverse languages. In this task, we adapted two large-scale cross-lingual open-retrieval QA datasets in 14 typologically diverse languages, and newly annotated open-retrieval QA data in 2 underrepresented languages: Tagalog and Tamil. Four teams submitted their systems. The best system leveraging iteratively mined diverse negative examples and larger pretrained models achieves 32.2 F1, outperforming our baseline by 4.5 points. The second best system uses entity-aware contextualized representations for document retrieval, and achieves significant improvements in Tamil (20.8 F1), whereas most of the other systems yield nearly zero scores.
Long-Tail Prediction Uncertainty Aware Trajectory Planning for Self-driving Vehicles
Venue: arXiv cs.AI 2022
Year: 2022

A typical trajectory planner of autonomous driving commonly relies on predicting the future behavior of surrounding obstacles. Recently, deep learning technology has been widely adopted to design prediction models due to their impressive performance. However, such models may fail in the "long-tail" driving cases where the training data is sparse or unavailable, leading to planner failures. To this end, this work proposes a trajectory planner to consider the prediction model uncertainty arising from insufficient data for safer performance. Firstly, an ensemble network structure estimates the prediction model's uncertainty due to insufficient training data. Then a trajectory planner is designed to consider the worst-case arising from prediction uncertainty. The results show that the proposed method can improve the safety of trajectory planning under the prediction uncertainty caused by insufficient data. At the same time, with sufficient data, the framework will not lead to overly conservative results. This technology helps to improve the safety and reliability of autonomous vehicles under the long-tail data distribution of the real world.
Two-Timescale Design for STAR-RIS Aided NOMA Systems
Venue: arXiv cs.IT 2022
Year: 2022

Simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) have emerged as a promising technology for achieving full-space coverage. Prior works on STAR-RISs mostly assumed the full and instantaneous channel state information (CSI) is available, which, however, is practically difficult to obtain due to the large number of elements. To address it, we investigate STAR-RIS aided NOMA systems, where two efficient two-timescale transmission protocols are proposed for different channel setups to maximize the average sum-rate. Specifically, 1) for line-of-sight (LoS) dominant channels, we propose the beamforming-then-estimate (BTE) Protocol, where the long-term STAR-RIS coefficients are optimized based on the statistical CSI, while the short-term power allocation at the base station (BS) is designed based on the effective channels; 2) for the rich scattering environment, we propose an alternative partition-then-estimate (PTE) Protocol, where the BS determines the long-term STAR-RIS surface-partition strategy; then the BS estimates the instantaneous subsurface channels and designs its power allocation and STAR-RIS phase-shifts accordingly. Simulation results validate the superiority of our proposed transmission protocols as compared to various benchmarks. It is shown that the BTE Protocol outperforms the PTE Protocol when the number of STAR-RIS elements is large and/or the LoS channel components are dominant, and vice versa.
Benchmarks for Industrial Inspection Based on Structured Light
Venue: arXiv cs.CV 2022
Year: 2022

Robustness and accuracy are two critical metrics for industrial inspection. In this paper, we propose benchmarks that can evaluate the structured light method's performance. Our evaluation metric was learning from a lot of inspection tasks from the factories. The metric we proposed consists of four detailed criteria such as flatness, length, height and sphericity. Then we can judge whether the structured light method/device can be applied to a specified inspection task by our evaluation metric quickly. A structured light device built for TypeC pin needles inspection performance is evaluated via our metrics in the final experimental section.
Corner-based implicit patches
Venue: arXiv cs.GR 2022
Year: 2022

Multi-sided surfaces are often defined by side interpolants (also called ribbons), i.e. the surface has to connect to the ribbons with a prescribed degree of smoothness. The I-patch is such a family of implicit surfaces capable of interpolating an arbitrary number of ribbons and can be used in design and approximation. While in the case of parametric surfaces describing ribbons is a well-discussed problem, defining implicit ribbons is a different task. This paper will introduce corner I-patches, a new representation that describes implicit surfaces based on corner interpolants. Those may be defined with much simpler surfaces, while the shape of the patch will depend on a handful of scalar parameters. Continuity between patches will be enforced via constraints on these parameters. Corner I-patches have several favorable properties that can be exploited for example in volume rendering or approximation.
Less is More: Adaptive Curriculum Learning for Thyroid Nodule Diagnosis
Venue: arXiv cs.CV 2022
Year: 2022

Thyroid nodule classification aims at determining whether the nodule is benign or malignant based on a given ultrasound image. However, the label obtained by the cytological biopsy which is the golden standard in clinical medicine is not always consistent with the ultrasound imaging TI-RADS criteria. The information difference between the two causes the existing deep learning-based classification methods to be indecisive. To solve the Inconsistent Label problem, we propose an Adaptive Curriculum Learning (ACL) framework, which adaptively discovers and discards the samples with inconsistent labels. Specifically, ACL takes both hard sample and model certainty into account, and could accurately determine the threshold to distinguish the samples with Inconsistent Label. Moreover, we contribute TNCD: a Thyroid Nodule Classification Dataset to facilitate future related research on the thyroid nodules. Extensive experimental results on TNCD based on three different backbone networks not only demonstrate the superiority of our method but also prove that the less-is-more principle which strategically discards the samples with Inconsistent Label could yield performance gains. Source code and data are available at https://github.com/chenghui-666/ACL/.
GOF-TTE: Generative Online Federated Learning Framework for Travel Time Estimation
Venue: arXiv cs.LG 2022
Year: 2022

Estimating the travel time of a path is an essential topic for intelligent transportation systems. It serves as the foundation for real-world applications, such as traffic monitoring, route planning, and taxi dispatching. However, building a model for such a data-driven task requires a large amount of users' travel information, which directly relates to their privacy and thus is less likely to be shared. The non-Independent and Identically Distributed (non-IID) trajectory data across data owners also make a predictive model extremely challenging to be personalized if we directly apply federated learning. Finally, previous work on travel time estimation does not consider the real-time traffic state of roads, which we argue can significantly influence the prediction. To address the above challenges, we introduce GOF-TTE for the mobile user group, Generative Online Federated Learning Framework for Travel Time Estimation, which I) utilizes the federated learning approach, allowing private data to be kept on client devices while training, and designs the global model as an online generative model shared by all clients to infer the real-time road traffic state. II) apart from sharing a base model at the server, adapts a fine-tuned personalized model for every client to study their personal driving habits, making up for the residual error made by localized global model prediction. % III) designs the global model as an online generative model shared by all clients to infer the real-time road traffic state. We also employ a simple privacy attack to our framework and implement the differential privacy mechanism to further guarantee privacy safety. Finally, we conduct experiments on two real-world public taxi datasets of DiDi Chengdu and Xi'an. The experimental results demonstrate the effectiveness of our proposed framework.
Neural Networks for Path Planning
Venue: arXiv cs.AI 2022
Year: 2022

The scientific community is able to present a new set of solutions to practical problems that substantially improve the performance of modern technology in terms of efficiency and speed of computation due to the advancement in neural networks architectures. We present the latest works considering the utilization of neural networks in robot path planning. Our survey shows the contrast between different formulations of the problems that consider different inputs, outputs, and environments and how different neural networks architectures are able to provide solutions to all of the presented problems.
Biological Robots: Perspectives on an Emerging Interdisciplinary Field
Venue: arXiv cs.RO 2022
Year: 2022

Advances in science and engineering often reveal the limitations of classical approaches initially used to understand, predict, and control phenomena. With progress, conceptual categories must often be re-evaluated to better track recently discovered invariants across disciplines. It is essential to refine frameworks and resolve conflicting boundaries between disciplines such that they better facilitate, not restrict, experimental approaches and capabilities. In this essay, we discuss issues at the intersection of developmental biology, computer science, and robotics. In the context of biological robots, we explore changes across concepts and previously distinct fields that are driven by recent advances in materials, information, and life sciences. Herein, each author provides their own perspective on the subject, framed by their own disciplinary training. We argue that as with computation, certain aspects of developmental biology and robotics are not tied to specific materials; rather, the consilience of these fields can help to shed light on issues of multi-scale control, self-assembly, and relationships between form and function. We hope new fields can emerge as boundaries arising from technological limitations are overcome, furthering practical applications from regenerative medicine to useful synthetic living machines.
Derivatives of Complete Weight Enumerators and New Balance Principle of Binary Self-Dual Codes
Venue: arXiv cs.IT 2022
Year: 2022

Let H be the standard Hadamard matrix of order two and let K=2^{-1/2}H. It is known that the complete weight enumerator $\ W$ of a binary self-dual code of length $n$ is an eigenvector corresponding to an eigenvalue 1 of the Kronecker power $K^{[n]}.$ For every integer $t$ in the interval [0,n] we define the derivative of order $t$, $W_{<t>},$ of $W$ in such a way that $W_{<t>}$ is in the eigenspace of $\ 1$ of the matrix $K^{[n-t]}.$ For large values of $t,$ $W_{<t>}$ contains less information about the code but has smaller length while $W_{<0>}=W$ completely determines the code. We compute the derivative of order $n-5$ for the extended Golay code of length 24, the extended quadratic residue code of length 48, and the putative [72,24,12] code and show that they are in the eigenspace of $\ 1$ of the matrix $% K^{[5]}.$ We use the derivatives to prove a new balance equation which involves the number of code vectors of given weight having 1 in a selected coordinate position. As an example, we use the balance equation to eliminate some candidates for weight enumerators of binary self-dual codes of length eight.
Software Engineering Process and Methodology in Blockchain-Oriented Software Development: A Systematic Study
Venue: arXiv cs.SE 2022
Year: 2022

Software Engineering is the process of a systematic, disciplined, quantifiable approach that has significant impact on large-scale and complex software development. Scores of well-established software process models have long been adopted in the software development life cycle that pour stakeholders towards the completion of final software product development. Within the boundary of advanced technology, various emerging and futuristic technology is evolving that really need the attention of the software engineering community whether the conventional software process techniques are capable to inherit the core fundamental into futuristic software development. In this paper, we study the impact of existing software engineering processes and models including Agile, and DevOps in Blockchain-Oriented Software Engineering. We also examine the essentiality of adopting state-of-art concepts and evolving the current software engineering process for blockchain-oriented systems. We discuss the insight of software project management practices in BOS development. The findings of this study indicate that utilizing state-of-art techniques in software processes for futuristic technology would be challenging and promising research is needed extensively towards addressing and improving state-of-the-art software engineering processes and methodology for novel technologies.