OpenRaiser/Intern-Atlas huggingface dataset:
VideoPipe 2022 Challenge: Real-World Video Understanding for Urban Pipe Inspection
Venue: arXiv cs.CV 2022
Year: 2022

Video understanding is an important problem in computer vision. Currently, the well-studied task in this research is human action recognition, where the clips are manually trimmed from the long videos, and a single class of human action is assumed for each clip. However, we may face more complicated scenarios in the industrial applications. For example, in the real-world urban pipe system, anomaly defects are fine-grained, multi-labeled, domain-relevant. To recognize them correctly, we need to understand the detailed video content. For this reason, we propose to advance research areas of video understanding, with a shift from traditional action recognition to industrial anomaly analysis. In particular, we introduce two high-quality video benchmarks, namely QV-Pipe and CCTV-Pipe, for anomaly inspection in the real-world urban pipe systems. Based on these new datasets, we will host two competitions including (1) Video Defect Classification on QV-Pipe and (2) Temporal Defect Localization on CCTV-Pipe. In this report, we describe the details of these benchmarks, the problem definitions of competition tracks, the evaluation metric, and the result summary. We expect that, this competition would bring new opportunities and challenges for video understanding in smart city and beyond. The details of our VideoPipe challenge can be found in https://videopipe.github.io.
User Value in Modern Payment Platforms: A Graph Approach
Venue: arXiv cs.SI 2022
Year: 2022

Payment platforms have significantly evolved in recent years to keep pace with the proliferation of online and cashless payments. These platforms are increasingly aligned with online social networks, allowing users to interact with each other and transfer small amounts of money in a Peer-to-Peer fashion. This poses new challenges for analysing payment data, as traditional methods are only user-centric or business-centric and neglect the network users build during the interaction. This paper proposes a first methodology for measuring user value in modern payment platforms. We combine quantitative user-centric metrics with an analysis of the graph created by users' activities and its topological features inspired by the evolution of opinions in social networks. We showcase our approach using a dataset from a large operational payment platform and show how it can support business decisions and marketing campaign design, e.g., by targeting specific users.
Content-based Graph Privacy Advisor
Venue: arXiv cs.CV 2022
Year: 2022

People may be unaware of the privacy risks of uploading an image online. In this paper, we present Graph Privacy Advisor, an image privacy classifier that uses scene information and object cardinality as cues to predict whether an image is private. Graph Privacy Advisor simplifies a state-of-the-art graph model and improves its performance by refining the relevance of the information extracted from the image. We determine the most informative visual features to be used for the privacy classification task and reduce the complexity of the model by replacing high-dimensional image feature vectors with lower-dimensional, more effective features. We also address the problem of biased prior information by modelling object co-occurrences instead of the frequency of object occurrences in each class.
Facial Expression Video Generation Based-On Spatio-temporal Convolutional GAN: FEV-GAN
Venue: arXiv cs.CV 2022
Year: 2022

Facial expression generation has always been an intriguing task for scientists and researchers all over the globe. In this context, we present our novel approach for generating videos of the six basic facial expressions. Starting from a single neutral facial image and a label indicating the desired facial expression, we aim to synthesize a video of the given identity performing the specified facial expression. Our approach, referred to as FEV-GAN (Facial Expression Video GAN), is based on Spatio-temporal Convolutional GANs, that are known to model both content and motion in the same network. Previous methods based on such a network have shown a good ability to generate coherent videos with smooth temporal evolution. However, they still suffer from low image quality and low identity preservation capability. In this work, we address this problem by using a generator composed of two image encoders. The first one is pre-trained for facial identity feature extraction and the second for spatial feature extraction. We have qualitatively and quantitatively evaluated our model on two international facial expression benchmark databases: MUG and Oulu-CASIA NIR&VIS. The experimental results analysis demonstrates the effectiveness of our approach in generating videos of the six basic facial expressions while preserving the input identity. The analysis also proves that the use of both identity and spatial features enhances the decoder ability to better preserve the identity and generate high-quality videos. The code and the pre-trained model will soon be made publicly available.
Drones e Inteligencia Artificial para Investigación y Competición
Venue: arXiv cs.RO 2022
Year: 2022

This work focuses on drones or UAVs (Unmanned Aerial Vehicles) for use in industry in general. These vehicles have a large number of uses and potential in the industry, as a tool for civil engineering, medicine, mining, among others. However, this vehicle is limited for use indoors due to the need for GPS and it does not work indoors. In this way, this work presents a UAV that works without GPS, thus being able to be used in closed spaces for example and have good precision. The work is based on an approach that uses computer vision and GPS.
Modelling and Explaining Legal Case-based Reasoners through Classifiers
Venue: arXiv cs.AI 2022
Year: 2022

This paper brings together two lines of research: factor-based models of case-based reasoning (CBR) and the logical specification of classifiers. Logical approaches to classifiers capture the connection between features and outcomes in classifier systems. Factor-based reasoning is a popular approach to reasoning by precedent in AI & Law. Horty (2011) has developed the factor-based models of precedent into a theory of precedential constraint. In this paper we combine the modal logic approach (binary-input classifier, BLC) to classifiers and their explanations given by Liu & Lorini (2021) with Horty's account of factor-based CBR, since both a classifier and CBR map sets of features to decisions or classifications. We reformulate case bases of Horty in the language of BCL, and give several representation results. Furthermore, we show how notions of CBR, e.g. reason, preference between reasons, can be analyzed by notions of classifier system.
A Study of Teacher Educators Skill and ICT Integration in Online Teaching during the Pandemic Situation in India
Venue: arXiv cs.CY 2022
Year: 2022

Information and communication technology prompted the sharing of information over the world. For its impact on education the government and the authorities like the University Grants Commission in India have energized the higher education institutions in India to implement online education during the pandemic situation. This paper attempts to know the teaching faculties ICT skills and related online class skills in higher educational institutions in India. In India like developing countries quick as the lightning change in traditional to fully online classes are like a rumble of thunder because faculties are adopting this situation but students are challenging to adopt.
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts
Venue: arXiv cs.CL 2022
Year: 2022

Prompt tuning is a parameter-efficient tuning (PETuning) method for utilizing pre-trained models (PTMs) that simply prepends a soft prompt to the input and only optimizes the prompt to adapt PTMs to downstream tasks. Although it is parameter- and deployment-efficient, its performance still lags behind other state-of-the-art PETuning methods. Besides, the training cost of prompt tuning is not significantly reduced due to the back-propagation through the entire model. Through empirical analyses, we shed some light on the lagging performance of prompt tuning and recognize a trade-off between the propagation distance from label signals to the inserted prompt and the influence of the prompt on model outputs. Further, we present Late Prompt Tuning (LPT) that inserts a late prompt into an intermediate layer of the PTM instead of the input layer or all layers. The late prompt is obtained by a neural prompt generator conditioned on the hidden states before the prompt insertion layer and therefore is instance-dependent. Through extensive experimental results across various tasks and PTMs, we show that LPT can achieve competitive performance to full model tuning and other PETuning methods under both full-data and few-shot scenarios while possessing faster training speed and lower memory cost.
A Study of Scalarisation Techniques for Multi-Objective QUBO Solving
Venue: arXiv cs.AI 2022
Year: 2022

In recent years, there has been significant research interest in solving Quadratic Unconstrained Binary Optimisation (QUBO) problems. Physics-inspired optimisation algorithms have been proposed for deriving optimal or sub-optimal solutions to QUBOs. These methods are particularly attractive within the context of using specialised hardware, such as quantum computers, application specific CMOS and other high performance computing resources for solving optimisation problems. These solvers are then applied to QUBO formulations of combinatorial optimisation problems. Quantum and quantum-inspired optimisation algorithms have shown promising performance when applied to academic benchmarks as well as real-world problems. However, QUBO solvers are single objective solvers. To make them more efficient at solving problems with multiple objectives, a decision on how to convert such multi-objective problems to single-objective problems need to be made. In this study, we compare methods of deriving scalarisation weights when combining two objectives of the cardinality constrained mean-variance portfolio optimisation problem into one. We show significant performance improvement (measured in terms of hypervolume) when using a method that iteratively fills the largest space in the Pareto front compared to a näive approach using uniformly generated weights.
The Effectiveness of Social Media Engagement Strategy on Disaster Fundraising
Venue: arXiv cs.SI 2022
Year: 2022

Social media has been a powerful tool and an integral part of communication, especially during natural disasters. Social media platforms help nonprofits in effective disaster management by disseminating crucial information to various communities at the earliest. Besides spreading information to every corner of the world, various platforms incorporate many features that give access to host online fundraising events, process online donations, etc. The current literature lacks the theoretical structure investigating the correlation between social media engagement and crisis management. Large nonprofit organisations like the Australian Red Cross have upscaled their operations to help nearly 6,000 bushfire survivors through various grants and helped 21,563 people with psychological support and other assistance through their recovery program (Australian Red Cross, 2021). This paper considers the case of bushfires in Australia 2019-2020 to inspect the role of social media in escalating fundraising via analysing the donation data of the Australian Red Cross from October 2019 - March 2020 and analysing the level of public interaction with their Facebook page and its content in the same period.
Deep reinforcement learning oriented for real world dynamic scenarios
Venue: arXiv cs.RO 2022
Year: 2022

Autonomous navigation in dynamic environments is a complex but essential task for autonomous robots. Recent deep reinforcement learning approaches show promising results to solve the problem, but it is not solved yet, as they typically assume no robot kinodynamic restrictions, holonomic movement or perfect environment knowledge. Moreover, most algorithms fail in the real world due to the inability to generate real-world training data for the huge variability of possible scenarios. In this work, we present a novel planner, DQN-DOVS, that uses deep reinforcement learning on a descriptive robocentric velocity space model to navigate in highly dynamic environments. It is trained using a smart curriculum learning approach on a simulator that faithfully reproduces the real world, reducing the gap between the reality and simulation. We test the resulting algorithm in scenarios with different number of obstacles and compare it with many state-of-the-art approaches, obtaining a better performance. Finally, we try the algorithm in a ground robot, using the same setup as in the simulation experiments.
Transcending Scaling Laws with 0.1% Extra Compute
Venue: arXiv cs.CL 2022
Year: 2022

Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. The key idea is to continue training a state-of-the-art large language model (e.g., PaLM) on a few more steps with UL2's mixture-of-denoiser objective. We show that, with almost negligible extra computational costs and no new sources of data, we are able to substantially improve the scaling properties of large language models on downstream metrics. In this paper, we continue training PaLM with UL2R, introducing a new set of models at 8B, 62B, and 540B scale which we call U-PaLM. Impressively, at 540B scale, we show an approximately 2x computational savings rate where U-PaLM achieves the same performance as the final PaLM 540B model at around half its computational budget (i.e., saving $\sim$4.4 million TPUv4 hours). We further show that this improved scaling curve leads to 'emergent abilities' on challenging BIG-Bench tasks -- for instance, U-PaLM does much better than PaLM on some tasks or demonstrates better quality at much smaller scale (62B as opposed to 540B). Overall, we show that U-PaLM outperforms PaLM on many few-shot setups, i.e., English NLP tasks (e.g., commonsense reasoning, question answering), reasoning tasks with chain-of-thought (e.g., GSM8K), multilingual tasks (MGSM, TydiQA), MMLU and challenging BIG-Bench tasks. Finally, we provide qualitative examples showing the new capabilities of U-PaLM for single and multi-span infilling.
GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network
Venue: arXiv cs.CV 2022
Year: 2022

Reconstructing 3D layouts from multiple $360^{\circ}$ panoramas has received increasing attention recently as estimating a complete layout of a large-scale and complex room from a single panorama is very difficult. The state-of-the-art method, called PSMNet, introduces the first learning-based framework that jointly estimates the room layout and registration given a pair of panoramas. However, PSMNet relies on an approximate (i.e., "noisy") registration as input. Obtaining this input requires a solution for wide baseline registration which is a challenging problem. In this work, we present a complete multi-view panoramic layout estimation framework that jointly learns panorama registration and layout estimation given a pair of panoramas without relying on a pose prior. The major improvement over PSMNet comes from a novel Geometry-aware Panorama Registration Network or GPR-Net that effectively tackles the wide baseline registration problem by exploiting the layout geometry and computing fine-grained correspondences on the layout boundaries, instead of the global pixel-space. Our architecture consists of two parts. First, given two panoramas, we adopt a vision transformer to learn a set of 1D horizon features sampled on the panorama. These 1D horizon features encode the depths of individual layout boundary samples and the correspondence and covisibility maps between layout boundaries. We then exploit a non-linear registration module to convert these 1D horizon features into a set of corresponding 2D boundary points on the layout. Finally, we estimate the final relative camera pose via RANSAC and obtain the complete layout simply by taking the union of registered layouts. Experimental results indicate that our method achieves state-of-the-art performance in both panorama registration and layout estimation on a large-scale indoor panorama dataset ZInD.
Application of artificial neural network to determine the thickness profile of thin film
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we introduce a novel artificial neural network (ANN) based scheme to estimate the thickness of thin films deposited on a given substrate. Here we consider the visible interference pattern between a plane wave and a diverging wave reflected from the thin film surface that records the thickness information of the thin film. We assume a uniform thickness profile of the film. However, the thickness increases as the deposition takes place. We extract the intensity data along a line through the center of the interference pattern. We train our network by using a number of such line information of known thickness profiles. The performance of the trained network is then tested by estimating the thickness of unknown surfaces. The numerical simulation results show that the proposed technique can be very much useful for automated measurement of thickness, quickly and in real time, during deposition
Text Enhancement for Paragraph Processing in End-to-End Code-switching TTS
Venue: arXiv cs.SD 2022
Year: 2022

Current end-to-end code-switching Text-to-Speech (TTS) can already generate high quality two languages speech in the same utterance with single speaker bilingual corpora. When the speakers of the bilingual corpora are different, the naturalness and consistency of the code-switching TTS will be poor. The cross-lingual embedding layers structure we proposed makes similar syllables in different languages relevant, thus improving the naturalness and consistency of generated speech. In the end-to-end code-switching TTS, there exists problem of prosody instability when synthesizing paragraph text. The text enhancement method we proposed makes the input contain prosodic information and sentence-level context information, thus improving the prosody stability of paragraph text. Experimental results demonstrate the effectiveness of the proposed methods in the naturalness, consistency, and prosody stability. In addition to Mandarin and English, we also apply these methods to Shanghaiese and Cantonese corpora, proving that the methods we proposed can be extended to other languages to build end-to-end code-switching TTS system.
The Natural Robotics Contest: Crowdsourced Biomimetic Design
Venue: arXiv cs.RO 2022
Year: 2022

Biomimetic and Bioinspired design is not only a potent resource for roboticists looking to develop robust engineering systems or understand the natural world. It is also a uniquely accessible entry point into science and technology. Every person on Earth constantly interacts with nature, and most people have an intuitive sense of animal and plant behavior, even without realizing it. The Natural Robotics Contest is novel piece of science communication that takes advantage of this intuition, and creates an opportunity for anyone with an interest in nature or robotics to submit their idea and have it turned into a real engineering system. In this paper we will discuss the competition's submissions, which show how the public thinks of nature as well as the problems people see as most pressing for engineers to solve. We will then show our design process from the winning submitted concept sketch through to functioning robot, to offer a case study in biomimetic robot design. The winning design is a robotic fish which uses gill structures to filter out microplastics. This was fabricated into an open source robot with a novel 3D printed gill design. By presenting the competition and the winning entry we hope to foster further interest in nature-inspired design, and increase the interplay between nature and engineering in the minds of readers.
G2NetPL: Generic Game-Theoretic Network for Partial-Label Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Multi-label image classification aims to predict all possible labels in an image. It is usually formulated as a partial-label learning problem, since it could be expensive in practice to annotate all the labels in every training image. Existing works on partial-label learning focus on the case where each training image is labeled with only a subset of its positive/negative labels. To effectively address partial-label classification, this paper proposes an end-to-end Generic Game-theoretic Network (G2NetPL) for partial-label learning, which can be applied to most partial-label settings, including a very challenging, but annotation-efficient case where only a subset of the training images are labeled, each with only one positive label, while the rest of the training images remain unlabeled. In G2NetPL, each unobserved label is associated with a soft pseudo label, which, together with the network, formulates a two-player non-zero-sum non-cooperative game. The objective of the network is to minimize the loss function with given pseudo labels, while the pseudo labels will seek convergence to 1 (positive) or 0 (negative) with a penalty of deviating from the predicted labels determined by the network. In addition, we introduce a confidence-aware scheduler into the loss of the network to adaptively perform easy-to-hard learning for different labels. Extensive experiments demonstrate that our proposed G2NetPL outperforms many state-of-the-art multi-label classification methods under various partial-label settings on three different datasets.
Choose Your Lenses: Flaws in Gender Bias Evaluation
Venue: arXiv cs.CL 2022
Year: 2022

Considerable efforts to measure and mitigate gender bias in recent years have led to the introduction of an abundance of tasks, datasets, and metrics used in this vein. In this position paper, we assess the current paradigm of gender bias evaluation and identify several flaws in it. First, we highlight the importance of extrinsic bias metrics that measure how a model's performance on some task is affected by gender, as opposed to intrinsic evaluations of model representations, which are less strongly connected to specific harms to people interacting with systems. We find that only a few extrinsic metrics are measured in most studies, although more can be measured. Second, we find that datasets and metrics are often coupled, and discuss how their coupling hinders the ability to obtain reliable conclusions, and how one may decouple them. We then investigate how the choice of the dataset and its composition, as well as the choice of the metric, affect bias measurement, finding significant variations across each of them. Finally, we propose several guidelines for more reliable gender bias evaluation.
Design Representation as Semantic Networks
Venue: arXiv cs.IR 2022
Year: 2022

Design representation is a common task in the design process to facilitate learning, analysis, redesign, communication, and other design activities. Traditional representation techniques rely on human expertise and manual construction and are difficult to repeat and scale. Here, we propose a methodology that utilizes a pre-trained large-scale cross-domain design knowledge base to automatically generate design representation as a semantic network, i.e., a network of the entities and relations, based on design descriptions in texts or natural languages. Our methodology requires no ad hoc statistics. Based on a participatory study, we reveal the effectiveness and differences of the semantic network representations that are automatically generated with alternative knowledge bases. The findings illuminate future research directions to enhance design representation as semantic networks.