OpenRaiser/Intern-Atlas huggingface dataset:
Personal Investigator: a Therapeutic 3D Game for Teenagers
Venue: arXiv cs.HC 2022
Year: 2022

This position paper describes the implementation and initial findings of a game called Personal Investigator (PI). PI is an online 3D detective game that implements a model of Brief Solution Focused Therapy (BSFT). It aims to help teenagers overcome mental health problems and engage with traditional mental health care services. It is predicted that the combination of goal-oriented gaming with a model of goal-oriented therapy will help to attract and sustain the interest of teenagers, a group that therapists often have difficulty engaging with. PI is the first game to integrate this established psychotherapy approach into an engaging online 3D game.
Demonstrating Eye Movement Biometrics in Virtual Reality
Venue: arXiv cs.HC 2022
Year: 2022

Thanks to the eye-tracking sensors that are embedded in emerging consumer devices like the Vive Pro Eye, we demonstrate that it is feasible to deliver user authentication via eye movement biometrics.
Finding a Hidden Edge
Venue: arXiv cs.DS 2022
Year: 2022

We consider the problem of finding an edge in a hidden undirected graph $G = (V, E)$ with $n$ vertices, in a model where we only allowed queries that ask whether or not a subset of vertices contains an edge. We study the non-adaptive model and show that while in the deterministic model the optimal algorithm requires $\binom{n}{2}$ queries (i.e., querying for any possible edge separately), in the randomized model $\tildeΘ(n)$ queries are sufficient (and needed) in order to find an edge. In addition, we study the query complexity for specific families of graphs, including Stars, Cliques, and Matchings, for both the randomized and deterministic models. Lastly, for general graphs, we show a trade-off between the query complexity and the number of rounds, $r$, made by an adaptive algorithm. We present two algorithms with $O(rn^{2/r})$ and $\tilde{O}(rn^{1/r})$ sample complexity for the deterministic and randomized models, respectively.
Swin Deformable Attention U-Net Transformer (SDAUT) for Explainable Fast MRI
Venue: arXiv cs.CV 2022
Year: 2022

Fast MRI aims to reconstruct a high fidelity image from partially observed measurements. Exuberant development in fast MRI using deep learning has been witnessed recently. Meanwhile, novel deep learning paradigms, e.g., Transformer based models, are fast-growing in natural language processing and promptly developed for computer vision and medical image analysis due to their prominent performance. Nevertheless, due to the complexity of the Transformer, the application of fast MRI may not be straightforward. The main obstacle is the computational cost of the self-attention layer, which is the core part of the Transformer, can be expensive for high resolution MRI inputs. In this study, we propose a new Transformer architecture for solving fast MRI that coupled Shifted Windows Transformer with U-Net to reduce the network complexity. We incorporate deformable attention to construe the explainability of our reconstruction model. We empirically demonstrate that our method achieves consistently superior performance on the fast MRI task. Besides, compared to state-of-the-art Transformer models, our method has fewer network parameters while revealing explainability. The code is publicly available at https://github.com/ayanglab/SDAUT.
Spatial Transformation for Image Composition via Correspondence Learning
Venue: arXiv cs.CV 2022
Year: 2022

When using cut-and-paste to acquire a composite image, the geometry inconsistency between foreground and background may severely harm its fidelity. To address the geometry inconsistency in composite images, several existing works learned to warp the foreground object for geometric correction. However, the absence of annotated dataset results in unsatisfactory performance and unreliable evaluation. In this work, we contribute a Spatial TRAnsformation for virtual Try-on (STRAT) dataset covering three typical application scenarios. Moreover, previous works simply concatenate foreground and background as input without considering their mutual correspondence. Instead, we propose a novel correspondence learning network (CorrelNet) to model the correspondence between foreground and background using cross-attention maps, based on which we can predict the target coordinate that each source coordinate of foreground should be mapped to on the background. Then, the warping parameters of foreground object can be derived from pairs of source and target coordinates. Additionally, we learn a filtering mask to eliminate noisy pairs of coordinates to estimate more accurate warping parameters. Extensive experiments on our STRAT dataset demonstrate that our proposed CorrelNet performs more favorably against previous methods.
Composite FORCE learning of chaotic echo state networks for time-series prediction
Venue: arXiv cs.LG 2022
Year: 2022

Echo state network (ESN), a kind of recurrent neural networks, consists of a fixed reservoir in which neurons are connected randomly and recursively and obtains the desired output only by training output connection weights. First-order reduced and controlled error (FORCE) learning is an online supervised training approach that can change the chaotic activity of ESNs into specified activity patterns. This paper proposes a composite FORCE learning method based on recursive least squares to train ESNs whose initial activity is spontaneously chaotic, where a composite learning technique featured by dynamic regressor extension and memory data exploitation is applied to enhance parameter convergence. The proposed method is applied to a benchmark problem about predicting chaotic time series generated by the Mackey-Glass system, and numerical results have shown that it significantly improves learning and prediction performances compared with existing methods.
Transformers are Adaptable Task Planners
Venue: arXiv cs.RO 2022
Year: 2022

Every home is different, and every person likes things done in their particular way. Therefore, home robots of the future need to both reason about the sequential nature of day-to-day tasks and generalize to user's preferences. To this end, we propose a Transformer Task Planner(TTP) that learns high-level actions from demonstrations by leveraging object attribute-based representations. TTP can be pre-trained on multiple preferences and shows generalization to unseen preferences using a single demonstration as a prompt in a simulated dishwasher loading task. Further, we demonstrate real-world dish rearrangement using TTP with a Franka Panda robotic arm, prompted using a single human demonstration.
A Learning System for Motion Planning of Free-Float Dual-Arm Space Manipulator towards Non-Cooperative Object
Venue: arXiv cs.RO 2022
Year: 2022

Recent years have seen the emergence of non-cooperative objects in space, like failed satellites and space junk. These objects are usually operated or collected by free-float dual-arm space manipulators. Thanks to eliminating the difficulties of modeling and manual parameter-tuning, reinforcement learning (RL) methods have shown a more promising sign in the trajectory planning of space manipulators. Although previous studies demonstrate their effectiveness, they cannot be applied in tracking dynamic targets with unknown rotation (non-cooperative objects). In this paper, we proposed a learning system for motion planning of free-float dual-arm space manipulator (FFDASM) towards non-cooperative objects. Specifically, our method consists of two modules. Module I realizes the multi-target trajectory planning for two end-effectors within a large target space. Next, Module II takes as input the point clouds of the non-cooperative object to estimate the motional property, and then can predict the position of target points on an non-cooperative object. We leveraged the combination of Module I and Module II to track target points on a spinning object with unknown regularity successfully. Furthermore, the experiments also demonstrate the scalability and generalization of our learning system.
Dual Decision Improves Open-Set Panoptic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Open-set panoptic segmentation (OPS) problem is a new research direction aiming to perform segmentation for both \known classes and \unknown classes, i.e., the objects ("things") that are never annotated in the training set. The main challenges of OPS are twofold: (1) the infinite possibility of the \unknown object appearances makes it difficult to model them from a limited number of training data. (2) at training time, we are only provided with the "void" category, which essentially mixes the "unknown thing" and "background" classes. We empirically find that directly using "void" category to supervise \known class or "background" classifiers without screening will lead to an unsatisfied OPS result. In this paper, we propose a divide-and-conquer scheme to develop a dual decision process for OPS. We show that by properly combining a \known class discriminator with an additional class-agnostic object prediction head, the OPS performance can be significantly improved. Specifically, we first propose to create a classifier with only \known categories and let the "void" class proposals achieve low prediction probability from those categories. Then we distinguish the "unknown things" from the background by using the additional object prediction head. To further boost performance, we introduce "unknown things" pseudo-labels generated from up-to-date models to enrich the training set. Our extensive experimental evaluation shows that our approach significantly improves \unknown class panoptic quality, with more than 30\% relative improvements than the existing best-performed method.
Ask Me What You Need: Product Retrieval using Knowledge from GPT-3
Venue: arXiv cs.IR 2022
Year: 2022

As online merchandise become more common, many studies focus on embedding-based methods where queries and products are represented in the semantic space. These methods alleviate the problem of vocab mismatch between the language of queries and products. However, past studies usually dealt with queries that precisely describe the product, and there still exists the need to answer imprecise queries that may require common sense knowledge, i.e., 'what should I get my mom for Mother's Day.' In this paper, we propose a GPT-3 based product retrieval system that leverages the knowledge-base (KB) of GPT-3 for question answering; users do not need to know the specific illustrative keywords for a product when querying. Our method tunes prompt tokens of GPT-3 to prompt knowledge and render answers that are mapped directly to products without further processing. Our method shows consistent performance improvement on two real-world and one public dataset, compared to the baseline methods. We provide an in-depth discussion on leveraging GPT-3 knowledge into a question answering based retrieval system.
Transaction Monitoring of Smart Contracts
Venue: arXiv cs.CR 2022
Year: 2022

Blockchains are modern distributed systems that provide decentralized financial capabilities with trustable guarantees. Smart contracts are programs written in specialized programming languages running on a blockchain and govern how tokens and cryptocurrency are sent and received. Smart contracts can invoke other contracts during the execution of transactions initiated by external users. Once deployed, smart contracts cannot be modified and their pitfalls can cause malfunctions and losses, for example by attacks from malicious users. Runtime verification is a very appealing technique to improve the reliability of smart contracts. One approach consists of specifying undesired executions (never claims) and detecting violations of the specification on the fly. This can be done by extending smart contracts with additional instructions corresponding to monitor specified properties, resulting in an onchain monitoring approach. In this paper, we study transaction monitoring that consists of detecting violations of complete transaction executions and not of individual operations within transactions. Our main contributions are to show that transaction monitoring is not possible in most blockchains and propose different execution mechanisms that would enable transaction monitoring.
Smart Education: Higher Education Instruction and the Internet of Things (IoT)
Venue: arXiv cs.CY 2022
Year: 2022

The Internet of Things (IoT) has many applications in our daily lives. One aspect in particular is how the IoT is making a substantial impact on education and learning; as we move into the 'Smart Educational' era. This article explores how the IoT continues to transform the education landscape, from classrooms and assessments to culture and attitudes. Smart Education is a pivotal tool in the fight to meet the educational challenges of tomorrow. The IoT tools are getting used more and more often in the area of education, aiming to increase student engagement, satisfaction and quality of learning. IoT will reshape student culture and habits beyond belief. As Smart Education is more than just using technologies, it involves a whole range of factors, from the educational management through to the pedagogical techniques and effectiveness. Educators in the 21st century now have access to gamification, smart devices, data management, and immersive technologies. Enabling academics to gather a variety of information from students. Ranging from monitoring student engagement to adapting the learning strategies for improved learning effectiveness. Through Smart Education, educators will be able to better monitor the needs of individual students and adjust their learning load correspondingly (i.e., optimal learning environment/workload to support and prevent students failing). One of the biggest challenges for educators is how new technologies will address growing problems (engagement and achievement). The scale and pace of change (technological IoT era) is unprecedented. Typically, jobs students are trained for today will not be here tomorrow. Education is not just about knowledge acquisition, but also the digital skills, adaptability and creativity (essential, if students are to thrive in the new world).
YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
Venue: arXiv cs.CV 2022
Year: 2022

YOLOv7 surpasses all known object detectors in both speed and accuracy in the range from 5 FPS to 160 FPS and has the highest accuracy 56.8% AP among all known real-time object detectors with 30 FPS or higher on GPU V100. YOLOv7-E6 object detector (56 FPS V100, 55.9% AP) outperforms both transformer-based detector SWIN-L Cascade-Mask R-CNN (9.2 FPS A100, 53.9% AP) by 509% in speed and 2% in accuracy, and convolutional-based detector ConvNeXt-XL Cascade-Mask R-CNN (8.6 FPS A100, 55.2% AP) by 551% in speed and 0.7% AP in accuracy, as well as YOLOv7 outperforms: YOLOR, YOLOX, Scaled-YOLOv4, YOLOv5, DETR, Deformable DETR, DINO-5scale-R50, ViT-Adapter-B and many other object detectors in speed and accuracy. Moreover, we train YOLOv7 only on MS COCO dataset from scratch without using any other datasets or pre-trained weights. Source code is released in https://github.com/WongKinYiu/yolov7.
Channel Estimation in RIS-Assisted MIMO Systems Operating Under Imperfections
Venue: arXiv cs.IT 2022
Year: 2022

Reconfigurable intelligent surface is a potential technology component of future wireless networks due to its capability of shaping the wireless environment. The promising MIMO systems in terms of extended coverage and enhanced capacity are, however, critically dependent on the accuracy of the channel state information. However, traditional channel estimation schemes are not applicable in RIS-assisted MIMO networks, since passive RISs typically lack the signal processing capabilities that are assumed by channel estimation algorithms. This becomes most problematic when physical imperfections or electronic impairments affect the RIS due to its exposition to different environmental effects or caused by hardware limitations from the circuitry. While these real-world effects are typically ignored in the literature, in this paper we propose efficient channel estimation schemes for RIS-assisted MIMO systems taking different imperfections into account. Specifically, we propose two sets of tensor-based algorithms, based on the parallel factor analysis decomposition schemes. First, by assuming a long-term model in which the RIS imperfections, modeled as unknown phase shifts, are static within the channel coherence time we formulate an iterative alternating least squares (ALS)-based algorithm for the joint estimation of the communication channels and the unknown phase deviations. Next, we develop the short-term imperfection model, which allows both amplitude and phase RIS imperfections to be non-static with respect to the channel coherence time. We propose two iterative ALS-based and closed-form higher order singular value decomposition-based algorithms for the joint estimation of the channels and the unknown impairments. Moreover, we analyze the identifiability and computational complexity of the proposed algorithms and study the effects of various imperfections on the channel estimation quality.
Incentivizing Proof-of-Stake Blockchain for Secured Data Collection in UAV-Assisted IoT: A Multi-Agent Reinforcement Learning Approach
Venue: arXiv cs.NI 2022
Year: 2022

The Internet of Things (IoT) can be conveniently deployed while empowering various applications, where the IoT nodes can form clusters to finish certain missions collectively. In this paper, we propose to employ unmanned aerial vehicles (UAVs) to assist the clustered IoT data collection with blockchain-based security provisioning. In particular, the UAVs generate candidate blocks based on the collected data, which are then audited through a lightweight proof-of-stake consensus mechanism within the UAV-based blockchain network. To motivate efficient blockchain while reducing the operational cost, a stake pool is constructed at the active UAV while encouraging stake investment from other UAVs with profit sharing. The problem is formulated to maximize the overall profit through the blockchain system in unit time by jointly investigating the IoT transmission, incentives through investment and profit sharing, and UAV deployment strategies. Then, the problem is solved in a distributed manner while being decoupled into two layers. The inner layer incorporates IoT transmission and incentive design, which are tackled with large-system approximation and one-leader-multi-follower Stackelberg game analysis, respectively. The outer layer for UAV deployment is undertaken with a multi-agent deep deterministic policy gradient approach. Results show the convergence of the proposed learning process and the UAV deployment, and also demonstrated is the performance superiority of our proposal as compared with the baselines.
Formalising Fisher's Inequality: Formal Linear Algebraic Proof Techniques in Combinatorics
Venue: arXiv cs.LO 2022
Year: 2022

The formalisation of mathematics is continuing rapidly, however combinatorics continues to present challenges to formalisation efforts, such as its reliance on techniques from a wide range of other fields in mathematics. This paper presents formal linear algebraic techniques for proofs on incidence structures in Isabelle/HOL, and their application to the first formalisation of Fisher's inequality. In addition to formalising incidence matrices and simple techniques for reasoning on linear algebraic representations, the formalisation focuses on the linear algebra bound and rank arguments. These techniques can easily be adapted for future formalisations in combinatorics, as we demonstrate through further application to proofs of variations on Fisher's inequality.
BFE and AdaBFE: A New Approach in Learning Rate Automation for Stochastic Optimization
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, a new gradient-based optimization approach by automatically adjusting the learning rate is proposed. This approach can be applied to design non-adaptive learning rate and adaptive learning rate. Firstly, I will introduce the non-adaptive learning rate optimization method: Binary Forward Exploration (BFE), and then the corresponding adaptive per-parameter learning rate method: Adaptive BFE (AdaBFE) is possible to be developed. This approach could be an alternative method to optimize the learning rate based on the stochastic gradient descent (SGD) algorithm besides the current non-adaptive learning rate methods e.g. SGD, momentum, Nesterov and the adaptive learning rate methods e.g. AdaGrad, AdaDelta, Adam... The purpose to develop this approach is not to beat the benchmark of other methods but just to provide a different perspective to optimize the gradient descent method, although some comparative study with previous methods will be made in the following sections. This approach is expected to be heuristic or inspire researchers to improve gradient-based optimization combined with previous methods.
Using Microservice Telemetry Data for System Dynamic Analysis
Venue: arXiv cs.SE 2022
Year: 2022

Microservices bring various benefits to software systems. They also bring decentralization and lose coupling across self-contained system parts. Since these systems likely evolve in a decentralized manner, they need to be monitored to identify when possibly poorly designed extensions deteriorate the overall system quality. For monolith systems, such tasks have been commonly addressed through static analysis. However, given the decentralization and possible language diversity across microservices, static analysis tools are lacking. On the other hand, there are available tools commonly used by practitioners that offer centralized logging, tracing, and metric collection for microservices. In this paper, we assess the opportunity to combine current dynamic analysis tools with anomaly detection in the form of quality metrics and anti-patterns. We develop a tool prototype that we use to assess a large microservice system benchmark demonstrating the feasibility and potential of such an approach.
Private Matrix Approximation and Geometry of Unitary Orbits
Venue: arXiv cs.DS 2022
Year: 2022

Consider the following optimization problem: Given $n \times n$ matrices $A$ and $Λ$, maximize $\langle A, UΛU^*\rangle$ where $U$ varies over the unitary group $\mathrm{U}(n)$. This problem seeks to approximate $A$ by a matrix whose spectrum is the same as $Λ$ and, by setting $Λ$ to be appropriate diagonal matrices, one can recover matrix approximation problems such as PCA and rank-$k$ approximation. We study the problem of designing differentially private algorithms for this optimization problem in settings where the matrix $A$ is constructed using users' private data. We give efficient and private algorithms that come with upper and lower bounds on the approximation error. Our results unify and improve upon several prior works on private matrix approximation problems. They rely on extensions of packing/covering number bounds for Grassmannians to unitary orbits which should be of independent interest.