OpenRaiser/Intern-Atlas huggingface dataset:
6G Network Business Support System
Venue: arXiv cs.AI 2023
Year: 2023

6G is the next-generation intelligent and integrated digital information infrastructure, characterized by ubiquitous interconnection, native intelligence, multi-dimensional perception, global coverage, green and low-carbon, native network security, etc. 6G will realize the transition from serving people and people-things communication to supporting the efficient connection of intelligent agents, and comprehensively leading the digital, intelligent and green transformation of the economy and the society. As the core support system for mobile communication network, 6 6G BSS need to integrate with new business models brought about by the development of the next-generation Internet and IT, upgrade from "network-centric" to "business and service centric" and "customer-centric". 6G OSS and BSS systems need to strengthen their integration to improve the operational efficiency and benefits of customers by connecting the digital intelligence support capabilities on both sides of supply and demand. This paper provides a detailed introduction to the overall vision, potential key technologies, and functional architecture of 6G BSS systems. It also presents an evolutionary roadmap and technological prospects for the BSS systems from 5G to 6G.
Single-cell RNA-seq data imputation using Feature Propagation
Venue: arXiv cs.CE 2023
Year: 2023

While single-cell RNA sequencing provides an understanding of the transcriptome of individual cells, its high sparsity, often termed dropout, hampers the capture of significant cell-cell relationships. Here, we propose scFP (single-cell Feature Propagation), which directly propagates features, i.e., gene expression, especially in raw feature space, via cell-cell graph. Specifically, it first obtains a warmed-up cell-gene matrix via Hard Feature Propagation which fully utilizes known gene transcripts. Then, we refine the k-Nearest Neighbor(kNN) of the cell-cell graph with a warmed-up cell-gene matrix, followed by Soft Feature Propagation which now allows known gene transcripts to be further denoised through their neighbors. Through extensive experiments on imputation with cell clustering tasks, we demonstrate our proposed model, scFP, outperforms various recent imputation and clustering methods. The source code of scFP can be found at https://github.com/Junseok0207/scFP.
Gradient Sparsification For Masked Fine-Tuning of Transformers
Venue: arXiv cs.CL 2023
Year: 2023

Fine-tuning pretrained self-supervised language models is widely adopted for transfer learning to downstream tasks. Fine-tuning can be achieved by freezing gradients of the pretrained network and only updating gradients of a newly added classification layer, or by performing gradient updates on all parameters. Gradual unfreezing makes a trade-off between the two by gradually unfreezing gradients of whole layers during training. This has been an effective strategy to trade-off between storage and training speed with generalization performance. However, it is not clear whether gradually unfreezing layers throughout training is optimal, compared to sparse variants of gradual unfreezing which may improve fine-tuning performance. In this paper, we propose to stochastically mask gradients to regularize pretrained language models for improving overall fine-tuned performance. We introduce GradDrop and variants thereof, a class of gradient sparsification methods that mask gradients during the backward pass, acting as gradient noise. GradDrop is sparse and stochastic unlike gradual freezing. Extensive experiments on the multilingual XGLUE benchmark with XLMR-Large show that GradDrop is competitive against methods that use additional translated data for intermediate pretraining and outperforms standard fine-tuning and gradual unfreezing. A post-analysis shows how GradDrop improves performance with languages it was not trained on, such as under-resourced languages.
Extended Graph Assessment Metrics for Graph Neural Networks
Venue: arXiv cs.SI 2023
Year: 2023

When re-structuring patient cohorts into so-called population graphs, initially independent data points can be incorporated into one interconnected graph structure. This population graph can then be used for medical downstream tasks using graph neural networks (GNNs). The construction of a suitable graph structure is a challenging step in the learning pipeline that can have severe impact on model performance. To this end, different graph assessment metrics have been introduced to evaluate graph structures. However, these metrics are limited to classification tasks and discrete adjacency matrices, only covering a small subset of real-world applications. In this work, we introduce extended graph assessment metrics (GAMs) for regression tasks and continuous adjacency matrices. We focus on two GAMs in specific: \textit{homophily} and \textit{cross-class neighbourhood similarity} (CCNS). We extend the notion of GAMs to more than one hop, define homophily for regression tasks, as well as continuous adjacency matrices, and propose a light-weight CCNS distance for discrete and continuous adjacency matrices. We show the correlation of these metrics with model performance on different medical population graphs and under different learning settings.
On the work of dynamic constant-time parallel algorithms for regular tree languages and context-free languages
Venue: arXiv cs.DS 2023
Year: 2023

Previous work on Dynamic Complexity has established that there exist dynamic constant-time parallel algorithms for regular tree languages and context-free languages under label or symbol changes. However, these algorithms were not developed with the goal to minimise work (or, equivalently, the number of processors). In fact, their inspection yields the work bounds $O(n^2)$ and $O(n^7)$ per change operation, respectively. In this paper, dynamic algorithms for regular tree languages are proposed that generalise the previous algorithms in that they allow unbounded node rank and leaf insertions, while improving the work bound from $O(n^2)$ to $O(n^ε)$, for arbitrary $ε> 0$. For context-free languages, algorithms with better work bounds (compared with $O(n^7)$) for restricted classes are proposed: for every $ε> 0$ there are such algorithms for deterministic context-free languages with work bound $O(n^{3+ε})$ and for visibly pushdown languages with work bound $O(n^{2+ε})$.
FABRIC: Personalizing Diffusion Models with Iterative Feedback
Venue: arXiv cs.CV 2023
Year: 2023

In an era where visual content generation is increasingly driven by machine learning, the integration of human feedback into generative models presents significant opportunities for enhancing user experience and output quality. This study explores strategies for incorporating iterative human feedback into the generative process of diffusion-based text-to-image models. We propose FABRIC, a training-free approach applicable to a wide range of popular diffusion models, which exploits the self-attention layer present in the most widely used architectures to condition the diffusion process on a set of feedback images. To ensure a rigorous assessment of our approach, we introduce a comprehensive evaluation methodology, offering a robust mechanism to quantify the performance of generative visual models that integrate human feedback. We show that generation results improve over multiple rounds of iterative feedback through exhaustive analysis, implicitly optimizing arbitrary user preferences. The potential applications of these findings extend to fields such as personalized content creation and customization.
RTVis: Research Trend Visualization Toolkit
Venue: arXiv cs.HC 2023
Year: 2023

When researchers are about to start a new project or have just entered a new research field, choosing a proper research topic is always challenging. To help them have an overall understanding of the research trend in real-time and find out the research topic they are interested in, we developed the Research Trend Visualization toolkit (RTVis) to analyze and visualize the research paper information. RTVis consists of a field theme river, a co-occurrence network, a specialized citation bar chart, and a word frequency race diagram, showing the field change through time, cooperating relationship among authors, paper citation numbers in different venues, and the most common words in the abstract part respectively. Moreover, RTVis is open source and easy to deploy. The demo of our toolkit and code with detailed documentation are both available online.
Techno-Utopians, Scammers, and Bullshitters: The Promise and Peril of Web3 and Blockchain Technologies According to Operators and Venture Capital Investors
Venue: arXiv cs.CY 2023
Year: 2023

Proponents and developers of Web3 and blockchain argue that these technologies can revolutionize how people live and work by empowering individuals and distributing decision-making power. While technologists often have expansive hopes for what their technologies will accomplish over the long term, the practical challenges of developing, scaling, and maintaining systems amidst present-day constraints can compromise progress toward this vision. How technologists think about the technological future they hope to enable and how they navigate day-to-day issues impacts the form technologies take, their potential benefits, and their potential harms. In our current work, we aimed to explore the visions of Web3 and blockchain technologists and identify the immediate challenges that could threaten their visions. We conducted semi-structured interviews with 29 operators and professional investors in the Web3 and blockchain field. Our findings revealed that participants supported several ideological goals for their projects, with decentralization being a pivotal mechanism to enable user autonomy, distribute governance power, and promote financial inclusion. However, participants acknowledged the practical difficulties in fulfilling these promises, including the need for rapid technology development, conflicts of interest among stakeholders due to platform financing dynamics, and the challenge of expanding to mainstream users who may not share the "Web3 ethos." If negotiated ineffectively, these challenges could lead to negative outcomes, such as corrupt governance, increased inequality, and increased prevalence of scams and dubious investment schemes. While participants thought education, regulation, and a renewed commitment to the original blockchain ideals could alleviate some problems, they expressed skepticism about the potential of these solutions.
CoNAN: Conditional Neural Aggregation Network For Unconstrained Face Feature Fusion
Venue: arXiv cs.CV 2023
Year: 2023

Face recognition from image sets acquired under unregulated and uncontrolled settings, such as at large distances, low resolutions, varying viewpoints, illumination, pose, and atmospheric conditions, is challenging. Face feature aggregation, which involves aggregating a set of N feature representations present in a template into a single global representation, plays a pivotal role in such recognition systems. Existing works in traditional face feature aggregation either utilize metadata or high-dimensional intermediate feature representations to estimate feature quality for aggregation. However, generating high-quality metadata or style information is not feasible for extremely low-resolution faces captured in long-range and high altitude settings. To overcome these limitations, we propose a feature distribution conditioning approach called CoNAN for template aggregation. Specifically, our method aims to learn a context vector conditioned over the distribution information of the incoming feature set, which is utilized to weigh the features based on their estimated informativeness. The proposed method produces state-of-the-art results on long-range unconstrained face recognition datasets such as BTS, and DroneSURF, validating the advantages of such an aggregation strategy.
Vision-Based Reactive Planning and Control of Quadruped Robots in Unstructured Dynamic Environments
Venue: arXiv cs.RO 2023
Year: 2023

Quadruped robots have received increasing attention for the past few years. However, existing works primarily focus on static environments or assume the robot has full observations of the environment. This limits their practical applications since real-world environments are often dynamic and partially observable. To tackle these issues, vision-based reactive planning and control (V-RPC) is developed in this work. The V-RPC comprises two modules: offline pre-planning and online reactive planning. The pre-planning phase generates a reference trajectory over continuous workspace via sampling-based methods using prior environmental knowledge, given an LTL specification. The online reactive module dynamically adjusts the reference trajectory and control based on the robot's real-time visual perception to adapt to environmental changes.
Evaluating and Enhancing Robustness of Deep Recommendation Systems Against Hardware Errors
Venue: arXiv cs.IR 2023
Year: 2023

Deep recommendation systems (DRS) heavily depend on specialized HPC hardware and accelerators to optimize energy, efficiency, and recommendation quality. Despite the growing number of hardware errors observed in large-scale fleet systems where DRS are deployed, the robustness of DRS has been largely overlooked. This paper presents the first systematic study of DRS robustness against hardware errors. We develop Terrorch, a user-friendly, efficient and flexible error injection framework on top of the widely-used PyTorch. We evaluate a wide range of models and datasets and observe that the DRS robustness against hardware errors is influenced by various factors from model parameters to input characteristics. We also explore 3 error mitigation methods including algorithm based fault tolerance (ABFT), activation clipping and selective bit protection (SBP). We find that applying activation clipping can recover up to 30% of the degraded AUC-ROC score, making it a promising mitigation method.
Dynamic Joint Scheduling of Anycast Transmission and Modulation in Hybrid Unicast-Multicast SWIPT-Based IoT Sensor Networks
Venue: arXiv cs.IT 2023
Year: 2023

The separate receiver architecture with a time- or power-splitting mode, widely used for simultaneous wireless information and power transfer (SWIPT), has a major drawback: Energy-intensive local oscillators and mixers need to be installed in the information decoding (ID) component to downconvert radio frequency (RF) signals to baseband signals, resulting in high energy consumption. As a solution to this challenge, an integrated receiver (IR) architecture has been proposed, and, in turn, various SWIPT modulation schemes compatible with the IR architecture have been developed. However, to the best of our knowledge, no research has been conducted on modulation scheduling in SWIPT-based IoT sensor networks while taking into account the IR architecture. Accordingly, in this paper, we address this research gap by studying the problem of joint scheduling for unicast/multicast, IoT sensor, and modulation (UMSM) in a time-slotted SWIPT-based IoT sensor network system. To this end, we leverage mathematical modeling and optimization techniques, such as the Lagrangian duality and stochastic optimization theory, to develop an UMSM scheduling algorithm that maximizes the weighted sum of average unicast service throughput and harvested energy of IoT sensors, while ensuring the minimum average throughput of both multicast and unicast, as well as the minimum average harvested energy of IoT sensors. Finally, we demonstrate through extensive simulations that our UMSM scheduling algorithm achieves superior energy harvesting (EH) and throughput performance while ensuring the satisfaction of specified constraints well.
Survey on Controlable Image Synthesis with Deep Learning
Venue: arXiv cs.CV 2023
Year: 2023

Image synthesis has attracted emerging research interests in academic and industry communities. Deep learning technologies especially the generative models greatly inspired controllable image synthesis approaches and applications, which aim to generate particular visual contents with latent prompts. In order to further investigate low-level controllable image synthesis problem which is crucial for fine image rendering and editing tasks, we present a survey of some recent works on 3D controllable image synthesis using deep learning. We first introduce the datasets and evaluation indicators for 3D controllable image synthesis. Then, we review the state-of-the-art research for geometrically controllable image synthesis in two aspects: 1) Viewpoint/pose-controllable image synthesis; 2) Structure/shape-controllable image synthesis. Furthermore, the photometrically controllable image synthesis approaches are also reviewed for 3D re-lighting researches. While the emphasis is on 3D controllable image synthesis algorithms, the related applications, products and resources are also briefly summarized for practitioners.
Semi-supervised Cycle-GAN for face photo-sketch translation in the wild
Venue: arXiv cs.CV 2023
Year: 2023

The performance of face photo-sketch translation has improved a lot thanks to deep neural networks. GAN based methods trained on paired images can produce high-quality results under laboratory settings. Such paired datasets are, however, often very small and lack diversity. Meanwhile, Cycle-GANs trained with unpaired photo-sketch datasets suffer from the \emph{steganography} phenomenon, which makes them not effective to face photos in the wild. In this paper, we introduce a semi-supervised approach with a noise-injection strategy, named Semi-Cycle-GAN (SCG), to tackle these problems. For the first problem, we propose a {\em pseudo sketch feature} representation for each input photo composed from a small reference set of photo-sketch pairs, and use the resulting {\em pseudo pairs} to supervise a photo-to-sketch generator $G_{p2s}$. The outputs of $G_{p2s}$ can in turn help to train a sketch-to-photo generator $G_{s2p}$ in a self-supervised manner. This allows us to train $G_{p2s}$ and $G_{s2p}$ using a small reference set of photo-sketch pairs together with a large face photo dataset (without ground-truth sketches). For the second problem, we show that the simple noise-injection strategy works well to alleviate the \emph{steganography} effect in SCG and helps to produce more reasonable sketch-to-photo results with less overfitting than fully supervised approaches. Experiments show that SCG achieves competitive performance on public benchmarks and superior results on photos in the wild.
SecureTrack- A contact tracing IoT platform for monitoring infectious diseases
Venue: arXiv cs.CR 2023
Year: 2023

The COVID-19 pandemic has highlighted the need for innovative solutions to monitor and control the spread of infectious diseases. With the potential for future pandemics and the risk of outbreaks particularly in academic institutions, there is a pressing need for effective approaches to monitor and manage such diseases. Contact tracing using Global Positioning Systems (GPS) has been found to be the most prevalent method to detect and tackle the extent of outbreaks during the pandemic. However, these services suffer from the inherent problems of infringement of data privacy that creates hindrance in adoption of the technology. Non-cellular wireless technologies on the other hand are well-suited to provide secure contact tracing methods. Such approaches integrated with the Internet of Things (IoT) have a great potential to aid in the fight against any type of infectious diseases. In response, we present a unique approach that utilizes an IoT based generic framework to identify individuals who may have been exposed to the virus, using contact tracing methods, without compromising the privacy aspect. We develop the architecture of our platform, including both the frontend and backend components, and demonstrate its effectiveness in identifying potential COVID-19 exposures (as a test case) through a proof-of-concept implementation. We also implement and verify a prototype of the device. Our framework is easily deployable and can be scaled up as needed with the existing infrastructure.
Post-pandemic mobility patterns in London
Venue: arXiv cs.SI 2023
Year: 2023

Understanding human mobility is crucial for urban and transport studies in cities. People's daily activities provide valuable insight, such as where people live, work, shop, leisure or eat during midday or after-work hours. However, such activities are changed due to travel behaviours after COVID-19 in cities. This study examines the mobility patterns captured from mobile phone apps to explore the behavioural patterns established since the COVID-19 lockdowns triggered a series of changes in urban environments.
DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation
Venue: arXiv cs.LG 2023
Year: 2023

The generation of synthetic tabular data that preserves differential privacy is a problem of growing importance. While traditional marginal-based methods have achieved impressive results, recent work has shown that deep learning-based approaches tend to lag behind. In this work, we present Differentially-Private TaBular AutoRegressive Transformer (DP-TBART), a transformer-based autoregressive model that maintains differential privacy and achieves performance competitive with marginal-based methods on a wide variety of datasets, capable of even outperforming state-of-the-art methods in certain settings. We also provide a theoretical framework for understanding the limitations of marginal-based approaches and where deep learning-based approaches stand to contribute most. These results suggest that deep learning-based techniques should be considered as a viable alternative to marginal-based methods in the generation of differentially private synthetic tabular data.
An Analysis of Bugs In Persistent Memory Application
Venue: arXiv cs.SE 2023
Year: 2023

Over the years of challenges on detecting the crash consistency of non-volatile persistent memory (PM) bugs and developing new tools to identify those bugs are quite stretching due to its inconsistent behavior on the file or storage systems. In this paper, we evaluated an open-sourced automatic bug detector tool (i.e. AGAMOTTO) to test NVM level hashing PM application to identify performance and correctness PM bugs in the persistent (main) memory. Furthermore, our faithful validation tool able to discovered 65 new NVM level hashing bugs on PMDK library and it outperformed the number of bugs (i.e. 40 bugs) that WITCHER framework was able to identified. Finally, we will propose a Deep-Q Learning search heuristic algorithm over the PM-Aware search algorithm in the state selection process to improve the searching strategy efficiently.
Novel Batch Active Learning Approach and Its Application to Synthetic Aperture Radar Datasets
Venue: arXiv cs.LG 2023
Year: 2023

Active learning improves the performance of machine learning methods by judiciously selecting a limited number of unlabeled data points to query for labels, with the aim of maximally improving the underlying classifier's performance. Recent gains have been made using sequential active learning for synthetic aperture radar (SAR) data arXiv:2204.00005. In each iteration, sequential active learning selects a query set of size one while batch active learning selects a query set of multiple datapoints. While batch active learning methods exhibit greater efficiency, the challenge lies in maintaining model accuracy relative to sequential active learning methods. We developed a novel, two-part approach for batch active learning: Dijkstra's Annulus Core-Set (DAC) for core-set generation and LocalMax for batch sampling. The batch active learning process that combines DAC and LocalMax achieves nearly identical accuracy as sequential active learning but is more efficient, proportional to the batch size. As an application, a pipeline is built based on transfer learning feature embedding, graph learning, DAC, and LocalMax to classify the FUSAR-Ship and OpenSARShip datasets. Our pipeline outperforms the state-of-the-art CNN-based methods.