OpenRaiser/Intern-Atlas huggingface dataset:
Developing a Framework for Heterotopias as Discursive Playgrounds: A Comparative Analysis of Non-Immersive and Immersive Technologies
Venue: arXiv cs.HC 2023
Year: 2023

The discursive space represents the reordering of knowledge gained through accumulation. In the digital age, multimedia has become the language of information, and the space for archival practices is provided by non-immersive technologies, resulting in the disappearance of several layers from discursive activities. Heterotopias are unique, multilayered epistemic contexts that connect other systems through the exchange of information. This paper describes a process to create a framework for Virtual Reality, Mixed Reality, and personal computer environments based on heterotopias to provide absent layers. This study provides virtual museum space as an informational terrain that contains a "world within worlds" and presents place production as a layer of heterotopia and the subject of discourse. Automation for the individual multimedia content is provided via various sorting and grouping algorithms, and procedural content generation algorithms such as Binary Space Partitioning, Cellular Automata, Growth Algorithm, and Procedural Room Generation. Versions of the framework were comparatively evaluated through a user study involving 30 participants, considering factors such as usability, technology acceptance, and presence. The results of the study show that the framework can serve diverse contexts to construct multilayered digital habitats and is flexible for integration into professional and daily life practices.
The Evolution of Web Search User Interfaces -- An Archaeological Analysis of Google Search Engine Result Pages
Venue: arXiv cs.IR 2023
Year: 2023

Web search engines have marked everyone's life by transforming how one searches and accesses information. Search engines give special attention to the user interface, especially search engine result pages (SERP). The well-known ''10 blue links'' list has evolved into richer interfaces, often personalized to the search query, the user, and other aspects. More than 20 years later, the literature has not adequately portrayed this development. We present a study on the evolution of SERP interfaces during the last two decades using Google Search as a case study. We used the most searched queries by year to extract a sample of SERP from the Internet Archive. Using this dataset, we analyzed how SERP evolved in content, layout, design (e.g., color scheme, text styling, graphics), navigation, and file size. We have also analyzed the user interface design patterns associated with SERP elements. We found that SERP are becoming more diverse in terms of elements, aggregating content from different verticals and including more features that provide direct answers. This systematic analysis portrays evolution trends in search engine user interfaces and, more generally, web design. We expect this work will trigger other, more specific studies that can take advantage of our dataset.
Solving PDEs with Unmeasurable Source Terms Using Coupled Physics-Informed Neural Network with Recurrent Prediction for Soft Sensors
Venue: arXiv cs.LG 2023
Year: 2023

Partial differential equations (PDEs) are a model candidate for soft sensors in industrial processes with spatiotemporal dependence. Although physics-informed neural networks (PINNs) are a promising machine learning method for solving PDEs, they are infeasible for the nonhomogeneous PDEs with unmeasurable source terms. To this end, a coupled PINN (CPINN) with a recurrent prediction (RP) learning strategy (CPINN- RP) is proposed. First, CPINN composed of NetU and NetG is proposed. NetU is for approximating PDEs solutions and NetG is for regularizing the training of NetU. The two networks are integrated into a data-physics-hybrid loss function. Then, we theoretically prove that the proposed CPINN has a satisfying approximation capability for solutions to nonhomogeneous PDEs with unmeasurable source terms. Besides the theoretical aspects, we propose a hierarchical training strategy to optimize and couple NetU and NetG. Secondly, NetU-RP is proposed for compensating information loss in data sampling to improve the prediction performance, in which RP is the recurrently delayed outputs of well-trained CPINN and hard sensors. Finally, the artificial and practical datasets are used to verify the feasibility and effectiveness of CPINN-RP for soft sensors.
Optimal multiple FSO transceiver configuration for using on High-altitude platforms
Venue: arXiv cs.NI 2023
Year: 2023

Free-space optical (FSO) communication requires light of sight (LoS) between the transmitter and the receiver. For long-distance communication, many research projects have been conducted towards using a network composed of high-altitude platforms (HAPs) flying at an elevation of 20 km to carry intermediate FSO transceivers that forward data between ground stations. The clear environment at high elevations prevents terrestrial obstacles from cutting the LoS between the transceivers. An FSO transceiver on a HAP can communicate with ground stations within a small area owing to its limited beam size. We suggest using multiple FSO transceivers on a HAP to extend its ground coverage. However, the use of too many FSO transceivers may quickly exhaust the onboard energy of the HAP. As a result, HAP must be lowered to recharge frequently. In this study, we first propose a configuration of multiple FSO transceivers to widen the ground coverage of a HAP. We then propose a set of closed-form expressions to calculate the extended coverage. Finally, to implement a HAP network using multiple FSO transceivers, we seek the optimal configuration of multiple FSO transceivers that minimizes the total cost of the HAP network, including amortization, energy, and maintenance costs. The simulation results show that the proposed multiple FSO transceiver configuration clearly increases the ground coverage of a HAP and significantly reduces the cost of the HAP network.
Key-and-Signature Compact Multi-Signatures for Blockchain: A Compiler with Realizations
Venue: arXiv cs.CR 2023
Year: 2023

Multi-signature is a protocol where a set of signatures jointly sign a message so that the final signature is significantly shorter than concatenating individual signatures together. Recently, it finds applications in blockchain, where several users want to jointly authorize a payment through a multi-signature. However, in this setting, there is no centralized authority and it could suffer from a rogue key attack where the attacker can generate his own keys arbitrarily. Further, to minimize the storage on blockchain, it is desired that the aggregated public-key and the aggregated signature are both as short as possible. In this paper, we find a compiler that converts a kind of identification (ID) scheme (which we call a linear ID) to a multi-signature so that both the aggregated public-key and the aggregated signature have a size independent of the number of signers. Our compiler is provably secure. The advantage of our results is that we reduce a multi-party problem to a weakly secure two-party problem. We realize our compiler with two ID schemes. The first is Schnorr ID. The second is a new lattice-based ID scheme, which via our compiler gives the first regular lattice-based multi-signature scheme with key-and-signature compact without a restart during signing process.
Holistically Explainable Vision Transformers
Venue: arXiv cs.CV 2023
Year: 2023

Transformers increasingly dominate the machine learning landscape across many tasks and domains, which increases the importance for understanding their outputs. While their attention modules provide partial insight into their inner workings, the attention scores have been shown to be insufficient for explaining the models as a whole. To address this, we propose B-cos transformers, which inherently provide holistic explanations for their decisions. Specifically, we formulate each model component - such as the multi-layer perceptrons, attention layers, and the tokenisation module - to be dynamic linear, which allows us to faithfully summarise the entire transformer via a single linear transform. We apply our proposed design to Vision Transformers (ViTs) and show that the resulting models, dubbed Bcos-ViTs, are highly interpretable and perform competitively to baseline ViTs on ImageNet. Code will be made available soon.
Blind Spots: Automatically detecting ignored program inputs
Venue: arXiv cs.CR 2023
Year: 2023

A blind spot is any input to a program that can be arbitrarily mutated without affecting the program's output. Blind spots can be used for steganography or to embed malware payloads. If blind spots overlap file format keywords, they indicate parsing bugs that can lead to exploitable differentials. For example, one could craft a document that renders one way in one viewer and a completely different way in another viewer. They have also been used to circumvent code signing in Android binaries, to coerce certificate authorities to misbehave, and to execute HTTP request smuggling and parameter pollution attacks. This paper formalizes the operational semantics of blind spots, leading to a technique based on dynamic information flow tracking that automatically detects blind spots. An efficient implementation is introduced and evaluated against a corpus of over a thousand diverse PDFs parsed through MuPDF, revealing exploitable bugs in the parser. All of the blind spot classifications are confirmed to be correct and the missed detection rate is no higher than 11%. On average, at least 5% of each PDF file is completely ignored by the parser. Our results show promise that this technique is an efficient automated means to detect exploitable parser bugs, over-permissiveness and differentials. Nothing in the technique is tied to PDF in general, so it can be immediately applied to other notoriously difficult-to-parse formats like ELF, X.509, and XML.
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
Venue: arXiv cs.CL 2023
Year: 2023

This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the prompts advised by ChatGPT to trigger its translation ability and find that the candidate prompts generally work well with minor performance differences. By evaluating on a number of benchmark test sets, we find that ChatGPT performs competitively with commercial translation products (e.g., Google Translate) on high-resource European languages but lags behind significantly on low-resource or distant languages. As for the translation robustness, ChatGPT does not perform as well as the commercial systems on biomedical abstracts or Reddit comments but exhibits good results on spoken language. Further, we explore an interesting strategy named $\mathbf{pivot~prompting}$ for distant languages, which asks ChatGPT to translate the source sentence into a high-resource pivot language before into the target language, improving the translation performance noticeably. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly boosted, becoming comparable to commercial translation products, even for distant languages. Human analysis on Google Translate and ChatGPT suggests that ChatGPT with GPT-3.5 tends to generate more hallucinations and mis-translation errors while that with GPT-4 makes the least errors. In other words, ChatGPT has already become a good translator. Please refer to our Github project for more details: https://github.com/wxjiao/Is-ChatGPT-A-Good-Translator
Inherent Limits on Topology-Based Link Prediction
Venue: arXiv cs.SI 2023
Year: 2023

Link prediction systems (e.g. recommender systems) typically use graph topology as one of their main sources of information. However, automorphisms and related properties of graphs beget inherent limits in predictability. We calculate hard upper bounds on how well graph topology alone enables link prediction for a wide variety of real-world graphs. We find that in the sparsest of these graphs the upper bounds are surprisingly low, thereby demonstrating that prediction systems on sparse graph data are inherently limited and require information in addition to the graph topology.
Robot Skill Learning Via Classical Robotics-Based Generated Datasets: Advantages, Disadvantages, and Future Improvement
Venue: arXiv cs.RO 2023
Year: 2023

Why do we not profit from our long-existing classical robotics knowledge and look for some alternative way for data collection? The situation ignoring all existing methods might be such a waste. This article argues that a dataset created using a classical robotics algorithm is a crucial part of future development. This developed classic algorithm has a perfect domain adaptation and generalization property, and most importantly, collecting datasets based on them is quite easy. It is well known that current robot skill-learning approaches perform exceptionally badly in the unseen domain, and their performance against adversarial attacks is quite limited as long as they do not have a very exclusive big dataset. Our experiment is the initial steps of using a dataset created by classical robotics codes. Our experiment investigated possible trajectory collection based on classical robotics. It addressed some advantages and disadvantages and pointed out other future development ideas.
Towards Flexibility and Interpretability of Gaussian Process State-Space Model
Venue: arXiv cs.LG 2023
Year: 2023

The Gaussian process state-space model (GPSSM) has garnered considerable attention over the past decade. However, the standard GP with a preliminary kernel, such as the squared exponential kernel or Matérn kernel, that is commonly used in GPSSM studies, limits the model's representation power and substantially restricts its applicability to complex scenarios. To address this issue, we propose a new class of probabilistic state-space models called TGPSSMs, which leverage a parametric normalizing flow to enrich the GP priors in the standard GPSSM, enabling greater flexibility and expressivity. Additionally, we present a scalable variational inference algorithm that offers a flexible and optimal structure for the variational distribution of latent states. The proposed algorithm is interpretable and computationally efficient due to the sparse GP representation and the bijective nature of normalizing flow. Moreover, we incorporate a constrained optimization framework into the algorithm to enhance the state-space representation capabilities and optimize the hyperparameters, leading to superior learning and inference performance. Experimental results on synthetic and real datasets corroborate that the proposed TGPSSM outperforms several state-of-the-art methods. The accompanying source code is available at \url{https://github.com/zhidilin/TGPSSM}.
CADA-GAN: Context-Aware GAN with Data Augmentation
Venue: arXiv cs.CV 2023
Year: 2023

Current child face generators are restricted by the limited size of the available datasets. In addition, feature selection can prove to be a significant challenge, especially due to the large amount of features that need to be trained for. To manage these problems, we proposed CADA-GAN, a \textbf{C}ontext-\textbf{A}ware GAN that allows optimal feature extraction, with added robustness from additional \textbf{D}ata \textbf{A}ugmentation. CADA-GAN is adapted from the popular StyleGAN2-Ada model, with attention on augmentation and segmentation of the parent images. The model has the lowest \textit{Mean Squared Error Loss} (MSEloss) on latent feature representations and the generated child image is robust compared with the one that generated from baseline models.
Improving Zero-Shot Action Recognition using Human Instruction with Text Description
Venue: arXiv cs.CV 2023
Year: 2023

Zero-shot action recognition, which recognizes actions in videos without having received any training examples, is gaining wide attention considering it can save labor costs and training time. Nevertheless, the performance of zero-shot learning is still unsatisfactory, which limits its practical application. To solve this problem, this study proposes a framework to improve zero-shot action recognition using human instructions with text descriptions. The proposed framework manually describes video contents, which incurs some labor costs; in many situations, the labor costs are worth it. We manually annotate text features for each action, which can be a word, phrase, or sentence. Then by computing the matching degrees between the video and all text features, we can predict the class of the video. Furthermore, the proposed model can also be combined with other models to improve its accuracy. In addition, our model can be continuously optimized to improve the accuracy by repeating human instructions. The results with UCF101 and HMDB51 showed that our model achieved the best accuracy and improved the accuracies of other models.
Submodular Hypergraph Partitioning: Metric Relaxations and Fast Algorithms via an Improved Cut-Matching Game
Venue: arXiv cs.DS 2023
Year: 2023

Despite there being significant work on developing spectral, and metric embedding based approximation algorithms for hypergraph generalizations of conductance, little is known regarding the approximability of hypergraph partitioning objectives beyond this. This work proposes algorithms for a general model of hypergraph partitioning that unifies both undirected and directed versions of many well-studied partitioning objectives. The first contribution of this paper introduces polymatroidal cut functions, a large class of cut functions amenable to approximation algorithms via metric embeddings and routing multicommodity flows. We demonstrate an $O(\sqrt{\log n})$-approximation, where $n$ is the number of vertices in the hypergraph, for these problems by rounding relaxations to metrics of negative-type. The second contribution of this paper generalizes the cut-matching game framework of Khandekar et. al. to tackle polymatroidal cut functions. This yields the first almost-linear time $O(\log n)$-approximation algorithm for standard versions of undirected and directed hypergraph partitioning. A technical consequence of our construction is that a cut-matching game which greatly relaxes the set of allowed actions for both players can be used to partition hypergraphs with negligible impact on the approximation ratio. We believe this to be of independent interest.
Counterfactual Explanation and Instance-Generation using Cycle-Consistent Generative Adversarial Networks
Venue: arXiv cs.CV 2023
Year: 2023

The image-based diagnosis is now a vital aspect of modern automation assisted diagnosis. To enable models to produce pixel-level diagnosis, pixel-level ground-truth labels are essentially required. However, since it is often not straight forward to obtain the labels in many application domains such as in medical image, classification-based approaches have become the de facto standard to perform the diagnosis. Though they can identify class-salient regions, they may not be useful for diagnosis where capturing all of the evidences is important requirement. Alternatively, a counterfactual explanation (CX) aims at providing explanations using a casual reasoning process of form "If X has not happend, Y would not heppend". Existing CX approaches, however, use classifier to explain features that can change its predictions. Thus, they can only explain class-salient features, rather than entire object of interest. This hence motivates us to propose a novel CX strategy that is not reliant on image classification. This work is inspired from the recent developments in generative adversarial networks (GANs) based image-to-image domain translation, and leverages to translate an abnormal image to counterpart normal image (i.e. counterfactual instance CI) to find discrepancy maps between the two. Since it is generally not possible to obtain abnormal and normal image pairs, we leverage Cycle-Consistency principle (a.k.a CycleGAN) to perform the translation in unsupervised way. We formulate CX in terms of a discrepancy map that, when added from the abnormal image, will make it indistinguishable from the CI. We evaluate our method on three datasets including a synthetic, tuberculosis and BraTS dataset. All these experiments confirm the supremacy of propose method in generating accurate CX and CI.
On the Algebraic Properties of Flame Graphs
Venue: arXiv cs.SE 2023
Year: 2023

Flame graphs are a popular way of representing profiling data. In this paper we propose a possible mathematical definition of flame graphs. In doing so, we gain some interesting algebraic properties almost for free, which in turn allow us to define some operations that can allow to perform an in-depth performance regression analysis. The typical documented use of a flame graph is via its graphical representation, whereby one scans the picture for the largest plateaux. Whilst this method is effective at finding the main sources of performance issues, it leaves quite a large amount of data potentially unused. By combining a mathematical precise definition of flame graphs with some statistical methods we show how to generalise this visual procedure and make the best of the full set of collected profiling data.
Time-Conditioned Generative Modeling of Object-Centric Representations for Video Decomposition and Prediction
Venue: arXiv cs.CV 2023
Year: 2023

When perceiving the world from multiple viewpoints, humans have the ability to reason about the complete objects in a compositional manner even when an object is completely occluded from certain viewpoints. Meanwhile, humans are able to imagine novel views after observing multiple viewpoints. Recent remarkable advances in multi-view object-centric learning still leaves some unresolved problems: 1) The shapes of partially or completely occluded objects can not be well reconstructed. 2) The novel viewpoint prediction depends on expensive viewpoint annotations rather than implicit rules in view representations. In this paper, we introduce a time-conditioned generative model for videos. To reconstruct the complete shape of an object accurately, we enhance the disentanglement between the latent representations of objects and views, where the latent representations of time-conditioned views are jointly inferred with a Transformer and then are input to a sequential extension of Slot Attention to learn object-centric representations. In addition, Gaussian processes are employed as priors of view latent variables for video generation and novel-view prediction without viewpoint annotations. Experiments on multiple datasets demonstrate that the proposed model can make object-centric video decomposition, reconstruct the complete shapes of occluded objects, and make novel-view predictions.
Soft Sensing Regression Model: from Sensor to Wafer Metrology Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

The semiconductor industry is one of the most technology-evolving and capital-intensive market sectors. Effective inspection and metrology are necessary to improve product yield, increase product quality and reduce costs. In recent years, many semiconductor manufacturing equipments are equipped with sensors to facilitate real-time monitoring of the production process. These production-state and equipment-state sensor data provide an opportunity to practice machine-learning technologies in various domains, such as anomaly/fault detection, maintenance scheduling, quality prediction, etc. In this work, we focus on the task of soft sensing regression, which uses sensor data to predict impending inspection measurements that used to be measured in wafer inspection and metrology systems. We proposed an LSTM-based regressor and designed two loss functions for model training. Although engineers may look at our prediction errors in a subjective manner, a new piece-wise evaluation metric was proposed for assessing model accuracy in a mathematical way. The experimental results demonstrated that the proposed model can achieve accurate and early prediction of various types of inspections in complicated manufacturing processes.