Miracle-dz/UniFAR-FTUs huggingface dataset:
A BenchCouncil View on Benchmarking Emerging and Future Computing

The measurable properties of the artifacts or objects in the computer, management, or ﬁnance disci- plines are extrinsic, not inherent — dependent on their problem deﬁnitions and solution instantiations. Only after the instantiation can the solutions to the problem be measured. The processes of deﬁnition, instantiation, and measurement are entangled, and they have complex mutual inﬂuences. Meanwhile, the technology inertia brings instantiation bias — trapped into a subspace or even a point at a high- dimension solution space. These daunting challenges, which emerging computing aggravates, make metrology can not work for benchmark communities. It is pressing to establish independent bench- mark science and engineering. This article presents a unifying benchmark deﬁnition, a conceptual framework, and a traceable and supervised learning-based benchmarking methodology, laying the foundation for benchmark science and engineering. I also discuss BenchCouncil’s plans for emerging and future computing. The ongoing projects include deﬁning the challenges of intelligence, instinct, quantum computers, Metaverse, planet-scale computers, and reformulating data centers, artiﬁcial intelligence for science, and CPU benchmark suites. Also, BenchCouncil will collaborate with ComputerCouncil on open-source computer systems for planet-scale computing, AI for science systems, and Metaverse.
Automation Slicing and Testing for in-App Deep Learning Models

Intelligent Apps (iApps), equipped with in-App deep learning (DL) models, are emerging to offer stable DL inference services. However, App marketplaces have trouble auto testing iApps because the in-App model is black-box and couples with ordinary codes. In this work, we propose an automated tool, ASTM, which can enable large-scale testing of in-App models. ASTM takes as input an iApps, and the outputs can replace the in-App model as the test object. ASTM proposes two reconstruction techniques to translate the in-App model to a backpropagation-enabled version and reconstruct the IO processing code for DL inference. With the ASTM's help, we perform a large-scale study on the robustness of 100 unique commercial in-App models and find that 56\% of in-App models are vulnerable to robustness issues in our context. ASTM also detects physical attacks against three representative iApps that may cause economic losses and security issues.
IRB-NLP at SemEval-2022 Task 1: Exploring the Relationship Between Words and Their Semantic Representations

What is the relation between a word and its description, or a word and its embedding? Both descriptions and embeddings are semantic representations of words. But, what information from the original word remains in these representations? Or more importantly, which information about a word do these two representations share? Definition Modeling and Reverse Dictionary are two opposite learning tasks that address these questions. The goal of the Definition Modeling task is to investigate the power of information laying inside a word embedding to express the meaning of the word in a humanly understandable way – as a dictionary definition. Conversely, the Reverse Dictionary task explores the ability to predict word embeddings directly from its definition. In this paper, by tackling these two tasks, we are exploring the relationship between words and their semantic representations. We present our findings based on the descriptive, exploratory, and predictive data analysis conducted on the CODWOE dataset. We give a detailed overview of the systems that we designed for Definition Modeling and Reverse Dictionary tasks, and that achieved top scores on SemEval-2022 CODWOE challenge in several subtasks. We hope that our experimental results concerning the predictive models and the data analyses we provide will prove useful in future explorations of word representations and their relationships.
Factors influencing the energy gap in topological states of antiferromagnetic MnBi$_2$Te$_4$

The experimentally measured angle-resolved photoemission dispersion maps for MnBi 2 Te 4 samples, which show diﬀerent energy gaps at the Dirac point (DP), are compared with the results of theoretical calculations to ﬁnd the conditions for the best agreement between theory and experi-ment. We have analyzed diﬀerent factors which inﬂuence the Dirac gap width: (i) the surface van der Waals (SvdW) distance between the ﬁrst and second septuple layers (SLs), (ii) the magnetic moment on Mn atoms, (iii) the spin-orbit coupling (SOC) strength for the surface Te and Bi atoms and related changes in the localization of the topological surface states (TSSs). It was shown that all these factors may change the gap width at the DP in a wide range from 5 to ∼ 90 meV. We show that the Dirac gap variation is mainly determined by the corresponding changes in the TSSs spatial distribution. The best agreement between the presented experimental data (with the Dirac gaps between ∼ 15 and 55 meV) and the calculations takes place for a slightly compressed SvdW interval (of about -3.5 % compared to the bulk value) with modiﬁed SOC for surface atoms (that can occur in the presence of various defects in the near-surface region). We show that upon changing the values of the SvdW interval and surface SOC strength the TSSs spatial distribution shifts between the SLs with opposite magnetizations, which leads to a non-monotonic change in the Dirac gap size.
Network embeddedness indicates the innovation potential of firms

Firms' innovation potential depends on their position in the R&D network. But details on this relation remain unclear because measures to quantify network embeddedness have been controversially discussed. We propose and validate a new measure, coreness, obtained from the weighted k-core decomposition of the R&D network. Using data on R&D alliances, we analyse the change of coreness for 14,000 firms over 25 years and patenting activity. A regression analysis demonstrates that coreness explains firms' R&D output by predicting future patenting.
DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

In this paper, we present DendroMap, a novel approach to interactively exploring large-scale image datasets for machine learning (ML). ML practitioners often explore image datasets by generating a grid of images or projecting high-dimensional representations of images into 2-D using dimensionality reduction techniques (e.g., t-SNE). However, neither approach effectively scales to large datasets because images are ineffectively organized and interactions are insufficiently supported. To address these challenges, we develop DendroMap by adapting Treemaps, a well-known visualization technique. DendroMap effectively organizes images by extracting hierarchical cluster structures from high-dimensional representations of images. It enables users to make sense of the overall distributions of datasets and interactively zoom into specific areas of interests at multiple levels of abstraction. Our case studies with widely-used image datasets for deep learning demonstrate that users can discover insights about datasets and trained models by examining the diversity of images, identifying underperforming subgroups, and analyzing classification errors. We conducted a user study that evaluates the effectiveness of DendroMap in grouping and searching tasks by comparing it with a gridified version of t-SNE and found that participants preferred DendroMap. DendroMap is available at https://div-lab.github. io/dendromap/.
MM-BD: Post-Training Detection of Backdoor Attacks with Arbitrary Backdoor Pattern Types Using a Maximum Margin Statistic

Backdoor attacks are an important type of adversarial threat against deep neural network classifiers, wherein test samples from one or more source classes will be (mis)classified to the attacker's target class when a backdoor pattern is embedded. In this paper, we focus on the post-training backdoor defense scenario commonly considered in the literature, where the defender aims to detect whether a trained classifier was backdoor-attacked without any access to the training set. Many post-training detectors are designed to detect attacks that use either one or a few specific backdoor embedding functions (e.g., patch-replacement or additive attacks). These detectors may fail when the backdoor embedding function used by the attacker (unknown to the defender) is different from the backdoor embedding function assumed by the defender. In contrast, we propose a post-training defense that detects backdoor attacks with arbitrary types of backdoor embeddings, without making any assumptions about the backdoor embedding type. Our detector leverages the influence of the backdoor attack, independent of the backdoor embedding mechanism, on the landscape of the classifier's outputs prior to the softmax layer. For each class, a maximum margin statistic is estimated. Detection inference is then performed by applying an unsupervised anomaly detector to these statistics. Thus, our detector does not need any legitimate clean samples, and can efficiently detect backdoor attacks with arbitrary numbers of source classes. These advantages over several state-of-the-art methods are demonstrated on four datasets, for three different types of backdoor patterns, and for a variety of attack configurations. Finally, we propose a novel, general approach for backdoor mitigation once a detection is made. The mitigation approach was the runner-up at the first IEEE Trojan Removal Competition. The code is online available.
Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction

Inspired by the great success of deep neural networks, learning-based methods have gained promising performances for metal artifact reduction (MAR) in computed tomography (CT) images. However, most of the existing approaches put less emphasis on modelling and embedding the intrinsic prior knowledge underlying this specific MAR task into their network designs. Against this issue, we propose an adaptive convolutional dictionary network (ACDNet), which leverages both model-based and learning-based methods. Specifically, we explore the prior structures of metal artifacts, e.g., non-local repetitive streaking patterns, and encode them as an explicit weighted convolutional dictionary model. Then, a simple-yet-effective algorithm is carefully designed to solve the model. By unfolding every iterative substep of the proposed algorithm into a network module, we explicitly embed the prior structure into a deep network , i.e., a clear interpretability for the MAR task. Furthermore, our ACDNet can automatically learn the prior for artifact-free CT images via training data and adaptively adjust the representation kernels for each input CT image based on its content. Hence, our method inherits the clear interpretability of model-based methods and maintains the powerful representation ability of learning-based methods. Comprehensive experiments executed on synthetic and clinical datasets show the superiority of our ACDNet in terms of effectiveness and model generalization. Code and supplementary material are available at https://github.com/hongwang01/ACDNet.
An automatic pipeline for atlas-based fetal and neonatal brain segmentation and analysis

BACKGROUND AND OBJECTIVE The automatic segmentation of perinatal brain structures in magnetic resonance imaging (MRI) is of utmost importance for the study of brain growth and related complications. While different methods exist for adult and pediatric MRI data, there is a lack for automatic tools for the analysis of perinatal imaging. METHODS In this work, a new pipeline for fetal and neonatal segmentation has been developed. We also report the creation of two new fetal atlases, and their use within the pipeline for atlas-based segmentation, based on novel registration methods. The pipeline is also able to extract cortical and pial surfaces and compute features, such as curvature, local gyrification index, sulcal depth, and thickness. RESULTS Results show that the introduction of the new templates together with our segmentation strategy leads to accurate results when compared to expert annotations, as well as better performances when compared to a reference pipeline (developing Human Connectome Project (dHCP)), for both early and late-onset fetal brains. CONCLUSIONS These findings show the potential of the presented atlases and the whole pipeline for application in both fetal, neonatal, and longitudinal studies, which could lead to dramatic improvements in the understanding of perinatal brain development.
Gaussian Processes and Bayesian Optimization for High Precision Experiments

High-precision measurements require optimal setups and analysis tools to achieve continuous im-provements. Systematic corrections need to be modeled with high accuracy and known uncertainty to reconstruct underlying physical phenomena. To this end, we present Gaussian processes for modeling experiments and usage with Bayesian optimization, on the example of an electron energy detector, achieving optimal performance. We demonstrate the method’s strengths and outline stochastic variational Gaussian processes for physics applications with large data sets, enabling new solutions for current problems.
Optimal Parameter-free Online Learning with Switching Cost

Parameter-freeness in online learning refers to the adaptivity of an algorithm with respect to the optimal decision in hindsight. In this paper, we design such algorithms in the presence of switching cost - the latter penalizes the optimistic updates required by parameter-freeness, leading to a delicate design trade-oﬀ. Based on a novel dual space scaling strategy, we propose a simple yet powerful algorithm for Online Linear Optimization (OLO) with switching cost , which improves the existing suboptimal regret bound [ZCP22a] to the optimal rate. The obtained beneﬁt is extended to the expert setting, and the practicality of our algorithm is demonstrated through a sequential investment task.
Realistic Defocus Blur for Multiplane Computer-Generated Holography

This paper introduces a new multiplane CGH computation method to reconstruct artifact-free high-quality holograms with natural-looking defocus blur. Our method introduces a new targeting scheme and a new loss function. While the targeting scheme accounts for defocused parts of the scene at each depth plane, the new loss function analyzes focused and defocused parts separately in reconstructed images. Our method support phase-only CGH calculations using various iterative (e.g., Gerchberg-Saxton, Gradient Descent) and non-iterative (e.g., Double Phase) CGH techniques. We achieve our best image quality using a modified gradient descent-based optimization recipe where we introduce a constraint inspired by the double phase method. We validate our method experimentally using our proof-of-concept holographic display, comparing various algorithms, including multi-depth scenes with sparse and dense contents.
Fast Conformal Parameterization of Disks and Sphere Sectors

We prove a novel method for the embedding of a 3-fold rotationally symmetric sphere-type mesh onto a subset of the plane with 3-fold rotational symmetry. The embedding is free-boundary with the only additional constraint on the image set is that its translations tile the plane, in turn this forces the angles at the embedding of the branch points in the construction. These parameterizations are optimal with respect to the Dirichlet energy functional defined on simplicial complexes. Since the parameterization is over a fixed area domain, it is conformal (i.e. a minimizer of the LSCM energy). The embedding is done by a novel construction of a torus from 63 copies of the original sphere. As a foundation for this result we first prove the optimality of the embedding of disk-type meshes onto special types of triangles in the plane, and rectangles. The embedding of the 3-fold symmetric torus is full rank and so cannot be reduced by simpler constructions. 3-fold symmetric surfaces appear in nature, for example the surface of the 3-fold symmetric proteins PIEZO1 and PIEZO2 which are an important target of current studies.
Voxel-wise Adversarial Semi-supervised Learning for Medical Image Segmentation

BACKGROUND AND OBJECTIVE Semi-supervised learning for medical image segmentation is an important area of research for alleviating the huge cost associated with the construction of reliable large-scale annotations in the medical domain. Recent semi-supervised approaches have demonstrated promising results by employing consistency regularization, pseudo-labeling techniques, and adversarial learning. These methods primarily attempt to learn the distribution of labeled and unlabeled data by enforcing consistency in the predictions or embedding context. However, previous approaches have focused only on local discrepancy minimization or context relations across single classes. METHODS In this paper, we introduce a novel adversarial learning-based semi-supervised segmentation method that effectively embeds both local and global features from multiple hidden layers and learns context relations between multiple classes. Our voxel-wise adversarial learning method utilizes a voxel-wise feature discriminator, which considers multilayer voxel-wise features (involving both local and global features) as an input by embedding class-specific voxel-wise feature distribution. Furthermore, our previous representation learning method is improved by overcoming information loss and learning stability problems, which enables rich representations of labeled data. RESULT In the experiments, we used the Left Atrial Segmentation Challenge dataset and the Abdominal Multi-Organ dataset to prove the effectiveness of our method in both single class and multiclass segmentation. The experimental results demonstrate that our method outperforms current best-performing state-of-the-art semi-supervised learning approaches. Our proposed adversarial learning-based semi-supervised segmentation method successfully leveraged unlabeled data to improve the network performance by 2% in Dice score coefficient for multi-organ dataset. CONCLUSION We compare our approach to a wide range of medical datasets, and showed our method can be adapted to embed class-specific features. Furthermore, visual interpretation of the feature space demonstrates that our proposed method enables a well-distributed and separated feature space from both labeled and unlabeled data, which improves the overall prediction results.
Robust-RRT: Probabilistically-Complete Motion Planning for Uncertain Nonlinear Systems

Robust motion planning entails computing a global motion plan that is safe under all possible uncertainty realizations, be it in the system dynamics, the robot's initial position, or with respect to external disturbances. Current approaches for robust motion planning either lack theoretical guarantees, or make restrictive assumptions on the system dynamics and uncertainty distributions. In this paper, we address these limitations by proposing the robust rapidly-exploring random-tree (Robust-RRT) algorithm, which integrates forward reachability analysis directly into sampling-based control trajectory synthesis. We prove that Robust-RRT is probabilistically complete (PC) for nonlinear Lipschitz continuous dynamical systems with bounded uncertainty. In other words, Robust-RRT eventually finds a robust motion plan that is feasible under all possible uncertainty realizations assuming such a plan exists. Our analysis applies even to unstable systems that admit only short-horizon feasible plans; this is because we explicitly consider the time evolution of reachable sets along control trajectories. Thanks to the explicit consideration of time dependency in our analysis, PC applies to unstabilizable systems. To the best of our knowledge, this is the most general PC proof for robust sampling-based motion planning, in terms of the types of uncertainties and dynamical systems it can handle. Considering that an exact computation of reachable sets can be computationally expensive for some dynamical systems, we incorporate sampling-based reachability analysis into Robust-RRT and demonstrate our robust planner on nonlinear, underactuated, and hybrid systems.
Natural Language Specifications in Proof Assistants

Interactive proof assistants are computer programs carefully constructed to check a human-designed proof of a mathematical claim with high confidence in the implementation. However, this only validates truth of a formal claim, which may have been mistranslated from a claim made in natural language. This is especially problematic when using proof assistants to formally verify the correctness of software with respect to a natural language specification. The translation from informal to formal remains a challenging, time-consuming process that is difficult to audit for correctness. This paper argues that it is possible to build support for natural language specifications within existing proof assistants, in a way that complements the principles used to establish trust and auditability in proof assistants themselves.
Evaluating Uncertainty Calibration for Open-Set Recognition

—Despite achieving enormous success in predictive accuracy for visual classiﬁcation problems, deep neural networks (DNNs) suffer from providing overconﬁdent probabilities on out-of-distribution (OOD) data. Yet, accurate uncertainty estimation is crucial for safe and reliable robot autonomy. In this paper, we evaluate popular calibration techniques for open-set conditions in a way that is distinctly different from the conventional evaluation of calibration methods on OOD data. Our results show that closed-set DNN calibration approaches are much less effective for open-set recognition , which highlights the need to develop new DNN calibration methods to address this problem.
cMelGAN: An Efficient Conditional Generative Model Based on Mel Spectrograms

Analysing music in the field of machine learning is a very difficult problem with numerous constraints to consider. The nature of audio data, with its very high dimensionality and widely varying scales of structure, is one of the primary reasons why it is so difficult to model. There are many applications of machine learning in music, like the classifying the mood of a piece of music, conditional music generation, or popularity prediction. The goal for this project was to develop a genre-conditional generative model of music based on Mel spectrograms and evaluate its performance by comparing it to existing generative music models that use note-based representations. We initially implemented an autoregressive, RNN-based generative model called MelNet . However, due to its slow speed and low fidelity output, we decided to create a new, fully convolutional architecture that is based on the MelGAN [4] and conditional GAN architectures, called cMelGAN.
Thermodynamics as Combinatorics: A Toy Theory

We discuss a simple toy model which allows, in a natural way, for deriving central facts from thermodynamics such as its fundamental laws, including Carnot’s version of the second principle. Our viewpoint represents thermodynamic systems as binary strings, and it links their temperature to their Hamming weight. From this, we can reproduce the possibility of negative temperatures, the notion of equilibrium as the coïncidence of two notions of temperature — statistical versus structural —, as well as the zeroth law of thermodynamics (transitivity of the thermal-equilibrium relation), which we find to be redundant, as other authors, yet at the same time not to be universally valid.