OpenRaiser/Intern-Atlas huggingface dataset:
Model Blending for Text Classification
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks (DNNs) have proven successful in a wide variety of applications such as speech recognition and synthesis, computer vision, machine translation, and game playing, to name but a few. However, existing deep neural network models are computationally expensive and memory intensive, hindering their deployment in devices with low memory resources or in applications with strict latency requirements. Therefore, a natural thought is to perform model compression and acceleration in deep networks without significantly decreasing the model performance, which is what we call reducing the complexity. In the following work, we try reducing the complexity of state of the art LSTM models for natural language tasks such as text classification, by distilling their knowledge to CNN based models, thus reducing the inference time(or latency) during testing.
Modeling community standards for metadata as templates makes data FAIR
Venue: arXiv cs.DL 2022
Year: 2022

It is challenging to determine whether datasets are findable, accessible, interoperable, and reusable (FAIR) because the FAIR Guiding Principles refer to highly idiosyncratic criteria regarding the metadata used to annotate datasets. Specifically, the FAIR principles require metadata to be "rich" and to adhere to "domain-relevant" community standards. Scientific communities should be able to define their own machine-actionable templates for metadata that encode these "rich," discipline-specific elements. We have explored this template-based approach in the context of two software systems. One system is the CEDAR Workbench, which investigators use to author new metadata. The other is the FAIRware Workbench, which evaluates the metadata of archived datasets for their adherence to community standards. Benefits accrue when templates for metadata become central elements in an ecosystem of tools to manage online datasets--both because the templates serve as a community reference for what constitutes FAIR data, and because they embody that perspective in a form that can be distributed among a variety of software applications to assist with data stewardship and data sharing.
Core and Periphery as Closed-System Precepts for Engineering General Intelligence
Venue: arXiv cs.AI 2022
Year: 2022

Engineering methods are centered around traditional notions of decomposition and recomposition that rely on partitioning the inputs and outputs of components to allow for component-level properties to hold after their composition. In artificial intelligence (AI), however, systems are often expected to influence their environments, and, by way of their environments, to influence themselves. Thus, it is unclear if an AI system's inputs will be independent of its outputs, and, therefore, if AI systems can be treated as traditional components. This paper posits that engineering general intelligence requires new general systems precepts, termed the core and periphery, and explores their theoretical uses. The new precepts are elaborated using abstract systems theory and the Law of Requisite Variety. By using the presented material, engineers can better understand the general character of regulating the outcomes of AI to achieve stakeholder needs and how the general systems nature of embodiment challenges traditional engineering practice.
A 35-Year Longitudinal Analysis of Dermatology Patient Behavior across Economic & Cultural Manifestations in Tunisia, and the Impact of Digital Tools
Venue: arXiv cs.CY 2022
Year: 2022

The evolution of behavior of dermatology patients has seen significantly accelerated change over the past decade, driven by surging availability and adoption of digital tools and platforms. Through our longitudinal analysis of this behavior within Tunisia over a 35-year time frame, we identify behavioral patterns across economic and cultural dimensions and how digital tools have impacted those patterns in preceding years. Throughout this work, we highlight the witnessed effects of available digital tools as experienced by patients, and conclude by presenting a vision for how future tools can help address the issues identified across economic and cultural manifestations. Our analysis is further framed around three types of digital tools: "Dr. Google", social media, and artificial intelligence (AI) tools, and across three stages of clinical care: pre-visit, in-visit, and post-visit.
Improved post-hoc probability calibration for out-of-domain MRI segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Probability calibration for deep models is highly desirable in safety-critical applications such as medical imaging. It makes output probabilities of deep networks interpretable, by aligning prediction probability with the actual accuracy in test data. In image segmentation, well-calibrated probabilities allow radiologists to identify regions where model-predicted segmentations are unreliable. These unreliable predictions often occur to out-of-domain (OOD) images that are caused by imaging artifacts or unseen imaging protocols. Unfortunately, most previous calibration methods for image segmentation perform sub-optimally on OOD images. To reduce the calibration error when confronted with OOD images, we propose a novel post-hoc calibration model. Our model leverages the pixel susceptibility against perturbations at the local level, and the shape prior information at the global level. The model is tested on cardiac MRI segmentation datasets that contain unseen imaging artifacts and images from an unseen imaging protocol. We demonstrate reduced calibration errors compared with the state-of-the-art calibration algorithm.
Differentially Private Counterfactuals via Functional Mechanism
Venue: arXiv cs.LG 2022
Year: 2022

Counterfactual, serving as one emerging type of model explanation, has attracted tons of attentions recently from both industry and academia. Different from the conventional feature-based explanations (e.g., attributions), counterfactuals are a series of hypothetical samples which can flip model decisions with minimal perturbations on queries. Given valid counterfactuals, humans are capable of reasoning under ``what-if'' circumstances, so as to better understand the model decision boundaries. However, releasing counterfactuals could be detrimental, since it may unintentionally leak sensitive information to adversaries, which brings about higher risks on both model security and data privacy. To bridge the gap, in this paper, we propose a novel framework to generate differentially private counterfactual (DPC) without touching the deployed model or explanation set, where noises are injected for protection while maintaining the explanation roles of counterfactual. In particular, we train an autoencoder with the functional mechanism to construct noisy class prototypes, and then derive the DPC from the latent prototypes based on the post-processing immunity of differential privacy. Further evaluations demonstrate the effectiveness of the proposed framework, showing that DPC can successfully relieve the risks on both extraction and inference attacks.
Quantifying the Sensitivity and Unclonability of Optical Physical Unclonable Functions
Venue: arXiv cs.CR 2022
Year: 2022

Due to their unmatched entropy, complexity, and security level, optical Physical Unclonable Functions (PUFs) currently receive a lot of interest in the literature. Despite the large body of existing works, however, one of their core features has never been quantified in detail, namely their physical unclonability. This paper tackles this fundamental and yet largely unaddressed issue. In simulations and/or experiments, the sensitivity of diffraction-based optical responses is investigated with respect to various small alterations such as variation in the position, size, and number of the scatterers, as well as perturbations in the spatial alignment between the physical unclonable function (PUF) and the measurement apparatus. Our analysis focuses on 2D optical PUFs because of their relevance in integrated applications and the need to reply to security concerns that can be raised when the physical structure of the geometry is accessible. Among the results of this study, the sensitivity analysis shows that a positional perturbation of scatterers on the order of \SI{30}{\nano\meter}, i.e., far below the wavelength of the probing laser light of \SI{632}{\nano\meter} wavelength, is sufficient to invalidate the PUF response and thus detect a forgery attempt. These results support and quantify the high adversarial efforts required to clone optical PUFs, even for 2D layouts.
Stochastic additive manufacturing simulation: from experiment to surface roughness and porosity prediction
Venue: arXiv cs.CE 2022
Year: 2022

Deterministic computational modeling of laser powder bed fusion (LPBF) process fails to capture irregularities and roughness of the scan track, unless expensive powder-scale analysis is used. In this work we developed a stochastic computational modeling framework based on Markov Chain Monte Carlo (MCMC) capable of capturing the irregularities of LPBF scan. The model is calibrated against AFRL single track scan data using a specially designed tensor decomposition method, i.e., Higher-Order Proper Generalized Decomposition (HOPGD) that relies on non-intrusive data learning and construction of reduced order surrogate models. Once calibrated, the stochastic model can be used to predict the roughness and porosity at part scale at a significantly reduced computational cost compared to detailed powder-scale deterministic simulations. The stochastic simulation predictions are validated against AFRL multi-layer and multitrack experiments and reported as more accurate when compared with regular deterministic simulation results.
Padding-only defenses add delay in Tor
Venue: arXiv cs.CR 2022
Year: 2022

Website fingerprinting is an attack that uses size and timing characteristics of encrypted downloads to identify targeted websites. Since this can defeat the privacy goals of anonymity networks such as Tor, many algorithms to defend against this attack in Tor have been proposed in the literature. These algorithms typically consist of some combination of the injection of dummy "padding" packets with the delay of actual packets to disrupt timing patterns. For usability reasons, Tor is intended to provide low latency; as such, many authors focus on padding-only defenses in the belief that they are "zero-delay." We demonstrate through Shadow simulations that by increasing queue lengths, padding-only defenses add delay when deployed network-wide, so they should not be considered "zero-delay." We further argue that future defenses should also be evaluated using network-wide deployment simulations
LATTE: LAnguage Trajectory TransformEr
Venue: arXiv cs.RO 2022
Year: 2022

Natural language is one of the most intuitive ways to express human intent. However, translating instructions and commands towards robotic motion generation and deployment in the real world is far from being an easy task. The challenge of combining a robot's inherent low-level geometric and kinodynamic constraints with a human's high-level semantic instructions traditionally is solved using task-specific solutions with little generalizability between hardware platforms, often with the use of static sets of target actions and commands. This work instead proposes a flexible language-based framework that allows a user to modify generic robotic trajectories. Our method leverages pre-trained language models (BERT and CLIP) to encode the user's intent and target objects directly from a free-form text input and scene images, fuses geometrical features generated by a transformer encoder network, and finally outputs trajectories using a transformer decoder, without the need of priors related to the task or robot information. We significantly extend our own previous work presented in Bucker et al. by expanding the trajectory parametrization space to 3D and velocity as opposed to just XY movements. In addition, we now train the model to use actual images of the objects in the scene for context (as opposed to textual descriptions), and we evaluate the system in a diverse set of scenarios beyond manipulation, such as aerial and legged robots. Our simulated and real-life experiments demonstrate that our transformer model can successfully follow human intent, modifying the shape and speed of trajectories within multiple environments. Codebase available at: https://github.com/arthurfenderbucker/LaTTe-Language-Trajectory-TransformEr.git
Applied monocular reconstruction of parametric faces with domain engineering
Venue: arXiv cs.CV 2022
Year: 2022

Many modern online 3D applications and videogames rely on parametric models of human faces for creating believable avatars. However, manual reproduction of someone's facial likeness with a parametric model is difficult and time-consuming. Machine Learning solution for that task is highly desirable but is also challenging. The paper proposes a novel approach to the so-called Face-to-Parameters problem (F2P for short), aiming to reconstruct a parametric face from a single image. The proposed method utilizes synthetic data, domain decomposition, and domain adaptation for addressing multifaceted challenges in solving the F2P. The open-sourced codebase illustrates our key observations and provides means for quantitative evaluation. The presented approach proves practical in an industrial application; it improves accuracy and allows for more efficient models training. The techniques have the potential to extend to other types of parametric models.
A Hybrid Observer for Estimating the State of a Distributed Linear System
Venue: arXiv cs.DC 2022
Year: 2022

A hybrid observer is described for estimating the state of a system of the form dot x=Ax, y_i=C_ix, i=1,...,m. The system's state x is simultaneously estimated by m agents assuming agent i senses y_i and receives appropriately defined data from its neighbors. Neighbor relations are characterized by a time-varying directed graph N(t). Agent i updates its estimate x_i of x at event times t_{i1},t_{i2} ... using a local continuous-time linear observer and a local parameter estimator which iterates q times during each event time interval [t_{i(s-1)},t_{is}), s>=1, to obtain an estimate of x(t_{is}). Subject to the assumptions that N(t) is strongly connected, and the system is jointly observable, it is possible to design parameters so that x_i converges to x with a pre-assigned rate. This result holds when agents communicate asynchronously with the assumption that N(t) changes slowly. Exponential convergence is also assured if the event time sequence of the agents are slightly different, although only if the system being observed is exponentially stable; this limitation however, is a robustness issue shared by all open loop state estimators with small modeling errors. The result also holds facing abrupt changes in the number of vertices and arcs in the inter-agent communication graph upon which the algorithm depends.
Analyzing the Impact of Shape & Context on the Face Recognition Performance of Deep Networks
Venue: arXiv cs.CV 2022
Year: 2022

In this article, we analyze how changing the underlying 3D shape of the base identity in face images can distort their overall appearance, especially from the perspective of deep face recognition. As done in popular training data augmentation schemes, we graphically render real and synthetic face images with randomly chosen or best-fitting 3D face models to generate novel views of the base identity. We compare deep features generated from these images to assess the perturbation these renderings introduce into the original identity. We perform this analysis at various degrees of facial yaw with the base identities varying in gender and ethnicity. Additionally, we investigate if adding some form of context and background pixels in these rendered images, when used as training data, further improves the downstream performance of a face recognition model. Our experiments demonstrate the significance of facial shape in accurate face matching and underpin the importance of contextual data for network training.
Task-Balanced Distillation for Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Mainstream object detectors are commonly constituted of two sub-tasks, including classification and regression tasks, implemented by two parallel heads. This classic design paradigm inevitably leads to inconsistent spatial distributions between classification score and localization quality (IOU). Therefore, this paper alleviates this misalignment in the view of knowledge distillation. First, we observe that the massive teacher achieves a higher proportion of harmonious predictions than the lightweight student. Based on this intriguing observation, a novel Harmony Score (HS) is devised to estimate the alignment of classification and regression qualities. HS models the relationship between two sub-tasks and is seen as prior knowledge to promote harmonious predictions for the student. Second, this spatial misalignment will result in inharmonious region selection when distilling features. To alleviate this problem, a novel Task-decoupled Feature Distillation (TFD) is proposed by flexibly balancing the contributions of classification and regression tasks. Eventually, HD and TFD constitute the proposed method, named Task-Balanced Distillation (TBD). Extensive experiments demonstrate the considerable potential and generalization of the proposed method. Specifically, when equipped with TBD, RetinaNet with ResNet-50 achieves 41.0 mAP under the COCO benchmark, outperforming the recent FGD and FRS.
Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognition
Venue: arXiv cs.CL 2022
Year: 2022

Named entity recognition (NER) task aims at identifying entities from a piece of text that belong to predefined semantic types such as person, location, organization, etc. The state-of-the-art solutions for flat entities NER commonly suffer from capturing the fine-grained semantic information in underlying texts. The existing span-based approaches overcome this limitation, but the computation time is still a concern. In this work, we propose a novel span-based NER framework, namely Global Pointer (GP), that leverages the relative positions through a multiplicative attention mechanism. The ultimate goal is to enable a global view that considers the beginning and the end positions to predict the entity. To this end, we design two modules to identify the head and the tail of a given entity to enable the inconsistency between the training and inference processes. Moreover, we introduce a novel classification loss function to address the imbalance label problem. In terms of parameters, we introduce a simple but effective approximate method to reduce the training parameters. We extensively evaluate GP on various benchmark datasets. Our extensive experiments demonstrate that GP can outperform the existing solution. Moreover, the experimental results show the efficacy of the introduced loss function compared to softmax and entropy alternatives.
Joint Beamforming Design in DFRC Systems for Wideband Sensing and OFDM Communications
Venue: arXiv cs.IT 2022
Year: 2022

Dual-function radar-communication (DFRC) systems, which can efficiently utilize the congested spectrum and costly hardware resources by employing one common waveform for both sensing and communication (S&C), have attracted increasing attention. While the orthogonal frequency division multiplexing (OFDM) technique has been widely adopted to support high-quality communications, it also has great potentials of improving radar sensing performance and providing flexible S&C. In this paper, we propose to jointly design the dual-functional transmit signals occupying several subcarriers to realize multi-user OFDM communications and detect one moving target in the presence of clutter. Meanwhile, the signals in other frequency subcarriers can be optimized in a similar way to perform other tasks. The transmit beamforming and receive filter are jointly optimized to maximize the radar output signal-to-interference-plus-noise ratio (SINR), while satisfying the communication SINR requirement and the power budget. An majorization minimization (MM) method based algorithm is developed to solve the resulting non-convex optimization problem. Numerical results reveal the significant wideband sensing gain brought by jointly designing the transmit signals in different subcarriers, and demonstrate the advantages of our proposed scheme and the effectiveness of the developed algorithm.
Neural Network Verification using Residual Reasoning
Venue: arXiv cs.NE 2022
Year: 2022

With the increasing integration of neural networks as components in mission-critical systems, there is an increasing need to ensure that they satisfy various safety and liveness requirements. In recent years, numerous sound and complete verification methods have been proposed towards that end, but these typically suffer from severe scalability limitations. Recent work has proposed enhancing such verification techniques with abstraction-refinement capabilities, which have been shown to boost scalability: instead of verifying a large and complex network, the verifier constructs and then verifies a much smaller network, whose correctness implies the correctness of the original network. A shortcoming of such a scheme is that if verifying the smaller network fails, the verifier needs to perform a refinement step that increases the size of the network being verified, and then start verifying the new network from scratch - effectively "wasting" its earlier work on verifying the smaller network. In this paper, we present an enhancement to abstraction-based verification of neural networks, by using residual reasoning: the process of utilizing information acquired when verifying an abstract network, in order to expedite the verification of a refined network. In essence, the method allows the verifier to store information about parts of the search space in which the refined network is guaranteed to behave correctly, and allows it to focus on areas where bugs might be discovered. We implemented our approach as an extension to the Marabou verifier, and obtained promising results.
A Fixpoint Characterization of Three-Valued Disjunctive Hybrid MKNF Knowledge Bases
Venue: arXiv cs.AI 2022
Year: 2022

The logic of hybrid MKNF (minimal knowledge and negation as failure) is a powerful knowledge representation language that elegantly pairs ASP (answer set programming) with ontologies. Disjunctive rules are a desirable extension to normal rule-based reasoning and typically semantic frameworks designed for normal knowledge bases need substantial restructuring to support disjunctive rules. Alternatively, one may lift characterizations of normal rules to support disjunctive rules by inducing a collection of normal knowledge bases, each with the same body and a single atom in its head. In this work, we refer to a set of such normal knowledge bases as a head-cut of a disjunctive knowledge base. The question arises as to whether the semantics of disjunctive hybrid MKNF knowledge bases can be characterized using fixpoint constructions with head-cuts. Earlier, we have shown that head-cuts can be paired with fixpoint operators to capture the two-valued MKNF models of disjunctive hybrid MKNF knowledge bases. Three-valued semantics extends two-valued semantics with the ability to express partial information. In this work, we present a fixpoint construction that leverages head-cuts using an operator that iteratively captures three-valued models of hybrid MKNF knowledge bases with disjunctive rules. This characterization also captures partial stable models of disjunctive logic programs since a program can be expressed as a disjunctive hybrid MKNF knowledge base with an empty ontology. We elaborate on a relationship between this characterization and approximators in AFT (approximation fixpoint theory) for normal hybrid MKNF knowledge bases.