OpenRaiser/Intern-Atlas huggingface dataset:
Representation Evaluation Block-based Teacher-Student Network for the Industrial Quality-relevant Performance Modeling and Monitoring
Venue: arXiv cs.LG 2021
Year: 2021

Quality-relevant fault detection plays an important role in industrial processes, while the current quality-related fault detection methods based on neural networks main concentrate on process-relevant variables and ignore quality-relevant variables, which restrict the application of process monitoring. Therefore, in this paper, a fault detection scheme based on the improved teacher-student network is proposed for quality-relevant fault detection. In the traditional teacher-student network, as the features differences between the teacher network and the student network will cause performance degradation on the student network, representation evaluation block (REB) is proposed to quantify the features differences between the teacher and the student networks, and uncertainty modeling is used to add this difference in modeling process, which are beneficial to reduce the features differences and improve the performance of the student network. Accordingly, REB and uncertainty modeling is applied in the teacher-student network named as uncertainty modeling teacher-student uncertainty autoencoder (TSUAE). Then, the proposed TSUAE is applied to process monitoring, which can effectively detect faults in the process-relevant subspace and quality-relevant subspace simultaneously. The proposed TSUAE-based fault detection method is verified in two simulation experiments illustrating that it has satisfactory fault detection performance compared to other fault detection methods.
One-way resynchronizability of word transducers
Venue: arXiv cs.FL 2021
Year: 2021

The origin semantics for transducers was proposed in 2014, and led to various characterizations and decidability results that are in contrast with the classical semantics. In this paper we add a further decidability result for characterizing transducers that are close to one-way transducers in the origin semantics. We show that it is decidable whether a non-deterministic two-way word transducer can be resynchronized by a bounded, regular resynchronizer into an origin-equivalent one-way transducer. The result is in contrast with the usual semantics, where it is undecidable to know if a non-deterministic two-way transducer is equivalent to some one-way transducer.
Deep Learning for Intelligent Demand Response and Smart Grids: A Comprehensive Survey
Venue: arXiv cs.LG 2021
Year: 2021

Electricity is one of the mandatory commodities for mankind today. To address challenges and issues in the transmission of electricity through the traditional grid, the concepts of smart grids and demand response have been developed. In such systems, a large amount of data is generated daily from various sources such as power generation (e.g., wind turbines), transmission and distribution (microgrids and fault detectors), load management (smart meters and smart electric appliances). Thanks to recent advancements in big data and computing technologies, Deep Learning (DL) can be leveraged to learn the patterns from the generated data and predict the demand for electricity and peak hours. Motivated by the advantages of deep learning in smart grids, this paper sets to provide a comprehensive survey on the application of DL for intelligent smart grids and demand response. Firstly, we present the fundamental of DL, smart grids, demand response, and the motivation behind the use of DL. Secondly, we review the state-of-the-art applications of DL in smart grids and demand response, including electric load forecasting, state estimation, energy theft detection, energy sharing and trading. Furthermore, we illustrate the practicality of DL via various use cases and projects. Finally, we highlight the challenges presented in existing research works and highlight important issues and potential directions in the use of DL for smart grids and demand response.
Adversarial Attacks for Tabular Data: Application to Fraud Detection and Imbalanced Data
Venue: arXiv cs.CR 2021
Year: 2021

Guaranteeing the security of transactional systems is a crucial priority of all institutions that process transactions, in order to protect their businesses against cyberattacks and fraudulent attempts. Adversarial attacks are novel techniques that, other than being proven to be effective to fool image classification models, can also be applied to tabular data. Adversarial attacks aim at producing adversarial examples, in other words, slightly modified inputs that induce the Artificial Intelligence (AI) system to return incorrect outputs that are advantageous for the attacker. In this paper we illustrate a novel approach to modify and adapt state-of-the-art algorithms to imbalanced tabular data, in the context of fraud detection. Experimental results show that the proposed modifications lead to a perfect attack success rate, obtaining adversarial examples that are also less perceptible when analyzed by humans. Moreover, when applied to a real-world production system, the proposed techniques shows the possibility of posing a serious threat to the robustness of advanced AI-based fraud detection procedures.
Designing Accountable Systems
Venue: arXiv cs.SE 2021
Year: 2021

Accountability is an often called for property of technical systems. It is a requirement for algorithmic decision systems, autonomous cyber-physical systems, and for software systems in general. As a concept, accountability goes back to the early history of Liberalism and is suggested as a tool to limit the use of power. This long history has also given us many, often slightly differing, definitions of accountability. The problem that software developers now face is to understand what accountability means for their systems and how to reflect it in a system's design. To enable the rigorous study of accountability in a system, we need models that are suitable for capturing such a varied concept. In this paper, we present a method to express and compare different definitions of accountability using Structural Causal Models. We show how these models can be used to evaluate a system's design and present a small use case based on an autonomous car.
Positive spectrahedra: Invariance principles and Pseudorandom generators
Venue: arXiv cs.CC 2021
Year: 2021

In a recent work, O'Donnell, Servedio and Tan (STOC 2019) gave explicit pseudorandom generators (PRGs) for arbitrary $m$-facet polytopes in $n$ variables with seed length poly-logarithmic in $m,n$, concluding a sequence of works in the last decade, that was started by Diakonikolas, Gopalan, Jaiswal, Servedio, Viola (SICOMP 2010) and Meka, Zuckerman (SICOMP 2013) for fooling linear and polynomial threshold functions, respectively. In this work, we consider a natural extension of PRGs for intersections of positive spectrahedrons. A positive spectrahedron is a Boolean function $f(x)=[x_1A^1+\cdots +x_nA^n \preceq B]$ where the $A^i$s are $k\times k$ positive semidefinite matrices. We construct explicit PRGs that $δ$-fool "regular" width-$M$ positive spectrahedrons (i.e., when none of the $A^i$s are dominant) over the Boolean space with seed length $\textsf{poly}(\log k,\log n, M, 1/δ)$. Our main technical contributions are the following: We first prove an invariance principle for positive spectrahedra via the well-known Lindeberg method. As far as we are aware such a generalization of the Lindeberg method was unknown. Second, we prove an upper bound on noise sensitivity and a Littlewood-Offord theorem for positive spectrahedra. Using these results, we give applications for constructing PRGs for positive spectrahedra, learning theory, discrepancy sets for positive spectrahedra (over the Boolean cube) and PRGs for intersections of structured polynomial threshold functions.
On the Readability of Abstract Set Visualizations
Venue: arXiv cs.HC 2021
Year: 2021

Set systems are used to model data that naturally arises in many contexts: social networks have communities, musicians have genres, and patients have symptoms. Visualizations that accurately reflect the information in the underlying set system make it possible to identify the set elements, the sets themselves, and the relationships between the sets. In static contexts, such as print media or infographics, it is necessary to capture this information without the help of interactions. With this in mind, we consider three different systems for medium-sized set data, LineSets, EulerView, and MetroSets, and report the results of a controlled human-subjects experiment comparing their effectiveness. Specifically, we evaluate the performance, in terms of time and error, on tasks that cover the spectrum of static set-based tasks. We also collect and analyze qualitative data about the three different visualization systems. Our results include statistically significant differences, suggesting that MetroSets performs and scales better.
Synthesizing Context-free Grammars from Recurrent Neural Networks (Extended Version)
Venue: arXiv cs.FL 2021
Year: 2021

We present an algorithm for extracting a subclass of the context free grammars (CFGs) from a trained recurrent neural network (RNN). We develop a new framework, pattern rule sets (PRSs), which describe sequences of deterministic finite automata (DFAs) that approximate a non-regular language. We present an algorithm for recovering the PRS behind a sequence of such automata, and apply it to the sequences of automata extracted from trained RNNs using the L* algorithm. We then show how the PRS may converted into a CFG, enabling a familiar and useful presentation of the learned language. Extracting the learned language of an RNN is important to facilitate understanding of the RNN and to verify its correctness. Furthermore, the extracted CFG can augment the RNN in classifying correct sentences, as the RNN's predictive accuracy decreases when the recursion depth and distance between matching delimiters of its input sequences increases.
secureTF: A Secure TensorFlow Framework
Venue: arXiv cs.CR 2021
Year: 2021

Data-driven intelligent applications in modern online services have become ubiquitous. These applications are usually hosted in the untrusted cloud computing infrastructure. This poses significant security risks since these applications rely on applying machine learning algorithms on large datasets which may contain private and sensitive information. To tackle this challenge, we designed secureTF, a distributed secure machine learning framework based on Tensorflow for the untrusted cloud infrastructure. secureTF is a generic platform to support unmodified TensorFlow applications, while providing end-to-end security for the input data, ML model, and application code. secureTF is built from ground-up based on the security properties provided by Trusted Execution Environments (TEEs). However, it extends the trust of a volatile memory region (or secure enclave) provided by the single node TEE to secure a distributed infrastructure required for supporting unmodified stateful machine learning applications running in the cloud. The paper reports on our experiences about the system design choices and the system deployment in production use-cases. We conclude with the lessons learned based on the limitations of our commercially available platform, and discuss open research problems for the future work.
PiChu: Accelerating Block Broadcasting in Blockchain Networks with Pipelining and Chunking
Venue: arXiv cs.DC 2021
Year: 2021

Blockchain technologies have been rapidly enhanced in recent years. However, its scalability still has limitations in terms of throughput and broadcast delay as the network and the amount of transaction data increase. To improve scalability of blockchain networks, we propose a novel approach named PiChu that accelerates block propagation in blockchain networks by pipelining and verifying chunks of a block in parallel. Accelerating block propagation reduces the mining interval and chance of fork occurring, which in turn increases throughput. Our approach can be applied to the blockchain networks either directly or with a minor modification to the consensus. Through an extensive and large scale simulations, we validate that the proposed PiChu scheme significantly enhances the scalability of blockchain networks. For instance, a 64 MB block can be broadcasted in just 80 seconds in a blockchain network with a million nodes. The efficiency of PiChu broadcasting increases with bigger block sizes and a larger number of nodes in the network.
TaNTIN: Terrestrial and Non-Terrestrial Integrated Networks-A collaborative technologies perspective for beyond 5G and 6G
Venue: arXiv cs.NI 2021
Year: 2021

The world is moving toward globalization rapidly. Everybody has easy access to information with the spread of Internet technology. Businesses are growing beyond national borders. Internationalization affects every aspect of life. In this scenario, by dispersing functions and tasks across organizational borders, time and space, global organizations have higher requirements for collaboration. In order to allow decision-makers and knowledge workers, situated at different times and spaces, to work more efficiently, collaborative technologies are needed. In this paper, we give an overview of potential collaborative technologies, their benefits, risks and challenges, types, and elements. Based on the conceptualization of terrestrial and non-terrestrial integrated networks (TaNTIN), we highlight artificial intelligence (AI), blockchains, tactile Internet, mobile edge computing (MEC)/fog computing, augmented reality and virtual reality, and so forth as the key features to ensure quality-of-service (QoS) guarantee of futuristic collaborative services such as telemedicine, e-education, online gaming, online businesses, the entertainment industry. We also discuss how these technologies will impact human life in the near future.
Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts
Venue: arXiv cs.HC 2021
Year: 2021

Charts often contain visually prominent features that draw attention to aspects of the data and include text captions that emphasize aspects of the data. Through a crowdsourced study, we explore how readers gather takeaways when considering charts and captions together. We first ask participants to mark visually prominent regions in a set of line charts. We then generate text captions based on the prominent features and ask participants to report their takeaways after observing chart-caption pairs. We find that when both the chart and caption describe a high-prominence feature, readers treat the doubly emphasized high-prominence feature as the takeaway; when the caption describes a low-prominence chart feature, readers rely on the chart and report a higher-prominence feature as the takeaway. We also find that external information that provides context, helps further convey the caption's message to the reader. We use these findings to provide guidelines for authoring effective chart-caption pairs.
An Empirical Study and Analysis on Open-Set Semi-Supervised Learning
Venue: arXiv cs.CV 2021
Year: 2021

Pseudo-labeling (PL) and Data Augmentation-based Consistency Training (DACT) are two approaches widely used in Semi-Supervised Learning (SSL) methods. These methods exhibit great power in many machine learning tasks by utilizing unlabeled data for efficient training. But in a more realistic setting (termed as open-set SSL), where unlabeled dataset contains out-of-distribution (OOD) samples, the traditional SSL methods suffer severe performance degradation. Recent approaches mitigate the negative influence of OOD samples by filtering them out from the unlabeled data. However, it is not clear whether directly removing the OOD samples is the best choice. Furthermore, why PL and DACT could perform differently in open-set SSL remains a mystery. In this paper, we thoroughly analyze various SSL methods (PL and DACT) on open-set SSL and discuss pros and cons of these two approaches separately. Based on our analysis, we propose Style Disturbance to improve traditional SSL methods on open-set SSL and experimentally show our approach can achieve state-of-the-art results on various datasets by utilizing OOD samples properly. We believe our study can bring new insights for SSL research.
What is all this new MeSH about? Exploring the semantic provenance of new descriptors in the MeSH thesaurus
Venue: arXiv cs.DL 2021
Year: 2021

The Medical Subject Headings (MeSH) thesaurus is a controlled vocabulary widely used in biomedical knowledge systems, particularly for semantic indexing of scientific literature. As the MeSH hierarchy evolves through annual version updates, some new descriptors are introduced that were not previously available. This paper explores the conceptual provenance of these new descriptors. In particular, we investigate whether such new descriptors have been previously covered by older descriptors and what is their current relation to them. To this end, we propose a framework to categorize new descriptors based on their current relation to older descriptors. Based on the proposed classification scheme, we quantify, analyse and present the different types of new descriptors introduced in MeSH during the last fifteen years. The results show that only about 25% of new MeSH descriptors correspond to new emerging concepts, whereas the rest were previously covered by one or more existing descriptors, either implicitly or explicitly. Most of them were covered by a single existing descriptor and they usually end up as descendants of it in the current hierarchy, gradually leading towards a more fine-grained MeSH vocabulary. These insights about the dynamics of the thesaurus are useful for the retrospective study of scientific articles annotated with MeSH, but could also be used to inform the policy of updating the thesaurus in the future.
Text Line Segmentation for Challenging Handwritten Document Images Using Fully Convolutional Network
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a method for text line segmentation of challenging historical manuscript images. These manuscript images contain narrow interline spaces with touching components, interpenetrating vowel signs and inconsistent font types and sizes. In addition, they contain curved, multi-skewed and multi-directed side note lines within a complex page layout. Therefore, bounding polygon labeling would be very difficult and time consuming. Instead we rely on line masks that connect the components on the same text line. Then these line masks are predicted using a Fully Convolutional Network (FCN). In the literature, FCN has been successfully used for text line segmentation of regular handwritten document images. The present paper shows that FCN is useful with challenging manuscript images as well. Using a new evaluation metric that is sensitive to over segmentation as well as under segmentation, testing results on a publicly available challenging handwritten dataset are comparable with the results of a previous work on the same dataset.
Aesthetics, Personalization and Recommendation: A survey on Deep Learning in Fashion
Venue: arXiv cs.CV 2021
Year: 2021

Machine learning is completely changing the trends in the fashion industry. From big to small every brand is using machine learning techniques in order to improve their revenue, increase customers and stay ahead of the trend. People are into fashion and they want to know what looks best and how they can improve their style and elevate their personality. Using Deep learning technology and infusing it with Computer Vision techniques one can do so by utilizing Brain-inspired Deep Networks, and engaging into Neuroaesthetics, working with GANs and Training them, playing around with Unstructured Data,and infusing the transformer architecture are just some highlights which can be touched with the Fashion domain. Its all about designing a system that can tell us information regarding the fashion aspect that can come in handy with the ever growing demand. Personalization is a big factor that impacts the spending choices of customers.The survey also shows remarkable approaches that encroach the subject of achieving that by divulging deep into how visual data can be interpreted and leveraged into different models and approaches. Aesthetics play a vital role in clothing recommendation as users' decision depends largely on whether the clothing is in line with their aesthetics, however the conventional image features cannot portray this directly. For that the survey also highlights remarkable models like tensor factorization model, conditional random field model among others to cater the need to acknowledge aesthetics as an important factor in Apparel recommendation.These AI inspired deep models can pinpoint exactly which certain style resonates best with their customers and they can have an understanding of how the new designs will set in with the community. With AI and machine learning your businesses can stay ahead of the fashion trends.
From Local Pseudorandom Generators to Hardness of Learning
Venue: arXiv cs.LG 2021
Year: 2021

We prove hardness-of-learning results under a well-studied assumption on the existence of local pseudorandom generators. As we show, this assumption allows us to surpass the current state of the art, and prove hardness of various basic problems, with no hardness results to date. Our results include: hardness of learning shallow ReLU neural networks under the Gaussian distribution and other distributions; hardness of learning intersections of $ω(1)$ halfspaces, DNF formulas with $ω(1)$ terms, and ReLU networks with $ω(1)$ hidden neurons; hardness of weakly learning deterministic finite automata under the uniform distribution; hardness of weakly learning depth-$3$ Boolean circuits under the uniform distribution, as well as distribution-specific hardness results for learning DNF formulas and intersections of halfspaces. We also establish lower bounds on the complexity of learning intersections of a constant number of halfspaces, and ReLU networks with a constant number of hidden neurons. Moreover, our results imply the hardness of virtually all improper PAC-learning problems (both distribution-free and distribution-specific) that were previously shown hard under other assumptions.
Do we need to go Deep? Knowledge Tracing with Big Data
Venue: arXiv cs.LG 2021
Year: 2021

Interactive Educational Systems (IES) enabled researchers to trace student knowledge in different skills and provide recommendations for a better learning path. To estimate the student knowledge and further predict their future performance, the interest in utilizing the student interaction data captured by IES to develop learner performance models is increasing rapidly. Moreover, with the advances in computing systems, the amount of data captured by these IES systems is also increasing that enables deep learning models to compete with traditional logistic models and Markov processes. However, it is still not empirically evident if these deep models outperform traditional models on the current scale of datasets with millions of student interactions. In this work, we adopt EdNet, the largest student interaction dataset publicly available in the education domain, to understand how accurately both deep and traditional models predict future student performances. Our work observes that logistic regression models with carefully engineered features outperformed deep models from extensive experimentation. We follow this analysis with interpretation studies based on Locally Interpretable Model-agnostic Explanation (LIME) to understand the impact of various features on best performing model pre-dictions.