Reviewerly/exHarmony huggingface dataset:
Improved Embeddings with Easy Positive Triplet Mining

Deep metric learning seeks to define an embedding where semantically similar images are embedded to nearby locations, and semantically dissimilar images are embedded to distant locations. Substantial work has focused on loss functions and strategies to learn these embeddings by pushing images from the same class as close together in the embedding space as possible. In this paper, we propose an alternative, loosened embedding strategy that requires the embedding function only map each training image to the most similar examples from the same class, an approach we call "Easy Positive" mining. We provide a collection of experiments and visualizations that highlight that this Easy Positive mining leads to embed-dings that are more flexible and generalize better to new unseen data. This simple mining strategy yields recall performance that exceeds state of the art approaches (including those with complicated loss functions and ensemble methods) on image retrieval datasets including CUB, Stanford Online Products, In-Shop Clothes and Hotels-50K. Code is available at: https://github.com/littleredxh/EasyPositiveHardNegative.
A many-objective optimization recommendation algorithm based on knowledge mining

Recommendation system (RS) is a technology that provides accurate recommendation for users. In order to make the recommendation results more accurate and diverse, we proposed a rating-based many-objective hybrid recommendation model that can optimize the accuracy, recall, diversity, novelty and coverage of the recommendation simultaneously. Additionally, a new generation-based fitness evaluation strategy and a partition-based knowledge mining strategy are proposed to improve the many-objective evolutionary algorithms (MaOEAs) to enhance the performance of the recommendations generated by the model. Finally, comparing the proposed many-objective optimization recommendation algorithm with the existing standard MaOEAs, experimental results demonstrate that the proposed algorithm can provide recommendations with the more and novel items on the basis of accuracy and diversity for users.
Recurrent Convolutional Structures for Audio Spoof and Video Deepfake Detection

Deepfakes, or artificially generated audiovisual renderings, can be used to defame a public figure or influence public opinion. With the recent discovery of generative adversarial networks, an attacker using a normal desktop computer fitted with an off-the-shelf graphics processing unit can make renditions realistic enough to easily fool a human observer. Detecting deepfakes is thus becoming important for reporters, social media platforms, and the general public. In this work, we introduce simple, yet surprisingly efficient digital forensic methods for audio spoof and visual deepfake detection. Our methods combine convolutional latent representations with bidirectional recurrent structures and entropy-based cost functions. The latent representations for both audio and video are carefully chosen to extract semantically rich information from the recordings. By feeding these into a recurrent framework, we can detect both spatial and temporal signatures of deepfake renditions. The entropy-based cost functions work well in isolation as well as in context with traditional cost functions. We demonstrate our methods on the FaceForensics++ and Celeb-DF video datasets and the ASVSpoof 2019 Logical Access audio datasets, achieving new benchmarks in all categories. We also perform extensive studies to demonstrate generalization to new domains and gain further insight into the effectiveness of the new architectures.
Detection of cervical cancer cells based on strong feature CNN-SVM network

Traditional screening of cervical cells largely depends on the experience of pathologists, which also has the problem of low accuracy and poor efficiency. Medical image processing combining deep learning and machine learning shows its superiority in the field of cell classification. A new framework based on strong feature Convolutional Neural Networks (CNN)-Support Vector Machine (SVM) model was proposed to accurately classify the cervical cells. A method fusing the strong features extracted by Gray-Level Co-occurrence Matrix (GLCM) and Gabor with abstract features from the hidden layers of CNN was conducted, meanwhile the fused ones were input into the SVM for classification. An effective dataset amplification method was designed to improve the robustness of the model. The proposed method was evaluated on two independent datasets with the metrics of accuracy (Acc), sensitivity (Sn), and specificity (Sp). Our approach outperformed than the state-of-the-art models with the Acc, Sn, and Sp of 99.3, 98.9, 99.4 for 2-class detection in the mass, respectively. The results indicated that the strong feature CNN-SVM model could be applied in cell classification for the early screening of cervical cancer.
SDC-Depth: Semantic Divide-and-Conquer Network for Monocular Depth Estimation

Monocular depth estimation is an ill-posed problem, and as such critically relies on scene priors and semantics. Due to its complexity, we propose a deep neural network model based on a semantic divide-and-conquer approach. Our model decomposes a scene into semantic segments, such as object instances and background stuff classes, and then predicts a scale and shift invariant depth map for each semantic segment in a canonical space. Semantic segments of the same category share the same depth decoder, so the global depth prediction task is decomposed into a series of category-specific ones, which are simpler to learn and easier to generalize to new scene types. Finally, our model stitches each local depth segment by predicting its scale and shift based on the global context of the image. The model is trained end-to-end using a multi-task loss for panoptic segmentation and depth prediction, and is therefore able to leverage large-scale panoptic segmentation datasets to boost its semantic understanding. We validate the effectiveness of our approach and show state-of-the-art performance on three benchmark datasets.
Behavioral Biometrics for Continuous Authentication in the Internet-of-Things Era: An Artificial Intelligence Perspective

In the Internet-of-Things (IoT) era, user authentication is essential to ensure the security of connected devices and the customization of passive services. However, conventional knowledge-based and physiological biometric-based authentication systems (e.g., password, face recognition, and fingerprints) are susceptible to shoulder surfing attacks, smudge attacks, and heat attacks. The powerful sensing capabilities of IoT devices, including smartphones, wearables, robots, and autonomous vehicles enable continuous authentication (CA) based on behavioral biometrics. The artificial intelligence (AI) approaches hold significant promise in sifting through large volumes of heterogeneous biometrics data to offer unprecedented user authentication and user identification capabilities. In this survey article, we outline the nature of CA in IoT applications, highlight the key behavioral signals, and summarize the extant solutions from an AI perspective. Based on our systematic and comprehensive analysis, we discuss the challenges and promising future directions to guide the next generation of AI-based CA research.
Meta-optics for spatial optical analog computing

Rapidly growing demands for high-performance computing, powerful data processing systems, and big data necessitate the advent of novel optical devices to perform demanding computing processes effectively. Due to its unprecedented growth in the past two decades, the field of meta-optics offers a viable solution for spatially, spectrally, and/or even temporally sculpting amplitude, phase, polarization, and/or dispersion of optical wavefronts. In this Review, we discuss state-of-the-art developments as well as emerging trends in computational meta-structures as disruptive platforms for spatial optical analog computation. Two fundamental approaches based on general concepts of spatial Fourier transformation and Green's function are discussed in detail. Moreover, numerical investigations and experimental demonstrations of computational optical surfaces and meta-structures for solving a diverse set of mathematical problems (e.g., integro-differentiation and convolution equations) necessary for on-demand applications (e.g., edge detection) are reviewed. Finally, we explore the current challenges and the potential resolutions in computational meta-optics followed by our perspective on future research directions and possible developments in this promising area.
Multilanguage-handwriting self-powered recognition based on triboelectric nanogenerator enabled machine learning

Handwriting signature is widely used and the main challenge for handwriting recognition is how to obtain comprehensive handwriting information. Triboelectric nanogenerator is sensitive to external triggering force and can be used to record personal handwriting signals and associated characteristics. In this work, micro/nano structure textured TENG acting as a smart self-powered handwriting pad is developed and its effectiveness for handwriting recognition is demonstrated. Three individuals' handwriting signals of English words, Chinese characters and Arabic numerals are acquired by leaf-inspired TENG, and the other three people's handwriting signals of English sentences and the corresponding Chinese sentences are obtained by cylindrical microstructured PDMS based TENG, and these signals exhibit unique features. Combined with the machine learning method, the people's handwriting was successfully identified. The classification accuracies of 99.66%, 93.63%, 91.36%, 99.05%, and 97.73% were reached for English words, Arabic numerals, Chinese characters, English sentences, and the corresponding Chinese sentences, respectively. The results strongly suggested that the textured TENG exhibited great potential in personal handwriting signature identification, security defense, and private information protection applications.
TRRespass: Exploiting the Many Sides of Target Row Refresh

After a plethora of high-profile RowHammer attacks, CPU and DRAM vendors scrambled to deliver what was meant to be the definitive hardware solution against the RowHammer problem: Target Row Refresh (TRR). A common belief among practitioners is that, for the latest generation of DDR4 systems that are protected by TRR, RowHammer is no longer an issue in practice. However, in reality, very little is known about TRR. How does TRR exactly prevent RowHammer? Which parts of a system are responsible for operating the TRR mechanism? Does TRR completely solve the RowHammer problem or does it have weaknesses? In this paper, we demystify the inner workings of TRR and debunk its security guarantees. We show that what is advertised as a single mitigation mechanism is actually a series of different solutions coalesced under the umbrella term Target Row Refresh. We inspect and disclose, via a deep analysis, different existing TRR solutions and demonstrate that modern implementations operate entirely inside DRAM chips. Despite the difficulties of analyzing in-DRAM mitigations, we describe novel techniques for gaining insights into the operation of these mitigation mechanisms. These insights allow us to build TRRespass, a scalable black-box RowHammer fuzzer that we evaluate on 42 recent DDR4 modules. TRRespass shows that even the latest generation DDR4 chips with in-DRAM TRR, immune to all known RowHammer attacks, are often still vulnerable to new TRR-aware variants of RowHammer that we develop. In particular, TRRespass finds that, on present-day DDR4 modules, RowHammer is still possible when many aggressor rows are used (as many as 19 in some cases), with a method we generally refer to as Many-sided RowHammer. Overall, our analysis shows that 13 out of the 42 modules from all three major DRAM vendors (i.e., Samsung, Micron, and Hynix) are vulnerable to our TRR-aware RowHammer access patterns, and thus one can still mount existing state-of-the-art system-level RowHammer attacks. In addition to DDR4, we also experiment with LPDDR4(X) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> chips and show that they are susceptible to RowHammer bit flips too. Our results provide concrete evidence that the pursuit of better RowHammer mitigations must continue.
Sensor-Based Continuous Authentication of Smartphones’ Users Using Behavioral Biometrics: A Contemporary Survey

Mobile devices and technologies have become increasingly popular, offering comparable storage and computational capabilities to desktop computers allowing users to store and interact with sensitive and private information. The security and protection of such personal information are becoming more and more important since mobile devices are vulnerable to unauthorized access or theft. User authentication is a task of paramount importance that grants access to legitimate users at the point of entry and continuously through the usage session. This task is made possible with today's smartphones' embedded sensors that enable continuous and implicit user authentication by capturing behavioral biometrics and traits. In this article, we survey more than 140 recent behavioral biometric-based approaches for continuous user authentication, including motion-based methods (28 studies), gait-based methods (19 studies), keystroke dynamics-based methods (20 studies), touch gesture-based methods (29 studies), voice-based methods (16 studies), and multimodal-based methods (34 studies). The survey provides an overview of the current state-of-the-art approaches for continuous user authentication using behavioral biometrics captured by smartphones' embedded sensors, including insights and open challenges for adoption, usability, and performance.
Using Dark States to Charge and Stabilize Open Quantum Batteries

We introduce an open quantum battery protocol using dark states to achieve both superextensive capacity and power density, with non-interacting spins coupled to a reservoir. Further, our power density actually scales with the of number of spins $N$ in the battery. We show that the enhanced capacity and power is correlated with entanglement. Whilst connected to the charger, the charged state of the battery is a steady state, stabilized through quantum interference in the open system.
Peer-to-Peer Energy Trading in Virtual Power Plant Based on Blockchain Smart Contracts

A novel Peer-to-peer (P2P) energy trading scheme for a Virtual Power Plant (VPP) is proposed by using Smart Contracts on Ethereum Blockchain Platform. The P2P energy trading is the recent trend the power society is keen to adopt carrying out several trial projects as it eases to generate and share the renewable energy sources in a distributed manner inside local community. Blockchain and smart contracts are the up-and-coming phenomena in the scene of the information technology used to be considered as the cutting-edge research topics in power systems. Earlier works on P2P energy trading including and excluding blockchain technology were focused mainly on the optimization algorithm, Information and Communication Technology, and Internet of Things. Therefore, the financial aspects of P2P trading in a VPP framework is focused and in that regard a P2P energy trading mechanism and bidding platform are developed. The proposed scheme is based on public blockchain network and auction is operated by smart contract addressing both cost and security concerns. The smart contract implementation and execution in a VPP framework including bidding, withdrawal, and control modules developments are the salient feature of this work. The proposed architecture is validated using realistic data with the Ethereum Virtual Machine (EVM) environment of Ropsten Test Network.
Chaotic image encryption algorithm based on hybrid multi-objective particle swarm optimization and DNA sequence

This paper studies an image encryption algorithm based on multi-objective particle swarm optimization (MOPSO), DNA encoding sequence and one-dimensional Logistic map. First, the key of this paper consists of the sub-key sequence selected by particle swarm optimization (PSO), a hash value of the plaintext image, and a shuffle mark bit. Create random DNA mask images using Logistic map and DNA encoding. Then use it and the block-shuffled plaintext DNA encoding sequence to operate to form an encryption system. In PSO, the position value of a particle represents a position of the plaintext image, the iterative PSO algorithm is based on the information entropy and correlation coefficient. Finally, obtains the best ciphertext, and returns the value of the best particle at this time. Simulation experiment and security analysis show that the correlation coefficient and entropy of ciphertext are excellent, and it can resist all kinds of typical attacks and has better encryption effect.
Designing a blockchain enabled supply chain

While blockchain technologies are gaining momentum within supply chains, academic understanding of concrete, real-life design and implementation is still lagging, hence offering very limited insights into the true implications of blockchain technology on supply chains. This paper reports a two-year design science research (DSR) study of a smart contract initiative piloted by a consortium in the UK’s construction sector. We seek answers to the research question, ‘How should a blockchain enabled supply chain be designed?’ Guided by the theory of business model, we explore how a group of supply chain actors collectively designs and pilots a blockchain solution that addresses the supply chain transparency and provenance problem. Our research is one of the very few longitudinal empirical studies to offer in-depth evidence about how blockchain is deployed in complex multi-tier supply chain networks. In compliance with DSR research paradigm, we make contributions at three levels: designing and instantiating the blockchain architect and proving its utility in addressing the target problem; developing a set of design principles as a mid-range theory that can be applied and tested in different blockchain supply chain contexts; and refining and extending the kernel theory of business value at supply chain network level.
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots

In this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (SA-BERT), is proposed in order to make the model aware of the speaker change information, which is an important and intrinsic property of multi-turn dialogues. Furthermore, a speaker-aware disentanglement strategy is proposed to tackle the entangled dialogues. This strategy selects a small number of most important utterances as the filtered context according to the speakers' information in them. Finally, domain adaptation is performed to incorporate the in-domain knowledge into pre-trained language models. Experiments on five public datasets show that our proposed model outperforms the present models on all metrics by large margins and achieves new state-of-the-art performances for multi-turn response selection.
Few-Shot Named Entity Recognition via Meta-Learning

Few-shot learning under the <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -way <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -shot setting (i.e., <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> annotated samples for each of <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> classes) has been widely studied in relation extraction (e.g., FewRel) and image classification (e.g., Mini-ImageNet). Named entity recognition (NER) is typically framed as a sequence labeling problem where the entity classes are inherently entangled together because the entity number and classes in a sentence are not known in advance, leaving the <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -way <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -shot NER problem so far unexplored. In this paper, we first formally define a more suitable <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -way <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -shot setting for NER. Then we propose <small>FewNER</small> , a novel meta-learning approach for few-shot NER. <small>FewNER</small> separates the entire network into a task-independent part and a task-specific part. During training in <small>FewNER</small> , the task-independent part is meta-learned across multiple tasks and the task-specific part is learned for each individual task in a low-dimensional space. At test time, <small>FewNER</small> keeps the task-independent part fixed and adapts to a new task via gradient descent by updating only the task-specific part, resulting in it being less prone to overfitting and more computationally efficient. Compared with pre-trained language models (e.g., BERT and ELMo) which obtain the transferability in an implicit manner (i.e., relying on large-scale corpora), <small>FewNER</small> explicitly optimizes the capability of “learning to adapt quickly” through meta-learning. The results demonstrate that <small>FewNER</small> achieves state-of-the-art performance against nine baseline methods by significant margins on three adaptation experiments (i.e., intra-domain cross-type, cross-domain intra-type and cross-domain cross-type).
House-GAN: Relational Generative Adversarial Networks for Graph-Constrained House Layout Generation

This paper proposes a novel graph-constrained generative adversarial network, whose generator and discriminator are built upon relational architecture. The main idea is to encode the constraint into the graph structure of its relational networks. We have demonstrated the proposed architecture for a new house layout generation problem, whose task is to take an architectural constraint as a graph (i.e., the number and types of rooms with their spatial adjacency) and produce a set of axis-aligned bounding boxes of rooms. We measure the quality of generated house layouts with the three metrics: the realism, the diversity, and the compatibility with the input graph constraint. Our qualitative and quantitative evaluations over 117,000 real floorplan images demonstrate that the proposed approach outperforms existing methods and baselines. We will publicly share all our code and data.
Long-Term Human Motion Prediction with Scene Context

Human movement is goal-directed and influenced by the spatial layout of the objects in the scene. To plan future human motion, it is crucial to perceive the environment – imagine how hard it is to navigate a new room with lights off. Existing works on predicting human motion do not pay attention to the scene context and thus struggle in long-term prediction. In this work, we propose a novel three-stage framework that exploits scene context to tackle this task. Given a single scene image and 2D pose histories, our method first samples multiple human motion goals, then plans 3D human paths towards each goal, and finally predicts 3D human pose sequences following each path. For stable training and rigorous evaluation, we contribute a synthetic dataset with clean annotations. In both synthetic and real datasets, our method shows consistent quantitative and qualitative improvements over existing methods. Project page: https://people.eecs.berkeley.edu/~zhecao/hmp/index.html (Please refer to our arXiv for a longer version of the paper with more visualizations.)