Reviewerly/exHarmony huggingface dataset:
Virtualizing a Post-Moore’s Law Analog Mesh Processor: The Case of a Photonic PDE Accelerator

Innovative processor architectures aim to play a critical role in future sustainment of performance improvements under severe limitations imposed by the end of Moore’s Law. The Reconfigurable Optical Computer (ROC) is one such innovative, Post-Moore’s Law processor. ROC is designed to solve partial differential equations in one shot as opposed to existing solutions, which are based on costly iterative computations. This is achieved by leveraging physical properties of a mesh of optical components that behave analogously to lumped electrical components. However, virtualization is required to combat shortfalls of the accelerator hardware. Namely, (1) the infeasibility of building large photonic arrays to accommodate arbitrarily large problems and (2) underutilization brought about by mismatches in problem and accelerator mesh sizes due to future advances in manufacturing technology. In this work, we introduce an architecture and methodology for lightweight virtualization of ROC that exploits advantages borne from optical computing technology. Specifically, we apply temporal and spatial virtualization to ROC and then extend the accelerator scheduling tradespace with the introduction of spectral virtualization. Additionally, we investigate multiple resource scheduling strategies for a system-on-chip (SoC)-based PDE acceleration architecture and show that virtual configuration management offers a speedup of approximately 2×. Finally, we show that overhead from virtualization is minimal, and our experimental results show two orders of magnitude increased speed as compared to microprocessor execution while keeping errors due to virtualization under 10%.
A Self-guided Framework for Radiology Report Generation

Automatic radiology report generation is essential to computer-aided diagnosis. Through the success of image captioning, medical report generation has been achievable. However, the lack of annotated disease labels is still the bottleneck of this area. In addition, the image-text data bias problem and complex sentences make it more difficult to generate accurate reports. To address these gaps, we present a self-guided framework (SGF), a suite of unsupervised and supervised deep learning methods to mimic the process of human learning and writing. In detail, our framework obtains the domain knowledge from medical reports without extra disease labels and guides itself to extract fined-grain visual features associated with the text. Moreover, SGF successfully improves the accuracy and length of medical report generation by incorporating a similarity comparison mechanism that imitates the process of human self-improvement through comparative practice. Extensive experiments demonstrate the utility of our SGF in the majority of cases, showing its superior performance over state-of-the-art methods. Our results highlight the capacity of the proposed framework to distinguish fined-grained visual details between words and verify its advantage in generating medical reports.
A Relative Information Gain-based Query Performance Prediction Framework with Generated Query Variants

Query performance prediction (QPP) methods, which aim to predict the performance of a query, often rely on evidences in the form of different characteristic patterns in the distribution of Retrieval Status Values (RSVs). However, for neural IR models, it is usually observed that the RSVs are often less reliable for QPP because they are bounded within short intervals, different from the situation for statistical models. To address this limitation, we propose a model-agnostic QPP framework that gathers additional evidences by leveraging information from the characteristic patterns of RSV distributions computed over a set of automatically generated query variants, relative to that of the current query. Specifically, the idea behind our proposed method—Weighted Relative Information Gain (WRIG), is that a substantial relative decrease or increase in the standard deviation of the RSVs of the query variants is likely to be a relative indicator of how easy or difficult the original query is. To cater for the absence of human-annotated query variants in real-world scenarios, we further propose an automatic query variant generation method. This can produce variants in a controlled manner by substituting terms from the original query with new ones sampled from a weighted distribution, constructed either via a relevance model or with the help of an embedded representation of query terms. Our experiments on the TREC-Robust, ClueWeb09B, and MS MARCO datasets show that WRIG, by the use of this relative changes in QPP estimate, leads to significantly better results than a state-of-the-art baseline method that leverages information from (manually created) query variants by the application of additive smoothing [ 64 ]. The results also show that our approach can improve the QPP effectiveness of neural retrieval approaches in particular.
Consider the Consequences: A Risk Assessment Approach for Industrial Control Systems

The development of information and communication technologies extended the application of digitalized industrial control systems (ICSs) to critical infrastructure. With this circumstance, emerging sophisticated cyberattacks by adversaries, including nation-backed terrorists, target ICSs due to their strategic value that critical infrastructure can cause severe consequences to equipment, people, and the environment due to the cyberattacks on ICSs. Therefore, critical infrastructure owners should provide high assurance to those involved, such as neighboring residents and governments, that the facility is adequately protected against cyberattacks. The risk assessment that identifies, estimates, and prioritizes risks is vital to provide high assurance. This study proposes a framework for evaluating risks by quantifying the likelihood of cyber exploitation and the consequences of cyberattacks. The quantification of the likelihood of cyber exploitation is inspired by research on Bayesian attack graphs (BAGs), allowing probability evaluation that considers the causal relationship between ICSs and multistage attacks. For the cyberattack consequences quantification, we propose a methodology to evaluate how far an impact will spread and thus how many functions will be influenced when an ICS is exploited. The methodology is conducted by ICS experts identifying and listing functional dependencies and essential function goals among ICSs that they are already familiar with and do not require in-depth cybersecurity knowledge. Through experiments, we demonstrated how to apply our framework to assess the risks of the plant protection system, which is a safety-grade digital system used in nuclear power plants. The result shows that risk can be multidimensionally assessed than previous literature, such as discovering that components that were not considered important have high risk due to their functional connectivity.
Industrial character recognition based on improved CRNN in complex environments

Automatic character recognition is being gradually adopted in several everyday applications. In industrial settings, it can free up manpower required for reading and tracking product information. However, character recognition in industrial environments is particularly challenging. The main challenges are as follows: (1) There are no publicly available datasets that would aid the development of robust industrial character recognition methods; (2) Industrial characters may be printed in complex and uneven shapes on the surface of various materials, resulting in blurred characters, low contrast and distortion; (3) Industrial character recognition in complex industrial environments can suffers from various difficulties, such as uneven lighting conditions, oxidation of characters caused by long-term storage, etc. To address these challenges, we propose a convolutional recurrent neural network based on a residual structure and a "Squeeze-and-Excitation" block, namely RS-CRNN. The feature extraction of this method is inspired by ResNet and SEnet, and the method uses gate recurrent units for sequence label prediction. The model was trained and tested on constructed real and synthetic datasets, and experimental results show that the method has better performance than state-of-the-art character recognition models.
TeleFest: Augmented Virtual Teleportation for Live Concerts

We present TeleFest, a novel system for live-streaming mixed reality 360° videos to online streaming services. TeleFest allows a producer to control multiple cameras in real time, providing viewers with different locations for experiencing the concert, and an intermediate software stack allows virtual content to be overlaid with coherent illumination that matches the real-world footage. TeleFest was evaluated by livestreaming a concert to almost 2,000 online viewers, allowing them to watch the performance from the crowd, the stage, or via a catered experience controlled by a producer in real time that included camera switching and augmented content. The results of an online survey completed by virtual and physical attendees of the festival are presented, showing positive feedback for our setup and suggesting that the addition of virtual and immersive content to live events could lead to a more enjoyable experience for viewers.
Do self-supervised speech models develop human-like perception biases?

Self-supervised models for speech processing form representational spaces without using any external labels.Increasingly, they appear to be a feasible way of at least partially eliminating costly manual annotations, a problem of particular concern for low-resource languages.But what kind of representational spaces do these models construct?Human perception specializes to the sounds of listeners' native languages.Does the same thing happen in self-supervised models?We examine the representational spaces of three kinds of stateof-the-art self-supervised models: wav2vec 2.0, HuBERT and contrastive predictive coding (CPC), and compare them with the perceptual spaces of French-speaking and Englishspeaking human listeners, both globally and taking account of the behavioural differences between the two language groups.We show that the CPC model shows a small native language effect, but that wav2vec 2.0 and Hu-BERT seem to develop a universal speech perception space which is not language specific.A comparison against the predictions of supervised phone recognisers suggests that all three self-supervised models capture relatively finegrained perceptual phenomena, while supervised models are better at capturing coarser, phone-level, effects of listeners' native language, on perception.
Camouflaged Poisoning Attack on Graph Neural Networks

Graph neural networks (GNNs) have enabled the automation of many web applications that entail node classification on graphs, such as scam detection in social media and event prediction in service networks. Nevertheless, recent studies revealed that the GNNs are vulnerable to adversarial attacks, where feeding GNNs with poisoned data at training time can lead them to yield catastrophically devastative test accuracy. This finding heats up the frontier of attacks and defenses against GNNs. However, the prior studies mainly posit that the adversaries can enjoy free access to manipulate the original graph, while obtaining such access could be too costly in practice. To fill this gap, we propose a novel attacking paradigm, named Generative Adversarial Fake Node Camouflaging (GAFNC), with its crux lying in crafting a set of fake nodes in a generative-adversarial regime. These nodes carry camouflaged malicious features and can poison the victim GNN by passing their malicious messages to the original graph via learned topological structures, such that they 1) maximize the devastation of classification accuracy (i.e., global attack) or 2) enforce the victim GNN to misclassify a targeted node set into prescribed classes (i.e., target attack). We benchmark our experiments on four real-world graph datasets, and the results substantiate the viability, effectiveness, and stealthiness of our proposed poisoning attack approach. Code is released in github.com/chao92/GAFNC.
TD-DNN: A Time Decay-Based Deep Neural Network for Recommendation System

In recent years, commercial platforms have embraced recommendation algorithms to provide customers with personalized recommendations. Collaborative Filtering is the most widely used technique of recommendation systems, whose accuracy is primarily reliant on the computed similarity by a similarity measure. Data sparsity is one problem that affects the performance of the similarity measures. In addition, most recommendation algorithms do not remove noisy data from datasets while recommending the items, reducing the accuracy of the recommendation. Furthermore, existing recommendation algorithms only consider historical ratings when recommending the items to users, but users’ tastes may change over time. To address these issues, this research presents a Deep Neural Network based on Time Decay (TD-DNN). In the data preprocessing phase of the model, noisy ratings are detected from the dataset and corrected using the Matrix Factorization approach. A power decay function is applied to the preprocessed input to provide more weightage to the recent ratings. This non-noisy weighted matrix is fed into the Deep Learning model, consisting of an input layer, a Multi-Layer Perceptron, and an output layer to generate predicted ratings. The model’s performance is tested on three benchmark datasets, and experimental results confirm that TD-DNN outperforms other existing approaches.
Unified dual-label semi-supervised learning with top-k feature selection

Semi-supervised feature selection alleviates the annotation burden of supervised feature learning by exploiting data under a handful of supervision information. The mainstream technique is to employ a linear regression framework that jointly learns labeled and unlabeled samples. However, existing approaches always encounter the deficiencies in two aspects: 1) the performance of models are severely degenerated once predicted labels are unreliable; 2) the balance of objectives in regards to two types of data are not well considered. In the article, we propose unified dual-label semi-supervised learning for top-k feature selection. The technique defines a soft label matrix to indicate the probability of samples belonging to each class. From the probability, the model could recognize unclassifiable samples that lay around the boundaries. Meanwhile, the label matrix is equipped with an exponent parameter γ. It endows the soft labels dual effects that the labeled and unlabeled data are tactfully discriminated. For the purpose of feature selection, we impose the ℓ2,0-norm constraint on the projection matrix, such that the exact top-k features are picked out. An iteration algorithm is designed to solve the given problem, by which large-scale data are facilely tackled. We conduct experiments that validate the superiority of the proposed method against the state-of-the-art competitors.
Kalman Filtering in Non-Gaussian Model Errors: A New Perspective [Tips &amp; Tricks]

It is well known that the optimality of the Kalman filter relies on the Gaussian distribution of process and observation model errors, which in many situations is well justified <xref ref-type="bibr" rid="ref1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[1]</xref> – <xref ref-type="bibr" rid="ref2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"/> <xref ref-type="bibr" rid="ref3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[3]</xref> . However, this optimality is useless in applications where the distribution assumptions of the model errors do not hold in practice. Even minor deviations from the assumed (or nominal) distribution may cause the Kalman filter’s performance to drastically degrade or completely break down. In particular, when dealing with perceptually important signals, such as speech, image, medical, campaign, and ocean engineering, measurements have confirmed the presence of non-Gaussian impulsive (heavy-tailed) and Laplace noises <xref ref-type="bibr" rid="ref4" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[4]</xref> . Therefore, the classical Kalman filter, which is derived under the nominal Gaussian probability model, is biased and even fails in such situations.
GRACE: A General Graph Convolution Framework for Attributed Graph Clustering

Attributed graph clustering (AGC) is an important problem in graph mining as more and more complex data in real-world have been represented in graphs with attributed nodes. While it is a common practice to leverage both attribute and structure information for improved clustering performance, most existing AGC algorithms consider only a specific type of relations, which hinders their applicability to integrate various complex relations into node attributes for AGC. In this article, we propose GRACE, an extended graph convolution framework for AGC tasks. Our framework provides a general and interpretative solution for clustering many different types of attributed graphs, including undirected, directed, heterogeneous and hyper attributed graphs. By building suitable graph Laplacians for each of the aforementioned graph types, GRACE can seamlessly perform graph convolution on node attributes to fuse all available information for clustering. We conduct extensive experiments on 14 real-world datasets of four different graph types. The experimental results show that GRACE outperforms the state-of-the-art AGC methods on the different graph types in terms of clustering quality, time, and memory usage.
Developing an evaluation framework for vocabulary-learning apps

Mobile technology is one of the most important approaches to facilitating vocabulary learning. However, despite an increasing use of mobile devices for this purpose, there is currently no dedicated evaluation framework to examine whether the vocabulary-learning applications on the market are well designed. Building on prior research from the spheres of instructional design, educational technology, and vocabulary learning, this study developed a six-component evaluation framework for vocabulary-learning apps, consisting of learning tasks, goal clarity, feedback, strategy instruction, repetition, and instructional control. Then, this framework was used to review 30 apps aimed at teaching English vocabulary to non-native speakers, and to inform various recommendations for future app development.
Interpretable Detection of Distribution Shifts in Learning Enabled Cyber-Physical Systems

The use of learning based components in cyber-physical systems (CPS) has created a gamut of possible avenues to use high dimensional real world signals generated from sensors like camera and LiDAR. The ability to process such signals can be largely attributed to the adoption of high-capacity function approximators like deep neural networks. However, this does not come without its potential perils. The pitfalls arise from possible over-fitting, and subsequent unsafe behavior when exposed to unknown environments. One challenge is that, in high dimensional input spaces it is almost impossible to experience enough training data in the design phase. What is required here, is an efficient way to flag out-of-distribution (OOD) samples that is precise enough to not raise too many false alarms. In addition, the system needs to be able to detect these in a computationally efficient manner at runtime. In this paper, our proposal is to build good representations for in-distribution data. We introduce the idea of a memory bank to store prototypical samples from the input space. We use these memories to compute probability density estimates using kernel density estimation techniques. We evaluate our technique on two challenging scenarios : a self-driving car setting implemented inside the simulator CARLA with image inputs, and an autonomous racing car navigation setting, with LiDAR inputs. In both settings, it was observed that a deviation from indistribution setting can potentially lead to deviation from safe behavior. An added benefit of using training samples as memories to detect out-of-distribution inputs is that the system is interpretable to a human operator. Explanation of this nature is generally hard to obtain from pure deep learning based alter-natives. Our code for reproducing the experiments is available at https://github.com/yangy96/interpretable_ood_detection.git
CNN based “Text in Image” Steganography using Slice Encryption Algorithm and LWT

This paper proposes CNN based "text in image" steganography using slice encryption algorithm and LWT technique. Generally, cryptography and steganography techniques are used to achieve the covert communication. However, these techniques may not provide good security to the secret data individually. This paper brings the solution by the combination of cryptography, steganography and eight layer CNN. The slice encryption algorithm generates a less length effective ciphertext when compared to the actual length of the readable secret data. This is because; the algorithm uses unique data conversion (DC) and "gray-code" techniques. Next, the new architecture of eight layer CNN is used to generate the feature map in such a way that it accepts the high capacity of ciphertext. Finally, two-level LWT provides the sufficient number of sub-bands which helps to generate high perceptual quality of stego image and good quality metric values. The experimental results shown that the stego image generated from proposed framework have good quality metric values for various payload capacities when compared to existing methods. Finally, the stego image generated with these combinations (Slice, CNN & LWT) yields more payload capacity, high perceptual quality, good quality metric values and can able to withstand for various attacks.
lifeXplore at the Lifelog Search Challenge 2022

Lifelogging creates substantial data archives that are challenging to manage and search. The annual Lifelog Search Challenge (LSC) aims at improving this situation by encouraging international teams to create interactive retrieval systems for searching large lifelog databases. The LSC challenge is held as a live event co-located at the ACM International Conference on Multimedia Retrieval (ICMR), where participating teams compete against each other by solving time-based retrieval tasks. In this paper, we present an improved version of lifeXplore -- our system already participating since LSC2018. For LSC2022, we focus on improving the result presentation as well as the system's interface.
SoK: Oblivious Pseudorandom Functions

In recent years, oblivious pseudorandom functions (OPRFs) have become a ubiquitous primitive used in cryptographic protocols and privacy-preserving technologies. The growing interest in OPRFs, both theoretical and applied, has produced a vast number of different constructions and functionality variations. In this paper, we provide a systematic overview of how to build and use OPRFs. We first categorize existing OPRFs into essentially four families based on their underlying PRF (Naor-Reingold, Dodis-Yampolskiy, Hashed Diffie-Hellman, and generic constructions). This categorization allows us to give a unified presentation of all oblivious evaluation methods in the literature, and to understand which properties OPRFs can (or cannot) have. We further demonstrate the theoretical and practical power of OPRFs by visualizing them in the landscape of cryptographic primitives, and by providing a comprehensive overview of how OPRFs are leveraged for improving the privacy of internet users. Our work systematizes 15 years of research on OPRFs and provides inspiration for new OPRF constructions and applications thereof.