Reviewerly/exHarmony huggingface dataset:
Extra-foveal Processing of Object Semantics Guides Early Overt Attention During Visual Search

Eye-tracking studies using arrays of objects have demonstrated that some high-level processing of object semantics can occur in extra-foveal vision, but its role on the allocation of early overt attention is still unclear. This eye-tracking visual search study contributes novel findings by examining the role of object-to-object semantic relatedness and visual saliency on search responses and eye-movement behaviour across arrays of increasing size (3, 5, 7). Our data show that a critical object was looked at earlier and for longer when it was semantically unrelated than related to the other objects in the display, both when it was the search target (target-present trials) and when it was a target's semantically related competitor (target-absent trials). Semantic relatedness effects manifested already during the very first fixation after array onset, were consistently found for increasing set sizes, and were independent of low-level visual saliency, which did not play any role. We conclude that object semantics can be extracted early in extra-foveal vision and capture overt attention from the very first fixation. These findings pose a challenge to models of visual attention which assume that overt attention is guided by the visual appearance of stimuli, rather than by their semantics.
Computable genuine multimode entanglement measure: Gaussian versus non-Gaussian

Genuine multimode entanglement in continuous variable systems can be quantified by exploring the geometry of the state-space, namely via the generalized geometric measure (GGM) which is defined as the shortest distance of a given multimode state from a nongenunely multimode entangled state. For multimode Gaussian states, we derive a closed form expression of GGM in terms of the symplectic invariants of the reduced states. Following that prescription, the characteristics of GGM for typical three- and four-mode Gaussian states are investigated. In the non-Gaussian paradigm, we compute GGM for photon-added as well as -subtracted states having three- and four-modes and find that both addition and subtraction of photons lead to enhancement of the genuine multimode entanglement content of the state compared to its Gaussian counterpart. Our analysis reveals that when an initial three-mode vacuum state is evolved according to an interacting Hamiltonian, photon addition is more beneficial in increasing GGM compared to photon subtraction while the picture is opposite for four-mode case. Specifically, subtracting photons from four-mode squeezed vacuum states almost always results in higher multimode entanglement content than that of photon addition with both single as well as multimode and constrained as well as unconstrained operations. Furthermore, we observe a novel freezing feature of GGM during some specific cases of photon subtraction with respect to the number of photons subtracted.
V4D: 4D Covolutional Neural Networks for Video-level Representations Learning

Most existing 3D CNN structures for video representation learning are clip-based methods, and do not consider video-level temporal evolution of spatio-temporal features. In this paper, we propose Video-level 4D Convolutional Neural Networks, namely V4D, to model the evolution of long-range spatio-temporal representation with 4D convolutions, as well as preserving 3D spatio-temporal representations with residual connections. We further introduce the training and inference methods for the proposed V4D. Extensive experiments are conducted on three video recognition benchmarks, where V4D achieves excellent results, surpassing recent 3D CNNs by a large margin.
Conflict Detection in Rule Based IoT Systems

An IoT environment with a complex rule structure containing a large number of sensing and actuating devices increases the chances of creating rules with faulty behaviors. An IoT system which contains conflicting and incomplete rules is one example of such faulty systems. In this paper, we present a mechanism to detect conflicting and incomplete rules. First, we introduce a rule conflict mechanism with three categories to detect rule conflicts. Second, we define rule incompleteness and introduce a method to detect incomplete rules. Third, we present two methods to resolve rule conflicts. We demonstrate our approach in several scenarios.
A Step Towards the Optimal Estimation of Image Orientation

Generally speaking, the orientation of an image can easily be recognized by human beings.On the other hand, recognizing the orientation of an image is one of the challenges facing digital image processing.Several attempts have been made in this research area.However, the optimal solution still needs to be determined.For such a solution, the main issue is to obtain the precise angle for possible rotation.Numerous attempts have been made to address this issue; some of them mainly focus on coarse angles, whereas others have applied fuzzy logic in order to determine more precise angles.In this paper, a two-stage method is introduced in which a coarse angle estimation is achieved through the use of the Convolutional Neural Network (CNN) approach, and a more precise angle is acquired via fuzzy logic.An extensive evaluation of the proposed method is carried out on different public datasets.The results indicate an outstanding level of performance in terms of optimizing image orientation.
Molo.news: Experimentally Developing a Relational Platform for Local Journalism

In this article we present a research project that experimentally develops a local news platform based on empirical research (interviews, group discussions, a survey) and a co-creation approach. What is presented here is not a typical empirical social science research study but the culmination of an entire approach that is oriented toward software development. This article’s aim is to present the project’s conceptual ideas, its interdisciplinary character, its research-based development approach and the concept for a local news platform that grew out of our preliminary work. At each level we focus on the &lt;em&gt;relationality&lt;/em&gt; which arises in the figurations of the actors involved and their various perspectives. First, we illustrate how relationality already shaped the objective of our project and how this results in its interdisciplinary structure and research design. We then discuss this idea with reference to our empirical findings, that is, the paradox of the local public sphere: While all the actors we interviewed—those who (professionally) produce content and those who use it—have a high appreciation for the idea of a local public sphere, the mediated connection to this sphere is diminishing at the same time. We understand this as the real challenge for local journalism and the local public sphere at large, and not just for individual media organizations. This is also the reason why we argue for a fundamentally relational approach: from a theoretical point of view, it can be used to grasp the crisis of the local public; from a practical point of view, relationality represents the core characteristic of the platform in development. On this basis, we will then show how the concept of the experimental local news platform evolved through the use of a prototype as a relational boundary object. This development lead to the conceptualization of the platform &lt;em&gt;molo.news&lt;/em&gt; which itself is characterized by a fourfold relationality. Our concluding argument is that approaching relationality in a more rigorous way could be the key to exploring the future of local journalism.
Provably Efficient Reinforcement Learning with Aggregated States

We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mathcal{O}}(\sqrt{H^5 M K} + \epsilon HK)$, where $H$ is the horizon, $M$ is the number of aggregate states, $K$ is the number of episodes, and $\epsilon$ is the largest difference between any pair of optimal state-action values associated with a common aggregate state. Notably, this regret bound does not depend on the number of states or actions and indicates that asymptotic per-period regret is no greater than $\epsilon$, independent of horizon. To our knowledge, this is the first such result that applies to reinforcement learning with nontrivial value function approximation without any restrictions on transition probabilities.
Graph Message Passing with Cross-location Attentions for Long-term ILI Prediction

Forecasting influenza-like illness (ILI) is of prime importance to epidemiologists and health-care providers. Early prediction of epidemic outbreaks plays a pivotal role in disease intervention and control. Most existing work has either limited long-term prediction performance or lacks a comprehensive ability to capture spatio-temporal dependencies in data. Accurate and early disease forecasting models would markedly improve both epidemic prevention and managing the onset of an epidemic. In this paper, we design a cross-location attention based graph neural network (Cola-GNN) for learning time series embeddings and location aware attentions. We propose a graph message passing framework to combine learned feature embeddings and an attention matrix to model disease propagation over time. We compare the proposed method with state-of-the-art statistical approaches and deep learning models on real-world epidemic-related datasets from United States and Japan. The proposed method shows strong predictive performance and leads to interpretable results for long-term epidemic predictions.
A Market Place Solution for Energy Transaction on Ethereum Blockchain

In the recent decade, self-consumption and local renewable energy communities are growing, especially in California, North Carolina, Arizona and Nevada. In these communities, approximately twenty percent of electricity comes from solar generation. However, what happens if a household generates more than what they consume? In most cases, the excess energy would go back to the grid with little to no compensation given by the energy utility. In today's market, there are countless homeowners who would like to profit from their own PV system. Unfortunately, there is no energy marketplace that would allow homeowners to sell or purchase energy to and from their neighbors. Luckily with blockchain, this idea becomes a reality. With the use of distributed ledger technology, it is possible to perform energy transactions in near-real time from a prosumer to consumer without the need of a central authority. Our thesis suggests ways to handle the facilitation of energy transactions using an Ethereum blockchain system in an effortless and efficient way.
The Placement of Virtual Machines Under Optimal Conditions in Cloud Datacenter

Nowadays cloud computing is progressing very fast and has resulted in advances in other technologies too. Cloud computing provides quite a convenient platform for millions of users to use computing resources through the internet. Cloud computing provides the possibility of only concentrating on business goals instead of expanding hardware resources for users. Using virtualization technology in computing resources results in the efficient use of resources. A challenging work in virtualization technology is the placement of virtual machines under optimal conditions on physical machines in cloud data centers. Optimal placement of virtual machines on physical machines in cloud data centers can lead to the management of resources and prevention of the resources waste. In this paper, a new method is proffered based on the combination of hybrid discrete multi-object sine cosine algorithm and multi-verse optimizer for optimal placement. The first goal of the proposed approach is to decrease the power consumption which is consumed in cloud data centers by reducing active physical machines. The second goal is to cut in resource wastage and managing resources using the optimal placement of virtual machines on physical machines in cloud data centers. With this approach, the increasing rate of virtual migration to physical machines is prevented. Finally, the results gained from our proposed algorithm are compared to some algorithms like the first fit (FF), virtual machine placement ant colony system (VMPACS), modified best fit decreasing (MBFD).
Accessibility and IoT / Smart and Connected Communities

The Internet of things (IoT) has unlimited potential to empower the lives of everyone.IoT devices increasingly appear in homes and power smart and connected communities.Related user experience design efforts must involve and consider people with disabilities, including the world's rapidly aging population of seniors.They must be able to use IoT device and app interfaces.While secure and private IoT device data collection and communication are important for everyone, related needs that are unique to people with disabilities must be addressed.Many current resources and developing efforts that can benefit UX designers to address these needs exist.
Graph-Based Transformer with Cross-Candidate Verification for Semantic Parsing

In this paper, we present a graph-based Transformer for semantic parsing. We separate the semantic parsing task into two steps: 1) Use a sequence-to-sequence model to generate the logical form candidates. 2) Design a graph-based Transformer to rerank the candidates. To handle the structure of logical forms, we incorporate graph information to Transformer, and design a cross-candidate verification mechanism to consider all the candidates in the ranking process. Furthermore, we integrate BERT into our model and jointly train the graph-based Transformer and BERT. We conduct experiments on 3 semantic parsing benchmarks, ATIS, JOBS and Task Oriented semantic Parsing dataset (TOP). Experiments show that our graph-based reranking model achieves results comparable to state-of-the-art models on the ATIS and JOBS datasets. And on the TOP dataset, our model achieves a new state-of-the-art result.
General Partial Label Learning via Dual Bipartite Graph Autoencoder

We formulate a practical yet challenging problem: General Partial Label Learning (GPLL). Compared to the traditional Partial Label Learning (PLL) problem, GPLL relaxes the supervision assumption from instance-level -- a label set partially labels an instance -- to group-level: 1) a label set partially labels a group of instances, where the within-group instance-label link annotations are missing, and 2) cross-group links are allowed -- instances in a group may be partially linked to the label set from another group. Such ambiguous group-level supervision is more practical in real-world scenarios as additional annotation on the instance-level is no longer required, e.g., face-naming in videos where the group consists of faces in a frame, labeled by a name set in the corresponding caption. In this paper, we propose a novel graph convolutional network (GCN) called Dual Bipartite Graph Autoencoder (DB-GAE) to tackle the label ambiguity challenge of GPLL. First, we exploit the cross-group correlations to represent the instance groups as dual bipartite graphs: within-group and cross-group, which reciprocally complements each other to resolve the linking ambiguities. Second, we design a GCN autoencoder to encode and decode them, where the decodings are considered as the refined results. It is worth noting that DB-GAE is self-supervised and transductive, as it only uses the group-level supervision without a separate offline training stage. Extensive experiments on two real-world datasets demonstrate that DB-GAE significantly outperforms the best baseline over absolute 0.159 F1-score and 24.8% accuracy. We further offer analysis on various levels of label ambiguities.
Ecological Reinforcement Learning

Much of the current work on reinforcement learning studies episodic settings, where the agent is reset between trials to an initial state distribution, often with well-shaped reward functions. Non-episodic settings, where the agent must learn through continuous interaction with the world without resets, and where the agent receives only delayed and sparse reward signals, is substantially more difficult, but arguably more realistic considering real-world environments do not present the learner with a convenient "reset mechanism" and easy reward shaping. In this paper, instead of studying algorithmic improvements that can address such non-episodic and sparse reward settings, we instead study the kinds of environment properties that can make learning under such conditions easier. Understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable. We first discuss what we term "environment shaping" -- modifications to the environment that provide an alternative to reward shaping, and may be easier to implement. We then discuss an even simpler property that we refer to as "dynamism," which describes the degree to which the environment changes independent of the agent's actions and can be measured by environment transition entropy. Surprisingly, we find that even this property can substantially alleviate the challenges associated with non-episodic RL in sparse reward settings. We provide an empirical evaluation on a set of new tasks focused on non-episodic learning with sparse rewards. Through this study, we hope to shift the focus of the community towards analyzing how properties of the environment can affect learning and the ultimate type of behavior that is learned via RL.
Blockchain based Data Distribution and Traceability Framework in the Electric Information Management System

Blockchain technology is playing an increasingly important role in the Management Information System especially in the data security aspect. In this paper, we propose a blockchain based system for the data distribution and traceability analysis in Electric Management Information System (EMIS). The blockchain and smart contract is studied to ensure the security of data share in the system. The experimental results indicate that the proposed demo system can efficiently support the data share and supply the undeniable traceability services in EMIS.
Pairwise Learning with Differential Privacy Guarantees

Pairwise learning has received much attention recently as it is more capable of modeling the relative relationship between pairs of samples. Many machine learning tasks can be categorized as pairwise learning, such as AUC maximization and metric learning. Existing techniques for pairwise learning all fail to take into consideration a critical issue in their design, i.e., the protection of sensitive information in the training set. Models learned by such algorithms can implicitly memorize the details of sensitive information, which offers opportunity for malicious parties to infer it from the learned models. To address this challenging issue, in this paper, we propose several differentially private pairwise learning algorithms for both online and offline settings. Specifically, for the online setting, we first introduce a differentially private algorithm (called OnPairStrC) for strongly convex loss functions. Then, we extend this algorithm to general convex loss functions and give another differentially private algorithm (called OnPairC). For the offline setting, we also present two differentially private algorithms (called OffPairStrC and OffPairC) for strongly and general convex loss functions, respectively. These proposed algorithms can not only learn the model effectively from the data but also provide strong privacy protection guarantee for sensitive information in the training set. Extensive experiments on real-world datasets are conducted to evaluate the proposed algorithms and the experimental results support our theoretical analysis.
RT-Trust: Automated refactoring for different trusted execution environments under real-time constraints

Real-time systems must meet strict timeliness requirements. These systems also often need to protect their critical program information (CPI) from adversarial interference and intellectual property theft. Trusted execution environments (TEE) execute CPI tasks on a special-purpose processor, thus providing hardware protection. However, adapting a system written to execute in environments without TEE requires partitioning the code into untrusted and trusted parts. This process involves complex manual program transformations that are not only laborious and intellectually tiresome, but also hard to validate and verify adherence to real-time constraints. To address these problems, this paper presents novel program analyses and transformation techniques, accessible to the developer via a declarative meta-programming model. The developer declaratively specifies the CPI portion of the system. A custom static analysis checks CPI specifications for validity, while probe-based profiling helps identify whether the transformed system would continue to meet the original real-time constraints, with a feedback loop suggesting how to modify the code, so its CPI can be isolated. Finally, an automated refactoring isolates the CPI portion for TEE-based execution, communicated with through generated calls to the TEE API. The reference implementation of our approach profiles and transforms real-time systems to isolate their CPI functions to execute on two different TEE platforms: OP-TEE and SGX. Although these platforms substantially differ in terms of their respective APIs and performance characteristics, our refactoring completely hides these differences from the developer by automatically synthesizing the correct CPI functionality required for these dissimilar TEE implementations. We have evaluated our approach by successfully enabling the trusted execution of the CPI portions of several microbenchmarks and a drone autopilot. Our approach shows the promise of declarative meta-programming in reducing the programmer effort required to adapt systems for trusted execution under real-time constraints.
Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings

Reinforcement learning is a general technique that allows an agent to learn an optimal policy and interact with an environment in sequential decision making problems. The goodness of a policy is measured by its value function starting from some initial state. The focus of this paper is to construct confidence intervals (CIs) for a policy's value in infinite horizon settings where the number of decision points diverges to infinity. We propose to model the action-value state function (Q-function) associated with a policy based on series/sieve method to derive its confidence interval. When the target policy depends on the observed data as well, we propose a SequentiAl Value Evaluation (SAVE) method to recursively update the estimated policy and its value estimator. As long as either the number of trajectories or the number of decision points diverges to infinity, we show that the proposed CI achieves nominal coverage even in cases where the optimal policy is not unique. Simulation studies are conducted to back up our theoretical findings. We apply the proposed method to a dataset from mobile health studies and find that reinforcement learning algorithms could help improve patient's health status. A Python implementation of the proposed procedure is available at https://github.com/shengzhang37/SAVE.