Reviewerly/exHarmony huggingface dataset:
Approximate object location deep visual representations for image retrieval

Image descriptor based on Convolution Neural Network (CNN) has been presented to represent image in recent lots of works, outperforming the traditional feature for image retrieval. In this paper, a new method is proposed to capture real object regions in images for better search. In contrast to the existing method that process image as a whole, our method focuses on searching for approximate object region in an image. The proposed method mainly has the following two aspects: (i) we employ the sliding windows with the multiple scales over the image, to extract corresponding local CNN features. After processing feature, an optimized image representation with several categories of weights has been prepared for image retrieval and re-rank. (ii) On the basis of (i), we construct a new framework called approximate object location to search out the most similar domain, which can be applied in renewed retrieval, in an image for the query. This proposed framework provides a vector consisting of many locals in different scales for an image. According to experiments, our proposed method outperforms most current approaches based on CNN, and excels many previous algorithms based on costly bag-of-words.
Biologically-inspired neuronal adaptation improves learning in neural networks

Since humans still outperform artificial neural networks on many tasks, drawing inspiration from the brain may help to improve current machine learning algorithms. Contrastive Hebbian learning (CHL) and equilibrium propagation (EP) are biologically plausible algorithms that update weights using only local information (without explicitly calculating gradients) and still achieve performance comparable to conventional backpropagation. In this study, we augmented CHL and EP with Adjusted Adaptation, inspired by the adaptation effect observed in neurons, in which a neuron's response to a given stimulus is adjusted after a short time. We add this adaptation feature to multilayer perceptrons and convolutional neural networks trained on MNIST and CIFAR-10. Surprisingly, adaptation improved the performance of these networks. We discuss the biological inspiration for this idea and investigate why Neuronal Adaptation could be an important brain mechanism to improve the stability and accuracy of learning.
Cyber Deception Metrics For Interconnected Complex Systems

Cyber attackers' evolving skills cause it challenging to secure the network. Thus it is paramount to characterize adversarial strategies and estimate the attacker's capability. Furthermore, estimating the adversarial capability can aid the cyber defender when deciding to place deceptive elements in the network. In this paper, we address the problem of characterizing adversarial strategies and develop a suite of metrics that quantify the opportunity and capability of the adversary. Using these metrics, the cyber defender can estimate the attacker's capability. In our simulation, we incorporated the developed metrics to estimate adversary capabilities based on the attacker's aggression, knowledge, and stealthiness level. To minimize the adversarial impact, we consider placing decoy nodes as deceptive elements in the network and measure the effectiveness of having decoy nodes. Our experimental evaluation suggests that placing decoy nodes in the network can effectively increase the attacker's resource usage and decrease the win percentage.
Scalable Ising Computer Based on Ultra-Fast Field-Free Spin Orbit Torque Stochastic Device with Extreme 1-Bit Quantization

The Ising computer has great potential to solve complicated combinatorial optimization problems which beyond the capability of conventional computer unless the two major obstacles of easy-control high-speed low-power individual P-Bit and scalable large P-Bit array are overcame. In this work, an ultra-fast field-free P-Bit tunable by Spin Orbit Torque (SOT) Effect is proposed and carefully verified by throughout experimental measurement. Further, for the first time, an extreme 1-Bit quantization method against device variation is proposed for the hardware implementation of Ising computer with large P-Bit array. A novel Pareto front analysis methodology of Kullback-Leibler Divergence (KLD) is proposed to provide the practical and convenient criteria for the scalability judgement to large P-Bit array.
Mask and Cloze: Automatic Open Cloze Question Generation Using a Masked Language Model

Open cloze questions have been attracting attention for both measuring the ability and facilitating the learning of L2 English learners. In spite of its benefits, the open cloze test has been introduced only sporadically on the educational front, largely because it is burdensome for teachers to manually create the questions. Unlike the more commonly used multiple choice questions (MCQ), open cloze questions are in free form and thus teachers have to ensure that only a ground truth answer and no additional words will be accepted in the blank. To help ease this burden, we developed CLOZER, an automatic open cloze question generator. In this work, we evaluate CLOZER through quantitative experiments on 1,600 answers and show statistically that it can successfully generate open cloze questions that only accept the ground truth answer. A comparative experiment with human-generated questions also reveals that CLOZER can generate OCQs better than the average non-native English teacher. Additionally, we conduct a field study at a local high school to clarify the benefits and hurdles when introducing CLOZER. The results demonstrate that while students found the application useful for their language learning. Finally, on the basis of our findings, we proposed several design improvements.
ResAt-UNet: A U-Shaped Network Using ResNet and Attention Module for Image Segmentation of Urban Buildings

Architectural image segmentation refers to the extraction of architectural objects from remote sensing images. At present, most neural networks ignore the relationship between feature information, and there are problems such as model overfitting and gradient explosion. Thus, this paper proposes an improved UNet based on ResNet34 and Attention Module (ResAt-UNet) to solve the related problems. The algorithm adds a two-layer residual structure (BasicBlock) and a regional enhancement attention mechanism (Space Enhancement Area Enhancement, SEAE) to the original framework of UNet, which enhances the network depth, improves the fitting performance, and extracts small objects more accurately. The experimental results show that the network has achieved MIOU of 78.81% in the Massachusetts dataset, and the newly developed model outperforms UNet in both quantitative and qualitative aspects.
Toward Estimating MRI-Ultrasound Registration Error in Image-Guided Neurosurgery

Image-guided neurosurgery allows surgeons to view their tools in relation to preoperatively acquired patient images and models. To continue using neuronavigation systems throughout operations, image registration between preoperative images [typically magnetic resonance imaging (MRI)] and intraoperative images (e.g., ultrasound) is common to account for brain shift (deformations of the brain during surgery). We implemented a method to estimate MRI-ultrasound registration errors, with the goal of enabling surgeons to quantitatively assess the performance of linear or nonlinear registrations. To the best of our knowledge, this is the first dense error estimating algorithm applied to multimodal image registrations. The algorithm is based on a previously proposed sliding-window convolutional neural network that operates on a voxelwise basis. To create training data where the true registration error is known, simulated ultrasound images were created from preoperative MRI images and artificially deformed. The model was evaluated on artificially deformed simulated ultrasound data and real ultrasound data with manually annotated landmark points. The model achieved a mean absolute error (MAE) of 0.977 ± 0.988 mm and a correlation of 0.8 ± 0.062 on the simulated ultrasound data, and an MAE of 2.24 ± 1.89 mm and a correlation of 0.246 on the real ultrasound data. We discuss concrete areas to improve the results on real ultrasound data. Our progress lays the foundation for future developments and ultimately implementation of clinical neuronavigation systems.
Visual Servoing Control of Robotics With A Neural Network Estimator Based on Spectral Adaptive Law

In recent years, with the development of machine vision and other relative techniques, visual servoing control of robotics has been wildly applied. A complex calibration process is usually required to get the accurate parameters of the camera and the robot, so that getting the projection relationship between changes of images and movement of robot joints usually takes much effort. In order to solve this problem, a rectified linear unit (ReLU) activating neural network (NN) estimator is proposed to estimate the compound Jacobian matrix of the system in this article. The weight of the NN is updated online by a project algorithm with a novel spectral adaptive law which can effectively improve the generalization ability of the NN and the robustness of the system. By constructing a new Lyapunov function with the spectral norm of weight of NN, the stability of the proposed adaptive algorithm and the controller can be proved. Simulations and experimental results validate the effectiveness of the proposed controller.
Robust Human Upper-Limbs Trajectory Prediction Based on Gaussian Mixture Prediction

Accurate prediction of human motion trajectory can improve the security of human-robot cooperation. Due to the unstructured nature of collaborative workspace and the uncertainty of sensor sensing data, the trajectory prediction accuracy of traditional prediction algorithms is low, and the uncertainty is difficult to estimate. Aiming at the complex characteristics of human upper limb movement patterns, this paper proposes a robust upper limb end trajectory prediction algorithm. The robust Gaussian mixture model was used to model the trajectory of human upper limb movement, and the statistical values of the future trajectory were obtained by combining Gaussian mixture regression. The advantage of this algorithm is that the prediction result is not only the predicted value of the position, but also the probability distribution of all possible future motion trajectories of the upper limb. The position prediction information in a specific motion mode can be obtained by using probability and statistical distribution characteristics. The algorithm is tested on both public and private datasets. Experimental results show that this method can predict human trajectories well.
Quaternion Relation Embedding for Scene Graph Generation

As an important visual understanding task, scene graph generation has been drawing widespread attention and could boost a broad range of downstream vision applications. Traditional scene graph generation methods based on different context refinements are trained with probabilistic chain rule, which treats objects and relationships as independent entities. Despite their surprisingly great progress, such a plain formulation unconsciously ignores the latent geometric structure of entities and relationships. To address this issue, we move beyond the traditional real-valued representations and use <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Quat</u> ernion <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</u> elation <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</u> mbedding (QuatRE) to generate scene graphs with more expressive hypercomplex representations. More specifically, we introduce the concept of quaternion representations, hyper-complex valued with three imaginary components for objects entities, then formulate the relation triplets with Hamilton product. Benefiting from explicitly modeling the latent inter-dependencies among all imaginary components and strong expressive capacity, our proposed QuatRE method could better capture the interactions between entities. More importantly, our novel QuatRE method can be treated as a plug-in and well generalized into other methods for performance improvement as it involves no additional layers. Finally, extensive comparisons of our proposed method against the state-of-the-art methods on two large-scale and widely-used datasets, i.e. Visual Genome and Open Images, demonstrated our superiority and generalization capability on various metrics for biased or unbiased inference.
An Empirical Study of Web Services Topics in Web Developer Discussions on Stack Overflow

Web Services (WSs) are gaining worldwide popularity due to reliable and fast intercommunication services for the development of web and mobile applications. WSs are provided to client application developers through web Application Programming Interfaces (APIs), such as YouTube API, Twitter API, Facebook API, etc. Due to the popularity of WSs, the developers frequently discuss various WSs-based application’ issues on online forums, such as Stack Overflow (SO). This study aims to highlight the problems faced by client developers in the development process of WSs-based applications using the dataset of SO. The comprehension of developers’ conversations on SO can give insight into the frequency, difficulty, and popularity of different WSs-related problems of developers. We downloaded 12,746 posts from SO relevant to WSs-related issues for this article. We used the topic modeling technique (LDA) to extract various topics from the SO dataset. The topics are labeled and organized into categories and sub-categories according to relationships among them. The difficulty and popularity of each topic have been analyzed. Our investigation yield several findings. First, developers focus on six topics related to WSs on SO: Client APIs development, Data Processing, Web services Authorization, Framework Support, Web APIs, and Mobile Applications. Secondly, the advantages and disadvantages of web applications topic (Fused_Popularity=0.39), from the Clients APIs development category have the highest prevalence, followed by Database (DB) and Data Processing in Applications topic (Fused_Popularity=0.38) from the Data Processing category. Third, most WSs-related topics in all categories are evolving promptly on SO, i.e., new questions are added daily about WSs development, deployment, and authorization. Fourth, the questions of type “how” are primarily asked in Framework support, Client APIs development, and Web APIs categories. Although, many questions in other categories are of the kind “What”. It is also observed that WSs developers not only used SO to ask How and What types of questions but they also used SO to ask information-seeking questions (i.e., in Data processing and Client APIs development categories). Fifth, the topics relevant to Web APIs (Fused_Popularity=10.8) and Client API Development ((Fused_Popularity=9.35) categories of WSs are very popular on SO. Sixth, the questions relevant to the Web APIs (Fused_Difficulty =3) & Client APIs development (Fused_Difficulty=2.25) categories are more difficult than the other four categories. The results of our research may be helpful for the following WSs stakeholders: WSs Client application developers, WSs Educators, and WSs researchers. The WSs Educators and investigators can get more knowledge of new methods and discover novel techniques to make challenging WSs topics easy to understand. WSs framework developers can utilize our extracted WSs topics and categories to know the preferences of WSs developers that may support them in upgrading existing frameworks or developing new ones.
Three Early Formal Approaches to the Verification of Concurrent Programs

Abstract This paper traces a relatively linear sequence of early research approaches to the formal verification of concurrent programs. It does so forwards and then backwards in time. After briefly outlining the context, the key insights from three distinct approaches from the 1970s are identified (Ashcroft/Manna, Ashcroft (solo) and Owicki). The main technical material in the paper focuses on a specific program taken from the last published of the three pieces of research (Susan Owicki’s): her own verification of her Findpos example is outlined followed by attempts at verifying the same example using the earlier approaches. Reconsidering the prior approaches on the basis of Owicki’s useful example illuminates similarities and differences between the proposals. Along the way, observations about interactions between researchers (and some “blind spots”) are noted.
Loitering Detection Using Spatial-Temporal Information for Intelligent Surveillance Systems on a Vision Sensor

As one of the essential modules in intelligent surveillance systems, loitering detection plays an important role in reducing theft incidents by analyzing human behavior. This paper introduces a novel strategy for detecting the loitering activities of humans in the monitoring area for an intelligent surveillance system based on a vision sensor. The proposed approach combines spatial and temporal information in the feature extraction stage to decide whether the human movement can be regarded as loitering. This movement has been previously tracked using human detectors and particle filter tracking. The proposed method has been evaluated using our dataset consisting of 20 videos. The experimental results show that the proposed method could achieve a relatively good accuracy of 85% when utilizing the random forest classifier in the decision stage. Thus, it could be integrated as one of the modules in an intelligent surveillance system.
Visualization and Visual Knowledge Discovery from Big Uncertain Data

In the current uncertain world, data are kept growing bigger. Big data refer to the data flow of huge volume, high velocity, wide variety, and different levels of veracity (e.g., precise data, imprecise/uncertain data). Embedded in these big data are implicit, previously unknown, but valuable information and knowledge. With huge volumes of information and knowledge that can be discovered by techniques like data mining, a challenge is to validate and visualize the data mining results. To validate data for better data aggregation in estimation and prediction and for establishing trustworthy artificial intelligence, the synergy of visualization models and data mining strategies are needed. Hence, in this paper, we present a solution for visualization and visual knowledge discovery from big uncertain data. Our solution aims to discover knowledge in the form of frequently co-occurring patterns from big uncertain data and visualize the discovered knowledge. In particular, the solution shows the upper and lower bounds on frequency of these patterns. Evaluation with real-life Coronavirus disease 2019 (COVID-19) data demonstrates the effectiveness and practicality of our solution in visualization and visual knowledge discovery from big health informatics data collected from the current uncertain world.
Augmenting the Reality of Situated Visualization

Situated Visualization (SV), encompassing all the visualizations that change their appearance based on context, by considering the visualizations that are relevant to the physical context in which they are displayed [1], has been recognized as a method with potential in many situations, as is the case of supporting decision making [2]. Augmented and Mixed Reality (AR/MR) are well suited to assist in such scenarios, given its ability to display additional data regarding the real-world context and be supported by context-driven visualization techniques [3]. Though some perspectives on the SV model have been proposed, such as space, time, place, activity and community, an appropriate systematization, covering the main definitions and perspectives has yet to be established. Hence, there is an urge to obtain a more comprehensive description. The work presented in this paper characterizes the SV model, within the scope of AR/MR, shows a critical analysis of the existing knowledge, expanding the SV model and in turn hoping to elicit discussion within the research community.
Using HoloLens for Remote Collaboration in Extended Data Visualization

Good collaboration is about clearance between all the team members to let everyone work transparently towards one goal. Augmented Reality was one of the technologies used for data visualization for an immersive and better understanding and analysis of the data. Hence, the collaboration between users on the visualized data in extended reality can combine the analytic power of multiple individuals, and potentially leading to increased quality of solutions and discoveries. The work presented in this paper discusses a remote collaboration technique for enhancing collaboration on visualized data in Augmented Reality. This was done through allowing the user to be immersed to a certain extent, but not completely replacing their reality. The Microsoft Hololens was used to create an Augmented Reality application that transfers the movements of the remote users in real-time using Microsoft Kinect v2 in the form of a hologram of an avatar. This allowed multiple participants to see each other (embodied as avatars), collaborate and interact with 3-Dimension visualized data as shareable holograms between the remote users. An experiment was conducted on 14 participants. The results indicated that the application was useful and fulfills the requirement. We found that the application successfully provided a rich, interesting, and immersive collaborative experience for the students regarding remote collaboration on data visualization in extended reality.