Reviewerly/exHarmony huggingface dataset:
A Method to Improve the Privacy and Security for Federated Learning

Federated learning is proposed as an approach that enables several participants to collaboratively train the machine learning model, but without directly expose the local data to others. Although federated learning is developed to prevent the participants' local sensitive data leakage. However, recent researches show that federated learning systems are still faced with privacy and security challenges. In the privacy view, if participants or the central server are curious, they can inference information about the training data or the model by the exchanged update parameters. From a security perspective, the malicious participants can make poisoning attacks during the model training phase. Differential privacy and security multiparty computation (SMC) is exploited to solve the privacy problem. But they often result in large communication overhead or low federated learning accuracy. Besides, there lacks a comprehensive scheme that has effects on both privacy and security issues. In this article, we proposed a homomorphic encryption-based privacy enhancement mechanism which has effectiveness on membership inference attacks. Our method works in two phases. In phase I, our method exploits homomorphic encryption to encrypts participants' update parameters before sending out to the aggregator. In phase II, we add a parameter selection method to the aggregator of the federated learning system to choose participants' updated information with a certain probability. The experimental results show that the proposed method effectively defends both against membership inference attacks and poisoning attacks and makes less model accuracy effects than other existing solutions.
Simulating and Predicting Dynamical Systems With Spatial Semantic Pointers

While neural networks are highly effective at learning task-relevant representations from data, they typically do not learn representations with the kind of symbolic structure that is hypothesized to support high-level cognitive processes, nor do they naturally model such structures within problem domains that are continuous in space and time. To fill these gaps, this work exploits a method for defining vector representations that bind discrete (symbol-like) entities to points in continuous topological spaces in order to simulate and predict the behavior of a range of dynamical systems. These vector representations are spatial semantic pointers (SSPs), and we demonstrate that they can (1) be used to model dynamical systems involving multiple objects represented in a symbol-like manner and (2) be integrated with deep neural networks to predict the future of physical trajectories. These results help unify what have traditionally appeared to be disparate approaches in machine learning.
Analysis of Gender Stereotypes for the Design of Service Robots

Service robots are entering all kinds of business areas, and the outbreak of COVID-19 speeds up their application. Many studies have shown that robots with matching gender-occupational roles receive larger acceptance. However, this can also enlarge the gender bias in society. In this paper, we identified gender norms embedded in service robots by iteratively coding 67 humanoid robot images collected from the Chinese e-commerce platform Alibaba. We then generated four-step guidance for designers to identify and challenge the gender norms in the robot design. Our research provides both the fundamental grounding and practical guidance for designing catering robots that challenge gender norms and promote social equality.
An Analysis On Sarcasm Detection Over Twitter During COVID-19

The 2019 Coronavirus (COVID-19) significantly affected our society, the country, and the world. During the Corona time, people mostly spent their time on the internet and actively connected with other people through online social media (OSM) or game chatboxes. Due to extensive use of the internet and social media, the sharing of offensive content increases. OSM provides a platform where people freely express their opinion, emotions, and thoughts. Sometimes people share their feelings and thoughts sarcastically, wherein it signifies the opposite of what it states. Sarcastic content shared by people can vary in many forms, such as videos, images, podcasts, audio, and text. This research mainly focuses on Twitter text data extracted by the Twitter API during COVID-19 and investigates sarcastic content with negative sentiments during COVID-19. We extracted the data with some specific keywords like hashtag-related sarcastic information, sarcasm, irony, etc., and performed an offensive and aggressive nature analysis of people at this stage. We have used the linear support vector classifier (libSVM), Naïve Bayes, and Decision Tree for this analysis. The Decision Tree achieved the highest accuracy as compared to libSVM and Naïve Bayes. It can detect sarcastic content with up to 90% accuracy.
Alexa, We Need to Talk: A Data Literacy Approach on Voice Assistants

Voice assistants (VA) collect data about users’ daily life including interactions with other connected devices, musical preferences, and unintended interactions. While users appreciate the convenience of VAs, their understanding and expectations of data collection by vendors are often vague and incomplete. By making the collected data explorable for consumers, our research-through-design approach seeks to unveil design resources for fostering data literacy and help users in making better informed decisions regarding their use of VAs. In this paper, we present the design of an interactive prototype that visualizes the conversations with VAs on a timeline and provides end users with basic means to engage with data, for instance allowing for filtering and categorization. Based on an evaluation with eleven households, our paper provides insights on how users reflect upon their data trails and presents design guidelines for supporting data literacy of consumers in the context of VAs.
CNN-SVM based vehicle detection for UAV platform

Conventional surveillance devices are deployed at fixed locations on road sideways, poles or on traffic lights, which provide a constant and fixed surveillance view of the urban traffic. Unmanned aerial vehicles (UAVs) have for last two decades received considerable attention in building smart and effective system with wider coverage using low cost, highly flexible unmanned platform for smart city infrastructure. Unlike fixed monitoring devices, the camera platform of aerial vehicles has many constraints, as it is in constant motion including titling and panning, and thus makes it difficult to process data for real time applications. The inaccuracy in object detection rates from UAV videos has motivated the research community to combine different approaches such as optical flow and supervised learning algorithms. The method proposed in this research incorporates steps that include Kanade-Lucas optical flow method for moving object detection, building connected graphs to isolate objects and convolutional neural network (CNN), followed by support vector machine (SVM) for final classification. The generated optical flow contains background (and tiny) objects detected as vehicle as the camera platform moves. The classifier introduced here rules out the presence of any other (moving) objects to be detected as vehicles. The methodology adopted is tested on a stationary and moving aerial videos. The system is shown to have performance accuracy of 100% in case of stationary video and 98% in case of video from aerial platform.
Multiplex Regulation System With Personalised Recommendation Using ML

The main aim of this research work is to conduct a feasibility study and requirement analysis for a Multiplex Management system (MMS). This research work includes the design diagrams that facilitate the production of the software and its use in making a frontend and backend for the system according to the elicited requirements, by integrating them. Its alpha version, which strictly adhering to relevant software engineering practices that test the software with suitable methods has also developed. It automates various functions of Multiplex Theater with particular emphasis on ticket reservation, show screening, personnel management, statistical report generation, and increasing ticket sales by giving personalized movie suggestions by learning from the previous booking history or characteristics of users. MMS is a Web application that provides different interfaces for different stakeholders like users, admins, and employees. The admins can add movies and schedule screenings and evaluate the booking statistics and the users and employees can book and cancel tickets. The system also analyzes various factors using Machine Learning to suggest movies to the User to maximize their chances of booking them. The data and model generated from this system are also a precious commodity that can be leveraged successfully to bring in large gains for the Service.
Adaptive Interval Type-2 Fuzzy Filter: An AI Agent for Handling Uncertainties to Preserve Image Naturalness

Artificial intelligence (AI) offers fuzzy set theory (FST) as one of the popular AI agents and decision making tools for digital image processing to increase the robustness of vision-based applications. FST is capable to handle uncertainties while enhancing image quality and preserving its naturalness. This article proposes a novel Adaptive Interval Type-2 Fuzzy Filter (AIT2FF) as an AI agent for preserving the naturalness of nonuniformly illuminated images. The proposed AIT2FF estimates coarse illumination of the input image, which is further processed to obtain the reflectance and refined coarse illumination for the composition of enhanced image. The estimated coarse illumination preserves the naturalness by eliminating uncertainties due to grayness ambiguities in homogeneous regions and spatial ambiguities at edges. The effectiveness of the proposed filter has been presented quantitatively in terms of lightness order error (LOE) and naturalness image quality evaluator (NIQE) score on images from the high dynamic range dataset and images captured with commercial digital cameras. The qualitative comparison visualizes that the enhanced images obtained using the proposed filter maintains visual realism and naturalness. The applications of the proposed filter have also been presented for image dehazing, vehicle tracking, and gradients estimation. Moreover, the detection results for dehazed images have been compared with state-of-the-art methods on real-world task driven testing set from REalistic Single Image DEhazing- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\beta$</tex-math></inline-formula> dataset. The comparisons demonstrate that the enhanced images obtained using the proposed AIT2FF approach are better and outperform others in terms of LOE and NIQE measures. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>Impact Statement</i>—The objective of vision-based applications is to automate the human visual perception. This requires a preprocessed input image with fine details. The image details to achieve human vision suffer from uncertainties in homogeneous regions and at edges. This result in an unnatural preprocessed image and degrades the performance. For example: Noisy images are not good to train deep learning models. AI offers one of the best tools, i.e., fuzzy set theory, which is capable to handle uncertainties. This article proposes an adaptive interval Type-2 fuzzy filter for elimination of uncertainties. The proposed filter enhances the image details and maintains its natural appearance. The elimination of uncertainties using proposed filter drops the error value from 2.573 to 1.929 for nonuniformly illuminated images. The proposed approach is able to boost the performance of various vision-based applications like object detection, vehicle tracking, security, medical diagnostics, etc.
Probabilistic Model Distillation for Semantic Correspondence

Semantic correspondence is a fundamental problem in computer vision, which aims at establishing dense correspondences across images depicting different instances under the same category. This task is challenging due to large intra-class variations and a severe lack of ground truth. A popular solution is to learn correspondences from synthetic data. However, because of the limited intra-class appearance and background variations within synthetically generated training data, the model’s capability for handling “real” image pairs using such strategy is intrinsically constrained. We address this problem with the use of a novel Probabilistic Model Distillation (PMD) approach which transfers knowledge learned by a probabilistic teacher model on synthetic data to a static student model with the use of unlabeled real image pairs. A probabilistic supervision reweighting (PSR) module together with a confidence-aware loss (CAL) is used to mine the useful knowledge and alleviate the impact of errors. Experimental results on a variety of benchmarks show that our PMD achieves state-of-the-art performance. To demonstrate the generalizability of our approach, we extend PMD to incorporate stronger supervision for better accuracy – the probabilistic teacher is trained with stronger key-point supervision. Again, we observe the superiority of our PMD. The extensive experiments verify that PMD is able to infer more reliable supervision signals from the probabilistic teacher for representation learning and largely alleviate the influence of errors in pseudo labels. Code is available at https://github.com/fanyang587/PMD.
Mention Flags (MF): Constraining Transformer-based Text Generators

This paper focuses on Seq2Seq (S2S) constrained text generation where the text generator is constrained to mention specific words, which are inputs to the encoder, in the generated outputs.Pre-trained S2S models such as T5 or a Copy Mechanism can be trained to copy the surface tokens from encoders to decoders, but they cannot guarantee constraint satisfaction.Constrained decoding algorithms always produce hypotheses satisfying all constraints.However, they are computationally expensive and can lower the generated text quality.In this paper, we propose Mention Flags (MF), which trace whether lexical constraints are satisfied in the generated outputs of an S2S decoder.The MF models are trained to generate tokens until all constraints are satisfied, guaranteeing high constraint satisfaction.Our experiments on the Common Sense Generation task (CommonGen) (Lin et al., 2020), End2end Data-to-Text task (E2ENLG) (Dušek et al., 2020) and Novel Object Captioning task (nocaps) (Agrawal et al., 2019) show that the MF models maintain higher constraint satisfaction and text quality than the baseline models and other constrained text generation algorithms, achieving state-of-the-art performance on all three tasks.These results are achieved with a much lower run-time than constrained decoding algorithms.We also show that the MF models work well in the low-resource setting. 1
Extending a Theory of Slow Technology for Design through Artifact Analysis

People’s daily experiences and the environments they inhabit have become saturated with digital devices and systems. With this shift, new concerns have emerged across the HCI community over the rol...
A Comprehensive Analysis of Low-Impact Computations in Deep Learning Workloads

Deep Neural Networks (DNNs) have achieved great successes in various machine learning tasks involving a wide range of domains. Though there are multiple hardware platforms available, such as GPUs, CPUs, FPGAs, and etc, CPUs are still preferred choices for machine learning applications, especially in low-power and resource-constrained computation environments such as embedded systems. However, the power and performance efficiency become critical issues in such computation environments when applying DNN techniques. An attractive optimization to DNNs is to remove redundant computations to enhance the execution efficiency. To this end, this paper conducts extensive experiments and analyses on popular state-of-the-art deep learning models. The experimental results include the numbers of instructions, branches, branch prediction misses, cache misses, and etc, during the execution of the models. Besides, we also investigate the performance and sparsity of each layer in the models. Based on the analysis results, this paper also proposes an instruction-level optimization, which achieves the performance improvement ranging from 10.26% to 28.0% for certain convolution layers.
HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences

In this paper, we address the problem of building dense correspondences between human images under arbitrary camera viewpoints and body poses. Prior art either assumes small motion between frames or relies on local descriptors, which cannot handle large motion or visually ambiguous body parts, e.g., left vs. right hand. In contrast, we propose a deep learning framework that maps each pixel to a feature space, where the feature distances reflect the geodesic distances among pixels as if they were projected onto the surface of a 3D human scan. To this end, we introduce novel loss functions to push features apart according to their geodesic distances on the surface. Without any semantic annotation, the proposed embeddings automatically learn to differentiate visually similar parts and align different subjects into an unified feature space. Extensive experiments show that the learned embeddings can produce accurate correspondences between images with remarkable generalization capabilities on both intra and inter subjects. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Towards standardization of morphosyntactic terminology for general linguistics

This paper proposes that just like phonologists, linguists working on morphosyntax should have a core set of standard terms that are understood in exactly the same way across the discipline. Most of these terms are traditional terms that are given a standard retro-definition, because linguists already behave as if these terms had the same meaning for everyone. The definitions are definitions of general concepts (i.e. comparative concepts, applicable to all languages in exactly the same way), but they are expected to be highly similar to language-particular categories with the same labels. If linguists were close to finding out the true natural-kind categories of Human Language that all grammars consist of, there would be no need for definitions, but since this seems to be a remote goal, research on general linguistics must rely on uniformly defined general terms.
SRKTDN: Applying Super Resolution Method to Dehazing Task

Nonhomogeneous haze removal is a challenging problem, which does not follow the physical scattering model of haze. Numerous existing methods focus on homogeneous haze removal by generating transmission map of the image, which is not suitable for nonhomogeneous dehazing tasks. Some methods use end-to-end model but are also designed for homogeneous haze. Inspired by Knowledge Transfer Dehazing Network and Trident Dehazing Network, we propose a model with super-resolution method and knowledge transfer method. Our model consists of a teacher network, a dehaze network and a super-resolution network. The teacher network provides the dehaze network with reliable prior, the dehaze network focuses primarily on haze removal, and the super-resolution network is used to capture details in the hazy image. Ablation study shows that the super-resolution network has significant benefit to image quality. And comparison shows that our model out- performs previous state-of-the-art methods in terms of perceptual quality on NTIRE2021 NonHomogeneous Dehazing Challenge dataset, and also performs well on other datasets.
Hierarchical Object Relationship Constrained Monocular Depth Estimation.

Monocular depth estimation has been gaining growing momentum in recent years. Despite significant advances of this task, due to the inherent difficulty of reliably capturing contextual cues from RGB images, it remains challenging to accurately predict depth in scenes with complicated and cluttered spatial arrangement of objects. Instead of naively utilizing the primary features in the single RGB image, in this paper we propose a hierarchical object relationship constrained network for monocular depth estimation, which could enable accurate and smooth depth prediction from monocular RGB image. The key idea of our method is to exploit object-centric hierarchical relationship as contextual constraints to compensate for the regularity of spatial depth changing. In particular, we design a semantics-guided CNN network to encode the original image into a global context feature map and encode the objects' relationship into a local relationship feature map simultaneously, so that we can leverage such effective and consolidated coding scheme over scenario samples to guide the depth prediction in a more accurate way. Benefiting from the local-to-global context constraints, our method can well respect the global depth changing and preserve the local depth details at the same time. In addition, our approach could make full use of the hierarchical semantic relationship across inner-object components and neighboring objects to define depth changing constraints. We conduct extensive experiments and make comprehensive evaluations on widely-used public datasets, and the experiments confirm that our method outperforms most state-of-the-art depth estimation methods in preserving the local details in depth.
Accurate 3D Object Detection using Energy-Based Models

Accurate 3D object detection (3DOD) is crucial for safe navigation of complex environments by autonomous robots. Regressing accurate 3D bounding boxes in cluttered environments based on sparse LiDAR data is however a highly challenging problem. We address this task by exploring recent advances in conditional energy-based models (EBMs) for probabilistic regression. While methods employing EBMs for regression have demonstrated impressive performance on 2D object detection in images, these techniques are not directly applicable to 3D bounding boxes. In this work, we therefore design a differentiable pooling operator for 3D bounding boxes, serving as the core module of our EBM network. We further integrate this general approach into the state-of-the-art 3D object detector SA-SSD. On the KITTI dataset, our proposed approach consistently outperforms the SA-SSD baseline across all 3DOD metrics, demonstrating the potential of EBM-based regression for highly accurate 3DOD. Code is available at https://github.com/fregu856/ebms_3dod.
Solar energy harvesting on building’s rooftops: A case of a Nigeria cosmopolitan city

In this paper, the possibility of meeting the growing electrical energy demands of the city of Ibadan (one of the Nigeria cosmopolitan city) using the rooftop solar PV systems is investigated with the aim of determining the available rooftop for possible PV deployment. First, the optimal tilt angle as well as solar irradiation on tilted surfaces are determined for the city, Thereafter, the available roof area for possible solar PV- rooftops placement are analysed using reduction factor technique with the aid of ArcGIS, Google earth Imagery and population sampling techniques. Some of the key results reveal that the total roof area for the city of Ibadan is about 49.54 sq km while the available roof area for rooftop PV deployment is about 7.54 sq km with a maximum installation capacity of 1734.8 MWp, at an optimum fixed tilt of 11°. This could generate PV electricity of about 6.67 TW h per annum. This paper is important as it could serve as first-hand scientific information for government agencies, NGOs, investors, researchers and policy makers at maximizing the potential of solar PV rooftop potentials in Nigeria.