Reviewerly/exHarmony huggingface dataset:
BCPriPIoT: BlockChain Utilized Privacy-Preservation Mechanism for IoT Devices

Security and privacy are the primary concerns for IoT devices but because of their inherent limitation both in terms of processing and energy, IoT devices cannot be deployed at their full scale. To alleviate such security and privacy issues, the interaction of blockchain with IoT systems has acquired significant attention these days because blockchain has presented an underlying mechanism of immutability, audibility, and transparency for data storage. However due to the intrinsic nature of a blockchain containing complex mathematical proof concepts such as Merkle Hash Tree and Proof of Work (PoW) which demands high computation power making it less viable for IoT devices to be connected with. To overcome these issues, a novel scheme is proposed in this paper, which deploys private by design based blockchain architecture for IoT devices using low complex consensus algorithm and low computation cryptographic mechanism which suits best for IoT devices to address the privacy concerns. Unlike the traditional blockchain network in which every node maintained a copy of the transaction, we have proposed a new architecture in which block validation and block generation logic has been modified so that a transaction will be limited to the trusted recipient only. The proposed scheme outperforms the contemporary approaches both in terms of throughput and latency as observed through simulation results as well as maintaining the privacy concerns which will encourage the actual implementation of IoT applications in the real world. Moreover, the evaluation analysis demonstrate that the approach has major potential in a trusted network computing system and provides a substantial secure environment for IoT users.
Verifying Pufferfish Privacy in Hidden Markov Models

Pufferfish is a Bayesian privacy framework for designing and analyzing privacy mechanisms. It refines differential privacy, the current gold standard in data privacy, by allowing explicit prior knowledge in privacy analysis. In practice, privacy mechanisms often need be modified or adjusted to specific applications. Their privacy risks have to be re-evaluated for different circumstances. Privacy proofs can thus be complicated and prone to errors. Such tedious tasks are burdensome to average data curators. In this paper, we propose an automatic verification technique for Pufferfish privacy. We use hidden Markov models to specify and analyze discrete mechanisms in Pufferfish privacy. We show that the Pufferfish verification problem in hidden Markov models is NP-hard. Using Satisfiability Modulo Theories solvers, we propose an algorithm to verify privacy requirements. We implement our algorithm in a prototypical tool called FAIER, and analyze several classic privacy mechanisms in Pufferfish privacy. Surprisingly, our analysis show that naïve discretization of well-established privacy mechanisms often fails, witnessed by counterexamples generated by FAIER. In discrete Above Threshold, we show that it results in absolutely no privacy. Finally, we compare our approach with state-of-the-art tools for differential privacy, and show that our verification technique can be efficiently combined with these tools for the purpose of certifying counterexamples and finding a more precise lower bound for the privacy budget $$\epsilon $$ .
Image Stitching via Convolutional Neural Network

Image stitching aims at generating a panorama from a sequence of images with overlapping areas. Many approaches have been dedicated to deal with the task. However, they usually require handcrafted features and multiple fulfilling steps, which may incur high time cost and intermediate interference. Recently, deep networks have been applied in this area, but mainly focused on specific steps of image stitching, such as feature extraction and image alignment, not enabling an end-to-end stitching. In this paper, we propose a novel convolutional neural network (CNN) based method which can perform image stitching in an end-to-end fashion. Qualitative and quantitative results from experiments on synthetic image datasets verify the excellent performance of the proposed method.
Metrics and Algorithms for Locally Fair and Accurate Classifications using Ensembles

To obtain accurate predictions of classifiers, model ensembles comprising multiple trained machine learning models are nowadays used. In particular, dynamic model ensembles pick the most accurate model for each query object, by applying the model that performed best on similar data. Dynamic model ensembles may however suffer, similarly to single machine learning models, from bias, which can eventually lead to unfair treatment of certain groups of a general population. To mitigate unfair classification, recent work has thus proposed fair model ensembles, that instead of focusing (solely) on accuracy also optimize global fairness. While such global fairness globally minimizes bias, imbalances may persist in different regions of the data, e.g., caused by some local bias maxima leading to local unfairness. Therefore, we extend our previous work by including a framework that bridges the gap between dynamic model ensembles and fair model ensembles. More precisely, we investigate the problem of devising locally fair and accurate dynamic model ensembles, which ultimately optimize for equal opportunity of similar subjects. We propose a general framework to perform this task and present several algorithms implementing the framework components. In this paper we also present a runtime-efficient framework adaptation that keeps the quality of the results on a similar level. Furthermore, new fairness metrics are presented as well as detailed informations about necessary data preparations. Our evaluation of the framework implementations and metrics shows that our approach outperforms the state-of-the art for different types and degrees of bias present in training data in terms of both local and global fairness, while reaching comparable accuracy.
Mobile Application Based on Augmented Reality to Encourage Tourism at the Wari Willka Museum

In this research, we have implemented a mobile application that allows to know through an inter-dimensional portal the history of the Wari culture in Junín (Peru) without the need to visit the museum in person. The methodology uses AR Foundation and Arcore tools for Unity, as well as plugins to work with augmented reality through a structure of files used to create scenes with interactive interfaces and 3D modeling directly. The validation was carried out through surveys with a qualitative approach and experimental design, verified by Cronbach’s alpha (α=0.876) whose result of the measurement range is between good and high. As a conclusion, a positive influence was obtained in the tourist experience of the museum since the implementation of the application, with a percentage of acceptance of 65% among the users.
Learning Deep Generative Clustering via Mutual Information Maximization

Deep clustering refers to joint representation learning and clustering using deep neural networks. Existing methods can be mainly categorized into two types: discriminative and generative methods. The former learns representations for clustering with discriminative mechanisms directly, and the latter estimate the latent distribution of each cluster for generating data points and then infers cluster assignments. Although generative methods have the advantage of estimating the latent distributions of clusters, their performances still significantly fall behind discriminative methods. In this work, we argue that this performance gap might be partly due to the overlap of data distribution of different clusters. In fact, there is little guarantee of generative methods to separate the distributions of different clusters in the data space. To tackle these problems, we theoretically prove that mutual information maximization promotes the separation of different clusters in the data space, which provides a theoretical justification for deep generative clustering with mutual information maximization. Our theoretical analysis directly leads to a model which integrates a hierarchical generative adversarial network and mutual information maximization. Moreover, we further propose three techniques and empirically show their effects to stabilize and enhance the model. The proposed approach notably outperforms other generative models for deep clustering on public benchmarks.
SVMAC: Unsupervised 3D Human Pose Estimation from a Single Image with Single-view-multi-angle Consistency

Recovering 3D human pose from 2D joints is still a challenging problem, especially without any 3D annotation, video information, or multi-view information. In this paper, we present an unsupervised GAN-based model consisting of multiple weight-sharing generators to estimate a 3D human pose from a single image without 3D annotations. In our model, we introduce single-view-multi-angle consistency (SVMAC) to significantly improve the estimation performance. With 2Djoint locations as input, our model estimates a 3D pose and a camera simultaneously. During training, the estimated 3D pose is rotated by random angles and the estimated camera projects the rotated 3D poses back to 2D. The 2D reprojections will be fed into weight-sharing generators to estimate the corresponding 3D poses and cameras, which are then mixed to impose SVMAC constraints to self-supervise the training process. The experimental results show that our method outpetforms the state-of-the-art unsupervised methods on Human 3.6M and MPI-INF-3DHP. Moreover, qualitative results on MPII and LSP show that our method can generalize well to unknown data.
Facial Expression Recognition Using Deep Learning Methods

Humans express their innate emotions and conditions through various channels such as body language and facial expressions. Facial expressions are the most direct and meaningful channel of non-verbal communication which forms a universal language of emotions that can instantly express a wide range of human emotional states, feelings, attitudes and thus assists in various cognitive tasks. The accurate analysis and interpretation of the emotional content of human facial expressions is essential for the deeper understanding of human behavior. In this work, we study the field of Facial Expression Recognition in-depth, and perform Emotion Recognition on 2D images. The task is supervised classification of data, for which we implement a wide range of Deep Learning models in order to recognize the existent emotion. Furthermore, an additional topic showcased in this work is Attention-based models. A concrete experimental study was designed and conducted and the results are quite interesting.
Robust phase unwrapping algorithm based on Zernike polynomial fitting and Swin-Transformer network

Abstract Phase unwrapping plays an important role in optical phase measurements. In particular, phase unwrapping under heavy noise conditions remains an open issue. In this paper, a deep learning-based method is proposed to conduct the phase unwrapping task by combining Zernike polynomial fitting and a Swin-Transformer network. In this proposed method, phase unwrapping is regarded as a regression problem, and the Swin-Transformer network is used to map the relationship between the wrapped phase data and the Zernike polynomial coefficients. Because of the self-attention mechanism of the transformer network, the fitting coefficients can be estimated accurately even under extremely harsh noise conditions. Simulation and experimental results are presented to demonstrate the outperformance of the proposed method over the other two polynomial fitting-based methods. This is a promising phase unwrapping method in optical metrology, especially in electronic speckle pattern interferometry.
FPGA Implementation of a Face Recognition System

Recently machine and deep learning algorithms are considered very often for solving real life problems. These algorithms are generally implemented on CPUs or GPUs, which are not meant to execute machine learning algorithms. While these algorithms are required for artificial intelligence-based systems, they have the problem of being computationally intensive with large power consumption and execution time. Some applications also require functional safety, and GPUs are expected to meet the functional safety requirements, which is a time-consuming challenge for GPU designers. As an alternative Field Programmable Gate Arrays (FPGAs) are preferred in avionics and defense-based applications where functional safety is a key factor. FPGA based systems offer increased performance, lower power consumption, low latency and low implementation cost compared to CPUs and GPUs. They also can configure hardware meant for realizing machine learning algorithms which incorporates parallel execution and customized data types. The primary goal of this paper is to implement a face recognition algorithm on Xilinx PYNQ Z2 FPGA and compare various performance parameters. FPGA was able to perform face recognition in real time with average precision of 91.6% and overall accuracy of 92%.
Efficient Multi-Lingual Sentence Classification Framework with Sentence Meta Encoders

Natural Language Understanding (NLU) has become a primary paradigm in enterprise settings for myriad industrial applications like user intent classification, smarter chatbots, sentiment analysis, and duplicate detection to name a few. With the advent of globalization, significant advancements have been recently achieved in transformers-based multi-lingual language models such as XLM and its variants for downstream multi-lingual sentence or short text classification tasks. However, fine-tuning such large pre-trained language models is highly resource-intensive as it assumes the adaptation of the full model, hampering its wide adoption in production grade applications due to the demanding computational and memory requirements.In this paper, we present a practical and efficient framework based on fusing various pre-trained sentence encoders leveraging the multi-lingual knowledge distillation approach. We demon-strate, for the first time, the practicality of utilizing such multi-lingual sentence embeddings for supervised learning tasks with a focus on sentence classification scenarios. We experimented our proposed framework on a wide range of open source classification datasets and exhibit very competitive performance compared to fine-tuning large pre-trained language models. We showcase that our light-weight framework provides the advantage of ease of training within minutes on a single CPU, competitive inference time, and robustness to parameter settings. In hope of facilitating and democratizing practical research focused on NLP, we are planning to release our code as well as a new pre-trained sentence embeddings for XLM-R-large model.
BFConv: Improving Convolutional Neural Networks with Butterfly Convolution

Convolutional neural network (CNN) is a basic neural network widely used in vision tasks. Many CNNs alleviate the redundancy in feature maps to reduce model complexity. Inspired by digital signal processing theories, this paper reviews discrete fourier transform (DFT), finding its similarities with standard convolution. In particular, DFT has a fast algorithm called FFT, which sparks our thinking: can we learn from the idea of FFT to realize a more efficient convolution filter? Based on the butterfly operation of FFT, we propose a novel butterfly convolution (BFConv). In addition, we illustrate that group weight sharing convolution is a basic unit of BFConv. Compared with the traditional group convolution structure, BFConv constructs group residual-like connections and increases the range of receptive fields for each sub-feature layer. Without changing the network architecture, we integrate BFConv into ResNet-50, ShuffleNet and VGG-16. Experimental results on CIFAR-10 and ImageNet demonstrate the above BFConv-equipped networks reduce parameters and computation, achieving similar or higher accuracy. Remarkably, when ResNet-50 embedded BFConv reaches nearly half of the compression ratio of the model, it performs favorably against its state-of-the-art competitors.
Federated Multitask Learning for HyperFace

Multitask learning (MTL) is a promising field in machine learning owing to its capability to improve the generalization performance of all tasks by sharing knowledge among the related tasks. MTL has attracted a large amount of attention in the multitask-related community. In recent years, with the rapid development of distributed machine learning, MTL in a distributed environment becomes a hot research topic. Although MTL can reap great fruit in a distributed environment, the sensitive information, such as photos and voice recordings of the owner, in distributed data may be leaked during training. Consequently, in the distributed settings, one prominent challenge of MTL is to prevent the disclosure of sensitive private data. In this article, we propose a novel approach, federated multitask learning (FMTL), which applies federated learning to MTL in a distributed environment in order to protect the MTL model from being leaked and to better optimize the utility of the MTL model. In FMTL, all the participants independently train their local models on their own dataset in parallel and only transmit their model updates to the central server for aggregation at every epoch. As a result, their learning accuracy is enhanced beyond what is achieved only on their own training data. In this manner, it can achieve the best tradeoff between the utility and privacy: participants not only preserve the privacy of their own data, but also benefit from the models of other participants. The experimental results on annotated facial landmarks in the wild (AFLW) and annotated faces in the wild (AFW) benchmark datasets verify the effectiveness of our framework.
Real-Time Generation of Comfort-Optimal Flight Trajectories for Urban Air Mobility Missions

In this paper, we explore the problem of planning the optimal trajectory for an unmanned aerial vehicle (UAV) with the task of providing its cargo or passengers with the most comfortable transit possible. The UAV has fixed boundary conditions for its initial and final position and velocity in three-dimensional (3D) space as well as a fixed flight time to perform its task. Three unique situations are considered and the solutions are compared for two solution techniques. The initial solutions are computed using a pseudospectral optimal control approach and these results are confirmed with a convex solver. The results from these two optimization methods adhere closely to one another and prove that comfort-optimized trajectories for UAVs can successfully be computed under the given constraints.
The Virtual Kinect

The COVID-19 pandemic impacted researches that depended on making tests on patients and teams that had to be divided to avoid crowding in laboratories. Sharing equipment is no longer a simple strategy; the acquisition of extra equipment is beyond the means of many researchers. Research with Kinect V2 applied to body tracking suffers from sanitary restrictions, product discontinuation and limited access to newer sensors (like the Azure Kinect). Kinect V2 is an RGB-D sensor with many applications in health, ergonomics, sports, games and other areas. That is why a lot of research is still under development with it. Because of the applicability of Kinect V2 on research and the current acquisition limitations, Virtual Kinect (VK) was created. VK is an open-source solution that enables the programmer to code using Kinect SDK functions and test it without a Kinect V2 sensor or a previous recording. It simulates the behavior of a Kinect V2 sensor through MediaPipe's Pose estimation, providing RGB image and joint tracking information, correlating the joints of the two devices. This correlation is possible due to the proximity of the devices' joint estimates. The VK was made to be simple and practical, so that its use only depends on the DLL exchange and the use of an ordinary RGB camera.
Hydrology model establishment of pit lake: Extreme event rainfall data analysis

Abstract Analysis of water management in the pit lake is divided into two conditions, namely Continuous Events and Extreme Events. The former is an analysis of pit lake management related to the water filling in a pit lake that takes place continuously. Meanwhile, the later is the analysis of pit lake management related to the possibility of extreme conditions that will occur, including extreme rainfall. This study is focused only on the Extreme Event conditions. The Gumbel method is used to calculate the planned return period rainfall T concerning the prediction of extreme rainfall. Meanwhile, for a certain return period, rainfall intensity can be predicted using the Mononobe formula. Based on the result of calculation the Gumbel method, it shows that the planned rainfall for a return period of 10 years is 132.9 mm / day. Then based on the results of the calculation of rainfall intensity using the Mononobe formula, it is obtained that the intensity of rainfall for a return period of 10 years with a concentration-time of 5 minutes is 241.5 mm/hour, while the amount of rainfall intensity with a concentration-time of 300 minutes or 5 hours is 15.8 mm/hour.
Political Ideology Detection of News Articles Using Deep Neural Networks

Individuals inadvertently allow emotions to drive their rational thoughts to predetermined conclusions regarding political partiality issues.Being wellinformed about the subject in question mitigates emotions' influence on humans' cognitive reasoning, but it does not eliminate bias.By nature, humans tend to pick a side based on their beliefs, personal interests, and principles.Hence, journalists' political leaning is defining factor in the rise of the polarity of political news coverage.Political bias studies usually align subjects or controversial topics of the news coverage to a particular ideology.However, politicians as private citizens or public officials are also consistently in the media spotlight throughout their careers.Detecting political polarity in the news coverage of politicians rather than topics adds a new perspective.Determining the best approach for detecting political polarity in the news relies on the news delivery method.Data types such as videos, audio, or text could summarize the news delivery methods.Text is one of the most prominent news delivery methods.Text pattern recognition and text classification are well-established research areas with applications in many multidisciplinary domains.We propose to use deep neural networks to detect ideology in news media articles that cover news related to political officials, namely, President Obama and Trump.Deep network models were able to identify the political ideology of articles with over 0.9 F1-Score.An evaluation and analysis of deep neural network performance in detecting political ideology of news articles, articles' authors, and news sources are presented in the paper.Furthermore, this paper experiments on and provides a detailed analysis of newly reconstructed datasets.