Reviewerly/exHarmony huggingface dataset:
Social Adaptive Module for Weakly-Supervised Group Activity Recognition

This paper presents a new task named weakly-supervised group activity recognition (GAR) which differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data. This eases us to collect and annotate a large-scale NBA dataset and thus raise new challenges to GAR. To mine useful information from weak supervision, we present a key insight that key instances are likely to be related to each other, and thus design a social adaptive module (SAM) to reason about key persons and frames from noisy data. Experiments show significant improvement on the NBA dataset as well as the popular volleyball dataset. In particular, our model trained on video-level annotation achieves comparable accuracy to prior algorithms which required strong labels.
PIPAL: A Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration

Image quality assessment (IQA) is the key factor for the fast development of image restoration (IR) algorithms. The most recent IR methods based on Generative Adversarial Networks (GANs) have achieved significant improvement in visual performance, but also presented great challenges for quantitative evaluation. Notably, we observe an increasing inconsistency between perceptual quality and the evaluation results. Then we raise two questions: (1) Can existing IQA methods objectively evaluate recent IR algorithms? (2) When focus on beating current benchmarks, are we getting better IR algorithms? To answer these questions and promote the development of IQA methods, we contribute a large-scale IQA dataset, called Perceptual Image Processing Algorithms (PIPAL) dataset. Especially, this dataset includes the results of GAN-based methods, which are missing in previous datasets. We collect more than 1.13 million human judgments to assign subjective scores for PIPAL images using the more reliable “Elo system”. Based on PIPAL, we present new benchmarks for both IQA and super-resolution methods. Our results indicate that existing IQA methods cannot fairly evaluate GAN-based IR algorithms. While using appropriate evaluation methods is important, IQA methods should also be updated along with the development of IR algorithms. At last, we improve the performance of IQA networks on GAN-based distortions by introducing anti-aliasing pooling. Experiments show the effectiveness of the proposed method.
Class-Wise Dynamic Graph Convolution for Semantic Segmentation

Recent works have made great progress in semantic segmentation by exploiting contextual information in a local or global manner with dilated convolutions, pyramid pooling or self-attention mechanism. In order to avoid potential misleading contextual information aggregation in previous works, we propose a class-wise dynamic graph convolution (CDGC) module to adaptively propagate information. The graph reasoning is performed among pixels in the same class. Based on the proposed CDGC module, we further introduce the Class-wise Dynamic Graph Convolution Network (CDGCNet), which consists of two main parts including the CDGC module and a basic segmentation network, formi2ng a coarse-to-fine paradigm. Specifically, the CDGC module takes the coarse segmentation result as class mask to extract node features for graph construction and performs dynamic graph convolutions on the constructed graph to learn the feature aggregation and weight allocation. Then the refined feature and the original feature are fused to get the final prediction. We conduct extensive experiments on three popular semantic segmentation benchmarks including Cityscapes, PASCAL VOC 2012 and COCO Stuff, and achieve state-of-the-art performance on all three benchmarks.
LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation

Deep learning approaches have achieved great success in addressing the problem of optical flow estimation. The keys to success lie in the use of cost volume and coarse-to-fine flow inference. However, the matching problem becomes ill-posed when partially occluded or homogeneous regions exist in images. This causes a cost volume to contain outliers and affects the flow decoding from it. Besides, the coarse-to-fine flow inference demands an accurate flow initialization. Ambiguous correspondence yields erroneous flow fields and affects the flow inferences in subsequent levels. In this paper, we introduce LiteFlowNet3, a deep network consisting of two specialized modules, to address the above challenges. (1) We ameliorate the issue of outliers in the cost volume by amending each cost vector through an adaptive modulation prior to the flow decoding. (2) We further improve the flow accuracy by exploring local flow consistency. To this end, each inaccurate optical flow is replaced with an accurate one from a nearby position through a novel warping of the flow field. LiteFlowNet3 not only achieves promising results on public benchmarks but also has a small model size and a fast runtime.
A deep facial recognition system using computational intelligent algorithms

The development of biometric applications, such as facial recognition (FR), has recently become important in smart cities. Many scientists and engineers around the world have focused on establishing increasingly robust and accurate algorithms and methods for these types of systems and their applications in everyday life. FR is developing technology with multiple real-time applications. The goal of this paper is to develop a complete FR system using transfer learning in fog computing and cloud computing. The developed system uses deep convolutional neural networks (DCNN) because of the dominant representation; there are some conditions including occlusions, expressions, illuminations, and pose, which can affect the deep FR performance. DCNN is used to extract relevant facial features. These features allow us to compare faces between them in an efficient way. The system can be trained to recognize a set of people and to learn via an online method, by integrating the new people it processes and improving its predictions on the ones it already has. The proposed recognition method was tested with different three standard machine learning algorithms (Decision Tree (DT), K Nearest Neighbor(KNN), Support Vector Machine (SVM)). The proposed system has been evaluated using three datasets of face images (SDUMLA-HMT, 113, and CASIA) via performance metrics of accuracy, precision, sensitivity, specificity, and time. The experimental results show that the proposed method achieves superiority over other algorithms according to all parameters. The suggested algorithm results in higher accuracy (99.06%), higher precision (99.12%), higher recall (99.07%), and higher specificity (99.10%) than the comparison algorithms.
Single image portrait relighting via explicit multiple reflectance channel modeling

Portrait relighting aims to render a face image under different lighting conditions. Existing methods do not explicitly consider some challenging lighting effects such as specular and shadow, and thus may fail in handling extreme lighting conditions. In this paper, we propose a novel framework that explicitly models multiple reflectance channels for single image portrait relighting, including the facial albedo, geometry as well as two lighting effects, i.e. , specular and shadow. These channels are finally composed to generate the relit results via deep neural networks. Current datasets do not support learning such multiple reflectance channel modeling. Therefore, we present a large-scale dataset with the ground-truths of the channels, enabling us to train the deep neural networks in a supervised manner. Furthermore, we develop a novel module named Lighting guided Feature Modulation (LFM). In contrast to existing methods which simply incorporate the given lighting in the bottleneck of a network, LFM fuses the lighting by layer-wise feature modulation to deliver more convincing results. Extensive experiments demonstrate that our proposed method achieves better results and is able to generate challenging lighting effects.
Model-Based Deep Learning

Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Such model-based methods utilize mathematical formulations that represent the underlying physics, prior information, and additional domain knowledge. Simple classical models are useful but sensitive to inaccuracies and may lead to poor performance when real systems display complex or dynamic behavior. On the other hand, purely data-driven approaches that are model-agnostic are becoming increasingly popular as datasets become abundant and the power of modern deep learning pipelines increases. Deep neural networks (DNNs) use generic architectures that learn to operate from data and demonstrate excellent performance, especially for supervised problems. However, DNNs typically require massive amounts of data and immense computational resources, limiting their applicability for some scenarios. In this article, we present the leading approaches for studying and designing model-based deep learning systems. These are methods that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches. Such model-based deep learning methods exploit both partial domain knowledge, via mathematical structures designed for specific problems, and learning from limited data. Among the applications detailed in our examples for model-based deep learning are compressed sensing, digital communications, and tracking in state-space models. Our aim is to facilitate the design and study of future systems at the intersection of signal processing and machine learning that incorporate the advantages of both domains.
Blockchain‐based efficient communication for food supply chain industry: Transparency and traceability analysis for sustainable business

Summary In recent years, blockchain (BC) networks have expanded to a broad spectrum of fields. They have garnered significant attention due to their feature of creating secure private and public networks in a decentralized manner. The traditional FSC fails to meet the customer requirements of quality, safety, reliability, transparency, and traceability. This paper proposes a novel 3‐stage methodology using principal component analysis (PCA), total iterative structural modeling (TISM), and Matrice d'Impacts Croisés Multiplication Appliquée à un Classement (MICMAC) analysis to develop an integrated BC with the food supply chain (FSC). The existing literature was surveyed and industry experts' opinion was gathered and various critical success factors (CSF) were identified which are associated with FSC and BC. The data collected from experts were then evaluated for data consistency and dimension reduction through PCA. The dimensionally reduced data set was analyzed by TISM and MICMAC analysis. TISM and MICMAC analysis categorizes the enabler factors according to their hierarchical level and is used to label or distinguish between dependence and driving power. The integrated methodology found nine CSFs, namely, transparent system, fraud detect, inventory management with tracking, scalability, secure system, cost reduction, safe and quality food, customer satisfaction, and government regulations, which are the main drivers for the FSC business. The paper integrates information technology, Blockchain, with operation management, supply chain. The proposed model overcomes the limitation associated with FSC and will motivate the academia and industry to adopt and achieve efficient and effective goals in this field of FSC.
PV power prediction in a peak zone using recurrent neural networks in the absence of future meteorological information

As the majority of daily PV power outputs is mostly obtained in a peak zone around noon, hourly PV power output prediction in a peak zone is considered as an essential function for more sophisticated operations of PV facilities. However, the prediction of PV power output in a peak zone is a challenging problem since meteorological information is continuously changing and difficult to obtain for a particular area. In addition, due to only using the meteorological information observed in the morning to estimate PV power outputs around noon, the input features which are utilized as a shorter horizon from the horizon of the prediction are making the problem even more complex. Therefore, this study proposes two PV power output prediction model by using long short-term memory (LSTM) and gate recurrent network (GRU). In particular, unlike the previous methods, the proposed models attempt to understand the hidden sequential patterns of PV power outputs based only on the information captured in the morning without utilizing future meteorological information observed around noon during training. The experiment results using a real-world dataset indicate that the proposed models perform better PV power prediction in the peak zone than conventional models.
Application of Bayesian networks to generate synthetic health data

This study seeks to develop a fully automated method of generating synthetic data from a real dataset that could be employed by medical organizations to distribute health data to researchers, reducing the need for access to real data. We hypothesize the application of Bayesian networks will improve upon the predominant existing method, medBGAN, in handling the complexity and dimensionality of healthcare data.We employed Bayesian networks to learn probabilistic graphical structures and simulated synthetic patient records from the learned structure. We used the University of California Irvine (UCI) heart disease and diabetes datasets as well as the MIMIC-III diagnoses database. We evaluated our method through statistical tests, machine learning tasks, preservation of rare events, disclosure risk, and the ability of a machine learning classifier to discriminate between the real and synthetic data.Our Bayesian network model outperformed or equaled medBGAN in all key metrics. Notable improvement was achieved in capturing rare variables and preserving association rules.Bayesian networks generated data sufficiently similar to the original data with minimal risk of disclosure, while offering additional transparency, computational efficiency, and capacity to handle more data types in comparison to existing methods. We hope this method will allow healthcare organizations to efficiently disseminate synthetic health data to researchers, enabling them to generate hypotheses and develop analytical tools.We conclude the application of Bayesian networks is a promising option for generating realistic synthetic health data that preserves the features of the original data without compromising data privacy.
Incomplete multi-modal representation learning for Alzheimer’s disease diagnosis

Alzheimers disease (AD) is a complex neurodegenerative disease. Its early diagnosis and treatment have been a major concern of researchers. Currently, the multi-modality data representation learning of this disease is gradually becoming an emerging research field, attracting widespread attention. However, in practice, data from multiple modalities are only partially available, and most of the existing multi-modal learning algorithms can not deal with the incomplete multi-modality data. In this paper, we propose an Auto-Encoder based Multi-View missing data Completion framework (AEMVC) to learn common representations for AD diagnosis. Specifically, we firstly map the original complete view to a latent space using an auto-encoder network framework. Then, the latent representations measuring statistical dependence learned from the complete view are used to complement the kernel matrix of the incomplete view in the kernel space. Meanwhile, the structural information of original data and the inherent association between views are maintained by graph regularization and Hilbert-Schmidt Independence Criterion (HSIC) constraints. Finally, a kernel based multi-view method is applied to the learned kernel matrix for the acquisition of common representations. Experimental results achieved on Alzheimers Disease Neuroimaging Initiative (ADNI) datasets validate the effectiveness of the proposed method.
On the Texture Bias for Few-Shot CNN Segmentation

Despite the initial belief that Convolutional Neural Networks (CNNs) are driven by shapes to perform visual recognition tasks, recent evidence suggests that texture bias in CNNs provides higher performing models when learning on large labeled training datasets. This contrasts with the perceptual bias in the human visual cortex, which has a stronger preference towards shape components. Perceptual differences may explain why CNNs achieve human-level performance when large labeled datasets are available, but their performance significantly degrades in low-labeled data scenarios, such as few-shot semantic segmentation. To remove the texture bias in the context of few-shot learning, we propose a novel architecture that integrates a set of Difference of Gaussians (DoG) to attenuate high-frequency local components in the feature space. This produces a set of modified feature maps, whose high-frequency components are diminished at different standard deviation values of the Gaussian distribution in the spatial domain. As this results in multiple feature maps for a single image, we employ a bi-directional convolutional long-short-term-memory to efficiently merge the multi scale-space representations. We perform extensive experiments on three well-known few-shot segmentation benchmarks -Pascal i5, COCO-20i and FSS-1000- and demonstrate that our method outperforms state-of-the-art approaches in two datasets under the same conditions.
Trear: Transformer-Based RGB-D Egocentric Action Recognition

In this article, we propose a transformer-based RGB-D egocentric action recognition framework, called Trear. It consists of two modules: 1) interframe attention encoder and 2) mutual-attentional fusion block. Instead of using optical flow or recurrent units, we adopt a self-attention mechanism to model the temporal structure of the data from different modalities. Input frames are cropped randomly to mitigate the effect of the data redundancy. Features from each modality are interacted through the proposed fusion block and combined through a simple yet effective fusion operation to produce a joint RGB-D representation. Empirical experiments on two large egocentric RGB-D data sets: 1) THU-READ and 2) first-person hand action, and one small data set, wearable computer vision systems, have shown that the proposed method outperforms the state-of-the-art results by a large margin.
SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving

State-of-the-art self-supervised learning approaches for monocular depth estimation usually suffer from scale ambiguity. They do not generalize well when applied on distance estimation for complex projection models such as in fisheye and omnidirectional cameras. This paper introduces a novel multi-task learning strategy to improve self-supervised monocular distance estimation on fisheye and pinhole camera images. Our contribution to this work is threefold: Firstly, we introduce a novel distance estimation network architecture using a self-attention based encoder coupled with robust semantic feature guidance to the decoder that can be trained in a one-stage fashion. Secondly, we integrate a generalized robust loss function, which improves performance significantly while removing the need for hyperparameter tuning with the reprojection loss. Finally, we reduce the artifacts caused by dynamic objects violating static world assumptions using a semantic masking strategy. We significantly improve upon the RMSE of previous work on fisheye by 25% reduction in RMSE. As there is little work on fisheye cameras, we evaluated the proposed method on KITTI using a pinhole model. We achieved state-of-the-art performance among self-supervised methods without requiring an external scale estimation.
<i>e-PoS</i>: Making Proof-of-Stake Decentralized and Fair

Blockchain applications that rely on the Proof-of-Work (PoW) have increasingly become energy inefficient with a staggering carbon footprint. In contrast, energy efficient alternative consensus protocols such as Proof-of-Stake (PoS) may cause centralization and unfairness in the blockchain system. To address these challenges, we propose a modular version of PoS-based blockchain systems called e-PoS that resists the centralization of network resources by extending mining opportunities to a wider set of stakeholders. Moreover, e-PoS leverages the in-built system operations to promote fair mining practices by penalizing malicious entities. We validate e-PoS 's achievable objectives through theoretical analysis and simulations. Our results show that e-PoS ensures fairness and decentralization, and can be applied to existing blockchain applications.
Barriers to blockchain adoption in health-care industry: an Indian perspective

Purpose This paper aims to identify barriers toward the adoption of blockchain (BC) technology in Indian health-care industry and also examines the significant issues of BC applications in health-care industry. Design/methodology/approach The barriers of the study are identified by two phases including the review of literature and semistructured interviews with hospital staff and administration operating in India. The experts ( N = 15) are being taken from top-level management, IT experts and patients from the hospitals. The study implemented integrated total interpretative structural modeling-FUZZY-Cross-impact matrix multiplication applied to classification (TISM-FUZZY-MICMAC) methods for identifying the interrelationship among the barriers. Findings A total of 15 barriers have been determined in the Indian health-care industry through discussion with the selected experts. TISM is applied to develop multilevel structure for BC barriers. Further, FUZZY-MICMAC has been used to compute driving and dependent barriers. The findings suggest that low awareness related to legal issues and low support from high level of management have maximum driving power. Research limitations/implications The present study applies multicriterion approach to identify the limited barriers in BC adoption in health care. Future studies may develop the relationship and mark down the steps for implementation of BC in health-care setting of a developing economy. Empirical study can be conducted to verify the results along with selected case studies. Practical implications The present study identifies the BC adoption barriers in health-care industry. The study examines the pertinent issues in context to major support required, bottlenecks in adoption, key benefits of adoption planning and activities. The technology adoption practices are expected to provide applications such as distributed, secured medical and clinical data and patient centric systems that will enhance the efficiency of the health-care industry. Originality/value The study is among few primary studies that identify and analyze the BC adoption in health-care industry.