Reviewerly/exHarmony huggingface dataset:
Temporal Context Aggregation Network for Temporal Action Proposal Refinement

Temporal action proposal generation aims to estimate temporal intervals of actions in untrimmed videos, which is a challenging yet important task in the video understanding field. The proposals generated by current methods still suffer from inaccurate temporal boundaries and inferior confidence used for retrieval owing to the lack of efficient temporal modeling and effective boundary context utilization. In this paper, we propose Temporal Context Aggregation Network (TCANet) to generate high-quality action proposals through "local and global" temporal context aggregation and complementary as well as progressive boundary refinement. Specifically, we first design a Local-Global Temporal Encoder (LGTE), which adopts the channel grouping strategy to efficiently encode both "local and global" temporal inter-dependencies. Furthermore, both the boundary and internal context of proposals are adopted for frame-level and segment-level boundary regressions, respectively. Temporal Boundary Regressor (TBR) is designed to combine these two regression granularities in an end-to-end fashion, which achieves the precise boundaries and reliable confidence of proposals through progressive refinement. Extensive experiments are conducted on three challenging datasets: HACS, ActivityNet-v1.3, and THUMOS-14, where TCANet can generate proposals with high precision and recall. By combining with the existing action classifier, TCANet can obtain remarkable temporal action detection performance compared with other methods. Not surprisingly, the proposed TCANet won the 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> place in the CVPR 2020 - HACS challenge leaderboard on temporal action localization task.
MetaMed: Few-shot medical image classification using gradient-based meta-learning

The occurrence of long-tailed distributions and unavailability of high-quality annotated images is a common phenomenon in medical datasets. The use of conventional Deep Learning techniques to obtain an unbiased model with high generalization accuracy for such datasets is a challenging task. Thus, we formulated a few-shot learning problem and presented a meta-learning-based “MetaMed” approach. The model presented here can adapt to rare disease classes with the availability of few images, and less compute. MetaMed is validated on three publicly accessible medical datasets – Pap smear, BreakHis, and ISIC 2018. We used advanced image augmentation techniques like CutOut, MixUp, and CutMix to overcome the problem of over-fitting. Our approach has shown promising results on all the three datasets with an accuracy of more than 70%. Inclusion of advanced augmentation techniques regularizes the model and increases the generalization capability by 2–5%. Comparative analysis of MetaMed against transfer learning demonstrated that MetaMed classifies images with a higher confidence score and on average outperforms transfer learning for 3, 5, and 10-shot tasks for both 2-way and 3-way classification.
Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

Supervised learning based object detection frameworks demand plenty of laborious manual annotations, which may not be practical in real applications. Semi-supervised object detection (SSOD) can effectively leverage unlabeled data to improve the model performance, which is of great significance for the application of object detection models. In this paper, we revisit SSOD and propose Instant-Teaching, a completely end-to-end and effective SSOD framework, which uses instant pseudo labeling with extended weak-strong data augmentations for teaching during each training iteration. To alleviate the confirmation bias problem and improve the quality of pseudo annotations, we further propose a co-rectify scheme based on Instant-Teaching, denoted as Instant-Teaching <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∗</sup> . Extensive experiments on both MS-COCO and PASCAL VOC datasets substantiate the superiority of our framework. Specifically, our method surpasses state-of-the-art methods by 4.2 mAP on MS-COCO when using 2% labeled data. Even with full supervised information of MS-COCO, the proposed method still outperforms state-of-the-art methods by about 1.0 mAP. On PASCAL VOC, we can achieve more than 5 mAP improvement by applying VOC07 as labeled data and VOC12 as unlabeled data.
DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors

In the paper, we present a new dual attention method called DanHAR, which blends channel and temporal attention on residual networks to improve feature representation ability for sensor-based HAR task. Specially, the channel attention plays a key role in deciding what to focus, i.e., sensor modalities, while the temporal attention can focus on the target activity from a long sensor sequence to tell where to focus. Extensive experiments are conducted on four public HAR datasets, as well as weakly labeled HAR dataset. The results show that dual attention mechanism is of central importance for many activity recognition tasks. We obtain 2.02%, 4.20%, 1.95%, 5.22% and 5.00% relative improvement over regular ConvNets respectively on WISDM dataset, UNIMIB SHAR dataset, PAMAP2 dataset, OPPORTUNITY dataset, as well as weakly labeled HAR dataset. The DanHAR is able to surpass other state-of-the-art algorithms at negligible computational overhead. Visualizing analysis is conducted to show that the proposed attention can capture the spatial–temporal dependencies of multimodal sensing data, which amplifies the more important sensor modalities and timesteps during classification. The results are in good agreement with normal human intuition.
No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency

The goal of No-Reference Image Quality Assessment (NR-IQA) is to estimate the perceptual image quality in accordance with subjective evaluations, it is a complex and unsolved problem due to the absence of the pristine reference image. In this paper, we propose a novel model to address the NR-IQA task by leveraging a hybrid approach that benefits from Convolutional Neural Networks (CNNs) and self-attention mechanism in Transformers to extract both local and non-local features from the input image. We capture local structure information of the image via CNNs, then to circumvent the locality bias among the extracted CNNs features and obtain a non-local representation of the image, we utilize Transformers on the extracted features where we model them as a sequential input to the Transformer model. Furthermore, to improve the monotonicity correlation between the subjective and objective scores, we utilize the relative distance information among the images within each batch and enforce the relative ranking among them. Last but not least, we observe that the performance of NR-IQA models degrades when we apply equivariant transformations (e.g. horizontal flipping) to the inputs. Therefore, we propose a method that leverages self-consistency as a source of self-supervision to improve the robustness of NR-IQA models. Specifically, we enforce self-consistency between the outputs of our quality assessment model for each image and its transformation (horizontally flipped) to utilize the rich self-supervisory information and reduce the uncertainty of the model. To demonstrate the effectiveness of our work, we evaluate it on seven standard IQA datasets (both synthetic and authentic) and show that our model achieves state-of-the-art results on various datasets. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Blockchain in the accounting, auditing and accountability fields: a bibliometric and coding analysis

Purpose This study aims to offer a bibliometric and coding analysis of blockchain articles published in the accounting, auditing and accountability fields. Design/methodology/approach The data were collected using the Scopus database and a bibliometric and qualitative coding analysis with the keywords “blockchain” and “accounting” or “auditing” or “accountability.” Of the 514 initial sources, 93 peer-reviewed papers, book chapters and conference proceedings in the areas of business, management and accounting were finally selected. Nonscientific sources such as nonpeer-reviewed books and white papers were excluded. Findings This study reveals a promising and multidisciplinary field of research dominated by scholars and less by practitioners. Qualitative research, especially discourse analysis, is the most used method among authors. This study gives some useful insights about blockchain's definition and characteristics, business models, processes involved, connection with other technologies and relationships with accounting theories. Among the most interesting insights, the results confirm that technology as an external force can create an intersection among several research areas: accounting, auditing, accountability, business, management, computer science and engineering fields. Finally, in terms of research themes, although blockchain has a clear effect on auditing accounting, the links with the area of accountability are less clear and validated. Originality/value This study highlights the current state of the field, combining methodological approaches and providing valuable future research insights. Additionally, it is also a starting point for professionals to fully understand blockchain's characteristics and potential with a constructive and systemic approach.
Security and Privacy-Enhanced Federated Learning for Anomaly Detection in IoT Infrastructures

Internet of Things (IoT) anomaly detection is significant due to its fundamental roles of securing modern critical infrastructures, such as falsified data injection detection and transmission line faults diagnostic in smart grids. Researchers have proposed various detection methods fostered by machine learning (ML) techniques. Federated learning (FL), as a promising distributed ML paradigm, has been employed recently to improve detection performance due to its advantages of privacy-preserving and lower latency. However, existing FL-based methods still suffer from efficiency, robustness, and security challenges. To address these problems, in this article, we initially introduce a blockchain-empowered decentralized and asynchronous FL framework for anomaly detection in IoT systems, which ensures data integrity and prevents single-point failure while improving the efficiency. Further, we design an improved differentially private FL based on generative adversarial nets, aiming to optimize data utility throughout the training process. To the best of our knowledge, it is the first system to employ a decentralized FL approach with privacy-preserving for IoT anomaly detection. Simulation results on the real-world dataset demonstrate the superior performance from aspects of robustness, accuracy, and fast convergence while maintaining high level of privacy and security protection.
Ego4D: Around the World in 3,000 Hours of Egocentric Video

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite. It offers 3,670 hours of dailylife activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 931 unique camera wearers from 74 worldwide locations and 9 different countries. The approach to collection is designed to uphold rigorous privacy and ethics standards, with consenting participants and robust de-identification procedures where relevant. Ego4D dramatically expands the volume of diverse egocentric video footage publicly available to the research community. Portions of the video are accompanied by audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event. Furthermore, we present a host of new benchmark challenges centered around understanding the first-person visual experience in the past (querying an episodic memory), present (analyzing hand-object manipulation, audio-visual conversation, and social interactions), and future (forecasting activities). By publicly sharing this massive annotated dataset and benchmark suite, we aim to push the frontier of first-person perception. Project page: https://ego4d-data.org/
FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration

3D human pose estimation can facilitate various applications, such as assistive technologies and AR/VR. Most existing monocular 3D pose estimation approaches only focus on a single body part, neglecting the fact that the essential nuance of human motion is conveyed through a concert of subtle movements of face, hands, and body. In this paper, we present FrankMocap, a fast and accurate whole-body 3D pose estimation system that can produce 3D face, hands, and body simultaneously from in-the-wild monocular images. The idea of FrankMocap is its modular design: We first run 3D pose regression methods for face, hands, and body independently, followed by composing the regression outputs via an integration module. The separate regression modules allow us to take full advantage of their state-of-the-art performances without compromising the original accuracy and reliability in practice. We develop three different integration modules that trade off between latency and accuracy. All of them are capable of providing simple yet effective solutions to unify the separate outputs into seamless whole-body pose estimation results. We quantitatively and qualitatively demonstrate that our modularized system outperforms both the optimization-based and end-to-end methods of estimating whole-body pose.
Pros and cons of GAN evaluation measures: New developments

This work is an update of my previous paper on the same topic published a few years ago (Borji, 2019). With the dramatic progress in generative modeling, a suite of new quantitative and qualitative techniques to evaluate models has emerged. Although some measures such as Inception Score, Fréchet Inception Distance, Precision–Recall, and Perceptual Path Length are relatively more popular, GAN evaluation is not a settled issue and there is still room for improvement. Here, I describe new dimensions that are becoming important in assessing models (e.g. bias and fairness) and discuss the connection between GAN evaluation and deepfakes. These are important areas of concern in the machine learning community today and progress in GAN evaluation can help mitigate them.
Monocular, One-stage, Regression of Multiple 3D People

This paper focuses on the regression of multiple 3D people from a single RGB image. Existing approaches predominantly follow a multi-stage pipeline that first detects people in bounding boxes and then independently regresses their 3D body meshes. In contrast, we propose to Regress all meshes in a One-stage fashion for Multiple 3D People (termed ROMP). The approach is conceptually simple, bounding box-free, and able to learn a per-pixel representation in an end-to-end manner. Our method simultaneously predicts a Body Center heatmap and a Mesh Parameter map, which can jointly describe the 3D body mesh on the pixel level. Through a body-center-guided sampling process, the body mesh parameters of all people in the image are easily extracted from the Mesh Parameter map. Equipped with such a fine-grained representation, our one-stage framework is free of the complex multi-stage process and more robust to occlusion. Compared with state-of-the-art methods, ROMP achieves superior performance on the challenging multi-person benchmarks, including 3DPW and CMU Panoptic. Experiments on crowded/occluded datasets demonstrate the robustness under various types of occlusion. The code, released at https://github.com/Arthur151/ROMP, is the first real-time implementation of monocular multi-person 3D mesh regression.
AR-RRNS: Configurable reliable distributed data storage systems for Internet of Things to ensure security

Benefits of Internet of Things and cloud–fog-edge computing are associated with the risks of confidentiality, integrity, and availability related with the loss of information, denial of access for a long time, information leakage, conspiracy and technical failures. In this article, we propose a configurable, reliable, and confidential distributed data storage scheme with the ability to process encrypted data and control results of computations. Our system utilizes Redundant Residue Number System (RRNS) with new method of error correction codes and secret sharing schemes. We introduce the concept of an approximate value of a rank of a number (AR), which allows us to reduce the computational complexity of the decoding from RNS to binary representation, and size of the coefficients. Based on the properties of the approximate value and arithmetic properties of RNS, we introduce AR-RRNS method for error detection, correction, and controlling computational results. We provide a theoretical basis to configure probability of information loss, data redundancy, speed of encoding and decoding to cope with different objective preferences, workloads, and storage properties. Theoretical analysis shows that by appropriate selection of RRNS parameters, the proposed scheme allows not only increasing safety, reliability, and reducing an overhead of data storage, but also processing of encrypted data.
Edge Computing Assisted Adaptive Mobile Video Streaming

Nearly all bitrate adaptive video content delivered today is streamed using protocols that run a purely client based adaptation logic. The resulting lack of coordination may lead to suboptimal user experience and resource utilization. As a response, approaches that include the network and servers in the adaptation process are emerging. In this article, we present an optimized solution for network assisted adaptation specifically targeted to mobile streaming in multi-access edge computing (MEC) environments. Due to NP-Hardness of the problem, we have designed a heuristic-based algorithm with minimum need for parameter tuning and having relatively low complexity. We then study the performance of this solution against two popular client-based solutions, namely Buffer-Based Adaptation (BBA) and Rate-Based Adaptation (RBA), as well as to another network assisted solution. Our objective is two fold: First, we want to demonstrate the efficiency of our solution and second to quantify the benefits of network-assisted adaptation over the client-based approaches in mobile edge computing scenarios. The results from our simulations reveal that the network assisted adaptation clearly outperforms the purely client-based DASH heuristics in some of the metrics, not all of them, particularly, in situations when the achievable throughput is moderately high or the link quality of the mobile clients does not differ from each other substantially.
Incentivizing Consensus Propagation in Proof-of-Stake Based Consortium Blockchain Networks

In proof-of-stake based consortium blockchain networks, pre-selected miners compete to solve a crypto-puzzle with a successfully mining probability proportional to the amount of their stakes. When the puzzle is solved, the miners are encouraged to take part in mined block propagation for verification to win a transaction fee from the blockchain user. The mined block should be propagated over wired or wireless networks, and be verified as quickly as possible to decrease consensus propagation delay. In this letter, we study incentivizing the consensus propagation considering the tradeoff between the network delay of block propagation process and offered transaction fee from the blockchain user. A Stackelberg game is then formulated to jointly maximize utility of the blockchain user and individual profit of the miners. The blockchain user acting as the leader sets the transaction fee for block verification. The miners acting as the followers decide on the number of recruited verifiers over wired or wireless networks. We apply the backward induction to analyze the existence and uniqueness of the Stackelberg equilibrium. Performance evaluation validates the feasibility and efficiency of the proposed game model in consensus propagation.
Passive Attacks Against Searchable Encryption

Searchable encryption (SE) provides a privacy-preserving mechanism for data users to search over encrypted data stored on a remote server. Researchers have designed a number of SE schemes with high efficiency yet allowing some degree of leakage profile to the remote server. The leakage, however, should be further measured to allow us to understand what types of attacks an SE scheme would encounter. This paper considers passive attacks that make inferences based on prior knowledge and observations on queries issued by users. This is in contrast to previously studied active attacks that adaptively inject files and queries. We consider several assumptions on the types or prior knowledge the attacker possessed and propose a few passive attacks. In particular, under the “full-fledged” assumption, the keyword recovery rate of our attack is optimal in the sense that it is equal to the theoretical upper bound. We further present several enhanced attacks under other weaker assumptions on various levels of the prior knowledge that the attacker can obtain, in which the keyword recovery rates are optimal or nearly optimal (i.e., approaching the theoretical upper bound). In addition, we provide extensive experiments to show the “power” of our passive attacks. This paper highlights the importance of minimizing the prior knowledge of a server and the leakage of search queries. It also shows that simply distorting the frequency of the keyword to hold against our passive attacks may not scale well.
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll" id="d1e10265" altimg="si80.gif"><mml:mi>β</mml:mi></mml:math>-Chaotic map enabled Grey Wolf Optimizer

The diversification (exploration) and intensification (exploitation) are two main attributes of any population-based metaheuristic algorithm. It is essential for any algorithm that in exploration phase the search space is utilized and explored properly through random behavior, on the other hand, the progression of the search in a viable direction to obtain global minima, should be performed through strategic behavior in exploitation phase. A proper balance between these two can be achieved by an adaptive mechanism in every algorithm. Robustness of an algorithm is judged by the efficacy of these two attributes along with the efficiency of the bridging mechanism. In literature, the positive impact of inculcation of chaotic sequences on the efficacy of these attributes has been reported. With this motivation, the paper presents an adaptive bridging mechanism based on β-chaotic sequence for the improvement of Grey Wolf Optimizer (GWO). The control vector of classical GWO is integrated with the β-chaotic sequence for better exploration and exploitation virtues. The new variant β-GWO is benchmarked on two benchmark suites 1 and 2 that include 12 shifted and biased functions and 29 Congress on Evolutionary Computation-2017 (CEC-2017) functions. Sensitivity Dependence of Initial Conditions (SDIC) is performed for tuning the initial parameters. The comparison of the proposed variant with other contemporary algorithms is carried out and different statistical tests are performed to judge the efficacy of the proposed variant. Further, the applicability of the proposed variant is checked with two real engineering problems namely frequency modulated sound waves parameter estimation problem and strategic bidding in the energy market. Results reveal that the proposed chaotic variant exhibits better exploration and exploitation qualities.