Reviewerly/exHarmony huggingface dataset:
Exploring Sub-Action Granularity for Weakly Supervised Temporal Action Localization

Modeling cross-video relationship is an important issue for the weakly supervised temporal action localization task. To this end, traditional methods operate at the action level and rely on complicated strategies to prepare triplet samples, which only mines the cross-video relationships among three videos from two categories. In this work, we observe that action instances from different categories could exhibit similar motion patterns, i.e. subaction, and propose to operate at the sub-action granularity to elaborately explore cross-video relationships. However, only given video-level category labels, the sub-actions are undefined and not annotated. To tackle this challenge, we represent video features via a group of sub-actions, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> the sub-action family. Specifically, the sub-action family contains multiple feature vectors, where each vector is in charge of representing a specific sub-action. The sub-action family is shared among all videos in the dataset, while all videos contribute to the learning of the sub-action family. Consequently, we can not only get rid of the complicated sampling strategy but also thoroughly mine cross-video relationships from all available videos in the dataset. To learn feature vectors within the sub-action family, we employ a bottom-up temporal action localization paradigm and introduce an extra top-down branch. The sub-action family is introduced into the top-down branch, and it learns feature vectors via representing raw video features. Moreover, we propose a consistency loss to guide the learning process and a diversity loss to mine distinct sub-actions. Extensive experiments are carried out on three benchmark datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> THUMOS14, ActivityNet v1.2 and ActivityNet v1.3, and the proposed method builds new high performance.
Combining max-pooling and wavelet pooling strategies for semantic image segmentation

This paper presents a novel multi-pooling architecture generated by combining the advantages of wavelet and max-pooling operations in convolutional neural networks (CNNs), focusing on semantic segmentation tasks. CNNs often use pooling to reduce the number of parameters, improve invariance to certain distortions, and enlarge the receptive field. However, pooling can cause information loss and thus is detrimental to further operations such as feature extraction and analysis. This problem is particularly critical for semantic segmentation, where each pixel of an image is assigned to a specific class to divide the image into disjoint regions of interest. To address this problem, pooling strategies based on wavelets-operations have been proposed with the promise to achieve a better trade-off between receptive field size and computational efficiency. Previous works have confirmed the superiority of wavelet pooling over the traditional one in semantic segmentation tasks. However, we have observed in our computational experiments that the expressive gains reported from the use of wavelet pooling in other segmentation tasks were not observed in the scope of aerial imagery due to imprecision in the segmentation of image details. The combination of wavelet pooling and max-pooling, a solution not yet reported in the literature, can address that issue. Such gap observed in the pooling area motivated the two proposals that are the main contributions of this paper: (a) A new multi-pooling strategy combining wavelet and traditional pooling in a new network structure suitable for aerial image segmentation tasks; (b) Two-stream architectures using the traditional max-pooling and wavelet pooling as streams. These proposals were implemented using the Segnet, a known architecture for semantic segmentation. The computational experiments, based on the IRRG images from the Potsdam and Vaihingen data sets, demonstrated that the proposed architectures surpassed the original Segnet architecture’s performance with results comparable to state-of-the-art approaches.
With Great Dispersion Comes Greater Resilience: Efficient Poisoning Attacks and Defenses for Linear Regression Models

With the rise of third parties in the machine learning pipeline, the service provider in “Machine Learning as a Service” (MLaaS), or external data contributors in online learning, or the retraining of existing models, the need to ensure the security of the resulting machine learning models has become an increasingly important topic. The security community has demonstrated that without transparency of the data and the resulting model, there exist many potential security risks, with new risks constantly being discovered. In this paper, we focus on one of these security risks - poisoning attacks. Specifically, we analyze how attackers may interfere with the results of regression learning by poisoning the training datasets. To this end, we analyze and develop a new poisoning attack algorithm. Our attack, termed Nopt, in contrast with previous poisoning attack algorithms, can produce larger errors with the same proportion of poisoning data-points. Furthermore, we also significantly improve the state-of-the-art defense algorithm, termed TRIM, proposed by Jagielsk et al. (IEEE S&P 2018), by incorporating the concept of probability estimation of clean data-points into the algorithm. Our new defense algorithm, termed Proda, demonstrates an increased effectiveness in reducing errors arising from the poisoning dataset through optimizing ensemble models. We highlight that the time complexity of TRIM had not been estimated; however, we deduce from their work that TRIM can take exponential time complexity in the worst-case scenario, in excess of Proda's logarithmic time. The performance of both our proposed attack and defense algorithms is extensively evaluated on four real-world datasets of housing prices, loans, health care, and bike sharing services. We hope that our work will inspire future research to develop more robust learning algorithms immune to poisoning attacks.
A decision model for programming language ecosystem selection: Seven industry case studies

Software development is a continuous decision-making process that mainly relies on the software engineer’s experience and intuition. One of the essential decisions in the early stages of the process is selecting the best fitting programming language ecosystem based on the project requirements. A significant number of criteria, such as developer availability and consistent documentation, in addition to the number of available options in the market, lead to a challenging decision-making process. As the selection of programming language ecosystems depends on the application to be developed and its environment, a decision model is required to analyze the selection problem using systematic identification and evaluation of potential alternatives for a development project. Recently, we introduced a framework to build decision models for technology selection problems in software production. Furthermore, we designed and implemented a decision support system that uses such decision models to support software engineers with their decision-making problems. This study presents a decision model based on the framework for the programming language ecosystem selection problem. The decision model has been evaluated through seven real-world case studies at seven software development companies. The case study participants declared that the approach provides significantly more insight into the programming language ecosystem selection process and decreases the decision-making process’s time and cost. With the decision model, software engineers can more rapidly evaluate and select programming language ecosystems. Having the knowledge in the decision model readily available supports software engineers in making more efficient and effective decisions that meet their requirements and priorities. Furthermore, such reusable knowledge can be employed by other researchers to develop new concepts and solutions for future challenges.
New Color Image Zero-Watermarking Using Orthogonal Multi-Channel Fractional-Order Legendre-Fourier Moments

Zero-watermarking methods provide promising solutions and impressive performance for copyright protection of images without changing the original images. In this paper, a novel zero-watermarking method for color images is envisioned. Our envisioned approach is based on multi-channel orthogonal Legendre Fourier moments of fractional orders, referred to as MFrLFMs. In this method, a highly precise Gaussian integration method is utilized to calculate MFrLFMs. Then, based on the selected accurate MFrLFMs moments, a zero-watermark is constructed. Due to their accuracy, geometric invariances, and numerical stability, the proposed MFrLFMs-based zero-watermarking method shows excellent resistance against various attacks. Performed experiments using the proposed watermarking method show the outperformance over existing watermarking algorithms.
Can humans perform mental regression on a graph? Accuracy and bias in the perception of scatterplots

Despite the widespread use of graphs, little is known about how fast and how accurately we can extract information from them. Through a series of four behavioral experiments, we characterized human performance in "mental regression", i.e. the perception of statistical trends from scatterplots. When presented with a noisy scatterplot, even as briefly as 100 ms, human adults could accurately judge if it was increasing or decreasing, fit a regression line, and extrapolate outside the original data range, for both linear and non-linear functions. Performance was highly consistent across those three tasks of trend judgment, line fitting and extrapolation. Participants' linear trend judgments took into account the slope, the noise, and the number of data points, and were tightly correlated with the t-test classically used to evaluate the significance of a linear regression. However, they overestimated the absolute value of the regression slope. This bias was inconsistent with ordinary least squares (OLS) regression, which minimizes the sum of square deviations, but consistent with the use of Deming regression, which treats the x and y axes symmetrically and minimizes the Euclidean distance to the fitting line. We speculate that this fast but biased perception of scatterplots may be based on a "neuronal recycling" of the human visual capacity to identify the medial axis of a shape.
SliceLine: Fast, Linear-Algebra-based Slice Finding for ML Model Debugging

Slice finding---a recent work on debugging machine learning (ML) models---aims to find the top-K data slices (e.g., conjunctions of predicates such as gender female and degree PhD), where a trained model performs significantly worse than on the entire training/test data. These slices may be used to acquire more data for the problematic subset, add rules, or otherwise improve the model. In contrast to decision trees, the general slice finding problem allows for overlapping slices. The resulting search space is huge as it covers all subsets of features and their distinct values. Hence, existing work primarily relies on heuristics and focuses on small datasets that fit in memory of a single node. In this paper, we address these scalability limitations of slice finding in a holistic manner from both algorithmic and system perspectives. We leverage monotonicity properties of slice sizes, errors and resulting scores to facilitate effective pruning. Additionally, we present an elegant linear-algebra-based enumeration algorithm, which allows for fast enumeration and automatic parallelization on top of existing ML systems. Experiments with different real-world regression and classification datasets show that effective pruning and efficient sparse linear algebra renders exact enumeration feasible, even for datasets with many features, correlations, and data sizes beyond single node memory.
Understanding decentralized civic engagement: Focus on peer-to-peer and blockchain-driven perspectives on e-participation

Decentralized civic engagement, i.e. engagement which is not initiated and managed by government, has been often used as a popular slogan in various public speeches to highlight the importance of genuine civic interests in governance, yet offering very limited options on how to actually promote such involvement. The advent of the digital era has facilitated the development of a plethora of digital participatory platforms that could presumably help to boost the effective engagement of civic communities in decision making processes in an explicitly decentralized manner and, more so, the demand for such decentralized e-participation services is increasing during the current Covid-19 crisis. However, there is a real shortage of studies that directly focus on the analysis of such autonomous cases and all the more so from real life contexts in their natural settings. In this regard, this paper aims to fill the gap by analyzing the phenomena from an intrinsically peer-to-peer networking perspective, which could be observed today in popular autonomous civic engagement initiatives that have been launched in recent times in the area.
Generic server-aided secure multi-party computation in cloud computing

Cloud computing has become one of the most popular distributed computing paradigms in recent years. With its advantages of low cost, on-demand flexibility, and high data processing abilities, more and more enterprises have adopted the cloud computing paradigm to build up their IT infrastructure. By performing collaborative computation tasks (e.g., big data analysis tasks) with multiple datasets of different correlated enterprises in cloud computing, the generated valuable information will provide the enterprises with higher productivity and financial gains. However, due to the privacy concerns from the enterprises, how to efficiently enable them to achieve secure multi-party joint datasets analysis in cloud computing without leaking their own private dataset becomes a critical but challenging problem for the enterprises. In this paper, focusing on securely performing any collaborative computation task in cloud computing, we construct a generic server-aided secure multi-party computation protocol to tackle the problem. Our solution can provide security guarantee in the setting where at most n-1 client parties are malicious while the server is semi-honest and there is no collusion between the server and clients. The security and experimental performance analysis show that this work is currently the most efficient server-aided secure multi-party computation protocol with the same security guarantee compared with all the previous works to the best of our knowledge.
Improved Noise2Noise Denoising with Limited Data

Deep learning methods have proven to be very effective for the task of image denoising even when clean reference images are not available. In particular, Noise2Noise, which requires pairs of noisy images during the training phase, has been shown to yield results as good as approaches using pairs of noisy and clean images (Noise2Clean). However, the performance of Noise2Noise drops when the amount of training data is reduced, limiting its capability in practical scenarios.In this work, an analysis of the Noise2Noise learning strategy is done using real noise and synthetic datasets. This paper demonstrates, using diverse network architectures and loss functions, that the duplicity of information in the noisy pairs can be exploited to reach increased denoising performance of Noise2Noise. Additionally, the issue of overfitting in Noise2Noise is analyzed, given its relevance when training with limited data, and an interpretable early termination criterion is proposed.
A novel image encryption scheme based on non-adjacent parallelable permutation and dynamic DNA-level two-way diffusion

As a medium that can convey information vividly, the application of digital images is increasingly widespread. Under various application environments, researchers are expected to provide more secure and efficient protection for digital images. In this paper, a novel image encryption scheme is proposed, which is based on non-adjacent parallelable permutation and dynamic DNA-level two-way diffusion (IES-NP-DDT). To achieve higher encryption efficiency, each encryption step of this scheme is designed with the parallelizability of operations in mind. Firstly, the plain image is logically divided into adjacent blocks and permutation blocks. And the permutation blocks are then randomly permutated in parallel by multiple computing units. Next, the permutated image is split and encoded into four DNA planes. When encoding each 2-bit plane, the computing units dynamically adopt encoding rules according to chaotic sequences. Combined with the dynamic DNA decoding, the excellent nonlinearity of the diffusion process is ensured. Finally, determined by the hash value, key components and chaotic values, various two-way DNA-level diffusions are carried out in parallel on the DNA planes. Consequently, a smallest change of any pixel is efficiently diffused to the whole image. Extensive simulation tests and theoretical analyses conducted in this paper have demonstrated the excellent performance of IES-NP-DDT.
Quantifying SARS‐CoV‐2 Infection Risk Within the Google/Apple Exposure Notification Framework to Inform Quarantine Recommendations

Most early Bluetooth-based exposure notification apps use three binary classifications to recommend quarantine following SARS-CoV-2 exposure: a window of infectiousness in the transmitter, ≥15 minutes duration, and Bluetooth attenuation below a threshold. However, Bluetooth attenuation is not a reliable measure of distance, and infection risk is not a binary function of distance, nor duration, nor timing. We model uncertainty in the shape and orientation of an exhaled virus-containing plume and in inhalation parameters, and measure uncertainty in distance as a function of Bluetooth attenuation. We calculate expected dose by combining this with estimated infectiousness based on timing relative to symptom onset. We calibrate an exponential dose-response curve based on infection probabilities of household contacts. The probability of current or future infectiousness, conditioned on how long postexposure an exposed individual has been symptom-free, decreases during quarantine, with shape determined by incubation periods, proportion of asymptomatic cases, and asymptomatic shedding durations. It can be adjusted for negative test results using Bayes' theorem. We capture a 10-fold range of risk using six infectiousness values, 11-fold range using three Bluetooth attenuation bins, ∼sixfold range from exposure duration given the 30 minute duration cap imposed by the Google/Apple v1.1, and ∼11-fold between the beginning and end of 14 day quarantine. Public health authorities can either set a threshold on initial infection risk to determine 14-day quarantine onset, or on the conditional probability of current and future infectiousness conditions to determine both quarantine and duration.
Decentralizing Private Blockchain-IoT Network with OLSR

With data transparency and immutability, the blockchain can provide trustless and decentralized services for Internet of Things (IoT) applications. However, most blockchain-IoT networks, especially those with a private blockchain, are built on top of an infrastructure-based wireless network (i.e., using Wi-Fi access points or cellular base stations). Hence, they are still under the risk of Single-Point-of-Failure (SPoF) on the network layer, hindering the decentralization merit, for example, when the access points or base stations get failures. This paper presents an Optimized Link State Routing (OLSR) protocol-based solution for that issue in a private blockchain-IoT application. By decentralizing the underlying network with OLSR, the private blockchain network can avoid SPoF and automatically recover after a failure. Single blockchain connections can be extended to multiple ad hoc hops. Services over blockchain become flexible to fit various IoT scenarios. We show the effectiveness of our solution by constructing a private Ethereum blockchain network running on IoT devices (i.e., Raspberry Pi model 4) with environmental data sensing (i.e., Particular Matter (PM)). The IoT devices use OLSR to form an ad hoc network. The environment data are collected and propagated in transactions to a pre-loaded smart contract periodically. We then evaluate the IoT blockchain network’s recovery time when facing a link error. The evaluation results show that OLSR can automatically recover after the failure. We also evaluate the transaction-oriented latency and block-oriented latency, which indicates the blocks have a high transmission quality, while transactions are transferred individually.
Multi-Branching Temporal Convolutional Network for Sepsis Prediction

Sepsisis among the leading causes of morbidity and mortality in modern intensive care units. Accurate sepsis prediction is of critical importance to save lives and reduce medical costs. The rapid advancements in sensing and information technology facilitate the effective monitoring of patients' health conditions, generating a wealth of medical data, and provide an unprecedented opportunity for data-driven diagnosis of sepsis. However, real-world medical data are often complexly structured with a high level of uncertainty (e.g., missing values, imbalanced data). Realizing the full data potential depends on developing effective analytical models. In this paper, we propose a novel predictive framework with Multi-Branching Temporal Convolutional Network (MB-TCN) to model the complexly structured medical data for robust prediction of sepsis. The MB-TCN framework not only efficiently handles the missing value and imbalanced data issues but also effectively captures the temporal pattern and heterogeneous variable interactions. We evaluate the performance of the proposed MB-TCN in predicting sepsis using real-world medical data from PhysioNet/Computing in Cardiology Challenge 2019. Experimental results show that MB-TCN outperforms existing methods that are commonly used in current practice.
NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization

Neural architecture search (NAS) typically consists of three main steps: training a super-network, training and evaluating sampled deep neural networks (DNNs), and training the discovered DNN. Most of the existing efforts speed up some steps at the cost of a significant slowdown of other steps or sacrificing the support of non-differentiable search metrics. The unbalanced reduction in the time spent per step limits the total search time reduction, and the inability to support non-differentiable search metrics limits the performance of discovered DNNs.In this paper, we present NetAdaptV2 with three innovations to better balance the time spent for each step while supporting non-differentiable search metrics. First, we propose channel-level bypass connections that merge network depth and layer width into a single search dimension to reduce the time for training and evaluating sampled DNNs. Second, ordered dropout is proposed to train multiple DNNs in a single forward-backward pass to decrease the time for training a super-network. Third, we propose the multi-layer coordinate descent optimizer that considers the interplay of multiple layers in each iteration of optimization to improve the performance of discovered DNNs while supporting non-differentiable search metrics. With these innovations, NetAdaptV2 reduces the total search time by up to 5.8× on ImageNet and 2.4× on NYU Depth V2, respectively, and discovers DNNs with better accuracy-latency/accuracy-MAC trade-offs than state-of-the-art NAS works. Moreover, the discovered DNN outperforms NAS-discovered MobileNetV3 by 1.8% higher top-1 accuracy with the same latency. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>