Reviewerly/exHarmony huggingface dataset:
Modelling Ambiguous Assignments for Multi-Person Tracking in Crowds

Multi-person tracking is often solved with a tracking-by-detection approach that matches all tracks and detections simultaneously based on a distance matrix. In crowded scenes, ambiguous situations with similar track-detection distances occur, which leads to wrong assignments. To mitigate this problem, we propose a new association method that separately treats such difficult situations by modelling ambiguous assignments based on the differences in the distance matrix. Depending on the numbers of tracks and detections, for which the assignment task is determined ambiguous, different strategies to resolve these ambiguous situations are proposed. To further enhance the performance of our tracking framework, we introduce a camera motion-aware interpolation technique and make an adaptation to the motion model, which improves identity preservation. The effectiveness of our approach is demonstrated through extensive ablative experiments with different detection models. Moreover, the superiority w.r.t. other trackers is shown on the challenging MOT17 and MOT20 datasets, where state-of-the-art results are obtained.
Location-aware Single Image Reflection Removal

This paper proposes a novel location-aware deep-learning-based single image reflection removal method. Our network has a reflection detection module to regress a probabilistic reflection confidence map, taking multi-scale Laplacian features as inputs. This probabilistic map tells if a region is reflection-dominated or transmission-dominated, and it is used as a cue for the network to control the feature flow when predicting the reflection and transmission layers. We design our network as a recurrent network to progressively refine reflection removal results at each iteration. The novelty is that we leverage Laplacian kernel parameters to emphasize the boundaries of strong reflections. It is beneficial to strong reflection detection and substantially improves the quality of reflection removal results. Extensive experiments verify the superior performance of the proposed method over state-of-the-art approaches. Our code and the pre-trained model can be found at https://github.com/zdlarr/Location-aware-SIRR.
Toward successful DevSecOps in software development organizations: A decision-making framework

Development and Operations (DevOps) is a methodology that aims to establish collaboration between programmers and operators to automate the continuous delivery of new software to reduce the development life cycle and produce quality software. Development, Security, and Operations (DevSecOps) is developing the DevOps concept, which integrates security methods into a DevOps process. DevSecOps is a software development process where security is built in to ensure application confidentiality, integrity, and availability. This paper aims to identify and prioritize the challenges associated with implementing the DevSecOps process. We performed a multivocal literature review (MLR) and conducted a questionnaire-based survey to identify challenges associated with DevSecOps-based projects. Moreover, interpretive structure modeling (ISM) was applied to study the relationships among the core categories of the challenges. Finally, we used the fuzzy technique for order preference by similarity to an ideal solution (TOPSIS) to prioritize the identified challenges associated with DevSecOps projects. We identified 18 challenges for the DevSecOps process and mapped them to 10 core categories. The ISM results indicate that the “standards” category has the most decisive influence on the other nine core categories of the identified challenges. Moreover, the fuzzy TOPSIS indicates that “lack of secure coding standards,” “lack of automated testing tools for security in DevOps,” and “ignorance in static testing for security due to lack of knowledge” are the highest priority challenges for the DevSecOps paradigm. Organizations using DevOps should consider the identified challenges in developing secure software.
KoDF: A Large-scale Korean DeepFake Detection Dataset

A variety of effective face-swap and face-reenactment methods have been publicized in recent years, democratizing the face synthesis technology to a great extent. Videos generated as such have come to be called deepfakes with a negative connotation, for various social problems they have caused. Facing the emerging threat of deepfakes, we have built the Korean DeepFake Detection Dataset (KoDF), a large-scale collection of synthesized and real videos focused on Korean subjects. In this paper, we provide a detailed description of methods used to construct the dataset, experimentally show the discrepancy between the distributions of KoDF and existing deepfake detection datasets, and underline the importance of using multiple datasets for real-world generalization. KoDF is publicly available at https://moneybrain-research.github.io/kodf in its entirety (i.e. real clips, synthesized clips, clips with adversarial attack, and metadata).
Learning to Track Objects from Unlabeled Videos

In this paper, we propose to learn an Unsupervised Single Object Tracker (USOT) from scratch. We identify that three major challenges, i.e., moving object discovery, rich temporal variation exploitation, and online update, are the central causes of the performance bottleneck of existing unsupervised trackers. To narrow the gap between unsupervised trackers and supervised counterparts, we propose an effective unsupervised learning approach composed of three stages. First, we sample sequentially moving objects with unsupervised optical flow and dynamic programming, instead of random cropping. Second, we train a naive Siamese tracker from scratch using single-frame pairs. Third, we continue training the tracker with a novel cycle memory learning scheme, which is conducted in longer temporal spans and also enables our tracker to update online. Extensive experiments show that the proposed USOT learned from unlabeled videos performs well over the state-of-the-art unsupervised trackers by large margins, and on par with recent supervised deep trackers. Code is available at https://github.com/VISION-SJTU/USOT.
Five Open Problems in Quantum Information Theory

We identify five selected open problems in the theory of quantum information, which are rather simple to formulate, are well studied in the literature, but are technically not easy. As these problems enjoy diverse mathematical connections, they offer a huge breakthrough potential. The first four concern existence of certain objects relevant for quantum information, namely a family of symmetric informationally complete generalized measurements in an infinite sequence of dimensions, mutually unbiased bases in dimension six, measurements saturating multiparameter Cramér-Rao bound and bound entangled states with negative partial transpose. The fifth problem requires checking whether a certain state of a two-ququart system is two-copy distillable.Received 21 December 2020Revised 1 December 2021DOI:https://doi.org/10.1103/PRXQuantum.3.010101Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasQuantum correlations in quantum informationQuantum measurementsQuantum InformationGeneral Physics
Evolving Fusion-Based Visibility Restoration Model for Hazy Remote Sensing Images Using Dynamic Differential Evolution

Remote sensing images taken during poor environmental conditions are degraded by the scattering of atmospheric particles, which affects the performance of many imaging systems. Hence, an efficient visibility restoration model is required to remove haze from distorted images. However, the design of visibility restoration models is an ill-posed problem as the physical information, such as depth information and attenuation model, is usually unknown. The physical parameters computed using existing models, such as dark channel prior and gradient channel prior, are not accurate, especially for images with large haze gradients. Therefore, in this article, an evolving visibility restoration model is proposed for remote sensing images. Initially, the fusion-based transmission map is computed from the foreground and sky regions. The transmission map is further improved by designing a hybrid constraint-based variational model. Finally, a dynamic differential evolution is utilized to optimize the control parameters of the proposed model. The proposed model is validated on 50 synthetic benchmarks and 50 real-life remote sensing images. For comparative analysis, ten well-known restoration models are also considered. The comparative analysis demonstrates that the proposed model outperforms the existing restoration models.
Few-Shot Semantic Segmentation with Cyclic Memory Network

Few-shot semantic segmentation (FSS) is an important task for novel (unseen) object segmentation under the data-scarcity scenario. However, most FSS methods rely on unidirectional feature aggregation, e.g., from support prototypes to get the query prediction, and from high-resolution features to guide the low-resolution ones. This usually fails to fully capture the cross-resolution feature relationships and thus leads to inaccurate estimates of the query objects. To resolve the above dilemma, we propose a cyclic memory network (CMN) to directly learn to read abundant support information from all resolution features in a cyclic manner. Specifically, we first generate N pairs (key and value) of multi-resolution query features guided by the support feature and its mask. Next, we circularly take one pair of these features as the query to be segmented, and the rest N-1 pairs are written into an external memory accordingly, i.e., this leave-one-out process is conducted for N times. In each cycle, the query feature is updated by collaboratively matching its key and value with the memory, which can elegantly cover all the spatial locations from different resolutions. Furthermore, we incorporate the query feature re-adding and the query feature recursive updating mechanisms into the memory reading operation. CMN, equipped with these merits, can thus capture cross-resolution relationships and better handle the object appearance and scale variations in FSS. Experiments on PASCAL-5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> and COCO-20 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> well validate the effectiveness of our model for FSS.
Multi-class sentiment analysis of urdu text using multilingual BERT

Sentiment analysis (SA) is an important task because of its vital role in analyzing people's opinions. However, existing research is solely based on the English language with limited work on low-resource languages. This study introduced a new multi-class Urdu dataset based on user reviews for sentiment analysis. This dataset is gathered from various domains such as food and beverages, movies and plays, software and apps, politics, and sports. Our proposed dataset contains 9312 reviews manually annotated by human experts into three classes: positive, negative and neutral. The main goal of this research study is to create a manually annotated dataset for Urdu sentiment analysis and to set baseline results using rule-based, machine learning (SVM, NB, Adabbost, MLP, LR and RF) and deep learning (CNN-1D, LSTM, Bi-LSTM, GRU and Bi-GRU) techniques. Additionally, we fine-tuned Multilingual BERT(mBERT) for Urdu sentiment analysis. We used four text representations: word n-grams, char n-grams,pre-trained fastText and BERT word embeddings to train our classifiers. We trained these models on two different datasets for evaluation purposes. Finding shows that the proposed mBERT model with BERT pre-trained word embeddings outperformed deep learning, machine learning and rule-based classifiers and achieved an F1 score of 81.49%.
An overview on twin support vector regression

Twin support vector regression (TSVR) is a useful extension of traditional support vector regression (SVR). As a new regression model, the basic idea of TSVR is generating a pair of nonparallel functions on both sides of the training data points, such that the ε-insensitive upper and lower bounds of the regression function can be determined. Owing to its excellent learning ability, TSVR has become a research hotspot in the field of machine learning. With the deepening of such research, scholars have found that TSVR also has certain limitations, and thus various improved models have been proposed. This review aims to report the recent developments in twin support vector regression. First, the basic concepts and basic models of TSVR are introduced. Second, the improved algorithms and applications of TSVR in recent years are summarized, and the advantages and disadvantages of its representative algorithms are analyzed and compared with the experiments. Finally, we discuss the research conducted on TSVR.
The shadow banning controversy: perceived governance and algorithmic folklore

In this paper, I approach platform governance through algorithmic folklore, consisting of beliefs and narratives about moderation systems that are passed on informally and can exist in tension with official accounts. More specifically, I analyse user discussions on ‘shadow banning’, a controversial, potentially non-existing form of content moderation on popular social media platforms. I argue that discursive mobilisations of the term can act as a methodological entry point to studying the shifting grounds and emerging logics of algorithmic governance, not necessarily in terms of the actual practices themselves, but in terms of its experiential dimension that, in turn, indicates broader modalities and relationalities of control. Based on my analysis of the user discussions, I argue that the constitutive logics of social media platforms increasingly seem to run counter to the values of good governance, such as clarity and stability of norms, and consistency of enforcement. This is reflected in how users struggle, desperately, to form expectations about system operation and police themselves according to perceived rules, yet are left in a state of dependency and frustration, unable to take hold of their digital futures.
Deep learning-based perception systems for autonomous driving: A comprehensive survey

With the rapid development of society and the economy, autonomous driving techniques are widely applied in many areas, such as autonomous vehicles, autonomous drones, and robotics. As a dominating technique, deep learning has become more and more popular for 2-D and 3-D object detection. Numerous deep learning-based methods have been proposed to solve various vision issues. To further help with the development of unmanned systems, this paper presents a comprehensive survey of the recent processes from the past five years for 3-D object detection, road detection, traffic sign detection, and traffic light detection and classification. To summarize and analyze previous works in detail, this paper only focuses on deep learning-based object detection tasks in autonomous driving that take place when the input is a point cloud or image(s). It also presents comparative results for insight comparison and inspiring future researches.
Accelerating DETR Convergence via Semantic-Aligned Matching

The recently developed DEtection TRansformer (DETR) establishes a new object detection paradigm by eliminating a series of hand-crafted components. However, DETR suffers from extremely slow convergence, which increases the training cost significantly. We observe that the slow convergence is largely attributed to the complication in matching object queries with target features in different feature embedding spaces. This paper presents SAM-DETR, a Semantic-Aligned-Matching DETR that greatly accelerates DETR's convergence without sacrificing its accuracy. SAM-DETR addresses the convergence issue from two perspectives. First, it projects object queries into the same embedding space as encoded image features, where the matching can be accomplished efficiently with aligned semantics. Second, it explicitly searches salient points with the most discriminative features for semantic-aligned matching, which further speeds up the convergence and boosts detection accuracy as well. Being like a plug and play, SAM-DETR complements existing convergence solutions well yet only introduces slight computational overhead. Extensive experiments show that the proposed SAM-DETR achieves superior convergence as well as competitive detection accuracy. The implementation codes are publicly available at https://github.com/ZhangGongjie/SAM-DETR.
Automatic Labeling to Generate Training Data for Online LiDAR-Based Moving Object Segmentation

Understanding the scene is key for autonomously navigating vehicles and the ability to segment the surroundings online into moving and non-moving objects is a central ingredient for this task. Often, deep learning-based methods are used to perform moving object segmentation (MOS). The performance of these networks, however, strongly depends on the diversity and amount of labeled training data, information that may be costly to obtain. In this paper, we propose an automatic data labeling pipeline for 3D LiDAR data to save the extensive manual labeling effort and to improve the performance of existing learning-based MOS systems by automatically generating labeled training data. Our proposed approach achieves this by processing the data offline in batches. It first exploits an occupancy-based dynamic object removal to detect possible dynamic objects coarsely. Second, it extracts segments among the proposals and tracks them using a Kalman filter. Based on the tracked trajectories, it labels the actually moving objects such as driving cars and pedestrians as moving. In contrast, the non-moving objects, e.g., parked cars, lamps, roads, or buildings, are labeled as static. We show that this approach allows us to label LiDAR data highly effectively and compare our results to those of other label generation methods. We also train a deep neural network with our auto-generated labels and achieve similar performance compared to the one trained with manual labels on the same data, and an even better performance when using additional datasets with labels generated by our approach. Furthermore, we evaluate our method on multiple datasets using different sensors and our experiments indicate that our method can generate labels in diverse environments.
Fast and Memory-Efficient Network Towards Efficient Image Super-Resolution

Runtime and memory consumption are two important aspects for efficient image super-resolution (EISR) models to be deployed on resource-constrained devices. Recent advances in EISR [16], [32] exploit distillation and aggregation strategies with plenty of channel split and concatenation operations to fully use limited hierarchical features. In contrast, sequential network operations avoid frequently accessing preceding states and extra nodes, and thus are beneficial to reducing the memory consumption and runtime overhead. Following this idea, we design our lightweight network backbone by mainly stacking multiple highly optimized convolution and activation layers and decreasing the usage of feature fusion. We propose a novel sequential attention branch, where every pixel is assigned an important factor according to local and global contexts, to enhance high-frequency details. In addition, we tailor the residual block for EISR and propose an enhanced residual block (ERB) to further accelerate the network inference. Finally, combining all the above techniques, we construct a fast and memory-efficient network (FMEN) and its small version FMEN-S, which runs 33% faster and reduces 74% memory consumption compared with the state-of-the-art EISR model: E-RFDN, the champion in [49]. Besides, FMEN-S achieves the lowest memory consumption and the second shortest runtime in NTIRE 2022 challenge on efficient super-resolution [28]. Code is available at https://github.com/NJU-Jet/FMEN.
Photovoltaic power forecasting: A hybrid deep learning model incorporating transfer learning strategy

Accurate forecasting of photovoltaic power is essential in the integration, operation, and scheduling of hybrid grid systems. In particular, modeling for newly built photovoltaic sites is restricted by insufficient data and training burden. In this study, a novel hybrid photovoltaic power forecasting model assisted with a transfer learning strategy is proposed. The hybrid model, named the attention-dilate convolution neural network-bidirectional long short-term memory network, consists of three steps. Step 1 - Input reconstruction: the historical power and meteorological factors are reconstructed as new inputs based on their relevance to the forecast by introducing a long short-term memory-based attention mechanism; Step 2 - Feature extraction: a hybrid structure is applied to extract spatial and temporal features from new inputs in parallel; Step 3 - Feature mapping: the extracted features are mapped into the forecasted photovoltaic output. Furthermore, to address the modeling for new sites, a transfer learning strategy that fine-tunes the pre-trained model is proposed in this work. The structure by step-wise division allows fine-tuning to be applied to the necessary parts rather than the entire model. Subsequently, the data from the actual photovoltaic system was acquired to validate the proposed model and transfer learning strategy. The proposed model showed significantly superior performance than the other models in the tests, and the parameter transferring not only makes up for the data shortage but also effectively accelerates the model training. With the transfer learning strategy, the maximum improvement in accuracy and training efficiency reached 69.51% and 71.42%, respectively.
Fair-AdaBoost: Extending AdaBoost method to achieve fair classification

Machine learning-based classification models are ubiquitous in a wide variety of classification tasks. These models usually rely on complex machine learning methods and vast historical data to make accurate predictions. Recently, a concern grows that these models lead to the unfair classifications, i.e., model predictions would hurt or benefit particular group of people based on sensitive features (e.g., race, age). To achieve fair classification, a popular machine learning method, AdaBoost, is extended into the Fair-AdaBoost method in this study. Fair-AdaBoost can achieve fair classification while preserving the advantages (i.e., interpretability, scalability, and accuracy) of the basic AdaBoost. To further enhance the performance of Fair-AdaBoost, the non-dominated sorting genetic algorithm-II is extended in this study to optimize the hyper-parameters of the base classifiers in Fair-AdaBoost. In the experiments, the proposed method and algorithm have been tested on three standard benchmark datasets widely used in the literature of fair classification, which has proven their superior performance.