Reviewerly/exHarmony huggingface dataset:
A New Memory Based on Sequence to Sequence Model for Video Captioning

In the field of computer vision, Video captioning is a very important and meaningful task, which could automatically generate textual descriptions of contents from videos. It is a challenging problem due to the difficulties of understanding the objects and activities in a video. Benefit from the rapid development of deep learning technology, e.g. sequence to sequence model, video captioning task has achieved very accurate results. However, there are two serious flaws, the first is that the pre-trained deep models are often used as visual feature abstractors as the training is highly time-consuming, so the feature generalization performance generated by these pre-trained Encoder is limited when we directly employ those networks in video captioning tasks. The second is that each frame in the video is processed separately, ignoring the correlation of video data in the time dimension. In this work, we propose video captioning model with attention-memory module to explore the role of capturing temporal correlations which with sequence to sequence model as the background and showing the importance of temporal structure to vision tasks by adding the correlation of videos when extracting features and enhancing the time-memory capability. Our experiments are based on two most famous benchmark datasets in the field of video captioning: MSVD and MSR-VTT. Then employ BLEU and METEOR to evaluate the accuracy of the description generated by different methods. Finally, the experimental results confirm that the proposed model could make significant improvements in description results compared with the baseline models.
A Recommended Method Based on the Weighted RippleNet Network Mode

Abstract User preferences were modeled by the RippleNet network and successfully applied in the recommender systems, but the weight of the entity was not considered. This paper proposes a RippleNet model incorporating the influence of complex network nodes. After the construction of complex networks based on knowledge Graphs, we build the maximum subnet model and calculate the influence of nodes in the graph network. We added it to the RippleNet as the weight of entities. The experimental results showed that new method increased the AUC and ACC values of RippleNet to 92.0% and 84.6%, solve the problem that entity influence was not considered in the RippleNet network, and made the recommended results more in line with users’ expectations.
Detecting Sexual Predatory Chats by Perturbed Data and Balanced Ensembles

Securing the safety of the children on online platforms is critical to avoid the mishaps of them being abused for sexual favors, which usually happens through predatory conversations. A number of approaches have been proposed to analyze the content of the messages to identify predatory conversations. However, due to the non-availability of large-scale predatory data, the state-of-the-art works employ a standard dataset that has less than 10% predatory conversations. Dealing with such heavy class imbalance is a challenge to devise reliable predatory detection approaches. We present a new approach for dealing with class imbalance using a hybrid sampling and class re-distribution to obtain an augmented dataset. To further improve the diversity of classifiers and features in the ensembles, we also propose to perturb the data along with augmentation in an iterative manner. Through a set of experiments, we demonstrate an improvement of 3% over the best state-of-the-art approach and results in an F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> -score of 0.99 and an F <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">β</inf> of 0.94 from the proposed approach.
Integrating Business Intelligence with Cloud Computing: State of the Art and Fundamental Concepts

The majority of the problems that organizations are currently facing is the lack of use of cloud computing as shared resources, and it is always looking to become smarter and more flexible by trying to use recent and powerful technologies, such as business intelligence solutions. The business intelligence solution is considered a quick investment and an easy-to-deploy solution. It has become very popular with organizations that process a huge amount of data. Thus, to make this solution more accessible, we thought to use cloud-computing technology to migrate the business intelligence system using frameworks and adapted models to process the data in a way that is efficient. The most important goal is to satisfy users, in terms of security and availability of information. There are many benefits to using cloud BI solution, especially in terms of cost reduction. This paper will address an important definition regarding cloud computing and business intelligence, the importance of each one, and the combination of both, evoking a cloud BI, we will present the management risks to take into account before proceeding to solutions, and the benefits and challenges of cloud computing will also discussed by comparing existing scenarios and their approach. The perspective of our future research will be based on this state of the art that remains an important opening for future contributions.
Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation

Depth estimation from monocular images is an important task in localization and 3D reconstruction pipelines for bronchoscopic navigation. Various supervised and self-supervised deep learning-based approaches have proven themselves on this task for natural images. However, the lack of labeled data and the bronchial tissue’s feature-scarce texture make the utilization of these methods ineffective on bronchoscopic scenes. In this work, we propose an alternative domain-adaptive approach. Our novel two-step structure first trains a depth estimation network with labeled synthetic images in a supervised manner; then adopts an unsupervised adversarial domain feature adaptation scheme to improve the performance on real images. The results of our experiments show that the proposed method improves the network’s performance on real images by a considerable margin and can be employed in 3D reconstruction pipelines.
Uncertainty-Aware Self-Improving Framework for Depth Estimation

In recent years, self-supervised paradigms for depth estimation have drawn lots of attention from the community. Promising as they are, in order to achieve wider application and better performance, reasoning about the uncertainty of the estimated depth maps is in great demand. In this paper, we propose a novel uncertainty-aware self-improving framework for self-supervised depth estimation, which could model and exploit uncertainty to boost depth accuracy without any external supervision. In detail, our framework includes two DepthNets–a teacher network and a student network. The teacher network is pre-trained in a self-supervised manner and the predicted depth is leveraged as pseudo-ground-truth to supervise the learning of the student network, leading to better estimation accuracy and the ability of modeling depth uncertainty. Afterward, the estimated uncertainty would be introduced back into the teacher network as the mask, preventing erroneous regions from contributing to network training and depth estimation. In this way, a better pseudo-ground-truth from the teacher network will be used for the next iteration of the student network training. We evaluate our proposed framework on the widely used KITTI dataset and TUM RGB-D dataset with both quantitative and qualitative experiments. The experimental results show that our framework achieves quite satisfactory performance among the self-supervised depth estimation methods. Extensive experiments which combine the student network and feature-based monocular SLAM system (i.e ORB-SLAM3) further demonstrate the benefits of the introduced depth uncertainty.
Object-based illumination transferring and rendering for applications of mixed reality

Abstract In applications of augmented reality or mixed reality, rendering virtual objects in real scenes with consistent illumination is crucial for realistic visualization experiences. Prior learning-based methods reported in the literature usually attempt to reconstruct complicated high dynamic range environment maps from limited input, and rely on a separate rendering pipeline to light up the virtual object. In this paper, an object-based illumination transferring and rendering algorithm is proposed to tackle this problem within a unified framework. Given a single low dynamic range image, instead of recovering lighting environment of the entire scene, the proposed algorithm directly infers the relit virtual object. It is achieved by transferring implicit illumination features which are extracted from its nearby planar surfaces. A generative adversarial network is adopted in the proposed algorithm for implicit illumination features extraction and transferring. Compared to previous works in the literature, the proposed algorithm is more robust, as it is able to efficiently recover spatially varying illumination in both indoor and outdoor scene environments. Experiments have been conducted. It is observed that notable experiment results and comparison outcomes have been obtained quantitatively and qualitatively by the proposed algorithm in different environments. It shows the effectiveness and robustness for realistic virtual object insertion and improved realism.
Towards Measuring Bias in Image Classification

Convolutional Neural Networks (CNN) have become de facto state-of-the-art for the main computer vision tasks. However, due to the complex underlying structure their decisions are hard to understand which limits their use in some context of the industrial world. A common and hard to detect challenge in machine learning (ML) tasks is data bias. In this work, we present a systematic approach to uncover data bias by means of attribution maps. For this purpose, first an artificial dataset with a known bias is created and used to train intentionally biased CNNs. The networks’ decisions are then inspected using attribution maps. Finally, meaningful metrics are used to measure the attribution maps’ representativeness with respect to the known bias. The proposed study shows that some attribution map techniques highlight the presence of bias in the data better than others and metrics can support the identification of bias.
Interpreting Attributions and Interactions of Adversarial Attacks

This paper aims to explain adversarial attacks in terms of how adversarial perturbations contribute to the attacking task. We estimate attributions of different image regions to the decrease of the attacking cost based on the Shapley value. We define and quantify interactions among adversarial perturbation pixels, and decompose the entire perturbation map into relatively independent perturbation components. The decomposition of the perturbation map shows that adversarially-trained DNNs have more perturbation components in the foreground than normally-trained DNNs. Moreover, compared to the normally-trained DNN, the adversarially-trained DNN have more components which mainly decrease the score of the true category. Above analyses provide new insights into the understanding of adversarial attacks.
Unsupervised Optimized Bipartite Graph Embedding

Graph embedding is a widely used method for dimensionality reduction due to its computational effectiveness. The quality of the graph and the efficiency of graph construction will directly affect the performance and the efficiency of the graph embedding methods. However, in the unsupervised graph embedding methods, the graph is not considered as an optimized graph since there is no label that can be used to construct this graph. In addition, the running of traditional graph embedding methods become very time-consuming on large-scale datasets due to the high computational cost in the step of graph construction. Aiming to solve these problems, we propose an unsupervised dimensionality reduction method based on bipartite graph, called Unsupervised Optimized Bipartite Graph Embedding (UOBGE). Representative anchors are firstly identified in the data. Then, we construct the bipartite graph between the projected samples and the projected anchors and the intrinsic graph connecting all the projected sample pairs with equal weights, which keep the local and global geometric structures of the data, respectively. Finally, the bipartite graph and the projection matrix are optimized simultaneously by introducing an alternating optimization procedure. Extensive experiments on several datasets demonstrate that the effectiveness and efficiency of the proposed method.
Tableaux for Free Logics with Descriptions

The paper provides a tableau approach to definite descriptions. We focus on several formalizations of the so-called minimal free description theory (MFD) usually formulated axiomatically in the setting of free logic. We consider five analytic tableau systems corresponding to different kinds of free logic, including the logic of definedness applied in computer science and constructive mathematics for dealing with partial functions (here called negative quasi-free logic). The tableau systems formalise MFD based on PFL (positive free logic), NFL (negative free logic), PQFL and NQFL (the quasi-free counterparts of the former ones). Also the logic \(\textsf {NQFL}^{-}\) is taken into account, which is equivalent to NQFL, but whose language does not comprise the existence predicate. It is shown that all tableaux are sound and complete with respect to the semantics of these logics.
Towards Efficient Inference: Adaptively Cooperate in Heterogeneous IoT Edge Cluster

New applications such as smart homes, autonomous vehicles are leading an increasing research topic of convolutional neural network (CNN) based inference on IoT edge devices. Unfortunately, this scenario meets a huge roadblock caused by the limited computing resources owned by these devices. One popular solution is to execute inference on an edge cluster with parallelization schemes instead of on a single device. However, the heterogeneous edge devices and varied neural layers bring challenges to this process. In this paper, we propose a pipelined cooperation scheme (PICO) to efficiently execute CNN inference for edge devices. Our goal is to maximize throughput by reducing redundant computing meanwhile to keep the inference latency under a certain value. PICO divides the neural layers and edge devices into several stages. The input data is fed into the first stage and the inference result is produced at the last stage. These stages compose an inference pipeline. The execution time of each stage is optimized to approach the maximum throughput as close as possible. We also implement an adaptive framework to choose the best inference scheme under different workloads. In our experiment with 8 RaspberryPi devices, the average inference latency can be reduced by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.7\sim 6.5\times$</tex> under different workloads, and the throughput can be improved by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1.8\sim 6.2\times$</tex> under various network settings.
OSVidCap: A Framework for the Simultaneous Recognition and Description of Concurrent Actions in Videos in an Open-Set Scenario

Automatically understanding and describing the visual content of videos in natural language is a challenging task in computer vision.Existing approaches are often designed to describe single events in a closed-set setting.However, in real-world scenarios, concurrent activities and previously unseen actions may appear in a video.This work presents the OSVidCap, a novel open-set video captioning framework that recognizes and describes, in natural language, concurrent known actions and deal with unknown ones.The OSVidCap is based on the encoder-decoder framework and uses a detection-andtracking-object-based mechanism followed by a background blurring method to focus on specific targets in a video.Additionally, we employ the TI3D Network with the Extreme Value Machine (EVM), which learns representations and recognizes unknown actions.We evaluate the proposed approach on the benchmark ActivityNet Captions dataset.Also, an enhanced version of the LIRIS human activity dataset was proposed by providing descriptions for each action.We also provide spatial, temporal, and caption annotations for existing unlabeled actions in the dataset -considered unknown actions in our experiments.Experimental results showed our method's effectiveness in recognizing and describing concurrent actions in natural language and the strong ability to deal with detected unknown activities.Based on these results, we believe that the proposed approach can be potentially helpful for many real-world applications, including human behavior analysis, safety monitoring, and surveillance.
A Feature Fusion Based Indicator for Training-Free Neural Architecture Search

Neural Architecture Search Without Training (NASWOT) has been proposed recently to replace the conventional Neural Architecture Search (NAS). Pioneer works only deploy one or two indicator(s) to search. Nevertheless, the quantitative assessment for indicators is not fully studied and evaluated. In this paper, we first review several indicators, which are used to evaluate the network in a training-free manner, including the correlation of Jacobian, the output sensitivity, the number of linear regions, and the condition number of the neural tangent kernel. Our observation is that each indicator is responsible for characterizing a network in a specific aspect and there is no single indicator that achieves good performance in all cases, e.g. highly correlated with the test accuracy. This motivated us to develop a novel indicator where all properties of a network are taken into account. To obtain better indicator that can consider various characteristics of networks in a harmonized form, we propose a Fusion Indicator (FI). Specifically, the proposed FI is formed by combining multiple indicators in a weighted sum manner. We minimize the mean squared error loss between the predicted and actual accuracy of networks to acquire the weights. Moreover, as the conventional training-free NAS researches used limited metrics to evaluate the quality of indicators, we introduce more desirable metrics that can evaluate the quality of training-free NAS indicator in terms of fidelity, correlation and rank-order similarity between the predicted quality value and actual accuracy of networks. That is, we introduce the Pearson Linear Coefficient Correlation (PLCC), the Root Mean Square Error (RMSE), the Spearman Rank-Order Correlation Coefficient (SROCC), and Kendall Rank-Order Correlation Coefficient (KROCC). Extensive experiments on NAS-Bench-101 and NAS-Bench-201 demonstrate the effectiveness of our FI, outperforming existing methods by a large margin.
RMC: Reordering Marking and Coding for Fine-Grained Load Balancing in Data Centers

Data center networks typically adopt multi-rooted tree topologies to provide high bisection bandwidth. Various fine-grained load balancing schemes have been proposed to split flows across multiple paths. However, data center networks suffer from many uncertainties such as highly dynamic traffic. These uncertainties easily make network become asymmetric, resulting in significant packet reordering. Unfortunately, existing solutions passively deal with packet reordering based on a threshold and hardly adapt to asymmetric networks because of lacking the explicit reordering feedback. These solutions either fail to quickly respond to packet loss or cause unnecessary fast retransmission, which reduces link utilization and increases flow completion time. In this paper, we propose a fine-grained load balancing scheme RMC to eliminate the impact of packet reordering and handle uncertainties in asymmetric networks. To avoid unnecessary fast retransmission, the switch proactively identifies reordered packet according to local queue length and global path latency. Furthermore, we employ a coding technique with redundancy optimization to reduce long-tailed flow completion time under network asymmetry. Through a series of large-scale NS2 simulations and testbed experiments, we demonstrate that RMC effectively avoids unnecessary fast retransmission under different network scenarios and reduces flow completion time by up to 72% compared with state-of-the-art schemes.
EC-DARTS: Inducing Equalized and Consistent Optimization into DARTS

Based on the relaxed search space, differential architecture search (DARTS) is efficient in searching for a high-performance architecture. However, the unbalanced competition among operations that have different trainable parameters causes the model collapse. Besides, the inconsistent structures in the search and retraining stages causes cross-stage evaluation to be unstable. In this paper, we call these issues as an operation gap and a structure gap in DARTS. To shrink these gaps, we propose to induce equalized and consistent optimization in differentiable architecture search (EC-DARTS). EC-DARTS decouples different operations based on their categories to optimize the operation weights so that the operation gap between them is shrinked. Besides, we introduce an induced structural transition to bridge the structure gap between the model structures in the search and retraining stages. Extensive experiments on CIFAR10 and ImageNet demonstrate the effectiveness of our method. Specifically, on CIFAR10, we achieve a test error of 2.39%, while only 0.3 GPU days on NVIDIA TITAN V. On ImageNet, our method achieves a top-1 error of 23.6% under the mobile setting.
Cooperative Task Allocation of Multiple VA Vs Based on Greedy Algorithm

Multiple VAVs can perform certain large-scale tasks by constructing VA V clusters in practical applications since its excellent performance such as strong operability and composability. However, it is difficult to determine the specific execution of each drone before performing the cluster task in a cluster composed of multiple drones. This paper studies the task assignment problem under the constraint of multi-VA V cooperative flight, and proposes a task planning algorithm based on the shortest flight path or time optimal under the condition of ensuring the cooperative flight between VAVs. The algorithm combines the directed graph in graph theory to represent the cooperative flight constraint relationship between tasks, and adopts the breadth-first search strategy to realize the traversal assignment of tasks. During the assignment process, according to the greedy algorithm, the tasks are locally optimally allocated to obtain the approximate optimal solution. Finally, a simulation experiment is performed on the constructed task scene to verify the effectiveness of the algorithm.