OpenRaiser/Intern-Atlas huggingface dataset:
Robustness of Greedy Approval Rules
Venue: arXiv cs.GT 2022
Year: 2022

We study the robustness of GreedyCC, GreedyPAV, and Phargmen's sequential rule, using the framework introduced by Bredereck et al. for the case of (multiwinner) ordinal elections and adopted to the approval setting by Gawron and Faliszewski. First, we show that for each of our rules and every committee size $k$, there are elections in which adding or removing a certain approval causes the winning committee to completely change (i.e., the winning committee after the operation is disjoint from the one before the operation). Second, we show that the problem of deciding how many approvals need to be added (or removed) from an election to change its outcome is NP-complete for each of our rules. Finally, we experimentally evaluate the robustness of our rules in the presence of random noise.
Debiasing Deep Chest X-Ray Classifiers using Intra- and Post-processing Methods
Venue: arXiv cs.CV 2022
Year: 2022

Deep neural networks for image-based screening and computer-aided diagnosis have achieved expert-level performance on various medical imaging modalities, including chest radiographs. Recently, several works have indicated that these state-of-the-art classifiers can be biased with respect to sensitive patient attributes, such as race or gender, leading to growing concerns about demographic disparities and discrimination resulting from algorithmic and model-based decision-making in healthcare. Fair machine learning has focused on mitigating such biases against disadvantaged or marginalised groups, mainly concentrating on tabular data or natural images. This work presents two novel intra-processing techniques based on fine-tuning and pruning an already-trained neural network. These methods are simple yet effective and can be readily applied post hoc in a setting where the protected attribute is unknown during the model development and test time. In addition, we compare several intra- and post-processing approaches applied to debiasing deep chest X-ray classifiers. To the best of our knowledge, this is one of the first efforts studying debiasing methods on chest radiographs. Our results suggest that the considered approaches successfully mitigate biases in fully connected and convolutional neural networks offering stable performance under various settings. The discussed methods can help achieve group fairness of deep medical image classifiers when deploying them in domains with different fairness considerations and constraints.
A Hybrid CNN-LSTM model for Video Deepfake Detection by Leveraging Optical Flow Features
Venue: arXiv cs.CV 2022
Year: 2022

Deepfakes are the synthesized digital media in order to create ultra-realistic fake videos to trick the spectator. Deep generative algorithms, such as, Generative Adversarial Networks(GAN) are widely used to accomplish such tasks. This approach synthesizes pseudo-realistic contents that are very difficult to distinguish by traditional detection methods. In most cases, Convolutional Neural Network(CNN) based discriminators are being used for detecting such synthesized media. However, it emphasise primarily on the spatial attributes of individual video frames, thereby fail to learn the temporal information from their inter-frame relations. In this paper, we leveraged an optical flow based feature extraction approach to extract the temporal features, which are then fed to a hybrid model for classification. This hybrid model is based on the combination of CNN and recurrent neural network (RNN) architectures. The hybrid model provides effective performance on open source data-sets such as, DFDC, FF++ and Celeb-DF. This proposed method shows an accuracy of 66.26%, 91.21% and 79.49% in DFDC, FF++, and Celeb-DF respectively with a very reduced No of sample size of approx 100 samples(frames). This promises early detection of fake contents compared to existing modalities.
Retrieval of surgical phase transitions using reinforcement learning
Venue: arXiv cs.CV 2022
Year: 2022

In minimally invasive surgery, surgical workflow segmentation from video analysis is a well studied topic. The conventional approach defines it as a multi-class classification problem, where individual video frames are attributed a surgical phase label. We introduce a novel reinforcement learning formulation for offline phase transition retrieval. Instead of attempting to classify every video frame, we identify the timestamp of each phase transition. By construction, our model does not produce spurious and noisy phase transitions, but contiguous phase blocks. We investigate two different configurations of this model. The first does not require processing all frames in a video (only <60% and <20% of frames in 2 different applications), while producing results slightly under the state-of-the-art accuracy. The second configuration processes all video frames, and outperforms the state-of-the art at a comparable computational cost. We compare our method against the recent top-performing frame-based approaches TeCNO and Trans-SVNet on the public dataset Cholec80 and also on an in-house dataset of laparoscopic sacrocolpopexy. We perform both a frame-based (accuracy, precision, recall and F1-score) and an event-based (event ratio) evaluation of our algorithms.
Effective Gesture Based Framework for Capturing User Input
Venue: arXiv cs.HC 2022
Year: 2022

Computers today aren't just confined to laptops and desktops. Mobile gadgets like mobile phones and laptops also make use of it. However, one input device that hasn't changed in the last 50 years is the QWERTY keyboard. Users of virtual keyboards can type on any surface as if it were a keyboard thanks to sensor technology and artificial intelligence. In this research, we use the idea of image processing to create an application for seeing a computer keyboard using a novel framework which can detect hand gestures with precise accuracy while also being sustainable and financially viable. A camera is used to capture keyboard images and finger movements which subsequently acts as a virtual keyboard. In addition, a visible virtual mouse that accepts finger coordinates as input is also described in this study. This system has a direct benefit of reducing peripheral cost, reducing electronics waste generated due to external devices and providing accessibility to people who cannot use the traditional keyboard and mouse.
Large-Scale Product Retrieval with Weakly Supervised Representation Learning
Venue: arXiv cs.CV 2022
Year: 2022

Large-scale weakly supervised product retrieval is a practically useful yet computationally challenging problem. This paper introduces a novel solution for the eBay Visual Search Challenge (eProduct) held at the Ninth Workshop on Fine-Grained Visual Categorisation workshop (FGVC9) of CVPR 2022. This competition presents two challenges: (a) E-commerce is a drastically fine-grained domain including many products with subtle visual differences; (b) A lacking of target instance-level labels for model training, with only coarse category labels and product titles available. To overcome these obstacles, we formulate a strong solution by a set of dedicated designs: (a) Instead of using text training data directly, we mine thousands of pseudo-attributes from product titles and use them as the ground truths for multi-label classification. (b) We incorporate several strong backbones with advanced training recipes for more discriminative representation learning. (c) We further introduce a number of post-processing techniques including whitening, re-ranking and model ensemble for retrieval enhancement. By achieving 71.53% MAR, our solution "Involution King" achieves the second position on the leaderboard.
CBAG: An Efficient Genetic Algorithm for the Graph Burning Problem
Venue: arXiv cs.NE 2022
Year: 2022

Information spread is an intriguing topic to study in network science, which investigates how information, influence, or contagion propagate through networks. Graph burning is a simplified deterministic model for how information spreads within networks. The complicated NP-complete nature of the problem makes it computationally difficult to solve using exact algorithms. Accordingly, a number of heuristics and approximation algorithms have been proposed in the literature for the graph burning problem. In this paper, we propose an efficient genetic algorithm called Centrality BAsed Genetic-algorithm (CBAG) for solving the graph burning problem. Considering the unique characteristics of the graph burning problem, we introduce novel genetic operators, chromosome representation, and evaluation method. In the proposed algorithm, the well-known betweenness centrality is used as the backbone of our chromosome initialization procedure. The proposed algorithm is implemented and compared with previous heuristics and approximation algorithms on 15 benchmark graphs of different sizes. Based on the results, it can be seen that the proposed algorithm achieves better performance in comparison to the previous state-of-the-art heuristics. The complete source code is available online and can be used to find optimal or near-optimal solutions for the graph burning problem.
Learning Multi-Object Symbols for Manipulation with Attentive Deep Effect Predictors
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we propose a concept learning architecture that enables a robot to build symbols through self-exploration by interacting with a varying number of objects. Our aim is to allow a robot to learn concepts without constraints, such as a fixed number of interacted objects or pre-defined symbolic structures. As such, the sought architecture should be able to build symbols for objects such as single objects that can be grasped, object stacks that cannot be grasped together, or other composite dynamic structures. Towards this end, we propose a novel architecture, a self-attentive predictive encoder-decoder network with binary activation layers. We show the validity of the proposed network through a robotic manipulation setup involving a varying number of rigid objects. The continuous sensorimotor experience of the robot is used by the proposed network to form effect predictors and symbolic structures that describe the interaction of the robot in a discrete way. We showed that the robot acquired reasoning capabilities to encode interaction dynamics of a varying number of objects in different configurations using the discovered symbols. For example, the robot could reason that (possible multiple numbers of) objects on top of another object would move together if the object below is moved by the robot. We also showed that the discovered symbols can be used for planning to reach goals by training a higher-level neural network that makes pure symbolic reasoning.
CircuitNet: An Open-Source Dataset for Machine Learning Applications in Electronic Design Automation (EDA)
Venue: arXiv cs.LG 2022
Year: 2022

The electronic design automation (EDA) community has been actively exploring machine learning (ML) for very large-scale integrated computer-aided design (VLSI CAD). Many studies explored learning-based techniques for cross-stage prediction tasks in the design flow to achieve faster design convergence. Although building ML models usually requires a large amount of data, most studies can only generate small internal datasets for validation because of the lack of large public datasets. In this essay, we present the first open-source dataset called CircuitNet for ML tasks in VLSI CAD.
Rethinking Quality of Experience for Metaverse Services: A Consumer-based Economics Perspective
Venue: arXiv cs.NI 2022
Year: 2022

The Metaverse is considered to be one prototype of the next-generation Internet, which contains people's expectations for the future world. However, the academic discussion of the Metaverse still mainly focused on the technical system design, and few research studied Metaverse challenges from the perspective of consumers, i.e., Metaverse users. One difficulty is that the analysis from the consumer's perspective requires interdisciplinary theoretical framework and quantifiable Quality of Experience (QoE) measurements. In this article, pioneering from consumers' point of view, we explore an interaction between Metaverse system design and consumer behaviors. Specifically, we rethink QoE and propose an framework that encompasses both the Metaverse service providers (MSPs) and consumer considerations. From the macro perspective, we introduce a joint optimization scheme that simultaneously considers the Metaverse system design, consumers' utility, and profitability of the MSPs. From the micro perspective, we advocate the Willingness-to-Pay (WTP) as an easy-to-implement measurement for future Metaverse system studies. To illustrate the usability of the proposed integrated framework, a use case, i.e., virtual traveling, is presented. We show that our framework can benefit the MSPs in offering competitive and economical service design to consumers while maximizing the profit.
On Matrix Multiplication and Polynomial Identity Testing
Venue: arXiv cs.CC 2022
Year: 2022

We show that lower bounds on the border rank of matrix multiplication can be used to non-trivially derandomize polynomial identity testing for small algebraic circuits. Letting $\underline{R}(n)$ denote the border rank of $n \times n \times n$ matrix multiplication, we construct a hitting set generator with seed length $O(\sqrt{n} \cdot \underline{R}^{-1}(s))$ that hits $n$-variate circuits of multiplicative complexity $s$. If the matrix multiplication exponent $ω$ is not 2, our generator has seed length $O(n^{1 - \varepsilon})$ and hits circuits of size $O(n^{1 + δ})$ for sufficiently small $\varepsilon, δ> 0$. Surprisingly, the fact that $\underline{R}(n) \ge n^2$ already yields new, non-trivial hitting set generators for circuits of sublinear multiplicative complexity.
Trust Challenges in Reusing Open Source Software: An Interview-based Initial Study
Venue: arXiv cs.SE 2022
Year: 2022

Open source projects play a significant role in software production. Most of the software projects reuse and build upon the existing open source projects and libraries. While reusing is a time and cost-saving strategy, some of the key factors are often neglected that create vulnerability in the software system. We look beyond the static code analysis and dependency chain tracing to prevent vulnerabilities at the human factors level. The literature lacks a comprehensive study of the human factors perspective on the issue of trust in reusing open source projects. We performed an interview-based initial study with software developers to get an understanding of the trust issue and limitations among the practitioners. We outline some of the key trust issues in this paper and lay out the first steps toward the trustworthy reuse of software.
Patents Phrase to Phrase Semantic Matching Dataset
Venue: arXiv cs.CL 2022
Year: 2022

There are many general purpose benchmark datasets for Semantic Textual Similarity but none of them are focused on technical concepts found in patents and scientific publications. This work aims to fill this gap by presenting a new human rated contextual phrase to phrase matching dataset. The entire dataset contains close to $50,000$ rated phrase pairs, each with a CPC (Cooperative Patent Classification) class as a context. This paper describes the dataset and some baseline models.
Late Fusion Multi-view Clustering via Global and Local Alignment Maximization
Venue: arXiv cs.LG 2022
Year: 2022

Multi-view clustering (MVC) optimally integrates complementary information from different views to improve clustering performance. Although demonstrating promising performance in various applications, most of existing approaches directly fuse multiple pre-specified similarities to learn an optimal similarity matrix for clustering, which could cause over-complicated optimization and intensive computational cost. In this paper, we propose late fusion MVC via alignment maximization to address these issues. To do so, we first reveal the theoretical connection of existing k-means clustering and the alignment between base partitions and the consensus one. Based on this observation, we propose a simple but effective multi-view algorithm termed LF-MVC-GAM. It optimally fuses multiple source information in partition level from each individual view, and maximally aligns the consensus partition with these weighted base ones. Such an alignment is beneficial to integrate partition level information and significantly reduce the computational complexity by sufficiently simplifying the optimization procedure. We then design another variant, LF-MVC-LAM to further improve the clustering performance by preserving the local intrinsic structure among multiple partition spaces. After that, we develop two three-step iterative algorithms to solve the resultant optimization problems with theoretically guaranteed convergence. Further, we provide the generalization error bound analysis of the proposed algorithms. Extensive experiments on eighteen multi-view benchmark datasets demonstrate the effectiveness and efficiency of the proposed LF-MVC-GAM and LF-MVC-LAM, ranging from small to large-scale data items. The codes of the proposed algorithms are publicly available at https://github.com/wangsiwei2010/latefusionalignment.
Fast Kernel Density Estimation with Density Matrices and Random Fourier Features
Venue: arXiv cs.LG 2022
Year: 2022

Kernel density estimation (KDE) is one of the most widely used nonparametric density estimation methods. The fact that it is a memory-based method, i.e., it uses the entire training data set for prediction, makes it unsuitable for most current big data applications. Several strategies, such as tree-based or hashing-based estimators, have been proposed to improve the efficiency of the kernel density estimation method. The novel density kernel density estimation method (DMKDE) uses density matrices, a quantum mechanical formalism, and random Fourier features, an explicit kernel approximation, to produce density estimates. This method has its roots in the KDE and can be considered as an approximation method, without its memory-based restriction. In this paper, we systematically evaluate the novel DMKDE algorithm and compare it with other state-of-the-art fast procedures for approximating the kernel density estimation method on different synthetic data sets. Our experimental results show that DMKDE is on par with its competitors for computing density estimates and advantages are shown when performed on high-dimensional data. We have made all the code available as an open source software repository.
Detecting and Characterizing Propagation of Security Weaknesses in Puppet-based Infrastructure Management
Venue: arXiv cs.CR 2022
Year: 2022

Despite being beneficial for managing computing infrastructure automatically, Puppet manifests are susceptible to security weaknesses, e.g., hard-coded secrets and use of weak cryptography algorithms. Adequate mitigation of security weaknesses in Puppet manifests is thus necessary to secure computing infrastructure that are managed with Puppet manifests. A characterization of how security weaknesses propagate and affect Puppet-based infrastructure management, can inform practitioners on the relevance of the detected security weaknesses, as well as help them take necessary actions for mitigation. To that end, we conduct an empirical study with 17,629 Puppet manifests mined from 336 open source repositories. We construct Taint Tracker for Puppet Manifests (TaintPup), for which we observe 2.4 times more precision compared to that of a state-of-the-art security static analysis tool. TaintPup leverages Puppet-specific information flow analysis using which we characterize propagation of security weaknesses. From our empirical study, we observe security weaknesses to propagate into 4,457 resources, i.e, Puppet-specific code elements used to manage infrastructure. A single instance of a security weakness can propagate into as many as 35 distinct resources. We observe security weaknesses to propagate into 7 categories of resources, which include resources used to manage continuous integration servers and network controllers. According to our survey with 24 practitioners, propagation of security weaknesses into data storage-related resources is rated to have the most severe impact for Puppet-based infrastructure management.
A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting
Venue: arXiv cs.CV 2022
Year: 2022

Earth Observatory is a growing research area that can capitalize on the powers of AI for short time forecasting, a Now-casting scenario. In this work, we tackle the challenge of weather forecasting using a video transformer network. Vision transformer architectures have been explored in various applications, with major constraints being the computational complexity of Attention and the data hungry training. To address these issues, we propose the use of Video Swin-Transformer, coupled with a dedicated augmentation scheme. Moreover, we employ gradual spatial reduction on the encoder side and cross-attention on the decoder. The proposed approach is tested on the Weather4Cast2021 weather forecasting challenge data, which requires the prediction of 8 hours ahead future frames (4 per hour) from an hourly weather product sequence. The dataset was normalized to 0-1 to facilitate using the evaluation metrics across different datasets. The model results in an MSE score of 0.4750 when provided with training data, and 0.4420 during transfer learning without using training data, respectively.
A ZK-SNARK based Proof of Assets Protocol for Bitcoin Exchanges
Venue: arXiv cs.CR 2022
Year: 2022

This paper proposes a protocol for Proof of Assets of a bitcoin exchange using the Zero-Knowledge Succinct Non-Interactive Argument of Knowledge (ZK-SNARK) without revealing either the bitcoin addresses of the exchange or balances associated with those addresses. The proof of assets is a mechanism to prove the total value of bitcoins the exchange has authority to spend using its private keys. We construct a privacy-preserving ZK-SNARK proof system to prove the knowledge of the private keys corresponding to the bitcoin assets of an exchange. The ZK-SNARK tool-chain helps to convert an NP-Statement for proving the knowledge of the private keys (known to the exchange) into a circuit satisfiability problem. In this protocol, the exchange creates a Pedersen commitment to the value of bitcoins associated with each address without revealing the balance. The simulation results show that the proof generation time, size, and verification time are efficient in practice.
Security Requirement Analysis of Blockchain-based E-Voting Systems
Venue: arXiv cs.CR 2022
Year: 2022

In democratic countries such as India, voting is a fundamental right given to citizens of their countries. Citizens need to physically present and cast their vote in ballot-paper-based voting systems. Most of the citizens fail to fulfill this constraint and have stayed away from their fundamental duty. Electronic-voting systems are often considered one efficient alternative in such situations. Blockchain Technology is an emerging technology that can provide a real solution as it is characterized by immutable, transparent, anonymous, and decentralized properties. This paper presents a security requirement analysis for e-voting systems and evaluates blockchain technology against these requirements.