OpenRaiser/Intern-Atlas huggingface dataset:
Privacy-Preserving Image Classification Using ConvMixer with Adaptive Permutation Matrix
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we propose a privacy-preserving image classification method using encrypted images under the use of the ConvMixer structure. Block-wise scrambled images, which are robust enough against various attacks, have been used for privacy-preserving image classification tasks, but the combined use of a classification network and an adaptation network is needed to reduce the influence of image encryption. However, images with a large size cannot be applied to the conventional method with an adaptation network because the adaptation network has so many parameters. Accordingly, we propose a novel method, which allows us not only to apply block-wise scrambled images to ConvMixer for both training and testing without the adaptation network, but also to provide a higher classification accuracy than conventional methods.
On the use of Artificial Neural Networks in Topology Optimisation
Venue: arXiv cs.CE 2022
Year: 2022

The question of how methods from the field of artificial intelligence can help improve the conventional frameworks for topology optimisation has received increasing attention over the last few years. Motivated by the capabilities of neural networks in image analysis, different model-variations aimed at obtaining iteration-free topology optimisation have been proposed with varying success. Other works focused on speed-up through replacing expensive optimisers and state solvers, or reducing the design-space have been attempted, but have not yet received the same attention. The portfolio of articles presenting different applications has as such become extensive, but few real breakthroughs have yet been celebrated. An overall trend in the literature is the strong faith in the "magic" of artificial intelligence and thus misunderstandings about the capabilities of such methods. The aim of this article is therefore to present a critical review of the current state of research in this field. To this end, an overview of the different model-applications is presented, and efforts are made to identify reasons for the overall lack of convincing success. A thorough analysis identifies and differentiates between problematic and promising aspects of existing models. The resulting findings are used to detail recommendations believed to encourage avenues of potential scientific progress for further research within the field.
Teaching Visual Accessibility in Introductory Data Science Classes with Multi-Modal Data Representations
Venue: arXiv cs.HC 2022
Year: 2022

Although there are various ways to represent data patterns and models, visualization has been primarily taught in many data science courses for its efficiency. Such vision-dependent output may cause critical barriers against those who are blind and visually impaired and people with learning disabilities. We argue that instructors need to teach multiple data representation methods so that all students can produce data products that are more accessible. In this paper, we argue that accessibility should be taught as early as the introductory course as part of the data science curriculum so that regardless of whether learners major in data science or not, they can have foundational exposure to accessibility. As data science educators who teach accessibility as part of our lower-division courses in two different institutions, we share specific examples that can be utilized by other data science instructors.
Hybrid cuckoo search algorithm for the minimum dominating set problem
Venue: arXiv cs.NE 2022
Year: 2022

The notions of dominating sets of graphs began almost 400 years ago with the game of chess, which sparked the analysis of dominating sets of graphs, at first relatively loosely until the beginnings of the 1960s, when the issue was given mathematical description. It's among the most important problems in graph theory, as well as an NP-Complete problem that can't be solved in polynomial time. As a result, we describe a new hybrid cuckoo search technique to tackle the MDS problem in this work. Cuckoo search is a well-known metaheuristic famed for its capacity for exploring a large area of the search space, making it useful for diversification. However, to enhance performance, we incorporated intensification techniques in addition to the genetic crossover operator in the suggested approach. The comparison of our method with the corresponding state-of-the-art techniques from the literature is presented in an exhaustive experimental test. The suggested algorithm outperforms the present state of the art, according to the obtained results.
AG2U -- Autonomous Grading Under Uncertainties
Venue: arXiv cs.RO 2022
Year: 2022

Surface grading, the process of leveling an uneven area containing pre-dumped sand piles, is an important task in the construction site pipeline. This labour-intensive process is often carried out by a dozer, a key machinery tool at any construction site. Current attempts to automate surface grading assume perfect localization. However, in real-world scenarios, this assumption fails, as agents are presented with imperfect perception, which leads to degraded performance. In this work, we address the problem of autonomous grading under uncertainties. First, we implement a simulation and a scaled real-world prototype environment to enable rapid policy exploration and evaluation in this setting. Second, we formalize the problem as a partially observable markov decision process and train an agent capable of handling such uncertainties. We show, through rigorous experiments, that an agent trained under perfect localization will suffer degraded performance when presented with localization uncertainties. However, an agent trained using our method will develop a more robust policy for addressing such errors and, consequently, exhibit a better grading performance.
Edge-centric Optimization of Multi-modal ML-driven eHealth Applications
Venue: arXiv cs.LG 2022
Year: 2022

Smart eHealth applications deliver personalized and preventive digital healthcare services to clients through remote sensing, continuous monitoring, and data analytics. Smart eHealth applications sense input data from multiple modalities, transmit the data to edge and/or cloud nodes, and process the data with compute intensive machine learning (ML) algorithms. Run-time variations with continuous stream of noisy input data, unreliable network connection, computational requirements of ML algorithms, and choice of compute placement among sensor-edge-cloud layers affect the efficiency of ML-driven eHealth applications. In this chapter, we present edge-centric techniques for optimized compute placement, exploration of accuracy-performance trade-offs, and cross-layered sense-compute co-optimization for ML-driven eHealth applications. We demonstrate the practical use cases of smart eHealth applications in everyday settings, through a sensor-edge-cloud framework for an objective pain assessment case study.
Investigating the Impact of Continuous Integration Practices on the Productivity and Quality of Open-Source Projects
Venue: arXiv cs.SE 2022
Year: 2022

Background: Much research has been conducted to investigate the impact of Continuous Integration (CI) on the productivity and quality of open-source projects. Most of studies have analyzed the impact of adopting a CI server service (e.g, Travis-CI) but did not analyze CI sub-practices. Aims: We aim to evaluate the impact of five CI sub-practices with respect to the productivity and quality of GitHub open-source projects. Method: We collect CI sub-practices of 90 relevant open-source projects for a period of 2 years. We use regression models to analyze whether projects upholding the CI sub-practices are more productive and/or generate fewer bugs. We also perform a qualitative document analysis to understand whether CI best practices are related to a higher quality of projects. Results: Our findings reveal a correlation between the Build Activity and Commit Activity sub-practices and the number of merged pull requests. We also observe a correlation between the Build Activity, Build Health and Time to Fix Broken Builds sub-practices and number of bug-related issues. The qualitative analysis reveals that projects with the best values for CI sub-practices face fewer CI-related problems compared to projects that exhibit the worst values for CI sub-practices. Conclusions: We recommend that projects should strive to uphold the several CI sub-practices as they can impact in the productivity and quality of projects.
Surgical Skill Assessment via Video Semantic Aggregation
Venue: arXiv cs.CV 2022
Year: 2022

Automated video-based assessment of surgical skills is a promising task in assisting young surgical trainees, especially in poor-resource areas. Existing works often resort to a CNN-LSTM joint framework that models long-term relationships by LSTMs on spatially pooled short-term CNN features. However, this practice would inevitably neglect the difference among semantic concepts such as tools, tissues, and background in the spatial dimension, impeding the subsequent temporal relationship modeling. In this paper, we propose a novel skill assessment framework, Video Semantic Aggregation (ViSA), which discovers different semantic parts and aggregates them across spatiotemporal dimensions. The explicit discovery of semantic parts provides an explanatory visualization that helps understand the neural network's decisions. It also enables us to further incorporate auxiliary information such as the kinematic data to improve representation learning and performance. The experiments on two datasets show the competitiveness of ViSA compared to state-of-the-art methods. Source code is available at: bit.ly/MICCAI2022ViSA.
Requirements Analysis and Management for Benefiting Openness
Venue: arXiv cs.SE 2022
Year: 2022

Requirements Engineering has recently been greatly influenced by the way how firms use Open Source Software (OSS) and Software Ecosystems (SECOs) as a part of their product development and business models. This is further emphasized by the paradigm of Open Innovation, which highlights how firms should strive to use both internal and external resources to advance their internal innovation and technology capabilities. The evolution from market-driven requirements engineering and management processes, has reshaped the understanding of what a requirement is, and how it is documented and used. In this work, we suggest a model for analyzing and managing requirements that is designed in the context of OSS and SECOs, including the advances and challenges that it brings. The model clarifies how the main stages of requirements engineering and management processes can be adjusted to benefit from the openness that the new context offers. We believe that the model is a first step towards the inevitable adaptation of requirements engineering to an open and informal arena, where processes and collaboration are decentralized, transparency and governance are the key success factors.
Age Appropriate Design: Assessment of TikTok, Twitch, and YouTube Kids
Venue: arXiv cs.CY 2022
Year: 2022

The presence of children in the online world is increasing at a rapid pace. As children interact with services such as video sharing, live streaming, and gaming, a number of concerns arise regarding their security and privacy as well as their safety. To address such concerns, the UK's Information Commissioner's Office (ICO) sets out 15 criteria alongside a risk management process for developers of online services for children. We present an analysis of 15 ICO criteria for age appropriate design. More precisely, we investigate whether those criteria provide actionable requirements for developers and whether video sharing and live streaming platforms that are used by children of different age ranges (i.e., TikTok, Twitch and YouTube Kids) comply with them. Our findings regarding the ICO criteria suggest that some criteria such as age verification and transparency provide adequate guidance for assessment whereas other criteria such as parental controls, reporting of inappropriate content, and handling of sensitive data need further clarification. Our findings regarding the platforms themselves suggest that they choose to implement the simplest form of self-declared age verification with limited parental controls and plenty of opportunities.
ATP: A holistic attention integrated approach to enhance ABSA
Venue: arXiv cs.CL 2022
Year: 2022

Aspect based sentiment analysis (ABSA) deals with the identification of the sentiment polarity of a review sentence towards a given aspect. Deep Learning sequential models like RNN, LSTM, and GRU are current state-of-the-art methods for inferring the sentiment polarity. These methods work well to capture the contextual relationship between the words of a review sentence. However, these methods are insignificant in capturing long-term dependencies. Attention mechanism plays a significant role by focusing only on the most crucial part of the sentence. In the case of ABSA, aspect position plays a vital role. Words near to aspect contribute more while determining the sentiment towards the aspect. Therefore, we propose a method that captures the position based information using dependency parsing tree and helps attention mechanism. Using this type of position information over a simple word-distance-based position enhances the deep learning model's performance. We performed the experiments on SemEval'14 dataset to demonstrate the effect of dependency parsing relation-based attention for ABSA.
Globally Consistent Video Depth and Pose Estimation with Efficient Test-Time Training
Venue: arXiv cs.CV 2022
Year: 2022

Dense depth and pose estimation is a vital prerequisite for various video applications. Traditional solutions suffer from the robustness of sparse feature tracking and insufficient camera baselines in videos. Therefore, recent methods utilize learning-based optical flow and depth prior to estimate dense depth. However, previous works require heavy computation time or yield sub-optimal depth results. We present GCVD, a globally consistent method for learning-based video structure from motion (SfM) in this paper. GCVD integrates a compact pose graph into the CNN-based optimization to achieve globally consistent estimation from an effective keyframe selection mechanism. It can improve the robustness of learning-based methods with flow-guided keyframes and well-established depth prior. Experimental results show that GCVD outperforms the state-of-the-art methods on both depth and pose estimation. Besides, the runtime experiments reveal that it provides strong efficiency in both short- and long-term videos with global consistency provided.
NP Decision Procedure for Monomial and Linear Integer Constraints
Venue: arXiv cs.LO 2022
Year: 2022

Motivated by satisfiability of constraints with function symbols, we consider numerical inequalities on non-negative integers. The constraints we consider are a conjunction of a linear system Ax = b and a conjunction of (non-)convex constraints of the form x_i >= x_j^n (x_i <= x_j^n). We show that the satisfiability of these constraints is NP-complete even if the solution to the linear part is given explicitly. As a consequence, we obtain NP completeness for an extension of certain quantifier-free constraints on sets with cardinalities (QFBAPA) with function images S = f[P^n].
New Differential Privacy Communication Pipeline and Design Framework
Venue: arXiv cs.HC 2022
Year: 2022

Organizations started to adopt differential privacy (DP) techniques hoping to persuade more users to share personal data with them. However, many users do not understand DP techniques, thus may not be willing to share. Previous research suggested that the design of DP mechanism communication could influence users' willingness to share data. Based on the prior work, we propose a new communication pipeline that starts by asking users about their privacy concerns and then provides a customized DP mechanism and communication. We also propose a design framework that systemically explores effective communication designs ranging from a text-based high-level description to a step-by-step interactive storyboard. Based on the framework, we created 17 designs and recruited five people to evaluate. Our user study showed that text-based descriptions have the highest clarity in all scenarios, while the step-by-step interactive storyboards have the potential to persuade users to trust central DP. Our future work will optimize the design and conduct a large-scale efficacy study.
Understanding Teenage Perceptions and Configurations of Privacy on Instagram
Venue: arXiv cs.HC 2022
Year: 2022

As teenage use of social media platform continues to proliferate, so do concerns about teenage privacy and safety online. Prior work has established that privacy on networked publics, such as social media, is complex, requiring users to navigate not only the technical affordances on the platform but also interpersonal relationships and social norms. We investigate how teenagers think about privacy on the popular image-sharing platform, Instagram. We draw on an online survey (N=144) and semi-structured interviews (N=21) with teenagers, ages 13-19, to gain a better understanding how teenagers configure privacy on the popular image-sharing platform Instagram and why they make these privacy decisions. Finally, based on our findings, we provide design recommendations towards the design of better privacy controls for promoting teenage safety online.
Cluster-to-adapt: Few Shot Domain Adaptation for Semantic Segmentation across Disjoint Labels
Venue: arXiv cs.CV 2022
Year: 2022

Domain adaptation for semantic segmentation across datasets consisting of the same categories has seen several recent successes. However, a more general scenario is when the source and target datasets correspond to non-overlapping label spaces. For example, categories in segmentation datasets change vastly depending on the type of environment or application, yet share many valuable semantic relations. Existing approaches based on feature alignment or discrepancy minimization do not take such category shift into account. In this work, we present Cluster-to-Adapt (C2A), a computationally efficient clustering-based approach for domain adaptation across segmentation datasets with completely different, but possibly related categories. We show that such a clustering objective enforced in a transformed feature space serves to automatically select categories across source and target domains that can be aligned for improving the target performance, while preventing negative transfer for unrelated categories. We demonstrate the effectiveness of our approach through experiments on the challenging problem of outdoor to indoor adaptation for semantic segmentation in few-shot as well as zero-shot settings, with consistent improvements in performance over existing approaches and baselines in all cases.
Leveraging the HW/SW Optimizations and Ecosystems that Drive the AI Revolution
Venue: arXiv cs.LG 2022
Year: 2022

This paper presents a state-of-the-art overview on how to architect, design, and optimize Deep Neural Networks (DNNs) such that performance is improved and accuracy is preserved. The paper covers a set of optimizations that span the entire Machine Learning processing pipeline. We introduce two types of optimizations. The first alters the DNN model and requires NN re-training, while the second does not. We focus on GPU optimizations, but we believe the presented techniques can be used with other AI inference platforms. To demonstrate the DNN model optimizations, we improve one of the most advanced deep network architectures for optical flow, RAFT arXiv:2003.12039, on a popular edge AI inference platform (Nvidia Jetson AGX Xavier).
Meta-learning from Learning Curves Challenge: Lessons learned from the First Round and Design of the Second Round
Venue: arXiv cs.LG 2022
Year: 2022

Meta-learning from learning curves is an important yet often neglected research area in the Machine Learning community. We introduce a series of Reinforcement Learning-based meta-learning challenges, in which an agent searches for the best suited algorithm for a given dataset, based on feedback of learning curves from the environment. The first round attracted participants both from academia and industry. This paper analyzes the results of the first round (accepted to the competition program of WCCI 2022), to draw insights into what makes a meta-learner successful at learning from learning curves. With the lessons learned from the first round and the feedback from the participants, we have designed the second round of our challenge with a new protocol and a new meta-dataset. The second round of our challenge is accepted at the AutoML-Conf 2022 and currently ongoing .
TIC: Text-Guided Image Colorization
Venue: arXiv cs.CV 2022
Year: 2022

Image colorization is a well-known problem in computer vision. However, due to the ill-posed nature of the task, image colorization is inherently challenging. Though several attempts have been made by researchers to make the colorization pipeline automatic, these processes often produce unrealistic results due to a lack of conditioning. In this work, we attempt to integrate textual descriptions as an auxiliary condition, along with the grayscale image that is to be colorized, to improve the fidelity of the colorization process. To the best of our knowledge, this is one of the first attempts to incorporate textual conditioning in the colorization pipeline. To do so, we have proposed a novel deep network that takes two inputs (the grayscale image and the respective encoded text description) and tries to predict the relevant color gamut. As the respective textual descriptions contain color information of the objects present in the scene, the text encoding helps to improve the overall quality of the predicted colors. We have evaluated our proposed model using different metrics and found that it outperforms the state-of-the-art colorization algorithms both qualitatively and quantitatively.
Self-Ensembling Vision Transformer (SEViT) for Robust Medical Image Classification
Venue: arXiv cs.CV 2022
Year: 2022

Vision Transformers (ViT) are competing to replace Convolutional Neural Networks (CNN) for various computer vision tasks in medical imaging such as classification and segmentation. While the vulnerability of CNNs to adversarial attacks is a well-known problem, recent works have shown that ViTs are also susceptible to such attacks and suffer significant performance degradation under attack. The vulnerability of ViTs to carefully engineered adversarial samples raises serious concerns about their safety in clinical settings. In this paper, we propose a novel self-ensembling method to enhance the robustness of ViT in the presence of adversarial attacks. The proposed Self-Ensembling Vision Transformer (SEViT) leverages the fact that feature representations learned by initial blocks of a ViT are relatively unaffected by adversarial perturbations. Learning multiple classifiers based on these intermediate feature representations and combining these predictions with that of the final ViT classifier can provide robustness against adversarial attacks. Measuring the consistency between the various predictions can also help detect adversarial samples. Experiments on two modalities (chest X-ray and fundoscopy) demonstrate the efficacy of SEViT architecture to defend against various adversarial attacks in the gray-box (attacker has full knowledge of the target model, but not the defense mechanism) setting. Code: https://github.com/faresmalik/SEViT