OpenRaiser/Intern-Atlas huggingface dataset:
Federated Learning: Balancing the Thin Line Between Data Intelligence and Privacy
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning holds great promise in learning from fragmented sensitive data and has revolutionized how machine learning models are trained. This article provides a systematic overview and detailed taxonomy of federated learning. We investigate the existing security challenges in federated learning and provide a comprehensive overview of established defense techniques for data poisoning, inference attacks, and model poisoning attacks. The work also presents an overview of current training challenges for federated learning, focusing on handling non-i.i.d. data, high dimensionality issues, and heterogeneous architecture, and discusses several solutions for the associated challenges. Finally, we discuss the remaining challenges in managing federated learning training and suggest focused research directions to address the open questions. Potential candidate areas for federated learning, including IoT ecosystem, healthcare applications, are discussed with a particular focus on banking and financial domains.
Channel Pruned YOLOv5-based Deep Learning Approach for Rapid and Accurate Outdoor Obstacles Detection
Venue: arXiv cs.CV 2022
Year: 2022

One-stage algorithm have been widely used in target detection systems that need to be trained with massive data. Most of them perform well both in real-time and accuracy. However, due to their convolutional structure, they need more computing power and greater memory consumption. Hence, we applied pruning strategy to target detection networks to reduce the number of parameters and the size of model. To demonstrate the practicality of the pruning method, we select the YOLOv5 model for experiments and provide a data set of outdoor obstacles to show the effect of model. In this specific data set, in the best circumstances, the volume of the network model is reduced by 49.7% compared with the original model, and the reasoning time is reduced by 52.5%. Meanwhile, it also uses data processing methods to compensate for the drop in accuracy caused by pruning.
The narrowing of literature use and the restricted mobility of papers in the sciences
Venue: arXiv cs.DL 2022
Year: 2022

It is a matter of debate whether a shrinking proportion of scholarly literature is getting most of the references over time. It is also less well understood how a narrowing literature usage would affect the circulation of ideas in the sciences. Here we show, that the utilization of scientific literature follows dual tendencies over time: while a larger proportion of literature is cited at least a few times, citations are also concentrating more on the top of the citation distribution. Parallel to the latter trend, a paper's future importance increasingly depends on its past citation performance. A random network model shows that the citation concentration is directly related to the greater stability of citation performance. The presented evidence suggests that the growing heterogeneity of citation impact restricts the mobility of research articles that do not gain attention early on. While concentration grows from the beginning of the studied period in 1970, citation dispersion manifest itself significantly only from the mid-1990s when the popularity of freshly published papers has also risen. Most likely, advanced information technologies to disseminate papers are behind both of these latter trends.
Learning to Split for Automatic Bias Detection
Venue: arXiv cs.LG 2022
Year: 2022

Classifiers are biased when trained on biased datasets. As a remedy, we propose Learning to Split (ls), an algorithm for automatic bias detection. Given a dataset with input-label pairs, ls learns to split this dataset so that predictors trained on the training split cannot generalize to the testing split. This performance gap suggests that the testing split is under-represented in the dataset, which is a signal of potential bias. Identifying non-generalizable splits is challenging since we have no annotations about the bias. In this work, we show that the prediction correctness of each example in the testing split can be used as a source of weak supervision: generalization performance will drop if we move examples that are predicted correctly away from the testing split, leaving only those that are mis-predicted. ls is task-agnostic and can be applied to any supervised learning problem, ranging from natural language understanding and image classification to molecular property prediction. Empirical results show that ls is able to generate astonishingly challenging splits that correlate with human-identified biases. Moreover, we demonstrate that combining robust learning algorithms (such as group DRO) with splits identified by ls enables automatic de-biasing. Compared to previous state-of-the-art, we substantially improve the worst-group performance (23.4% on average) when the source of biases is unknown during training and validation.
Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization
Venue: arXiv cs.CL 2022
Year: 2022

Despite recent advances in abstractive summarization, current summarization systems still suffer from content hallucinations where models generate text that is either irrelevant or contradictory to the source document. However, prior work has been predicated on the assumption that any generated facts not appearing explicitly in the source are undesired hallucinations. Methods have been proposed to address this scenario by ultimately improving `faithfulness' to the source document, but in reality, there is a large portion of entities in the gold reference targets that are not directly in the source. In this work, we show that these entities are not aberrations, but they instead require utilizing external world knowledge to infer reasoning paths from entities in the source. We show that by utilizing an external knowledge base, we can improve the faithfulness of summaries without simply making them more extractive, and additionally, we show that external knowledge bases linked from the source can benefit the factuality of generated summaries.
CKH: Causal Knowledge Hierarchy for Estimating Structural Causal Models from Data and Priors
Venue: arXiv cs.AI 2022
Year: 2022

Structural causal models (SCMs) provide a principled approach to identifying causation from observational and experimental data in disciplines ranging from economics to medicine. However, SCMs, which is typically represented as graphical models, cannot rely only on data, rather require support of domain knowledge. A key challenge in this context is the absence of a methodological framework for encoding priors (background knowledge) into causal models in a systematic manner. We propose an abstraction called causal knowledge hierarchy (CKH) for encoding priors into causal models. Our approach is based on the foundation of "levels of evidence" in medicine, with a focus on confidence in causal information. Using CKH, we present a methodological framework for encoding causal priors from various information sources and combining them to derive an SCM. We evaluate our approach on a simulated dataset and demonstrate overall performance compared to the ground truth causal model with sensitivity analysis.
Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer and more accurate features from heterogeneous models, which make it possible to reconstruct higher-quality denoised images under extreme conditions. Extensive experiments show that our MDRN achieves favorable performance against other SID models with fewer parameters. Meanwhile, plenty of ablation studies demonstrate that the introduced HMDS can improve the performance of tiny models or the model under high noise levels, which is extremely useful for related applications.
A hybrid privacy protection scheme for medical data
Venue: arXiv cs.CR 2022
Year: 2022

Healthcare data contains sensitive information, and it is challenging to persuade healthcare data owners to share their information for research purposes without any privacy assurance. The proposed hybrid medical data privacy protection scheme explores the possibility of providing adaptive privacy protection and data utility levels. The evaluation result demonstrates that the scheme can provide adaptive privacy and data utility levels, and the data holder can choose their preferred risk level and data utility through the scheme. The evaluation results on the heart disease and diabetes data demonstrate that the scheme can provide a wide range of adaptive privacy protection and data utility levels to meet different privacy protection and data utility requirements.
Six Feet Apart: Online Payments During the COVID-19 Pandemic
Venue: arXiv cs.HC 2022
Year: 2022

Since the COVID-19 pandemic, businesses have faced unprecedented challenges when trying to remain open. Because COVID-19 spreads through aerosolized droplets, businesses were forced to distance their services; in some cases, distancing may have involved moving business services online. In this work, we explore digitization strategies used by small businesses that remained open during the pandemic, and survey/interview small businesses owners to understand preliminary challenges associated with moving online. Furthermore, we analyze payments from 400K businesses across Japan, Australia, United States, Great Britain, and Canada. Following initial government interventions, we observe (at minimum for each country) a 47% increase in digitizing businesses compared to pre-pandemic levels, with about 80% of surveyed businesses digitizing in under a week. From both our quantitative models and our surveys/interviews, we find that businesses rapidly digitized at the start of the pandemic in preparation of future uncertainty. We also conduct a case-study of initial digitization in the United States, examining finer relationships between specific government interventions, business sectors, political orientation, and resulting digitization shifts. Finally, we discuss the implications of rapid & widespread digitization for small businesses in the context of usability challenges and interpersonal interactions, while highlighting potential shifts in pre-existing social norms.
Maxmin Participatory Budgeting
Venue: arXiv cs.GT 2022
Year: 2022

Participatory Budgeting (PB) is a popular voting method by which a limited budget is divided among a set of projects, based on the preferences of voters over the projects. PB is broadly categorised as divisible PB (if the projects are fractionally implementable) and indivisible PB (if the projects are atomic). Egalitarianism, an important objective in PB, has not received much attention in the context of indivisible PB. This paper addresses this gap through a detailed study of a natural egalitarian rule, Maxmin Participatory Budgeting (MPB), in the context of indivisible PB. Our study is in two parts: (1) computational (2) axiomatic. In the first part, we prove that MPB is computationally hard and give pseudo-polynomial time and polynomial-time algorithms when parameterized by certain well-motivated parameters. We propose an algorithm that achieves for MPB, additive approximation guarantees for restricted spaces of instances and empirically show that our algorithm in fact gives exact optimal solutions on real-world PB datasets. We also establish an upper bound on the approximation ratio achievable for MPB by the family of exhaustive strategy-proof PB algorithms. In the second part, we undertake an axiomatic study of the MPB rule by generalizing known axioms in the literature. Our study leads to the proposal of a new axiom, maximal coverage, which captures fairness aspects. We prove that MPB satisfies maximal coverage.
Deep Geometry Post-Processing for Decompressed Point Clouds
Venue: arXiv cs.CV 2022
Year: 2022

Point cloud compression plays a crucial role in reducing the huge cost of data storage and transmission. However, distortions can be introduced into the decompressed point clouds due to quantization. In this paper, we propose a novel learning-based post-processing method to enhance the decompressed point clouds. Specifically, a voxelized point cloud is first divided into small cubes. Then, a 3D convolutional network is proposed to predict the occupancy probability for each location of a cube. We leverage both local and global contexts by generating multi-scale probabilities. These probabilities are progressively summed to predict the results in a coarse-to-fine manner. Finally, we obtain the geometry-refined point clouds based on the predicted probabilities. Different from previous methods, we deal with decompressed point clouds with huge variety of distortions using a single model. Experimental results show that the proposed method can significantly improve the quality of the decompressed point clouds, achieving 9.30dB BDPSNR gain on three representative datasets on average.
SusTrainable: Promoting Sustainability as a Fundamental Driver in Software Development Training and Education. Teacher Training, November 1-5, Nijmegen, The Netherlands. Revised lecture notes
Venue: arXiv cs.CY 2022
Year: 2022

These are the proceedings of the first teacher training of the Erasmus+ project No. 2020-1-PT01-KA203-078646 -- Sustrainable. The full title of this project is Promoting Sustainability as a Fundamental Driver in Software Development Training and Education and the interested reader may know more about it at: https://sustrainable.github.io The flagship contribution is the organization of two summer schools on sustainable software production. The first summer school is moved due to the Corona pandemic. It is now scheduled for July 2022 in Rijeka, Croatia. This summer school will consist of lectures and practical work for master and PhD students in the area of computing science and closely related fields. There will be contributions from Plovdiv University, University of Košice, University of Coimbra, University of Minho, Eötvös Loránd University, University of Rijeka, Radboud University Nijmegen, University of Pula, University of Amsterdam and Babeş-Bolyai University. To prepare and streamline this summer school, the consortium organized a teacher training in Nijmegen, the Netherlands. This was an event of five full days on November 1-5, 2022 organized by Ingrid Berenbroek, Mart Lubbers and Pieter Koopman on the campus of the Radboud University Nijmegen, The Netherlands. Sustainability is one of the themes within the strategy of the Radboud University called `A Significant Impact', see https://www.ru.nl/sustainability/organisation/radboud-sustainable/. Sustainability plays an important role in education, research, and management. The contributions were reviewed and give a good overview of what can be expected from the summer school. Based on these papers and the very positive and constructive atmosphere, we expect a high quality and pleasant summer school. We are looking forward to contributing to a sustainable future.
Statistical applications of contrastive learning
Venue: arXiv cs.LG 2022
Year: 2022

The likelihood function plays a crucial role in statistical inference and experimental design. However, it is computationally intractable for several important classes of statistical models, including energy-based models and simulator-based models. Contrastive learning is an intuitive and computationally feasible alternative to likelihood-based learning. We here first provide an introduction to contrastive learning and then show how we can use it to derive methods for diverse statistical problems, namely parameter estimation for energy-based models, Bayesian inference for simulator-based models, as well as experimental design.
Biologically-inspired neuronal adaptation improves learning in neural networks
Venue: arXiv cs.NE 2022
Year: 2022

Since humans still outperform artificial neural networks on many tasks, drawing inspiration from the brain may help to improve current machine learning algorithms. Contrastive Hebbian Learning (CHL) and Equilibrium Propagation (EP) are biologically plausible algorithms that update weights using only local information (without explicitly calculating gradients) and still achieve performance comparable to conventional backpropagation. In this study, we augmented CHL and EP with Adjusted Adaptation, inspired by the adaptation effect observed in neurons, in which a neuron's response to a given stimulus is adjusted after a short time. We add this adaptation feature to multilayer perceptrons and convolutional neural networks trained on MNIST and CIFAR-10. Surprisingly, adaptation improved the performance of these networks. We discuss the biological inspiration for this idea and investigate why Neuronal Adaptation could be an important brain mechanism to improve the stability and accuracy of learning.
Distributed Software Evolution: a Survey
Venue: arXiv cs.SE 2022
Year: 2022

Distribution can be a feature of the software evolution process. In other words, temporally and spatially distributed teams and organizations can develop and work on a software application. The simplest case is to outsource production and employ workforce at distributed sites so that multiple distributed teams can work on a project within a parallel framework. If this distribution is global, it will be called the global software evolution or development. A higher level of distribution is defined as decentralization and decentralized software evolution, which means that software development can be independent of the initial provider. It also means that software execution is independent of the initial provider and the initial system so that the software application can easily be reused in different and new projects. However, the high level architecture is managed within a practically centralized framework in the decentralized software evolution. Most of the large scale open-source projects are exemplars of this level. In terms of distribution, there is a higher level of decentralized software evolution called "distributed cognition and leadership". At this level of distribution, all system levels evolve within a distributed framework, and there are no centralized points in the project network and its evolution process. Some open-source software applications are the exemplars of this last level. Not only is the distributed software evolution faced with certain challenges and opportunities to reach its goals, but it has also caused some challenges and opportunities in other fields. This paper conducts a general review of the distributed software evolution. For this purpose, the paper first addresses the importance of the distributed software evolution, and then introduces its noteworthy paradigms.
Improving the estimation of directional area scattering factor (DASF) from canopy reflectance: theoretical basis and validation
Venue: arXiv cs.CE 2022
Year: 2022

Directional area scattering factor (DASF) is a critical canopy structural parameter for vegetation monitoring. It provides an efficient tool for decoupling of canopy structure and leaf optics from canopy reflectance. Current standard approach to estimate DASF from canopy bidirectional reflectance factor (BRF) is based on the assumption that in the weakly absorbing 710 to 790 nm spectral interval, leaf scattering does not change much with the concentration of dry matter and thus its variation can be neglected. This results in biased estimates of DASF and consequently leads to uncertainty in DASF-related applications. This study proposes a new approach to account for variations in concentrations of this biochemical constituent, which additionally uses the canopy BRF at 2260 nm. In silico analysis of the proposed approach suggests significant increase in accuracy over the standard technique by a relative root mean square error (rRMSE) of 49% and 34% for one- and three dimensional scenes, respectively. When compared with indoor multi-angular hyperspectral measurements reported in literature, the mean absolute error has reduced by 68% for needle leaf and 20% for broadleaf canopies. Thus, the proposed DASF estimation approach outperforms the current one and can be used more reliably in DASF-related applications, such as vegetation monitoring of functional traits, dynamics, and radiation budget.
Classifications as Linked Open Data. Challenges and Opportunities
Venue: arXiv cs.DL 2022
Year: 2022

Linked Data (LD) as a web--based technology enables in principle the seamless, machine--supported integration, interplay and augmentation of all kinds of knowledge, into what has been labeled a huge knowledge graph. Despite decades of web technology and, more recently, the LD approach, the task to fully exploit these new technologies in the public domain is only commencing. One specific challenge is to transfer techniques developed preweb to order our knowledge into the realm of Linked Open Data (LOD) This paper illustrates two different models in which a general analytico--synthetic classification can be published and made available as LD. In both cases, an LD solution deals with the intricacies of a pre--coordinated indexing language.
Strategy Synthesis for Global Window PCTL
Venue: arXiv cs.LO 2022
Year: 2022

Given a Markov decision process (MDP) $M$ and a formula $Φ$, the strategy synthesis problem asks if there exists a strategy $σ$ s.t. the resulting Markov chain $M[σ]$ satisfies $Φ$. This problem is known to be undecidable for the probabilistic temporal logic PCTL. We study a class of formulae that can be seen as a fragment of PCTL where a local, bounded horizon property is enforced all along an execution. Moreover, we allow for linear expressions in the probabilistic inequalities. This logic is at the frontier of decidability, depending on the type of strategies considered. In particular, strategy synthesis is decidable when strategies are deterministic while the general problem is undecidable.
A Comparative Study of Meter Detection Methods for Automated Infrastructure Inspection
Venue: arXiv cs.CV 2022
Year: 2022

In order to read meter values from a camera on an autonomous inspection robot with positional errors, it is necessary to detect meter regions from the image. In this study, we developed shape-based, texture-based, and background information-based methods as meter area detection techniques and compared their effectiveness for meters of different shapes and sizes. As a result, we confirmed that the background information-based method can detect the farthest meters regardless of the shape and number of meters, and can stably detect meters with a diameter of 40px.
Randomized Smoothing under Attack: How Good is it in Pratice?
Venue: arXiv cs.CR 2022
Year: 2022

Randomized smoothing is a recent and celebrated solution to certify the robustness of any classifier. While it indeed provides a theoretical robustness against adversarial attacks, the dimensionality of current classifiers necessarily imposes Monte Carlo approaches for its application in practice. This paper questions the effectiveness of randomized smoothing as a defense, against state of the art black-box attacks. This is a novel perspective, as previous research works considered the certification as an unquestionable guarantee. We first formally highlight the mismatch between a theoretical certification and the practice of attacks on classifiers. We then perform attacks on randomized smoothing as a defense. Our main observation is that there is a major mismatch in the settings of the RS for obtaining high certified robustness or when defeating black box attacks while preserving the classifier accuracy.