OpenRaiser/Intern-Atlas huggingface dataset:
A Comprehensive Study on Machine Learning Methods to Increase the Prediction Accuracy of Classifiers and Reduce the Number of Medical Tests Required to Diagnose Alzheimer'S Disease
Venue: arXiv cs.LG 2022
Year: 2022

Alzheimer's patients gradually lose their ability to think, behave, and interact with others. Medical history, laboratory tests, daily activities, and personality changes can all be used to diagnose the disorder. A series of time-consuming and expensive tests are used to diagnose the illness. The most effective way to identify Alzheimer's disease is using a Random-forest classifier in this study, along with various other Machine Learning techniques. The main goal of this study is to fine-tune the classifier to detect illness with fewer tests while maintaining a reasonable disease discovery accuracy. We successfully identified the condition in almost 94% of cases using four of the thirty frequently utilized indicators.
Implicit Mixture of Interpretable Experts for Global and Local Interpretability
Venue: arXiv cs.LG 2022
Year: 2022

We investigate the feasibility of using mixtures of interpretable experts (MoIE) to build interpretable image classifiers on MNIST10. MoIE uses a black-box router to assign each input to one of many inherently interpretable experts, thereby providing insight into why a particular classification decision was made. We find that a naively trained MoIE will learn to 'cheat', whereby the black-box router will solve the classification problem by itself, with each expert simply learning a constant function for one particular class. We propose to solve this problem by introducing interpretable routers and training the black-box router's decisions to match the interpretable router. In addition, we propose a novel implicit parameterization scheme that allows us to build mixtures of arbitrary numbers of experts, allowing us to study how classification performance, local and global interpretability vary as the number of experts is increased. Our new model, dubbed Implicit Mixture of Interpretable Experts (IMoIE) can match state-of-the-art classification accuracy on MNIST10 while providing local interpretability, and can provide global interpretability albeit at the cost of reduced classification accuracy.
CUNI Systems for the WMT22 Czech-Ukrainian Translation Task
Venue: arXiv cs.CL 2022
Year: 2022

We present Charles University submissions to the WMT22 General Translation Shared Task on Czech-Ukrainian and Ukrainian-Czech machine translation. We present two constrained submissions based on block back-translation and tagged back-translation and experiment with rule-based romanization of Ukrainian. Our results show that the romanization only has a minor effect on the translation quality. Further, we describe Charles Translator, a system that was developed in March 2022 as a response to the migration from Ukraine to the Czech Republic. Compared to our constrained systems, it did not use the romanization and used some proprietary data sources.
Duplicate Bug Report Detection: How Far Are We?
Venue: arXiv cs.SE 2022
Year: 2022

Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects in our benchmark. In addition, we compared the DBRD techniques proposed in research with those used in Mozilla and VSCode. Surprisingly, we observe that a simple technique already adopted in practice can achieve comparable results as a recently proposed research tool. Our study gives reflections on the current state of DBRD, and we share our insights to benefit future DBRD research.
Probabilistic Neural Circuits leveraging AI-Enhanced Codesign for Random Number Generation
Venue: arXiv cs.ET 2022
Year: 2022

Stochasticity is ubiquitous in the world around us. However, our predominant computing paradigm is deterministic. Random number generation (RNG) can be a computationally inefficient operation in this system especially for larger workloads. Our work leverages the underlying physics of emerging devices to develop probabilistic neural circuits for RNGs from a given distribution. However, codesign for novel circuits and systems that leverage inherent device stochasticity is a hard problem. This is mostly due to the large design space and complexity of doing so. It requires concurrent input from multiple areas in the design stack from algorithms, architectures, circuits, to devices. In this paper, we present examples of optimal circuits developed leveraging AI-enhanced codesign techniques using constraints from emerging devices and algorithms. Our AI-enhanced codesign approach accelerated design and enabled interactions between experts from different areas of the microelectronics design stack including theory, algorithms, circuits, and devices. We demonstrate optimal probabilistic neural circuits using magnetic tunnel junction and tunnel diode devices that generate an RNG from a given distribution.
On Terrain-Aware Locomotion for Legged Robots
Venue: arXiv cs.RO 2022
Year: 2022

(Simplified Abstract) To accomplish breakthroughs in dynamic whole-body locomotion, legged robots have to be terrain aware. Terrain-Aware Locomotion (TAL) implies that the robot can perceive the terrain with its sensors, and can take decisions based on this information. This thesis presents TAL strategies both from a proprioceptive and an exteroceptive perspective. The strategies are implemented at the level of locomotion planning, control, and state estimation, and using optimization and learning techniques. The first part is on TAL strategies at the Whole-Body Control (WBC) level. We introduce a passive WBC (pWBC) framework that allows the robot to stabilize and walk over challenging terrain while taking into account the terrain geometry (inclination) and friction properties. The pWBC relies on rigid contact assumptions which makes it suitable only for stiff terrain. As a consequence, we introduce Soft Terrain Adaptation aNd Compliance Estimation (STANCE) which is a soft terrain adaptation algorithm that generalizes beyond rigid terrain. The second part of the thesis focuses on vision-based TAL strategies. We present Vision-Based Terrain-Aware Locomotion (ViTAL) which is an online planning strategy that selects the footholds based on the robot capabilities, and the robot pose that maximizes the chances of the robot succeeding in reaching these footholds. ViTAL relies on a set of robot skills that characterizes the capabilities of the robot and its legs. The skills include the robot's ability to assess the terrain's geometry, avoid leg collisions, and avoid reaching kinematic limits. Our strategies are based on optimization and learning methods and are validated on HyQ and HyQReal in simulation and experiment. We show that with the help of these strategies, we can push dynamic legged robots one step closer to being fully autonomous and terrain aware.
Game Implementation: What Are the Obstructions?
Venue: arXiv cs.GT 2022
Year: 2022

In many applications, we want to influence the decisions of independent agents by designing incentives for their actions. We revisit a fundamental problem in this area, called GAME IMPLEMENTATION: Given a game in standard form and a set of desired strategies, can we design a set of payment promises such that if the players take the payment promises into account, then all undominated strategies are desired? Furthermore, we aim to minimize the cost, that is, the worst-case amount of payments. We study the tractability of computing such payment promises and determine more closely what obstructions we may have to overcome in doing so. We show that GAME IMPLEMENTATION is NP-hard even for two players, solving in particular a long open question (Eidenbenz et al. 2011) and suggesting more restrictions are necessary to obtain tractability results. We thus study the regime in which players have only a small constant number of strategies and obtain the following. First, this case remains NP-hard even if each player's utility depends only on three others. Second, we repair a flawed efficient algorithm for the case of both small number of strategies and small number of players. Among further results, we characterize sets of desired strategies that can be implemented at zero cost as a kind of stable core of the game.
High Performance Computing and Computational Intelligence Applications with MultiChaos Perspective
Venue: arXiv cs.CY 2022
Year: 2022

The experience of the COVID-19 pandemic, which has accelerated many chaotic processes in modern society, has highlighted in a very serious and urgent way the need to understand complex processes in order to achieve the common well-being. Modern High performance computing technologies, Quantum Computing, Computational Intelligence are shown to be extremely efficient and useful in safeguarding the fate of mankind. These technologies are the state of the art of IT evolution and are fundamental to be competitive and efficient today. If a company is familiar with these techniques and technologies, will be able to deal with any unexpected and complicated scenario more efficiently and effectively. The main contribution of our work is a set of best practices and case studies that can help the researcher address computationally complex problems. We offer a range of software technologies, from high performance computing to machine learning and quantum computing, which represent today the state of the art to deal with extremely complex computational issues, driven by chaotic events and not easily predictable. In this chapter we analyse the different technologies and applications that will lead mankind to overcome this difficult moment, as well as to understand more and more deeply the profound aspects of very complex phenomena. In this environment of rising complexity, both in terms of technology, algorithms, and changing lifestyles, it is critical to emphasize the importance of achieving maximum efficiency and outcomes while protecting the integrity of everyone's personal data and respecting the human being as a whole.
Predicting Digital Asset Prices using Natural Language Processing: a survey
Venue: arXiv cs.CY 2022
Year: 2022

Blockchain technology has changed how people think about how they used to store and trade their assets, as it introduced us to a whole new way to transact: using digital currencies. One of the major innovations of blockchain technology is decentralization, meaning that traditional financial intermediaries, such as asset-backed security issuers and banks, are eliminated in the process. Even though blockchain technology has been utilized in a wide range of industries, its most prominent application is still cryptocurrencies, with Bitcoin being the first proposed. At its peak in 2021, the market cap for Bitcoin once surpassed 1 trillion US dollars. The open nature of the crypto market poses various challenges and concerns for both potential retail investors and institutional investors, as the price of the investment is highly volatile, and its fluctuations are unpredictable. The rise of Machine Learning, and Natural Language Processing, in particular, has shed some light on monitoring and predicting the price behaviors of cryptocurrencies. This paper aims to review and analyze the recent efforts in applying Machine Learning and Natural Language Processing methods to predict the prices and analyze the behaviors of digital assets such as Bitcoin and Ethereum.
Prioritizing Policies for Furthering Responsible Artificial Intelligence in the United States
Venue: arXiv cs.CY 2022
Year: 2022

Several policy options exist, or have been proposed, to further responsible artificial intelligence (AI) development and deployment. Institutions, including U.S. government agencies, states, professional societies, and private and public sector businesses, are well positioned to implement these policies. However, given limited resources, not all policies can or should be equally prioritized. We define and review nine suggested policies for furthering responsible AI, rank each policy on potential use and impact, and recommend prioritization relative to each institution type. We find that pre-deployment audits and assessments and post-deployment accountability are likely to have the highest impact but also the highest barriers to adoption. We recommend that U.S. government agencies and companies highly prioritize development of pre-deployment audits and assessments, while the U.S. national legislature should highly prioritize post-deployment accountability. We suggest that U.S. government agencies and professional societies should highly prioritize policies that support responsible AI research and that states should highly prioritize support of responsible AI education. We propose that companies can highly prioritize involving community stakeholders in development efforts and supporting diversity in AI development. We advise lower levels of prioritization across institutions for AI ethics statements and databases of AI technologies or incidents. We recognize that no one policy will lead to responsible AI and instead advocate for strategic policy implementation across institutions.
P(Expression|Grammar): Probability of deriving an algebraic expression with a probabilistic context-free grammar
Venue: arXiv cs.FL 2022
Year: 2022

Probabilistic context-free grammars have a long-term record of use as generative models in machine learning and symbolic regression. When used for symbolic regression, they generate algebraic expressions. We define the latter as equivalence classes of strings derived by grammar and address the problem of calculating the probability of deriving a given expression with a given grammar. We show that the problem is undecidable in general. We then present specific grammars for generating linear, polynomial, and rational expressions, where algorithms for calculating the probability of a given expression exist. For those grammars, we design algorithms for calculating the exact probability and efficient approximation with arbitrary precision.
High-dimensional density estimation with tensorizing flow
Venue: arXiv cs.LG 2022
Year: 2022

We propose the tensorizing flow method for estimating high-dimensional probability density functions from the observed data. The method is based on tensor-train and flow-based generative modeling. Our method first efficiently constructs an approximate density in the tensor-train form via solving the tensor cores from a linear system based on the kernel density estimators of low-dimensional marginals. We then train a continuous-time flow model from this tensor-train density to the observed empirical distribution by performing a maximum likelihood estimation. The proposed method combines the optimization-less feature of the tensor-train with the flexibility of the flow-based generative models. Numerical results are included to demonstrate the performance of the proposed method.
The purpose of qualia: What if human thinking is not (only) information processing?
Venue: arXiv cs.AI 2022
Year: 2022

Despite recent breakthroughs in the field of artificial intelligence (AI) - or more specifically machine learning (ML) algorithms for object recognition and natural language processing - it seems to be the majority view that current AI approaches are still no real match for natural intelligence (NI). More importantly, philosophers have collected a long catalogue of features which imply that NI works differently from current AI not only in a gradual sense, but in a more substantial way: NI is closely related to consciousness, intentionality and experiential features like qualia (the subjective contents of mental states) and allows for understanding (e.g., taking insight into causal relationships instead of 'blindly' relying on correlations), as well as aesthetical and ethical judgement beyond what we can put into (explicit or data-induced implicit) rules to program machines with. Additionally, Psychologists find NI to range from unconscious psychological processes to focused information processing, and from embodied and implicit cognition to 'true' agency and creativity. NI thus seems to transcend any neurobiological functionalism by operating on 'bits of meaning' instead of information in the sense of data, quite unlike both the 'good old fashioned', symbolic AI of the past, as well as the current wave of deep neural network based, 'sub-symbolic' AI, which both share the idea of thinking as (only) information processing. In the following I propose an alternative view of NI as information processing plus 'bundle pushing', discuss an example which illustrates how bundle pushing can cut information processing short, and suggest first ideas for scientific experiments in neuro-biology and information theory as further investigations.
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding
Venue: arXiv cs.CV 2022
Year: 2022

Performing 3D dense captioning and visual grounding requires a common and shared understanding of the underlying multimodal relationships. However, despite some previous attempts on connecting these two related tasks with highly task-specific neural modules, it remains understudied how to explicitly depict their shared nature to learn them simultaneously. In this work, we propose UniT3D, a simple yet effective fully unified transformer-based architecture for jointly solving 3D visual grounding and dense captioning. UniT3D enables learning a strong multimodal representation across the two tasks through a supervised joint pre-training scheme with bidirectional and seq-to-seq objectives. With a generic architecture design, UniT3D allows expanding the pre-training scope to more various training sources such as the synthesized data from 2D prior knowledge to benefit 3D vision-language tasks. Extensive experiments and analysis demonstrate that UniT3D obtains significant gains for 3D dense captioning and visual grounding.
Focus! Relevant and Sufficient Context Selection for News Image Captioning
Venue: arXiv cs.CV 2022
Year: 2022

News Image Captioning requires describing an image by leveraging additional context from a news article. Previous works only coarsely leverage the article to extract the necessary context, which makes it challenging for models to identify relevant events and named entities. In our paper, we first demonstrate that by combining more fine-grained context that captures the key named entities (obtained via an oracle) and the global context that summarizes the news, we can dramatically improve the model's ability to generate accurate news captions. This begs the question, how to automatically extract such key entities from an image? We propose to use the pre-trained vision and language retrieval model CLIP to localize the visually grounded entities in the news article and then capture the non-visual entities via an open relation extraction model. Our experiments demonstrate that by simply selecting a better context from the article, we can significantly improve the performance of existing models and achieve new state-of-the-art performance on multiple benchmarks.
Modeling Mobile Health Users as Reinforcement Learning Agents
Venue: arXiv cs.LG 2022
Year: 2022

Mobile health (mHealth) technologies empower patients to adopt/maintain healthy behaviors in their daily lives, by providing interventions (e.g. push notifications) tailored to the user's needs. In these settings, without intervention, human decision making may be impaired (e.g. valuing near term pleasure over own long term goals). In this work, we formalize this relationship with a framework in which the user optimizes a (potentially impaired) Markov Decision Process (MDP) and the mHealth agent intervenes on the user's MDP parameters. We show that different types of impairments imply different types of optimal intervention. We also provide analytical and empirical explorations of these differences.
Progressive Feature Upgrade in Semi-supervised Learning on Tabular Domain
Venue: arXiv cs.LG 2022
Year: 2022

Recent semi-supervised and self-supervised methods have shown great success in the image and text domain by utilizing augmentation techniques. Despite such success, it is not easy to transfer this success to tabular domains. It is not easy to adapt domain-specific transformations from image and language to tabular data due to mixing of different data types (continuous data and categorical data) in the tabular domain. There are a few semi-supervised works on the tabular domain that have focused on proposing new augmentation techniques for tabular data. These approaches may have shown some improvement on datasets with low-cardinality in categorical data. However, the fundamental challenges have not been tackled. The proposed methods either do not apply to datasets with high-cardinality or do not use an efficient encoding of categorical data. We propose using conditional probability representation and an efficient progressively feature upgrading framework to effectively learn representations for tabular data in semi-supervised applications. The extensive experiments show superior performance of the proposed framework and the potential application in semi-supervised settings.
Human-instructed Deep Hierarchical Generative Learning for Automated Urban Planning
Venue: arXiv cs.AI 2022
Year: 2022

The essential task of urban planning is to generate the optimal land-use configuration of a target area. However, traditional urban planning is time-consuming and labor-intensive. Deep generative learning gives us hope that we can automate this planning process and come up with the ideal urban plans. While remarkable achievements have been obtained, they have exhibited limitations in lacking awareness of: 1) the hierarchical dependencies between functional zones and spatial grids; 2) the peer dependencies among functional zones; and 3) human regulations to ensure the usability of generated configurations. To address these limitations, we develop a novel human-instructed deep hierarchical generative model. We rethink the urban planning generative task from a unique functionality perspective, where we summarize planning requirements into different functionality projections for better urban plan generation. To this end, we develop a three-stage generation process from a target area to zones to grids. The first stage is to label the grids of a target area with latent functionalities to discover functional zones. The second stage is to perceive the planning requirements to form urban functionality projections. We propose a novel module: functionalizer to project the embedding of human instructions and geospatial contexts to the zone-level plan to obtain such projections. Each projection includes the information of land-use portfolios and the structural dependencies across spatial grids in terms of a specific urban function. The third stage is to leverage multi-attentions to model the zone-zone peer dependencies of the functionality projections to generate grid-level land-use configurations. Finally, we present extensive experiments to demonstrate the effectiveness of our framework.