OpenRaiser/Intern-Atlas huggingface dataset:
Blind Quality Assessment of 3D Dense Point Clouds with Structure Guided Resampling
Venue: arXiv cs.MM 2022
Year: 2022

Objective quality assessment of 3D point clouds is essential for the development of immersive multimedia systems in real-world applications. Despite the success of perceptual quality evaluation for 2D images and videos, blind/no-reference metrics are still scarce for 3D point clouds with large-scale irregularly distributed 3D points. Therefore, in this paper, we propose an objective point cloud quality index with Structure Guided Resampling (SGR) to automatically evaluate the perceptually visual quality of 3D dense point clouds. The proposed SGR is a general-purpose blind quality assessment method without the assistance of any reference information. Specifically, considering that the human visual system (HVS) is highly sensitive to structure information, we first exploit the unique normal vectors of point clouds to execute regional pre-processing which consists of keypoint resampling and local region construction. Then, we extract three groups of quality-related features, including: 1) geometry density features; 2) color naturalness features; 3) angular consistency features. Both the cognitive peculiarities of the human brain and naturalness regularity are involved in the designed quality-aware features that can capture the most vital aspects of distorted 3D point clouds. Extensive experiments on several publicly available subjective point cloud quality databases validate that our proposed SGR can compete with state-of-the-art full-reference, reduced-reference, and no-reference quality assessment algorithms.
SIM-Trans: Structure Information Modeling Transformer for Fine-grained Visual Categorization
Venue: arXiv cs.CV 2022
Year: 2022

Fine-grained visual categorization (FGVC) aims at recognizing objects from similar subordinate categories, which is challenging and practical for human's accurate automatic recognition needs. Most FGVC approaches focus on the attention mechanism research for discriminative regions mining while neglecting their interdependencies and composed holistic object structure, which are essential for model's discriminative information localization and understanding ability. To address the above limitations, we propose the Structure Information Modeling Transformer (SIM-Trans) to incorporate object structure information into transformer for enhancing discriminative representation learning to contain both the appearance information and structure information. Specifically, we encode the image into a sequence of patch tokens and build a strong vision transformer framework with two well-designed modules: (i) the structure information learning (SIL) module is proposed to mine the spatial context relation of significant patches within the object extent with the help of the transformer's self-attention weights, which is further injected into the model for importing structure information; (ii) the multi-level feature boosting (MFB) module is introduced to exploit the complementary of multi-level features and contrastive learning among classes to enhance feature robustness for accurate recognition. The proposed two modules are light-weighted and can be plugged into any transformer network and trained end-to-end easily, which only depends on the attention weights that come with the vision transformer itself. Extensive experiments and analyses demonstrate that the proposed SIM-Trans achieves state-of-the-art performance on fine-grained visual categorization benchmarks. The code is available at https://github.com/PKU-ICST-MIPL/SIM-Trans_ACMMM2022.
Rethinking Conversational Recommendations: Is Decision Tree All You Need?
Venue: arXiv cs.IR 2022
Year: 2022

Conversational recommender systems (CRS) dynamically obtain the user preferences via multi-turn questions and answers. The existing CRS solutions are widely dominated by deep reinforcement learning algorithms. However, deep reinforcement learning methods are often criticised for lacking interpretability and requiring a large amount of training data to perform. In this paper, we explore a simpler alternative and propose a decision tree based solution to CRS. The underlying challenge in CRS is that the same item can be described differently by different users. We show that decision trees are sufficient to characterize the interactions between users and items, and solve the key challenges in multi-turn CRS: namely which questions to ask, how to rank the candidate items, when to recommend, and how to handle negative feedback on the recommendations. Firstly, the training of decision trees enables us to find questions which effectively narrow down the search space. Secondly, by learning embeddings for each item and tree nodes, the candidate items can be ranked based on their similarity to the conversation context encoded by the tree nodes. Thirdly, the diversity of items associated with each tree node allows us to develop an early stopping strategy to decide when to make recommendations. Fourthly, when the user rejects a recommendation, we adaptively choose the next decision tree to improve subsequent questions and recommendations. Extensive experiments on three publicly available benchmark CRS datasets show that our approach provides significant improvement to the state of the art CRS methods.
Fine-Grained Distribution-Dependent Learning Curves
Venue: arXiv cs.LG 2022
Year: 2022

Learning curves plot the expected error of a learning algorithm as a function of the number of labeled samples it receives from a target distribution. They are widely used as a measure of an algorithm's performance, but classic PAC learning theory cannot explain their behavior. As observed by Antos and Lugosi (1996 , 1998), the classic `No Free Lunch' lower bounds only trace the upper envelope above all learning curves of specific target distributions. For a concept class with VC dimension $d$ the classic bound decays like $d/n$, yet it is possible that the learning curve for \emph{every} specific distribution decays exponentially. In this case, for each $n$ there exists a different `hard' distribution requiring $d/n$ samples. Antos and Lugosi asked which concept classes admit a `strong minimax lower bound' -- a lower bound of $d'/n$ that holds for a fixed distribution for infinitely many $n$. We solve this problem in a principled manner, by introducing a combinatorial dimension called VCL that characterizes the best $d'$ for which $d'/n$ is a strong minimax lower bound. Our characterization strengthens the lower bounds of Bousquet, Hanneke, Moran, van Handel, and Yehudayoff (2021), and it refines their theory of learning curves, by showing that for classes with finite VCL the learning rate can be decomposed into a linear component that depends only on the hypothesis class and an exponential component that depends also on the target distribution. As a corollary, we recover the lower bound of Antos and Lugosi (1996 , 1998) for half-spaces in $\mathbb{R}^d$. Finally, to provide another viewpoint on our work and how it compares to traditional PAC learning bounds, we also present an alternative formulation of our results in a language that is closer to the PAC setting.
Audiogram Digitization Tool for Audiological Reports
Venue: arXiv cs.CV 2022
Year: 2022

A number of private and public insurers compensate workers whose hearing loss can be directly attributed to excessive exposure to noise in the workplace. The claim assessment process is typically lengthy and requires significant effort from human adjudicators who must interpret hand-recorded audiograms, often sent via fax or equivalent. In this work, we present a solution developed in partnership with the Workplace Safety Insurance Board of Ontario to streamline the adjudication process. In particular, we present the first audiogram digitization algorithm capable of automatically extracting the hearing thresholds from a scanned or faxed audiology report as a proof-of-concept. The algorithm extracts most thresholds within 5 dB accuracy, allowing to substantially lessen the time required to convert an audiogram into digital format in a semi-supervised fashion, and is a first step towards the automation of the adjudication process. The source code for the digitization algorithm and a desktop-based implementation of our NIHL annotation portal is publicly available on GitHub (https://github.com/GreenCUBIC/AudiogramDigitization).
Mapping aids using source location tracking increase novices' performance in programming cyber-physical systems
Venue: arXiv cs.SE 2022
Year: 2022

Novices need to overcome initial barriers while programming cyber-physical systems behavior, like coding quadcopter missions, and should thus be supported by an adequately designed programming environment. Using multiple representations by including graphical previews is a common approach to ease coding and program understanding. However, novices struggle to map information of the code and graphical previews. Previous studies imply that mapping aids in a live programming environment might support novices while programming and foster a deeper understanding of the content. To implement these mapping aids in a domain independent way Source Location Tracking based on run-time information can be used. In our study, we tested N=82 participants while interacting and learning in an online programming environment. Using our 2x2 between-subject design study, we investigated the effects of two mapping aids: highlighting and dynamic linking on coding correctness including typical errors, and learning outcomes. Based on process data, successful strategies were analyzed. Combining both mapping aids compared to one aid resulted in higher performance. While highlights were more helpful for implementing the quadcopter missions, dynamic linking improved learning outcomes on the comprehension and application level . Traces of learning strategies were related to higher coding correctness and higher learning outcomes. Based on process data, users in the group with both aids had a higher chance of avoiding certain typical implementation mistakes. Implementing dynamic linking and highlighting through source location tracking is a promising approach to support novices to develop a better semantic understanding of the domain specific language. Depending on the coding tasks different mapping aids might be effective.
Evolving Microstructures in Relaxed Continuum Damage Mechanics for Strain Softening
Venue: arXiv cs.CE 2022
Year: 2022

A new relaxation approach is proposed which allows for the description of stress- and strain-softening at finite strains. The model is based on the construction of a convex hull replacing the originally non-convex incremental stress potential which in turn represents damage in terms of the classical $(1-D)$ approach. This convex hull is given as the linear convex combination of weakly and strongly damaged phases and thus, it represents the homogenization of a microstructure bifurcated in the two phases. As a result thereof, damage evolves in the convexified regime mainly by an increasing volume fraction of the strongly damaged phase. In contrast to previous relaxed incremental formulations in Gürses and Miehe [16] and Balzani and Ortiz [2], where the convex hull has been kept fixated after construction, here, the strongly damaged phase is allowed to elastically unload upon further loading. At the same time, its volume fraction increases nonlinearly within the convexified regime. Thus, strain-softening in the sense of a decreasing stress with increasing strain can be modeled. The major advantage of the proposed approach is that it ensures mesh-independent structural simulations without the requirement of additional length-scale related parameters or nonlocal quantities, which simplifies an implementation using classical material subroutine interfaces. In this paper, focus is on the relaxation of one-dimensional models for fiber damage which are combined with a microsphere approach to allow for the description of three-dimensional fiber dispersions appearing in fibrous materials such as soft biological tissues. Several numerical examples are analyzed to show the overall response of the model and the mesh-independence of resulting structural calculations.
A Real-Time Tempo and Meter Tracking System for Rhythmic Improvis
Venue: arXiv cs.SD 2022
Year: 2022

Music is a form of expression that often requires interaction between players. If one wishes to interact in such a musical way with a computer, it is necessary for the machine to be able to interpret the input given by the human to find its musical meaning. In this work, we propose a system capable of detecting basic rhythmic features that can allow an application to synchronize its output with the rhythm given by the user, without having any prior agreement or requirement on the possible input. The system is described in detail and an evaluation is given through simulation using quantitative metrics. The evaluation shows that the system can detect tempo and meter consistently under certain settings, and could be a solid base for further developments leading to a system robust to rhythmically changing inputs.
A Finite-Automaton Based Stream Cipher As a Quasigroup Based Cipher
Venue: arXiv cs.CR 2022
Year: 2022

In this paper we show that a recently published finite automaton stream cipher can be considered as a quasigroup based stream cipher. Some additional properties of the discussed cipher are also given.
Open Challenges in Musical Metacreation
Venue: arXiv cs.SD 2022
Year: 2022

Musical Metacreation tries to obtain creative behaviors from computers algorithms composing music. In this paper I briefly analyze how this field evolved from algorithmic composition to be focused on the search for creativity, and I point out some issues in pursuing this goal. Finally, I argue that hybridization of algorithms can be a useful direction for research.
Scatter Points in Space: 3D Detection from Multi-view Monocular Images
Venue: arXiv cs.CV 2022
Year: 2022

3D object detection from monocular image(s) is a challenging and long-standing problem of computer vision. To combine information from different perspectives without troublesome 2D instance tracking, recent methods tend to aggregate multiview feature by sampling regular 3D grid densely in space, which is inefficient. In this paper, we attempt to improve multi-view feature aggregation by proposing a learnable keypoints sampling method, which scatters pseudo surface points in 3D space, in order to keep data sparsity. The scattered points augmented by multi-view geometric constraints and visual features are then employed to infer objects location and shape in the scene. To make up the limitations of single frame and model multi-view geometry explicitly, we further propose a surface filter module for noise suppression. Experimental results show that our method achieves significantly better performance than previous works in terms of 3D detection (more than 0.1 AP improvement on some categories of ScanNet). The code will be publicly available.
Harmonization and Evaluation; Tweaking the Parameters on Human Listeners
Venue: arXiv cs.SD 2022
Year: 2022

Kansei models were used to study the connotative meaning of music. In multimedia and mixed reality, automatically generated melodies are increasingly being used. It is important to consider whether and what feelings are communicated by this music. Evaluation of computer-generated melodies is not a trivial task. Considered the difficulty of defining useful quantitative metrics of the quality of a generated musical piece, researchers often resort to human evaluation. In these evaluations, often the judges are required to evaluate a set of generated pieces along with some benchmark pieces. The latter are often composed by humans. While this kind of evaluation is relatively common, it is known that care should be taken when designing the experiment, as humans can be influenced by a variety of factors. In this paper, we examine the impact of the presence of harmony in audio files that judges must evaluate, to see whether having an accompaniment can change the evaluation of generated melodies. To do so, we generate melodies with two different algorithms and harmonize them with an automatic tool that we designed for this experiment, and ask more than sixty participants to evaluate the melodies. By using statistical analyses, we show harmonization does impact the evaluation process, by emphasizing the differences among judgements.
Python Type Hints are Turing Complete
Venue: arXiv cs.PL 2022
Year: 2022

Grigore showed that Java generics are Turing complete by describing a reduction from Turing machines to Java subtyping. We apply Grigore's algorithm to Python type hints and deduce that they are Turing complete. In addition, we present an alternative reduction in which the Turing machines are simulated in real time, resulting in significantly lower compilation times. Our work is accompanied by a Python implementation of both reductions that compiles Turing machines into Python subtyping machines.
Graph Distance Neural Networks for Predicting Multiple Drug Interactions
Venue: arXiv cs.LG 2022
Year: 2022

Since multidrug combination is widely applied, the accurate prediction of drug-drug interaction (DDI) is becoming more and more critical. In our method, we use graph to represent drug-drug interaction: nodes represent drug; edges represent drug-drug interactions. Based on our assumption, we convert the prediction of DDI to link prediction problem, utilizing known drug node characteristics and DDI types to predict unknown DDI types. This work proposes a Graph Distance Neural Network (GDNN) to predict drug-drug interactions. Firstly, GDNN generates initial features for nodes via target point method, fully including the distance information in the graph. Secondly, GDNN adopts an improved message passing framework to better generate each drug node embedded expression, comprehensively considering the nodes and edges characteristics synchronously. Thirdly, GDNN aggregates the embedded expressions, undergoing MLP processing to generate the final predicted drug interaction type. GDNN achieved Test Hits@20=0.9037 on the ogb-ddi dataset, proving GDNN can predict DDI efficiently.
A Fair Experimental Comparison of Neural Network Architectures for Latent Representations of Multi-Omics for Drug Response Prediction
Venue: arXiv cs.LG 2022
Year: 2022

Recent years have seen a surge of novel neural network architectures for the integration of multi-omics data for prediction. Most of the architectures include either encoders alone or encoders and decoders, i.e., autoencoders of various sorts, to transform multi-omics data into latent representations. One important parameter is the depth of integration: the point at which the latent representations are computed or merged, which can be either early, intermediate, or late. The literature on integration methods is growing steadily, however, close to nothing is known about the relative performance of these methods under fair experimental conditions and under consideration of different use cases. We developed a comparison framework that trains and optimizes multi-omics integration methods under equal conditions. We incorporated early integration and four recently published deep learning methods: MOLI, Super.FELT, OmiEmbed, and MOMA. Further, we devised a novel method, Omics Stacking, that combines the advantages of intermediate and late integration. Experiments were conducted on a public drug response data set with multiple omics data (somatic point mutations, somatic copy number profiles and gene expression profiles) that was obtained from cell lines, patient-derived xenografts, and patient samples. Our experiments confirmed that early integration has the lowest predictive performance. Overall, architectures that integrate triplet loss achieved the best results. Statistical differences can, overall, rarely be observed, however, in terms of the average ranks of methods, Super.FELT is consistently performing best in a cross-validation setting and Omics Stacking best in an external test set setting. The source code of all experiments is available under \url{https://github.com/kramerlab/Multi-Omics_analysis}
Attentive pooling for Group Activity Recognition
Venue: arXiv cs.CV 2022
Year: 2022

In group activity recognition, hierarchical framework is widely adopted to represent the relationships between individuals and their corresponding group, and has achieved promising performance. However, the existing methods simply employed max/average pooling in this framework, which ignored the distinct contributions of different individuals to the group activity recognition. In this paper, we propose a new contextual pooling scheme, named attentive pooling, which enables the weighted information transition from individual actions to group activity. By utilizing the attention mechanism, the attentive pooling is intrinsically interpretable and able to embed member context into the existing hierarchical model. In order to verify the effectiveness of the proposed scheme, two specific attentive pooling methods, i.e., global attentive pooling (GAP) and hierarchical attentive pooling (HAP) are designed. GAP rewards the individuals that are significant to group activity, while HAP further considers the hierarchical division by introducing subgroup structure. The experimental results on the benchmark dataset demonstrate that our proposal is significantly superior beyond the baseline and is comparable to the state-of-the-art methods.
Improving Operational Efficiency In EV Ridepooling Fleets By Predictive Exploitation of Idle Times
Venue: arXiv cs.LG 2022
Year: 2022

In ridepooling systems with electric fleets, charging is a complex decision-making process. Most electric vehicle (EV) taxi services require drivers to make egoistic decisions, leading to decentralized ad-hoc charging strategies. The current state of the mobility system is often lacking or not shared between vehicles, making it impossible to make a system-optimal decision. Most existing approaches do not combine time, location and duration into a comprehensive control algorithm or are unsuitable for real-time operation. We therefore present a real-time predictive charging method for ridepooling services with a single operator, called Idle Time Exploitation (ITX), which predicts the periods where vehicles are idle and exploits these periods to harvest energy. It relies on Graph Convolutional Networks and a linear assignment algorithm to devise an optimal pairing of vehicles and charging stations, in pursuance of maximizing the exploited idle time. We evaluated our approach through extensive simulation studies on real-world datasets from New York City. The results demonstrate that ITX outperforms all baseline methods by at least 5% (equivalent to $70,000 for a 6,000 vehicle operation) per week in terms of a monetary reward function which was modeled to replicate the profitability of a real-world ridepooling system. Moreover, ITX can reduce delays by at least 4.68% in comparison with baseline methods and generally increase passenger comfort by facilitating a better spread of customers across the fleet. Our results also demonstrate that ITX enables vehicles to harvest energy during the day, stabilizing battery levels and increasing resilience to unexpected surges in demand. Lastly, compared to the best-performing baseline strategy, peak loads are reduced by 17.39% which benefits grid operators and paves the way for more sustainable use of the electrical grid.