OpenRaiser/Intern-Atlas huggingface dataset:
Understanding User Awareness and Behaviors Concerning Encrypted DNS Settings
Venue: arXiv cs.CR 2023
Year: 2023

Recent developments to encrypt the Domain Name System (DNS) have resulted in major browser and operating system vendors deploying encrypted DNS functionality, often enabling various configurations and settings by default. In many cases, default encrypted DNS settings have implications for performance and privacy; for example, Firefox's default DNS setting sends all of a user's DNS queries to Cloudflare, potentially introducing new privacy vulnerabilities. In this paper, we confirm that most users are unaware of these developments -- with respect to the rollout of these new technologies, the changes in default settings, and the ability to customize encrypted DNS configuration to balance user preferences between privacy and performance. Our findings suggest several important implications for the designers of interfaces for encrypted DNS functionality in both browsers and operating systems, to help improve user awareness concerning these settings, and to ensure that users retain the ability to make choices that allow them to balance tradeoffs concerning DNS privacy and performance.
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation
Venue: arXiv cs.CL 2023
Year: 2023

Although the problem of hallucinations in neural machine translation (NMT) has received some attention, research on this highly pathological phenomenon lacks solid ground. Previous work has been limited in several ways: it often resorts to artificial settings where the problem is amplified, it disregards some (common) types of hallucinations, and it does not validate adequacy of detection heuristics. In this paper, we set foundations for the study of NMT hallucinations. First, we work in a natural setting, i.e., in-domain data without artificial noise neither in training nor in inference. Next, we annotate a dataset of over 3.4k sentences indicating different kinds of critical errors and hallucinations. Then, we turn to detection methods and both revisit methods used previously and propose using glass-box uncertainty-based detectors. Overall, we show that for preventive settings, (i) previously used methods are largely inadequate, (ii) sequence log-probability works best and performs on par with reference-based methods. Finally, we propose DeHallucinator, a simple method for alleviating hallucinations at test time that significantly reduces the hallucinatory rate. To ease future research, we release our annotated dataset for WMT18 German-English data, along with the model, training data, and code.
The Effectiveness of Embedded Values Analysis Modules in Computer Science Education: An Empirical Study
Venue: arXiv cs.CY 2023
Year: 2023

Embedding ethics modules within computer science courses has become a popular response to the growing recognition that CS programs need to better equip their students to navigate the ethical dimensions of computing technologies like AI, machine learning, and big data analytics. However, the popularity of this approach has outpaced the evidence of its positive outcomes. To help close that gap, this empirical study reports positive results from Northeastern's program that embeds values analysis modules into CS courses. The resulting data suggest that such modules have a positive effect on students' moral attitudes and that students leave the modules believing they are more prepared to navigate the ethical dimensions they will likely face in their eventual careers. Importantly, these gains were accomplished at an institution without a philosophy doctoral program, suggesting this strategy can be effectively employed by a wider range of institutions than many have thought.
Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP
Venue: arXiv cs.LG 2023
Year: 2023

Web-crawled datasets have enabled remarkable generalization capabilities in recent image-text models such as CLIP (Contrastive Language-Image pre-training) or Flamingo, but little is known about the dataset creation processes. In this work, we introduce a testbed of six publicly available data sources - YFCC, LAION, Conceptual Captions, WIT, RedCaps, Shutterstock - to investigate how pre-training distributions induce robustness in CLIP. We find that the performance of the pre-training data varies substantially across distribution shifts, with no single data source dominating. Moreover, we systematically study the interactions between these data sources and find that combining multiple sources does not necessarily yield better models, but rather dilutes the robustness of the best individual data source. We complement our empirical findings with theoretical insights from a simple setting, where combining the training data also results in diluted robustness. In addition, our theoretical model provides a candidate explanation for the success of the CLIP-based data filtering technique recently employed in the LAION dataset. Overall our results demonstrate that simply gathering a large amount of data from the web is not the most effective way to build a pre-training dataset for robust generalization, necessitating further study into dataset design. Code is available at https://github.com/mlfoundations/clip_quality_not_quantity.
Multi-fidelity wavelet neural operator with application to uncertainty quantification
Venue: arXiv cs.LG 2023
Year: 2023

Operator learning frameworks, because of their ability to learn nonlinear maps between two infinite dimensional functional spaces and utilization of neural networks in doing so, have recently emerged as one of the more pertinent areas in the field of applied machine learning. Although these frameworks are extremely capable when it comes to modeling complex phenomena, they require an extensive amount of data for successful training which is often not available or is too expensive. However, this issue can be alleviated with the use of multi-fidelity learning, where a model is trained by making use of a large amount of inexpensive low-fidelity data along with a small amount of expensive high-fidelity data. To this end, we develop a new framework based on the wavelet neural operator which is capable of learning from a multi-fidelity dataset. The developed model's excellent learning capabilities are demonstrated by solving different problems which require effective correlation learning between the two fidelities for surrogate construction. Furthermore, we also assess the application of the developed framework for uncertainty quantification. The results obtained from this work illustrate the excellent performance of the proposed framework.
PA-Seg: Learning from Point Annotations for 3D Medical Image Segmentation using Contextual Regularization and Cross Knowledge Distillation
Venue: arXiv cs.CV 2023
Year: 2023

The success of Convolutional Neural Networks (CNNs) in 3D medical image segmentation relies on massive fully annotated 3D volumes for training that are time-consuming and labor-intensive to acquire. In this paper, we propose to annotate a segmentation target with only seven points in 3D medical images, and design a two-stage weakly supervised learning framework PA-Seg. In the first stage, we employ geodesic distance transform to expand the seed points to provide more supervision signal. To further deal with unannotated image regions during training, we propose two contextual regularization strategies, i.e., multi-view Conditional Random Field (mCRF) loss and Variance Minimization (VM) loss, where the first one encourages pixels with similar features to have consistent labels, and the second one minimizes the intensity variance for the segmented foreground and background, respectively. In the second stage, we use predictions obtained by the model pre-trained in the first stage as pseudo labels. To overcome noises in the pseudo labels, we introduce a Self and Cross Monitoring (SCM) strategy, which combines self-training with Cross Knowledge Distillation (CKD) between a primary model and an auxiliary model that learn from soft labels generated by each other. Experiments on public datasets for Vestibular Schwannoma (VS) segmentation and Brain Tumor Segmentation (BraTS) demonstrated that our model trained in the first stage outperformed existing state-of-the-art weakly supervised approaches by a large margin, and after using SCM for additional training, the model's performance was close to its fully supervised counterpart on the BraTS dataset.
Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
Venue: arXiv cs.SD 2023
Year: 2023

Lyric-to-melody generation is an important task in songwriting, and is also quite challenging due to its unique characteristics: the generated melodies should not only follow good musical patterns, but also align with features in lyrics such as rhythms and structures. These characteristics cannot be well handled by neural generation models that learn lyric-to-melody mapping in an end-to-end way, due to several issues: (1) lack of aligned lyric-melody training data to sufficiently learn lyric-melody feature alignment; (2) lack of controllability in generation to better and explicitly align the lyric-melody features. In this paper, we propose Re-creation of Creations (ROC), a new paradigm for lyric-to-melody generation. ROC generates melodies according to given lyrics and also conditions on user-designated chord progression. It addresses the above issues through a generation-retrieval pipeline. Specifically, our paradigm has two stages: (1) creation stage, where a huge amount of music fragments generated by a neural melody language model are indexed in a database through several key features (e.g., chords, tonality, rhythm, and structural information); (2) re-creation stage, where melodies are re-created by retrieving music fragments from the database according to the key features from lyrics and concatenating best music fragments based on composition guidelines and melody language model scores. ROC has several advantages: (1) It only needs unpaired melody data to train melody language model, instead of paired lyric-melody data in previous models. (2) It achieves good lyric-melody feature alignment in lyric-to-melody generation. Tested by English and Chinese lyrics, ROC outperforms previous neural based lyric-to-melody generation models on both objective and subjective metrics.
A new anisotropic bending model for nonlinear shells: Comparison with existing models and isogeometric finite element implementation
Venue: arXiv cs.CE 2023
Year: 2023

A new nonlinear hyperelastic bending model for shells formulated directly in surface form is presented, and compared to four prominently used bending models. Through an essential set of elementary nonlinear bending test cases, the stresses and moments of each model are examined analytically. Only the proposed bending model passes all the test cases while the other bending models either fail or only pass the test cases for small deformations. The proposed new bending model can handle large deformations and initially curved surfaces. It is based on the principal curvatures and their directions in the initial configuration, and it thus can have different bending moduli along those directions. These characteristics make it flexible in modeling a given material, while it does not suffer from the pathologies of existing bending models. Further, the bending models are compared computationally through four classical benchmark examples and one contact example. As the underlying shell theory is based on Kirchhoff-Love kinematics, isogeometric NURBS shape functions are used to discretize the shell surface. The linearization and efficient finite element implementation of the proposed new model are also provided.
Slicing4Meta: An Intelligent Integration Framework with Multi-dimensional Network Resources for Metaverse-as-a-Service in Web 3.0
Venue: arXiv cs.NI 2023
Year: 2023

As the next-generation Internet paradigm, Web 3.0 encapsulates the expectations of user-centric immersion and interaction experiences in a decentralized manner. Metaverse, a virtual world interacting with the physical world, is becoming one of the most potential technology to push forward with Web 3.0. In the Metaverse, users expect to tailor immersive and interactive experiences by customizing real-time Metaverse services (e.g., augmented/virtual reality and digital twin) in the three-dimensional virtual world. Nevertheless, there are still no unified solutions for the Metaverse services in terms of management and orchestration. It is calling for a continuous evolution of scalable solutions and mobile systems to support Metaverse services, and thus bring Metaverse into reality. In this paper, to provide scalable subscription solutions for Metaverse services, we propose a new concept, named Metaverse-as-a-Service (MaaS), in which various physical-virtual components and technologies in the Metaverse can be delivered as services. Furthermore, to unify the management and orchestration of MaaS, we propose a novel framework, called Slicing4Meta, to customize Metaverse services by intelligently integrating MaaS models and the associated multi-dimensional resources on the components and technologies. Additionally, we propose the classification for typical Metaverse services based on the quality of experience (QoE) requirements and illustrate how to fulfill the QoE requirements under the Slicing4Meta framework. We then illustrate a virtual traveling study case, in which we examine the relationship between the QoE and the multi-dimensional resources by quantitatively modeling the QoE of Metaverse users. Finally, we discuss some open challenges of Slicing4Meta and propose potential solutions to address the challenges.
Understanding the stochastic dynamics of sequential decision-making processes: A path-integral analysis of multi-armed bandits
Venue: arXiv cs.LG 2023
Year: 2023

The multi-armed bandit (MAB) model is one of the most classical models to study decision-making in an uncertain environment. In this model, a player chooses one of $K$ possible arms of a bandit machine to play at each time step, where the corresponding arm returns a random reward to the player, potentially from a specific unknown distribution. The target of the player is to collect as many rewards as possible during the process. Despite its simplicity, the MAB model offers an excellent playground for studying the trade-off between exploration versus exploitation and designing effective algorithms for sequential decision-making under uncertainty. Although many asymptotically optimal algorithms have been established, the finite-time behaviors of the stochastic dynamics of the MAB model appear much more challenging to analyze, due to the intertwine between the decision-making and the rewards being collected. In this paper, we employ techniques in statistical physics to analyze the MAB model, which facilitates the characterization of the distribution of cumulative regrets at a finite short time, the central quantity of interest in an MAB algorithm, as well as the intricate dynamical behaviors of the model. Our analytical results, in good agreement with simulations, point to the emergence of an interesting multimodal regret distribution, with large regrets resulting from excess exploitation of sub-optimal arms due to an initial unlucky output from the optimal one.
Developing a Philosophical Framework for Fair Machine Learning: Lessons From The Case of Algorithmic Collusion
Venue: arXiv cs.LG 2023
Year: 2023

Fair machine learning research has been primarily concerned with classification tasks that result in discrimination. However, as machine learning algorithms are applied in new contexts the harms and injustices that result are qualitatively different than those presently studied. The existing research paradigm in machine learning which develops metrics and definitions of fairness cannot account for these qualitatively different types of injustice. One example of this is the problem of algorithmic collusion and market fairness. The negative consequences of algorithmic collusion affect all consumers, not only particular members of a protected class. Drawing on this case study, I propose an ethical framework for researchers and practitioners in machine learning seeking to develop and apply fairness metrics that extends to new domains. This contribution ties the development of formal metrics of fairness to specifically scoped normative principles. This enables fairness metrics to reflect different concerns from discrimination. I conclude with the limitations of my proposal and discuss promising avenues for future research.
Relational Action Bases: Formalization, Effective Safety Verification, and Invariants (Extended Version)
Venue: arXiv cs.AI 2023
Year: 2023

Modeling and verification of dynamic systems operating over a relational representation of states are increasingly investigated problems in AI, Business Process Management, and Database Theory. To make these systems amenable to verification, the amount of information stored in each relational state needs to be bounded, or restrictions are imposed on the preconditions and effects of actions. We introduce the general framework of relational action bases (RABs), which generalizes existing models by lifting both these restrictions: unbounded relational states can be evolved through actions that can quantify both existentially and universally over the data, and that can exploit numerical datatypes with arithmetic predicates. We then study parameterized safety of RABs via (approximated) SMT-based backward search, singling out essential meta-properties of the resulting procedure, and showing how it can be realized by an off-the-shelf combination of existing verification modules of the state-of-the-art MCMT model checker. We demonstrate the effectiveness of this approach on a benchmark of data-aware business processes. Finally, we show how universal invariants can be exploited to make this procedure fully correct.
Uni6Dv2: Noise Elimination for 6D Pose Estimation
Venue: arXiv cs.CV 2023
Year: 2023

Uni6D is the first 6D pose estimation approach to employ a unified backbone network to extract features from both RGB and depth images. We discover that the principal reasons of Uni6D performance limitations are Instance-Outside and Instance-Inside noise. Uni6D's simple pipeline design inherently introduces Instance-Outside noise from background pixels in the receptive field, while ignoring Instance-Inside noise in the input depth data. In this paper, we propose a two-step denoising approach for dealing with the aforementioned noise in Uni6D. To reduce noise from non-instance regions, an instance segmentation network is utilized in the first step to crop and mask the instance. A lightweight depth denoising module is proposed in the second step to calibrate the depth feature before feeding it into the pose regression network. Extensive experiments show that our Uni6Dv2 reliably and robustly eliminates noise, outperforming Uni6D without sacrificing too much inference efficiency. It also reduces the need for annotated real data that requires costly labeling.
Multinomial Logistic Regression Algorithms via Quadratic Gradient
Venue: arXiv cs.LG 2023
Year: 2023

Multinomial logistic regression, also known by other names such as multiclass logistic regression and softmax regression, is a fundamental classification method that generalizes binary logistic regression to multiclass problems. A recently work proposed a faster gradient called $\texttt{quadratic gradient}$ that can accelerate the binary logistic regression training, and presented an enhanced Nesterov's accelerated gradient (NAG) method for binary logistic regression. In this paper, we extend this work to multiclass logistic regression and propose an enhanced Adaptive Gradient Algorithm (Adagrad) that can accelerate the original Adagrad method. We test the enhanced NAG method and the enhanced Adagrad method on some multiclass-problem datasets. Experimental results show that both enhanced methods converge faster than their original ones respectively.
Analogical proportions in monounary algebras
Venue: arXiv cs.AI 2023
Year: 2023

This paper studies analogical proportions in monounary algebras consisting only of a universe and a single unary function. We show that the analogical proportion relation is characterized in the infinite monounary algebra formed by the natural numbers together with the successor function via difference proportions.
Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning
Venue: arXiv cs.AI 2023
Year: 2023

Integrating logical reasoning and machine learning by approximating logical inference with differentiable operators is a widely used technique in Neuro-Symbolic systems. However, some differentiable operators could bring a significant bias during backpropagation and degrade the performance of Neuro-Symbolic learning. In this paper, we reveal that this bias, named \textit{Implication Bias} is common in loss functions derived from fuzzy logic operators. Furthermore, we propose a simple yet effective method to transform the biased loss functions into \textit{Reduced Implication-bias Logic Loss (RILL)} to address the above problem. Empirical study shows that RILL can achieve significant improvements compared with the biased logic loss functions, especially when the knowledge base is incomplete, and keeps more robust than the compared methods when labelled data is insufficient.
A novel approach for FPGA-to-server data transmission over an Ethernet-based network using the eXpress Data Path technology
Venue: arXiv cs.NI 2023
Year: 2023

In the context of the upgrade of the Large Hadron Collider at CERN for high-luminosity operation, the particle detectors have to cope with much higher data rates and therefore need to upgrade their data acquisition systems. This upgrade is taken as an opportunity to exchange the currently used highly customized hardware by commercial solutions. Nevertheless, some part of the data processing still needs to be done within Field Programmable Gate Arrays (FPGA), requiring the transfer of data between the FPGAs and the commercial servers. This paper reports on a study of direct data transmission from FPGAs to servers via a commercial network. Large data buffers as required for reliable data-transmission protocols are avoided by using an emerging technique named eXpress Data Path (XDP). Based on XDP, the transmission of 5.2 PB (i.e. 2.92 * 10^{12} packets) was achieved within 168 h without a single missing packet.
Tetrahedralization of a Hexahedral Mesh
Venue: arXiv cs.CG 2023
Year: 2023

Two important classes of three-dimensional elements in computational meshes are hexahedra and tetrahedra. While several efficient methods exist that convert a hexahedral element to a tetrahedral elements, the existing algorithm for tetrahedralization of a hexahedral complex is the marching tetrahedron algorithm which limits pre-selection of face divisions. We generalize a procedure for tetrahedralizing triangular prisms to tetrahedralizing cubes, and combine it with certain heuristics to design an algorithm that can triangulate any hexahedra.
A Survey of Recommender System Techniques and the Ecommerce Domain
Venue: arXiv cs.IR 2023
Year: 2023

In this big data era, it is hard for the current generation to find the right data from the huge amount of data contained within online platforms. In such a situation, there is a need for an information filtering system that might help them find the information they are looking for. In recent years, a research field has emerged known as recommender systems. Recommenders have become important as they have many real-life applications. This paper reviews the different techniques and developments of recommender systems in e-commerce, e-tourism, e-resources, e-government, e-learning, and e-library. By analyzing recent work on this topic, we will be able to provide a detailed overview of current developments and identify existing difficulties in recommendation systems. The final results give practitioners and researchers the necessary guidance and insights into the recommendation system and its application.