OpenRaiser/Intern-Atlas huggingface dataset:
Going where, by whom, and at what time: Next location prediction considering user preference and temporal regularity
Authors: Tianao Sun, Ke Fu, Weiming Huang 0001, Kai Zhao 0011, Yongshun Gong, Meng Chen 0003
Venue: Conference KDD 2024
Year: 2024

Next location prediction is a crucial task in human mobility modeling, and is pivotal for many downstream applications like location-based recommendation and transportation planning. Although there has been a large body of research tackling this problem, the usefulness of user preference and temporal regularity remains underrepresented. Specifically, previous studies usually neglect the explicit user preference information entailed from human trajectories and fall short in utilizing the arrival time of next location, as a key determinant on next location. To address these limitations, we propose a Multi-Context aware Location Prediction model (MCLP) to predict next locations for individuals, where it explicitly models user preference and the next arrival time as context. First, we utilize a topic model to extract user preferences for different types of locations from historical human trajectories. Second, we develop an arrival time estimator to construct a robust arrival time embedding based on the multi-head attention mechanism. The two components provide pivotal contextual information for the subsequent prediction. Finally, we utilize the Transformer architecture to mine sequential patterns and integrate multiple contextual information to predict the next locations. Experimental results on two real-world mobility datasets show that our proposed MCLP outperforms baseline methods.
Orcdf: An oversmoothing-resistant cognitive diagnosis framework for student learning in online education systems
Authors: Hong Qian, Shuo Liu 0017, Mingjia Li 0002, Bingdong Li, Zhi Liu, Aimin Zhou
Venue: Conference KDD 2024
Year: 2024

Cognitive diagnosis models (CDMs) are designed to learn students' mastery levels using their response logs. CDMs play a fundamental role in online education systems since they significantly influence downstream applications such as teachers' guidance and computerized adaptive testing. Despite the success achieved by existing CDMs, we find that they suffer from a thorny issue that the learned students' mastery levels are too similar. This issue, which we refer to as oversmoothing, could diminish the CDMs' effectiveness in downstream tasks. CDMs comprise two core parts: learning students' mastery levels and assessing mastery levels by fitting the response logs. This paper contends that the oversmoothing issue arises from that existing CDMs seldom utilize response signals on exercises in the learning part but only use them as labels in the assessing part. To this end, this paper proposes an oversmoothing-resistant cognitive diagnosis framework (ORCDF) to enhance existing CDMs by utilizing response signals in the learning part. Specifically, ORCDF introduces a novel response graph to inherently incorporate response signals as types of edges. Then, ORCDF designs a tailored response-aware graph convolution network (RGC) that effectively captures the crucial response signals within the response graph. Via ORCDF, existing CDMs are enhanced by replacing the input embeddings with the outcome of RGC, allowing for the consideration of response signals on exercises in the learning part. Extensive experiments on real-world datasets show that ORCDF not only helps existing CDMs alleviate the oversmoothing issue but also significantly enhances the models' prediction and interpretability performance. Moreover, the effectiveness of ORCDF is validated in the downstream task of computerized adaptive testing.
Be3r: Bert based early-exit using expert routing
Authors: Sourab Mangrulkar, Ankith M. S, Vivek Sembium
Venue: Conference KDD 2022
Year: 2022

Pre-trained language models like BERT have reported state-of-the-art performance on several Natural Language Processing (NLP) tasks, but high computational demands hinder its widespread adoption for large scale NLP tasks. In this work, we propose a novel routing based early exit model called BE3R (BERT based Early-Exit using Expert Routing), where we learn to dynamically exit in the earlier layers without needing to traverse through the entire model. Unlike the exiting early-exit methods, our approach can be extended to a batch inference setting. We consider the specific application of search relevance filtering in Amazon India marketplace services (a large e-commerce website). Our experimental results show that BE3R improves the batch inference throughput by 46.5% over the BERT-Base model and 35.89% over the DistilBERT-Base model on large dataset with 50 Million samples without any trade-off on the performance metric. We conduct thorough experimentation using various architectural choices, loss functions and perform qualitative analysis. We perform experiments on public GLUE Benchmark and demonstrate comparable performance to corresponding baseline models with 23% average throughput improvement across tasks in batch inference setting.
Finding prerequisite relations using the wikipedia clickstream
Authors: Mohsen Sayyadiharikandeh, Jonathan Gordon, José Luis Ambite, Kristina Lerman
Venue: Conference WWW 2019
Year: 2019

The increased availability of online learning resources in the form of courses, videos, and tutorials has created new opportunities for independent learners, but it has also increased the difficulty of planning a course of study. Where should the learner start? What should the learner know before tackling a new course? Manually identifying these prerequisite relations between learning resources or concepts is expensive in terms of time and expertise, and it is particularly difficult to do so for new or rapidly changing areas of knowledge. To address this challenge, we present a new method for identifying prerequisite relations based on naturally occurring data, namely the navigation patterns of users on the Wikipedia online encyclopedia. Our supervised learning approach shows that the navigation network structure can be used to identify dependencies among concepts in several domains.
A learning-path based supervised method for concept prerequisite relations extraction in educational data
Authors: Jingwen Sun, Yu He, Yiyu Xu, Jingwei Sun, Guangzhong Sun
Venue: Conference CIKM 2024
Year: 2024

In educational data mining, concept prerequisite relations extraction determines which concepts need to be learned before learning another concept. It plays a crucial role in pedagogical practices, such as learning path planning and curriculum design. Deep neural networks, especially graph neural networks, have recently made significant strides in concept prerequisite relations extraction. However, existing methods face two primary limitations. (1) Methods with better performance construct heterogeneous complete graphs, leading to higher model complexity and training cost. Meanwhile, the performance of low-complexity methods is inferior to the former. (2) A disregard for temporal context, essential for learning, limits both the performance and the application of these methods. To address these issues, we propose a novel graph-based approach, called Learning-path based Concept Prerequisite Relations Extraction (LCPRE). LCPRE constructs a lightweight sparse graph in a simple manner, which reduces complexity from quadratic to linear and captures the temporal feature through learning-path, a comprehensible learning approach from one concept to another. Experimental results on three benchmark datasets demonstrate that LCPRE outperforms existing methods, establishing a new state-of-the-art in concept prerequisite relations extraction.
Prerequisite concept maps extraction for automaticassessment
Authors: Shuting Wang, Lei Liu
Venue: Conference WWW 2016
Year: 2016

Traditional assessment modes usually give identical set of questions to each student, thus are inefficient for students to fix their problems. In order to perform an efficient assessment, we utilize prerequisite concept maps to find students' learning gaps and work on closing these gaps and proposed a two-phase model for concept map construction. Experiments on concept pairs with prerequisite relationships which are manually created show the promise of our proposed method. In order to meet the challenge of using concept maps in automatic assessments, we also derive a top-k concept selection algorithm which allows students to view different numbers of concepts.
Improving text categorization methods for event tracking
Authors: Yiming Yang, Tom Ault, Thomas Pierce, Charles W. Lattimer
Venue: Conference SIGIR 2000
Year: 2000

Automated tracking of events from chronologically ordered document streams is a new challenge for statistical text classification. Existing learning techniques must be adapted or improved in order to effectively handle difficult situations where the number of positive training instances per event is extremely small, the majority of training documents are unlabelled, and most of the events have a short duration in time. We adapted several supervised text categorization methods, specifically several new variants of the k-Nearest Neighbor (kNN) algorithm and a Rocchio approach, to track events. All of these methods showed significant improvement (up to 71% reduction in weighted error rates) over the performance of the original kNN algorithm on TDT benchmark collections, making kNN among the top-performing systems in the recent TDT3 official evaluation. Furthermore, by combining these methods, we significantly reduced the variance in performance of our event tracking system over different data collections, suggesting a robust solution for parameter optimization.
A lattice-based approach to query-by-example spoken document retrieval
Authors: Tee Kiah Chia, Khe Chai Sim, Haizhou Li, Hwee Tou Ng
Venue: Conference SIGIR 2008
Year: 2008

Recent efforts on the task of spoken document retrieval (SDR) have made use of speech lattices: speech lattices contain information about alternative speech transcription hypotheses other than the 1-best transcripts, and this information can improve retrieval accuracy by overcoming recognition errors present in the 1-best transcription. In this paper, we look at using lattices for the query-by-example spoken document retrieval task - retrieving documents from a speech corpus, where the queries are themselves in the form of complete spoken documents (query exemplars). We extend a previously proposed method for SDR with short queries to the query-by-example task. Specifically, we use a retrieval method based on statistical modeling: we compute expected word counts from document and query lattices, estimate statistical models from these counts, and compute relevance scores as divergences between these models. Experimental results on a speech corpus of conversational English show that the use of statistics from lattices for both documents and query exemplars results in better retrieval accuracy than using only 1-best transcripts for either documents, or queries, or both. In addition, we investigate the effect of stop word removal which further improves retrieval accuracy. To our knowledge, our work is the first to have used a lattice-based approach to query-by-example spoken document retrieval.
Model-based motion clustering using boosted mixture modeling
Authors: Vladimir Pavlovic 0001
Venue: Conference CVPR 2004
Year: 2004

Model-based clustering of motion trajectories can be posed as the problem of learning an underlying mixture density function whose components correspond to motion classes with different statistical properties. We propose a general framework for boosted modeling of mixtures of parametric densities. A density is represented as a parametric mixture of kernels where mixture components are now being added recursively, one at a time, until a best fit to data occurs and an optimal number of mixture components is selected. Optimal ML and MAP solutions to this problem are found using the functional gradient techniques. Unlike traditional mixture modeling techniques, the new method does not rely on random parameter initialization and exhaustive exploration of varying model orders (such as the number of mixture components.) The method justifies parameter estimation of new mixture components independently of that of the rest of the mixture, thus allowing tractable use of complex kernels such as HMMs or switching linear dynamic models. The relationship to traditional parametric EM-based mixture modeling algorithms is established We demonstrate the utility of the new algorithm on the problem of discovering motion sequence clusters. Our generative modeling framework has an important advantage over nonparametric approaches in that it can be used for classification as well as synthesis of the learned motion categories.
Robust boosting and its relation to bagging
Authors: Saharon Rosset
Venue: Conference KDD 2005
Year: 2005

Several authors have suggested viewing boosting as a gradient descent search for a good fit in function space. At each iteration observations are re-weighted using the gradient of the underlying loss function. We present an approach of weight decay for observation weights which is equivalent to "robustifying" the underlying loss function. At the extreme end of decay this approach converges to Bagging, which can be viewed as boosting with a linear underlying loss function. We illustrate the practical usefulness of weight decay for improving prediction performance and present an equivalence between one form of weight decay and "Huberizing" --- a statistical method for making loss functions more robust.
VoisTV: Voice-Enabled Social TV
Authors: Bernard Renger, Junlan Feng, Ovidiu Dan, Harry Chang, Luciano Barbosa
Venue: Conference WWW 2011
Year: 2011

Until recently, the TV viewing experience has not been a very social activity compared to activities on the World Wide Web. In this work, we will present a Voice-enabled Social TV system (VoiSTV) which allows users to interact, follow and monitor the online social media messages related to a TV show while watching it. Users can create, send, and reply to messages using spoken language. VoiSTV also provides metadata information about TV shows such as trends, hot topics, popularity as well as aggregated sentiment of show-related messages, all of which are valuable for TV program search and recommendation.
Quasi-Succinct Indices
Authors: Sebastiano Vigna
Venue: Conference WSDM 2013
Year: 2013

Compressed inverted indices in use today are based on the idea of gap compression: documents pointers are stored in increasing order, and the gaps between successive document pointers are stored using suitable codes which represent smaller gaps using less bits. Additional data such as counts and positions is stored using similar techniques. A large body of research has been built in the last 30 years around gap compression, including theoretical modeling of the gap distribution, specialized instantaneous codes suitable for gap encoding, and ad hoc document reorderings which increase the efficiency of instantaneous codes. This paper proposes to represent an index using a different architecture based on quasi-succinct representation of monotone sequences. We show that, besides being theoretically elegant and simple, the new index provides expected constant-time operations, space savings, and, in practice, significant performance improvements on conjunctive, phrasal and proximity queries.
Alexa, can you help me shop?
Authors: Yoelle Maarek
Venue: Conference SIGIR 2019
Year: 2019

In a previous keynote address [1], we have described how voice-enabled intelligent assistants are revolutionizing the way humans interact with machines and how their ubiquitous presence in homes, cars, offices are taking us closer to the old dream of ambient computing. We also explained how Alexa, Amazon's intelligent assistant, is pioneering the domain of voice shopping, considering the customer's shopping journey in a holistic manner. In this talk, we will discuss in more details the key stages required in addressing customer's needs, namely (1) understanding customers, (2) satisfying their needs, and (3) predicting them. We will also provide a finer analysis of customers' needs, be they informational, transactional or navigational needs [2]. We will conclude by presenting some associated research challenges and encouraging the community to explore the unchartered area of voice-enabled product discovery.
Decomposition Theory and Transformations of Visual Directions
Authors: Fredrik Bergholm
Venue: Conference ICCV 1990
Year: 1990

The author shows that there are two basic ways to eliminate rotation; either you eliminate actual (effective) rotation R or abstract rotation R. The transformation of visual directions P is decomposed into pure deformation (pure strain) and abstract rotation. Methods can be grouped as to whether they eliminate actual or abstract rotation. An interesting elimination of R can be done for equatorial motion fields on the image hemisphere, using equator-normal flow, having a moving sensor in a fixed environment. Then rotation can be eliminated by detecting the phase and amplitude of a sine wave hidden in a 1-D signal (the signal is a function of longitudes). Both sparse velocities and normal flow for nearly equator-parallel image contours can be used (directly) as input data. This approach seems robust and can be used in conjunction with the algorithm by R.C. Nelson and J. Aloimonos (1988).< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Using Planar Parallax to Estimate the Time-to-Contact
Authors: Manolis I. A. Lourakis, Stelios C. Orphanoudakis
Venue: Conference CVPR 1999
Year: 1999

To avoid collisions, mobile robots need a measure for assessing the proximity of obstacles. A measure that is well suited to this purpose is provided by the time-to-contact. In this paper, a novel method for estimating the time-to-contact is presented. The method is based on the assumption that the robot is moving on a locally planar ground, parts of which are visible in the acquired images. Using the optical flow field induced by the robot's motion, the time-to-contact with points on the ground is first estimated and then the concept of planar parallax is exploited to recover the time-to-contact with obstacle points. The computation of high order derivatives of the image flow, which are known to be very difficult to estimate accurately, is completely avoided. Moreover, no knowledge of the egomotion is necessary. Experimental results from the application of the method on real and simulated flow fields are also reported.
Multilinear independent components analysis
Authors: M. Alex O. Vasilescu, Demetri Terzopoulos
Venue: Conference CVPR 2005
Year: 2005

Independent components analysis (ICA) maximizes the statistical independence of the representational components of a training image ensemble, but it cannot distinguish between the different factors, or modes, inherent to image formation, including scene structure, illumination, and imaging. We introduce a nonlinear, multifactor model that generalizes ICA. Our multilinear ICA (MICA) model of image ensembles learns the statistically independent components of multiple factors. Whereas ICA employs linear (matrix) algebra, MICA exploits multilinear (tensor) algebra. We furthermore introduce a multilinear projection algorithm which projects an unlabeled test image into the N constituent mode spaces to simultaneously infer its mode labels. In the context of facial image ensembles, where the mode labels are person, viewpoint, illumination, expression, etc., we demonstrate that the statistical regularities learned by MICA capture information that, in conjunction with our multilinear projection algorithm, improves automatic face recognition.
Remote-sensing issues for intelligent underwater systems
Authors: W. Kenneth Stewart
Venue: Conference CVPR 1991
Year: 1991

For certain applications an autonomous underwater system may benefit from a perception of its environment in terms of magnetics, gravity, chemical signature, or other sensing modalities. An introduction is presented for researchers unfamiliar with issues in underwater remote sensing. Application areas include machine perception for intelligent underwater systems, large-scale seafloor mapping, survey, and environmental characterization.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Qualitative image based localization in indoors environments
Authors: Jana Kosecká, Liang Zhou, Philip Barber, Zoran Duric
Venue: Conference CVPR 2003
Year: 2003

Man made indoor environments possess regularities, which can be efficiently exploited in automated model acquisition by means of visual sensing. In this context we propose an approach for inferring a topological model of an environment from images or the video stream captured by a mobile robot during exploration. The proposed model consists of a set of locations and neighborhood relationships between them. Initially each location in the model is represented by a collection of similar, temporally adjacent views, with the similarity defined according to a simple appearance based distance measure. The sparser representation is obtained in a subsequent learning stage by means of learning vector quantization (LVQ). The quality of the model is tested in the context of qualitative localization scheme by means of location recognition: given a new view, the most likely location where that view came from is determined.
Class-specific material categorisation
Authors: Barbara Caputo, Eric Hayman, P. Mallikarjuna
Venue: Conference ICCV 2005
Year: 2005

Although a considerable amount of work has been published on material classification, relatively little of it studies situations with considerable variation within each class. Many experiments use the exact same sample, or different patches from the same image, for training and test sets. Thus, such studies are vulnerable to effectively recognising one particular sample of a material as opposed to the material category. In contrast, this paper places firm emphasis on the capability to generalise to previously unseen instances of materials. We adopt an appearance-based strategy, and conduct experiments on a new database which contains several samples of each of eleven material categories, imaged under a variety of pose, illumination and scale conditions. Together, these sources of intra-class variation provide a stern challenge indeed for recognition. Somewhat surprisingly, the difference in performance between various state-of-the-art texture descriptors proves rather small in this task. On the other hand, we clearly demonstrate that very significant gains can be achieved via different SVM-based classification techniques. Selecting appropriate kernel parameters proves crucial. This motivates a novel recognition scheme based on a decision tree. Each node contains an SVM to split one class from all others with a kernel parameter optimal for that particular node. Hence, each decision is made using a different, optimal, class-specific metric. Experiments show the superiority of this approach over several state-of-the-art classifiers
Evaluation of stereo confidence indoors and outdoors
Authors: Xiaoyan Hu, Philippos Mordohai
Venue: Conference CVPR 2010
Year: 2010

We present an extensive evaluation of 13 confidence metrics for stereo matching that compares the most widely used metrics as well as four novel techniques proposed here. We begin by categorizing the methods according to which aspects of stereo computation they take into account and, then, assess their strengths and weaknesses. The evaluation is conducted on indoor and outdoor datasets with ground truth and measures the capability of each confidence metric to rank depth estimates according to their likelihood for being correct, to detect occluded pixels and to generate low-error depth maps by selecting among multiple hypotheses for each pixel. We believe that such an evaluation is missing from the rapidly maturing stereo literature and that our findings will be helpful to researchers in binocular and multi-view stereo.