SoroushVahidi/scidocs huggingface dataset:
Primed for Violence : The Role of Gender Inequality in Predicting Internal Conflict
(2008)

We know, most notably through Ted Gurr’s research, that ethnic discrimination can lead to ethnopolitical rebellion–intrastate conflict. I seek to discover what impact, if any, gender inequality has on intrastate conflict. Although democratic peace scholars and others highlight the role of peaceful domestic behavior in predicting state behavior, many scholars have argued that a domestic environment of inequality and violenceFstructural and cultural violenceFresults in a greater likelihood of violence at the state and the international level. This project contributes to this line of inquiry and further tests the grievance theory of intrastate conflict by examining the norms of violence that facilitate a call to arms. And in many ways, I provide an alternative explanation for the significance of some of the typical economic measuresFthe greed theoryFbased on the link between discrimination, inequality, and violence. I test whether states characterized by higher levels of gender inequality are more likely to experience intrastate conflict. Ultimately, the basic link between gender inequality and intrastate conflict is confirmedFstates characterized by gender inequality are more likely to experience intrastate conflict, 1960–2001.
Sales Taxes and Internet Commerce∗
(2012)

We estimate the sensitivity of Internet retail purchasing to sales taxes using data from the eBay marketplace. Our first approach exploits the fact that seller locations are revealed only after buyers have expressed interest in an item by clicking on its listing. We use millions of location “surprises”to estimate price elasticities with respect to the charged sales tax. We then use aggregated data to estimate cross-state substitution parameters, and substitution between offl ine and online purchases, relying both on cross-sectional variation in state and local sales taxes, and on changes in these rates over time. We find substantial sensitivity to sales taxes. Using our item-level approach, we find an average price elasticity of around -2 for interested buyers. Using our aggregate approach, we find that a one percentage point increase in a state’s sales tax increases online purchases by state residents by just under two percent, but decreases their online purchases from home-state retailers by 3-4 percent. (JEL: D12, H20, H71, L81) ∗We are grateful to Glenn Ellison and many seminar participants for useful comments. We appreciate support from the Stanford Institute for Economic Policy Research and the Toulouse Network on Information Technology. Data access for this study was obtained under a consulting agreement between the Stanford authors (Einav, Knoepfle and Levin) and eBay Research. †Department of Economics, Stanford University, Stanford, CA 94305-6072 (Einav, Knoepfle, Levin), NBER (Einav and Levin), and eBay Labs (Sundaresan). Email: leinav@stanford.edu, knoepfle@stanford.edu, jdlevin@stanford.edu, nsundaresan@ebay.com.
Internet Web servers: workload characterization and performance implications
(1997)

This paper presents a workload characterization study for Internet Web servers. Six different data sets are used in the study: three from academic environments, two from scientific research organizations, and one from a commercial Internet provider. These data sets represent three different orders of magnitude in server activity, and two different orders of magnitude in time duration, ranging from one week of activity to one year. The workload characterization focuses on the document type distribution, the document size distribution, the document referencing behavior, and the geographic distribution of server requests. Throughout the study, emphasis is placed on finding workload characteristics that are common to all the data sets studied. Ten such characteristics are identified. The paper concludes with a discussion of caching and performance issues, using the observed workload characteristics to suggest performance enhancements that seem promising for Internet Web servers.
TIARA: a visual exploratory text analytic system
(2010)

In this paper, we present a novel exploratory visual analytic system called TIARA (Text Insight via Automated Responsive Analytics), which combines text analytics and interactive visualization to help users explore and analyze large collections of text. Given a collection of documents, TIARA first uses topic analysis techniques to summarize the documents into a set of topics, each of which is represented by a set of keywords. In addition to extracting topics, TIARA derives time-sensitive keywords to depict the content evolution of each topic over time. To help users understand the topic-based summarization results, TIARA employs several interactive text visualization techniques to explain the summarization results and seamlessly link such results to the original text. We have applied TIARA to several real-world applications, including email summarization and patient record analysis. To measure the effectiveness of TIARA, we have conducted several experiments. Our experimental results and initial user feedback suggest that TIARA is effective in aiding users in their exploratory text analytic tasks.
Dynamic topic models
(2006)

A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections. The approach is to use state space models on the natural parameters of the multinomial distributions that represent the topics. Variational approximations based on Kalman filters and nonparametric wavelet regression are developed to carry out approximate posterior inference over the latent topics. In addition to giving quantitative, predictive models of a sequential corpus, dynamic topic models provide a qualitative window into the contents of a large document collection. The models are demonstrated by analyzing the OCR'ed archives of the journal Science from 1880 through 2000.
EventRiver: Visually Exploring Text Collections with Temporal References
(2012)

Many text collections with temporal references, such as news corpora and weblogs, are generated to report and discuss real life events. Thus, event-related tasks, such as detecting real life events that drive the generation of the text documents, tracking event evolutions, and investigating reports and commentaries about events of interest, are important when exploring such text collections. To incorporate and leverage human efforts in conducting such tasks, we propose a novel visual analytics approach named EventRiver. EventRiver integrates event-based automated text analysis and visualization to reveal the events motivating the text generation and the long term stories they construct. On the visualization, users can interactively conduct tasks such as event browsing, tracking, association, and investigation. A working prototype of EventRiver has been implemented for exploring news corpora. A set of case studies, experiments, and a preliminary user test have been conducted to evaluate its effectiveness and efficiency.
Evolutionary hierarchical dirichlet processes for multiple correlated time-varying corpora
(2010)

Mining cluster evolution from multiple correlated time-varying text corpora is important in exploratory text analytics. In this paper, we propose an approach called evolutionary hierarchical Dirichlet processes (EvoHDP) to discover interesting cluster evolution patterns from such text data. We formulate the EvoHDP as a series of hierarchical Dirichlet processes~(HDP) by adding time dependencies to the adjacent epochs, and propose a cascaded Gibbs sampling scheme to infer the model. This approach can discover different evolving patterns of clusters, including emergence, disappearance, evolution within a corpus and across different corpora. Experiments over synthetic and real-world multiple correlated time-varying data sets illustrate the effectiveness of EvoHDP on discovering cluster evolution patterns.
The Word Tree, an Interactive Visual Concordance
(2008)

We introduce the Word Tree, a new visualization and information-retrieval technique aimed at text documents. A Word Tree is a graphical version of the traditional "keyword-in-context" method, and enables rapid querying and exploration of bodies of text. In this paper we describe the design of the technique, along with some of the technical issues that arise in its implementation. In addition, we discuss the results of several months of public deployment of word trees on Many Eyes, which provides a window onto the ways in which users obtain value from the visualization.
Total variation regularization of local-global optical flow
(2011)

More data fidelity terms in variational optical flow methods improve the estimation's robustness. A robust and anisotropic smoother enhances the specific fill-in process. This work presents a combined local-global (CLG) approach with total variation regularization. The combination of bilateral filtering and anisotropic (image driven) regularization is used to control the propagation phenomena. The resulted method, CLG-TV, is able to compute larger displacements in a reasonable time. The numerical scheme is highly parallelizable and runs in real-time on current generation graphic processing units.
Learning intermediate object affordances: Towards the development of a tool concept
(2014)

Inspired by the extraordinary ability of young infants to learn how to grasp and manipulate objects, many works in robotics have proposed developmental approaches to allow robots to learn the effects of their own motor actions on objects, i.e., the objects affordances. While holding an object, infants also promote its contact with other objects, resulting in object-object interactions that may afford effects not possible otherwise. Depending on the characteristics of both the held object (intermediate) and the acted object (primary), systematic outcomes may occur, leading to the emergence of a primitive concept of tool. In this paper we describe experiments with a humanoid robot exploring object-object interactions in a playground scenario and learning a probabilistic causal model of the effects of actions as functions of the characteristics of both objects. The model directly links the objects' 2D shape visual cues to the effects of actions. Because no object recognition skills are required, generalization to novel objects is possible by exploiting the correlations between the shape descriptors. We show experiments where an affordance model is learned in a simulated environment, and is then used on the real robotic platform, showing generalization abilities in effect prediction. We argue that, despite the fact that during exploration no concept of tool is given to the system, this very concept may emerge from the knowledge that intermediate objects lead to significant effects when acting on other objects.
Stereovision-Based Object Segmentation for Automotive Applications
(2005)

Obstacle detection and classification in a complex urban area are highly demanding, but desirable for pedestrian protection, stop & go, and enhanced parking aids. The most difficult task for the system is to segment objects from varied and complicated background. In this paper, a novel position-based object segmentationmethod has been proposed to solve this problem. According to the method proposed, object segmentation is performed in two steps: in depth map (X-Z plane) and in layered images (X-Y planes). The stereovision technique is used to reconstruct image points and generate the depth map. Objects are detected in the depth map. Afterwards, the original edge image is separated into different layers based on the distance of detected objects. Segmentation performed in these layered images can be easier and more reliable. It has been proved that the proposed method offers robust detection of potential obstacles and accurate measurement of their location and size.
DPIL@FIRE2016: Overview of the Shared task on Detecting Paraphrases in Indian language
(2016)

This paper explains the overview of the shared task "Detecting Paraphrases in Indian Languages" (DPIL) conducted at FIRE 2016. Given a pair of sentences in the same language, participants are asked to detect the semantic equivalence between the sentences. The shared task is proposed for four Indian languages namely Tamil, Malayalam, Hindi and Punjabi. The dataset created for the shared task has been made available online and it is the first open-source paraphrase detection corpora for Indian languages.
A comparison of trading algorithms based on machine learning classifiers : application on the S & P 500
(2017)

In this paper we analyse whether different machine learning classifiers can be useful when it comes to financial data prediction. We first measure the accuracy and the volatility of our algorithm on the stocks included in the S&P 500 stock index. We then back-test it against the same benchmark for two different periods. Overall, our results show that an algorithm that trades just according to the predictions of the classifiers, is not profitable even if the accuracy obtained by the algorithm is higher than the accuracy that should be obtained according to the random walk theory. However, it can boost the results obtained by other strategies, maintaining the same level of volatility. JEL classification: C4, C8, G14.
Visual Perception of Parallel Coordinate Visualizations
(2009)

Parallel coordinates is a visualization technique that provides an unbiased representation of high-dimensional data. The parallel configuration of axes treats data dimensions uniformly and is well suited for exploratory visualization. However, first-time users of parallel coordinate visualizations can find the representation confusing and difficult to understand.We used eye tracking to study how parallel coordinate visualizations are perceived, and compared the results to the optimal visual scan path required to complete the tasks. The results indicate that even first-time users quickly learn how to use parallel coordinate visualizations, pay attention to the correct task-specific areas in the visualization, and become rapidly proficient with it.
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
(2016)

Deep models are the defacto standard in visual decision problems due to their impressive performance on a wide array of visual tasks. However, they are frequently seen as opaque and are unable to explain their decisions. In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which supports their decisions. We propose a method which incorporates a novel explanation attention mechanism; our model is trained using textual rationals, and infers latent attention to visually ground explanations. We collect two novel datasets in domains where it is interesting and challenging to explain decisions. First, we extend the visual question answering task to not only provide an answer but also visual and natural language explanations for the answer. Second, we focus on explaining human activities in a contemporary activity recognition dataset. We extensively evaluate our model, both on the justification and pointing tasks, by comparing it to prior models and ablations using both automatic and human evaluations.
A chaincode based scheme for fingerprint feature extraction
(2006)

A feature extraction method using the chaincode representation of fingerprint ridge contours is presented for use by Automatic Fingerprint Identification Systems. The representation allows efficient image quality enhancement and detection of fine minutiae feature points. For image enhancement a given fingerprint image is first binarized after a quick averaging to generate its chaincode representation. The direction field is estimated from a set of selected chaincodes. The original gray scale image is then enhanced using connected component analysis and a dynamic filtering scheme that takes advantage of the knowledge gained from the estimated direction flow of the contours. For feature extraction, the enhanced fingerprint image is carefully binarized using a local-global binarization algorithm for generating the chaincode representation. The minutiae are generated using a sophisticated ridge contour following procedure. Visual inspection of the experiment images shows that the method is very effective.
Membrane Bioreactor (MBR) Technology for Wastewater Treatment and Reclamation: Membrane Fouling
(2016)

The membrane bioreactor (MBR) has emerged as an efficient compact technology for municipal and industrial wastewater treatment. The major drawback impeding wider application of MBRs is membrane fouling, which significantly reduces membrane performance and lifespan, resulting in a significant increase in maintenance and operating costs. Finding sustainable membrane fouling mitigation strategies in MBRs has been one of the main concerns over the last two decades. This paper provides an overview of membrane fouling and studies conducted to identify mitigating strategies for fouling in MBRs. Classes of foulants, including biofoulants, organic foulants and inorganic foulants, as well as factors influencing membrane fouling are outlined. Recent research attempts on fouling control, including addition of coagulants and adsorbents, combination of aerobic granulation with MBRs, introduction of granular materials with air scouring in the MBR tank, and quorum quenching are presented. The addition of coagulants and adsorbents shows a significant membrane fouling reduction, but further research is needed to establish optimum dosages of the various coagulants/adsorbents. Similarly, the integration of aerobic granulation with MBRs, which targets biofoulants and organic foulants, shows outstanding filtration performance and a significant reduction in fouling rate, as well as excellent nutrients removal. However, further research is needed on the enhancement of long-term granule integrity. Quorum quenching also offers a strong potential for fouling control, but pilot-scale testing is required to explore the feasibility of full-scale application.
BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
(2016)

We introduce BinaryNet, a method which trains DNNs with binary weights and activations when computing parameters’ gradient. We show that it is possible to train a Multi Layer Perceptron (MLP) on MNIST and ConvNets on CIFAR-10 and SVHN with BinaryNet and achieve nearly state-of-the-art results. At run-time, BinaryNet drastically reduces memory usage and replaces most multiplications by 1-bit exclusive-not-or (XNOR) operations, which might have a big impact on both general-purpose and dedicated Deep Learning hardware. We wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST MLP 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for BinaryNet is available.
Lagrangian Texture Advection: Preserving both Spectrum and Velocity Field
(2011)

Texturing an animated fluid is a useful way to augment the visual complexity of pictures without increasing the simulation time. But texturing flowing fluids is a complex issue, as it creates conflicting requirements: we want to keep the key texture properties (features, spectrum) while advecting the texture with the underlying flow-which distorts it. In this paper, we present a new, Lagrangian, method for advecting textures: the advected texture is computed only locally and follows the velocity field at each pixel. The texture retains its local properties, including its Fourier spectrum, even though it is accurately advected. Due to its Lagrangian nature, our algorithm can perform on very large, potentially infinite scenes in real time. Our experiments show that it is well suited for a wide range of input textures, including, but not limited to, noise textures.
Producing radiologist-quality reports for interpretable artificial intelligence
(2018)

Current approaches to explaining the decisions of deep learning systems for medical tasks have focused on visualising the elements that have contributed to each decision. We argue that such approaches are not enough to “open the black box” of medical decision making systems because they are missing a key component that has been used as a standard communication tool between doctors for centuries: language. We propose a model-agnostic interpretability method that involves training a simple recurrent neural network model to produce descriptive sentences to clarify the decision of deep learning classifiers. We test our method on the task of detecting hip fractures from frontal pelvic x-rays. This process requires minimal additional labelling despite producing text containing elements that the original deep learning classification model was not specifically trained to detect. The experimental results show that: 1) the sentences produced by our method consistently contain the desired information, 2) the generated sentences are preferred by doctors compared to current tools that create saliency maps, and 3) the combination of visualisations and generated text is better than either alone.
On Divergences and Informations in Statistics and Information Theory
(2006)

The paper deals with the f-divergences of Csiszar generalizing the discrimination information of Kullback, the total variation distance, the Hellinger divergence, and the Pearson divergence. All basic properties of f-divergences including relations to the decision errors are proved in a new manner replacing the classical Jensen inequality by a new generalized Taylor expansion of convex functions. Some new properties are proved too, e.g., relations to the statistical sufficiency and deficiency. The generalized Taylor expansion also shows very easily that all f-divergences are average statistical informations (differences between prior and posterior Bayes errors) mutually differing only in the weights imposed on various prior distributions. The statistical information introduced by De Groot and the classical information of Shannon are shown to be extremal cases corresponding to alpha=0 and alpha=1 in the class of the so-called Arimoto alpha-informations introduced in this paper for 0<alpha<1 by means of the Arimoto alpha-entropies. Some new examples of f-divergences are introduced as well, namely, the Shannon divergences and the Arimoto alpha-divergences leading for alphauarr1 to the Shannon divergences. Square roots of all these divergences are shown to be metrics satisfying the triangle inequality. The last section introduces statistical tests and estimators based on the minimal f-divergence with the empirical distribution achieved in the families of hypothetic distributions. For the Kullback divergence this leads to the classical likelihood ratio test and estimator
A column store engine for real-time streaming analytics
(2016)

This paper describes novel aspects of the column store implemented in the MemSQL database engine and describes the design choices made to support real-time streaming workloads. Column stores have traditionally been restricted to data warehouse scenarios where low latency queries are a secondary goal, and where restricting data ingestion to be offline, batched, append-only, or some combination thereof is acceptable. In contrast, the MemSQL column store implementation treats low latency queries and ongoing writes as first class citizens, with a focus on avoiding interference between read, ingest, update, and storage optimization workloads through the use of fragmented snapshot transactions and optimistic storage reordering. This implementation broadens the range of serviceable column store workloads to include those with more stringent demands on query and data latency, such as those backing operational systems used by adtech, financial services, fraud detection and other real-time or data streaming applications.