pinecone/core-2020-05-10-deduplication huggingface dataset:
A multi-way divergence metric for vector spaces

The majority of work in similarity search focuses on the efficiency of threshold and nearest-neighbour queries. Similarity join has been less well studied, although efficient indexing algorithms have been shown. The multi-way similarity join, extending similarity join to multiple spaces, has received relatively little treatment. Here we present a novel metric designed to assess some concept of a mutual similarity over multiple vectors, thus extending pairwise distance to a more general notion taken over a set of values. In outline, when considering a set of values X, our function gives a single numeric outcome D(X) rather than calculating some compound function over all of d(x, y) where x,y are elements of X. D(X) is strongly correlated with various compound functions, but costs only a little more than a single distance to evaluate. It is derived from an information-theoretic distance metric; it correlates strongly with this metric, and also with other metrics, in high-dimensional spaces. Although we are at an early stage in its investigation, we believe it could potentially be used to help construct more efficient indexes, or to construct indexes more efficiently. The contribution of this short paper is simply to identify the function, to show that it has useful semantic properties, and to show also that it is surprisingly cheap to evaluate. We expect uses of the function in the domain of similarity search to follow
Fuzzy equivalence relation based clustering and its use to restructuring websites' hyperlinks and web pages

Quality design of websites implies that among other factors, hypelinks’ structure should allow the users to reach the information they seek with the minimum number of clicks. This paper utilises the fuzzy equivalence relation based clustering in adapting website hyperlinks’ structure so that the redesigned website allows users to meet as effectively as possible their informational and navigational requirements. The fuzzy tolerance relation is calculated based on the usage rate of hyperlinks in a website. The equivalence relation identifies clusters of hyperlinks. The clusters are then used to realocate hyperlinks in webpages and to rearrange webpages into the website structure hierarchy
Log File Analysis with Context-Free Grammars

Classical ways of intrusion analysis from textual communication log files are either AI-based (such as by combinations of data mining with various techniques of machine learning), or they are based on regular expressions (such as the scanners implemented in the CISCO boxes). Whereas AI-based heuristics are not analytically exact, methods based on regular expressions do not reach very far in Chomsky's hierarchy of languages. In this short chapter we describe work in progress on the topic of parsing traces of network traffic with context-free grammars. "Green" grammars describe acceptable log files, whereas "red" grammars represent already known specific patterns of intrusion attempts. This technique can complement or augment the aready existing AI-approaches with additional precision. Analytically it is also more powerful than CISCO's technique on the basis of regular expressions.http://link.springer.com/chapter/10.1007/978-3-642-41148-9_10mv201
Free and open source software underpinning the european forest data centre

Excerpt:

Worldwide, governments are growingly focusing on free and open source software (FOSS) as a move toward transparency and the freedom to run, copy, study, change and improve the software. The European Commission (EC) is also supporting the development of FOSS [...]. In addition to the financial savings, FOSS contributes to scientific knowledge freedom in computational science (CS) and is increasingly rewarded in the science-policy interface within the emerging paradigm of open science. Since complex computational science applications may be affected by software uncertainty, FOSS may help to mitigate part of the impact of software errors by CS community- driven open review, correction and evolution of scientific code. The continental scale of EC science-based policy support implies wide networks of scientific collaboration. Thematic information systems also may benefit from this approach within reproducible integrated modelling. This is supported by the EC strategy on FOSS: "for the development of new information systems, where deployment is foreseen by parties outside of the EC infrastructure, [F]OSS will be the preferred choice and in any case used whenever possible". The aim of this contribution is to highlight how a continental scale information system may exploit and integrate FOSS technologies within the transdisciplinary research underpinning such a complex system. A European example is discussed where FOSS innervates both the structure of the information system itself and the inherent transdisciplinary research for modelling the data and information which constitute the system content. [...]
An Evaluation of Aggregation Techniques in Crowdsourcing

As the volumes of AI problems involving, human knowledge are likely to soar, crowdsourcing has become essential in a wide range of world-wide-web applications. One of the biggest challenges of crowdsourcing is aggregating the answers collected from the crowd since the workers might have wide-ranging levels of expertise. In order to tackle this challenge, many aggregation techniques have been proposed. These techniques, however, have never been compared and analyzed under the same setting, rendering a ‘right’ choice for a particular application very difficult. Addressing this problem, this paper presents a benchmark that offers a comprehensive empirical study on the performance comparison of the aggregation techniques. Specifically, we integrated several state-of-the-art methods in a comparable manner, and measured various performance metrics with our benchmark, including computation time, accuracy, robustness to spammers, and adaptivity to multi-labeling. We then provide in-depth analysis of benchmarking results, obtained by simulating the crowdsourcing process with different types of workers. We believe that the findings from the benchmark will be able to serve as a practical guideline for crowdsourcing applications.Full Tex
From Rational Number Reconstruction to Set Reconciliation and File Synchronization

This work revisits set reconciliation, the problem of synchronizing two multisets of fixed-size values while minimizing transmission complexity. We propose a new number-theoretic reconciliation protocol called Divide and Factor (D&F;) that achieves optimal asymptotic transmission complexity — as do previously known alternative algorithms. We analyze the computational complexities of various D&F; variants, study the problem of synchronizing sets of variable-size files using hash functions and apply D&F; to synchronize file hierarchies taking file locations into account.
We describe btrsync, our open-source D&F; implementation, and benchmark it against the popular software rsync. It appears that btrsync transmits much less data than rsync, at the expense of a relatively modest computational overhead
A type system for flexible role assignment in multiparty communicating systems

Communication protocols in distributed systems often specify the roles of the parties involved in the communications, namely for enforcing security policies or task assignment purposes. Ensuring that implementations follow role-based protocol specifications is challenging, especially in scenarios found, e.g., in business processes and web applications, where multiple peers are involved, single peers impersonate several roles, or single roles are carried out by several peers. We present a type-based analysis for statically verifying role-based multi-party interactions, based on a simple π-calculus model and prior work on conversation types. Our main result ensures that well-typed systems follow the role-based protocols prescribed by the types, including systems where roles are flexibly assigned to processes
Linking Unlinkability

International audienceUnlinkability is a privacy property of crucial importance for several systems (such as RFID or voting systems). Informally, unlinkability states that, given two events/items in a system, an attacker is not able to infer whether they are related to each other. However, in the literature we find several definitions for this notion, which are apparently unrelated and shows a potentially problematic lack of agreement. This paper sheds new light on unlinkability by comparing different ways of defining it and showing that in many practical situations the various definitions coincide. It does so by (a) expressing in a unifying framework four definitions of unlinkability from the literature (b) demonstrating how these definitions are different yet related to each other and to their dual notion of ''inseparability'' and (c) by identifying conditions under which all these definitions become equivalent. We argue that the conditions are reasonable to expect in identification systems, and we prove that they hold for a generic class of protocols
Evaluation of Statistical Features for Medical Image Retrieval

In this paper we present a complete system allowing the classification of medical images in order to detect possible diseases present in them. The proposed method is developed in two distinct stages: calculation of descriptors and their classification. In the first stage we compute a vector of thirty-three statistical features: seven are related to statistics\ud
of the first level order, fifteen to that of second level where thirteen are calculated by means of co-occurrence matrices and two with absolute gradient; the last thirteen finally are calculated using run-length matrices. In the second phase, using the descriptors already calculated, there is the actual image classification. Naive Bayes, RBF, Support VectorMa-\ud
chine, K-Nearest Neighbor, Random Forest and Random Tree classifiers are used. The results obtained from the proposed system show that the analysis carried out both on textured and on medical images lead to have a high accuracy
Hierarchical Image Representation Simplification Driven by Region Complexity

International audienceThis article presents a technique that arranges the elements of hierarchical representations of images according to a coarseness attribute. The choice of the attribute can be made according to prior knowledge about the content of the images and the intended application. The transformation is similar to filtering a hierarchy with a non-increasing attribute, and comprises the results of multiple simple filterings with an increasing attribute. The transformed hierarchy can be used for search space reduction prior to the image analysis process because it allows for direct access to the hierarchy elements at the same scale or a narrow range of scales
VSCAN: An Enhanced Video Summarization using Density-based Spatial
  Clustering

In this paper, we present VSCAN, a novel approach for generating static video
summaries. This approach is based on a modified DBSCAN clustering algorithm to
summarize the video content utilizing both color and texture features of the
video frames. The paper also introduces an enhanced evaluation method that
depends on color and texture features. Video Summaries generated by VSCAN are
compared with summaries generated by other approaches found in the literature
and those created by users. Experimental results indicate that the video
summaries generated by VSCAN have a higher quality than those generated by
other approaches.Comment: arXiv admin note: substantial text overlap with arXiv:1401.3590 by
  other authors without attributio
SVM-based classification of class C GPCRs from alignment-free physicochemical transformations of their sequences

G protein-coupled receptors (GPCRs) have a key function in regulating the function of cells due to their ability to transmit extracelullar signals. Given that the 3D structure and the functionality of most GPCRs is unknown, there is a need to construct robust classification models based on the analysis of their amino acid sequences for protein homology detection. In this paper, we describe the supervised classification of the different subtypes of class C GPCRs using support vector machines (SVMs). These models are built on different transformations of the amino acid sequences based on their physicochemical properties. Previous research using semi-supervised methods on the same data has shown the usefulness of such transformations. The obtained classification models show a robust performance, as their Matthews correlation coefficient is close to 0.91 and their prediction accuracy is close to 0.93.Postprint (author’s final draft
Toward an Integrated System for Surveillance\ud
and Behaviour Analysis of Groups and People

Security and INTelligence SYStem is an Italian research project\ud
which aims to create an integrated system for the analysis of multi-modal data\ud
sources (text, images, video, audio), to assist operators in homeland security\ud
applications. Within this project the Scientific Research Unit of the University\ud
of Palermo is responsible of the image and video analysis activity. The SRU of\ud
Palermo developed a web service based architecture that provides image and\ud
video analysis capabilities to the integrated analysis system. The developed architecture\ud
uses both state of the art techniques, adapted to cope with the particular\ud
problem at hand, and new algorithms to provide the following services: image\ud
cropping, image forgery detection, face and people detection, weapon detection\ud
and classification, and terrorist logo recognition. In the last phase of the\ud
project we plan to include in our system new services, mainly oriented to the\ud
video analysis, to study and understand the behaviour of individuals, either\ud
alone or in a group
Online Signature Verification: Improving Performance through Pre-classification Based on Global Features

In this paper, a pre-classification stage based on global features is incorporated to an online signature verification system for the purposes of improving its performance. The pre-classifier makes use of the discriminative power of some global features to discard (by declaring them as forgeries) those signatures for which the associated global feature is far away from its respective mean. For the remaining signatures, features based on a wavelet approximation of the time functions associated with the signing process, are extracted, and a Random Forest based classification is performed. The experimental results show that the proposed pre-classification approach, when based on the apppropriate global feature, is capable of getting error rate improvements with respect to the case where no pre-classification is performed. The approach also has the advantages of simplifying and speeding up the verification process.Fil: Parodi, Marianela. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico - CONICET - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y Sistemas; Argentina;Fil: Gomez, Juan Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico - CONICET - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y Sistemas; Argentina
Arrestins in host-pathogen interactions.

International audienceIn the context of host-pathogen interaction, host cell receptors and signaling pathways are essential for both invading pathogens, which exploit them for their own profit, and the defending organism, which activates early mechanism of defense, known as innate immunity, to block the aggression. Because of their central role as scaffolding proteins downstream of activated receptors, β-arrestins are involved in multiple signaling pathways activated in host cells by pathogens. Some of these pathways participate in the innate immunity and the inflammatory response. Other β-arrestin-dependent pathways are actually hijacked by microbes and toxins to penetrate into host cells and spread in the organism
Deadline analysis of AUTOSAR OS periodic tasks in the presence of interrupts

Lecture Notes in Computer Science 8144, 2013AUTOSAR, the open and emerging global standard for automotive embedded systems, oﬀers a timing protection mechanism to protect tasks from missing their deadlines. However, in practice, it is difficult to predict when a deadline is violated, because a task missing its deadline may be caused by unrelated tasks or by the presence of interrupts. In this paper, we propose an abstract formal model to rep- resent AUTOSAR OS programs with timing protection. We are able to determine schedulability properties and to calculate constraints on the allowed time that interrupts can take for a given task in a given period. We implement our model in Mathematica and give a case study to illus- trate the utility of our method. Based on the results, we believe that our work can help designers and implementors of AUTOSAR OS programs check whether their programs satisfy crucial timing properties.This work was partly supported by the Danish National Research Foundation and the National Natural Science Foundation of China (Grant No. 61061130541) for the Danish-Chinese Center for Cyber Physical Systems. And, it is also supported by National Basic Research Program of China (Grant No. 2011CB302904), National High Technology Research and Development Program of China (Grant Nos. 2011AA010101 and 2012AA011205).Themajority of the work reported in this paper was done while Yanhong Huang was a visiting researcher at Teesside University (funded by the Teesside University Research Fund). The support of Teesside University is gratefully acknowledged
Automated Specification Discovery via User-Defined Predicates

Automated discovery of specifications for heap-manipulating programs is a challenging task due to the complexity of aliasing and mutability of data structures. This task is further complicated by an expressive domain that combines shape, numerical and bag information. In this paper, we propose a compositional analysis framework in the presence of user-defined predicates, which would derive the summary for each method in the expressive abstract domain, independently from its callers. We propose a novel abstraction method with a bi-abduction technique in the combined domain to discover pre-/post-conditions that could not be automatically inferred before. The analysis does not only prove the memory safety properties, but also finds relationships between pure and shape domains towards full functional correctness of programs. A prototype of the framework has been implemented and initial experiments have shown that our approach can discover interesting properties for non-trivial programs
OWA-FRPS : a prototype selection method based on ordered weighted average fuzzy rough set theory

The Nearest Neighbor (NN) algorithm is a well-known and effective classification algorithm. Prototype Selection (PS), which provides NN with a good training set to pick its neighbors from, is an important topic as NN is highly susceptible to noisy data. Accurate state-of-the-art PS methods are generally slow, which motivates us to propose a new PS method, called OWA-FRPS. Based on the Ordered Weighted Average (OWA) fuzzy rough set model, we express the quality of instances, and use a wrapper approach to decide which instances to select. An experimental evaluation shows that OWA-FRPS is significantly more accurate than state-of-the-art PS methods without requiring a high computational cost
Rough Set-Based Information Dilution by Non-deterministic Information

We have investigated rough set-based concepts for a given Non-deterministic Information System (NIS). In this paper, we consider generating a NIS from a Deterministic Information System (DIS) intentionally. A NIS Φ is seen as a diluted DIS ϕ, and we can hide the actual values in ϕ by using Φ. We name this way of hiding Information Dilution by non-deterministic information. This paper considers information dilution and its application to hiding the actual values in a table.Rough sets, fuzzy sets, data mining, and granular computing : 14th international conference, RSFDGrC 2013 : October 11-14, 2013,  Halifax, NS, Canad
Recommending tripleset interlinking through a social network approach

Tripleset interlinking is one of the main principles of Linked Data. However, the discovery of existing triplesets relevant to be linked with a new tripleset is a non-trivial task in the publishing process. Without prior knowledge about the entire Web of Data, a data publisher must perform an exploratory search, which demands substantial effort and may become impracticable, with the growth and dissemination of Linked Data. Aiming at alleviating this problem, this paper proposes a recommendation approach for this scenario, using a Social Network perspective. The experimental results show that the proposed approach obtains high levels of recall and reduces in up to 90% the number of triplesets to be further inspected for establishing appropriate links. The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-642-41230-1_13.CNPq/160326/2012-5CNPq/301497/2006-0CNPq/475717/2011-2CNPq/57128/2009-9FAPERJ/E-26/170028/2008FAPERJ/E-26/103.070/2011CAPES/PROCAD/NF 1128/201
Verifying transactional requirements of web service compositions using temporal logic templates

Lecture notes in computer science, 2013, vol. 8180 LNCS (Part 1)Ensuring reliability in Web service compositions is of crucial interest as services are composed and executed in long-running, distributed mediums that cannot guarantee reliable communications. Towards this, transactional behavior has been proposed to handle and undo the effects of faults of individual components. Despite significant research interest, challenges remain in providing an easy-to-use, formal approach to verify transactional behavior of Web service compositions before costly development. In this paper, we propose the use of temporal logic templates to specify component-level and composition-level transactional requirements over a Web service composition. These templates are specified using a simple format, configured according to scope and cardinality, and automatically translated into temporal logic. To verify design conformance to a set of implemented templates, we employ model checking. We propose an algorithm to address state space explosion by reducing the models into semantically equivalent Kripke structures. Our approach facilitates the implementation of expressive transactional behavior onto existing complex services, as demonstrated in our experimental study.Scott Bourne, Claudia Szabo, and Quan Z. Shen
A novel method for finding similarities between unordered trees using matrix data model

Trees are capable of portraying the semi-structured data which is common in web domain. Finding similarities between trees is mandatory for several applications that deal with semi-structured data. Existing similarity methods examine a pair of trees by comparing through nodes and paths of two trees, and find the similarity between them. However, these methods provide unfavorable results for unordered tree data and result in yielding NP-hard or MAX-SNP hard complexity. In this paper, we present a novel method that encodes a tree with an optimal traversing approach first, and then, utilizes it to model the tree with its equivalent matrix representation for finding similarity between unordered trees efficiently. Empirical analysis shows that the proposed method is able to achieve high accuracy even on the large data sets