A computer implemented method for generating a similarity measure between a reference input and a target input, the reference input and the target input each comprising a sequence of tokens, the method comprising: receiving a reference profile for said reference input having eigenvalues, w.i, wherein the reference profile is obtained by diagonalizing a matrix A(j, k) derived from a Markov model indicating the probability that said reference input emits a token r.k, given a prior history tuple of tokens q.j, the prior history tuple comprising a predetermined number of tokens; receiving a transition matrix, b(p, q), based on the target input, wherein the transition matrix, b(p, q) is also derived from a Markov model and indicates the probability that the target input emits a token q, given a prior history tuple of tokens p, the prior history tuple comprising a predetermined number of tokens; generating a left eigenvector, u.i for each said w.i; generating a right eigenvector, v.i, for each said w.i; generating a complex conjugate of said u.i; and generating a similarity measure according to the formula: similarity measure = sum (i, ∥ u.i.conjugate * b * v.i ∥^2), wherein each term of said summation is said transition matrix premultiplied by said complex conjugate of i-th said left eigenvector, and postmultiplied by i-th said right eigenvector and wherein norm squared ∥.∥^2 is a square of magnitude of a complex number resulting from said i-th term in said sum; wherein said similarity measure is a measure for the similarity between the reference input and the target input; and wherein said tokens of both said reference input and said target input are selected from the group consisting of letters, groups of letters, words, phrases, sentences, paragraphs, sections, spaces, punctuation, one or more documents, XML, textual input, HTML, SGML, and sets of text. receiving a reference profile for said reference input having eigenvalues, w.i, wherein the reference profile is obtained by diagonalizing a matrix A(j, k) derived from a Markov model indicating the probability that said reference input emits a token r.k, given a prior history tuple of tokens q.j, the prior history tuple comprising a predetermined number of tokens; receiving a transition matrix, b(p, q), based on the target input, wherein the transition matrix, b(p, q) is also derived from a Markov model and indicates the probability that the target input emits a token q, given a prior history tuple of tokens p, the prior history tuple comprising a predetermined number of tokens; generating a left eigenvector, u.i for each said w.i; generating a right eigenvector, v.i, for each said w.i; generating a complex conjugate of said u.i; and generating a similarity measure according to the formula: similarity measure = sum (i, ∥ u.i.conjugate * b * v.i ∥^2), wherein each term of said summation is said transition matrix premultiplied by said complex conjugate of i-th said left eigenvector, and postmultiplied by i-th said right eigenvector and wherein norm squared ∥.∥^2 is a square of magnitude of a complex number resulting from said i-th term in said sum; wherein said similarity measure is a measure for the similarity between the reference input and the target input; and wherein said tokens of both said reference input and said target input are selected from the group consisting of letters, groups of letters, words, phrases, sentences, paragraphs, sections, spaces, punctuation, one or more documents, XML, textual input, HTML, SGML, and sets of text. similarity measure = sum (i, ∥ u.i.conjugate * b * v.i ∥^2), wherein each term of said summation is said transition matrix premultiplied by said complex conjugate of i-th said left eigenvector, and postmultiplied by i-th said right eigenvector and wherein norm squared ∥.∥^2 is a square of magnitude of a complex number resulting from said i-th term in said sum; wherein said similarity measure is a measure for the similarity between the reference input and the target input; and wherein said tokens of both said reference input and said target input are selected from the group consisting of letters, groups of letters, words, phrases, sentences, paragraphs, sections, spaces, punctuation, one or more documents, XML, textual input, HTML, SGML, and sets of text.