Patent ID: 8060512
Filing Date: 2011-11-15
Classification: G06F,G06N

Abstract:
1. A method for identifying clusters within data sets in a document processing system, the method comprising: receiving, from an electronic document storage system, a plurality of digital documents for which multi-dimensional probabilistic relationships are to be determined; parsing, with a computer processor, the plurality of digital documents to identify multi-dimensional count data within each of the plurality of digital documents, the multi-dimensional count data comprising at least three dimensions with each dimension comprising a respective data class comprising one of metadata associated with a respective document and text within a respective document; producing a data set comprising at least a three dimensional tensor representing the multi-dimensional count data; defining cluster definition matrices comprising estimated cluster membership probability of each element of each dimension of the multi-dimensional count data within the data set, the estimated cluster membership probability indicating a probability of membership of each element in a respective data cluster; setting the cluster definition matrices to current cluster definition matrices comprising random entry values for the data set; setting a pre-defined convergence criteria for iterative cluster definition refinement processing; setting the first tensor factorization model to any one of a NParafac Factorization Model and a NTucker3 Tensor Decomposition Model; setting the second tensor factorization models to a complementary model of the first tensor factorization model, the complementary model selected from one of a ParaAspect Factorization Model and a TuckAspect Tensor Decomposition Model, wherein the first tensor factorization model and the second factorization model each have respective objective functions that are equivalent to one another, and wherein the first tensor factorization model and the second factorization model each use respective optimization algorithms that are different from one another to solve their equivalent respective objective functions; iteratively processing the current cluster definition matrices with a sequence of iterations to produce a sequence of current cluster definition matrices, the iteratively processing being performed until the convergence criteria has been satisfied, the iteratively processing comprising: determining at least one likely cluster membership for any of the multi-dimensional count data within the data set based upon refinements made to the cluster definition matrices, a likely cluster membership comprising an indicator of membership of an element of the multi-dimensional count data in a respective cluster; and outputting the at least one likely cluster membership.