Source: http://www.google.de/patents/US8738635
Timestamp: 2018-01-19 13:40:20
Document Index: 769645786

Matched Legal Cases: ['Application No. 2008', 'Application No. 200510088212', 'Application No. 200510088212', 'Application No. 2005', 'Application No. 2005', 'art5', 'art 4', 'Application No. 566532', 'Application No. 2008105758', 'Application No. 2008110731', 'Application No. 2010141559', 'Application No. 2008']

Patent US8738635 - Detection of junk in search result ranking - Google Patentsuche
Embodiments are directed to ranking search results using a junk profile. For a given corpus of documents, one or more junk profiles may be created and maintained. The junk profile provides reference metrics to represent known junk documents. For example, a junk profile may comprise a dictionary of document...http://www.google.de/patents/US8738635?utm_source=gb-gplus-sharePatent US8738635 - Detection of junk in search result ranking
Veröffentlichungsnummer US8738635 B2
Anmeldenummer US 12/791,756
Eingetragen 1. Juni 2010
Auch veröffentlicht unter CN102918532A, CN102918532B, EP2577521A2, EP2577521A4, US20110295850, WO2011152925A2, WO2011152925A3
Veröffentlichungsnummer 12791756, 791756, US 8738635 B2, US 8738635B2, US-B2-8738635, US8738635 B2, US8738635B2
Erfinder Vladimir Tankovich, Dmitriy Meyerzon, Victor Poznanski
Patentzitate (383), Nichtpatentzitate (452), Klassifizierungen (13), Juristische Ereignisse (2)
US 8738635 B2
Embodiments are directed to ranking search results using a junk profile. For a given corpus of documents, one or more junk profiles may be created and maintained. The junk profile provides reference metrics to represent known junk documents. For example, a junk profile may comprise a dictionary of document data that is automatically inserted into documents created using a particular system or template. A junk profile may also comprise one or more representations (e.g., histograms) of a distribution of a particular junk variable for known junk documents. The junk profile provides a usable representation of known junk documents, and the present systems and methods employ the junk profile to predict the likelihood that documents in the corpus are junk. In embodiments, junk scores are calculated and used to rank such documents higher or lower in response to a search query.
1. A computer-implemented method for ranking candidate documents in response to a search query, comprising steps of:
creating, by at least a first processor, an index of a plurality of documents in a corpus;
calculating a junk score for at least a first document in the corpus, wherein calculating the junk score comprises:
using a first candidate histogram for the first document in the corpus, wherein the first candidate histogram is specific to the first document; and
using a junk profile, wherein the junk profile comprises:
a first reference histogram for a first known junk document, wherein the first reference histogram is specific to the first known junk document and is based on a first junk variable; and
comparing the first candidate histogram to the first reference histogram;
identifying, based on the search query and the index, candidate documents from the plurality of documents in the corpus, wherein the candidate documents include at least the first document;
ranking the candidate documents.
2. The computer-implemented method of claim 1, wherein ranking the candidate documents comprises ranking the candidate documents based at least in part on the junk score for the first document, and wherein the ranking of the first document is decreased where the first document is more similar to the first known junk document.
wherein calculating the junk score further comprises determining a first similarity metric.
4. The computer-implemented method of claim 3, wherein the junk profile comprises a second reference histogram for the first junk variable of a second known junk document, and wherein calculating the junk score comprises comparing the candidate histogram to the second reference histogram to determine a second similarity metric.
5. The computer-implemented method of claim 4, wherein calculating the junk score comprises at least one of: calculating a maximum of the first and second similarity metrics and calculating an average of the first and second similarity metrics.
6. The computer-implemented method of claim 1, further comprising the step of displaying the ranked candidate documents and displaying a junk status for at least the first document.
7. The computer-implemented method of claim 1, wherein the first junk variable comprises chunk size.
8. The computer-implemented method of claim 1, wherein: the junk profile comprises a dictionary of automatically generated data, and wherein creating the index comprises ignoring document data that matches the automatically generated data.
the junk profile comprises a dictionary of automatically generated data;
calculating the junk score further comprises comparing document data from the plurality of documents in the corpus to the dictionary of automatically generated data; and
creating the index comprises delineating in the index document data that matches the automatically generated data.
10. The computer-implemented method of claim 9, wherein identifying the candidate documents includes comparing the search query to document data in the index, and wherein ranking the candidate documents includes determining whether document data matching the search query has been delineated as matching the automatically generated data.
11. The computer-implemented method of claim 9, wherein calculating the junk score for the first document comprises determining a similarity metric between document data in the first document and the automatically generated data.
12. The computer-implemented of claim 9, further comprising:
creating the junk profile, comprising creating the dictionary of automatically generated data by:
creating a blank template containing automatically generated data; and
extracting the automatically generated data from the blank template.
13. The computer-implemented method of claim 1, wherein the step of calculating includes calculating a junk score for a second document in the corpus and wherein the step of identifying comprises excluding the second document from the candidate documents when the junk score for the second document exceeds a predetermined threshold.
14. The computer-implemented method of claim 1, wherein the step of calculating occurs after the step of identifying and wherein the step of calculating comprises calculating a junk score for a plurality of the candidate documents.
15. The computer-implemented method of claim 1, wherein the corpus is an intranet, the plurality of documents is created using a particular template, and the junk profile is specific to the particular template.
16. The computer-implemented method of claim 1, wherein the search query comprises a query for documents in the corpus that have a junk score that exceeds a predetermined threshold.
17. The computer-implemented method of claim 1, wherein the junk score for the first document exceeds a predetermined threshold, further comprising:
sending to an administrator a message identifying the first document as junk.
18. A system for ranking candidate documents in response to a search query, comprising:
a memory, operatively connected to the at least one processor and containing instructions that, when executed by the at least one processor, perform a method comprising:
creating an index of a plurality of documents in a corpus;
ranking the candidate documents based at least in part on the junk score for the first document;
wherein creating the index comprises separately delineating document data from the plurality of documents if the document data matches the junk profile.
creating, for at least the first document, a candidate histogram for at least a first junk variable;
wherein calculating the junk score comprises comparing the candidate histogram to the first reference histogram to determine a first similarity metric;
wherein the junk profile comprises a dictionary of automatically generated data;
wherein calculating the junk score further comprises comparing document data from the plurality of documents in the corpus to the dictionary of automatically generated data; and
wherein creating the index comprises delineating in the index document data that matches the automatically generated data.
20. A computer storage medium including computer-executable instructions that, when executed by at least one processor, perform a method comprising:
creating, for at least a first document of the plurality of documents, a candidate histogram specific to the first document for at least a first junk variable;
calculating a junk score for at least the first document using a junk profile, wherein:
the junk profile comprises:
a first reference histogram for a first known junk document, wherein the first reference histogram is specific to at least the first known junk document and is based on the first junk variable, and
a dictionary of automatically generated data; and
calculating a junk score comprises at least (a) comparing the candidate histogram to the first reference histogram to determine a first similarity metric and (b) determining a second similarity metric between document data in the first document and the dictionary of automatically generated data;
ranking the candidate documents based at least in part on the junk score for the first document.
This application is related to co-pending U.S. patent application Ser. No. 11/874,579, filed Oct. 18, 2007, entitled “Ranking and Providing Search Results Based in Part on a Number of Click Through Parameters,” and U.S. patent application Ser. No. 11/874,844, “now U.S. Pat. No. 7,840,569,” filed Oct. 18, 2007, entitled “Enterprise Relevancy Ranking Using a Neural Network,” both of which applications are incorporated by reference herein in their entirety.
Computers users have different ways to locate information that may be locally or remotely stored. For example, search engines may be used to locate documents using a search query. A search engine attempts to return relevant results based on the particular search query.
Embodiments are configured to rank search results using a junk profile. In embodiments, ranking features, such as junk scores, may be calculated and used by a ranking algorithm to rank candidate documents in response to a search query. In an embodiment, an index is created to facilitate the identification of candidate documents in response to a search query. The index, in embodiments, is optimized by eliminating or separately indexing document data that may have been automatically inserted when the document was created. In embodiments, junk scores for documents may be further calculated by determining the similarity between a document and known junk documents. In embodiments, the junk scores are based on a comparison of histograms for the document to histograms for known junk documents. Histograms may be based upon different junk variables, such as term frequency and chunk size. In embodiments, search results may be identified as possible junk documents based on a calculated junk score, whether or not the search results are ranked based on the junk score. Such identification as possible junk may be displayed to the user and may be maintained as a searchable parameter.
FIG. 1 illustrates an embodiment of a system for ranking search results in accordance with the present disclosure.
FIG. 2 illustrates a method for ranking search results in accordance with the present disclosure.
FIG. 3 illustrates a method for creating an index in accordance with the present disclosure.
FIG. 4 illustrates a method for calculating junk scores in accordance with the present disclosure.
FIG. 5 illustrates an exemplary operating environment in accordance with the present disclosure.
Junk documents that are returned in response to a search query can frustrate users and waste valuable time and energy. In embodiments, “junk” documents may include documents that do not contain much human readable document data or that contain little document data other than data that was added automatically by the system or template used to create the document. For example, in enterprise environments, junk documents may be documents that were generated, but no significant content was ever added. Often, such documents may have a title that is a close match to particular search query terms, a popular type of uniform resource locator (“URL”) and matching anchor text. Accordingly, a default search-result ranker may have no way to distinguish such junk documents from documents that have useful information and may rank such junk documents highly.
As briefly described above, embodiments disclosed herein are directed to ranking search results using a junk profile. For a given corpus of documents, one or more junk profiles may be created and maintained. In general, the junk profile provides reference metrics or models to represent known junk documents. For example, a junk profile may comprise a dictionary of document data that is automatically inserted into documents created using a particular system or template. A junk profile may also comprise one or more representations (e.g., histograms) of a distribution of a particular junk variable for known junk documents. The junk profile effectively provides a usable representation of known junk documents, and the present systems and methods employ the junk profile to predict the likelihood that documents in the corpus are junk. In embodiments, junk scores are calculated and used to rank such documents higher or lower in response to a search query.
The present systems and methods, although not so limited, may be particularly useful in an enterprise environment in which a corpus of documents is created by one or more known systems and/or templates. In embodiments, a search component of the present disclosure can use the functionality of an integrated server platform, such as MICROSOFT OFFICE SHAREPOINT SERVER® system, to calculate, collect, store, and update junk scores and other ranking features that can be used as part of a ranking determination. Because the MICROSOFT OFFICE SHAREPOINT SERVER® system includes standard and customizable “templates” by which documents may be created, the creation of a junk profile may be simplified.
In one embodiment, a system includes a search component which comprises a searching application that can be included as part of a computer-readable storage medium. The searching application can be used to provide search results based in part on a user query. For example, a user can input keywords or other search parameters to the search application and the search application can use the search parameters to identify candidate documents. The candidate documents may be ranked and presented to the user according, in part, using junk profile(s).
FIG. 1 is a block diagram of a system 100 which includes indexing, searching, and other functionality. For example, the system 100 can include indexing, searching, and other applications that can be used to index information as part of an indexed data structure and search for relevant data using the indexed data structure. As described below, components of the system 100 can be used to rank and return search results based at least in part on one or more junk scores for the documents. A user can submit queries to the search component 102 using a user interface 103, such as a browser or search window for example.
As shown in FIG. 1, the system 100 includes a search component 102, such as a search engine for example, that can be configured to return results based in part on a query input. For example, the search component 102 can operate to use a word, words, phrases, and other data to locate candidate documents. The search component 102 can operate to locate information and can be used by an operating system (OS), file system, web-based system, or other system. The search component 102 can also be included as an add-in component, wherein the searching functionality can be used by a host system or application. As described further herein, the search component 102 may also use junk scores as ranking features to rank candidate documents.
The search component 102 can be configured to provide search results (uniform resource locaters (URLs) for example) that may be associated with documents. For example, the search component 102 may use text, property information, formatting, and/or metadata when returning search results associated with local files, remotely networked files, combinations of local and remote files, etc. In one embodiment, the search component 102 can interact with a file system, virtual web, network, or other information source when providing search results.
The search component 102 includes a ranking component 104 that can be configured to rank search results (such as candidate documents) based at least in part on a ranking algorithm 106 and one or more ranking features 108. In one embodiment, the ranking algorithm 106 can be configured to provide a number or other variable that can be used for sorting purposes by the search component 102. The ranking features 108 can be described as basic inputs or raw numbers that can be used when identifying relevance of a search result. The ranking features 108 can be collected, stored, and maintained in a database component 110.
Alternatively, ranking features 108, such as junk scores, can be stored and maintained in a dedicated store, including local, remote, and other storage mediums. One or more of the ranking features 108 can be input to the ranking algorithm 106, and the ranking algorithm 106 can operate to rank search results as part of a ranking determination. As described below, in one embodiment, the ranking component 104 can use one or more ranking features 108 as part of the ranking determination.
Correspondingly, the search component 102 can use the ranking component 104 and associated ranking algorithm 106 when using one or more of the ranking features 108 as part of a ranking determination to provide search results. Search results can be provided based on a relevance ranking or some other ranking. For example, the search component 102 can render the search results from most relevant to least relevant based at least in part on the relevance determination provided by the ranking component 104 using one or more of the ranking features 108, including junk scores.
With continuing reference to FIG. 1, the system 100 also includes an index component 112 that can be used to index information. The index component 112 can be used to index and catalog information to be stored in the database component 110. Moreover, the index component 102 can use the metadata, content, and/or other document data when indexing against a number of disparate information sources. For example, the index component 112 can be used to build an inverted index data structure that maps keywords and other document data to documents, including URLs associated with documents.
The search component 102 can use the indexed information when returning relevant search results, such as candidate documents, according to the ranking provided by the ranking component 104. In an embodiment, as part of a search, the search component 102 can be configured to identify a set of candidate documents that contain a portion or all of a user's query information, such as keywords and phrases for example. For example, query information may be located in a document's body or metadata, or additional metadata associated with a document that can be stored in other documents or data stores (such as anchor text for example). As described below, rather than returning an entire set of search results if the set is large, the search component 102 can use the ranking component 104 to rank the candidates with respect to relevance or some other criteria, and return a subset of the entire set based at least in part on the ranking determination. However, if the set of candidates is not too large, the search component 102 can operate to return the entire set.
In an embodiment, the ranking component 104 can use the ranking algorithm 106 to order candidate documents associated with a particular query according to relevance. For example, the ranking algorithm 106 can calculate a rank value associated with a candidate search result, wherein a higher rank value corresponds with a more relevant candidate. Multiple features, including one or more ranking features 108 (such as junk scores), can be input into the ranking algorithm 106 which can then compute an output that enables the search component 102 to sort candidates by a rank or some other criteria. The search component 102 can use the ranking algorithm 106 to prevent the user from having to inspect an entire set of candidates, such as enterprise URL collections for example, by limiting a set of candidates according to rank.
In embodiments, the search component 102 calculates and selects one or more junk scores for candidate documents as part of a relevance determination when returning candidate documents. Candidate documents may have zero or more junk scores associated with them and ranking of candidate documents using junk scores may include averaging, ignoring, or finding the maximum or minimum among a set of junk scores for a candidate document.
In one embodiment, the search component 102 can use the functionality of an integrated server platform, such as MICROSOFT OFFICE SHAREPOINT SERVER® system, to calculate, collect, store, and update junk scores and other ranking features 108 that can be used as part of a ranking determination. The functionality of the server platform can include web content management, enterprise content services, enterprise search, shared business processes, business intelligence services, and other services. For example, as will be described herein, templates created using the MICROSOFT OFFICE SHAREPOINT SERVER® system, can be used to collect reference information for known junk documents.
As described below, a two layer neural network can be used as part of a relevance determination. In one embodiment, the implementation of the two layer neural network includes a training phase and a ranking phase as part of a forward propagation process using the two layer neural network. LambdaRank can be used as a training algorithm during the training phase, and a neural net forward propagation model can be used as part of the ranking determination (see C. Burges, R. Ragno, Q. V. Le, “Learning To Rank With Nonsmooth Cost Functions” in Schölkopf, Platt and Hofmann (Ed.) Advances in Neural Information Processing Systems 19, Proceedings of the 2006 Conference, (MIT Press, 2006), incorporated by reference in its entirety herein). For example, a standard neural net forward propagation model can be used as part of the ranking phase. One or more junk scores can be used as ranking features 108 in conjunction with the two layer neural network as part of ranking candidate documents based on a user query.
In an embodiment, the ranking component 104 utilizes a ranking algorithm 106 which comprises a two layer neural network scoring function (also referred to herein as a “scoring function”) which includes:
Score ( x 1 , … , x n ) = ( ∑ j = 1 m h j · w 2 j ) wherein , ( 1 ) h j = tan h ( ( ∑ i = 1 n x i · w ij ) + t j ) ( 1 a )
wij is a weight to be applied to input value x, by hidden node j,
and, tan h is the hyperbolic tangent function:
h j = tan h ( ( ∑ i - 1 n x i · w ij ) + t j ) ( 1 c )
In an embodiments, other functions having similar properties and characteristics as the tan h function can be used above. In embodiments, the variable x, can represent one or more junk scores or other ranking features. A X-rank training algorithm can be used to train the two-layer neural network scoring function before ranking as part of a relevance determination. Moreover, new features and parameters can be added to the scoring function without significantly affecting a training accuracy or training speed.
One or more ranking features 108 can be input and used by the ranking algorithm 106, the two layer neural network scoring function for this embodiment, when making a relevance determination when returning search results based on a user query. In embodiments, one or more junk scores can be input and used as ranking features 108 by the ranking algorithm 106 when making a relevance determination as part of returning search results based on a user query.
Other features can also be used when ranking and providing search results. In an embodiment, click distance (CD), URL depth (UD), file type or type prior (T), language or language prior (L), metadata, BM25F, and/or other ranking features can be used to rank and provide search results. Additional details regarding use of a two-layer neural network to rank search results based upon ranking features (including transformation and normalization of ranking features), are provided in U.S. patent application Ser. No. 11/874,579, filed Oct. 18, 2007, entitled “Ranking and Providing Search Results Based in Part on a Number of Click Through Parameters,” and U.S. patent application Ser. No. 11/874,844, filed Oct. 18, 2007, entitled “Enterprise Relevancy Ranking Using a Neural Network,” now U.S. Pat. No. 7,840,569, both of which applications are incorporated by reference herein in their entirety. In embodiments, other types of ranking algorithms 106 can be employed. For example, one or more of these (or additional) ranking features 108, including junk scores, can also be used as part of a linear ranking determination or other ranking algorithm 106 employed by ranking component 104.
FIG. 2 illustrates an embodiment of a method 200 for determining and using junk scores as ranking features in ranking candidate documents in response to a search query. In embodiments, the steps of the method 200 illustrated in FIG. 2 and other figures herein may be performed in a different order, and steps may be added, eliminated, or combined. The method of FIG. 2 may be performed by a system, such as system 100. At step 201, an index of documents in a corpus is created. In embodiments, the index is an inverted index that maps document data to documents within a corpus. As used herein, document data may include words, numbers, phrases, text, formatting, metadata, and other human-readable and non-human-readable data within a document. In addition, a corpus may be any collection of documents that is crawled to create the index. As used herein, documents include word processing documents, spreadsheets, web sites, lists, document libraries, webs, presentations or other files. The corpus may be defined by a collection of files within a specific network (Internet, extranet, or otherwise), site, or other grouping. In embodiments, the present system and methods may be used advantageously to identify probable junk documents within a corpus that employs particular templates for creation of documents. For example, the MICROSOFT OFFICE SHAREPOINT SERVER® system includes standard document templates and permits users to define document templates specific to their MICROSOFT OFFICE SHAREPOINT SERVER® environment.
FIG. 3 illustrates an embodiment of a method 300 for creating the index in step 201, which may be performed, in embodiments, by an indexing component of a system, such as index component 112. In this embodiment, the index is created with reference to a dictionary of automatically generated data. At step 301, the corpus of documents is crawled. At step 302, a dictionary of automatically generated data is selected.
The dictionary of automatically generated data may be created, in embodiments, by creating a blank document using a system associated with the corpus. As used herein, “blank” means substantially lacking document data other than document data automatically created and inserted in the document by a system associated with the corpus when the document is created. For example, the MICROSOFT OFFICE SHAREPOINT SERVER® system permits users to define templates for particular document libraries, and a document library may comprise a corpus of documents that can be indexed, queried and ranked as set forth herein. A blank document created with such a template will include certain document data (such as text, formatting, metadata, etc.) that is automatically generated by the MICROSOFT OFFICE SHAREPOINT SERVER® system. The dictionary of automatically generated data, then, can be created by extracting and compiling the document data in the blank document. The dictionary may also be created or augmented by examining some or all of the existing documents in the corpus and identifying document data that is common to a significant percentage of documents in the corpus. The dictionary may include both content and position information for document data extracted from the blank document.
In embodiments, different dictionaries of automatically generated data may be created for different corpuses, for different templates, and for different systems used to generate the blank documents. In addition, a user (such as an administrator) may be provided the ability to create a new dictionary to be used with respect to a particular type of document by permitting the user to specify a particular blank document to be used. For example, a computer readable storage media may be provided that includes predefined templates generated by a particular system, such as the MICROSOFT OFFICE SHAREPOINT SERVER® system. For example, one such predefined template may comprise a contact management template, while a second such predefined template may be a sales lead pipeline template. Dictionaries of automatically generated data for each of these templates may be compiled and preloaded as part of the system. An administrator, however, may customize an existing template or create a new custom template using the system. In embodiments, the administrator may be prompted to (or the system may automatically) generate a blank document using such custom template and compile a dictionary of automatically generated data from such custom template. In this manner, a dictionary of automatically generated data that is specific to a particular document being indexed may be utilized in embodiments as set forth below.
At step 302, at least one dictionary of automatically generated data is selected. As described, more than one dictionary may be available for selection depending on the type of documents in the corpus. In embodiments, if all of the documents in the corpus were created using the same system and employing the same template, then a single dictionary may be selected and used for all of the documents in the corpus. Alternatively, in embodiments, the corpus may include documents created by disparate systems or using different templates, and different dictionaries may be selected for different documents. In addition, in embodiments, a single dictionary may be defined that combines the content and position information of documents that were created using disparate systems or templates across the corpus. In embodiments, the selected dictionary or dictionaries may be considered part of the junk profile.
At step 304, document data for the crawled documents is compared against the selected one or more dictionaries of automatically generated data. In embodiments, the document data for each crawled document is compared against the dictionary (or dictionaries) selected for that document to determine what document information was likely generated automatically by the system (and/or template) that was used to create that crawled document. For example, a system-defined template used to create a document may include the word “Task” automatically in the title of every document created using that template. The word “Task” and its position within the template are included in the dictionary of automatically generated data selected for that document. Upon comparison of the document to the dictionary, the word “Task” in the title of the document may be determined to “match” a corresponding entry in the selected dictionary. As used herein, “match” may comprise exact or substantial correlation of content, position, or both. In addition, in embodiments, matching the document data to the dictionary (or dictionaries) of automatically generated data may be considered calculating a junk score as described herein.
At step 306, document data that matches the selected dictionary of automatically generated data is delineated. In embodiments, such matching document data may be delineated by marking the document or the matching document data in the index, indexing matching document data separately from non-matching document data, ignoring the matching document data and indexing only non-matching document data, or by other methods.
Consider the following simple example. Although this example is provided using textual document data, any document data can be used (e.g., metadata, formatting, non-human readable data, etc.). Given the texts T0=“you know what it is”; T1=“what is it”; and T2=“it is a bird,” the following full inverted file index would typically be created (where the pairs of numbers refer to the document number (Tx) and the word position). E.g., the word “bird” is in the third document (T2) and it is the fourth word in that document (position 3):
“a”: [(2, 2)]
“bird”: [(2, 3)]
“is”: [(0, 4), (1, 1), (2, 1)]
“it”: [(0, 3), (1, 2), (2, 0)]
“know”: [(0, 1)]
“what”: [(0, 2), (1, 0)]
“you”: [(0, 0)]
Now assume that the same dictionary is selected for all three documents, and the selected dictionary includes the word “is” in position 1 (e.g., because a blank document created using the same system and template used to create documents 0, 1, and 2 contained the word “is” in position 1). In embodiments, the word “is” in position 1 matches document data in each of documents 1 and 2. That matching document data may be delineated in several manners. For example, the matching document data from documents 1 and 2 could be ignored in indexing. In this embodiment, the index for the example above would become:
“is”: [(0, 4)]
In this way, the index that will be searched in response to a search query is minimized (for faster searching) and more focused on user-added content (rather than document data added automatically by a system or template used to create the document). In other embodiments, the matching document data could be indexed separately so that subsequent search queries could be run against both the main index (derived from non-matching document data) and a secondary index (derived from matching document data). Accordingly, in embodiments, a ranking algorithm, such as ranking algorithm 106, may weight the appearance of search query terms in the primary index as more important than the appearance of such terms in the secondary index.
At step 308, documents in the corpus are indexed. In embodiments, steps 306 and 308 are combined, and document data that matches the selected dictionary may be delineated as matching by the manner in which such document data (and documents) are indexed (or not indexed) as discussed above.
At step 310, a junk score is calculated for some or all of the crawled documents in the corpus. In embodiments, a junk score is calculated at step 310 that is a function of a similarity metric (such as Kullback-Leibler divergence) between the crawled document and the blank document(s) used to create the selected dictionary of automatically generated data. For example, the similarity metric may comprise a ratio of document data that matches a corresponding entry in the selected dictionary to document data that does not match a corresponding entry in the selected dictionary. A relatively high similarity metric, in embodiments, indicates a document that does not include much document data other than what is in the blank document(s) used to create the selected dictionary. This may be considered an indication that the document is likely “junk” and a corresponding junk score may be assigned to the document and stored as a ranking feature, such as ranking features 108 in database component 110. The junk score may comprise the similarity metric itself or may comprise a function of the similarity metric in order to normalize and use the junk score as a ranking feature.
In embodiments, if the document is compared 304 to more than one selected dictionary, the junk score may be calculated 310 as a function of the average, maximum, minimum or other calculation of similarity metrics for the document against the selected dictionaries. The junk score calculated at step 310 may be used as a ranking feature as provided herein.
Referring back to FIG. 2, at step 202, one or more junk score(s) are calculated using junk profile(s). In embodiments, the calculation of junk score(s) may be done in combination with creating 201 an index of documents in the corpus. For example, as discussed, the similarity metric(s) between a crawled document and the blank document(s) used to create the selected dictionary(ies) of automatically generated data may be used to determine a junk score. In other embodiments, the junk score calculated at step 310 may not be calculated or may be combined with, or used as an additional ranking feature in conjunction with, calculating 202 a junk score using junk variable(s) from the junk profile.
As used in embodiments herein, junk variable may mean a variable that can be used to determine the similarity between a document and a known junk document. For example, as discussed below, in embodiments a junk variable may comprise term frequency. In other embodiments, a junk variable may comprise chunk size. Junk variables may be used individually or may be combined.
At step 202, in embodiments, the junk score for a document is calculated as a function of a similarity metric between one or more histograms created for the document and a junk profile, wherein the junk profile comprises one or more histograms for known junk document(s). FIG. 4 illustrates one embodiment of a method 400 for calculating 202 a junk score for a document using one or more junk profile(s).
In embodiments, method 400 is performed for all documents in the corpus while the documents are being indexed at step 201 or otherwise before a search query is received. In other embodiments, method 400 may be performed after a search query is received and candidate documents have been identified. Method 400 is described with respect to one document, however, method 400 may be repeated, for example, for any or all documents in a corpus or for candidate documents identified in response to a search query.
At step 401, in embodiments, at least one histogram is generated for the document based on a junk variable. As used herein, a histogram may be a representation of a variable within defined categories (or bins). In other embodiments, alternative representations of the distribution of a junk variable may be used instead of or in addition to, histograms. For example, a continuous function can be calculated or estimated to represent such distribution without converting it to a histogram. A histogram for a document being analyzed is described herein as a “candidate histogram.”
In the example embodiment shown in method 400, at step 401, a candidate histogram is generated for the document. In the following example, the candidate histogram is based on term frequency for the document (e.g., the percentage of the unique terms in the document that have a corresponding frequency of occurrence in the corpus). For example, assume in a simple example that a document includes four terms: T1, T2, T3, and T4. Example total corpus frequencies (i.e., total number of documents in the corpus that contain this term) is shown for each term as follows:
T2: 300
T3: 100,000
T4: 50
A term frequency candidate histogram for this document may be defined in relation to categories, or “bins.” In this example, four bins are defined for total corpus frequencies: [1 . . . 20], [21 . . . 400], [401 . . . 12000], [12001 . . . max]. Accordingly, the term frequency candidate histogram for this document may be represented as: [0.25, 0.5, 0.0, 0.25]. This reflects that one out of the four terms falls into the first bin, two of the four terms fall into the second bin, none of the four terms falls into the third bin, and one of the four terms falls into the fourth bin.
At step 402 the candidate histogram is compared to at least one junk profile. In addition to the junk profile(s) described in relation to FIGS. 2 and 3 above, the junk profile may further comprise one or more histograms of known junk document(s). It may also comprise one or more rules that approximate a representation of known junk document(s). “Junk documents” may be defined differently in different embodiments. For example, in some embodiments, junk documents comprise documents that include a large percentage of non-human readable document data versus the document as a whole. In other embodiments, junk documents may comprise documents that are nearly empty. In embodiments, administrators are permitted (e.g., through user interface 103) to define particular existing documents as “known junk” for the purposes of creating reference histogram(s).
For example, with reference to the simple example discussed above with respect to step 401, assume that a known junk document includes the following terms (with total corpus frequencies for each):
T6: 1,000
T7: 12
A term frequency reference histogram for this known junk document may be defined in relation to the same categories, or “bins” as the candidate histogram: [1 . . . 20], [21 . . . 400], [401 . . . 12000], [12001 . . . max]. Accordingly, the term frequency reference histogram for this known junk document may be represented as: [0.4, 0.0, 0.4, 0.2]. This reflects that two out of the five terms fall into the first bin, none of the five terms falls into the second bin, two of the five terms fall into the third bin, and one of the five terms falls into the fourth bin.
Comparison 402 of the candidate histogram to the junk profile (in this example, the junk profile comprises a reference histogram), may take many forms. For example, the histograms could be compared by calculating a similarity metric as the distance between the candidate and reference histograms:
Similarity Metric = ∑ 0 n ( B 1 ( i ) - B 2 ( i ) ) 2
In this example comparison function, B1 is the candidate histogram, B2 is the reference histogram, B1(i) is the value for the candidate histogram for the i'th bin, and B2(i) is the value for the reference histogram for the i'th bin. This calculation yields a similarity metric between 0 and 1.
Referring again to the above example, the comparison step 402 calculates a similarity metric between the candidate and reference histograms as follows:
Similarity Metric ( B 1 , B 2 ) = ( .4 - .25 ) 2 + ( 0.0 - .5 ) 2 + ( .4 - 0.0 ) 2 + ( .2 - .25 ) 2 ≈ .66
In this example calculation, the more similar the candidate and reference histograms are, the closer the similarity metric is to zero (thus indicating a higher likelihood that the document is junk).
In step 404 at least one junk score is calculated for the document. In embodiments the junk score may comprise the similarity metric itself. In other embodiments, the similarity metric may be converted to a different scale. In addition, the junk profile used in step 402 may include more than one reference histogram. For example, more than one known junk document may be used to create the junk profile. In embodiments, the junk score(s) may comprise a function of an average, weighted average, maximum, minimum or some other function of multiple similarity metrics calculated during the comparison step 402. In addition, more than one junk score for the document may be calculated based on a junk variable, such as term frequency, and all such junk scores may be used by a ranking algorithm, as described further herein. Junk scores may be stored as metadata with the document itself, separate from the document, or otherwise.
At step 406, a determination is made whether alerts have been requested. For example, in embodiments, administrators may request alerts when a junk score is calculated that exceeds a certain threshold. It should be understood that “exceeding a threshold” as used herein may refer to a measurement (such as a junk score) falling above or below a particular threshold, depending on how a particular scale is defined. If alerts have not been requested, method 400 may end (and, in embodiments, control returns to step 204 in FIG. 2). If alerts have been requested, a determination is made 408 whether the calculated junk score(s) exceed a threshold. In embodiments, the threshold is tunable by an administrator. If the threshold has not been exceeded, method 400 may end. If it has been exceeded, then an alert is sent 410. The alert can take different forms, including email, voice message, text, etc. and may be sent to a user, an administrator, etc. In this manner, administrators and others may be alerted to junk documents that may need to be deleted. In addition, junk scores may be employed as a searchable property. For example, a user may wish to search on documents having high junk scores because they may be good candidates for archiving. Another user may wish to use junk scores as search criteria, automatically causing documents with junk scores over a particular threshold to be filtered out from the returned search results. In other embodiments, junk scores may be queried by an administrator or user in order to find documents that are candidates for deletion.
Method 400 is described in relation to using histogram(s) for the document based on one junk variable, however, method 400 may be repeated for histograms using other junk variables. For example, another junk variable that can be used to predict whether a document is junk is “chunk size.” Chunk size is the length of continuous text. Some document types, such as spreadsheets, have large amounts of non-human readable data. Other documents, such as word processing documents, may be expected to have longer chunks of continuous text. A histogram based on chunk size for the document may illustrate the distribution of chunk sizes (runs of continuous text) within the document. Depending on the document type, chunks may be measured differently by scanning the text of the document and measuring the distance between logical breaks (such as cells, paragraphs, sentences, page breaks, etc.). In embodiments, similarity between the chunk size distribution within a document to the chunk size distribution in a known junk document is an indicator of whether the document should be characterized as junk.
Similarity metrics between the document and a junk profile based on different histograms and/or junk variables may be combined into a single junk score or used as individual junk scores that are employed as ranking features by ranking algorithm, such as ranking algorithm 106.
Referring again to FIG. 2, the method 200 continues at step 204, where a search query is received. In embodiments, a user may enter a search query using a user interface, such as user interface 103. The search query may include key words, phrases or other search parameters, including non-text search parameters (such as formatting, etc.). At step 206, candidate documents are identified. For example, as described in relation to FIG. 1, a search component, such as search component 102, may return candidate documents that match the search query.
At step 208, the candidate documents are ranked. As described in relation to FIG. 1, the candidate documents may be ranked using a ranking component 104, employing one or more ranking algorithms 106 and one or more ranking features 108. Junk scores, calculated as set forth herein, may be used as ranking features 108. Other ranking features 108 may also be used. In embodiments, the rank of an individual document may thus be affected by the document's junk score(s). As such, a document that would otherwise be highly ranked because it matches search query terms closely may be ranked lower because of a high junk score. In other embodiments, the actual ranking of the documents is unaffected by the junk score. Rather, junk scores are used to provide an indication of possible junk to the user without affecting the order in which the documents are listed. At step 210, the ranked candidate documents are presented. For example, a subset (e.g., the first ten) documents may be displayed to a user in order of their ranking. In embodiments, documents that have a junk score that exceeds a threshold may not be presented at all (e.g., may be eliminated as a candidate document). In other embodiments, a document's junk score is presented along with the candidate document (e.g., in a ranked list of candidate documents) so that a user can make an independent determination whether to click on a document having a particular junk score. In other embodiments, a junk threshold may be used. If a document's junk score exceeds the junk threshold, the document may be displayed with an indication that the document is likely junk. For example, if the junk score exceeds the junk threshold, a junk symbol or the actual junk score may be displayed for the document in the ranked list of candidate documents. As used herein, a displayed junk score or junk symbol shall be considered a “junk status.”
In embodiments, the present systems and methods are useful in detecting “accidental junk documents” (as opposed to documents that contain malicious or unwanted information, i.e., “adversarial junk”). For example, a user may have started creating a presentation, inserted a title, saved the presentation without adding any additional content, and then forgotten about it. Ordinarily, in response to a search query containing terms that are used in the title of that presentation, a search component would likely return that presentation and rank it highly. Detecting, however, that the presentation is similar (e.g., has a similar term frequency distribution) to a known junk document, allows a search component to rank the presentation more appropriately.
FIG. 5 illustrates one example of a suitable operating environment 500 in which a software embodiment may be implemented. This is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality. Other well known computing systems, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
In its most basic configuration, operating environment 500 typically includes at least one processing unit 502 and memory 504. Depending on the exact configuration and type of computing device, memory 504 (storing, among other things, junk scores calculated as described herein) may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in FIG. 5 by dashed line 506. Further, environment 500 may also include storage devices (removable, 508, and/or non-removable, 510) including, but not limited to, magnetic or optical disks or tape. Similarly, environment 500 may also have input device(s) 514 such as keyboard, mouse, pen, voice input, etc. and/or output device(s) 516 such as a display, speakers, printer, etc. Also included in the environment may be one or more communication connections, 512, such as LAN, WAN, point to point, etc.
Operating environment 500 typically includes at least some form of computer readable media. Computer readable media can be any available media that can be accessed by processing unit 502 or other devices comprising the operating environment. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information. Communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.
The operating environment 500 may be a single computer operating in a networked environment using logical connections to one or more remote computers. The remote computer may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above. The logical connections may include any method supported by available communications media. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
US5729730 28. März 1995 17. März 1998 Dex Information Systems, Inc. Method and apparatus for improved information storage and retrieval system
US5765150 9. Aug. 1996 9. Juni 1998 Digital Equipment Corporation Method for statistically projecting the ranking of information
US5848404 24. März 1997 8. Dez. 1998 International Business Machines Corporation Fast query search in large dimension database
US5870739 15. Nov. 1996 9. Febr. 1999 Novell, Inc. Hybrid query apparatus and method
US5893116 30. Sept. 1996 6. Apr. 1999 Novell, Inc. Accessing network resources using network resource replicator and captured login script for use when the computer is disconnected from the network
US5905866 24. Apr. 1997 18. Mai 1999 A.I. Soft Corporation Data-update monitoring in communications network
US5913210 27. März 1998 15. Juni 1999 Call; Charles G. Methods and apparatus for disseminating product information via the internet
US5943670 21. Nov. 1997 24. Aug. 1999 International Business Machines Corporation System and method for categorizing objects in combined categories
US6029164 16. Juni 1997 22. Febr. 2000 Digital Equipment Corporation Method and apparatus for organizing and accessing electronic mail messages using labels and full text and label indexing
US6167402 27. Apr. 1998 26. Dez. 2000 Sun Microsystems, Inc. High performance message store
US6178419 18. Juli 1997 23. Jan. 2001 British Telecommunications Plc Data access system
US6314421 12. Mai 1998 6. Nov. 2001 David M. Sharnoff Method and apparatus for indexing documents for message filtering
US6324551 31. Aug. 1998 27. Nov. 2001 Xerox Corporation Self-contained document management based on document properties
US6381597 7. Okt. 1999 30. Apr. 2002 U-Know Software Corporation Electronic shopping agent which is capable of operating with vendor sites which have disparate formats
US6415319 7. Febr. 1997 2. Juli 2002 Sun Microsystems, Inc. Intelligent network browser using incremental conceptual indexer
US6484204 24. Mai 2001 19. Nov. 2002 At&T Corp. System and method for allocating requests for objects and managing replicas of objects on a network
US6557036 7. März 2000 29. Apr. 2003 Sun Microsystems, Inc. Methods and apparatus for site wide monitoring of electronic mail systems
US6598040 14. Aug. 2000 22. Juli 2003 International Business Machines Corporation Method and system for processing electronic search expressions
US6631369 30. Juni 1999 7. Okt. 2003 Microsoft Corporation Method and system for incremental web crawling
US6636853 30. Aug. 1999 21. Okt. 2003 Morphism, Llc Method and apparatus for representing and navigating search results
US6654742 11. Febr. 2000 25. Nov. 2003 International Business Machines Corporation Method and system for document collection final search result by arithmetical operations between search results sorted by multiple ranking metrics
US6678692 10. Juli 2000 13. Jan. 2004 Northrop Grumman Corporation Hierarchy statistical analysis system and method
US6868411 13. Aug. 2001 15. März 2005 Xerox Corporation Fuzzy text categorizer
US6898592 27. Dez. 2000 24. Mai 2005 Microsoft Corporation Scoping queries in a search engine
US7003442 * 12. März 1999 21. Febr. 2006 Fujitsu Limited Document file group organizing apparatus and method thereof
US7010532 13. Okt. 2000 7. März 2006 International Business Machines Corporation Low overhead methods and apparatus for shared access storage devices
US7065523 6. Okt. 2004 20. Juni 2006 Microsoft Corporation Scoping queries in a search engine
US7085755 7. Nov. 2002 1. Aug. 2006 Thomson Global Resources Ag Electronic document repository management and access system
US7283997 14. Mai 2003 16. Okt. 2007 Apple Inc. System and method for ranking the relevance of documents retrieved by a query
US7415459 19. Okt. 2004 19. Aug. 2008 Microsoft Corporation Scoping queries in a search engine
US7496561 1. Dez. 2003 24. Febr. 2009 Science Applications International Corporation Method and system of ranking and clustering for document indexing and retrieval
US7580568 31. März 2004 25. Aug. 2009 Google Inc. Methods and systems for identifying an image as a representative image for an article
US7606793 27. Sept. 2004 20. Okt. 2009 Microsoft Corporation System and method for scoping searches using index keys
US7644107 30. Sept. 2004 5. Jan. 2010 Microsoft Corporation System and method for batched indexing of network documents
US7685084 9. Febr. 2007 23. März 2010 Yahoo! Inc. Term expansion using associative matching of labeled term pairs
US7689531 28. Sept. 2005 30. März 2010 Trend Micro Incorporated Automatic charset detection using support vector machines with charset grouping
US7689559 8. Febr. 2006 30. März 2010 Telenor Asa Document similarity scoring and ranking method, device and computer program product
US7693829 * 25. Apr. 2005 6. Apr. 2010 Google Inc. Search engine with fill-the-blanks capability
US7962462 31. Mai 2005 14. Juni 2011 Google Inc. Deriving and using document and site quality signals from search query streams
US8370331 2. Juli 2010 5. Febr. 2013 Business Objects Software Limited Dynamic visualization of search results on a graphical user interface
US8412702 12. März 2008 2. Apr. 2013 Yahoo! Inc. System, method, and/or apparatus for reordering search results
US20020016787 28. Juni 2001 7. Febr. 2002 Matsushita Electric Industrial Co., Ltd. Apparatus for retrieving similar documents and apparatus for extracting relevant keywords
US20020026390 23. Aug. 2001 28. Febr. 2002 Jonas Ulenas Method and apparatus for obtaining consumer product preferences through product selection and evaluation
US20020083054 27. Dez. 2000 27. Juni 2002 Kyle Peltonen Scoping queries in a search engine
US20020165873 19. Febr. 2002 7. Nov. 2002 International Business Machines Corporation Retrieving handwritten documents using multiple document recognizers and techniques allowing both typed and handwritten queries
US20020168106 * 11. Mai 2001 14. Nov. 2002 Miroslav Trajkovic Palette-based histogram matching with recursive histogram vector generation
US20020169800 5. Jan. 2001 14. Nov. 2002 International Business Machines Corporation XML: finding authoritative pages for mining communities based on page structure criteria
US20030004952 16. Aug. 2002 2. Jan. 2003 Mark Nixon Accessing and updating a configuration database from distributed physical locations within a process control system
US20030028520 * 20. Juni 2001 6. Febr. 2003 Alpha Shamim A. Method and system for response time optimization of data query rankings and retrieval
US20030088545 18. Juni 2001 8. Mai 2003 Pavitra Subramaniam System and method to implement a persistent and dismissible search center frame
US20030101183 26. Nov. 2001 29. Mai 2003 Navin Kabra Information retrieval index allowing updating while in use
US20030195882 24. Sept. 2002 16. Okt. 2003 Lee Chung Hee Homepage searching method using similarity recalculation based on URL substring relationship
US20040064442 27. Sept. 2002 1. Apr. 2004 Popovitch Steven Gregory Incremental search engine
US20040141354 28. Jan. 2003 22. Juli 2004 Carnahan John M. Query string matching method and apparatus
US20040215606 25. Apr. 2003 28. Okt. 2004 David Cossock Method and apparatus for machine learning a document relevance function
US20040249795 5. Juni 2003 9. Dez. 2004 International Business Machines Corporation Semantics-based searching for information in a distributed data processing system
US20040267722 30. Juni 2003 30. Dez. 2004 Larimore Stefan Isbein Fast ranked full-text searching
US20050033742 22. Aug. 2003 10. Febr. 2005 Kamvar Sepandar D. Methods for ranking nodes in large directed graphs
US20050089215 25. Okt. 2003 28. Apr. 2005 Carl Staelin Image artifact reduction using a neural network
US20050154710 8. Jan. 2004 14. Juli 2005 International Business Machines Corporation Dynamic bitmap processing, identification and reusability
US20050210079 17. März 2004 22. Sept. 2005 Edlund Stefan B Method for synchronizing documents for disconnected operation
US20050210105 3. Sept. 2004 22. Sept. 2005 Fuji Xerox Co., Ltd. Conference information processing apparatus, and conference information processing method and storage medium readable by computer
US20050256865 14. Mai 2004 17. Nov. 2005 Microsoft Corporation Method and system for indexing and searching databases
US20050289133 25. Juni 2004 29. Dez. 2005 Yan Arrouye Methods and systems for managing data
US20050289193 22. Apr. 2005 29. Dez. 2005 Yan Arrouye Methods and systems for managing data
US20060031183 * 4. Aug. 2004 9. Febr. 2006 Tolga Oral System and method for enhancing keyword relevance by user's interest on the search result documents
US20060036598 9. Aug. 2005 16. Febr. 2006 Jie Wu Computerized method for ranking linked information items in distributed sources
US20060041521 * 4. Aug. 2004 23. Febr. 2006 Tolga Oral System and method for providing graphical representations of search results in multiple related histograms
US20060047643 29. Aug. 2005 2. März 2006 Chirag Chaman Method and system for a personalized search engine
US20060047649 31. Okt. 2005 2. März 2006 Ping Liang Internet and computer information retrieval and mining with intelligent conceptual filtering, visualization and automation
US20060059144 16. Sept. 2005 16. März 2006 Telenor Asa Method, system, and computer program product for searching for, navigating among, and ranking of documents in a personal web
US20060074871 30. Sept. 2004 6. Apr. 2006 Microsoft Corporation System and method for incorporating anchor text into ranking search results
US20060074903 30. Sept. 2004 6. Apr. 2006 Microsoft Corporation System and method for ranking search results using click distance
US20060136411 21. Dez. 2004 22. Juni 2006 Microsoft Corporation Ranking search results using feature extraction
US20060149723 6. März 2006 6. Juli 2006 Microsoft Corporation System and method for providing search results with configurable scoring formula
US20060173828 9. Dez. 2005 3. Aug. 2006 Outland Research, Llc Methods and apparatus for using personal background data to improve the organization of documents retrieved in response to a search query
US20060212423 16. März 2006 21. Sept. 2006 Rosie Jones System and method for biasing search results based on topic familiarity
US20060248074 * 28. Apr. 2005 2. Nov. 2006 International Business Machines Corporation Term-statistics modification for category-based search
US20060282306 21. Apr. 2006 14. Dez. 2006 Unicru, Inc. Employee selection via adaptive assessment
US20060294100 26. Apr. 2006 28. Dez. 2006 Microsoft Corporation Ranking search results using language types
US20070050338 1. Mai 2006 1. März 2007 Strohm Alan C Mobile sitemaps
US20070067284 21. Sept. 2005 22. März 2007 Microsoft Corporation Ranking functions using document usage statistics
US20070085716 30. Sept. 2005 19. Apr. 2007 International Business Machines Corporation System and method for detecting matches of small edit distance
US20070094285 21. Okt. 2005 26. Apr. 2007 Microsoft Corporation Question answering over structured content on the web
US20080005068 28. Juni 2006 3. Jan. 2008 Microsoft Corporation Context-based search, retrieval, and awareness
US20080016053 1. Aug. 2006 17. Jan. 2008 Bea Systems, Inc. Administration Console to Select Rank Factors
US20080140641 7. Dez. 2006 12. Juni 2008 Yahoo! Inc. Knowledge and interests based search term ranking for search results validation
US20080154888 * 10. Dez. 2007 26. Juni 2008 Florian Michel Buron Viewport-Relative Scoring For Location Search Queries
US20080195596 9. Febr. 2007 14. Aug. 2008 Jacob Sisk System and method for associative matching
US20090006356 27. Juni 2007 1. Jan. 2009 Oracle International Corporation Changing ranking algorithms based on customer settings
US20090006358 27. Juni 2007 1. Jan. 2009 Microsoft Corporation Search results
US20090024606 20. Juli 2007 22. Jan. 2009 Google Inc. Identifying and Linking Similar Passages in a Digital Text Corpus
US20090070306 7. Sept. 2007 12. März 2009 Mihai Stroe Systems and Methods for Processing Inoperative Document Links
US20090106221 18. Okt. 2007 23. Apr. 2009 Microsoft Corporation Ranking and Providing Search Results Based In Part On A Number Of Click-Through Features
US20090106223 18. Okt. 2007 23. Apr. 2009 Microsoft Corporation Enterprise relevancy ranking using a neural network
US20090106235 10. Sept. 2008 23. Apr. 2009 Microsoft Corporation Document Length as a Static Relevance Feature for Ranking Search Results
US20090157607 * 12. Dez. 2007 18. Juni 2009 Yahoo! Inc. Unsupervised detection of web pages corresponding to a similarity class
US20090164929 11. Juni 2008 25. Juni 2009 Microsoft Corporation Customizing Search Results
US20090259651 11. Apr. 2008 15. Okt. 2009 Microsoft Corporation Search results ranking using editing distance and document information
US20090276421 4. Mai 2009 5. Nov. 2009 Gang Qiu Method and System for Re-ranking Search Results
US20090307209 10. Juni 2008 10. Dez. 2009 David Carmel Term-statistics modification for category-based search
US20100191744 26. Jan. 2009 29. Juli 2010 Dmitriy Meyerzon Ranking functions using document usage statistics
US20100268707 1. Juli 2010 21. Okt. 2010 Microsoft Corporation System and method for ranking search results using click distance
US20110106850 29. Okt. 2009 5. Mai 2011 Microsoft Corporation Relevant Individual Searching Using Managed Property and Ranking Features
US20110137893 4. Dez. 2009 9. Juni 2011 Microsoft Corporation Custom ranking model schema
US20110235909 * 26. März 2010 29. Sept. 2011 International Business Machines Corporation Analyzing documents using stored templates
US20130198174 27. Jan. 2012 1. Aug. 2013 Microsoft Corporation Re-ranking search results
CA2279119C 29. Juli 1999 19. Okt. 2004 Ibm Canada Limited-Ibm Canada Limitee Heuristic-based conditional data indexing
EP0950961A2A3 Titel nicht verfügbar
EP0950961A3 15. Apr. 1999 22. März 2000 Xerox Corporation Methods for interactive visualization of spreading activation using time tubes and disk trees
EP1462950A1 27. März 2003 29. Sept. 2004 Sony International (Europe) GmbH Method of analysis of a text corpus
EP1862916A1 * 1. Juni 2006 5. Dez. 2007 Microsoft Corporation Indexing Documents for Information Retrieval based on additional feedback fields
JP4950444B2 Titel nicht verfügbar
JPH11232300A Titel nicht verfügbar
KR20030080826A Titel nicht verfügbar
KR1020060048716A Titel nicht verfügbar
MY147720A Titel nicht verfügbar
RU2138076C1 Titel nicht verfügbar
RU2236699C1 Titel nicht verfügbar
RU2273879C2 Titel nicht verfügbar
RU2319202C2 Titel nicht verfügbar
RU2001128643A Titel nicht verfügbar
TWI396984B Titel nicht verfügbar
WO2007089289A2 * 27. Sept. 2006 9. Aug. 2007 Louis Wang Method for ranking and sorting electronic documents in a search result list based on relevance
WO2007123416A1 * 24. Apr. 2007 1. Nov. 2007 Telenor Asa Method and device for efficiently ranking documents in a similarity graph
1 "Microsoft FAST Search Server 2010 for SharePoint, Evaluation Guide", Published on Aug. 12, 2010, Available at: http://www.microsoft.com/downloads/info.aspx?na=41&srcfamilyid=f1e3fb39-6959-4185-8b28-5315300b6e6b&srcdisplaylang=en&u=http%3a%2f%2download.microsoft.com%2fdownload%2fA%2f7%2fF%2fA7F98D88-BC15-4F3C-8B71-D42A5ED79964%, 60 pgs.
2 "Okapi Similarity Measurement (Okapi"), 11th International Web Conference, www2002, 2002, p. 1.
3 Agarwal et al., "Ranking Database Queries Using User Feedback: A Neural Network Approach", Fall 2006, 9 pp.
4 Agichten et al., "Improving Web Search Ranking by Incorporating User Behavior Information"-http://www.mathcs.emory.edu/~eugene/papers/sigir2006ranking.pdf, 8 pp.
5 Agichten et al., "Improving Web Search Ranking by Incorporating User Behavior Information"—http://www.mathcs.emory.edu/˜eugene/papers/sigir2006ranking.pdf, 8 pp.
6 Australian Exam Report in Application No. 2008 00521-7, mailed Mar. 11, 2009, 4 pgs.
7 Australian First Examiner's Report in 2006279520 mailed Oct. 5, 2010.
8 Australian Notice of Allowance in Application 2006279520, mailed Mar. 2, 2011, 3 pgs.
9 Australian Office Action in Application 2009234120, mailed Feb. 26, 2014, 3 pgs.
10 Bandinelli, Luca, "Using Microsoft SharePoint Products and Technologies in Multilingual Scenarios", http://www.microsoft.com/technet/prodtechnol/office/sps2003/maintain/spmultil.mspx, published on Nov. 1, 2003, printed on May 22, 2006, 32 pp.
11 Becker, Hila et al., "Learning Similarity Metrics for Event Identification in Social Media," Published Date: Feb. 4-6, 2010 http://infolab.stanford.edu/~mor/research/becker-wsdm10.pdf.
12 Becker, Hila et al., "Learning Similarity Metrics for Event Identification in Social Media," Published Date: Feb. 4-6, 2010 http://infolab.stanford.edu/˜mor/research/becker-wsdm10.pdf.
13 Becker, Hila et al., "Learning Similarity Metrics for Event Identification in Social Media," Published Date: Feb. 4-6, 2010, http://infolab.stanford.edu/~mor/research/becker-wsdm10.pdf, 10 pp.
14 Becker, Hila et al., "Learning Similarity Metrics for Event Identification in Social Media," Published Date: Feb. 4-6, 2010, http://infolab.stanford.edu/˜mor/research/becker-wsdm10.pdf, 10 pp.
15 Bohm et al., "Multidimensional Index Structures in Relational Databases", Journal of Intelligent Information Systems, Jul. 2000, vol. 15, Issue 1, pp. 1-20, found at: http://springerlink.com/content/n345270t27538741/fulltext.pdf.
16 Brin, S. et al., "The Anatomy of a Large-Scale Hypertextual Web Search Engine", Proceedings of the Seventh International World-Wide Web Conference, ′Online! Apr. 14, 1998, pp. 1-26.
17 Brin, S. et al., "The Anatomy of a Large-Scale Hypertextual Web Search Engine", Proceedings of the Seventh International World-Wide Web Conference, 'Online! Apr. 14, 1998, pp. 1-26.
18 Burges, Christopher J.C. et al. "Learning to Rank with Nonsmooth Cost Functions"-http://books.NIPS.cc/papers/txt/nips19/NIPS2006-0574.txt, 2 pp.
19 Burges, Christopher J.C. et al. "Learning to Rank with Nonsmooth Cost Functions"—http://books.NIPS.cc/papers/txt/nips19/NIPS2006—0574.txt, 2 pp.
20 Canadian Notice of Allowance in Application 2618854, received Jan. 13, 2014, 1 pg.
21 Canadian Office Action in Application 2618854, mailed Mar. 27, 2013, 2 pgs.
22 Carmel, D. et al., "Searching XML Documents Via XML Fragments", SIGIR Toronto, Canada, Jul.-Aug. 2003, pp. 151-158.
23 Chakrabarti, S., "Recent Results in Automatic Web Resource Discovery", ACM Computing Surveys, vol. 31, No. 4es, Dec. 1999, pp. 1-7.
24 Chen, Hsinchun et al., "A Smart Itsy Bitsy Spider for the Web", Journal of the American Society for Information Science, 49(7), 1998, pp. 604-618.
25 Chen, Michael et al., Cha Cha, "A System for Organizing Intranet Search Results", Computer Science Department, University of California, Berkeley, 1999, pp. 1-12.
26 Chinese 1st Office Action in Application 200980112928.6, mailed Jun. 8, 2012, 8 pgs.
27 Chinese 2nd Office Action in Application 200980112928.6, mailed Mar. 4, 2013, 9 pgs.
28 Chinese Application 200510088213.5, Notice of Allowance mailed Apr. 20, 2010, 4 pgs.
29 Chinese Application No. 200510088212.0, First Office Action mailed Jul. 4, 2008, 10 pgs.
30 Chinese Application No. 200510088212.0, Notice of Allowance mailed Jan. 8, 2010, 4 pgs.
31 Chinese Decision on Reexamination cited in 200680029645.1, mailed Dec. 14, 2012, 15 pgs.
32 Chinese Decision on Re-Examination in Application 200510084707.6 mailed Aug. 22, 2011, 12 pgs.
33 Chinese Decision on Rejection in 200680029645.1 mailed Aug. 12, 2010.
34 Chinese Final Rejection in 200510084707.6 mailed Aug. 21, 2009, 13 pgs.
35 Chinese Final Rejection in 200510088213.5 mailed Mar. 6, 2009.
36 Chinese First Office Action in 200510084707.6 mailed Mar. 28, 2008, 10 pgs.
37 Chinese First Office Action in 200680034531.6 mailed Sep. 11, 2009, 7 pgs.
38 Chinese First Office Action in Chinese Application/Patent No. 200880112416.5, mailed Aug. 12, 2011, 11 pgs.
39 Chinese First Official Action in 200510088213.5 mailed May 9, 2008.
40 Chinese First Official Action in 200510088527.5 mailed Apr. 18, 2008.
41 Chinese First Official Action in 200680029645.1 mailed Jun. 19, 2009.
42 Chinese First Official Action in 200680035828.4 mailed Jun. 19, 2009.
43 Chinese Notice of Allowance in 200510088527.5 mailed Jul. 24, 2009, 4 pgs.
44 Chinese Notice of Allowance in 200680034531.6 mailed Oct. 14, 2010, 6 pgs.
45 Chinese Notice of Allowance in Application 200510084707.6, mailed Sep. 25, 2012, 4 pgs.
46 Chinese Notice of Allowance in Application 200880112416.5, mailed Jul. 18, 2012,4 pgs.
47 Chinese Notice of Allowance in Application 2009801129286, mailed Aug. 30, 2013, 4 pgs.
48 Chinese Notice of Reexamination in Application 200680029645.1, mailed Aug. 20, 2012, 11 pgs.
49 Chinese Second Office Action in 200510084707.6 mailed Nov. 7, 2008, 10 pgs.
50 Chinese Second Office Action in 200680029645.1 mailed Apr. 6, 2010.
51 Chinese Second Official Action in 200510088213.5 mailed Oct. 10, 2008.
52 Chinese Second Official Action in 200510088527.5 mailed Dec. 26, 2008.
53 Chinese Third Office Action in 200510084707.6 mailed Feb. 20, 2009, 12 pgs.
54 Chinese Third Official Action in 200510088213.5 mailed Sep. 4, 2009.
55 Cho et al., "Efficient Crawling Through URL Ordering", In Proceedings of the 7th International World Wide Web Conference, Apr. 1998, pp. 161-180.
56 Conlon, M., "Inserts Made Simple", American Printer, Nov. 1, 2002, retrieved from internet on Dec. 17, 2010: http://americanprinter.com/press/other/printing-inserts-made-simple/, 4 pp. (Cited in Search Report Jan. 21, 2011).
57 Conlon, M., "Inserts Made Simple", American Printer, Nov. 1, 2002, retrieved from internet on Dec. 17, 2010: http://americanprinter.com/press/other/printing—inserts—made—simple/, 4 pp. (Cited in Search Report Jan. 21, 2011).
58 Craswell, Nick et al., "Relevance Weighting for Query Independent Evidence", Aug. 15-19, 2005, ACM, pp. 416-423.
59 Creswell, N. et al., "TREC12 Web Track as CSIRO", TREC 12, Nov. 2003, 11 pp.
60 Cutler, M. et al., "A New Study on Using HTML Structures to Improve Retrieval", 11th IEEE International Conference on Chicago, IL, Nov. 9-11, 1999, pp. 406-409.
61 Desmet, P. et al., "Estimation of Product Category Sales Responsiveness to Allocated Shelf Space", Intern. J. of Research in Marketing, vol. 15, No. 5, Dec. 9, 1998, pp. 443-457. (Cited in Search Report Feb. 23, 2011).
62 Ding, Chen et al., "An Improved Usage-Based Ranking", obtained online Jul. 1, 2009 at: http://www.springerlink.com/content/h0jut6d1dnrk5227/fulltext.pdf, 8 pgs.
63 Egyptian Official Action in PCT 269/2008 mailed Feb. 1, 2010.
64 Eiron, N. et al., "Analysis of Anchor Text for Web Search", Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 28-Aug. 1, 2003, Toronto, Canada, 8 pgs.
65 EP 2nd Office Action in Application 05105672.9, mailed Oct. 15, 2009, 4 pgs.
66 EP Communication to cancel the oral summons in Application 05105048.2, mailed Jul. 16, 2012, 1 pg.
67 EP Exam Report in EP 00309121.2-1522 mailed Jul. 4, 2003.
68 EP Exam Report in EP 00309121.2-1527 mailed Feb. 8, 2007.
69 EP Exam Report in EP 00309121.2-1527 mailed Jun. 16, 2004.
70 EP Exam Report in EP 05105048.2-2201 mailed Apr. 23, 2007.
71 EP Examination Report in Application 05105672.9, mailed Oct. 24, 2006, 4 pgs.
72 EP Notice of Allowance in Application 05105048.2, mailed Aug. 13, 2012, 8 pgs.
73 EP Office Action in Application 05105107.6, mailed Mar. 28, 2008, 6 pgs.
74 EP Result of consultation in Application 05105048.2, mailed Aug. 8, 2012, 3 pgs.
75 EP Search Report in Application 05105107.6, mailed Apr. 7, 2006, 3 pgs.
76 EP Search Report in Application 05105672.9, mailed Feb. 6, 2006, 3 pgs.
77 EP Search Report in EP 00309121 mailed Jul. 18, 2002.
78 EP Search Report in EP 05105048 mailed Jan. 17, 2006.
79 EP Search Report in EP 05105110 dated Aug. 11, 2006.
80 EP Summons to Attend Oral Proceedings in EP 05105048.2-2201 mailed Apr. 3, 2012.
81 European Communication in Application 05105107.6, mailed Dec. 17, 2012, 4 pgs.
82 European Extended Search Report in Application 06836141.9 mailed Dec. 27, 2011, 8 pgs.
83 European Extended Search Report in Application 097308084, mailed Oct. 2, 2012, 7 pgs.
84 European Notice of Allowance in Application 00309121.2, mailed Jun. 15, 2009, 5 pgs.
85 European Notice of Allowance in Application EP 06836141.9, mailed Jan. 31, 2013, 6 pgs.
86 European Official Action in 05105110.0-1527 mailed Aug. 4, 2010.
87 European Report on Result of Consultation in Application EP 06836141.9, mailed Jan. 9, 2013, 3 pgs.
88 European Search Report in 08840594.9-2201 mailed Feb. 23, 2011.
89 European Search Report in 08840594.9-2201 mailed Jan. 21, 2011.
90 European Search Report in Application 06789800.7 mailed Oct. 13, 2011, 11 pgs.
91 Extended European Search Report in Application 06804098.9, mailed Dec. 19, 2011, 7 pgs.
92 Fagin, R. et al., "Searching the Workplace Web", IBM Almaden Research Center, In Proceedings of the Twelfth International World Wide Web Conference, Budapest, May 20, 2003, 21 pgs.
93 Fagin, Ronald, "Searching the Workplace Web", Mar. 3, 2005, pp. 1-10.
94 Fiedler, J. et al., Using the Web Efficiently: Mobile Crawlers, 17th Annual Int'l. Conference of the Association of Management on Computer Science, Aug. 1999, pp. 324-329.
95 Gross, Christian, Microsoft Interactive Developer, No. 2, "Integrating the Microsoft Index Server with Active Server Pages", Jun. 1997, 21 pgs.
96 Hawking, D. et al., "Overview of the TREC-8 Web Track", TREC, Feb. 2000, pp. 1-18.
97 Hawking, D. et al., "Overview of TREC-7 Very Large Collection Track", TREC, Jan. 1999, pp. 1-13.
98 Hawking, D., "Overview of the TREC-9 Track", TREC, 2000, pp. 1-16.
99 Heery, Rachel, "Review of Metadata Formats", Program, vol. 30, No. 4, Oct. 1996, 1996 IEEE, pp. 345-373.
100 Hiemstra, D. et al., "Relevance Feedback for Best Match Term Weighting Algorithms in Information Retrieval", Proceedings of the Joint DELOS-NSF Workshop on Personalisation and Recommender Systems in Digital Libraries, ERCIM Workshop Proceedings 01/W03, pp. 37-42, Jun. 2001.
101 Hoeber, Orland et al., "Evaluating the Effectiveness of Term Frequency Histograms for Supporting Interactive Web Search Tasks," Published Date: Feb. 25-27, 2008 http://delivery.acm.org/10.1145/1400000/1394484/p360-hoeber.pdf?key1=1394484&key2=1611170721&coll=GUIDE&d1=GUIDE&CFID=83362159&CFTOKEN=63982632.
102 Hoeber, Orland et al., "Evaluating the Effectiveness of Term Frequency Histograms for Supporting Interactive Web Search Tasks," Published Date: Feb. 25-27, 2008, http://delivery.acm.org/10.1145/1400000/1394484/p360-hoeber.pdf?key1=1394484&key2=1611170721&coll=GUIDE&d1=GUIDE&CFID=83362159&CFTOKEN=63982632, 9 pp.
103 Horikawa, Akira, "Table design correcting room of Access user", Visual Basic Magazine, vol. 6, No. 3, pp. 158-170, Shoeisha Col. Ltd., Japan, Mar. 1, 2000. (No English translation). As cited in JP 2005-175174.
104 Huang et al., "Design and Implementation of a Chinese Full-Text Retrieval System Based on Probabilistic Model", IEEE, 1993, pp. 1090-1093.
105 Huuhka, "Google: Data Structures and Algorithms".
106 Indonesian Notice of Allowance in Application W00200800848 mailed Jun. 9, 2011, 4 pgs.
107 Japanese Appeal Decision and Notice of Allowance in Application 2005-175174, mailed Jun. 18, 2013, 4 pgs.
108 Japanese Appeal Decision in 2008-527094 (Appeal No. 2010-011037) mailed Nov. 4, 2011—31 pgs., only first page translated.
109 Japanese Final Notice of Rejection in Application No. 2005-187816 mailed Mar. 16, 2012, 5 pgs.
110 Japanese Final Rejection in 2005-175172 mailed Jun. 7, 2011, 5 pgs.
111 Japanese Final Rejection in 2008-527094 mailed Jan. 22, 2010.
112 Japanese Final Rejection in JP Application 2008-532469, mailed Jan. 29, 2010, 19 pgs.
113 Japanese Interrogation in Application 2005-175174, mailed Jul. 24, 2012, 7 pgs.
114 Japanese Notice of Allowance in 2005-175172 mailed Mar. 6, 2012, 6 pgs.
115 Japanese Notice of Allowance in 2005-175173 mailed Jun. 7, 2011, 6 pgs.
116 Japanese Notice of Allowance in Application 2011-021985, mailed Dec. 25, 2012, 6 pgs.
117 Japanese Notice of Allowance in Application 2011-194741, mailed Sep. 6, 2013, 4 pgs.
118 Japanese Notice of Allowance in Application 2011-504031, mailed Jan. 30, 2014, 4 pgs.
119 Japanese Notice of Allowance in JP Application 2008-532469, mailed Feb. 22, 2011, 6 pgs.
120 Japanese Notice of Final Rejection in 2005-175174 (0292JP01), mailed Aug. 5, 2011, 5 pgs.
121 Japanese Notice of Rejection in 2005-175172 mailed Sep. 28, 2010.
122 Japanese Notice of Rejection in 2005-175173 mailed Oct. 1, 2010.
123 Japanese Notice of Rejection in 2005-175174 (0292JP01), mailed Oct. 29, 2010, 13 pgs.
124 Japanese Notice of Rejection in 2008-527094 mailed Sep. 11, 2009.
125 Japanese Notice of Rejection in Application 2011-194741, mailed May 14, 2013, 4 pgs.
126 Japanese Notice of Rejection in Application 2011-266249, mailed Sep. 2, 2013, 7 pgs.
127 Japanese Notice of Rejection in Application 2011-504031, mailed May 14, 2013, 4 pgs.
128 Japanese Notice of Rejection in Application 2011-527079, mailed Oct. 8, 2013, 15 pgs.
129 Japanese Notice of Rejection in Application No. 2005-187816 mailed May 20, 2011, 13 pgs.
130 Japanese Office Action in JP Application 2008-532469, mailed Sep. 29, 2009, 18 pgs.
131 Jones, K. et al., "A probabilistic model of information retrieval: development and status", Department of Information Science, City University, London, Aug. 1998, 76 pgs.
132 Kazama, K., "A Searching and Ranking Scheme Using Hyperlinks and Anchor Texts", IPSJ SIG Technical Report, vol. 2000, No. 71, Information Processing Society of Japan, Japan, Jul. 28, 2000, pp. 17-24. (cited in Oct. 1, 2010 OA).
133 Kleinberg, Jon M., "Authoritative Sources in a Hyperlinked Environment", Proceedings of the aCM-SIAM symposium on Discrete Algorithms, 1998, 34 pp.
134 Korean Notice of Preliminary Rejection in Application 10-2008-7003121, mailed Jan. 21, 2013, 11 pgs.
135 Korean Notice of Preliminary Rejection in Application 1020087006775, mailed Feb. 4, 2013, 1 pg.
136 Korean Notice of Preliminary Rejection in Application 10-2008-7007702, mailed Feb. 4, 2013, 4 pgs.
137 Korean Official Action in 2005-0057199 mailed Aug. 4, 2011, pgs.
138 Korean Official Action in 2005-0057199 mailed Mar. 26, 2012, 5 pgs.
139 Kotsakis, E., "Structured Information Retrieval in XML Documents", Proceedings of the ACM Symposium on Applied Computing, Madrid, Spain, 2002, pp. 663-667.
140 Kucuk, Mehmet Emin, et al., "Application of Metadata Concepts to Discovery of Internet Resources", ADVIS 2000, INCS 1909, pp. 304-313, 2000.
141 Kwok, K.L., "A Network Approach to Probabilistic Information Retrieval", ACM Transactions on Information Systems, vol. 13, No. 3, Jul. 1995, pp. 324-353.
142 Lalmas, M., "Uniform Representation of Content and Structure for Structured Document Retrieval", 20th SGES International Conference on Knowledge Based Systems and Applied Artificial Intelligence, Cambridge, UK, Dec. 2000, pp. 1-12.
143 Lam et al, "Automatic document classification based on probabilistic reasoning: model and performance analysis," Oct. 12-15, 1997, IEEE, Computational Cybernetics and Simulation vol. 3, pp. 2719-2723.
144 Larkey, Leah S., et al., "Collection Selection and Results Merging with Topically Organized US Patents and TREC Data", Proceedings of the Ninth International Conference on Information Knowledge Management, CIKM 2000, Nov. 6-11, 2000, pp. 282-289.
145 Lee, J.K.W. et al., "Intelligent Agents for Matching Information Providers and Consumers on the Worl-Wide Web", IEEE, 1997, pp. 189-199.
146 Ljosland, Mildrid, "Evaluation of Web Search Engines and the Search for Better Ranking Algorithms," http://www.aitel.hist.no/˜mildrid/dring/paper/SIGIR.html, SIGIR99 Workshop on Evaluation of Reb Retrieval, Aug. 19, 1999, 5 pages.
147 Losee, R. et al., "Research in Information Organization", Literature Review, School of Information and Library Science, Section 4, pp. 53-96, Jan. 2001.
148 Losee, Robert M. et al., "Measuring Search Engine Quality and Query Difficulty: Ranking with Target and Freestyle," http://ils.unc.edu/˜losee/paril.pdf, Journal of the American Society for Information Science, Jul. 29, 1999, 20 pages.
149 Luxenburger et al., "Matching Task Profiles and User Needs in Personalized Web Search", CIKM Proceeding of the 17th ACM Conference on Information and Knowledge Mining, Oct. 2008, pp. 689-698.
150 Malaysia Adverse Report in Application PI20063920, mailed Jul. 31, 2012, 3 pgs.
151 Malaysia Adverse Search Report in Application PI20080638, mailed Jul. 31, 2012, 4 pgs.
152 Malaysian Notice of Allowance in Application PI 20080638, mailed Jun. 28, 2013, 2 pgs.
153 Malaysian Notice of Allowance in Application PI20063920, mailed Dec. 14, 2012, 2 pgs.
154 Malaysian Substantive Examination Report in Application PI 20063920, mailed Jul. 31, 2012, 3 pgs.
155 Managing External Content in Microsoft Office SharePoint Portal Server 2003, http://www.microsoft.com/technet/prodtechnol/sppt/reskit/c2261881x.mspx, published on Jun. 9, 2004, printed on May 22, 2006, 20 pp.
156 Manning, C. et al., "CS276A Text Information Retrieval, Mining, and Exploitation: Lecture 12", Stanford University CS276A/SYMBSYS2391/LING2391 Test Information Retrieval, Mining, and Exploitation, Fall 2002, last modified Nov. 18, 2002, 8 pgs.
157 Matsuo, Y., "A New Definition of Subjective Distance Between Web Pages," IPSJ Journal, vol. 44, No. 1, Information Processing Society of Japan, Japan, Jan. 15, 2003, pp. 88-94. (Cited in Sep. 28, 2010 OA).
158 Matveeva, Irina et al., "High Accuracy Retrieval with Multiple Nested Ranker," http://people.cs.uchicago.edu/˜matveeva/RankerSIGIR06.pdf, SIGIR'06, Seattle, WA Aug. 6-11, 2006, 8 pages.
159 Mexican Office Action with Summary in PA/a/2008/02173 mailed Jun. 5, 2012, 7 pgs.
160 Microsoft Full-Text Search Technologies, http://www.microsoft.com/technet/prodtechnol/sppt/sharepoint/evaluate/featfunc/mssearc . . . , published on Jun. 1, 2001, printed on May 22, 2006, 13 pp.
161 Microsoft SharePoint Portal Server 2001 Resource Kit: Chapter 24, Analyzing the Default Query for the Dashboard, http://www.microsoft.com/technet/prodtechnol/sppt/sharepoint/reskit/part5/c24spprk.mspx, printed on May 22, 2006, 5 pp.
162 Microsoft SharePoint Portal Server 2001 White Paper, "Microsoft SharePoint Portal Server: Advanced Technologies for Information Search and Retrieval," http://download.microsoft.com/download/3/7/a/37a762d7-dbe6-4b51-a6ec-f6136f44fd65/SPS—Search.doc, Jun. 2002, 12 pages.
163 Mittal et al., "Framework for Synthesizing Semantic-Level Indices", Multimedia Tools and Applications, Jun. 2003, vol. 20, Iss. 2., pp. 1-24, found online at: http://www.springerlink.com/content/tv632274r1267305/fulltext.pdf.
164 MSDN, "Understanding Ranking," http://msdn.microsoft.com/en-us/library/ms142524.aspx, Sep. 2007, 4 pages.
165 Murata, Shin Ya, et al., "Ranking Search Results based on Information Needs in Conjunction with Click-Log Analysis", Journal of Japan Database Society, Japan Database Society, Mar. 27, 2009, vol. 7, Part 4, pp. 37-42.
166 Najork, Marc et al., "Breadth-First Crawling Yields High-Quality Pages", ACM, Compaq Systems Research Center, Hong Kong, 2001, pp. 114-118.
167 Nelson, Chris, "Use of Metadata Registries for Searching for Statistical Data", IEEE 2002, Dimension EDI Ltd., pp. 232-235, 2002.
168 New Zealand Examination Report in Application No. 566532, mailed Oct. 15, 2009, 2 pgs.
169 Nie, Jien Yun, "Introduction to Information Retrieval", University of Montreal Canada, 1989 pp. 1-11.
170 Numerico, T., "Search engines organization of information and Web Topology", http://www.cafm.lsbu.ac.uk/eminars/sse/numerico-6-dec-2004.pdf, Dec. 6, 2004, 32 pgs.
171 Ogilvie, P. et al., "Combining Document Representations for Known-Item Search", Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, Canada, 2003, pp. 143-150.
172 Page, L. et al., "The PageRank Citation Ranking: Bringing Order To The Web", Internet Citation, found online at: http://citeseer.nj.nec.com/page98pagerank.html, retrieved Sep. 16, 2002, 18 pgs.
173 PCT International Search Report and Written Opinion in International Application PCT/US2011/033125, mailed Dec. 15, 2011, 8 pgs.
174 PCT International Search Report in PCT/US2009/036597 dated Aug. 28, 2009, 11 pgs.
175 PCT International Search Report, Application No. PCT/US2006/037206, mailed Jan. 16, 2007, 10 pgs.
176 PCT Search Report in Application PCT/US2013/022825, mailed Apr. 30, 2013, 11 pgs.
177 PCT Search Report in PCT/US2006/031965 mailed Jan. 11, 2007.
178 PCT Search Report in PCT/US2008/011894 mailed Feb. 27, 2009, 12 pgs.
179 PCT Search Report in PCT/US2009/063333 dated Apr. 22, 2010, 10 pgs.
180 Pera, Maria S. et al., "Using Word Similarity to Eradicate Junk Emails," Published Date: Nov. 6-8, 2007 http://delivery.acm.org/10.1145/1330000/1321581/p943-pera.pdf?key1=1321581&key2=8421170721&coll=GUIDE&d1=GUIDE&CFID=83362328&CFTOKEN=17563913, 4 pp.
181 Pera, Maria S. et al., "Using Word Similarity to Eradicate Junk Emails," Published Date: Nov. 6-8, 2007 http://delivery.acm.org/10.1145/1330000/1321581/p943-pera.pdf?key1=1321581&key2=8421170721&coll=GUIDE&d1=GUIDE&CFID=83362328&CFTOKEN=17563913.
182 Philippines Letters Patent in Application 12008500189, issued Jan. 6, 2012, 2 pgs.
183 Philippines Office Action in 1-2008-500189 mailed Mar. 11, 2011, 1 page.
184 Philippines Official Action in 1-2008-500189 mailed Jun. 22, 2011, 1 page.
185 Philippines Official Action in 1-2008-500433 mailed Mar. 24, 2011, 1 page
186 Planning Your Information Structure Using Microsoft Office SharePoint Portal Server 2003, http://www.microsoft.com/technet/prodtechnol/sppt/reskit/c0861881x.mspx, published on Jun. 9, 2004, printed on May 22, 2006, 22 pp.
187 Radlinski, Filip, et al.,, Query Chains: Learning to Rank from Implicit Feedback, http://delivery.acm.org/10.1145/1090000/1081899/p239-radlinski. pdf?key1=1081899&key2=3628533811&coll=GUIDE&CFID=27212902&CFTOKEN=53118399, KDD'05, Chicago, IL, Aug. 21-24, 2005, 10 pages.
188 Robertson, S. et al., "Okapi at TREC-3", Centre for Interactive Systems Research Department of Information Science, Third Text Retrieval Conference, 1995, 19 pp. (cited in US Official Action, Dec. 21, 2011).
189 Robertson, S. et al., "Okapi at TREC-4", 1996, 24 pp.
190 Robertson, S. et al., "Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval", Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1994, pp. 232-241.
191 Russian Application No. 2008105758, Notice of Allowance mailed Dec. 16, 2010, 5 pgs.
192 Russian Notice of Allowance in Application 2011108842, mailed Dec. 16, 2013, 7 pgs. (English translation).
193 Russian Notice of Allowance in Application No. 2008110731/08, mailed Oct. 25, 2010, 7 pgs.
194 Russian Notice of Allowance in Application No. 2010141559, mailed Jun. 27, 2013, 6 pgs.
195 Russian Office Action in Application 2010141559, mailed Jan. 28, 2013, 6 pgs.
196 Russian Official Action in 2008105758 mailed Jun. 29, 2010.
197 Russian Official Action in 2010141559, mailed Jan. 28, 2013, 4 pgs. (in foreign language).
198 Schulz, Stefan, et al., "Indexing Medical WWW Documents by Morphemes", MEDINFO 2001 Proceedings of the 10th World Congress on Medical Informatics, Park I, IOS Press, Inc., pp. 266-270, 2001.
199 Senecal, Sylvain, "Consumers' Decision-Making Process and Their Online Shopping Behavior: A Clickstream Analysis", Jun. 1, 2004, pp. 1600-1607.
200 Shamsfard, Mehrnoush, et al., "ORank: An Ontology Based System for Ranking Documents," http://www.waset.org/ijcs/v1/v1-3-30.pdf, International Journal of Computer Science, vol. 1, No. 3, Apr. 10, 2006, pp. 225-231.
201 SharePoint Portal Server 2001 Planning and Installation Guide, http://www.microsoft.com/technet/prodtechnol/sppt/sharepoint/plan/planinst.mspx, printed on May 22, 2006, 86 pp.
202 Singhal, A. et al., "AT&T at TREC-9", Proceedings of the Ninth Text Retrieval Conference, NIST Special Publication 500-249, ′Online! 2001, pp. 103-105.
203 Singhal, A. et al., "Document Length Normalization", Cornell University, vol. 32, No. 5, 1996, pp. 619-633.
204 Smyth, Barry, "Relevance at a Distance—An Investigation of Distance-Biased Personalization on the Mobile Internet", no date, pp. 1-6.
205 Song et al., "Exploring URL Hit Priors for Web Search", vol. 3936, Springer Berlin / Heidelberg, 2006.
206 South Africa Notice of Allowance in Application No. 2008/02250 mailed Jul. 23, 2009, 1 page.
207 Sturdy, Derek, "Squirrels and nuts: metadata and knowledge management", Business Information Review, 18(4), pp. 34-42, Dec. 2001.
208 Svore, Krysta M. et al., "Improving Web Spam Classification using Rank-time Features," Published Date: May 8, 2007 http://www2007.org/workshops/paper-101.pdf.
209 Svore, Krysta M. et al., "Improving Web Spam Classification using Rank-time Features," Published Date: May 8, 2007 http://www2007.org/workshops/paper—101.pdf.
210 Svore, Krysta M. et al., "Improving Web Spam Classification using Rank-time Features," Published Date: May 8, 2007, http://www2007.org/workshops/paper-101.pdf, 8 pp.
211 Svore, Krysta M. et al., "Improving Web Spam Classification using Rank-time Features," Published Date: May 8, 2007, http://www2007.org/workshops/paper—101.pdf, 8 pp.
212 Taiwanese Notice of Allowance in Application 95129817, mailed Jan. 29, 2013, 4 pgs.
213 Taiwanese Search Report in Application 95129817, mailed Oct. 19, 2012, 1 pg. (OA is in foreign language).
214 Takeda, Takaharu et al., "Multi-Document Summarization by efficient text processing", Proceedings of the FIT2007, Sixth Forum on Information Technology, vol. 2, No. E-014, pp. 165-168, Information Processing Society of Japan, Japan, Aug. 22, 2007. (not an English document).
215 Taylor et al., "Optimisation Methods for Ranking Functions with Multiple Parameters"—http://delivery.acm.org/10.1145/1190000/1183698/p585-taylor.pdf?key1=1183698&key2=3677533811&coll=GUIDE&dl=GUIDE&CFID=22810237&CFTOKEN=34449120, Nov. 5-11, 2006, pp. 585-593.
216 U.S. Appl. No. 09/493,748, Advisory Action mailed Jan. 4, 2005, 2 pgs.
217 U.S. Appl. No. 09/493,748, Amendment and Response filed Apr. 20, 2004, 16 pgs.
218 U.S. Appl. No. 09/493,748, Amendment and Response filed Oct. 12, 2004, 18 pgs.
219 U.S. Appl. No. 09/493,748, Final Office Action mailed Jul. 20, 2004, 14 pgs.
220 U.S. Appl. No. 09/493,748, Office Action mailed Sep. 25, 2003, 11 pgs.
221 U.S. Appl. No. 09/603,695, Advisory Action mailed Aug. 27, 2004, 3 pgs.
222 U.S. Appl. No. 09/603,695, Amendment and Response filed Feb. 27, 2004, 13 pgs.
223 U.S. Appl. No. 09/603,695, Amendment and Response filed Jul. 22, 2004, 13 pgs.
224 U.S. Appl. No. 09/603,695, Amendment and Response filed Nov. 5, 2004, 9 pgs.
225 U.S. Appl. No. 09/603,695, Final Office Action mailed May 18, 2004, 12 pgs.
226 U.S. Appl. No. 09/603,695, Notice of Allowance mailed Dec. 21, 2004, 8 pgs.
227 U.S. Appl. No. 09/603,695, Office Action mailed Nov. 7, 2003, 11 pgs.
228 U.S. Appl. No. 09/749,005, Amendment and Response filed Apr. 28, 2003, 12 pgs.
229 U.S. Appl. No. 09/749,005, Amendment and Response filed Jun. 21, 2004, 14 pgs.
230 U.S. Appl. No. 09/749,005, Notice of Allowance mailed Apr. 7, 2005, 4 pgs.
231 U.S. Appl. No. 09/749,005, Notice of Allowance mailed Aug. 30, 2004, 9 pgs.
232 U.S. Appl. No. 09/749,005, Notice of Allowance mailed Mar. 4, 2005, 4 pgs.
233 U.S. Appl. No. 09/749,005, Office Action mailed Jun. 12, 2003, 10 pgs.
234 U.S. Appl. No. 09/749,005, Office Action mailed Oct. 28, 2002, 12 pgs.
235 U.S. Appl. No. 10/609,315, Amendment and Response filed Mar. 17, 2006, 14 pgs.
236 U.S. Appl. No. 10/609,315, Amendment and Response filed Nov. 29, 2006, 23 pgs.
237 U.S. Appl. No. 10/609,315, Notice of Allowance mailed Jan. 24, 2007, 6 pgs.
238 U.S. Appl. No. 10/609,315, Notice of Allowance mailed May 30, 2007, 4 pgs.
239 U.S. Appl. No. 10/804,326, Advisory Action mailed Feb. 21, 2008, 3 pgs.
240 U.S. Appl. No. 10/804,326, Amendment and Response filed Feb. 11, 2008, 28 pgs.
241 U.S. Appl. No. 10/804,326, Amendment and Response filed Jun. 10, 2008, 27 pgs.
242 U.S. Appl. No. 10/804,326, Amendment and Response filed Mar. 16, 2007, 21 pgs.
243 U.S. Appl. No. 10/804,326, Amendment and Response filed Mar. 9, 2009, 8 pgs.
244 U.S. Appl. No. 10/804,326, Amendment and Response filed Sep. 7, 2007, 26 pgs.
245 U.S. Appl. No. 10/804,326, Final Office Action mailed Dec. 11, 2007, 24 pgs.
246 U.S. Appl. No. 10/804,326, Notice of Allowance mailed May 29, 2009, 8 pgs.
247 U.S. Appl. No. 10/951,123, Advisory Action mailed Dec. 31, 2007, 3 pgs.
248 U.S. Appl. No. 10/951,123, Amendment and Response filed Apr. 25, 2007, 15 pgs.
249 U.S. Appl. No. 10/951,123, Amendment and Response filed Apr. 6, 2009, 18 pgs.
250 U.S. Appl. No. 10/951,123, Amendment and Response filed Dec. 13, 2007, 10 pgs.
251 U.S. Appl. No. 10/951,123, Amendment and Response filed Jan. 14, 2008, 10 pgs.
252 U.S. Appl. No. 10/951,123, Amendment and Response filed Sep. 17, 2008, 15 pgs.
253 U.S. Appl. No. 10/951,123, Final Office Action mailed Jan. 5, 2009, 23 pgs.
254 U.S. Appl. No. 10/951,123, Final Office Action mailed Jul. 13, 2007, 15 pgs.
255 U.S. Appl. No. 10/951,123, Notice of Allowance mailed Jun. 25, 2009, 5 pgs.
256 U.S. Appl. No. 10/951,123, Office Action mailed Jan. 25, 2007, 16 pgs.
257 U.S. Appl. No. 10/951,123, Office Action mailed Mar. 18, 2008, 20 pgs.
258 U.S. Appl. No. 10/955,462 Amendment and Response filed Aug. 8, 2007, 21 pgs.
259 U.S. Appl. No. 10/955,462 Amendment and Response filed Mar. 10, 2008, 17 pgs.
260 U.S. Appl. No. 10/955,462 Amendment and Response filed Mar. 5, 2007, 18 pgs.
261 U.S. Appl. No. 10/955,462 Notice of Allowance mailed Feb. 24, 2009, 7 pgs.
262 U.S. Appl. No. 10/955,462 Notice of Allowance mailed Jan. 25, 2010, 6 pgs.
263 U.S. Appl. No. 10/955,462 Notice of Allowance mailed Jun. 10, 2009, 6 pgs.
264 U.S. Appl. No. 10/955,462 Notice of Allowance mailed Jun. 17, 2008, 12 pgs.
265 U.S. Appl. No. 10/955,462 Notice of Allowance mailed Oct. 16, 2009, 7 pgs.
266 U.S. Appl. No. 10/955,462 Notice of Allowance mailed Sep. 23, 2008, 6 pgs.
267 U.S. Appl. No. 10/955,983, Amendment and Response filed Aug. 22, 2007, 13 pgs.
268 U.S. Appl. No. 10/955,983, Amendment and Response filed Mar. 18, 2009, 18 pgs.
269 U.S. Appl. No. 10/955,983, Amendment and Response filed May 13, 2008, 14 pgs.
270 U.S. Appl. No. 10/955,983, Amendment and Response filed Oct. 13, 2009, 12 pgs.
271 U.S. Appl. No. 10/955,983, Amendment and Response filed Sep. 25, 2008, 13 pgs.
272 U.S. Appl. No. 10/955,983, Notice of Allowance mailed Jan. 12, 2010, 10 pgs.
273 U.S. Appl. No. 10/955,983, Notice of Allowance mailed Jun. 4, 2010, 5 pgs.
274 U.S. Appl. No. 10/956,891, Advisory Action mailed Mar. 21, 2008, 3 pgs.
275 U.S. Appl. No. 10/956,891, Amendment and Response filed Aug. 22, 2007, 11 pgs.
276 U.S. Appl. No. 10/956,891, Amendment and Response filed Jun. 1, 2009, 12 pgs.
277 U.S. Appl. No. 10/956,891, Amendment and Response filed Mar. 3, 2008, 11 pgs.
278 U.S. Appl. No. 10/956,891, Amendment and Response filed May 1, 2008, 11 pgs.
279 U.S. Appl. No. 10/956,891, Amendment and Response filed Oct. 16, 2008, 12 pgs.
280 U.S. Appl. No. 10/956,891, Final Office Action filed Nov. 1, 2007, 18 pgs.
281 U.S. Appl. No. 10/956,891, Final Office Action mailed Dec. 31, 2008, 16 pgs.
282 U.S. Appl. No. 10/956,891, Notice of Allowance mailed Aug. 20, 2009, 6 pgs.
283 U.S. Appl. No. 10/956,891, Office Action mailed Jul. 16, 2008, 19 pgs.
284 U.S. Appl. No. 10/956,891, Office Action mailed Mar. 22, 2007, 15 pgs.
285 U.S. Appl. No. 10/959,330, Amendment and Response filed Jan. 6, 2006, 10 pgs.
286 U.S. Appl. No. 10/959,330, Amendment and Response filed Sep. 14, 2005, 12 pgs.
287 U.S. Appl. No. 10/959,330, Notice of Allowance mailed Apr. 3, 2006, 6 pgs.
288 U.S. Appl. No. 10/959,330, Office Action mailed Dec. 14, 2005, 6 pgs.
289 U.S. Appl. No. 10/959,330, Office Action mailed Jun. 27, 2005, 10 pgs.
290 U.S. Appl. No. 10/968,716, Amendment and Response filed Aug. 13, 2007, 6 pgs.
291 U.S. Appl. No. 10/968,716, Amendment and Response filed Jan. 25, 2008, 8 pgs.
292 U.S. Appl. No. 10/968,716, Amendment and Response filed Jun. 15, 2007, 13 pgs.
293 U.S. Appl. No. 10/968,716, Notice of Allowance mailed Jun. 2, 2008, 8 pgs.
294 U.S. Appl. No. 10/968,716, Office Action mailed Mar. 15, 2007, 13 pgs.
295 U.S. Appl. No. 10/968,716, Office Action mailed Oct. 26, 2007, 14 pgs.
296 U.S. Appl. No. 10/981,962, Advisory Action mailed Jan. 23, 2007, 3 pgs.
297 U.S. Appl. No. 10/981,962, Amendment and Response filed Aug. 18, 2008, 10 pgs.
298 U.S. Appl. No. 10/981,962, Amendment and Response filed Feb. 7, 2007, 1 pg.
299 U.S. Appl. No. 10/981,962, Amendment and Response filed Jul. 27, 2007, 16 pgs.
300 U.S. Appl. No. 10/981,962, Amendment and Response filed Jun. 27, 2006, 23 pgs.
301 U.S. Appl. No. 10/981,962, Amendment and Response filed Nov. 27, 2007, 10 pgs.
302 U.S. Appl. No. 10/981,962, Notice of Allowance mailed Aug. 20, 2009, 6 pgs.
303 U.S. Appl. No. 10/981,962, Notice of Allowance mailed Jan. 29, 2009, 6 pgs.
304 U.S. Appl. No. 10/981,962, Notice of Allowance mailed Jan. 9, 2009, 6 pgs.
305 U.S. Appl. No. 10/981,962, Notice of Allowance mailed May 8, 2009, 6 pgs.
306 U.S. Appl. No. 10/981,962, Notice of Allowance mailed Oct. 15, 2008, 6 pgs.
307 U.S. Appl. No. 10/981,962, Notice of Allowance mailed Sep. 11, 2008, 14 pgs.
308 U.S. Appl. No. 10/981,962, Office Action mailed Nov. 13, 2007, 3 pgs.
309 U.S. Appl. No. 11/019,091, Amendment and Response filed Dec. 20, 2007, 23 pgs.
310 U.S. Appl. No. 11/019,091, Amendment and Response filed Jun. 11, 2009, 12 pgs.
311 U.S. Appl. No. 11/019,091, Amendment and Response filed Nov. 30, 2009, 11 pgs.
312 U.S. Appl. No. 11/019,091, Amendment and Response filed Oct. 3, 2008, 15 pgs.
313 U.S. Appl. No. 11/019,091, Notice of Allowance mailed Dec. 23, 2009, 16 pgs.
314 U.S. Appl. No. 11/022,054, Amendment and Response filed Aug. 24, 2007, 19 pgs.
315 U.S. Appl. No. 11/022,054, Notice of Allowance mailed Nov. 15, 2007, 10 pgs.
316 U.S. Appl. No. 11/022,054, Office Action mailed Jun. 19, 2007, 19 pgs.
317 U.S. Appl. No. 11/073,381, Amendment and Response filed Dec. 13, 2010, 10 pgs.
318 U.S. Appl. No. 11/073,381, Amendment and Response filed Dec. 28, 2009, 9 pgs.
319 U.S. Appl. No. 11/073,381, Amendment and Response filed Dec. 9, 2008, 11 pgs.
320 U.S. Appl. No. 11/073,381, Amendment and Response filed Jul. 15, 2009, 10 pgs.
321 U.S. Appl. No. 11/073,381, Amendment and Response filed Jul. 9, 2010, 10 pgs.
322 U.S. Appl. No. 11/073,381, Amendment and Response filed Mar. 18, 2008, 14 pgs.
323 U.S. Appl. No. 11/206,286, Amendment and Response filed Jul. 22, 2009, 3 pgs.
324 U.S. Appl. No. 11/206,286, Amendment and Response filed Mar. 24, 2009, 13 pgs.
325 U.S. Appl. No. 11/206,286, Amendment and Response filed Sep. 30, 2008, 11 pgs.
326 U.S. Appl. No. 11/206,286, Notice of Allowance mailed Apr. 22, 2009, 9 pgs.
327 U.S. Appl. No. 11/231,955, filed Sep. 21, 2005, Amendment and Response filed Apr. 30, 2008, 12 pgs.
328 U.S. Appl. No. 11/231,955, filed Sep. 21, 2005, Amendment and Response filed Sep. 15, 2008, 16 pgs.
329 U.S. Appl. No. 11/231,955, filed Sep. 21, 2005, Final Office Action mailed Jun. 4, 2008, 8 pgs.
330 U.S. Appl. No. 11/231,955, filed Sep. 21, 2005, Notice of Allowance mailed Oct. 21, 2008, 5 pgs.
331 U.S. Appl. No. 11/231,955, filed Sep. 21, 2005, Office Action mailed Jan. 30, 2008, 8 pgs.
332 U.S. Appl. No. 11/238,906, Amendment and Response filed Feb. 26, 2009, 9 pgs.
333 U.S. Appl. No. 11/238,906, Amendment and Response filed Jun. 9, 2008, 10 pgs.
334 U.S. Appl. No. 11/238,906, Amendment and Response filed May 28, 2010, 9 pgs.
335 U.S. Appl. No. 11/238,906, Amendment and Response filed Sep. 1, 2009, 9 pgs.
336 U.S. Appl. No. 11/238,906, Notice of Allowance mailed Aug. 5, 2010, 4 pgs.
337 U.S. Appl. No. 11/238,906, Notice of Allowance mailed Jul. 22, 2010, 10 pgs.
338 U.S. Appl. No. 11/412,723, Amendment and Response filed Jun. 23, 2009, 11 pgs.
339 U.S. Appl. No. 11/412,723, Amendment and Response filed May 31, 2010, 11 pgs.
340 U.S. Appl. No. 11/412,723, Amendment and Response filed Nov. 26, 2008, 10 pgs.
341 U.S. Appl. No. 11/412,723, Amendment and Response filed Nov. 30, 2009, 10 pgs.
342 U.S. Appl. No. 11/412,723, Notice of Allowance mailed Jul. 9, 2010, 10 pgs.
343 U.S. Appl. No. 11/874,579, Amendment and Response filed Dec. 10, 2013, 17 pgs.
344 U.S. Appl. No. 11/874,579, filed Oct. 18, 2007, Amendment and Response filed May 16, 2011, 14 pgs.
345 U.S. Appl. No. 11/874,579, filed Oct. 18, 2007, Amendment and Response filed Nov. 22, 2010, 8 pgs.
346 U.S. Appl. No. 11/874,579, Office Action mailed Mar. 28, 2014, 30 pgs.
347 U.S. Appl. No. 11/874,579, Office Action mailed Sep. 10, 2013, 27 pgs.
348 U.S. Appl. No. 11/874,844, Amendment and Response filed Mar. 15, 2010, 16 pgs.
349 U.S. Appl. No. 11/874,844, Notice of Allowance mailed Jun. 25, 2010, 2 pgs.
350 U.S. Appl. No. 11/874,844, Notice of Allowance mailed May 18, 2010, 9 pgs.
351 U.S. Appl. No. 12/101,951, Advisory Action mailed Jun. 27, 2012, 3 pgs.
352 U.S. Appl. No. 12/101,951, Amendment and Response filed Dec. 3, 2010, 16 pgs.
353 U.S. Appl. No. 12/101,951, Amendment and Response filed Jan. 9, 2012, 10 pgs.
354 U.S. Appl. No. 12/101,951, Amendment and Response filed Jun. 21, 2012, 8 pgs.
355 U.S. Appl. No. 12/101,951, Amendment and Response filed Jun. 3, 2011, 12 pgs.
356 U.S. Appl. No. 12/101,951, Amendment filed Oct. 30, 2013, 8 pgs.
357 U.S. Appl. No. 12/101,951, Notice of Allowance mailed Dec. 16, 2013, 3 pgs.
358 U.S. Appl. No. 12/101,951, Notice of Allowance mailed Jul. 30, 2013, 5 pgs.
359 U.S. Appl. No. 12/101,951, Notice of Allowance mailed Mar. 26, 2014, 8 pgs.
360 U.S. Appl. No. 12/101,951, Notice of Allowance mailed Sep. 18, 2013, 2 pgs.
361 U.S. Appl. No. 12/101,951, Notice of Allowance mailed Sep. 5, 2013, 2 pgs.
362 U.S. Appl. No. 12/101,951, Office Action mailed Aug. 3, 2010, 26 pgs.
363 U.S. Appl. No. 12/101,951, Office Action mailed Feb. 24, 2012, 28 pgs.
364 U.S. Appl. No. 12/101,951, Office Action mailed Mar. 4, 2011, 25 pgs.
365 U.S. Appl. No. 12/101,951, Office Action mailed Oct. 7, 2011, 28 pgs.
366 U.S. Appl. No. 12/101,951, Petition and Response filed Dec. 16, 2013, 5 pgs.
367 U.S. Appl. No. 12/207,910, Amendment and Response filed Mar. 12, 2012, 13 pgs.
368 U.S. Appl. No. 12/207,910, Amendment and Response filed Sep. 7, 2011, 14 pgs.
369 U.S. Appl. No. 12/207,910, Office Action mailed Dec. 12, 2011, 27 pgs.
370 U.S. Appl. No. 12/359,939, Amendment and Response filed Mar. 11, 2014, 10 pgs.
371 U.S. Appl. No. 12/359,939, Amendment and Response filed Mar. 23, 2012, 11 pgs.
372 U.S. Appl. No. 12/359,939, Amendment and Response filed Oct. 11, 2013, 11 pgs.
373 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Amendment and Response filed Jul. 21, 2011, 8 pgs.
374 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Amendment and Response filed May 23, 2011, 8 pgs.
375 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Amendment and Response filed Nov. 29, 2012, 9 pgs.
376 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Amendment and Response filed Oct. 26, 2012, 11 pgs.
377 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Amendment and Response filed Sep. 28, 2011, 14 pgs.
378 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Office Action mailed Dec. 6, 2011, 14 pgs.
379 U.S. Appl. No. 12/359,939, filed Jan. 26, 2009, Office Action mailed Jan. 21, 2011, 15 pgs.
380 U.S. Appl. No. 12/359,939, Office Action mailed Jan. 2, 2014, 18 pgs.
381 U.S. Appl. No. 12/359,939, Office Action mailed Jul. 17, 2012, 21 pgs.
382 U.S. Appl. No. 12/359,939, Office Action mailed Jun. 17, 2013, 19 pgs.
383 U.S. Appl. No. 12/569,028, Amendment and Response filed Aug. 2, 2013, 17 pgs.
384 U.S. Appl. No. 12/569,028, Amendment and Response filed Dec. 28, 2011, 8 pgs.
385 U.S. Appl. No. 12/569,028, Amendment and Response filed Jan. 15, 2013, 14 pgs.
386 U.S. Appl. No. 12/569,028, Amendment and Response filed Jan. 28, 2014, 13 pgs.
387 U.S. Appl. No. 12/569,028, Amendment and Response filed Jun. 27, 2012, 8 pgs.
388 U.S. Appl. No. 12/569,028, Notice of Allowance mailed Feb. 21, 2014, 8 pgs.
389 U.S. Appl. No. 12/569,028, Office Action mailed Apr. 2, 2013, 21 pgs.
390 U.S. Appl. No. 12/569,028, Office Action mailed Aug. 28, 2013, 21 pgs.
391 U.S. Appl. No. 12/569,028, Office Action mailed Feb. 27, 2012, 11 pgs.
392 U.S. Appl. No. 12/569,028, Office Action mailed Oct. 15, 2012, 14 pgs.
393 U.S. Appl. No. 12/569,028, Office Action mailed Sep. 28, 2011, 14 pgs.
394 U.S. Appl. No. 12/828,508, Amendment and Response filed Jan. 13, 2011, 11 pgs.
395 U.S. Appl. No. 12/828,508, Amendment and Response filed Sep. 6, 2011, 3 pgs.
396 U.S. Appl. No. 12/828,508, Notice of Allowance mailed Jul. 6, 2011, 8 pgs.
397 U.S. Appl. No. 12/828,508, Notice of Allowance mailed Mar. 31, 2011, 9 pgs.
398 U.S. Appl. No. 13/360,536, Office Action mailed Mar. 20, 2014, 14 pgs.
399 US Official Action in U.S. Appl. No. 10/609,315 mailed Dec. 15, 2005, 13 pgs.
400 US Official Action in U.S. Appl. No. 10/609,315 mailed Jun. 1, 2006, 12 pgs.
401 US Official Action in U.S. Appl. No. 10/804,326 mailed Dec. 10, 2008, 7 pgs.
402 US Official Action in U.S. Appl. No. 10/804,326 mailed Jun. 7, 2007, 19 pgs.
403 US Official Action in U.S. Appl. No. 10/804,326 mailed Oct. 16, 2006, 18 pgs.
404 US Official Action in U.S. Appl. No. 10/955,462 mailed May 11, 2007, 26 pgs.
405 US Official Action in U.S. Appl. No. 10/955,462 mailed Nov. 3, 2006, 19 pgs.
406 US Official Action in U.S. Appl. No. 10/955,462 mailed Sep. 10, 2007, 22 pgs.
407 US Official Action in U.S. Appl. No. 10/955,983 mailed Dec. 18, 2008, 29 pgs.
408 US Official Action in U.S. Appl. No. 10/955,983 mailed Jun. 10, 2009, 30 pgs.
409 US Official Action in U.S. Appl. No. 10/955,983 mailed Mar. 22, 2007, 25 pgs.
410 US Official Action in U.S. Appl. No. 10/955,983 mailed Nov. 13, 2007, 27 pgs.
411 US Official Action in U.S. Appl. No. 10/981,962 mailed Apr. 30, 2007, 21 pgs.
412 US Official Action in U.S. Appl. No. 10/981,962 mailed Apr. 5, 2006, 15 pgs.
413 US Official Action in U.S. Appl. No. 10/981,962 mailed Mar. 17, 2008, 20 pgs.
414 US Official Action in U.S. Appl. No. 10/981,962 mailed Sep. 21, 2006, 16 pgs.
415 US Official Action in U.S. Appl. No. 11/019,091 mailed Apr. 3, 2008.
416 US Official Action in U.S. Appl. No. 11/019,091 mailed Dec. 11, 2008, 24 pgs.
417 US Official Action in U.S. Appl. No. 11/019,091 mailed Jun. 20, 2007.
418 US Official Action in U.S. Appl. No. 11/019,091 mailed Sep. 1, 2009, 26 pgs.
419 US Official Action in U.S. Appl. No. 11/073,381 mailed Apr. 12, 2010, 25 pgs.
420 US Official Action in U.S. Appl. No. 11/073,381 mailed Apr. 15, 2009, 20 pgs.
421 US Official Action in U.S. Appl. No. 11/073,381 mailed Feb. 23, 2011, 27 pgs.
422 US Official Action in U.S. Appl. No. 11/073,381 mailed Jul. 10, 2008, 19 pgs.
423 US Official Action in U.S. Appl. No. 11/073,381 mailed Sep. 13, 2010, 24 pgs.
424 US Official Action in U.S. Appl. No. 11/073,381 mailed Sep. 18, 2007, 17 pgs.
425 US Official Action in U.S. Appl. No. 11/073,381 mailed Sep. 29, 2009, 21 pgs.
426 US Official Action in U.S. Appl. No. 11/206,286 mailed Dec. 24, 2008, 16 pgs.
427 US Official Action in U.S. Appl. No. 11/206,286 mailed Jul. 14, 2008, 15 pgs.
428 US Official Action in U.S. Appl. No. 11/238,906 mailed Dec. 18, 2009, 21 pgs.
429 US Official Action in U.S. Appl. No. 11/238,906 mailed Jan. 8, 2008, 18 pgs.
430 US Official Action in U.S. Appl. No. 11/238,906 mailed May 19, 2009, 20 pgs.
431 US Official Action in U.S. Appl. No. 11/238,906 mailed Sep. 16, 2008, 17 pgs.
432 US Official Action in U.S. Appl. No. 11/412,723 mailed Mar. 11, 2010, 20 pgs.
433 US Official Action in U.S. Appl. No. 11/412,723 mailed Mar. 6, 2009, 22 pgs.
434 US Official Action in U.S. Appl. No. 11/412,723 mailed May 28, 2008, 22 pgs.
435 US Official Action in U.S. Appl. No. 11/412,723 mailed Sep. 3, 2009, 20 pgs.
436 US Official Action in U.S. Appl. No. 11/874,579 mailed Jan. 14, 2011, 23 pgs.
437 US Official Action in U.S. Appl. No. 11/874,579 mailed Jun. 22, 2010, 23 pgs.
438 US Official Action in U.S. Appl. No. 11/874,844 mailed Nov. 13, 2009, 14 pgs.
439 US Official Action in U.S. Appl. No. 12/207,910 mailed Jun. 7, 2011, 30 pgs.
440 US Official Action in U.S. Appl. No. 12/828,508 mailed Aug. 13, 2010, 16 pgs.
441 US Official Action in U.S. Appl. No.10/955,983 mailed Jul. 21, 2008, 28 pgs.
442 Utiyama, Masao et al., "Implementation of an IR package", IPSJ SIG Notes, vol. 2001, No. 74 (2001-FI-63-8), pp. 57-64, Information Processing Society of Japan, Japan, Jul. 25, 2001. (not an English document).
443 Voorhees, E., "Overview of TREC 2002", Gaithersburg, Maryland, Nov. 19-22, 15 pp.
444 Web Page "Reuters: Reuters Corpus", http://about.reuter.com/researchandstandards/corpus/, viewed Mar. 18, 2004.
445 Wen, JI-Rong, "Query Clustering Using User Logs", Jan. 2002, pp. 59-81.
446 Westerveld, T. et al., "Retrieving Web pages using Content, Links, URLs and Anchors", Proceedings of the Tenth Text Retrieval Conference, NIST Special Publication, ′Online! Oct. 2001, pp. 1-10.
447 Wilkinson, R., "Effective Retrieval of Structured Documents", Annual ACM Conference on Research and Development, 1994, 7 pp.
448 Xue, Gui-Rong et al., "Optimizing Web Search Using Web Click-Through Data," http://people.cs.vt.edu/˜xwensi/Publication/p118-xue.pdf, CIKM'04, Nov. 8-13, 2004, 9 pp.
449 Yi, Jeonghe,e et al., "Metadata Based Web Mining for Topic-Specific Information Gathering", IEEE, pp. 359-368, 2000.
450 Yi, Jeonghee, et al., "Using Metadata to Enhance Web Information Gathering", D. Suciu and G. Vossen (eds.): WebDB 2000, LNCS 1997, pp. 38-57, 2001.
451 Yuwono, Budi and Lee, Dik L., "Search and Ranking Algorithms for Locating Resources on the World Wide Web", IEEE, 1996, pp. 164-170.
452 Zamir, O. et al., "Grouper: A Dynamic Clustering Interface to Web Search Results", Computer Networks (Amsterdam, Netherlands: 1999), 31(11-16): 1361-1374, 1999.
US-Klassifikation 707/748, 715/215, 707/723, 715/259, 707/727, 707/749, 707/758
Unternehmensklassifikation G06F17/30, G06F17/00, G06F17/30864, G06F17/30657
2. Juni 2010 AS Assignment
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TANKOVICH, VLADIMIR;MEYERZON, DMITRIY;POZNANSKI, VICTOR;SIGNING DATES FROM 20100526 TO 20100601;REEL/FRAME:024475/0413