Source: http://www.google.ca/patents/US8706735
Timestamp: 2017-10-17 20:41:43
Document Index: 231677031

Matched Legal Cases: ['Application No. 09', 'Application No. 09', 'Application No. 09175979', 'Application No. 09175979', 'Application No. 09175979', 'Application No. 09175979', 'Application No. 09180776', 'application No. 10167947']

Patent US8706735 - Method and system for indexing and searching timed media information based ... - Google Patents
A method and system for indexing, searching, and retrieving information from timed media files based upon relevance intervals. The method and system for indexing, searching, and retrieving this information is based upon relevance intervals so that a portion of a timed media file is returned, which is...http://www.google.ca/patents/US8706735?utm_source=gb-gplus-sharePatent US8706735 - Method and system for indexing and searching timed media information based upon relevance intervals
Publication number US8706735 B2
Application number US 13/955,582
Also published as US7490092, US8117206, US8527520, US9244973, US9542393, US20050216443, US20090125534, US20120117078, US20130318121, US20140250131, US20160098396
Publication number 13955582, 955582, US 8706735 B2, US 8706735B2, US-B2-8706735, US8706735 B2, US8706735B2
Inventors Michael Scott Morton, Sibley Verbeck Simon, Noam Carl Unger, Robert Rubinoff, Anthony Ruiz Davis, Kyle Aveni-Deforge
Original Assignee Streamsage, Inc.
Patent Citations (162), Non-Patent Citations (53), Referenced by (1), Classifications (15), Legal Events (3)
US 8706735 B2
determining, by at least one computer, based on a timed media file, data representing at least a word or phrase; and
using, by the at least one computer, the data to determine a first portion of the timed media file based on the word or phrase.
2. The method of claim 1, further comprising determining a plurality of occurrences of the word or phrase in the timed media file, wherein said using comprises using the data to determine the first portion such that the plurality of occurrences are within beginning and end points of the first portion.
using, by the at least one computer, the data to determine a plurality of occurrences of the word or phrase in the timed media file; and
determining, by the at least one computer, a first subset and a different second subset of the plurality of occurrences,
wherein said using comprises using the data to determine the first portion such that the first subset of the plurality of occurrences, and not the second subset of the plurality of occurrences, are within beginning and end points of the first portion.
4. The method of claim 3, further comprising using, by the at least one computer, the data to determine a second portion of the timed media file having a beginning point and an end point such that the second subset of the plurality of occurrences, and not the first subset of the plurality of occurrences, are within the beginning and end points of the second portion.
5. The method of claim 1, wherein said using the data to determine the first portion comprises determining the first portion to have at least a minimum length.
6. The method of claim 1, wherein the data represents at least a plurality of words or phrases, the method further comprising using, by the at least one computer, the data to determine a plurality of portions of the timed media file, one for each of the plurality of words or phrases, wherein the plurality of portions overlap each another within the timed media file.
7. The method of claim 1, wherein the data represents at least a plurality of words or phrases, the method further comprising:
using, by the at least one computer, the data to determine a plurality of portions of the timed media file, one for each of the plurality of words or phrases, wherein at least some of the plurality of portions are not contiguous with any others of the plurality of portions within the timed media file; and
generating, by the at least one computer, another timed media file comprising the plurality of portions, such that each of the plurality of portions is contiguous with another of the plurality of portions within the another timed media file.
8. The method of claim 1, wherein the timed media file comprises video, and said determining the data comprises determining the data using optical character recognition.
9. The method of claim 1, wherein the timed media file comprises audio, and said determining the data comprises determining the data using speech recognition.
10. The method of claim 1, wherein the timed media file comprises metadata, and said determining the data comprises determining data based on at least the metadata.
receiving, by at least one computer, a query;
determining, by the at least one computer, based on the query, a beginning point and an end point of each of a plurality of portions of first timed media data, wherein at least some of the portions are not contiguous with any others of the plurality of portions within the first timed media data; and
generating, by the at least one computer, second timed media data comprising the plurality of portions, such that each of the portions is contiguous with another of the plurality of portions within the second timed media data.
12. The method of claim 11, wherein said determining comprises determining the portions such that at least some of the portions overlap each other within the first timed media data.
13. The method of claim 11, wherein the timed media data comprises video, and said determining comprises determining at least some of the plurality of portions using optical character recognition.
14. The method of claim 11, wherein the timed media data comprises audio, and said determining comprises determining at least some of the plurality of portions using speech recognition.
15. The method of claim 11, wherein the timed media data comprises metadata, and said determining comprises determining at least some of the plurality of portions based on at least the metadata.
receiving, by at least one computer, a search term;
determining, by the at least one computer, based on the search term, a beginning point and an end point of each of a plurality of portions of first audio and video data, wherein at least some of the portions are not contiguous with any others of the portions within the first audio and video data; and
generating, by the at least one computer, second data comprising the plurality of portions, such that each of the portions is contiguous with another of the portions within the second data.
17. The method of claim 16, wherein said determining comprises determining the portions such that at least some of the portions overlap each other within the first audio and video data.
18. The method of claim 16, wherein said determining comprises determining at least some of the plurality of portions using optical character recognition.
19. The method of claim 16, wherein determining comprises determining at least some of the plurality of portions using speech recognition.
20. The method of claim 16, wherein the audio and video data further comprises metadata, and said determining comprises determining at least some of the plurality of portions based on at least the metadata.
This application is a continuation of U.S. patent application Ser. No. 13/347/914, filed Jan. 11, 2012 (U.S. Pat. No. 8,527,520), which is a continuation of U.S. patent application Ser. No. 12/349/934, filed Jan. 7, 2009 (U.S. Pat. No. 8,117,206), which is a continuation of U.S. patent application Ser. No. 10/364,408 (U.S. Pat. No. 7,490,092), filed Feb. 12, 2003, which is a continuation-in-part of U.S. patent application Ser. No. 09/611,316 (abandoned), filed Jul. 6, 2000. This application further claims priority to U.S. Provisional Patent Application Ser. No. 60/356,632, filed Feb. 12, 2002, all hereby incorporated by reference as to their entireties.
A portion of this invention was made with United States Government support under ATP Award #70NANB1H3037 awarded by the National Institute of Standards and Technology (NIST).
Traditionally, search engines create a large table that is indexed by words, phrases, or other information such as hyperlinks. Each word or phrase points to documents that contain it. The pointing is rated by a relevance magnitude that is calculated by some algorithm, typically including information such as the frequency with which the word or phrase appears and whether it occurs in the title, keywords, etc. Advanced search engines augment the foregoing system by adding the capability to check synonyms or by letting the user indicate the intended definition of the word in question, either by choosing it manually or by entering a natural language query. Other functions are plentiful, such as putting the words searched for in bold in an HTML document or organizing the returned results into customized folders, as is done by Northern Light®.
Referring to FIG. 1, the timed media indexing system and method for searching and retrieving timed media according to the present invention is illustrated. A timed media file is entered into the system in step 20. In step 21, data associated with the timed media file may be entered into the system. This data can include meta-data, such as the title or subject of the timed media file; descriptive information; categorization of the media file according to genre (e.g. news, presentation, instructional content), number of speakers, or expected use; text on subject matter closely related to the timed media file; HTML from a Web page that is associated with or that includes the timed media file; and other types of data. In step 22, the system then extracts data from the timed media file and the associated data. The extracted data can include spoken words, speech and sound events or parameters, on-screen text, meta-tag information, and other types of data. The extracted data is then analyzed in step 24 using natural language processing, conceptual reasoning, logical structure analysis, and other techniques. The results of the analysis are saved in a raw data index in step 26, so that users can access the raw data for highly accurate multi-information representation queries and for the creation of customized or updated search indices at a later date. In some embodiments of the current invention, step 26 may not be necessary, as in the case where only the need for rapid results is anticipated and it is not intended to customize the search index or update the relevance interval calculation at a later date. In step 28, relevance intervals and their associated magnitudes of relevance are calculated for each information representation, and the relevance intervals and the corresponding magnitudes of relevance are stored in the search index in step 30, along with other calculated data that can be used to adjust the intervals for a given user or calculate relevance intervals for complex queries.
The term information representation, as used herein, denotes an indicator of material relevant to a concept, search term, or more complex query. An information representation, therefore, can comprise any single item or combination of particular definitions of particular words, parts of speech, words, phrases, sentences, grammatical structures, and linguistic objects. An information representation need not be a complete sentence. It is expected that an information representation would not be a single article, preposition or conjunction since such parts of speech, absent a connection to other parts of speech, do not convey meaning. For example, the word “the” is one of the most commonly occurring words in the English language. A search for “the” alone would yield meaningless results. On the other hand, an information representation such as in the spoken phrase “the Great Depression” uses the article “the” in connection with the adjective “great” and the noun “depression,” forming a concept that is distinct from any combination of one or two of its three words. Such an information representation would be much more likely to provide useful relevance intervals. Similarly, the co-occurrence of the phrase “the depression” with the phrase “the nineteen thirties” is likely to be an information representation with the same meaning, even though these phrases may be separated by other words and independently they can have other meanings. In the context of search and retrieval, query information representations comprise indicators of that which is sought in a query, and information representations within texts and timed media files indicate the presence of that which is relevant to the search and should be returned.
In addition, the output of the sentence segmentation module 96 is used by the named entity identification module 107 to identify proper nouns and determine which proper nouns within a media file refer to the same entity. The name co-references determined by module 107 are used, along with the output of the centrality calculation module 100, to resolve direct and indirect anaphora (including pronouns, definite references such as “the company,” and indirect references such as “the door” following a discussion of a house) in module 108.
FIG. 6 shows a screen 240 that is presented to a user to perform a search using the system of the present invention. The screen 240 includes a query input space 242, which in FIG. 6 contains the input query “science legislation Shelby amendment HR88.” Below the input query space 242 are boxes 244 and 246 through which the user designates the type of timed media files to be searched, i.e. audio files and/or video files, respectively. Next the user can designate how the results are returned, either entire documents only or including partial documents, by marking box 248. The user can further elect a general topic search, via box 250 or a specific information search via box 252. Finally, the user selects the degree of accuracy by choosing the time for the search via boxes 254. The more time selected, the more accurate the search.
The results may be displayed in a screen 308 as shown in FIG. 7. The results can be sorted by document or by date. In the example shown in FIG. 7, the search returned 15 results, of which two are shown in the window. Each result includes a relevance magnitude, information describing the timed media file, sample text from the relevant portion of the file, the time intervals of the relevance intervals, and the date. For each result, a box 310 is provided that allows the user to combine that result with others that are checked into one large file for playback. The playback of the combined results is achieved by accessing the “play checked” box 320. Alternately, the user can select any individual interval listed in the time interval section 330 or the full document. Instead of playing the timed media file, the user can access a text transcript of the timed media file or a full summary 334, both of which are automatically created by the system. The results display screen 308 also allows the user to manipulate the results by clicking on the “find similar” indicator 340 to sort the results so that results similar to the selected result are displayed.
For visual logical object information representations, which are the output of visual logical structure analysis module 94 that parses outlines, headlines, etc. from PowerPoint® slides or typed overhead projections, the raw data index 114 includes data such as the logical object number, the time-code at which the information representation occurs in the file, and the logical tree information.
The lemmatization module 98 reduces words such as nouns, verbs, adjectives, and adverbs to a canonical form. This function allows the system to identify that “mouse” and “mice” are the singular and plural of the same concept, and should thus be treated as occurrences of the same or very related information representations. Using the canonical form of each word in the raw data index 114 and the search index 120 also allows the system to respond to queries with virtual documents constructed for alternative forms of a given word or phrase.
“We are now in a new era. To label this time ‘the post-Cold War era’ belies its uniqueness and its significance. We are now in a Global Age. Like it or not, we live in an age when our destinies and the destinies of billions of people around the globe are increasingly intertwined.”
A simple example of centrality analysis can be seen in the fact that the proper noun “Cold War” is used as an adjective. This indicates that the Cold War may not be the central topic of this portion of the speech. Further evidence includes the fact that “the post-Cold War era” is itself the direct object of the infinitive “to label”. Altogether this evidence suggests that the relevance of this information representation probably does not, in the absence of other occurrences or semantic information to the contrary, span beyond the passage quoted.
The tokenization module 102 filters the text extracted from the media to leave only those words that are by themselves units of semantic meaning. For example, the tokenization module 102 filters out determiners such as “the” and “a” because they do not individually carry semantic information. The term token, as used herein, refers to a meaningful word unit, and excludes words such as conjunctions, articles, and prepositions that have been filtered out by the tokenization module 102.
N is the number of words in the corpus,
#A is the number of occurrences of the pair A in the corpus, and
#A&B is the number of times the pair A occurs within k tokens of the pair B.
In addition, this formula is further modified to take into account a weighting factor. Because the grammatical structure of sentences can help indicate the likelihood that a word is semantically related to most of the surrounding context, each occurrence of a pair in the corpus is weighted according to its centrality, as determined by the centrality calculation module 100. The mutual information score then becomes
Let the “centrality sum” CS(A)=Σi(cent(Ai)), where the sums are over all occurrences within the corpus and cent(Ai) is the centrality of the ith occurrence of A in the corpus, and similarly let CS(A,B)=Σi(cent(Ai)*cent(B)) where the sum is over co-occurrences within the corpus. Then, MI(A,B)=(N/2k)[CS(A,B)]/[CS(A)*CS(B)].
Let Tabs and Trel be constants.
1) If A and B have no co-occurrences, MI(A,B)=min[(N/4k)(1/CS(A,B), 1].
2) If CS(A)*CS(B)<(N/2kTabs) and CS(A,B)<1/Trel then MI(A,B)=1.
In the above adjustments, the constants Tabs and Trel are used to set the maximum absolute and relative uncertainty in the mutual information score, respectively. In the preferred embodiment of the present invention, Tabs is set at 10, meaning that the mutual information scores are within 10 of its true value to a high degree of confidence and Trel is set to 0.2, meaning that the mutual information scores are within 20% of their true value to a high degree of confidence. When these criteria are not met, setting the mutual information to 1 is equivalent to stating that we do not have any knowledge of the correlation between the words beyond random chance.
Referring to FIG. 11, the first step in topic segmentation is the pair-wise comparison of sentences 152 performed on the tokenized text 150 that has been extracted via speech recognition 80. This step creates an n×n matrix of relatedness values for each sentence with each other sentence, where n is the number of sentences within the tokenized text 150. Each of these relatedness values is in turn calculated from an i×j matrix, where i is the number of words in the first sentence and j is the number of words in the second of the two sentences being compared. Note that because the file has been tokenized, i and j are the number of tokens in their respective sentences, which is probably different than the number of words contained within these sentences before tokenization. This i×j matrix contains relatedness values for all pairs of words in which one of the words is contained in the first sentence and the other word is contained in the second sentence. In the preferred embodiment of the present invention, the relatedness value between two words a and b, contained in sentence A and sentence B respectively, is
(MI−1)/(MI−1+C)
From the i×j matrix of word-relatedness values, the relatedness value for the pair of sentences being compared is calculated via the following steps. Let L1 be the largest word-pair relatedness value in the i×j matrix of word-relatedness values. Similarly, let L2 be the largest word-pair relatedness value in the matrix that does not involve either of the words in the pair that have relatedness L1 and let Lm be the largest word-pair relatedness value in the matrix that does not involve any of the words in the pairs that have relatedness L1, . . . Lm−1. Continue to determine values of Lm until either of the sentences no longer has additional words to consider (and thus n will equal the lesser of i or j). Then, let the relatedness value of the two sentences being considered be
Σn L n/√(i×j)
where i and j are the number of words in the two sentences.
In this way, step 152 creates V, the n×n matrix of relatedness values between pairs of sentences in the text. Next, the topic segmentation module 106 performs the calculation of the rank matrix 154. This step creates R, an n×n matrix calculated from V. In particular,
For each element vi,j in V, look at the m×m sub-matrix that is centered on vi,j, where m is an odd number. In other words, look at the elements va,b where a ranges from i−(m−1)/2 to i+(m−1)/2 and b ranges from j−(m−1)/2 to j+(m−1)/2;
Let ri,j=(the number of elements in the said sub-matrix that are <vi,j)/the number of elements considered in the matrix (not counting vi,j itself).
The calculation of the rank matrix 154, thus replaces the quantitative measurement vi,j with a qualitative one ri,j that depends upon the relatedness as compared to its local environment. Dividing by the number of elements within the sub-matrix properly treats the boundary cases near the edges of V and R. In the preferred embodiment of the present invention, values of M between 9 and 15 have been found to be most useful with no discernable difference between them.
1. rsi,i=ri,i for each i in {1, . . . , n};
2. rsi+1,i=2ri+1,i+rsi,i+rsi+1,i+1
rsi,i+1=rsi+1,i
for each i in {1, . . . , n−1};
3. rsi+k,i=2ri+k,i+rsi+k−1,i+rsi+k,i+1−rsi+k−1,i+1
rsi,i+k=rsi+k,i
for each k in {2, . . . , n−1}, for each i in {1, . . . , n−k}.
Let B={b(1), . . . , b(k−1)} be the sequence of sentence numbers that indicate which sentences have been identified as topic boundaries by the first k−1 iterations of steps 156-160 (where the values b(n) are in order from least to greatest). For example, if the 27th sentence was found to be the first sentence of a new topic segment in the first iteration and the 5th sentence was found to be the first sentence of a new topic segment in the second iteration, B would equal {5, 27} during the third iteration. At step k,
D=[Σk i=1(rsb(i),b(i−1)+1)]/[Σk i=1(ab(i),b(i−1)+1)] where b(0)=0.
Let size=(M−1)/2 where M is the constant used in calculating the rank matrix R in step 154.
Let WithinV(n)=Σ(Vi,j) where either (n−size)<i,j≦n or n<i, j≦(n+size).
Let CrossV(n)=Σ(Vi,j) where either (n-size)<i≦n and n<j≦(n+size) or vice versa.
BR(n)=WithinV(n)−CrossV(n).
Once this boundary rating BR(n) has been calculated for all of the boundaries that have been identified thus far, the boundary rating step 160 then computes a list of local maxima by calculating the boundary rating for segments near identified boundaries. These local maxima are used to adjust and to filter the identified boundaries. A boundary at sentence n is considered a local maximum if the following three conditions hold:
1) BR(n)>f1*(average boundary rating);
2) BR(n)>BR(i) for (n−size)≦i≦(n+size) except those values of i for which there is a j between i and n for which BR(j)<(f2*BR(i));
3) BR(n)>BR(i) for (n−f3)≦i≦(n+f3).
In the preferred embodiment of the present invention, the values of these constants are f1=1.05, f2=0.85, f3=2.
Once the local maxima have been calculated, this set of sentences is used to adjust and to filter the identified boundaries. Each boundary that has been identified by the boundary insertion step 158 is compared to the set of local maxima. If it matches a local maximum, it is accepted. If it is not a local maximum but it is within two sentences of a local maximum, then that boundary is changed to the local maximum. If there is not a local maximum within two sentences of the boundary, then that boundary is rejected by excluding it from the eventual output of the module.
There have been more than d1 unique boundaries identified by step 158 (these must be unique boundaries because a sentence may be selected as a boundary more than once due to adjustment or rejection) AND EITHER
The percentage of rejected boundaries is >d2, OR
d3 unique boundaries in a row are rejected.
The named entity identification module 107 identifies named entities, classifies the entities by type, such as people, places, organizations, and events, and determines whether multiple instances of named entities are referring to the same entity (e.g. “Bill Clinton” and “the former president”). The operation of the named entity identification module 107 of the present invention is illustrated in FIG. 12.
The name-word identifier 272 reads in the parsed text 270 that was produced by the extraction modules 78 and parsed by the parsing module 92. It then checks the text against the named entity lists 274 and tags those words that may be part of named entities. The named entity lists 274 include lists of words and phrases that are often a part of named entities (e.g. “Robert” and “United States”). These lists also indicate the type of named entity that each word or phrase is a part of as well as qualitative or quantitative indications as to the likelihood that the given word or phrase is a named entity (e.g. the spoken word “Bill” could be the first name of a human, a legislative act, a part of a duck, a piece of currency, or an invoice). These likelihoods may vary according to register (e.g. content from the US Senate vs. the Audubon Society), and thus multiple values may be recorded and selected according to assumptions by the system or instructions by the system administrator.
For example, the words “Mrs. Sue Clayton” would be analyzed by the identifier and classifier 276 and identified as referring to a single named entity, in this case a person. “Sue” would have been tagged as a possible human first name and “Clayton” would have been tagged as a human name that could be either a first or last name. The fact that “Sue” precedes “Clayton” increases the likelihood that “Sue” is a first name and “Clayton” is a last name, but these two words could also be referring to legal action against a human with the first or last name of “Clayton” (recall that capitalization is not known for spoken words). Using the prefix/suffix lists 278, however, the identifier and classifier 276 would recognize that “Mrs.” is a title of a female human that requires that it be followed by at least one name, the last of which is a human last name. In this example, the identifier and classifier 276 would also check for the presence of other prefixes and suffixes contained in the prefix/suffix lists 278 to exclude cases such as “the Mrs. Sue Clayton Memorial Scholarship” that would indicate that the named entity be further expanded to include the words “memorial scholarship” and be reclassified as a proper noun (or perhaps more specifically as a scholarship or financial entity) rather than as a human.
Once the named entities have been fully identified and classified, the co-reference identifier 280 inspects each named entity to determine whether it refer to the same entity as other named entities identified within the media file. For example, “Ms. Amanda Hourihan” may be referred to later within the file as “Ms. Hourihan” or simply “Amanda”. The co-reference identifier 280 applies co-reference lists and rules 282 to know that human first names occurring after human first/last name pairs often refer to the same person, while analogous cases with other types of named entities do not follow such a pattern, such as “agriculture” occurring after “Agriculture Department.” Similarly, the co-references lists and rules 282 indicate variations within words that are equivalent, such as “Mandy” referring to the same person as “Amanda.”
Some of the co-reference rules 282 make use of topic boundaries, and thus the co-reference identifier 280 also uses the topic boundaries 164 that have been identified within the media file. For example, “Amanda” probably does not refer to “Ms. Amanda Hourihan” if “Amanda” occurs in a separate news story from “Ms. Amanda Hourihan”. Once all co-references have been identified, the co-reference identifier 280 produces the output of the named entity identification module 107, namely the co-reference table 284. This table includes the named entities identified, classified, and grouped according to the entity to which they refer.
The anaphora resolution module 108 adds to the named entity co-reference table 284 by identifying antecedents for all types of anaphora, including pronouns, definite references, and indirect references. FIG. 13 illustrates the operation of the anaphora resolution module 108. The module first identifies anaphora in step 292. This step reads in parsed text that includes centrality numbers, as represented by block 290. Step 292 identifies pronouns that are likely to be anaphora with antecedents, thus excluding, for example, non-referential uses of “it”, “that,” and “there” such as “I was told that . . . .” Step 292 also identifies potential definite and indirect references such as “the company” which may refer to a specific company previously mentioned.
Potential antecedents for a given anaphor are located by stepping backward through the text and by looking at the named entities contained in the named entity co-reference table 284. Personal pronoun resolution in step 296 filters the potential antecedents according to whether they can represent a human, a group of humans, or a gendered non-human, as well as by number. For example, “she” cannot generally refer to “waiter,” “John,” “Mr. Lewis,” or “waitresses.” Some of this filtration makes use of an ontology in order to recognize, for example, that a “waiter” is an entity that is usually a male human.
Impersonal pronoun resolution in step 298 also uses the ontology to filter potential antecedents according to number. Step 298 also filters potential antecedents according to semantic constraints on the pronouns that can be detected by analyzing the sentence structure. For example, in resolving the pronoun “it” that occurs in the phrase “it runs,” step 298 recognizes that because “it” is the subject of the verb “runs,” then it must represent an entity that can run, such as a human, animal, human organization, machine, event, liquid, linear object, etc. Something that does not run, such as “the sun” is therefore not a potential antecedent and can be ignored.
The non-pronominal anaphora resolution step 300 resolves references that are not pronouns. This might include the definite reference “the ship,” which might refer to “the Titanic” which occurs previously within the media. Step 300 makes use of the named entity co-reference table 284 and the ontology 294 to identify that entities such as “the Titanic” are examples of the definite reference in question, in this case “the ship.”Step 300 resolves indirect anaphora as well. These include part/whole references, as in the sentences “This was the first time I had built a house of classical architecture. The awning is constructed of . . . ” where “the awning” makes an implicit reference to the “house” of which it is a part. In other words, it is the awning of a very particular house that is the subject of the second sentence in the passage, and thus part of the subject of the sentence is assumed from the previous sentence. Such relationships are not strictly co-references, but they are references that convey continuation of topic and are therefore useful to the system.
The hierarchical list of rules is kept in a separate text file that is compiled into code automatically by the system so that experts can easily adjust and add to the rule base. Useful rules include those that locate: summary or conclusion statements (e.g. “In conclusion . . . ”); indications of a particular logical relationship with the immediately previous text (e.g. “On the other hand . . . ”); indications of the presence of, or a specific role within, a list (e.g. “Lastly . . . ” or “There are three reasons why . . . ”); explicit mentions of the main topic of a segment of the media (e.g. “Let's now address the issue of . . . ”); identifications of the subsequent, current, or past speaker (e.g. “Now that we've finished that section of the meeting, I'll turn things over to Roger . . . ”); and numerous other structures.
Once a set of simultaneously visible text is divided into logical objects, a hierarchical structure to the logical objects is discerned by analyzing the position on the page, size, indentation, alignment, and other characteristics of the logical objects. The hierarchical structures from multiple sets of simultaneously visible logical objects (such as multiple slides in a presentation) are then compared for logical connections. When connections are identified, the hierarchies are joined. For example, if one slide has the title “Video Analysis Software” and the following slide has the title “Video Analysis Software, cont.”, the second slide is clearly a continuation of the first. If there were a bulleted list of points on each of these two slides, then the two lists could be joined in a single hierarchy under the equivalent title. Other examples of connections include the continuation of outlines and table of contents slides. In addition, the time interval during which each logical object is visible within the media is recorded and associated with that logical object.
To illustrate, consider an information representation that is located in a certain logical object on a PowerPoint® slide that is visible during a particular portion of a timed media file. Often a speaker will address that logical object while the slide is visible. By comparing the words in the logical object with spoken words, the system can identify the time interval of the timed media that is most relevant to the logical object, which can then be used to adjust the length of the relevance interval. For example, if the visible logical object is a minor detail in a list, then it may be sufficient for the relevance interval to include a small portion of timed media, including the time when the speaker mentions the detail, rather than the entire time the information representation is visible on the screen. A natural extension of this adjustment process would be to create a feedback loop between the calculation of relevance intervals in step 116 with this adjustment of time intervals associated in logical objects in step 112.
FIG. 14 depicts the operation of the relevance interval calculation module 116. These steps are operated for each unique information representation that has been identified within the media file. These steps operate for a given information representation (called the “indexing term”) as follows: in the first step, occurrence identification step 400, the module locates every occurrence of the indexing term within the media file by time-code. For spoken occurrences of the indexing term, the list of occurrences includes the time interval of the sentence that includes the spoken occurrence. For visual occurrences of the indexing term, the list of occurrences includes the intervals of time that have been associated with the visual occurrence by the temporal logical structure analysis module 112.
1) At least one block of sentences between the nth block and the (n-maxSkipped)th block preceding the interval (inclusive) is already a part of the interval, either because it was a part of the interval as defined by step 402 or because it has become part of the interval already in step 404.
2) The product P and the largest mutual information between an information representation within the block of sentences and the indexing term is greater than globalConstant^n.
Similarly, step 404 expands the end of each interval by performing the analogous steps proceeding forward through the timed media file. In the preferred embodiment of the present invention, the constants maxSkipped and globalConstant are 3 and 1.95 respectively. After completing the expansion based upon the mutual information model 115, the system once more joins adjacent or overlapping intervals.
1) The interval is less than maxExpansion seconds from the topic boundary in question.
2) The interval is less than [expansionMultiple*Length+expansionConstant] seconds from the topic boundary in question, where Length is the length of the interval.
1) There is a gap less than maxExpansion seconds between the two consecutive intervals.
2) There is a gap of less than [expansionMultiple*Length+expansionConstant] seconds between the two consecutive intervals, where Length is the length of the longer interval.
For example, given information representations A and B and relevance intervals <A 1> and <B1> for each information representation A and B respectively, if
1. B is an example of A, given their grammatical and lexicographic context within the file (such as B=“natural gas” and A=“petroleum”),
2. <A1>⊃<B1>, and
3. <B1> is “almost all” of <A1>,
then <B1> may be adjusted to equal <A1>.
After calculating relevance intervals in steps 400-410, in step 412 the relevance interval calculation module 116 calculates a relevance magnitude for each relevance interval for each indexing term. These magnitudes, much like the relevance numbers associated with indexed static text documents, are used to rank the search results in an approximate order of usefulness, thereby greatly enhancing the usefulness of the search results. In addition, the magnitudes are used to make decisions concerning the combination of relevance intervals into virtual documents and creation of merged virtual documents in response to multiple information representation queries.
Y=max (1, RICS(anaphora that refer IT, R)),
AMI is a relevance interval-wide measurement of mutual information explained below, and
R is the relevance interval in question.
In other words, Z is a modified version of a count of the number of occurrences of the indexing term within the relevance interval. The count is weighted by the centrality of each occurrence in order to make it a truer measure of the degree to which the information representation “occurs” within the media. The maximum of this weighted count and 1 is used because in some relevance intervals that contain very few occurrences, the count may well be less than 1, which would make logZ negative in the definition of MI. While logZ is used in part because a) there is diminishing relative value for the purpose of judging relevance in each additional occurrence of the indexing term and b) the existence of the first occurrence is of little value because of the error-prone nature of speech recognition, it is not useful to actually decrease the magnitude of intervals that have a weighted occurrence sum of less than 1. Similarly, Y represents a weighted count of anaphoric occurrences of the indexing term.
where the average is over all of the sentences in the relevance interval, and MMI is the maximum mutual information score between the indexing term and an information representation that is within the given sentence in the relevance interval but not a noun separated by fewer than two words from an occurrence of the indexing term itself.
The exclusion of nouns very nearby an indexing term eliminates unnaturally high mutual information values that stem from very common complete phrases of which the indexing term is a part. Averaging the maximum mutual information values from each sentence in the media then gives a sense of the overall connectedness of the relevance interval to the indexing term.
1) The statistical similarity of information representations in the intervals. In particular, if information representations that are not very common and are not relevant to the entire file are located in both relevance intervals, then there is a higher likelihood of a contextual similarity between the relevance intervals. Another way of making the same comparison is to examine the number of other information representations that have relevance intervals that are closely synchronized to both of the intervals in question. If a number of information representations have similar sets of relevance intervals, it is very likely that the content between their relevance intervals is related.
2) Rules involving relevance intervals and the output of the natural language processing of various parts of the system. As in the case of the construction of relevance intervals, some embodiments of the present invention apply specialized language rules involving relevance intervals that can be invoked. For example:
Let <B1> and <B2> be relevance intervals for the information representation B.
Let F<Bi>≡{information representations W|∃<Wj>⊃<Bi> and B is an example of W}
Then, if F<B1> is sufficiently similar to F<B2>, <B1> and <B2> should be a part of the same virtual document.
3) Rules involving knowledge about the media file. For example, if it is known that the indexed file is a recording of a single event, such as a hearing or a press conference, then it is likely that all of the intervals from a media file should be combined to give the viewer all of the information pertaining to the indexing term from that event. If the media is a lengthy stream of news broadcasts, however, then it is much less likely that relevance intervals from separate news stories or exposés should be combined simply because they are saved in the same media file.
4) Analysis of the relevance and quality of the relevance intervals. The relevance of relevance intervals can vary dramatically. This is especially true in the case where speech recognition is used to determine the spoken text contained in the media file, as a relevance interval may be created entirely based upon a speech recognition error and therefore have no connection whatsoever with the indexing term. If such intervals were combined with highly relevant intervals to form a virtual document, the virtual document may still be assigned a very high relevance magnitude, and yet the irrelevant relevance interval would significantly degrade the user experience.
Let maxBadRelevance and minGoodRelevance be constants.
i) At least one relevance interval in the media file for the given indexing term has a relevance magnitude>minGoodRelevance and
ii) The relevance interval in question has a relevance magnitude<maxBadRelevance.
In the preferred embodiment of the present invention, the values of maxBadRelevance and minGoodRelevance are 2.7 and 4 respectively.
The system then performs the process referred to in FIG. 5 for increasing the accuracy for compound searches in step 560. This includes querying the search index 120 for relevance intervals for each query information representation in step 562, and calculating relevance intervals for the multi-information representation search in step 564. Finally the system returns the results to the user in step 570, which may include providing multiple links per timed media file, in block 572.
1) each interval in the merged virtual document contains at least one occurrence of each query information representation;
2) the start time of each interval in the merged virtual document is the start time of a relevance interval for one of the query information representations; and
3) the end time of each interval in the merged virtual document is the end time of a relevance interval for one of the query information representations.
A magnitude is calculated for each interval in the merged virtual document. First, each interval in the intersection between the virtual documents for each query information representation is given a merge value (MV), which equals
1) 1 if the intersection interval contains an occurrence of all of the query information representations in a single sentence;
2) 0.8 if the intersection interval does not contains an occurrence of all of the query information representations in a single sentence, but the intersection interval does contain occurrences of all of the query information representations; or
3) 0.5 if the intersection interval does not contain occurrences of all of the query information representations.
Second, each interval in the intersection between the virtual documents for each query information representation is given a relevance magnitude (RM) that is the average of the relevance magnitude for each relevance interval in the intersecting virtual documents that includes the intersection interval. The final magnitude assigned to the merged virtual document is then the largest RM*MV for any of the intersection intervals.
The displayed results of a search include links to each virtual document. Because these virtual documents are not timed media files, but simply pointers to the time-code of actual timed media files, additional features must be included in the system in order to utilize the advantages of the system. FIG. 16 shows a hierarchy of playback features. When a user follows a link to a virtual document, the appropriate media player, i.e. QuickTime®, NetMeeting®, RealPlayer®, Windows Media Player®, etc., begins playing the streamed media, the feature represented by block 600. The virtual document will be automatically played continuously, block 602, despite the fact that it may comprise several discontinuous sections of one or more timed media files. Furthermore, an additional navigation bar can appear below the media player indicating the portion of the document that is currently being played. In block 604 of FIG. 16, the player's standard navigation bar allows the user to navigate within the whole document from which the virtual document being played was created. Additional buttons can be provided to allow semantic navigation in block 610. The semantic navigation 610 includes navigating within relevance intervals, block 612, and between relevance intervals, block 614. The user can thus easily find the desired feature within the timed media file, block 620. The user can therefore easily navigate the portions of the timed media files that have been deemed relevant to the query information representation(s), while also easily being able to view the rest of the original timed media file.
US4227177 * 27 Apr 1978 7 Oct 1980 Dialog Systems, Inc. Continuous speech recognition method
US5799315 * 7 Jul 1995 25 Aug 1998 Sun Microsystems, Inc. Method and apparatus for event-tagging data files automatically correlated with a time of occurence in a computer system
US5857200 * 21 Feb 1996 5 Jan 1999 Fujitsu Limited Data retrieving apparatus used in a multimedia system
US6188976 * 23 Oct 1998 13 Feb 2001 International Business Machines Corporation Apparatus and method for building domain-specific language models
US6320588 * 18 Apr 1997 20 Nov 2001 Compaq Computer Corporation Audio/video storage and retrieval for multimedia workstations
US6345253 * 18 Jun 1999 5 Feb 2002 International Business Machines Corporation Method and apparatus for retrieving audio information using primary and supplemental indexes
US6415434 * 18 Jun 1999 2 Jul 2002 Hewlett-Packard Company Apparatus and method for a runtime method overloading resolver
US6792426 * 8 Jan 2002 14 Sep 2004 International Business Machines Corporation Generic servlet for browsing EJB entity beans
US6970639 * 7 Sep 2000 29 Nov 2005 Sony United Kingdom Limited System and method for editing source content to produce an edited content sequence
US7272558 * 23 Mar 2007 18 Sep 2007 Coveo Solutions Inc. Speech recognition training method for audio and video file indexing on a search engine
US7376642 * 30 Mar 2004 20 May 2008 Microsoft Corporation Integrated full text search system and method
US7548934 * 30 Mar 2006 16 Jun 2009 Microsoft Corporation Auto playlist generator
US7584102 * 15 Nov 2002 1 Sep 2009 Scansoft, Inc. Language model for use in speech recognition
US7739286 * 17 Mar 2006 15 Jun 2010 University Of Southern California Topic specific language models built from large numbers of documents
US7788266 * 27 Sep 2005 31 Aug 2010 Veveo, Inc. Method and system for processing ambiguous, multi-term search queries
US7925506 * 5 Oct 2004 12 Apr 2011 Inago Corporation Speech recognition accuracy via concept to keyword mapping
US7958119 * 4 Feb 2008 7 Jun 2011 Sony Deutschland Gmbh Method for content recommendation
US8041566 * 12 Nov 2004 18 Oct 2011 Nuance Communications Austria Gmbh Topic specific models for text formatting and speech recognition
US8078467 * 8 Mar 2007 13 Dec 2011 Nec (China) Co., Ltd. Device and method for language model switching and adaptation
US8117206 * 7 Jan 2009 14 Feb 2012 Streamsage, Inc. Method and system for indexing and searching timed media information based upon relevance intervals
US8527520 * 11 Jan 2012 3 Sep 2013 Streamsage, Inc. Method and system for indexing and searching timed media information based upon relevant intervals
US20020194181 * 25 Jul 2001 19 Dec 2002 Wachtel David C. Method and apparatus for intelligent data assimilation
US20030163443 * 26 Feb 2002 28 Aug 2003 Sony Corporation System and method for effectively implementing a personal channel for interactive television
US20040254795 * 22 Jul 2002 16 Dec 2004 Atsushi Fujii Speech input search system
US20050114130 * 9 Jul 2004 26 May 2005 Nec Laboratories America, Inc. Systems and methods for improving feature ranking using phrasal compensation and acronym detection
US20050152362 * 30 Dec 2004 14 Jul 2005 Yi-Chieh Wu Data classification management system and method thereof
US20050222975 * 30 Mar 2004 6 Oct 2005 Nayak Tapas K Integrated full text search system and method
US20060074671 * 5 Oct 2004 6 Apr 2006 Gary Farmaner System and methods for improving accuracy of speech recognition
US20060100898 * 10 Nov 2005 11 May 2006 Pearce Jonathan P Method for booking and scheduling actors for projects using a computer network
US20060253780 * 10 Jul 2006 9 Nov 2006 Mastushita Electric Industrial Co., Ltd. Data processing device and method for selecting media segments on the basis of a score
US20070050343 * 23 Aug 2006 1 Mar 2007 Infosys Technologies Ltd. Semantic-based query techniques for source code
US20070050366 * 16 Dec 2005 1 Mar 2007 Harris Corporation System, program product, and methods to enhance media content management
US20070067285 * 22 Sep 2005 22 Mar 2007 Matthias Blume Method and apparatus for automatic entity disambiguation
US20070208567 * 1 Mar 2006 6 Sep 2007 At&T Corp. Error Correction In Automatic Speech Recognition Transcripts
US20070239707 * 30 Mar 2007 11 Oct 2007 Collins John B Method of searching text to find relevant content
US20070260700 * 4 May 2006 8 Nov 2007 Samsung Electronics Co., Ltd. Method and system for providing persistent media across web pages using only browser plug-ins
US20080204595 * 26 Jun 2007 28 Aug 2008 Samsung Electronics Co., Ltd. Method and system for extracting relevant information from content metadata
US20080208864 * 26 Feb 2007 28 Aug 2008 Microsoft Corporation Automatic disambiguation based on a reference resource
US20080222142 * 6 Mar 2008 11 Sep 2008 Utopio, Inc. Context based data searching
US20080250010 * 5 Apr 2007 9 Oct 2008 Samsung Electronics Co., Ltd. Method and system for determining and pre-processing potential user queries related to content in a network
US20080256097 * 16 Apr 2007 16 Oct 2008 Samsung Electronics Co., Ltd. Method and system for location identification
US20090006315 * 29 Jun 2007 1 Jan 2009 Sougata Mukherjea Structured method for schema matching using multiple levels of ontologies
US20090006391 * 27 Jun 2007 1 Jan 2009 T Reghu Ram Automatic categorization of document through tagging
US20090013002 * 4 Feb 2008 8 Jan 2009 Sony Deutschland Gmbh Method for content recommendation
US20090025054 * 17 Jul 2007 22 Jan 2009 Samsung Electronics Co., Ltd. Method and system for access to content in a content space
US20090144260 * 30 Nov 2007 4 Jun 2009 Yahoo! Inc. Enabling searching on abbreviated search terms via messaging
US20090204599 * 13 Feb 2008 13 Aug 2009 Microsoft Corporation Using related users data to enhance web search
US20090205018 * 6 Feb 2009 13 Aug 2009 Ferraiolo David F Method and system for the specification and enforcement of arbitrary attribute-based access control policies
US20090240650 * 5 Jun 2009 24 Sep 2009 Wang Fangshan System and method for managing media contents
US20100070507 * 16 Mar 2009 18 Mar 2010 Kabushiki Kaisha Toshiba Hybrid content recommending server, system, and method
US20100094845 * 11 Dec 2008 15 Apr 2010 Jin Young Moon Contents search apparatus and method
US20100138653 * 25 Nov 2009 3 Jun 2010 Elliot Spencer Method and system for searching for, and collecting, electronically-stored information
US20100250598 * 30 Mar 2009 30 Sep 2010 Falk Brauer Graph based re-composition of document fragments for name entity recognition under exploitation of enterprise databases
US20110077943 * 18 Jun 2007 31 Mar 2011 Nec Corporation System for generating language model, method of generating language model, and program for language model generation
US20110125728 * 31 Jan 2011 26 May 2011 Smyros Athena A Systems and Methods for Indexing Information for a Search Engine
US20110191099 * 8 Mar 2011 4 Aug 2011 Inago Corporation System and Methods for Improving Accuracy of Speech Recognition
CA2685833A1 * 12 Nov 2009 20 May 2010 Comcast Cable Communications, Llc Method and apparatus for delivering video and video-related content at sub-asset level
EP1241587A1 * 12 Mar 2001 18 Sep 2002 THOMSON multimedia Method for processing content data
EP1501305A1 * 23 Sep 1997 26 Jan 2005 Thomson Inc. Method for collating program guide information from a plurality of sources with equivalence mapping
GB2448874A * Title not available
GB2448875B * Title not available
WO2006099621A2 * 17 Mar 2006 21 Sep 2006 University Of Southern California Topic specific language models built from large numbers of documents
WO2008053132A1 * 12 Jun 2007 8 May 2008 Nds Limited Program guide search
WO2009052277A1 * 16 Oct 2008 23 Apr 2009 Evri, Inc. Nlp-based entity recognition and disambiguation
1 * "Ying Zhang and Phil Vines. 2004. Using the web for automated translation extraction in cross-language information retrieval. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '04). ACM, New York, NY, USA, 162-169".
2 * Boulgouris N. V. et al., "Real-Time Compressed-Domain Spatiotemporal Segmentation and Ontologies for Video Indexing and Retrieval", IEEE Transactions on Circuits and Systems for Video Technology, vol. 14, No. 5, pp. 606-621, May 2004.
3 * Changsheng Xu et al., "Using Webcast Text for Semantic Event Detection in Broadcast Sports Video", IEEE Transactions on Multimedia, vol. 10, No. 7, pp. 1342-1355, Nov. 2008.
4 * Chen, "Extraction of Indicative Summary Sentences from Imaged Documents", IEEE publication, 1997, pp. 227-232.
5 * Eberman, et al., "Indexing Multimedia for the Internet", Compaq, Cambridge Research laboratory, Mar. 1999, pp. 1-8 and Abstract.
6 * EP Application No. 09 175 979.5-Office Action mailed Apr. 11, 2011.
7 * EP Application No. 09 175 979.5—Office Action mailed Apr. 11, 2011.
8 * ESR-EP10154725.5-Nov. 2, 2010.
9 * ESR—EP10154725.5—Nov. 2, 2010.
10 * ESR-EP10155340.2-Nov. 25, 2010.
11 * ESR—EP10155340.2—Nov. 25, 2010.
12 * ESR-EP10162666.1-Aug. 4, 2011.
13 * ESR—EP10162666.1—Aug. 4, 2011.
14 * ESR-EP10167947.0-Sep. 28, 2010.
15 * ESR—EP10167947.0—Sep. 28, 2010.
16 * European Application No. 09175979.5-Office Action mailed Mar. 15, 2010.
17 * European Application No. 09175979.5—Office Action mailed Mar. 15, 2010.
18 * European Patent Application No. 09175979.5-Office Action dated Dec. 13, 2011.
19 * European Patent Application No. 09175979.5—Office Action dated Dec. 13, 2011.
20 * European Search Report EP 09180762, dated Mar. 22, 2010.
21 * European Search Report EP09179987.4, dated Jun. 4, 2010.
22 * European Search Report for Application No. 09180776.8, mailed Jun. 7, 2010, 9 pages.
23 * European Search Report for application No. 10167947.0, mailed Sep. 28, 2010.
24 * Experiments in Spoken Document Retrieval at CMU, M.A. Siegler, M.J. Wittbrock, S.T. Slattery, K. Seymore, R.E. Jones, and A.G. Hauptmann, School of Computer Science Carniegie Mellon University, Pittsburgh, PA 15213-3890, Justsystem Pittsburgh Research Center, 4616 Henry Street, Pittsburgh, PA 15213 (No Date Provided).
25 * Extended European Search Report-EP 09815446.1-mailind date: May 7, 2013.
26 * Extended European Search Report—EP 09815446.1—mailind date: May 7, 2013.
27 * First Office Action in EP01950739.1-1244 dated Mar. 27, 2009.
28 * First Office Action in EP01950739.1—1244 dated Mar. 27, 2009.
29 * Hsin-Min Wang and Berlin Chen, "Content-based Language Models for Spoken Document Retrieval", ACM, 2000, pp. 149-155.
30 * International Preliminary Examination Report for PCT/US01/20894, dated Feb. 4, 2002.
31 * IPRP PCT/US2009/069644-Jun. 29, 2011.
32 * IPRP PCT/US2009/069644—Jun. 29, 2011.
33 * Ishitani, et al., "Logical Structure Analysis of Document Images Based on emergent Computation", IEEE Publication, pp. 189-192, Jul. 1999.
34 * ISR PCT/US2001/020894-Nov. 25, 2003.
35 * ISR PCT/US2001/020894—Nov. 25, 2003.
36 * isr pct/us2009/069644-Mar. 4, 2010.
37 * isr pct/us2009/069644—Mar. 4, 2010.
38 * Kalina Bontcheva et al "Shallow Methods for Named Entity Coreference Resolution", Pro. of Taln 2002, Jan. 1, 2002.
39 * Kontothoanassis, Ledonias et al. "Design, Implementation, and Analysis of a Multimedia Indexing and Delivery Server", Technical Report Series, Aug. 1999, Cambridge Research Laboratory.
40 * Koskela M. et al., "Measuring Concept Similarities in Multimedia Ontologies: Analysis and Evaluations", IEEE Transactions on Multimedia, vol. 9, No. 5, pp. 912-922, Aug. 2007.
41 * Li, Y. et al., "Reliable Video Clock Time Recognition," Pattern Recognition, 2006, 1CPR 1006, 18th International Conference on Pattern Recognition, 4 pages.
42 * Liang Bai et al., "Video Semantic Content Analysis based on Ontology", International Machine Vision and Image Processing Conference, pp. 117-124, Sep. 2007.
43 * Marin, Feldman, Ostendorf and Gupta, "Filtering Web Text to Match Target Genres", International Conference on Acoustics, Speech and Signal Processing, 2009, Piscataway, NJ, Apr. 19, 2009, pp. 3705-3708.
44 * Messer, Alan et al., "SeeNSearch: A Context Directed Search Facilitator for Home Entertainment Devices", Paper, Samsung Information Systems America Inc., San Jose CA (No Date Provided).
45 * Partial ESR-EP10155340.2-Jul. 12, 2010.
46 * Partial ESR—EP10155340.2—Jul. 12, 2010.
47 * Raphael Volz et al., "Towards ontologybased disambiguation of geographical identifiers", Proceedings of the WWW2007 Workship l3: Identity, Identifiers, Identification, Entity-Centric Approaches to Information and Knowledge Management on the Web, Jan. 1, 2007.
48 * Salton et al., Computer Evaluation of Indexing and Text Processing Journal of the Association for Computing Machinery, vol. 15, No. 1, Jan. 1968, pp. 8-36.
49 * Shahraray: "Impact and Applications of Video Content Analysis and Coding in the internet and Telecommunications", AT&T Labs Research, A Position Statement for Panel 4: Applications the 1998 International Workshop on Very Low Bitrate Video Coding, 3 pages.
50 * Smith, J.R. et al., "An Image and Video Search Engine for the World-Wide Web" Storage and Retrieval for Image and Video Databases 5, San Jose, Feb. 13-14, 1997, Proceedings of Spie, Belingham, Spie, US vol. 3022, Feb. 13, 1997, pp. 84-95.
51 * Steffen Staab et al., "Semantic Multimedia", Reasoning Web; Lecture Notes in Computer Science, pp. 125-170, Sep. 2008.
52 * Towards a Multimedia World-Wide Web Information retrieval engines, Sougata Mukherjea, Kyoji Hirata, and Yoshinori Hara Computer Networks and ISDN Systems 29 (1997) 1181-1191.
53 * Wacholder N. et al., "Disambiguation of Proper Names in Text", Proceedings of the Conference on Applied Natural Language Processing, Association Computer Linguistics, Morrisontown, NJ, Mar. 1, 2007.
U.S. Classification 707/746
Cooperative Classification G06F17/3002, G06F17/30424, G06F17/30064, G06F17/30613, G06F17/30684, G06F17/275, G06F17/2705, G06F17/274, Y10S707/913, Y10S707/99931, Y10S707/99935, G06F17/30796, G06F17/30029
Owner name: STREAMSAGE, INC., DISTRICT OF COLUMBIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MORTON, MICHAEL SCOTT;SIBLEY, TIM V.;UNGER, NOAM CARL;AND OTHERS;SIGNING DATES FROM 20030211 TO 20030212;REEL/FRAME:030914/0989
Free format text: MERGER;ASSIGNOR:STREAMSAGE, INC.;REEL/FRAME:042028/0766
Free format text: MERGER;ASSIGNOR:STREAMSAGE, INC.;REEL/FRAME:042028/0670