Source: https://insight.rpxcorp.com/pat/US20110106813A1
Timestamp: 2019-10-16 01:50:22
Document Index: 484508083

Matched Legal Cases: ['art 1', 'art 1', 'art 1', 'art 1', 'art 3', 'art 2', 'art 3', 'arts 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2', 'art 2']

Patent US 20110106813A1
1. A data indexing and perusal system comprising:
A data acquisition and perusal system and method including a database selection module, a database index generator module and a search module. The database selection module enables selection of a plurality of files for inclusion into at least one selectable database. The database index generator module enables generation of a searchable index of the data contained in the selectable database. The search module enables a search to be performed of the searchable index according to search criteria. The system allows for the capture of HTML data which is automatically indexed without human intervention and has the ability to automatically and accurately locate or &#x201c;pinpoint,&#x201d; and highlight specific text or groups of text designated by the user within the resulting database.
AUTOMATED SUBJECT ANNOTATOR CREATION USING SUBJECT EXPANSION, ONTOLOGICAL MINING, AND NATURAL LANGUAGE PROCESSING TECHNIQUES
US 20140006373A1
US 9,471,613 B2
US 9,824,109 B2
US 10,169,388 B2
US 6,389,434 B1
US 6,199,060 B1
Survivors of THW Shoah Visual History Foundation
US 6,212,530 B1
US 6,247,018 B1
US 6,272,534 B1
US 6,334,132 B1
US 6,138,129 A
US 5,744,123 A
US 5,752,816 A
US 5,848,249 A
US 5,652,880 A
US 5,687,367 A
US 5,519,865 A
US 5,423,034 A
US 5,466,891 A
US 5,222,234 A
US 4,955,056 A
US 3,670,310 A
Infodata Systems Incorporated Webster NY
2. The data indexing and perusal system of claim 1, wherein the searchable index comprises a plurality of complementary index files.
3. The data indexing and perusal system of claim 1, wherein the index module is operable to store any custom links within the index.
4. The data indexing and perusal system of claim 1, wherein the link module enables creation of valid custom links only.
5. The data indexing and perusal system of claim 1, wherein the link module enables designation of a link term and designation of one of the plurality of selected source files to be linked to the designated link term;
the link module being operable to automatically link multiple instances of the designated link term in the plurality of selected source files with the designated file.
6. The data indexing and perusal system of claim 1, wherein the selected source files include HTML files, the system further comprising a browser for displaying the HTML files that meet the search criterion and which utilizes word locations retrieved from an index of word locations to visually distinguish the searched words and phrases from any surrounding text in the displayed HTML files.
7. A data indexing and perusal system comprising:
a display utility means for displaying the HTML files that meet the search criterion and visually distinguishing the searched words and phrases from any surrounding text in the displayed HTML files.
8. A data indexing and perusal system comprising:
a search module that enables a search to be performed of the index-according to a search criterion.
9. The data indexing and perusal system of claim 8, wherein the selection module is operable to save the original source internet addresses of the selectively saved linked elements when modifying the saved web page's links to point to the selectively saved linked elements' local addresses.
10. A method of annotating, indexing, searching, and displaying a plurality of selected source files, the method comprising:
11. The method of claim 10, wherein the step of generating a searchable index creates a searchable index comprising a plurality of complementary index files.
12. The method of claim 10, wherein the enablement step also enables a user to designate a link term and designate one of the plurality of selected source files to be linked to the designated link term.
13. The method of claim 12, further comprising the step of verifying the validity of any designated links, wherein the incorporating step incorporates links only if the links are valid.
14. The method of claim 12, further comprising the step of automatically generating links between all instances of a designated link term within the plurality of selected source files and the designated file.
15. The method of claim 14, wherein the automatic link generation step generates only valid links.
16. The method of claim 10, wherein the index generating step generates an index operable to be searched according to negative searching principles using conflation logic.
Computers were intended to provide an effective and efficient way for humans to manage, locate, peruse and manipulate data or objects. For example, a first, basic, system and method is that demonstrated by modern word processor applications which have some search and text access capabilities; however, as far as in known, they are limited to the current file that is open. Employing this method, the user can request the location of a word in the text. Within an individual file, the computer will then take the user sequentially to each location of that text. Only string searches are allowed. By repeatedly running the search, the user can sequentially move from result to result. While it might be possible to open, many files simultaneously, the available resources and memory make this impractical.
A third system and method used by software applications provides improved search capabilities and is commonly known as a &#x201c;search/retrieval engine&#x201d;. Among other things, search/retrieval engines can essentially search and access many thousands of files simultaneously and very quickly by using pre-generated indexes of the data. For example, a user can query an encyclopedia converted to an indexed database, and by the use of highlighted text, quickly determine every place a word or phrase occurs in the text, and have the ability to instantly view those occurrences as desired. These products even take the user sequentially to each incident of highlighted text or &#x201c;hit.&#x201d; The computer can then take the user from hit to hit.
Using the current art in the third system and method above, users can add electronic bookmarks to enable them to quickly return to any part of any volume of an encyclopedia, referred to in the example above, and they can copy portions for insertion into other documents of their own creation. By use of hypertext links appearing within the database, a user is able to instantly view related data for which he had not searched. The links are generated according to a rationale applied when the database index was prepared. Adding hypertext links usable within a database is generally a more complex process. The links are intended to appear to the user in a color or format distinguishable from other data, and when activated, the computer is directed to display another highlighted portion of the database. By naming the instructions to the computer within links as &#x201c;pointers&#x201d; and what they link to as &#x201c;targets&#x201d;, the process will be facilitated. A database can theoretically have an unlimited number of identical pointers (even though what the user sees can be different for some or all of them), but any pointer can generally only have one target (a specific area of the database to display),
If a user desires to have the searchable data include context-sensitive links, the choices are generally reduced to: (1) obtaining a pre-linked database from an electronic publisher; or (2) creating a custom database and manually inserting links individually or by use of a custom program written for the unique situation. Beyond the problems of availability and lack of customization, a fundamental problem with the first choice is that a publisher may not consider the same links to be important as a user does. Thus, the publisher may include links that are not important to the user and may not include links that would have been important. A fundamental problem with the second choice is that, manually inserting links requires a substantial amount of time and trouble that quickly outweighs any potential benefit to manually inserting links as the quantity of data increases. As far as is known, the current art does not include a system to create links by designating &#x201c;pointers&#x201d; and &#x201c;targets&#x201d; and having the program automatically create links that are all valid.
In one embodiment, commercial electronic database publishers could use a system according to the present invention as a publishing system to create databases with more or less homogeneous content. For example, one publisher may produce a monthly searchable, linked database containing issued United States patents, another might produce a linked database containing decisions of appellate courts, and another might produce a linked database containing documents required to be filed by various regulatory agencies, etc. Using prior systems to produce such databases requires substantial programming skills to incorporate reference links within the database, but in practice, many such links are invalid because a referenced document does not exist. Using the system according to the present invention does not require such skills because it automatically creates only valid and verified links. The graphical user interface is easily modified to comport with a particular &#x201c;look and feel&#x201d; desired by the publisher.
Referring to the drawings, FIG. 1A is a block diagram an exemplary computer system 100 that could be used to illustrate various aspects of a data acquisition and perusal system implemented according to the present invention. The computer system 100 is preferably a conventional IBM brand compatible, personal computer (PC) system or the like, and includes a motherboard and bus system 102 coupled to at least one central processing unit (CPU) 104 and a memory system 106. The motherboard and bus system 102 include any kind of bus system configuration, such as any combination of a host bus, one or more peripheral component interconnect (PCI) buses, an industry standard architecture (ISA) bus, an extended ISA (EISA) bus, micro-channel architecture (MCA) bus, an AGP bus, a universal serial bus (USB), etc., along with corresponding bus driver circuitry and bridge interfaces, etc., as known to those skilled in the art. The CPU 104 preferably incorporates anyone of several microprocessors, such as the 80486, Pentium&#x2122;, Pentium II&#x2122;, Pentium III&#x2122;, etc. microprocessors from Intel Corp., or other similar type microprocessors such as the K6 microprocessor by Advanced Micro Devices, and supporting external circuitry typically used in PCs. The external circuitry preferably includes an external or level two (L2) cache or the like (not shown). The memory system 106 may include a memory controller or the like and be implemented with one or more memory boards (not shown) plugged into compatible memory slots on the motherboard, although any memory configuration is contemplated. The invention is also applicable to other microprocessors, other architectures and other operating systems.
Referring now to FIG. 1B, a block diagram is shown of a network system 150 that communicatively couples a plurality of computer systems or computing devices 152, 154, 156, 158, 160, etc. together via a communication medium 162. Any one or more of the computing devices 152-160 may be implemented in the same or a similar manner as the computer system 100. The network system 150 may include anyone or more network devices (not shown), such as hubs, switches, repeaters, bridges, routers, brouters, etc. The network system 150 may operate according to any network architecture, such as Ethernet&#x2122;, Token Ring, Token Bus, ATM, etc., or combinations of such architectures at any available speed, such as 10 Megabits per second (Mbps), 100 Mbps, 1 Gigabits per second (1 Gbps), etc. The network 150 may form any type of Local Area Network (LAN) or Wide Area Network (WAN), and may comprise an intranet and be connected to the internet.
Database source files (or documents) are typically divided into fields or areas when they are created. These fields may result from word processing application that is used to create the documents. For example, WordPerfect&#xae; formatted files/documents contain both hidden and visible fields in almost every document that is created. Likewise, Microsoft&#xae; Word (hereinafter MS-Word) formatted files/documents contain certain fields. In addition, internet or HTM (or HTML, HyperText Markup Language) type format files, objects or documents contain many hidden and unhidden fields. Thus, the files/documents/objects referred to herein should be understood to contain fields.
Further, a system according to the present invention includes special features for handling composite file types, such as HTML format files used over the internet. Composite files can include display codes for arrangement, graphics, fonts, hyperlinks, and other characteristics that allow &#x201c;assembly&#x201d; of what appears to be a single document presented on the computer monitor but which actually may be a compilation of multiple text and graphic elements stored in separate files. Unlike integral files, composite files are more efficient from a disk storage space standpoint than integral files because their reusable components, such as graphics, can be used many times by different files without having to be replicated for each file. Composite files can also include small integral programs called scripts (e.g., Java applets or Java scripts) that instruct the computer to perform other tasks while the HTML page is displayed. Regardless of the visual complexity of an HTML composite file, from a searchable database perspective, the crucial parts of the HTML composite file are those parts that contain text.
The file database 171 includes one or more files of type A, shown as files AF1, AF2 . . . AFn, where &#x201c;n&#x201d; is any positive integer. The file database 171 may further include one or more files of type B, shown as files BF1, BF2 . . . BFn, one or more files of type C (not shown), etc. Examples of file types include Internet or HTML format (or simply HTM), word processor format including DOC files generated by MS-Word, or similar word processing files generated by WordPerfect&#xae;, text format, RTF (Rich Text Format) files, drawing files, database files, etc. The incompatibilities and between various formats has become less since several type of formats may be included in a single file, object, or document. In this manner, the present invention contemplates any number of files or documents of any type. It is noted that anyone or more of the files may be copied into local storage or may be simply accessed via an existing link to that
The system 170 shown in FIG. 1C includes a database selection module 173 that enables a user to select any number of any type of files from the file database 171 for inclusion into a selectable database 175. Of note, the term &#x201c;module&#x201d; represents any combination of hardware and software implemented to achieve the desired functions. For example, one or more modules described herein may be incorporated into a database application, which is stored on the storage system 120 and retrieved into the memory system 106 for execution by the CPU 104. The selectable database 175 comprises one or more databases, shown as D1, D2, D3, etc., where each database includes one or more files selected by the user from the file database 171. The selectable database 175 may include a single database with a single file or multiple files, or a plurality of databases, each including a single file or multiple files. The
may define the terms &#x201c;grape&#x201d;, &#x201c;tomato&#x201d;, &#x201c;raspberry&#x201d;, etc., as aliases of a link term &#x201c;vine fruit&#x201d;. Each alias is treated in a similar manner as its corresponding link term. Each of the files in the selected database 175 may further include one or more fields. The link 20 module 177 enables the user to define field links to link similar fields between two or more files. Such field links may be generated according to patterns, where such patterns may further be defined using wildcard characters that each substitute for one or more digits or characters depending upon the function of the respective wildcards, as further described below.
In operation, the search module 183 first compares each term of the search query against &#x201c;words&#x201d; contained in a stop word list 201 of the database index 200. The stop word list 201&#x2033; is a file containing a list of &#x201c;noise words&#x201d;, or words that frequently occur In a file/document that do not contain distinguishable characteristics. For 5 example, stop words are &#x201c;words&#x201d; such as &#x201c;and&#x201d;, &#x201c;as&#x201d;, &#x201c;the&#x201d;, &#x201c;a&#x201d;, &#x201c;I&#x201d;, &#x201c;for&#x201d;, certain punctuation, etc. Although a default stop word list is provided for each database index that is to be generated, a user may edit the stop word list 201 for a particular database index that is to be generated and include additional stop words or remove unwanted stop words from the default stop word list. If a stop word is found among the terms of a search query, the search for that term is terminated because the search module 183 considers that term to be a noise word and does not allocate further resources toward searching the files for that term. However, the length of the term is stored in the search engine's dynamic buffers for future phrase analysis. For example, if the search query contains the terms &#x201c;big for till&#x201d;, the word &#x201c;for&#x201d; is considered a stop word and a length of the stop word, i.e., five (three letters plus two white space delimiters), is stored in place of the spaces and the word &#x201c;for&#x201d;. Thus, as described in greater detail herein, the search query becomes a search for files/documents that include the words &#x201c;big&#x201d; and &#x201c;tall&#x201d; with five spaces/characters between the words. If the search had been the search query &#x201c;big as tall&#x201d;, where the word &#x201c;as&#x201d; is considered a noise word, the search query becomes a search for the words &#x201c;big&#x201d; and &#x201c;tall&#x201d; with four spaces/characters there between.
(b) Following the tagged binary string is a sequence of twelve bytes comprised of three sets of four-byte integers or &#x201c;long integers&#x201d;. Each long integer provides additional information necessary to find the word string in its database file(s). These twelve bytes are broken down thusly:
<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="center"/><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE I</entry></row></thead><tbody valign="top"><row><entry namest="1" nameend="1" align="center" rowsep="1"/></row><row><entry>Long Integer Interpretation</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="1" colwidth="63pt" align="left"/><colspec colname="2" colwidth="56pt" align="left"/><colspec colname="3" colwidth="98pt" align="left"/><tbody valign="top"><row><entry>If first long</entry><entry>And second long</entry><entry/></row><row><entry>integer (x) is:</entry><entry>integer (y) is:</entry><entry>Interpretation:</entry></row><row><entry namest="1" nameend="3" align="center" rowsep="1"/></row><row><entry>Positive and</entry><entry>Positive.</entry><entry>First number (x) is the</entry></row><row><entry>less than the</entry><entry/><entry>number of files in the</entry></row><row><entry>number of</entry><entry/><entry>database containing the word.</entry></row><row><entry>files in the</entry><entry/><entry>Second number (y) is an</entry></row><row><entry>database.</entry><entry/><entry>index to the file position in</entry></row><row><entry/><entry/><entry>Part 1 of the Master Word</entry></row><row><entry/><entry/><entry>Index which starts the list of file</entry></row><row><entry/><entry/><entry>numbers containing this</entry></row><row><entry/><entry/><entry>word. The list of numbers</entry></row><row><entry/><entry/><entry>is x entries long.</entry></row><row><entry>Positive, and</entry><entry>Positive.</entry><entry>x indicates the number of</entry></row><row><entry>greater than</entry><entry/><entry>files that DO NOT contain</entry></row><row><entry>the number of</entry><entry/><entry>the given word. This</entry></row><row><entry>files in the database.</entry><entry/><entry>number is determined by</entry></row><row><entry/><entry/><entry>subtracting the number of</entry></row><row><entry/><entry/><entry>files in the database from x.</entry></row><row><entry/><entry/><entry>y is an index to the</entry></row><row><entry/><entry/><entry>fileposition in Part 1 of the</entry></row><row><entry/><entry/><entry>Master Word Index at</entry></row><row><entry/><entry/><entry>whichstarts the list of the</entry></row><row><entry/><entry/><entry>file numbers that do NOT</entry></row><row><entry/><entry/><entry>contain this word. The</entry></row><row><entry/><entry/><entry>length of this list is the</entry></row><row><entry/><entry/><entry>number x, less the number</entry></row><row><entry/><entry/><entry>of files in the database.</entry></row><row><entry>Positive</entry><entry>&#x2212;1</entry><entry>x is the file number of the</entry></row><row><entry/><entry/><entry>one and only file in the</entry></row><row><entry/><entry/><entry>database which contains</entry></row><row><entry/><entry/><entry>this word.</entry></row><row><entry/><entry/><entry>(No entry is needed</entry></row><row><entry/><entry/><entry>in Part 1.)</entry></row><row><entry>&#x2212;1</entry><entry>&#x2212;1</entry><entry>All files in the database contain</entry></row><row><entry/><entry/><entry>this word. (No entry</entry></row><row><entry/><entry/><entry>is needed in Part 1.)</entry></row><row><entry namest="1" nameend="3" align="center" rowsep="1"/></row></tbody></tgroup></table></tables>
Part 3 is a sequence of three indices, herein referred to as a first index, a second index, and a third index, for eliminating search terms that do not appear in Part 2 of the master word index file. Essentially, once a database index has been generated, the search module 183 uses Part 3 as a &#x201c;negative search&#x201d; index, i.e., an index to quickly eliminate search terms that do not appear in the database. In one embodiment, before the first of these three indices, there is a two-byte ASCII 5, ASCII NULL pair that serves as a dividing point between Parts 2 and 3.
The first ten long integers are indices into the Part 2 information for words starting with &#x201c;0&#x201d;-&#x201c;9&#x201d;. Thus, when the database index 200 is generated, offsets for the words starting with &#x201c;0&#x201d;-&#x201c;9&#x201d; in the Part 2 data are recorded in each of the first ten long integers. If no word in Part 2 starts with the given single digit, four ASCII 255's are written into the corresponding long integer of the first ten long integers. Following these ten long integers are 100 long integers for words starting with the pairs &#x201c;00&#x201d;-&#x201c;99&#x201d;. Similar to the first ten long integers, offsets for words in the Part 2 data are recorded, but if no word starts with the given pair, four ASCII 255's are written to that long integer of the first index.
The second index is an index for &#x201c;odd&#x201d; leading characters. This index is a list of 255 long integers, corresponding to ANSI characters 1-255. Like the first index, offsets for words in the Part 2 data are recorded, but if no word in Part 2 starts with a given character, four ASCII 255's are written to the corresponding long integer of the second index. Also, if the given character is a letter, a numeric digit, or any other character that a user is not intended to find with the search module 183, four ASCII 255's are written to the long integer that represents that character.
The third index is a list of long integers that index words with alphabetical leading characters. The third index is of variable length depending on whether the index is a two or a three dimensional index (to be described herein). The first 26 long integers in the third index are offsets for words in the Part 2 data that begin with the single letters &#x201c;a&#x201d; through &#x201c;z&#x201d;. If no words in Part 2 begin with a given letter, four ASCII 255's are written to the corresponding long integer. The next 676 (26 squared) long integers of the third index are offsets for words that begin with the pairs &#x201c;aa&#x201d;, &#x201c;ab&#x201d;, &#x201c;ac&#x201d;, etc., through &#x201c;zz&#x201d;, thus, creating a &#x201c;two dimensional&#x201d; index from the third index. Offsets for these words in the Part 2 data, are recorded in the 676 long integers, but if no word begins with a given pair, four ASCII 255's are written to the corresponding long integer. If desired, the third index can be a &#x201c;three dimensional&#x201d; index, i.e., an index including references to single alpha characters (26), pairs of alpha characters (676), and three alpha characters. If the index is three dimensional, then 26 cubed (17576) long integers follow &#x201c;zz&#x201d;. These long integers index words beginning with the triplets &#x201c;aaa&#x201d;, &#x201c;aab&#x201d;, &#x201c;aac&#x201d;, etc., through &#x201c;zzz&#x201d;. Again, if no word begins with a given triplet, four ASCII 255's are written to the corresponding long integer for that triplet.
After the search module 183 determines which files contain the search terms, a word number index 203 is accessed to find the exact location of the search terms in each file of the database. The word number index 203 is included in the database index 200 and can be described by two files, a OSI file 204, and a OSF file 205. The terms &#x201c;OSI&#x201d; and &#x201c;OSF&#x201d; are somewhat arbitrary character strings and are commonly used as file extensions for the respective files in the word number index 203. Broadly speaking, the terms represent a file (OSF) and an index (OSI) to that file, but, for purposes of understanding, each term is referred to as a file from a portion of the database index 200. It should be noted that, in a similar manner, the remaining portions of the database index 200 are also designated with similar character strings to designate files included in the respective portions of the database index 200.
The word number index 203 is used by the search module 183 to find the character and slot positions of words in database files. A character position is defined as the number of the logical byte or character in a file at which a word starts. For text files this is straightforward. For RTF, DOC (MS-Word), and HTM files, a translation from the actual binary file as stored on the disk to the logical file is necessary. A slot position is defined as the numeric position of the word in the file, a &#x201c;word&#x201d; being defined as any contiguous unit of text, including stop words, that appears between white space. Hence; for a file whose sole contents is the string &#x201c;Have a nice day!&#x201d;, the word &#x201c;nice&#x201d; has a character position of 7 because the count starts at 0, where &#x2018;H&#x2019; is at position 0. In addition, the word &#x201c;nice&#x201d; has a slot position of 3 because the count starts at 1, where &#x201c;Have&#x201d; is at position 1.
Referring to FIG. 2A, a number of elements in locator string 226 comprises the fourth and last column in the word position table 219. The number of elements in locator string 226 is a long integer&#x201d; and stores the number of sub-segments in the locator string 224.
In this embodiment, the data in the WDN file 216 consists of segments, one segment per each word in the database, where each segment consists of 52 bytes. The first 40 bytes contain the string representation of a given search word (e.g. &#x201c;apple&#x201d;). This string is padded on the right with spaces, so that it is always 40 bytes long, thus allowing easier loading into the word map. The next twelve bytes precisely duplicate the data in the three long integers stored in Part 2 of the master word index 202. In other words, the first long integer of the twelve bytes encode the word's word number. The next eight bytes encode two long integers, whose interpretations depend upon one another. Refer to Table I for possible interpretations.
Referring to FIG. 2, the WDN file 216 is part of a word lists structure 214. The word lists structure 214 includes files that contain different organizations of information associated with tile words from the selected databases, the files being available to expedite the search of the database index 200 for the terms of a search phrase. In this embodiment, the word lists structure 214 includes a word length (WDL) file 215 that comprises an index of words according to their length, a reverse word order (WDR) file 217 that comprises an index of words spelled in reverse order (i.e., right to left order) and that are alphabetized according to the reverse spelling of the words, and the WDN file 216. Thus, the word lists structure 214 is useful when a search query includes terms such as leading conflation searches, i.e. searches that call for all words meeting a search criteria in which only the last few letters of the search term are required to be met in the search query. For example, a search for &#x201c;*ample&#x201d; creates a hit for the words &#x201c;sample&#x201d;, &#x201c;example&#x201d;, &#x201c;ample&#x201d;, etc.
In this embodiment, if the search term is not found in the WDN file 216, the search for that term is terminated because the files/documents of the selected databases do not contain the term of the search query. If the search term is found in the WDN file 216, the exact location of additional information about the term stored in the master word index 202 is provided to the search module 183. If the computer does not have enough memory 106 to store the WDN file 216 in a memory map, the master word index 202 is searched directly for all information about the word, thus bypassing the WDN file 216 of the database index 200. In one embodiment, WDN files 216 of three databases are stored in memory 106, if possible, because users frequently select three or less databases to search and, typically, three or less WDN files 216 do not overly burden the &#x201c;memory 106 of a computer system operating the search module 183.
To assist in the understanding of the database index 200, the following narrative of a search for the word &#x201c;unique&#x201d; from the perspective of FIG. 2 is offered. In this example, a database index is created for each of three databases. One database includes three HTM files, a second database includes three RTF files, and a third database includes four DOC files. In each of the databases, the word &#x201c;unique&#x201d; appears twice in one document and once in another document. Therefore, upon a search for the word &#x201c;unique&#x201d;, each database has two files with at least one hit, one file with two hits and one file with one hit. The user selects the three databases and generates database indexes. The user presses &#x201c;Enter&#x201d; in the search dialog, requesting a search of the selected databases for the word &#x201c;unique&#x201d;. The search module 183 determines that there are three databases selected, and all are primary databases. Because they are primary databases, the corresponding WON files 216 are loaded into memory 106.
Starting with database 1 (the HTM database), the search module 183 searches the HTM WDN file for the word &#x201c;unique&#x201d;. The return value indicates that &#x201c;unique&#x201d; exists in this database, has a given word number (e.g., 138), and has two associated numeric values. In this case, the two values might be 4 and 68. The interpretation of the numeric values is carried out according to the interpretations described in Table I, where x=4 and y=68.
Because the HTM database is a three-file database, and x is 4, then row 2 of Table I applies, i.e., x (or 4) minus the number of files (3) equals one. Thus, one file does NOT contain the word &#x201c;unique&#x201d;, but the other files do. The file number of the single file that does not contain the word &#x201c;unique&#x201d; maybe found at position y=68 in the master word index 202. The search module 183 next looks in the master word index 202 at position 68 and reads one four-byte binary&#x2019; encoded long integer, whose value is 1. This is interpreted to mean that files 2 and 3 in this database contain, the word &#x201c;unique&#x201d;. Thus, all the files in the first database that contain the word &#x201c;unique&#x201d; are known. The search module 183 next performs a search on the second RTF database with similar results, perhaps finding that &#x201c;unique&#x201d; was word number 122 and files 1 and 3 contain the word &#x201c;unique&#x201d;.
This is followed by a check of the third database, i.e., the four-file MS-Word DOC database, where the word number is 190 and the numeric values are x=6 and y=156. Again, according to Table I, the return values indicate that two (6&#x2212;4=2) of the four files in the database do not contain the word &#x201c;unique&#x201d;, and those two files are recorded at position 156 of the master word index file 202. Reading the two four-byte binary encoded long integers at position 156 in the master word index 202 indicates that files 1 and 2 do not contain the word &#x201c;unique&#x201d;, and thus files 3 and 4 do contain the word &#x201c;unique&#x201d;. Thus, at this point, the user knows that each of the three databases has two files that contain the word &#x201c;unique&#x201d;. These files include Files 2 and 3 of Database 1, Files 1 and 3 of Database 2 and Files 3 and 4 of Database 3.
final display selected by the user. Initially, the file number order may be represented as the following ordered pairs (database number, file number): (1,2), (1,3), (2,1), (2,3), (3,3) and (3,4). The search module 183 begins by loading the full contents of tile first database's C04 file (213, member of 212), since ordering is by field number four. A comparison of the ordered contents of the C04 file to the two &#x201c;hit&#x201d; file numbers for database 1 indicates that file 3 should be displayed before file 2. This process is repeated for databases 2 and 3, resulting in a final sorted list of: (1,3), (1,2), (2,1), (2,3), (3,4), (3,3).
Now that the search module 183 has sorted the complete hits list, the numeric pairs are translated to field list strings 212. The search module 183 begins by looking in the COI file 210 of Database 1's contents table 209. In this example, the COI file 210 indicates that the field information for file 3 begins at position 112. Further, because 112 is the third and final number stored in the COI file 210, and the total file length for the COF file 211 is 172, the field information for file 3 ends at position 172. Reading the data in the COF file 211 from position 112 to 172, the search module 183 gives the fields for the file, including a file name (field one) of &#x201c;1uniq.htm&#x201d;, a title field (field four) of &#x201c;Unique appears only once&#x201d;, and a closing date field, with blank fields in between. The search module 183 sorts these fields and composes a string in which field four is presented first, followed by the database name, followed by a number of other mostly blank fields (excluding the file name), and concluding with the file date. This string is output to the display. A similar process is carried out for each file hit, allowing a total of six field strings to be output to the dialog display 112.
In the case of file 1, a DSF 205 entry exists, so the first two long integers in the DSI file 204 indicate the beginning and ending ranges for this entry in the DSF file 205. The search module 183 temporarily extracts this segment into main memory 106 and examines it. The layout of information in this segment is determined by first examining the last four bytes of this segment, and translating it into a number. The number is the number of elements in the segment's locator string 224, which immediately precedes the last four bytes of the segment. The search module 183 knows that each locator string 224 entry is twelve bytes long, and thus the locator string 224 is 1200 bytes long if the number of elements is 100. The search module 183 then examines the first entry in the locator string 224. This entry, as is true of all the entries, codes three long integers in its twelve bytes. The first four bytes code the word number 228 for the first indexed word in the file. For example, the file may begin with the word &#x201c;zebra&#x201d; and end with the word &#x201c;aardvark&#x201d;, but since &#x201c;aardvark&#x201d; lexically precedes &#x201c;zebra&#x201d;, &#x201c;aardvark&#x201d; is considered the first indexed word in the file. The second four bytes indicate the character position index 230 information for this first word, which should be 0, indicating the beginning of this DSF 205 segment. The third set of four bytes indicates the start of the slot position index 232 information for this first word, which will thus be the position in this DSF 205 segment at which the word slots list 222 information begins. Thus, the DSF 205 segment has been divided into four parts, including the character positions 220 addressed by the second byte of each locator string 224; the word slots list 222 addressed by the third byte of each locator string 224; the locator string 224, in this case containing 100 twelve-byte segments; and the number of elements in locator string 226, in this case 100.
As stated earlier, if the word number for &#x201c;unique&#x201d; in database 2 is 122, the 20 locator string 224 is searched for an entry whose word number portion is 122. Once this locator string 224 entry is found, the second long integer in the locator string 224 is read and interpreted, for example, a value of 68. Following this, the next locator string 224 entry is read and interpreted, for example, a value of 76. Thus, the eight bytes starting at 68 and ending at 76 in this segment indicate the starting positions for the word &#x201c;unique&#x201d; in file 1. Since these bytes are interpreted as four-byte long integers, this indicates that &#x201c;unique&#x201d; occurs twice in file 1. For example, the first long integer could indicate that &#x201c;unique&#x201d; begins at character position 100 and the second long integer could indicate another instance beginning at character position 200.
With this information, plus the knowledge that &#x201c;unique&#x201d; is six characters long, the search module 183 is able to identify character positions 100 to 106 and 200 to 206 of 10 file 1 in database 2 as the location of the two instances of the search term in this file. These text ranges are indicated through operations such as highlighting, and the file is finally displayed for the user. Of course, the search module 183 treats the character positions in the remaining files in a similar fashion for indicating or highlighting the terms for a user.
Block 406's functions regarding HTML format files are more fully illustrated by FIG. 4A. The format is first determined to be an HTML file or a non-HTML file at block 417. If the file is not an HTML file, a fast and straightforward string analysis method is used to determine the locations of words within the displayable text string of the file. For example, if a file consists solely of the string &#x201c;hello, world&#x201d;, the first word occupies file positions 1-5, and the second word occupies file positions 8-12. Once the search engine reports that &#x201c;world&#x201d; is in the file, it determines its file positions so the word can be set off with different color text or by some other means. If the file position information for the word is not accurate, then the retrieved word will not be highlighted accurately.
The string analysis method first requires obtaining an index string wherein all visible characters occupy positions absolutely relative to each other. The index string is then parsed into words entered into an index along with the numeric word location in the string. In the &#x201c;hello, world&#x201d; example, the search engine can then go to the absolute position of 8 as the beginning of &#x201c;world&#x201d; instead of the relative position of &#x201c;the end of &#x2018;hello&#x2019; plus 3&#x201d; to get the display data for the word.
HTML files involve major complications for using a string analysis method to determine file positions. HTML control tags are placed in line with visible characters. Some of the tags cause the file position to increase, and some do not. Furthermore, the parameters and tag content can be of unlimited and indeterminate length. A simple HTML file that only displays &#x201c;hello, world&#x201d;, can have thousands of invisible control characters before the first word, thousands between it and the second word, and thousands after that. Furthermore, whether those control characters cause the file position of a visible character to increase or not depends on the type of HTML tag and the interaction of other HTML tags. Consequently, obtaining an accurate index string to parse is immensely difficult when HTML files are involved. Other mark-up language file types, such as SGML, etc., present similar but less egregious problems in obtaining accurate index strings. The method described herein for HTML files can also be used for other types of mark-up language files.
The problem is that there is no known accurate way to determine what the effect of present and future HTML control tags will be relative to the file positions of visible words displayed by an HTML viewer when using a string analysis method. HTML viewer technology includes a text ranging method to determine where visible characters are displayed. Essentially, this method assigns a null value to non-incrementing control tags, including their parameters, and a byte value to tags that cause the display to advance the &#x201c;file position pointer&#x201d; when they are encountered. The technology also includes rules for determining whether the interaction of tags changes their behavior with respect to advancing the file position pointer. An accurate index string representing not only the relative file positions of words within an HTML file but also the starting position can be generated using a text ranging method. However, the method is slow compared to a string analysis method because each byte in the file has to be analyzed individually, and single byte analysis using the text range method requires beginning at the first byte of the html string. Thus, the time required for analysis increases exponentially with increasing lengths of files to be analyzed. The present invention overcomes the inaccuracy of the string analysis method used on HTML files and the slowness of the text ranging method.
At block 418, all HTML control tags and their contents are converted to single characters in the non-displayable range, typically ASCII 1 through ASCII 31. In the same block 418, adjacent strings of these control characters are then combined into just one control character. Thus, the example of &#x201c;hello, world&#x201d;, would be reduced at most to 15 characters regardless of the length and complexity of embedded HTML tags. This is the visible character string.
The process of block 407 on FIG. 4 is straightforward. Link patterns and field matches are designated by the user through the Linking Control Panel depicted by FIG. 11 and the Options for Field Links dialog depicted by FIG. 13. When a user designates a custom link word by entering it in text box 1101, associates it with a specific file (such as a glossary) by entering its path into text box 1102, and then clicks the Add New Link button 1104, instructions for that link have been programmed into the index generator. Likewise, when a user specifies a link pattern by entering it (with or without optional wildcard characters) in text box 1106, associates it with a particular field number by selecting one in the options box 1107, and then clicks the Add New Link button 1108, instructions for that link pattern have been programmed into the index generator. The user selectable options depicted on FIG. 13 allow refinement of the link pattern choices. For example, a user may want to use aliases or synonyms so that &#x201c;equine&#x201d; is also linked when &#x201c;horse&#x201d; is the primary pattern.
Functionally, generating valid links automatically as depicted by block 407 of the database index generation process of FIG. 4 is a two step process. First, the virtual list of link pointers (words and patterns) is checked each time a word is extracted in block 405: If the word is on the list, the virtual list of all the files that will be in the final database (that is, a virtual table of contents) is checked to determine if a link target exists for the link pointer. For example, a pattern of &#x201c;# S.W.2d #&#x201d; might match a potential link pointer of &#x201c;877 S.W.2d 200&#x201d; that designates a file with a field likewise containing &#x201c;877 S.W.2d 200&#x201d; as the target. However, if the target file is not in the virtual table of contents, the pattern will not be designated as a link pointer. This avoids having link pointers that have no target being created.
FIG. 7 is a screen display of an exemplary index generator dialog of the GUI database application program introduced in FIG. 5 as it might appear after a user presses the database generator button 501. The index generator dialog includes a source file location edit box 700, a database output directory edit box 701, a generator type selection box 702, a set link properties or Linking button 703, a New Database Name edit box 704, a Register New Database check box 705, an enable Pause feature button 706, a Run button 707, and an Exit button 708. The index generator dialog is used for registering a database or regenerating the database index 200 from a previously registered but changed database. Should the user press the Run button 707 without changing any of the FIG. 7 parameters, the database indicated is registered and appears as shown at 503 in the database display window 504. If the database has already been registered, the database index 200 is regenerated when the Run button 20707 is pressed. Checking the register new database check box 705 causes the generator to register new databases or to reregister changed databases and add them to the database display window 504. A user might choose to regenerate a database index in this manner if any of the source files in the source file location edit box 700 have been changed or if any files matching the generator type selection box 702 were added or deleted. The Pause button 706 toggles a feature that allows the user to suspend database processing indefinitely. When the pause feature is disabled, the generator completes its tasks faster.
Database indexes are made from documents or files located at a path to a directory or folder indicated in the source file location edit box 700 and according to the file type indicated in the generator type selection box 702. If the documents of the database index are located remotely, e.g., on the World Wide Web (WWW) of the internet, the source file location edit box 700 contains a hypertext transfer protocol address, i.e., an &#x201c;http&#x201d; (HyperText Translation Protocol) address to the location. Of course, other types of addresses/designations are available for remotely accessible files, and these various types of addresses/designations are entered into the source file location edit box 700 in a similar manner. A database index is placed in the location shown in the database index output directory edit box 701 when generated from the selected files. Before pressing the Run button 707, the user can press the Linking button 703 in order to cause the documents of a database to have custom links to one another automatically generated at the same time the database index is generated (see FIG. 11 and related discussion). However, in order to understand searching operations of the software of the invention, at this point it is assumed that links have already been set and a database index has already been generated.
FIG. 8 is a screen display of an exemplary search/retrieval dialog of the GUI database application program introduced in FIG. 5 that is displayed when a user presses the search button 502. The search/retrieval dialog presents the user with a search expression edit box 803 in which the user enters search terms of interest. In this case, the search terms &#x201c;second amended petition&#x201d; (including the quote marks) have been entered into the search expression edit box 803. The search expression edit box 803 supports search expressions of any degree of complexity by using the following techniques: parentheses; phrases set off by double quotations; proximity expressions; single- and multiple-character conflation in any combination of leading, middle, and trailing conflation; and default or overriding explicit Boolean operators, such as AND, OR, XOR, etc. Other search expression techniques are also contemplated.
In addition, the search/retrieval dialog includes default Boolean operator controls 805 to determine how the system&#x2019; interprets multiple words entered in the search expression edit box 803. For example, if only two terms are entered without being surrounded by double quotation marks and the default Boolean operator is AND, the system finds all occurrences of both terms in documents that contain both terms. If the default Boolean operator is set to OR using the same example, the system finds all occurrences of either term in all documents with either term. If the default Boolean operator is set to XOR, the system finds all occurrences of either term only in documents that contain one term but not the other. Further, when checked, a search within current results box 801 causes the system to perform the search called for in the search expression edit box 803 only for those documents found by the previous search.
A field link 925 is illustrated in the text in FIG. 9, in which the underlying text is shown highlighted with selectable color and font different from the surrounding text to indicate the link, where the highlight selections are made in a Search Terms display control 1011 (FIG. 10). When the user double clicks on the field link 925, the system displays the document that the field link 925 targets. To return to the text displayed, the user need only press a jump backward button 916. The document display window 928 then shows the text of the document 812. A found terms display 927 shows that two terms were found in the highlighted document 812 of the documents found window 815. The same information about the document 812 is accessible through activation of a title bar 906. The Document Position controls 800 were set to display the document at the first search term, and the order search results by controls 802 were set to sort the results by database name 503. A database named &#x201c;RTF12231&#x201d; is the first one shown in the selected databases 503, and the system assumes that the user prefers that order.
The search expression edit box 803 shows that the phrase &#x201c;second amended petition&#x201d; was searched for and the document display window 928 shows two instances 926 of the phrase appearing near the center of the screen display for user convenience in determining the context of a term. The terms of the phrase are shown in font attributes determined by the Search Terms display control 1011. The previous search term button 911 is not available because the first search term in the document is displayed and current as indicated by a text cursor 950. The next search term button 912 is available because there is one more instance 926 in the document. Both the next document with search terms button 915 and the previous document with search terms button 914 are shown as available because the document displayed is the thirteenth of forty documents found as shown in the document counter 807.
Also shown in the document display window 928 is a phrase 909, &#x201c;Texas Rules of Appellate Procedure&#x201d;. The phrase 909 is shown in bold italics to indicate that it has a legal pad note attached to it, where the bold italics is determined by a LegalPad Notes display control 1009. Legal pad notes allow a user to create reference notes that are accessible from a document in a manner similar to document access through the field link 925. The LegalPad Notes display control 1009 shows that bold italics is used when the system displays text where legal pad notes are attached. As discussed in relation to FIG. 12, a legal pad button 918 is used to create new legal pads from highlighted text.
A SmartScreen button 900 causes the system to display the same screen shown when the database application program is started (initialized) as in the example embodiment of FIG. 5. The first document in universe button 901, the &#x201c;universe&#x201d; including all files/documents in all selected databases, is not available and thus not highlighted because the document shown in the document display window 928 just happens to be the first document in all of the documents in the selected databases.
The link generator dialog of FIG. 11 also allows a user to instruct the database index generator to insert field links that are based on link field patterns. A field link based on a link field pattern is a pattern sequence found in a file that exactly matches the same pattern sequence that is found in the appropriate field in another file in the database source files. To create these types of field links, a pattern is entered into a New Link Pattern edit box 1106. When the pattern of the New Link Pattern edit box 1106 matches another pattern located in a certain field of another file, a field link can be created between the files. The &#x201c;certain field&#x201d; of a file that is linked to corresponds to a link field number that is selected in a Link Field Number edit box 1107. After the pattern is entered into the New Link Pattern edit box 1106 and the link field number is entered into the Link Field Number edit box 1107, in this case &#x201c;1&#x201d;, an Add New Link button 1108 is pressed and the database index is updated with the new information. The resultant field link pattern corresponding to the link field number then appears in the link field pattern window 1105.
Advantageously, a pattern entered into the New Link Pattern edit box 1106 can use &#x201c;wildcard&#x201d; characters. Wildcard characters are characters such as %, ?, *, and #, where each of the characters has a special meaning. In the embodiment shown, the &#x201c;%&#x201d; character substitutes for any digit, the &#x201c;#&#x201d; character substitutes for any integer greater than zero, the &#x201c;*&#x201d; character substitutes for any number of characters or digits between delimiters, and the &#x201c;7&#x201d; character substitutes for any single character. For example, a pattern &#x201c;# s*d #&#x201d; matches &#x201c;9 sand 977&#x201d;, &#x201c;843 S.W.2d 955&#x201d;, etc. An essentially unlimited number of field link pairs can be created. Of course, any number of wildcard characters may be defined depending on a particular embodiment.
Also of note, files/documents have many different file formats for their respective fields (e.g., WordPerfect&#xae; format). These formats provide for normally hidden fields to contain data about the file such as title, subject, author, etc. A system according to the present invention provides for placing visible fields in the first line of the file with each field separated by a delimiter such as a tab character.
FIG. 13 is a screen display of an optional field links dialog 1300 of the GUI database application program-introduced in FIG. 5 that is displayed when the Options button 1111 is pressed. The optional field links dialog 1300 includes additional options that are available to the user for creating custom field links. An Alias Control section 1301 allows the user to define an unlimited number of aliases in a Current Aliases window 1303 for a link term so that strict correlation between terms linked to target files is not necessary. For example, whenever the term &#x201c;vine fruit&#x201d; appears in any of the database files, the user may want the term to be linked to a glossary file that defines the term. By setting or defining aliases for &#x201c;vine fruit&#x201d; to include alias terms &#x201c;grape&#x201d;, &#x201c;tomato&#x201d;, and &#x201c;raspberry&#x201d;, those words also have a link generated to the glossary file just as &#x201c;vine fruit&#x201d; does. Pressing an OK button 1302 sets the options and restores the display of the linking control panel to its previous state.
In addition to the above described example embodiment, FIG. 14 is an example screen display of a Browser Mode Window showing an HTM document retrieved from the internet using the GUI database application program introduced in FIG. 5. The Browser Mode works in a similar manner as commonly used browsers, such as Netscape Navigator or Microsoft&#xae; Explorer. The internet address of the document is shown in an Address bar 1405. If the user puts the cursor (or focus) on the Address bar 1405 and presses the Return or Enter keyboard key, or presses a Refresh button 1404, the document would again be retrieved from its internet source. By pressing a Back button 1400, the Browser Mode Window displays the previous document that was viewed. By pressing a Forward button 1402, the Browser Mode Window displays the document that was previously viewed before the Back button 1400 was pressed to display the document shown.
Pressing a Stop button 1403 terminates any internet retrieval action currently underway. Pressing a Home button 1406 causes the Browser Mode Window to retrieve and display the document at the specific internet address designated as the &#x201c;Home Page&#x201d; for the Browser Mode Window. Pressing a Search button 1407 causes the Browser Mode Window to retrieve and display the internet search engine page designated by a user option in an Options dropdown menu 1401. A Print button 1408 allows the user to print the document displayed and to set printing options in a dialog box that is displayed. A DB Name button 1409 displays a dialog box and list of previous database names that have been used. The current database selected is shown in a database name label 1412. A more extensive dialog box that allows the user to change other database particulars is also available as a user option in an Options dropdown menu 1401.
A SpeedSave button 1411 immediately saves the displayed document, along with all of its pictures, graphics, images, hypertext links, and layout into the database named in the database name label 1412. The first time the SpeedSave button 1411 is pressed in a session of the software, the same dialog displayed by pressing the DB Name button 1409 is displayed to safeguard against the user inadvertently saving a file into a forgotten about database. Double clicking the database name label 1412 also displays the same dialog. Depending upon the settings for the database particulars accessible under the Options dropdown menu 1401, the file can be saved as a normal &#x201c;Text&#x201d; file, an HTM file without images, an HTM file with images linked to their internet source, or an HTM file with all images retrieved and saved on the local computer's hard drive. Pressing an Exit Browser button 1410 causes the software of the system 100 to create a fully indexed and searchable database of all files saved into the database name shown in the database name label 1412 according to the default behavior. The database is automatically registered and shown on the database display window 504. The default behavior can be changed to accommodate a variety of user preferences through the appropriate selection on the Options dropdown menu 1401. A document location label 1413 indicates to the user whether the source of the document being viewed is remote or local, and the label 1413 changes automatically when the viewed document changes its source. A status bar message 1414 changes as appropriate to give the user information about the status of the Browser Mode Window. A Browse Mode label 1420 indicates to the user whether the software is functioning in its Browser Mode or its Viewer Mode.
The document depicted in FIG. 14 has several elements referenced in order to illustrate the capability of a system according to the present invention for allowing the user to easily edit content and arrangement of documents saved in the Browser Mode. For example, FIG. 14 illustrates a &#x201c;Contact Information&#x201d; graphic 1415, a &#x201c;What's New&#x201d; graphic 1417&#x2033; a &#x201c;Services&#x201d; graphic 1418, a Footer Text 1416, and a Body Text 1419 which have all been manipulated, deleted, or changed as shown in FIG. 15.
FIG. 15 is an example screen display of the HTML document of FIG. 14 after being saved and edited in the Browser Mode window. FIG. 15 shows the &#x201c;Contact Information&#x201d; graphic 1415 as having been moved in the left column of the document, which is now shown as Contact Information graphic 1500. The &#x201c;What's New&#x201d; graphic 1417 and the &#x201c;Services&#x201d; graphic 1418 have been deleted. A new &#x201c;Super Sweeps&#x201d; graphic 1503 has been added. The Footer Text 1416 has been moved to be the first paragraph of the document's new body text. The Body Text 1419 has been moved down and edited to delete the text &#x201c;(&#x201c;IDC&#x201d;)&#x201d; from it. Since the internet address shown in the Address Bar 1405 of the document of FIG. 14 has been saved, the Address Bar 1501 has changed to a pathname to reflect the document's location on the local computer's hard drive. The document location label 1413 indicating &#x201c;internet&#x201d; has also changed to be Document Location Label'1502 indicating &#x201c;Local&#x201d; to help ensure that the user knows the source location of the document being viewed. The editing process automatically makes all adjustments to HyperText links and other HTM codes associated with text or graphic elements that are added, deleted, or moved. When the SpeedSave button 1411 is pressed, the edited file is saved after the user selects an option to save it under a different name or to replace the existing file.
Jensen, Robert Leland, Smith, Daniel Victor
Current Assignee: Indacon Incorporated
Sponsoring Entity: Integrated Data Control, Inc.
US 7,836,043 B2
Sponsoring Entity: Indacon Incorporated
Current Assignee: Daniel Victor Smith, Robert Leland Jensen
Sponsoring Entity: Daniel Victor Smith, Robert Leland Jensen