Abstract:
A method of extracting sections of a data stream, the sections including a set of sequences. Each sequence is encoded separately and coupled together to define a section. The method involves determining a combination of at least two sequences of the set, comparing the combination of sequences with sequences in the data stream, and rejecting or accepting extraction of the section of the data stream based upon the result of the comparison. If the combination of sequences does not include a start and end marker for the section, a search for the start and end markers is carried out before the section is extracted.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is a continuation of PCT International Application No. PCT/GB2008/000184, filed Jan. 18, 2008, which claims priority under 35 U.S.C. § 119 to Great Britain Patent Application No. 0700926.9, filed Jan. 18, 2007, and Great Britain Patent Application No. 0700928.5, filed Jan. 18, 2007, the entire disclosures of the aforementioned applications are herein expressly incorporated by reference. 
         [0002]    The present application is also related to U.S. patent application Ser. No. ______, entitled “A Method of Filtering Sections of a Data Stream” and filed on even date herewith, which is a continuation of PCT International Application No. PCT/GB2008/000172, filed Jan. 18, 2008, which claims priority under 35 U.S.C. § 119 to Great Britain Patent Application No. 0700926.9, filed Jan. 18, 2007, and Great Britain Patent Application No. 0700928.5, filed Jan. 18, 2007, the entire disclosures of the aforementioned applications are herein expressly incorporated by reference. 
     
    
     BACKGROUND AND SUMMARY OF THE INVENTION 
       [0003]    This invention relates to a method of extracting sections of a data stream. 
         [0004]    There are many instances where a user wishes to find and extract only certain data types from a larger body of data. The data is typically presented as a data stream, whether from a store, or in real time, and if all of the data were processed fully, this would be very slow. 
         [0005]    A particular example of searching data streams is in SPAM filtering where it is desirable to extract data having a particular label, or end point identifier, such as an email address, a domain name, a uniform resource locator, or telephone number. 
         [0006]    In accordance with the present invention, a method of extracting sections of a data stream, the sections comprising a set of sequences, wherein each sequence is encoded separately and coupled together to define the section, comprises determining a combination of at least two sequences of the set; comparing the combination of sequences with sequences in the data stream; and rejecting or accepting extraction of the section of the data stream based upon the result of the comparison; wherein if the combination of sequences does not include a start and end marker for the section, a search for the start and end markers is carried out before the section is extracted. 
         [0007]    The present invention provides a high performance generic extraction framework which allows data stream content to be processed at high speed and used in a real time context. 
         [0008]    Preferably, extraction of the section is accepted if the combination of sequences in any order matches stored sequences in the section of the data stream. 
         [0009]    Preferably, extraction of the section is rejected if the combination of sequences does not match any of the sequences in the section of the data stream; and thereafter the search continues for further instances of the combination of sequences in another section. 
         [0010]    Preferably, a sequence comprises a series of bits having a predetermined format, such as an anchor, or a bridge. 
         [0011]    Preferably, the anchor is a statistically rare, or low probability sequence in the data stream. 
         [0012]    Typically, the probability of occurrence is less than about 1%. 
         [0013]    Preferably, the combination of sequences comprises an anchor and a sequence adjacent to the anchor. 
         [0014]    This improves throughput by reducing the likelihood of a match. 
         [0015]    Preferably, the combination of sequences comprises at least the first and last sequence of the section. 
         [0016]    This allows the section to be extracted immediately if a match is found, whereas a successful match with a combination of sequences which does not include both start and end points requires the additional step of identifying these before extracting the section. 
         [0017]    In one embodiment, the combination of sequences comprises more than one sequence associated with an anchor; wherein the combination of anchor and sequences to form the section is determined; and wherein the section is only extracted if all sequences forming the section are present. 
         [0018]    This has the effect of only extracting sections where there is a complete match. 
         [0019]    Preferably, searches for combinations of sequences are carried out in parallel on different sections of the data stream. 
         [0020]    This could be by splitting the data stream, or looking for different combinations of sequences in the same part of the data stream. 
         [0021]    Preferably, each sequence comprises a series of bits of data, or multiple bytes of data. 
         [0022]    Preferably, the section comprises an end point identifier, such as a domain name; an email address; a uniform resource locator; or a telephone number. 
         [0023]    Choosing a particular type of end point identifier allows a large amount of irrelevant data to be immediately discarded without having to search for a specific instance. For example, a SPAM filter could search for the domain name structure, so data lacking that format would not need to be considered. 
         [0024]    Preferably, each sequence is encoded in a separate state machine and multiple state machines are combined to represent the section. 
         [0025]    This makes the method more flexible. 
         [0026]    Preferably, a bridge provides a transition between separate state machines representing the sequences of the section. 
         [0027]    This allows the super state machine to be built up. 
         [0028]    Preferably, the method further comprises filtering the extracted sections of the data stream; the filtering comprising determining a set of characters of interest; testing each section of the data stream for the presence of one or more of the set of characters of interest; and extracting sections in which at least one of the characters is present. 
         [0029]    Having extracted sections which satisfy a minimum requirement, for example having a domain name format, then filtering is carried out to reduce the number of results more specifically, such as only emails having “.roke.” in their address. 
         [0030]    Preferably, the method further comprises determining a further set of characters of interest; testing for at least one character from the further set of characters in the portion of the data stream; and extracting sections in which at least one of the characters from the further sets of characters is also present in the section. 
         [0031]    This step can be repeated until the amount of data which needs to be tested for a complete match is reduced to a reasonable amount. 
         [0032]    Although, all the processing steps could be carried out in real time, preferably, the extracted sections are stored in a store and extracted as and when needed. 
         [0033]    Preferably, the extracted sections are input to a comparison stage; compared with specific examples of end point identifiers; and discarded if the section does not match a specific example in the comparison stage. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWING FIGURES 
         [0034]    An example of a method of extracting sections of a data stream will now be described with reference to the accompanying drawings in which: 
           [0035]      FIG. 1  is a block diagram of a typical system to which the method of the present invention is applied; 
           [0036]      FIG. 2  illustrates domain name and DNIV state machines; 
           [0037]      FIG. 3  illustrates state machines when used with the ‘.’ anchor point; 
           [0038]      FIG. 4  illustrates state machine modifications for diagram operation; 
           [0039]      FIG. 5  illustrates an example of extracting a page title; 
           [0040]      FIG. 6  shows an example of searching for a hyperlink; 
           [0041]      FIG. 7  shows an example of a search for a data and time format; 
           [0042]      FIGS. 8A-8D  illustrate exemplary extraction and filtering arrangements for electronic mail in accordance with the present invention; and 
           [0043]      FIGS. 9A-9D  illustrate exemplary extraction and filtering arrangements for URLs in accordance with the present invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0044]    The present invention describes a technique which allows structural forms of data to be identified and extracted, such as identifying and extracting data based on it being a domain name, an email address, or a data and time format. Other examples include, in search engine indexing automating the process of document retrieval and classification, e.g. if using a web spider for extraction of hyperlinks from html documents in order to construct a list of URLs to subsequently retrieve. Given the vast quantities of html content available on the Internet efficient extraction of hyperlinks from web pages is required. Another example is use in real time SPAM classification. Part of SPAM classification involves the identification of URLs/URLs, domain names or email addresses associated with SPAM objects. Such identification is used with whitelist/blacklists of SPAM items to filter out SPAM content. Due to the large quantities of SPAM present in modern communications networks, an efficient identification and filtering of SPAM content is desired. 
         [0045]    A section of data, typically representing an end point identifier, label, or meta-data, which section is to be identified and extracted, is broken down by encoding each subsection of the format within an individual state machine. Particular characters can then be used as bridges to move between one state machine and another, where a bridge character is used to move between the different machines describing a meta-data format. Thus, a complete format is defined by creating a number of smaller machines that describe each subsection of the format. The machines are then used with the bridges to create a super machine that describes the entire format. Complete traversal of the super machine from its start state to its terminal state is used to identify the end point identifier format. Anchors are signatures that are associated with the label of interest, in particular, single characters or sequences of characters that are statistically rare in free text, or binary data. This property can be used to quickly lock on to a location in free text that has a higher than average probability of being a subpart of the label of interest. 
         [0046]    For example of the present invention may be described with respect to identification and extraction of a hyperlink consisting of a sequence of characters followed by a domain name e.g. href=http://www.roke.co.uk. In general a hyperlink can be identified by recognising the domain name part of the format. The domain name part of the hyperlink can be described using the following syntax: 
         [0000]    DNIV domain.domain[.domain] DNIV. 
         [0047]    Within this syntax the following subgroups are identified: 
         [0000]    [ ]—square brackets are used to signify one or more optional components.
 
DNIV—this is the set of characters that are illegal within the domain name part.
 
domain—this is the set of character that are legal within the domain name part.
 
.—the dot symbol is a bridge between two domain name parts.
 
         [0048]    In general the set of characters that compose the DNIV, and domain name parts of the syntax are defined by the standards for internet based computer names. DNIV is also defined by the expression-!domain. 
         [0049]      FIG. 1  illustrates a typical system for operating the method of the present invention. An input data stream  30  which could be from a store (not shown), or a real time data source, is input to a processor  31  which applies the method of the present invention. Whenever a section of the data stream satisfies the test criteria, the section is output  39  to a store  32 , or output  40  to a comparison stage  33 , such as a look up table. Data which is not extracted is discarded  34 , although the discarded data steam could be subjected to additional tests, for example for an alternative label, or end user identifier. For convenience, the extracted sections of data may be stored before an optional filtering step  35  is applied and the sections which are filtered out can be returned to the store, or sent on for further processing in the comparison stage  33 . Sections which are not extracted in the filter stage  35  are discarded  36 . Thus, the output  38  of the extracted and optionally, filtered data stream may be obtained from the store  32 , or as an output  39  from the comparison stage  33 . 
         [0050]    The mechanism for extracting sections of the data stream is described in more detail with respect to  FIGS. 2 and 3 . Let a single valid domain name character be Ch d , the term !Ch d  means not in the set Ch d  then an example of a possible state machine for the domain name is defined in  FIG. 2 . In the example the ‘.’ symbols are examples of bridge characters. The ‘.’ character is used as a bridge between the sub-domains of the complete domain name. 
         [0051]    From start domain name    1 , if a valid domain name character Ch d    2  is identified, the test moves on to the next point  3 . If an invalid character  4 , or bridge character  5 , are found, the test fails  6 . From point  3 , an invalid character  7  causes a fail  8  and a valid character  9  loops back on itself, but a bridge character  10  moves the test on to the next point  11 . From point  11  a bridge character  12 , or an invalid character  13  cause a fail  6 , whereas a valid character  14  moves on to the next point  15 . A bridge character  16  moves to point  11 , a valid character  17  loops back on itself to point  15  and an invalid character  18  moves to the end point, end domain name    19 . For start DNIV , an invalid character moves the test to end DNIV  (not shown). Having determined a start and end point for the domain name, the series of sequences making up this section of the data stream can be extracted for storage, or further processing. In the state machine the domain name format is identified in a left to right fashion as the text is examined. However, in principal the sub parts of the format can be identified in any order. 
         [0052]    The label or end point identifier which is used to determine which sections of the data stream are extracted is made up of parts, some of which may be statistically rarer than others in free text. Consequently, an effective method to increase the practical performance of the identification algorithm is to look for these parts before the others. These parts, known as anchor points, can be used to ‘lock on’ to a position in the data stream that may be an instance of the end point identifier type sought. 
         [0053]    Once an anchor point has been found in the data stream, validation of the data is carried out by parsing outwards (forward and backwards) around the anchor point. For the domain name example the ‘.’ symbols are statistically rarer in free text than the other characters contained in the domain name format. This modification splits the domain name algorithm into two distinct machines as shown in  FIG. 3   a  and  FIG. 3   b . The identification algorithm first finds the signature ‘.domain’ using the machine defined in  FIG. 3   a  and then starting at the ‘.’ position in the data stream moves backwards and applies the smaller state machine defined in  FIG. 3   b . The domain name part is validated first as failure at any point allows the algorithm to continue moving forward through the data stream without expending unnecessary effort on validating the smaller part. From start point, start. domain name    41 , a bridge character  42  moves the test to the next point  43 , where an invalid character  44  causes the test to fail  45  and a valid character  46  moves on to the next point  47 . From here the process steps and results are the same as for the equivalent reference numbers in  FIG. 2 . From point  43  a bridge character  48  moves back to start. domain name    41 . The machine in  FIG. 3A  moves from left to right starting at point  41 , whereas the machine in  FIG. 3B  moves from right to left starting at  41 . So for the pattern roke.co.uk,  FIG. 3A  would find the part ‘.co.uk’ at character position  5 .  FIG. 3B  would then start at position  5  and move from right to left to find the part ‘roke’. The pattern roke.co.uk is then subsequently extracted. 
         [0054]    The series of steps in  FIG. 3B  starts at the same position in the text as point  41 , a valid character  148  takes us from start domain name    41  to the next state  149 . From this state  149  an invalid domain name character  150  identifies the start of the complete pattern  151  (i.e. start domain name  or the ‘r’ in roke.co.uk). A valid domain name character  152  loops back on itself. A dot  153  indicates another sub-domain and moves us to the next state  154 . From here a valid domain name character  155  moves us back and an invalid domain name character  156  results in failure  157 . 
         [0055]    Finally performance can be further improved by exploiting the machine word size. The meta-data format is defined as a collection of bytes. However, modern processors have register sizes that are multiple bytes wide. The machine register size can be exploited by adapting the state machines so that the state machine transitions are labelled with multi byte values rather than single byte values. In this instance the input byte stream is processed multiple bytes at a time instead of a single byte at a time. Thus, in effect the multi-byte state machine runs multiple instances of the single byte state machine each starting at different byte offset, i.e. the throughput is increased by processing the data in multiple machines operating in parallel. 
         [0056]    An example of a simplified ‘.domain’ state machine that processes two bytes at a time is shown in  FIG. 4 . Starting the state machine at the upper most arc in  FIG. 4 , the machine is entered when any of the 16 bit patterns defined by Ch d . or .Ch d  is found. Let a single valid domain name character be Ch d , the term !Ch d  means not in the set Ch d . The term Ch d Ch d  means a valid domain name character followed by a valid domain name character. The term Ch d !Ch d  means a valid domain name character followed by an invalid domain name character. The term ! Ch d Ch d  means an invalid domain name character followed by a valid domain name character. The term Ch d . means a valid domain name character followed by a dot character. The term Ch d  means a dot character followed by a valid domain name character. 
         [0000]    The machine is started by finding a pair of bytes defined by either of the following sequences Ch d . or .Ch d    50  followed by a valid domain name that satisfies this version of the domain name state machine. 
         [0057]    Thus, the algorithm no longer looks for the ‘.’ symbol specifically but searches for a 16 bit sequence containing the ‘.’ symbol. This modification also has the advantage that a 16 bit sequence containing an ‘.’ is statistically rarer than a bare‘.’ symbol. Consequently, the algorithm rejects a larger fraction of potential alignments by enforcing the formatting of the characters around the ‘.’. 
         [0058]    The machine is started by finding a pair of bytes defined by either of the following sequences, Ch d . or .Ch d    50  and in this case the test moves to the next point  51 . At point  51  if the next two bytes are Ch d . or .Ch d  the search loops back on itself  52 . At point  51  if the next two bytes are Ch d Ch d    53  the test moves to the next point  54 . At point  54  if the next two bytes are Ch d . or .Ch d    55  the search moves back to point  51 . At point  54  if the next two bytes are Ch d Ch d  the search loops back on itself  56 . At point  54  if the next two bytes are any of the following Ch d ! Ch d  or !Ch d Ch d  or !Ch d !Ch d    57  the search has failed  58 . At point  54  if the next two bytes are Ch d . or .Ch d    59  then the search moves to point  60 . At point  60  if the next two bytes are Ch d . or .Ch d    61  then the search moves to point  51 . At point  60  if the next two bytes are Ch d Ch d    62  then the search loops back on itself. At point  60  if the next two bytes are Ch d !Ch d  or !Ch d Ch d  or !Ch d  ! Ch d    64  then a domain name has been found  65 . At point  60  if the next two bytes are Ch d . or .Ch d    63  then a domain name has been found  69 . At point  69  if the next two bytes are Ch d Ch d    66  then the search moves to point  54 . At point  69  if the next two bytes are Ch d . or .Ch d    67  then the search moves back to point  51 . 
         [0059]    In summary, the invention uses a set of state machines to describe the format of an end point identifier, label or meta-data. A super machine is created by linking the smaller machines using bridge characters. Anchor points may be defined in the format, so these are identified first to increase throughput. A further feature is that multi-byte versions of the state machines may be defined to enable the input to be processed in parallel. Rather than process the byte stream 8 bits at a time a pointer is used to access the data several bytes at a time. Each vertex of the machine is labelled using a multi byte value. The value of the sequence of bytes pointed at by the pointer is then used to traverse the vertices of the machine. This means that several bytes of the input are processed for each transition of the machine which improves the throughput. In effect this can be thought of as running several single character machines in parallel i.e. the state machine design exploits the machine word size to enable parallel processing in software. 
         [0060]    More generally, in the example of searching for a hyperlink_. The pattern is: 
         [0061]    href=“http://URL” 
         [0000]    In this case the pair of labels are: 
         [0062]    href=“http://and” 
         [0000]    The labels are separated by a sequence of characters from the valid set of characters that can be used within a URL. The example is shown in  FIG. 5   
         [0063]    Starting at point  78 , the sequence href=“http:// 79  takes the search to point  80 . From point  80  a symbol from the set ChURL (the set of valid URL characters)  82  takes the search to point  85 . From point  80  a symbol that is not in the set ChURL (!ChURL)  81  takes the search to point  83  and the search fails. From point  85  a valid URL character  86  loops the search back to point  85 . From point  85  an invalid URL character  84  results in failure  83 . From point  85  the quote character  87  takes the search to point  88 . At this point a valid hyperlink has been found and can be extracted. 
         [0064]    When searching a page for a title, having a pattern 
         [0065]    &lt;title&gt; page title &lt;/title&gt; 
         [0000]    In this case the pair of labels are: 
         [0066]    &lt;title&gt; and &lt;/title&gt; 
         [0000]    The labels are separated by a sequence of characters from the set A-Z, a-z, 0-9 as illustrated in  FIG. 6   
         [0067]    Starting at  70  the sequence &lt;title&gt; 71  takes the search to point  72 . At point  72  the characters A-Z, a-z, 0-9 ( 73 ) loop the search back to point  72 . At point  72  the symbols in the set !(A-Z, a-z, 0-9)! (&lt;/title&gt;)  76  take the search to point  77  and the search fails. At point  72  the sequence &lt;/title&gt;74 takes the search to point  75  and the end. Thus, the identification of the pair of sequences &lt;title&gt; &lt;/title&gt; identifies a page title between them. 
         [0068]    Alternatively, when the search may be for a Date-Time format. 
         [0000]    The pattern is: 
         [0069]    Jan. 1 2008 SPACE10:20:22 
         [0000]    In this case the pair of labels are: 
         [0070]    Month and :NUM NUM !(NUM) 
         [0000]    The month can be one from the set of patterns Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec. NUM indicates one of the characters 0-9 and !(NUM) means not one of the characters 0-9. In this case a bridge character is needed to link the date and time parts. A suitable bridge is the SPACE character after the year. The example is shown in  FIG. 7 . 
         [0071]    Starting at point  89 , a valid month  90  moves the search to point  91 . From point  91  any character  92  takes the search to point  93 . At point  93  any character loops the search back to point  93 . At point  93  the SPACE character  95  takes the search to point  96 . At point  96  any character  97  takes the search to point  98 . At point  98  any character  99  loops the search back to point  98 . At point  98  the sequence: NUMNUM!(NUM)  100  completes the search  101 . 
         [0072]    The present invention allows sections of data to be identified and extracted. Although the examples have been described using hyperlinks and domain names, the invention can be applied to many other end user identifier types including email address identification; URI/URL identification; Session Initiation Protocol (SIP) URI identification; E.164 telephone number detection; tag detection in other data formats; IP addresses, port range, protocol and session identifier detection; xml data structures, xml objects; HTML structures and objects; and detection of content types and identification of content from packet payloads. The basic method can be improved to increase throughput and processing speed by use of an anchor structure, or looking for an ngram containing an anchor symbol. 
         [0073]    The combination of separate encoded sequences represented by smaller state machines into a group of state machines to produce the full format of an end user identifier, or label, allows labels of arbitrary complexity to be detected. Further improvements in throughput arise from the use of parallel processing, exploiting machine word size to run several instances of a super machine in parallel. 
         [0074]      FIGS. 8A-8D  illustrate exemplary extraction and filtering arrangements for electronic mail in accordance with the present invention. In the arrangement of  FIG. 8A  a separate filtering and extraction server (e.g., SPAM filter)  805 A is provided for performing the extraction and filtering described above. SPAM filter  805 A can be executed by an application specific integrated circuit (ASIC), microprocessor executing computer code, field programmable gate array and/or the like to perform the extraction and filtering functions. The SPAM filter  805 A is coupled to an e-mail server  810 A, which in turn is coupled to a terminal  815 A. Terminal  815 A can be any type of terminal, including a desktop computer, laptop computer and/or a wireless computing device (e.g., a wireless telephone and/or e-mail device). Terminal  815 A includes an e-mail client  820 A for receiving the e-mails that pass from SPAM filter  805 A through e-mail server  810 A to terminal  815 A. Terminal  815 A can include an application specific integrated circuit (ASIC), microprocessor executing computer code, field programmable gate array and/or the like to execute the e-mail client. The e-mails can be output on printer  825 , display  830  or any other type of output device. In particular, if an e-mail is filtered then it would not be provided to terminal  815 A, whereas those that are not filtered would be provided to the terminal. In other words, the e-mails that are discarded by SPAM filter  805 A are those that are passed to the terminal, whereas those that are output from lookup table  6  are filtered and not passed to the terminal. 
         [0075]    In the arrangement of  FIG. 8B  SPAM filter  805 B can be included in e-mail server  810 B. In this arrangement SPAM filter  805 B can be a separate program on the same hardware as the e-mail server  810 B and/or can be a program executing within the e-mail server program. 
         [0076]    In the arrangement of  FIG. 8C  SPAM filter  805 C can be included in terminal  815 C. In this arrangement SPAM filter  805 C can be a program executing on terminal  815 C. 
         [0077]    In the arrangement of  FIG. 8D  SPAM filter  805 D can be included in e-mail client  820 D. In this arrangement SPAM filter  805 D can be, for example, a plug-in for e-mail client  820 D. 
         [0078]      FIGS. 9A-9D  illustrate exemplary extraction and filtering arrangements for URLs in accordance with the present invention. In the arrangement of  FIG. 9A  a separate server  905 A is provided for performing the extraction and filtering described above. Server  905 A can include an application specific integrated circuit (ASIC), microprocessor executing computer code, field programmable gate array and/or the like to perform the extraction and filtering functions. The extraction and filtering server  905 A is coupled to a web server  910 A, which in turn is coupled to a terminal  915 A. Terminal  915 A can be any type of terminal, including a desktop computer, laptop computer and/or a wireless computing device (e.g., a wireless telephone and/or e-mail device). Terminal  915 A includes a browser client  920 A for browsing web pages that pass from the extraction and filtering server  905 A through web server  910 A to client  915 A. Terminal  915 A can include an application specific integrated circuit (ASIC), microprocessor executing computer code, field programmable gate array and/or the like to execute the e-mail client. The web pages can be output on printer  925 , display  930  or any other type of output device. In particular, if a web page passes through the filter then it would not be provided to terminal  815 A, whereas those that are not filtered would be provided to the terminal. In other words, the web pages that are discarded by the extraction and filtering server  805 C are those that are passed to the terminal, whereas those that are output from lookup table  6  are filtered and not passed to the terminal. 
         [0079]    In the arrangement of  FIG. 9B  extraction and filtering server  905 B can be included in web server  910 B. In this arrangement server  905 B can be a separate server executing on the same hardware as the web server  910 B and/or can be a program executing within the web server program. 
         [0080]    In the arrangement of  FIG. 9C  extraction and filtering server  905 C can be included in terminal  915 C. In this arrangement server  905 C can be a separate server or can be a program executing on terminal  915 C. 
         [0081]    In the arrangement of  FIG. 9D  extraction and filtering server  905 D can be included in browser client  920 D. In this arrangement server  905 D can be, for example, a plug-in for browser client  920 D. 
         [0082]    Although  FIGS. 8A-8D  and  9 A- 9 D are described in connection with so-called blacklists, in which a match with the lookup table causes the email or web page to be excluded and not delivered to the terminal, the present invention can also be implemented with so-called whitelists. In this case a match with the lookup table allows the email or web page to be delivered to the terminal and a failure to match with the lookup table excludes the email or web page from being delivered. 
         [0083]    It will be recognized from the discussion of  FIGS. 8A-8D  and  9 A- 9 D above that the designation of the filtering and extraction element as being a server is used to cover a variety of different arrangements, including a physical server, a server program, a regular executable program and a plug-in program. Accordingly, the term server should be interpreted accordingly in connection with the claims. 
         [0084]    The foregoing disclosure has been set forth merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to persons skilled in the art, the invention should be construed to include everything within the scope of the appended claims and equivalents thereof.