A METHOD, DEVICE, AND SYSTEM FOR DATA REPRESENTATION

Abstract: Various methods are known for data representation, a number of which describe encoding/compression of data, using "dictionaries" or references to recurring sequences of bytes, and mapping of frequent events onto the short codes. The method, device, and system according to the invention constitute a representation method using a priori knowledge, stored in a so-called a priori model, holding information on many forms of dependencies among parts of the data. The method can also be used for automatic learning by adapting the a priori model according to processed data.

Description: Title: A method, device, and system for data representation The invention relates to a method for data representation. The invention also relates to a system for data representation. Furthermore, the invention relates to a device for data representation. One of the methods for data representation is data compression. In addition, methods for encoding and/or conversion of data are known. Many methods are known for data compression, which are based on recognition of recurring sequences of symbols (bits or bytes) . An extensive overview of such methods can be found in the book "Text Compression" (T.C. Bell, J.G. Cleary, Ian H. Witten, Prentice Hall, 1990, ISBN 0-31-911991-4) . Also methods are known based on general a priori knowledge. An extensive example is described in US patent 0083393. Typically, a priori knowledge is incorporated as one or more tables of frequently occurring sequences of symbols. A special kind of use of a priori knowledge is to model data according to a formal context-free grammar. Such a method is described in the article "Source Encoding Using Syntactic Information Source Models" (R.D. Cameron, IEEE Trans. Info. Theory, Vol. 34, No. 4, p. 834-850) for strong compression of Pascal programs. However, context-free grammars can model only a very limited part of all possible data types. In addition, described methods for exploiting probabilistic dependencies are very limited, while the method can't handle any unexpected circumstances, such as typing errors. In principle, there are methods to autodidactically learn underlying relationships from (sample) data, such as a method described in "An introduction to the application of the theory of probabilistic functions of a Markov process to automate speech recognition" (Bell Syst. Techn. Journ. , Vol. 62, No. 4, p. 1035-1074) . However, such methods require extreme amounts of computer time, are based on too simple models, and demand lots of memory. The invention aims to provide a method, system, and device for efficient data representation. It is assumed that this data (also called source data) is already available in some digital format. Therefore, the goal is to find a more efficient representation, as well as methods for encoding, decoding, and other relevant operations. The invention makes use of general a priori knowledge on the source data. Examples of such a priori knowledge are: grammar, morphology, formatting, style of writing, instruction set, register allocation, geometric dependencies, etc. Therefore, for some implementation, especially data to which the a priori knowledge applies will be very efficiently represented (see figure 1) . For instance, while Dutch text (which happens to be the domain of the a priori knowledge) could be very efficiently represented, English text would probably be encoded less efficiently. The main idea behind the new approach is that one of the most efficient representations of data is its natural meaning. In order to infer this meaning - to some extent - a very large amount of knowledge on underlying structures and dependencies is required, while algorithms to apply this knowledge tend to be extremely intricate. The method, system, and device according to the invention entail several means and/or steps to handle this complexity, both conceptually and implementationally. For instance, whether a text is written in present or past tense can be encoded using only 1 bit. To get this far, verbs need be recognized (grammar) , and their inflection rules need be known (morphology) . This knowledge can be captured relatively easily using a combination of explicit specification and automated learning from sample data. A precise description will now be given of what is meant by efficient representation, structured as a list of a minimum number of characteristics and feasible operations to be accomplished simultaneously. These features and operations are directly related to the way data is being represented, but have not been recognized as such until now. It is often feasible to yield an encoded data size considerably smaller than the size of the corresponding source data. This increase in information density, also called data compression, is of major economical interest. For complex data types, the information density accomplished by the invention is remarkably higher than achieved by currently known data compression algorithms. For small amounts of data, the difference is particularly large. For efficiently handling of encoded data, it need be quickly accessible. With the invention, data can remain randomly accessible without lowering information density. Associative access entails swift access to parts of the data, based on characteristics of its contents. For example, searching for all paragraphs in an encyclopedia which contain a particular word. The invention relates to a system to allow for fast searching by analyzing only a tiny part of the encoded data. Being based on general a priori knowledge on a data type, this knowledge can also be applied to determine whether, and in what way, the data adheres to given expectations. It can be detected, for instance, whether words are spelled correctly, whether grammar is used correctly (within certain limits) , and with what style a text is written. Moreover, it is possible to efficiently determine which of a large number of similar source data elements best adheres to the given a priori knowledge. An important use of this feature is improvement of hand-written and speech recognition. The potentially very high complexity of encoding and decoding algorithms allow the encoded data to be made inaccessible to outsiders (encryption) with very little extra effort. Furthermore, an inherent property of the system according to the invention is the ability to incorporate data conversions into the encoding-decoding-path at none or very little extra computational effort. This is of use in, for example, reaching compatibility between data formats (such as among WordPerfect and other text editing systems and/or various formatted text formats) , where one version of the encoded data suffices to render multiple instances of the decoded data. Due to the potentially very complicated structure of the data it is not unreasonable to also include the following goals on implementational and conceptual complexity. Programs enacting the previously described methods need be very compact. Otherwise, the goal of high information density would be overshadowed by considerable overhead. In addition, a priori knowledge should be made known to the system in a relatively simple manner, such as by means of a dedicated specification language and automated learning from sample data. Since high information density presents a major advantage to data transmission, it would be very useful if all functions related to a specific representation can be transferred among, and efficiently employed by different types of computers. In practice, data is often created by few people, or at low speed, while being referenced by many, at high speed. Examples are books, computer programs, and news messages. In other cases, such as archiving, things are just the other way around. Therefore it is reasonable to demand data representation methods to be able to exchange cost of encoding and decoding in order to minimize overall expenses. There are two major aspects to be considered: 1) the ability to exchange the amount of memory needed during encoding and decoding, 2) and the ability to exchange computational complexity between the encoding and decoding algorithms. To accomplish efficient representation, the invention offers a method to represent data, based on logical and probabilistic dependencies among its parts. The invention also offers a device, containing means to represent data as logical and probabilistic dependencies among its parts. In addition, the invention offers a system to represent data as logical and probabilistic dependencies among its parts. With parts of the data is meant: units of information residing within the data (bits, bytes, etc.) , interpreted as symbols or numerical values, as well as compound structures built on these units of information, such as sequences, morphological and grammatical constructions, formatting, etc. Embodiments of methods according to the invention are characterized in that the method comprises a step to represent a priori knowledge, which knowledge is used by a following step for representation - i.e. encoding and decoding - of the data. Embodiments of devices according to the invention are characterized in that they comprise an a priori model. Other embodiments of devices according to the invention are characterized by a machine-independent priori model. Embodiments of devices according to the invention are characterized in that the device comprises an input to receive incoming data (encoded data) and an output to emit encoded data (outgoing data) . Advantages of the method, system, and embodiments according to the invention are: The a priori model forms a unification of functionality. No separate programs need be made by hand, resulting in memory savings while rendering a machine-independent description. In addition, the flexibility of an a priori model allows for automatic learning. Automatic learning by transferring information from the dynamic m αel to the a priori model. Automatic learning by conveying chance distributions (or parts thereof) used by symbol coders, from the dynamic model to the a priori model. Automatic learning by transferring patterns detected by self-resemblance processes from the dynamic model to the a priori model. Automatic learning by taking into account most probable solutions for the evaluation order during back-tracking. Automatic learning by changing the structure of the a priori model, based on analysis of " the dynamic model, for example: 1) Merging parts of the a priori model (with or without parameterization) which turn out to be used in a similar manner, such as for similar patterns or symbol encoders which happen to be based on similar chance distributions. 2) Splitting parts of the model into multiple specific instances, such as with symbol encoders whose chance distributions depend on events elsewhere in the encoding or decoding process. 3) Removing unused, or infrequently used parts of the a priori model. A very limited number of primitive transformations as building blocks for pattern recognition, symbol encoding, and abstraction mechanisms, yielding a much smaller, more easily comprehensible, and more clearly organized a priori model. Reduction of the a priori model by automatic detection of possible use of abstraction mechanisms. A non-temporal way of specifying interactions among primitive processes, leaving open a degree of freedom which can be exploited in several optimizations. The ability to exchange memory requirements and computational complexity between the encoder and decoder. The ability to exchange between the encoder and decoder the necessity for back-tracking. The ability to exchange between the encoder and decoder memory requirements for storage of recurring patterns. The ability to change the temporal order of execution of the decoder in order to decrease memory requirements of the encoder (or decoder) for multiplexing (or demultiplexing) multiple data streams. The ability to merge multiple data streams without overhead, while decreasing computational complexity of the decoder. Selection of symbol encoding methods based on actual chance distributions. Random access by efficient encoding of small parts of the data. Rarrdcrπraccess by siroriπg- g-lobal characteristics -separately to attain very high information density when separately encoding even smaller parts of the data. Random access by storing multiple sets of local characteristics, possibly hierarchically interrelated. Associative access by storing potential search keys without significant overhead. Extremely compact representation of a priori models by encoding the a priori model itself, using to an a priori model of the a priori model, according to the invention. Swift associative access by splitting off information on the occurrence of potential search keys, without significant overhead. Still faster associative access by storing information on potential search keys in a hierarchical manner. Interpolation by maximizing information density. Interpolation by maximizing information density, based on incremental evaluation of dependencies. Interpolation by maximizing information density, based on heuristics residing in chance distributions. Classification based on acceptation by the encoder. Classification based on attained information density. Conversion based on differences between a priori models used for encoding and decoding. Encryption by using part of the a priori model as the decryption key. Encryption by using part of the encoded data as the decryption key. The description of the invention will be illustrated with the following figures, which serve as examples. The figures show: Figure 1 a representation of source data and encoded data, Figure 2 the a priori model, Figure 3 the a priori model as a graph, Figure 4 primitive processes arranged in a table, Figure 5 the a priori model and the dynamic model, (schematic) , Figure 6 translation of a priori knowledge to an a priori model, (schematic), Figure 7 a stream as a two-dimensional array, Figure 8 a stream between a sender and a receiver, Figure 9 a process accepting only constants, Figure 10 a process for copying frames, Figure 11 a process transferring frames to one of its outputs, Figure 12 a tabular view of accepted frames, Figure 13 a process for splitting frames, Figure 14 a tabular view of splitting frames, Figure 15 a process to switch between two outputs, Figure 16 a tabular view of switching between outputs, Figure 17 a process for merging frames, Figure 18 a tabular view of merging frames, Figure 19 a process repeating frames, Figure 20 a tabular view of repeating frames, Figure 21 a process interpreting symbols as numerical values, Figure 22 a process detecting recurring patterns, Figure 23 a tabular view of detection of recurring patterns, Figure 24 a process to encode incoming symbols into outgoing streams, Figure 25 a process to multiplex incoming streams, Figure 26 an abstraction mechanism for reuse and parameterization of transformations, Figure 27 an abstraction mechanism for translation of patterns, Figure 28 an abstraction mechanism for automatic learning of high-level patterns, Figure 29 a process to remove information, Figure 30 an outline of an embodiment according to the invention, Figure 31 an outline of an embodiment according to currently known methods. Figure 2 shows a so-called a priori model, which describes a specific data representation method. It, in fact, holds all employed a priori knowledge on the data. All functions related to the data type can be efficiently derived from this a priori model. Examples- of such functions are: Encoding, decoding, random access addressing, associative access (searching) , classification, interpolation, encryption, decryption, and conversion. Although these functions can be derived from the same a priori model, each function interprets it in a different way. This common source of information allows for a significant memory reduction. An a priori model describes a transformation, constructed from a limited set of primitive transformations (building blocks) and a specification of the way they are interconnected. The model does not describe the exact behavior of its primitive transformations, but rather abstract references. Figure 3 shows an example of a graph in which primitive transformation are indicated by numbers. Figure 4 shows, in a tabular format, how actual primitive processes can be derived from the primitive transformation type and the desired function by means of table lookup. To avoid misunderstandings, there are three remarks to be made here. 1) Primitive processes run in parallel, although in some implementations, interconnected subsets might execute sequentially. 2) The arrows in figure 3 do not denote the direction of flow of information. 3) Furthermore, primitive processes are themselves functional descriptions, of which several machine-dependent implementations may exists, which can all relate to the same a priori model. The proposed modeling and parameterization is only one of the means offered to handle large complexity, and it is not a prerequisite. The compound network of primitives could be compiled into a single monolithic program. This may yield a speed increase (sacrificing flexibility and compactness) . Furthermore, many kinds of hybrid implementations are feasible, with fewer supported functions, dynamic compilation of (parts of) the model, etc. Also, the a priori model may be represented in other ways, possibly using parameterization to compactly describe it. Primitives are contained in separate processes, each holding its own current state. The compound set of all these states is called the dynamic model, for it forms a structure similar to the a priori model. This dissection can be seen from two different viewpoints: Figure 5 presents a schematic view of the a priori and dynamic model. The a priori model can be regarded as a program which (in a unusual fashion) tells which actions to perform. The dynamic model constitutes the memory in which each process registers its internal variables. On the other hand, the a priori model can also be regarded as the knowledge available at program start (hence its name) , while the dynamic model starts out empty, gathering specific information during data processing. This makes clear why such a system can learn by transferring information from the dynamic model to the a priori model. For instance, the data in the dynamic model may suggest that in practice an adjective is often followed by a noun. Such information can then be stored in the a priori model, so the next round of data processing may yield, for example, higher information density or higher execution speed. Other learning methods are based on analysis of patterns and dependencies among information from different primitives in the dynamic model, which can be used to change the a priori model. Figure 6 schematically shows the transition of initial a priori knowledge. An initial a priori model can be created from a high-level specification. A compiler can be used to translate this specification into a graph of primitive transformations, in such a way that derived functions adhere to the goals of efficient representation. An example will now be given of a small set of primitive transformations, with which a very large expressivity can be attained. This set can applied to analyze even natural language with high fidelity. But, using the same primitives, also machine language, data bases, spreadsheets, vector graphics, and many other data types can be very efficiently represented. Between two primitive processes (or between one process and the "outside world") , information can be exchanged via a common variable. Such variables can be structured, and can adhere to constraints governing accessibility. In the rest of this document communication variables of a. type called "stream" will be used. Streams contain an ordered sequence of "frames", while frames consist of an ordered sequences of symbols. Symbols can be addressed by their index within a given frame. As depicted in figure 7, a stream can be regarded as a 2-dimensional array of symbols, with each symbol related to a frame number and an index. Figure 8 shows a stream with associated processes, of which one is a "sender" and one a "receiver" of symbols. For the sender, this stream is an "outgoing" stream, for the receiver an "incoming" stream. The sender's task is to "emit" symbols, while the receiver's task is to "accept" them. At this point, transmission of symbols is not bound to a temporal or sequential order. Nor need the order of emission and acceptation of symbols be the same. Senders can "withdraw" previously emitted symbols. A receiver can ask the respective sender to emit a particular symbol, which it can either accept or not. A receiver may also choose to accept a symbol, but "reject" it at a later time. Interconnected senders and receivers are aware of each others emissions, acceptations, withdrawals, and rejections. This communication mechanism also applies to entire frames, and also frames holding zero symbols can be accepted. In effect, sending and receiving processes can make use of an extensive back-tracking mechanism in order to agree on the contents of shared stream variables in such a way that their internal constraints are satisfied. In addition to the binary choice of acceptance of symbols and frames, a receiver can also make know to what degree it is acceptable, or at what cost it can be accepted, allowing the sender to make a well-founded choice between acceptable alternatives. For instance, in order to optimize information density. In practice, the implementation of streams can sometimes be simplified if some information is known a priori with both the sender and receiver. For example, an external file can be regarded as stream with a single frame, holding symbols (bytes, bits, etc.) which will be neither be rejected nor withdrawn. Primitive transformations in fact consist of a set of relations-h-lps among, incoming- and outgoing streams. Primitives are- implemented in a way related to the actual direction of symbol transfer over streams, depending on the desired function. In general, for decompression this direction will be the inverse of compression. To keep things clear, the following primitives will be described for only one direction. The inverse operation can be derived by reversing all directions of symbol transfer. Figure 9 depicts a process (CONST) which accepts from its incoming stream only frames with a given content (I) . (The inverse of the CONST process emits such frames.) Figure 10 shows a process (COPY) copying frames from its incoming (I) to its outgoing (0) stream. For each relayed frame, one constant frame is emitted into stream K. Figure 11 shows a process (OR) copying each incoming frame (I) to one (1) of its output streams (A,B,..) . The choice regarding the selected output stream, is expressed via stream K. In the case where more than one output stream can accept the frame, priorities can be used to solve the ambiguity. For example, by. preferring certain output streams to others, and emitting the identity of the chosen output stream as enumerated symbols into stream K. An example in figure 12 also shows which frames have been accepted. For instance, the word "light" could be classified by an OR process as either a noun or an adjective, depending on acceptation by its output streams and acceptation of the choice entered into the stream K. Figure 13 shows a process (SEQ) which splits frames, taken from the incoming stream (I) , into contiguous parts. The order and number of symbols emitted into each output streams adheres to given priorities. The choices being made are represented in outgoing stream K. For instance, the number of symbols sent to stream A can be maximal, the number of symbols sent to B minimal, as in the example in figure 14. For example, a SEQ process could be used to split a phrase like "a big house" into five parts: article, space, adjective, space, noun. Figure 15 shows a so-called SWITCH process, sending frames from its input stream (I) to one of its output streams (A,B,..), depending on choices input from stream K. An example is given in figure 16. Figure 17 displays a process (MERGE) which merges several incoming streams (A,B,..) into a single outgoing stream (0) . The order of emitted frames is related to those of incoming frames, according to given priorities. Choices being made in this process are encoded into outgoing stream K. Incoming streams can, for instance, be interwoven in a strict order, in which case stream K happens to be redundant (see the example in figure 18) . Figure 19 shows a process (REPEAT) which splits up incoming frames (from stream I) into parts, placing these parts into different output frames. Choices being made are represented in outgoing stream K. For instance, a REPEAT process could split frames in such a way that the number of symbols per outgoing frame is maximized, with a given priority for ambiguous cases, with stream K registering the number of outgoing frames per input frame (as in figure 20) . For example, words can be split into a, preferably minimal, number of components to be accepted as syllables. Figure 21 shows a process (NUM) which interprets symbols (I) as numerical values, performing a reversible numerical operation on them, the results being encoded as symbols into stream O. For instance, two incoming streams (X and Y) , suppl\ symbols 'a' through 'z', being interpreted as the numbers C thr." -i 25. The output might reflect the value of Z=X+26*Y, which a ct. --ecutive process may choose to interpret as 676 distinct possible symbols. Figure 22 depicts a so-called self-resemblance (ΞR) process, detecting repetitive combinations of symbols and frames in its input streams (II, 12,..) . Repetitions are marked as references into stream K. All other symbols and frames are copie to their respective output (01, 02,..) . For example, suppose there is only one input stream, and no a priori known frames. In figure 23, symbol R[0] indicates th repetition of a frame indicated by symbol S[0]. Figure 24 shows a so-called symbol encoder (SE) process (not to be confused with the encoding function) , which converts an incoming stream (I) into one or more outgoing streams (01, 02, ..), using an encoding scheme based on the actual and/or a prior chance distributions of symbols. No further dependencies among incoming symbols are assumed. Examples of symbol coding methods which generate binary symbols according to given chance distributions, are Huffman coding and Arithmetic coding. In general, the closer these distributions adhere to the actual distribution of symbols, the more compact the resulting code will be. Figure 25 shows a so-called multiplex (MX) process, joining several incoming streams (II, 12,..) into a single frame in its output stream (O) . In contrast to the MERGE process, identification of which frame originates from which input stream is recorded within the output frame itself, by means of added (combinations of) symbols. In order to show how powerful and expressive models based on these primitives can be, several abstraction mechanisms will be presented which can be realized with these building blocks. Using OR and SEQ primitives, formal grammars can be analyzed and encoded. Primitives such as MERGE and REPEAT are able to gather multiple frames into a single stream, in effect employing an consecutive network of transformations multiple times. Figure 26 outlines an abstraction mechanism by which, via COPY processes, data is fed to a MERGE process, followed by a common transformation. However, using a SWITCH and an other MERGE process, specific transformations (A,B) can still be applied to each of the original input streams (II, 12) , in effect a parameterization. This mechanism allows for large, resembling parts of the model to be included only once. Figure 27 shows an example of an abstraction mechanism, in which patterns are recognized and reconstructed as new patterns. For instance, a simple Dutch sentence like "neem de telefoon op" can, via grammatical recognition and reordering, be converted to its English equivalent, "pick up the phone". Figure 28 outlines an abstraction mechanism in which an SR process is employed to recognize repetitive patterns in choices made by OR processes. This, in effect, constitutes the capability to automatically learn new patterns in the source data at a higher level. For example, the fact that a possessive pronoun is often followed by a noun could be automatically detected and exploited in this manner. The way in which information can be fed to symbol encoders (SE) can be used to exploit the existence of complex stochastic relationships within the data. For instance, a MERGE process can be used to merge choices made by several processes into a single symbol encoder. For instance, to express that chance distributions for lower and upper case letters are equal. Alternatively, using a SWITCH process, choices from a single process can be fed to different symbol encoders, depending on external circumstances. For example, a letter starting with the phrase "Dear Sir" is likely to contain a different style from one opening with "hi John", and will probably yield different chance distributions. Using numerical transformations, chance distributions can be reshaped. Stock market exchange figures, for instance, can be represented by incremental differences, which could be encoded assuming a Gauss distribution. The rest of this description discusses how the previously mentioned functions can be derived from an a priori model, and how these functions can be optimized. Implementations of primitives depends on the particular function required, while for each function some simple processes may need to be added to communicate with the outside world. As the example in figure 30 shows, information can be handed to the instantiated model directly, or it can be pre- or post-processed (such as selection of a word from a line of text) . Also extra parameters can be offered (such as an address for random access) , or extracted (for instance, whether a word is spelled correctly) . This example shows the enormous benefits of unification of functionality. Traditionally, much more work need be done (outlined in figure 31) , while the knowledge of the spelling checker is not available for encoding and decoding. Data can be encoded by transformation according to the model, in a given direction. A final step within this transformation can be to combine all streams containing relevant information into a single frame, using a multiplex process (MX) , rendering a linear sequence of symbols which can be stored or transported. The decoding function can be obtained by enacting individual transformations in the opposite direction (relative to encoding) , using primitives optimized for this kind of information processing. The description of encoding and decoding functions is based on the concept of streams. The order in which symbols and frames are emitted and accepted is not fixed. This deliberately introduced degree of freedom can be exploited to speed up the decoding function considerably, and to further reduce the size of the encoded data. This can be accomplished by choosing an actual evaluation order in a way yielding minimal computational overhead (back-tracking) for the decoder. Moreover, extra symbols added by the multiplex process (MX) for identification of its incoming streams can be left out completely, putting symbols and frames in the exact order in which the actual implementation of the decoder happens to accept them. This is feasible if, for given encoded data, behavior of the decoder is deterministic. Freedom in the order of evaluation can also be employed to reduce the need for temporary memory, such as memory required for multiplexing and demultiplexing data streams. For self-resemblance (SR) transformations, the encoder can explicitly indicate whether patterns (combinations of symbols) recur. Other patterns need not be stored for future reference by the decoder. Chance distributions, used by symbol encoders (SE) , can be encoded explicitly, allowing for ad-hoc selection of symbol encoding methods which need only be (near) optimal for the distribution at hand. This allows for a much broader range of feasible encoding methods, of which - for instance - the fastest ones can be selected. The consequence of such measures is in fact an exchange of computational complexity and memory requirements between the encoding and decoding function. The ability to include large amounts of a priori knowledge in the model allows for high information density, even for relative short files. Consequently, large files may be split up into smaller segments without incurring major losses, each segment being separately accessible. Each page of a book, for instance, could be encoded separately, so even accessing the last page of an encyclopedia could be very fast. As indicated before, typical characteristics of some data can be learnt by transferring information from the dynamic model to the a priori model. For instance, characteristics all segments of a file have in common. An a priori model modified in this way results in high information density for even shorter segments. For example, the modified a priori model could contain specific chance distributions for symbol encoders for a given book. Now, even individual sentences could be encoded with high information density, so decoding or recoding small pieces of the data incurs only very little computational overhead. Also multiple versions of data-specific knowledge can be contained in the a priori model, possibly hierarchically structured. This is very useful if, for example, a document contains multiple alternating text styles. Efficient associative access can be obtained by the ability to quickly determine whether a given search key (of a certain type) occurs in parts of the data. These parts can be encoded files, or segments of encoded files with random-access. The way to efficiently test for the occurrence of search keys is to have separate access to the encoded data pertaining to it. For example, to allow an encoded text to be searched for nouns, information pertaining to nouns should be stored separately. • No other encoded data need be inspected during a ■ - search. Information on potential search keys can also be split up. For example: A) the fact whether each of the potential search keys occurs, B) the number of times keys occur, C) and the permutation in which those keys appear in the data. In this manner, the information marked as A) suffices to answer most queries, so an even smaller part of the data need be decoded . For instance, when searching for a given proper name, onl the list of employed proper names is decoded. If the item searched for does not appear, the current segment or file can b quickly skipped. Information on the occurrence of search keys can also be organized in a hierarchical manner, in which each level registe the employed search keys as a subset of those on the next highe level. For example, when looking for a material noun within an encyclopedia, one could first find out in which volumes (s) it occurs. Subsequently, relevant topics and, thereafter, paragrap could be identified. Interpolation is regarded as the process of finding the most likely contents of part of the data, given the rest of the data and the knowledge residing in an a priori model. This problem can be solved by minimizing the size of the encoded data over all possible contents of the unknown part. Fo the a priori model is generally constructed in a way rendering maximum information density for the most likely data. Computational complexity can be dramatically reduced by keeping track of incremental relationships with the unknown (varying) part. For instance, in the sentence "This is a ra??it" (the question marks denoting unknown characters) , not all possible sentences with combinations of the two characters need be encoded. For only the last word causes variations in the encode data size. In addition,, unlikely solutions can be excluded, based o information in the a priori model and chance distributions gathered from the known part of the data. For instance, for the sentence "This is a ra??it" it can be inferred that the last word must be a noun, probably "rabbit unless some other derivation appears in the surrounding text. The classification function can be based on the acceptation or rejection of data by (part of) an encoding syste Multiple parts of the model can each determine whether the data adheres to given constraints, possibly classifying the data int multiple groups. For example, various (possibly overlapping) parts of the model could detect whether a given word is a verb, an adverb, an article, etc. Together they can determine whether the word is spelled correctly or not. An other classification method is based on classification of the information density attained. Also in this case, the amount of information generated by parts of the model can be regarded separately. For instance, with a model holding Dutch grammar rules, a much higher information density can be obtained for Dutch texts than for English texts. In the same way, style of writing can be classified, with a peak information density for some particular author. Conversion of data within the encode-decode path can be accomplished by using decoding primitives which are not exactly the inverse of the encoding primitives (or vice versa) . In addition, during encoding and decoding information can be removed using a so-called Dump process (shown in figure 29) , accepting arbitrary input frames, without emitting any information. During encoding, for instance, undesired words or phrases can be automatically removed (censorship) . During decoding, the length of lines of text could be adjusted to a given column width. Besides seamless integration with encoding and decoding functions, many conversions performed in this way take up non or very little extra time. Data is usually analyzed and dissected anyway; work that need not be -done twice. When a small part of the-*,a priori model is missing, correct decoding can become extremely difficult. This missing part can be regarded as the decryption key for the decoding algorithm. Alternatively, taking out a small part of the encoded data can make correct decoding unrewardingly difficult. Both techniques rely on the very large potential complexity of functions derived from the model. Decoding slightly incomplete or incorrect encoded data renders data which adheres to the a priori knowledge, but not the original contents. For instance, decoding slightly mutilated encoded text generally yields a text with correct grammar, but without any comprehensible meaning, making a mechanical search for the correct decryption key very difficult. Encryption of encoded data hardly requires any extra work. Only a very small part of the model or data need be transferred via a safe channel, the rest of the encoded data requires no further processing. Finally, a number of methods will be mentioned pertaining to all listed functions, with regard to automatic learning and representation of the. a priori model. For symbol encoder (SE) primitives, (parts of) chance distributions gathered in the dynamic model can be transferred to the a priori model. The same goes for patterns observed by self-resemblance (SR) processes. In addition, back-tracking schemes can be biased towards trying most probable solutions first. The ability to change the structure of the a priori model gives rise to several other techniques. For instance, parts of the dynamic model which happen to be similar, or mutually dependent, can be shared in the a priori model (with or without parameterization) . An example is sharing of symbol encoders (SE) with similar or dependent chance distributions. For instance, symbol encoders (SE) could be instantiated multiple times if the resulting distributions turn out to correlate with events elsewhere in the model. Unused, or infrequently used parts of. the . a priori model can be automatically deleted. The learning methods can be applied to different kinds of data, yielding multiple specialized a priori models, biased, for example, towards a given style of writing or topic. These specialized models can be very efficiently represented by means of their mutual differences, or differences with the original model. An other mechanism to reduce the size of an a-priori model is to automatically detect the possible use of abstraction mechanisms and parameterization, whereby resembling parts can be unified. An a priori model itself can also be the subject of encoding, for instance based on an a priori model of the a priori model. Employing such methods makes it possible to represent an exceptionally high amount of a priori knowledge within a small memory, or to transmit such information to others at very low cost. Finally, from an other point view an a priori model can be regarded as a data transmission standard, with the model indicating how data should be interpreted. Such a standard is flexible, and can be distributed along the same channels as the data itself, so improvements and extensions can be employed very quickly, possibly without bothering end-users. What need be agreed upon is a meta-standard: the way an a priori model is represented and the exact functionality of its primitives.

Claims: CLAIMS 1. Method to represent data, based on logical and probabilistic dependencies among its parts. 2. Method according to claim 1, characterized in that the method comprises a step to represent a priori knowledge, which knowledge is used by a following step for representation - i.e. encoding and decoding - of the data. 3. Device to represent data, containing means to represent data as logical and probabilistic dependencies among its parts. 4. Device according to claim 3, characterized in that the means comprise an a priori model. 5. Device according to claim 4, characterized by a machine-independent a priori model. 6. Device according to claim 4 or 5, characterized in that the a priori model is built with sample data. 7. Device according to claim 3, 4, 5 or 6, characterized in that the device comprises a dynamic model. 8. Device according to claim 3, 4, 5, 6 or 7, characterized in that the device comprises an input to receive incoming data (encoded data) and an output to emit encoded data (outgoing data) , an encoding (decoding) circuit to encode (decode) the data. 9. System to represent data, based on logical and probabilistic dependencies among its parts. 10. System according to claim 9, characterized in that the system comprises an a priori model and a dynamic model.

Inventors: KOEKOEK ROBERT MARTIN, KOEKOEK, ROBERT, MARTIN
Applicants: KOEKOEK ROBERT MARTIN, KOEKOEK, ROBERT, MARTIN
Application year: 1994
Main classification: H03M7
IPC codes: H03M7