Patent ID: 7031910

Claim:
Method of encoding linguistic frequency data, the method comprising: mapping each character string occurring in a source text to a numeric identifier identifying the character string, identifying a plurality of sets of n character strings in a source text, n being an integer number greater than 1, each set forming an n-gram comprising at least a first and a second successive character string, for each set, obtaining frequency data indicative of the frequency of the respective set in the source text, creating a memory array f for containing the frequency data, n−1 pointer memory arrays p 2 . . . p n for containing pointers, and n offset positional arrays r 1 –r n for containing indexing offsets, for each character string that is a first character string in at least one of the sets, assigning a memory position in memory array f to the respective character string and storing at said memory position the frequency data of each set comprising the respective character string as the first character string, grouping the frequencies relating to n-grams that have the same first character string into a block within the array f, for each character string that is a second character string in at least one of the sets, assigning a memory position in pointer memory array p 2 to the respective character string and storing at said memory position, for each set comprising the respective character string as the second character string, a pointer pointing to a memory position in the memory array f assigned to the corresponding first character string of the respective set and having stored the frequency data of the respective set, grouping the pointers relating to n-grams that have the same second character string together as a block within the pointer memory array p 2 , storing an offset position for each respective first character string in positional array r 1 that indexes to the corresponding block in frequency array f relating to the first character string, and storing an offset position for each respective i th character string in positional array r i that indexes to the corresponding block in pointer array p i relating to the i th character string.