Input device with deferred translation

Improved data entry for handwriting entry computer systems, including "delayed translation" of written strokes, is provided. After a user has entered data into a handwriting entry computer system, the writing is associated with a target field or object. In one embodiment of the invention, a translation of the entered data is performed, and the untranslated "ink" is displayed. At a later time, if desired, the user can elect to display the translated text. If there are errors or mistranslations, the user can make corrections or revert back to the ink. In another embodiment, a user enters data into the computer system. A translation of the data is performed by a background process. If any of the characters cannot be translated, the original ink is displayed. If all of the characters are translated, the resultant word is compared to a dictionary of words, numbers and symbols. If the resultant translated word does not match a dictionary entry, the original ink is displayed. If there is a match, the translated text is displayed. In another embodiment, the user can selectably choose between translated text and the original ink on a field by field basis.

BACKGROUND OF THE INVENTION 
1. Field of the Invention 
The present invention relates to the field of data entry in computer 
systems. 
2. Background Art 
A typical computer system consists of a central processing unit (CPU), main 
memory such as random access memory (RAM), a data entry device, including 
a positioning device, a mass storage device such as one or more disk 
drives, a display and/or a printer. In the prior art, the data entry 
device often consists of a keyboard, on which a user enters data by 
typing. The positioning device of a prior art computer system may consist 
of a "mouse" or other cursor positioning device. 
Computer systems have also been developed that are directed to handwritten 
data entry rather than keyboard data entry. These systems are often 
characterized by the use of a pen, stylus, or other writing device, to 
enter handwritten data directly on the display of the computer system. 
Alternatively, these systems may provide for a user to enter data on a 
digitizing tablet or other input device, with the image of the written 
input displayed on a separate computer display output device. The writing 
device for entering handwritten or freestyle stroke input information is 
not limited to a pen or stylus, but may be any input device such as a 
mouse, trackball, pointer, or even a person's fingers. Such systems are 
not necessarily limited to receiving data generated by human users; for 
example, machine generated data may also be inputted and accepted to such 
systems. 
One class of this handwriting entry computer system that receives 
handwritten data input is referred to as a "pen based" computer system. In 
a pen based computer system, a writer can input information on a display 
by "writing" directly on the display. A writing device, such as a pen or 
stylus, is used to enter information on the display. In a typical 
pen-based computer system, a user touches the stylus to the display and 
writes as he would on a piece of paper, by making a series of pen strokes 
to form letters and words. A line appears on the display that follows the 
path of travel of the pen point, so that the pen strokes appear on the 
display as ink would appear on a handwritten page. Thus, the user can 
enter information into the computer by writing on the display. These 
strokes are referred to herein as "ink". A related series of strokes or 
characters are referred to as a "block of ink". 
In addition to handwriting, special functions based on input strokes are 
supported on pen based computer systems. For example, a writer may strike 
certain previously entered handwritten input by making the strokes of an 
"X" over the entry. These special functional strokes or actions are 
referred to herein as "gestures". Pen based computers typically have a 
display surface that serves as both an input receiving device and as an 
output display device. 
The display of a pen-based entry or other handwriting entry computer system 
may have bordered regions called "fields" that define locations or areas 
where data is to be entered, or that are associated with particular 
actions. For example, if the software application provides a data entry 
display that is an information form with fields for "First Name", "Last 
Name", "Age", "Occupation", etc., certain information is generally 
associated with each field. These locations are referred to here as 
"objects" or "fields". Each object has a boundary associated with it. When 
data is entered, the computer must determine which object is to be 
associated with the writing or gesture. The object intended to be 
associated with the data input is called the target object. The process of 
identifying the target object is referred to here as targeting. The method 
and apparatus that performs the function of targeting is referred to here 
as a targeting system. 
One characteristic of handwriting entry computer systems is the ability to 
translate original handwritten strokes of ink or blocks of ink into 
machine readable words or characters for display. This translation is 
accomplished via a "character recognition" algorithm. The strokes of ink 
that form handwritten words are translated into, for example, ASCII 
characters. After the translation, the appearance of the displayed 
characters is as if they had been typed in via a keyboard. 
To translate a handwritten character into a machine readable character, the 
handwritten character is compared to a library of characters to determine 
if there is a match. A description, or "template" for each character is 
defined and stored in memory. Handwritten characters are compared to the 
stored templates. Match coefficients, reflecting how closely a handwritten 
character matches the template of a stored character, are calculated for 
each template character. The template character with the highest match 
coefficient is identified. The character represented by this template 
provides the "best fit" for the handwritten character. If the match 
coefficient for the "best fit" character exceeds a predetermined minimum 
threshold., the "best fit" character is adopted. If the match coefficient 
for the "best fit" character is less than the minimum threshold value, no 
translation is done. If the handwritten character cannot be translated, 
the character must be re-entered. 
A disadvantage of current character recognition algorithms is limited 
accuracy. Often, handwritten characters are not translated at all or are 
mistranslated as an ASCII character other than the handwritten character. 
The mistranslated character must then be rewritten by the user, sometimes 
repeatedly, until a correct translation is made. 
A number of prior art handwriting recognition and translation schemes have 
been described in United States patents. These prior art schemes are 
described below. 
Fukunaga, U.S. Pat. No. 4,641,354, is directed to an apparatus for 
recognizing and displaying handwritten characters and figures. The 
apparatus includes a stylus, a liquid crystal display, and handwritten 
character and figure recognition means, based upon input stroke 
information. The Fukunaga system displays the result of recognition as it 
is completed, erasing from the display screen the stroke information used 
for recognition, and retaining on the display the stroke information which 
has not yet been used for recognition. Fukunaga discusses the use of a 
character recognition algorithm that converts handwritten text as soon as 
the algorithm determines that the stroke information for a character is 
completed. To conserve memory, all handwritten information is erased from 
the display as soon as it is recognized. 
Conoval, U.S. Pat. No. 4,703,511, is directed to a handwriting input 
apparatus and method of determining handwriting dynamics information. The 
Conoval system uses a portable unit and a code generating stylus that 
modulates handwriting markings, thereby embedding time code information 
within the writing path. The coded markings are delivered to the writing 
surface via electromechanical, electrothermal, or electrostatic printhead 
devices. After the writing is produced, an image detection means may 
convert the written image to an electrical signal representation thereof 
for further dynamic analysis. The writing device of Conoval emits a 
modulated signal which is memorialized on the writing surface as the user 
hand writes text. Temporal information is stored in the strokes comprising 
the handwritten characters. The handwritten characters are displayed as 
they are written. However, the characters are not line-segmented. Conoval 
suggests that the writing surface, on which the characters are written, 
can later be provided to a character recognition algorithm for conversion 
to typewritten fonts. 
The system described in More, U.S. Pat. No. 4,839,634, is a portable 
electro-optic data input/output, storage, processing, and display device 
responsive to hand printed text and hand drawn graphics. The More system 
incorporates a character recognition method optimized for automatic and 
computationally efficient on-line recognition of hand printed text 
characters. Hand printed text characters are stored in a compact and 
standard format, like ASCII, for later display, processing or output to 
external equipment. In one preferred embodiment, the user places hand 
printed characters into a text conversion area, where they are recognized, 
converted to typewritten fonts on a line-by-line basis, and displayed in a 
document viewing/processing area. With the device of More, a user hand 
writes a line of characters in a text conversion area. The characters are 
displayed in this area as they are written. When the user activates a 
"CONVERT" button on the device, the entire line of handwritten text is 
operated on by a character recognition algorithm. Any handwritten 
characters in the text conversion area are erased. The recognition output 
is displayed as typewritten fonts in a larger, separate document 
viewing/word processing area. All editing functions are performed on the 
typewritten fonts in a display area which is separate from the area where 
data is inputted. Even though character recognition does not take place 
until the user initiates the recognition algorithm, the user can enter 
only a single line of unconverted text at a time. More does not allow the 
user to work with and perform functions on the untranslated handwritten 
characters. Further, the user is prompted to correct any translating 
errors as each line is converted. More requires untranslated, handwritten 
characters to be resident in a specific and limited region. 
A portable keyboardless entry computer system is described in Sklarew, U.S. 
Pat. No. 4,972,496. The invention in Sklarew temporarily displays the hand 
printed characters and then translates them into pre-selected fonts by way 
of a character recognition algorithm. The user is prompted to make any 
corrections to handwritten data as it is being inputted. The system in 
Sklarew also includes a word processing editor. The portable unit 
described in Sklarew also suffers from some of the disadvantages of 
Fukunaga. The device of Sklarew concurrently displays handwritten text and 
the result of its recognition algorithm on the screen. The user is 
prompted to correct any errors before the translated characters are 
permanently transferred to the screen. 
Shojima U.S. Pat. No. 4,718,103, and Jeanty, U.S. Pat. No. 5,001,765, are 
both specifically related to character recognition algorithms. 
A disadvantage of the prior art translation systems is that the character 
recognition and translation functions are invoked in real time, 
immediately after entry of a handwritten word. When the characters are not 
translated correctly, the user is prompted to correct the translation or 
reenter the handwritten data. This interrupts the flow of data entry, 
increasing data entry time, frustrating the user, and decreasing the 
usability of handwriting entry computer systems. 
SUMMARY OF THE INVENTION 
The present invention provides for improved data entry for handwriting 
entry computer systems by disabling the automatic display of a translation 
of the entered data. Instead, the original ink entered as data is 
displayed. This is referred to here as "deferred translation". The 
invention allows a user to select modal deferred translation, automatic 
deferred translation, and/or manual deferred translation. 
Modal deferred translation provides for the automatic display of original 
ink input in all cases. A translated version of the input data is 
generated and retained. The user may elect to display either the original 
ink or the translated text at a later time. In operation, a user enters 
data into a handwriting entry computer system, and the writing is 
associated with a target field or object. A translation of the entered 
data is performed and the untranslated "ink" is displayed. At a later 
time, if desired, the user can elect to display the translated text. If 
there are errors or mistranslations, the user can make corrections or 
revert back to the ink. 
Automatic deferred translation provides for the display of the original ink 
whenever there are errors in the translated text. If there are no errors 
in the translated text, it is displayed instead of the original ink. In 
operation, a user enters data into the computer system, and a translation 
of the data is performed. If any of the characters cannot be translated, 
the original ink is displayed. In an optional step, if all of the 
characters are translated, the resultant word is compared to a dictionary 
of words, numbers and symbols. If the resultant translated word does not 
match a dictionary entry, the original ink is displayed. If there is a 
match, the translated text is displayed. 
In manual deferred translation, a user enters data, and a translated 
version of the data is displayed. The user may elect to correct the 
translated text, if necessary, display the original ink instead, or do 
nothing. This decision can be made on a field by field basis. 
The present invention retains both the original ink and translated text so 
that a user can elect to display either version as desired.

DETAILED DESCRIPTION OF THE INVENTION 
A method and apparatus for providing deferred translation of data entered 
into a computer system is described. In the following description, 
numerous specific details, such as processor type, processor speed, 
operating system type, etc. are set forth in detail to provide a more 
thorough description of the present invention. It will be apparent, 
however, to one skilled in the art, that the present invention may be 
practiced without these specific details. In other instances, well known 
features have not been described in detail so as not to unnecessarily 
obscure the present invention. 
The preferred embodiment of the invention is incorporated in computer 
software for use, for example, with the PenPoint.TM. operating system for 
pen-based computers, such as those manufactured by GO.TM. Corporation of 
Foster City, Calif. However, the invention can be used with a wide variety 
of other operating systems and other computers and computer systems. 
FIG. 1 illustrates an example of a handwriting entry computer of a type 
that may be used with the present invention. It consists of a pen 
sensitive display 102 disposed in a housing 101. The front of-the housing 
features an on/off button 110. Input/output ports for a disk drive, 
communications, and a printer, a power supply socket, and a contrast 
control (not visible) are located along the top 111 of housing 101. 
The display 102 senses the position of a special stylus 103 when the tip of 
the stylus is adjacent to the display. The display 102, for example, shows 
as display output, the outline of a form for data entry. This form can be 
generated, for example, by an application program running on the pen based 
computer system. The form may include a number of objects or fields. For 
example, in the example of FIG. 1, the form includes the "First Name" 
field 104, the "Last Name" field 105, the "Company" field 106 and the 
"Date" field 107. A user may write data directly in one or more of the 
fields using the stylus 103. For example, the name "Tom" is entered in the 
"First Name" field 104, the name "Jones" is entered in the "Last Name" 
field 105, and the name "International" is entered in the "Company" field 
106. 
Internally, the computer of FIG. 1 uses an Intel 80286 microprocessor chip 
running at 12 Mhz or more. Any suitable processor may be used with the 
present invention. Preferably, at least four megabytes of random access 
memory (RAM) is included.. 
"Gestures" are simple types of one or two stroke pen movements that invoke 
certain specified commands. The present invention utilizes approximately a 
dozen standard "gestures" that form part of its user interface. Examples 
of these "gestures" are left and right brackets ("[" and "]"), often used 
for selecting words or a section of text, an up-caret (" "), for accessing 
an editing pad that can be used for data entry, single strokes or "flicks" 
right, left, up or down, which are used for scrolling, and "taps" or 
"presses" which are used to invoke commands. 
The operating system of the preferred embodiment of the present invention 
also provides data to an application about any writing that is done on the 
display with the stylus. The operating system records events such as "pen 
down", "pen up", pen position, and pen movement, and makes the record of 
such events available to applications running on the computer system. 
An example of the entering and translation of data in a handwriting entry 
computer system is illustrated in FIGS. 2A-2C. Referring first to FIG. 2A, 
an array of fields 201-209 is illustrated. These fields are areas of a 
display or data entry means that are to be associated with particular 
data. In the example of FIG. 2A, the boundaries between the fields are 
illustrated. In actual implementation, however, it is not required that 
the boundaries be displayed. In the example, a user has written the words 
"Acme Sales" into field 205. After a user has written a block of ink, such 
as the words "Acme Sales", the block of ink is targeted to a target field. 
One method of targeting is described in U.S. patent application copending, 
commonly-assigned Ser. No. 07/819,450, filed Jan. 10, 1992 entitled 
"Targeting.". 
In this scheme, the bounds of the smallest rectangle that contains all of 
the strokes in the block is determined. This bound is shown as bound 210 
surrounding the strokes that comprise the word "Acme Sales" in FIG. 2A. 
The area overlap between the bounded rectangle and every object or field 
touched by the rectangle is calculated. If a preselected threshold 
percentage of the bounded rectangle overlaps a single field or object, 
that field or object is identified as the target. If no one field or 
object is overlapped by at least the preselected percentage area of the 
bounded rectangle, bounding rectangles are defined for each character or 
stroke in the block of data. The amount of overlap of the bounded 
characters is determined, and the field having the greatest area of 
overlap is identified as the target field. 
Any suitable targeting scheme may be utilized without departing from the 
scope of the present invention. For example, in an alternate embodiment, a 
bounded rectangle is generated for the data block and the center of the 
rectangle is determined. The object overlapped by the center of the 
writing rectangle is identified as the target object. 
After the block of ink has been targeted, a translation is performed to 
translate the strokes and characters of ink into translated text. In the 
preferred embodiment, the strokes are translated into characters (e.g., 
Unicode or ASCII). This translation can be performed using any of several 
well known translation algorithms. In one embodiment of the present 
invention, a multi-tiered translation scheme is employed. This scheme is 
described in U.S. patent application Ser. No. copending, commonly-assigned 
Ser. No. 07,795,009, filed Nov. 18, 1991, entitled "Method for Entering 
and Manipulating Spreadsheet Cell Data" assigned to the assignee of the 
present invention. 
Fields in the present invention may be one of three types of data formats: 
text, numbers, or "ink". A field may be designated as having a particular 
format prior to data being entered in the field. That is, a field may be 
designated to be a "text" field, a "value" field, or an "ink" field. In 
the preferred embodiment of the present invention, designating a field a 
particular format does not preclude data having a different format from 
being entered into the field. It does, however, indicate what kind of data 
is expected to be entered into the field, thereby aiding the 
character-recognition process. 
Regardless of the translation scheme used, there can sometimes be errors in 
the translation. These errors can take the form of untranslatable 
characters or mistranslated characters. After a translation is attempted, 
the translation results are displayed to the user. Referring to FIG. 2B, 
the attempted translation of "Acme Sales" is displayed as "Ac?e Sole?". 
The translation system correctly recognizes the "A", "c" and "e" of "Acme" 
and the "S", "l" and "e" of "Sales". It is unable to recognize the "m" of 
"Acme" and the "s" of "Sales", and incorrectly translates the "a" in 
"Sales" as an "o". A question mark "?" is used to represent an 
untranslatable character. 
After the system has displayed the translation, the user may seek to 
correct the translation, if necessary. This is accomplished by invoking an 
"editing pad" for the translated text. 
FIG. 2C illustrates an embodiment of an editing pad that can be invoked to 
correct the entry in field 205 of FIG. 2B. The editing pad 220 shown in 
FIG. 2C consists of a rectangular box containing a format display 221 that 
indicates the format of the field (in this case, Text), an entry display 
222 that displays the content of the field in a format based on the 
designated format of the field, and navigation buttons 223A to 223F that 
are used to, for example, accept an edited entry. 
To edit the field entry, corrections are made with the stylus by writing on 
the entry display 222. Entry display 222 illustrates the existing field 
content in a format that depends on the format of the field. The format of 
field 205 is Text. For a field that has a text format, the field content 
is displayed on a character by character basis in translated form, to the 
extent translation is possible. The same multi-tiered recognition process 
described above for translating handwritten field entries is used, with 
the following two exceptions: (1) an entry is treated on a character by 
character basis, rather than as a whole, such that each recognized 
character is displayed regardless of whether other characters are 
recognized and regardless whether all characters of an entry have the same 
format (or, in the case of a text entry, whether the entry is found in a 
dictionary) and (2) if a character is not recognized, a question mark 
("?") is displayed in lieu of that character. 
To correct the entry, the correct characters are handwritten over the 
incorrect or unrecognized characters. To replace the "?" in "Ac?e", an "m" 
is written over the "?" with the stylus. Occasionally, a character must be 
corrected more than once before the proper translation is made. The same 
process used to correct the "?" in "Ac?e" can be used to correct the "o" 
and the "?" in "Sole?". 
The need to correct translation errors interrupts the flow of data entry in 
prior art handwriting entry computer systems. The present invention 
eliminates the need to correct translation errors by enabling the user to 
elect to display only the original ink as data is entered. This is 
referred to here as "deferred translation". The translation of entered 
data may take place in a background process, but only the original ink is 
displayed. At a later, more convenient time, the user can elect to display 
translated text and perform corrections, if necessary, at that time. 
The present invention provides a number of methods for disabling the 
automatic display of translated text. These methods are referred to as 
"modal deferred translation", "automatic deferred translation", and 
"manual deferred translation". These methods are described below. 
Modal Deferred Translation 
A user may wish to prevent or disable the automatic display of translated 
text that is characteristic of prior art handwriting entry computer 
systems. For example, a user may wish to disable the automatic display of 
the translation of ink when entering large volumes of data, or 
sequentially entering data as part of completing a sales order form, 
inspection sheet, medical history form, when entering data quickly, or in 
any other situation where the automatic display of translated text is a 
barrier to data entry. 
One method provided by the present invention to disable automatic 
translation of ink to translated text is referred to here as "modal 
deferred translation". In modal deferred translation, the data entry 
system is configured so that no translated text is displayed unless 
specifically requested by the user. When modal deferred translation is 
enabled, only ink strokes and characters are displayed for all fields into 
which data may be entered. 
FIG. 3 is a flow diagram illustrating the operation of modal deferred 
translation using the present invention. At step 301, a block of ink is 
collected. At step 302, the block of ink is targeted and the appropriate 
target field is identified. At decision block 303, the argument "signature 
field?" is made. If the argument is true, the system proceeds to step 304. 
A signature field is never translated. A signature field is reserved for 
written signatures and other items that are to remain in the written form. 
If the argument at decision block 303 is false, the system proceeds to step 
305 and the field is identified as a data field. At decision block 306, 
the argument "defer translation?" is made. If the argument is true, the 
system proceeds to step 307 and the block of ink is displayed as ink only. 
At step 308, the block of ink is translated so that if the user 
subsequently wishes to display the translated characters, they are 
available. 
If the argument at decision block 306 is false, the block of ink is 
translated at step 309 and the result is displayed at step 310. After 
either of steps 310 or 308, the user may, at step 311, initiate a gesture 
at the target field associated with the block of ink. This gesture may be 
a "tap" gesture, where the stylus or data entering device is tapped once 
over the target field. The gesture may be translated by a multi-tiered 
translation process. Alternatively, another gesture or activating means 
may be used. 
At decision block 312 the argument "Deferred ink here?" is made. This 
argument determines if the contents of the target field are currently 
displaying ink or translated text. If the argument is true, meaning the 
target field is displaying ink, the system proceeds to step 313 and the 
ink is replaced with the translated text. The translated text may or may 
not have errors that require correction at this time. The user may elect 
to make corrections if necessary, or may elect to redisplay the ink. This 
can be accomplished by initiating the tap gesture (or its equivalent), or 
by some other activating means. If the argument at decision block 312 is 
false, the system proceeds to step 314 and some other action is taken. 
This action may be the provision of an editing pad to aid in the 
correction of the translated text. Alternatively, another action or 
actions may be taken. 
Modal deferred translation may be enabled by selecting and activating a 
menu command or its equivalent. The user may also use a menu command to 
disable manual deferred translation. In one embodiment of this invention, 
when modal deferred translation is disabled, subsequent entry of strokes 
and characters are automatically translated. Ink that had been entered 
while the computer system was in modal deferred translation remains 
displayed as ink until the user selectively indicates, on a field by field 
basis, that the translated text for that target field is to be displayed. 
In another embodiment of this invention, when modal deferred translation is 
disabled, all strokes and characters that had been entered while the 
computer system was in modal deferred translation are displayed as 
translated text. The user may, on a field by field basis, elect to display 
the original ink instead. 
Automatic Deferred Translation 
In some circumstances, the user may wish for the translated text to be 
automatically displayed, but only if it is a correct translation. 
Otherwise, the user prefers that the original ink be displayed instead. 
The present invention provides a method, referred to here as "automatic 
deferred translation", in which translated text is reviewed for 
untranslatable characters or for mistranslated words or characters. Only 
if there are no translation errors is the translated text displayed. 
Otherwise, the original ink is displayed. 
The present invention provides a two part test to implement automatic 
deferred translation. After the translation process has been performed, 
the translation is reviewed. If there are any untranslatable characters, 
the translated text is not displayed. In an optional step, if each 
character has been translated, the word is compared to entries in a 
dictionary. If the translated block is not found in the dictionary, it is 
assumed that a mistranslation has occurred and the original ink is 
displayed. The dictionary may be a system dictionary or a user-selected 
dictionary. 
FIG. 4 is a flow diagram illustrating automatic deferred translation in the 
present invention. At step 401, a block of ink is collected. At step 402, 
the target field of the block of ink is identified and the block of ink is 
associated with the target field. At decision block 403, the argument 
"signature field?" is made. If the argument is true, the system proceeds 
to step 404 and no translation takes place. If the argument at decision 
block 403 is false, the system proceeds to step 405 and the block of ink 
is translated, (translation may be performed in the background). 
At decision block 406, the argument "is there a translation error?" is 
made. That is, whether one or more strokes or characters are 
untranslatable. If the argument at decision block 406 is true, the system 
proceeds to step 407. At step 407, the ink is displayed and at step 408 
the attempted translation is saved for future use. 
If the argument at decision block 406 is false, the system displays the 
text. In an optional step, the system proceeds to decision block 409. At 
decision block 409, the argument "Is word in dictionary?" is made. If the 
translated word, although having no untranslatable characters, is not 
found in the dictionary, i.e., the argument is false, the system returns 
to step 407 where the ink is displayed and the translation is saved. If 
the argument at step 409 is true, an acceptable translation has been made 
and the text is displayed at step 410. 
Automatic deferred translation can be enabled and disabled by a menu 
command or some other suitable activation means. In one embodiment, after 
automatic deferred translation is disabled, target fields displaying ink 
continue to display ink. The user can select the attempted translations on 
a field by field basis. In another embodiment, all fields display 
translated text when automatic deferred translation is disabled. The user 
can select the original ink on a field by field basis. 
Manual Deferred Translation 
The present invention also provides a method that allows the user, on a 
field by field basis, to display the translated text or the original ink. 
This method is referred to here as "manual deferred translation". In 
manual deferred translation, the user is first shown the attempted 
translation of the original ink. If the user elects, the translated text 
is displayed and the user may make corrections as necessary. The user, 
after viewing the translated text, may elect to display the original ink 
and correct or display the translated text at a later time. 
FIG. 5 is a flow diagram illustrating the manual deferred translation 
scheme of the present invention. At step 501, a block of ink is collected. 
At step 502, the target field of the block of ink is identified and the 
block of ink is associated with the target field. At decision block 503, 
the argument "signature field?" is made. If the argument is true, the 
system proceeds to step 504 and there is no translation of the block of 
ink. If the argument is false, the system proceeds to step 505 and the 
block of ink is translated. 
At step 506, the translated text is displayed. At this point, the user may 
determine to display the original ink or the translated text. At decision 
block 507, the argument "user selects ink?" is made. If the argument at 
decision block 507 is false, the system proceeds to step 508 and the text 
is displayed if the argument at decision block 507 is true, the system 
proceeds to step 509 and the original block of ink is displayed in the 
target field. 
In one embodiment of this invention, manual deferred translation is always 
enabled, and the user may elect at any time to display the original ink 
instead of the translated text. This is true even if the user has 
"approved" the translated text. The original ink is retained and 
associated with the target field so that the user may redisplay it at any 
time. 
Generation of Block of Ink 
FIG. 6 illustrates a flow diagram of the generation of a block of ink. At 
step 601, a stroke is entered as data. (A stroke is defined here as the 
the ink entered between a pen down event and a pen up event). At step 602, 
the motion of the pen is sampled periodically so that point locations and 
deltas between point locations are determined. These points are collected 
to define a stroke. At step 603, the stroke data is forwarded to a data 
structure form. 
At step 604, a preliminary targeting analysis for the individual stroke is 
performed. At step 605, the stroke is sent to an ink object stroke 
manager. The ink object stroke manager collects strokes and combines them 
into blocks of strokes as required. At decision block 606, the argument 
"block completed?" is made. This argument is used to determine when a 
stroke is to be associated with a current block of strokes or if a new 
block of strokes is being generated. If the argument at decision block 606 
is false, the system returns to step 601 and another stroke is generated. 
If the argument is true, the system proceeds to step 607. At step 607, an 
ink object having n strokes is created. This ink object is the collected 
block of ink referred to in the initial steps of the flow charts of FIGS. 
3-5. 
Branching of Fields 
A number of steps may occur prior to the determination that text has been 
entered into a field. The present invention first determines the type of 
field that is being written to and whether that field has restrictions as 
to the type of data that can be entered into the field. The present 
invention also determines whether the writing is a gesture to take an 
action related to the field or is data to be entered into the field. 
FIG. 7 is a flow diagram illustrating the operation of the present 
invention in connection with querying fields as to their type and 
determining if a series of strokes comprise a gesture or data. At decision 
block 701, after a block of ink has been collected, the argument "Ask 
target if it wants raw ink?" is made. This determines if the target field 
is an ink-only field or not. If the argument at decision block 701 is 
true, the system proceeds to decision block 702. (One type of field that 
requests raw ink is a comb field such as illustrated in FIG. 2C). At 
decision block 702 the argument "Is target field a comb?" is made. If the 
argument at decision block 702 is true, the system proceeds to decision 
block 703 and the argument "Does ink look like special gesture?" is made. 
A comb field accepts a number of gestures in addition to the gestures 
accepted by non-comb fields. This step is used to determine if one of the 
additional gestures is being invoked in the comb field. If the argument at 
decision block 703 is true, the system proceeds to step 704 and the 
gesture is performed. 
If the argument at any of decision blocks 701,702 or 703 is false, the 
system proceeds to block 705. At step 705, an attempt is made to translate 
the ink as a gesture. If the ink does represent a gesture, the system 
proceeds to step 706 and the gesture is sent to the target field. The 
target field may refuse the gesture or it may accept the gesture. At 
decision block 707, the argument "Target refuses gesture" is made. If the 
gesture is not refused, the system proceeds to step 708 and the gesture is 
performed. If the argument at decision block 707 is true, the system 
proceeds to block 709 where it is determined that the ink entered in the 
field is not a gesture. 
At step 710, the target is polled to determine appropriate action. The 
target has three possible responses. The target can request the ink, it 
can request the translated text, and it can request that the ink or text 
be appended to ink or text already in the target field. 
If the system requests text, the system proceeds to step 711. At decision 
block 712, the argument "Append text?" is made. If the argument is false, 
the ink is translated in the background at step 713. At step 714, the 
translated text and ink are sent to the target. If the argument at 
decision block 712 is true, the present ink object is combined with the 
prior ink objects in the target field at step 715. The combined ink 
objects are sent to the target field at step 716 and the system then 
proceeds to step 713 and 714, where the combined ink object is translated 
in the background and the translated text and the ink object are sent to 
the target field. 
When the target field requests ink, the system proceeds to step 717. At 
decision block 718, the argument "Append ink?" is made. If the argument is 
false, the system proceeds to step 719 and a new ink object is sent to the 
target. If the argument at decision block 718 is true, the system proceeds 
to step 720 and the ink is combined with the existing ink object into a 
new ink object. At step 721, the combined ink object is sent to the target 
field. 
Label Object 
The present invention permits a user to choose between displaying ink or 
displaying translated text at any time. This contrasts with prior art 
schemes where the original ink is discarded after it has been translated. 
The present invention is implemented in an object oriented programming 
environment in the preferred embodiment. An object, referred to here as a 
"label object", is defined. The label object is an object that can display 
text or ink. The label object is associated with a field object, retains 
the ink and translated text, and displays one or the other when requested 
by the associated target field object. 
A block diagram of the present invention is illustrated in FIG. 8. A 
writing instrument 103 is used to write data on the display 102 of 
computer 101. Information about the strokes of data are provided to 
identifying means 801 on line 800. The identifying means 801 is used to 
determine blocks of data. When a block of data is identified, it is 
provided on line 802 to targeting means 803. Targeting means 803 is used 
to determine the target field of the block of data. 
After targeting, the block of data is provided on line 804 to translating 
means 805 and storage means 807. The translating means is used to 
translate the block of data and provides translated text on line 806 to 
storage means 807. The storage means 807 is used to store both the 
original block of ink and the translated text associated with a target 
field. 
A mode enabling means 809 is coupled to the storage means through line 808. 
The mode enabling means is used to indicate whether one of the modes of 
deferred translation are to be enabled. The mode enabling means selects 
one of the original ink or translated text stored in the storage means to 
output to display 102 based on user input on line 810. 
An error detecting means 813 is coupled to the translating means through 
line 806. The error detection means is optionally coupled to a dictionary 
815 through line 814. The error detecting means 813 is used to determine 
the presence of untranslated characters in the translated block of text. 
If there are no untranslated characters, the error detecting means may 
compare the translated block of text to acceptable translations stored in 
the dictionary 815. The dictionary contains a list of words that are to be 
expected in the field of use of the computer system. For example, the 
dictionary 815 may be an English language dictionary. If desired, the 
dictionary may append words or blocks of characters that have been 
accepted by a user as part of its database. The error detection means 813 
is coupled to the mode enabling means 809 on line 808. 
A switching means 811 is coupled to the storage means 807 via line 816. The 
switching means, in response to user input on line 812, selectably enables 
the display of the translated block of data or the original ink. Because 
both images are stored in the storage means 807, a user can select between 
the two images at any time. 
In the preferred embodiment of the present invention, the blocks within 
dashed line 817 of FIG. 8 are implemented as a series of instructions on a 
microprocessor based computer system. The storage means 807 may be 
integrated circuit memory such as RAM or DRAM, or it may be a moving media 
storage system such as a magnetic disk drive, magneto-optical drive or 
optical drive. 
The present invention may be practiced in a processor based computer system 
by executing the instructions of Appendix 1 and Appendix 2. 
Appendix 1 is an example of code suitable for implementing the deferred 
translation scheme of the present invention. Appendix 2 is an example of 
code for implementing a label object that can display either of text or 
ink. 
Thus, a method and apparatus for providing deferred translation of entry 
data is described. 
##SPC1##