Digitized handwriting sample ingestion systems and methods

Certain aspects of the present methods and systems may focus on computer implemented methods of obtaining digitized hand-writing data corresponding to a sample of a needed code point of a set of code points. Such methods may include: obtaining a sample of digitized handwritten text, the sample of digitized handwritten text including glyph data corresponding to a first glyph, the first glyph corresponding to the needed code point of the set of code points; associating the first glyph with the needed code point; identifying stroke data in the glyph data, the stroke data corresponding to a stroke component of the first glyph, determining a plurality of dimensional values of the stroke component in the stroke data; and associating the plurality of dimensional values with a new code point sample of the needed code point in a code point set data structure.

FIELD

The present disclosure relates to computer handwriting analysis and synthesis, and more particularly, to systems and methods for digitized handwriting data collection and analysis.

BACKGROUND

Since the advent of desktop publishing in the mid-1980s, it has become increasingly easy to use commonly-available software to create and print letters, cards, documents, and other printed matter. Moreover, at the present time, a computer user may have scores or even hundreds of high-quality fonts installed on his or her computer, with thousands of additional free and commercial fonts available via the Internet. As a result, many people have become accustomed to receiving printed materials that are not hand-written. Indeed, hand-written notes and cards may signal to a recipient a sense of importance and particular care because the sender personally took the effort to hand-craft the message.

There are numerous fonts that are intended to mimic generic handwriting to a certain extent. There are even services that will create a font to mimic a particular person's handwriting. However, existing personalized-handwriting fonts may appear mechanical and/or unnatural because individual glyphs may always be printed with identical geometry, whereas in an actual hand-written document, each individual character may have its own subtly unique geometry. Moreover, existing personalized-handwriting fonts and personalized-handwriting-font-creation services may have difficulty isolating individual glyph within a sample of cursive handwriting or other handwriting in which adjacent letters may be connected to one another.

Other techniques may use a variable glyph representation to mimic an individual's handwriting, such as the systems and methods described in U.S. Pat. Nos. 8,351,700 and 8,699,794.

DESCRIPTION

The detailed description that follows is represented largely in terms of processes and symbolic representations of operations by conventional computer components, including a processor, memory storage devices for the processor, connected display devices and input devices. Furthermore, these processes and operations may utilize conventional computer components in a heterogeneous distributed computing environment, including remote file servers, computer servers, and/or memory storage devices. Each of these conventional distributed computing components is accessible by the processor via a communication network, which may include, but is not limited to, the Internet.

The phrases “in one embodiment,” “in various embodiments,” “in some embodiments,” and the like are used repeatedly. Such phrases do not necessarily refer to the same embodiment. The terms “comprising,” “having,” and “including” are synonymous, unless the context dictates otherwise.

Reference is now made in detail to the description of the embodiments as illustrated in the drawings. While embodiments are described in connection with the drawings and related descriptions, there is no intent to limit the scope to the embodiments disclosed herein. On the contrary, the intent is to cover all alternatives, modifications, and equivalents. In alternate embodiments, additional devices, or combinations of illustrated devices, may be added to, or combined, without limiting the scope to the embodiments disclosed herein. For example, the embodiments set forth below are primarily described in the context of obtaining data corresponding to digitized samples of handwritten text in the English language via a digitizing surface device and a digitizing marking device. However, these embodiments are exemplary and are in no way limited to the type of item for which recommendations are being generated.

Exemplary Network Topology of a Client/Server-Based Handwriting Ingestion System

FIG. 1illustrates a first exemplary client/server-based handwriting ingestion system100in accordance with various embodiments. Client devices200A-B and a remote front-end server300A are in data communication with a network103. In various embodiments, network103may include the Internet, one or more local area networks (“LANs”), one or more wide area networks (“WANs”), cellular data networks, and/or other data networks. Network103may, at various points, be a wired and/or wireless network. Remote front-end server300A may be in data communication with a remote handwriting ingestion server300B and an administrative data store105. Remote handwriting ingestion server300B may be in data communication with a handwriting sample set data store108.

In these and other embodiments, client devices200, such as client device200A and client device200B, may be networked computing devices having form factors including general purpose computers (including “desktop,” “laptop,” “notebook,” “tablet” computers, or the like); mobile phones; watches, glasses, or other wearable computing devices; or the like. For simplified exemplary purposes, two client devices are shown, one of which is depicted as a laptop computer and the other of which is depicted as a tablet computer. In various embodiments there may be many more client devices200. The primary functional components of an exemplary, form-factor-independent client device200are described below in reference toFIG. 2.

Client device200A may be in data communication with an external digitizing surface device218A and a digitizing marking device (e.g. a “stylus”)220A. Digitizing marking device220A may be in data communication with client device200A via external digitizing surface device218A or may be in direct data communication with the client device (as indicated by dotted lines).

Client device200B includes a built in digitizing surface device218B and may be in data communication with a digitizing marking device220B.

In various embodiments, remote front-end server300A and remote handwriting ingestion server300B may be networked computing devices generally capable of accepting requests over network108, e.g. from client devices200, each other, various databases, and/or other networked computing devices, such as a remote handwriting generation server (not shown), and providing responses accordingly. The primary functional components of an exemplary remote server300, such as remote front-end server300A and remote handwriting ingestion server300B, are described below in reference toFIG. 3.

Exemplary Client Device

FIG. 2illustrates several components of an exemplary client device200, such as any of client devices200A-B, is illustrated. In some embodiments, a client device200may include many more components than those shown inFIG. 2. However, it is not necessary that all of these generally conventional components be shown in order to disclose an illustrative embodiment. The data communications between various components of client device200may be accomplished by wired and/or wireless connections

As shown inFIG. 2, exemplary client device200includes a central processing unit203in data communication with memory205via a bus208. Central processing unit203is an electronic circuit designed to carry out instructions of a computer program, e.g. obtained from memory205, by performing the basic arithmetic, logical, control and input/output (I/O) operations specified by the program's instructions. Memory205generally comprises some or all of random access memory (RAM), read-only memory (ROM), and/or a permanent mass storage device, such as a disk drive, flash memory, or the like. Bus208is a communication system that transfers data between components within client device200, and encompasses any related hardware components (wire, optical fiber, etc.) and software, including communication protocols.

Client device200may also include a network interface210for connecting to a network such as network103, one or more optional user input device(s)213, e.g. an alphanumeric keyboard, keypad, a mouse or other pointing device, a touchscreen, and/or a microphone, (or a user input port for connecting an external user input device), an optional digitizing surface device215, such as digitizing surface device215B, (or a port for connecting an external digitizing surface device, such as digitizing surface device215A), an optional digitizing marking device (or a port for connecting to an external digitizing marking device, such as digitizing marking devices218A-B), and the like, all interconnected, along with the network interface210, to central processing unit203and memory205via bus208.

Memory205of exemplary client device200may store program code, executable by central processing unit203, corresponding to an operating system223, as well as program code corresponding to various software applications, such as a browser application225, a handwriting ingestion application228, and other software applications (not shown). Operating system223and such various software applications may be loaded into memory205via network interface210or via a computer readable storage medium230, such as a hard-disk drive, a solid-state drive, an optical disc, a removable memory card, and/or the like.

Browser application225is a software application for retrieving, presenting, and traversing information resources on a network, such as network108. Although browser application225may be primarily intended to use the World Wide Web, it may also be used to access information resources provided by remote servers in private networks. An information resource may be a web page, an image, a video, or other piece of content and may be identified by a Uniform Resource Identifier (URI/URL) on network108. An information resource may also provide browser application225executable program code for web applications, i.e. a software application that runs in and is rendered by browser application225.

In operation, operating system223manages the hardware and software resources of client device200and provides common services and memory allocation for various software applications, such as research study data acquisition and quality control application228. For hardware functions such as network communications via network interface210, receiving data via input213, outputting data via optional display215, and allocation of memory205for various software applications, such as handwriting ingestion application228, operating system223acts as an intermediary between software executing on the client device and the device's hardware.

For example, operating system223may cause a representation of available software applications, such as browser application225and handwriting ingestion228, to be presented to a user of client device200via display215. If client device200obtains an indication from a user, e.g. via user input213, a desire to use handwriting ingestion application228, operating system223may instantiate a handwriting ingestion application process (not shown), i.e. cause central processing unit203to begin executing the executable instructions of the handwriting ingestion application and allocate a portion of memory205for its use.

In the case of a web application, browser application225may act as an intermediary between a software service operating on a remote server and the operating system223. For example, a software service equivalent of handwriting ingestion application228may be executing on front-end server400A.

Although an exemplary client device200has been described with hardware components that generally conforms to conventional general purpose computing devices, a client device may be any of a great number of devices capable of communicating with network103and executing instructions for performing handwriting ingestion application228.

Exemplary Server

FIG. 3illustrates several components of an exemplary server300, such as front-end server300A and handwriting ingestion server300B, in accordance with at least one exemplary embodiment are illustrated. In some embodiments, a server300may include many more components than those shown inFIG. 3. However, it is not necessary that all of these generally conventional components be shown in order to disclose an illustrative embodiment. As shown inFIG. 3, a server300includes a central processing unit303and memory305connected by a bus308.

Central processing unit303is an electronic circuit designed to carry out instructions of a computer program, e.g. obtained from memory305, by performing the basic arithmetic, logical, control and input/output (I/O) operations specified by the program's instructions. Memory305may generally include some or all of random access memory (RAM), read-only memory (ROM), and/or a permanent mass storage device, such as a disk drive, flash memory, or the like. Bus308is a communication system that transfers data between components within exemplary server300, and includes any related hardware components (wire, optical fiber, etc.) and software, including communication protocols.

Server300may also include a network interface310for connecting to a network such as network103, one or more optional user input device(s)313, e.g. an alphanumeric keyboard, keypad, a mouse or other pointing device, a touchscreen, and/or a microphone, (or a user input port for connecting an external user input device) and/or an optional display315(or a display port for connecting an external display device), both interconnected along with the network interface310via bus308.

Memory305may store an operating system320and program code for various software services323. For example, front-end server300A may include executable instructions for performing user session management service323A (indicated by dotted lines) and handwriting ingestion server300B may include executable instructions for performing handwriting ingestion service323B (indicated by dotted lines).

Program code for these and other such software services, such as a software services (not shown) equivalent to handwriting ingestion application228, may be loaded into memory305from a non-transient computer readable storage medium325using a drive mechanism (not shown) associated with the non-transient computer readable storage medium, such as, but not limited to, a DVD/CD-ROM drive, memory card, or the like. Software components may also be loaded into memory304via the network interface310. A server300may also communicate via bus308with a database (not shown), such as admin database105and/or trial data database108, or other local or remote data store.

In operation, operating system320manages the hardware and software resources of server300and provides common services and memory allocation for various software services, such as user session management service323A or handwriting ingestion service323B. For hardware functions such as network communications via network interface310and allocation of memory305for various software services, such as handwriting ingestion service323B, operating system320may act as an intermediary between software executing on server300and the server's hardware.

Although an exemplary server300has been described having hardware components that generally conform to a conventional general purpose computing device, a server may be any of a great number of devices capable of communicating with network103and executing instructions for performing user session management service323A and/or handwriting ingestion service323B.

In some embodiments, a server300may comprise one or more replicated and/or distributed physical or logical devices. In some embodiments, one or more of front-end server300A and handwriting ingestion server300B may be embodied by the same physical device.

Client/Server-Based Handwriting Ingestion Systems

Referring collectively toFIGS. 1-3, and as is described in more detail below, an instance of client handwriting ingestion application228operating on a client device, such as client devices200A-B, remote user session management service323A operating on front-end server300A, and remote handwriting ingestion service323B operating on handwriting ingestion server300B (collectively a client/server-based handwriting ingestion system) in accordance with the present embodiments may operate to create a code-point sample data set of a user's handwriting comprehensive enough to allow the provider to obtain text data including a message from a user and generate an image, e.g. for electronic display and/or hard-copy printing, including a sequence of glyphs corresponding the message and having the appearance of being rendered in the user's handwriting. Such a client/server based handwriting ingestion system may be operated in furtherance of a handwriting digitization service provider (not shown) providing handwriting reproduction services.

Instantiations of client handwriting ingestion application228may act as an interface between a user of client device200and user session management service323A operating on front-end server300A. Upon instantiation, client handwriting ingestion application228may send a “create new session” request to user session management service323A, for example including identifying information for client device200, identifying information for the particular instantiation of client handwriting ingestion application228, and/or user-account credentials such as a user name and password, obtained from the user or stored in memory205. If presented, the user-account credentials may be associated with an existing user account or with a generic, temporary, and/or anonymous “guest” account.

User session management service323A may create a corresponding user session (not shown) associated with the particular instantiation of client handwriting ingestion application228, identified by a user-session identifier (referred to herein as a “session ID”) and may obtain user account information, such as a user identifier, associated with the provided identifying information from administrative data store105and provide a response to client handwriting ingestion application228, which may include information related to features and services provided by the handwriting digitization service provider which the user-account associated with the user-account credentials are authorized to access.

Client handwriting ingestion application228may then present the user with a menu of options, e.g. via optional display215, and wait for the user to indicate a selection of a specific option, e.g. via optional input213. Such options may include obtaining a new handwriting sample, viewing a measure of completeness of a handwriting sample set associated with the user identifier, creating a new handwriting sample set, and the like. Upon obtaining a selection of one of the presented options, e.g. via user input213, client handwriting ingestion application215may process the selection and generate a request corresponding to the selected option.

These requests may be provided to user session management service323A operating on front-end server300A, e.g. via network103. User session management service323A may process the requests, provide related internal requests to handwriting ingestion service323B, obtain responses from handwriting ingestion service323B, provide responses to client handwriting ingestion application228and store records of these requests and responses and other related data, e.g. in administrative data store105indexed by a user identifier and/or a user session identifier.

When a user indicates a desire to create a new handwriting sample set, the handwriting ingestion system may identify a desired style of communication for a user, such as general English language communication, formal Japanese language communication, technical Arabic language communication, and the like. The handwriting ingestion system may then determine a code-point sub-set, applicable to the desired style of communication, from one or more universal code-point sets, such as the set of standard Unicode character code-points.

For each code-point within the applicable code-point sub-set, the handwriting ingestion system may collect one or more digitized samples of a handwritten glyphs corresponding to the code-point. For example, as is explained in more detail below, handwriting ingestion application228may provide a user interface that presents a user of client device200with a visual display of one or more code-points and instructs the user to write out glyphs corresponding to the one or more code-points using digitizing marking device220to make one or more strokes on digitizing surface device218. Depending on the desired style of communication and the user's individual writing style, a digitized glyph sample may be made up of one or more strokes and multiple digitized glyph samples may also be contained within a single stroke. Therefore, handwriting ingestion application228may (1) accommodate glyph samples being made up of multiple, non-sequential strokes, for example if the digitizing marking device leaves the digitizing surface device during the glyph sample capture (as may be the case when hand writing the lower case English letters “f,” “i,” “j,” “k,” “t,” and “x” for example) (2) provide a user with the opportunity to segment strokes into individual glyphs, and then provide the user with the opportunity to associate one or more strokes and/or one or more stroke segments with a code-point

Referring toFIG. 4by way of example, the word “This”403is made up of four sequential characters represented by four distinct code-points: a “T” code-point405A, an “h” code-point405B, an “i” code-point405C, and a “s” code-point405D; a corresponding digitized hand-written sample408of the word “This” may be made up of four sequential glyphs: a “T” glyph410A, an “h” glyph410B, an “i” glyph410C, and an “s” glyph410D. In this example, the digitized hand-written sample of the word “This” is made up of four strokes413A-D, however strokes413A-D do not respectively correspond to glyphs410A-D. Rather, a ‘horizontal’ stroke413A and ‘vertical’ stroke413B together make up “T” glyph410A; a third stroke413C makes up “h” glyph410B, “s” glyph410D, and part of “i” glyph410C. A ‘dot’ stroke413D makes up the remainder of the “i” glyph410C.

In accordance with various embodiments, after a user finishes ‘dot’ stroke413D, the user may then selectively segment third stroke413C into an “h” stroke segment415A, an “i” stroke segment415B, and an “s” stroke segment415C. The user may then associate ‘horizontal’ stroke413A and ‘vertical’ stroke413B as a digitized glyph sample for “T” code-point405A, “h” stroke segment415A as a digitized glyph sample for “h” code-point405B, “i” stroke segment415B and ‘dot’ stroke413D as a digitized glyph sample for “i” code-point405B, and “s” stroke segment415C as a digitized glyph sample for “s” code-point405D.

For each digitized glyph sample, the handwriting ingestion system may record code-point sample contextual data related to the digitized glyph sample. For example, the handwriting ingestion system may record where the digitized glyph sample was collected within the context of a group of glyphs (e.g. at the beginning, or in the middle of a sequence of glyphs written as a continuous stroke), whether the previous glyph, if any, had an upward or downward exit angle (described in more detail below) and a low or high exit point, and whether the subsequent glyph, if any, had an upward or downward entry angle and a low or high entry point, and the like.

For each stroke and/or stroke segment making up the digitized glyph sample, the handwriting ingestion system may also record stroke dimension data. For example, for each stroke, the handwriting ingestion system may obtain and record data values relating to the stroke overall temporal duration; the stroke's relative horizontal displacement over time, the stroke's relative vertical displacement displacement over time, the stroke's relative rotational displacement over time, the stroke's relative angular displacement over time, and the stroke's downward pressure over time.

Each glyph sample may be added to a code-point sample data set associated with the user (e.g. via a user identifier). The handwriting ingestion system will continue to collect digitized glyph samples for a given code-point until the system determines it can reproduce a full range of glyph variations of the code-point.

Handwriting Ingestion User Interface

FIGS. 5A-Killustrate various states of an exemplary user interface500provided by handwriting ingestion application228operating on a client device200for enabling a user of the client device to selectively provide digitized samples of the user's handwriting via a digitizing surface device in conjunction with a digitizing marking device. Exemplary user interface500may, for example, be rendered by display215in response to the execution of instructions of an instantiation of client handwriting ingestion application228operating on a client device200. Handwriting ingestion UI500may be rendered in a window503including an instructive prompt505, a sample text display508, and an input reproduction rendering display510.

InFIG. 5A, handwriting ingestion UI500is illustrated in an initial prompt state500A. In initial prompt state500A, instructive prompt505displays text instructing a user operating client device200to input the sample text (“This is a test”) displayed in sample text display508via digitizing surface device218and digitizing marking device220. In the illustrated example, the displayed sample text includes: one instance of a “T” code-point; one instance of an “h” code-point; two instances of an “i” code-point; three instances of an “s” code-point; one instance of an “a” code-point; two instances of a “t” code-point; and one instance of an “e” code-point.

InFIG. 5B, handwriting ingestion UI500is illustrated in a first potential sample ingestion display state500B. In first potential sample ingestion display state500B, input reproduction rendering display510displays a visual rendering of input data corresponding to a first stroke513A, e.g. obtained via digitizing surface device218in combination with digitizing marking device220. The characteristics of such input data are discussed in more detail below, particularly in reference toFIGS. 7A-B.

InFIG. 5C, handwriting ingestion UI500is illustrated in a second potential sample ingestion display state500C. In second potential sample ingestion display state500C, input reproduction rendering display510displays a visual rendering of input data corresponding to first stroke513A and a second stroke513B, e.g. obtained via digitizing surface device218in combination with digitizing marking device220subsequent to the first stroke. Similar to the example discussed above with reference toFIG. 4, first and second strokes513A-B make up a glyph sample corresponding to the first, “T” code-point of the sample text.

InFIG. 5D, handwriting ingestion UI500is illustrated in a third potential sample ingestion display state500D. In third potential sample ingestion display state500D, input reproduction rendering display510displays a visual rendering of input data corresponding to first stroke513A, second stroke513B, and a third stroke513C. Similar to the example discussed above with reference toFIG. 4, third stroke513C includes glyph samples corresponding to the “h” code-point, part of the “i” code-point, and one of the “s” code points of the sample text.

InFIG. 5E, handwriting ingestion UI500is illustrated in a fourth potential sample ingestion display state500E. In fourth potential sample ingestion display state500E, input reproduction rendering display510displays a visual rendering of input data corresponding to first stroke513A, second stroke513B, third stroke513C, and a fourth stroke513D. Similar to the example discussed above with reference toFIG. 4, fourth stroke513D completes the glyph sample of the “i” code-point of the sample text began by stroke513C.

InFIG. 5F, handwriting ingestion UI500is illustrated in a fifth potential sample ingestion display state500F. In fifth potential sample ingestion display state500F, input reproduction rendering display510displays a visual rendering of input data corresponding to first stroke513A, second stroke513B, third stroke513C, and fourth stroke513D, along with a fifth stroke513E, a sixth stroke513F, a seventh stroke513G, an eighth stroke513H, a ninth stroke513I, a tenth stroke513J, and an eleventh stroke513K. First through eleventh strokes513A-K collectively make up glyph samples corresponding to the sample text in sample text display508.

InFIG. 5G, handwriting ingestion UI500is illustrated in a sample segmentation state500G. In sample segmentation state500G, instructive prompt505displays text instructing a user operating client device200to segment any multi-character strokes displayed in input reproduction rendering display510. For example, the user may user a mouse or other pointer-based user input213to select segmentation points515within a stroke where one glyph sample ends and another begins. In the illustrated example, third stroke513C is segmented at two segmentation points515A-B, fifth stroke513C is segmented at one segmentation point515C, and eighth stroke513H is segmented at two segmentation points515D-E. As is shown inFIG. 5H, third stroke513C is segmented into three stroke segments520A-C, fifth stroke513C is segmented into two stroke segments520D-E, and eighth stroke513H is segmented into three stroke segments520F-H.

InFIG. 5H, handwriting ingestion UI500is illustrated in an initial stroke assignment state500H. In initial stroke assignment state500G, the sample text display has been replaced by a character sequence display523and instructive prompt505displays text instructing a user operating client device200to assign the various strokes and stroke segments displayed in reproduction rendering display510to a corresponding character from the character sequence display.

InFIG. 5I, handwriting ingestion UI500is illustrated in a first ongoing stroke assignment state500I. In first ongoing stroke assignment state500G, the first, “T” character of character sequence display523has been highlighted and first stroke513A has been selected, indicating the first stroke should be associated with a glyph sample for the “T” code point.

InFIG. 5J, handwriting ingestion UI500is illustrated in a second ongoing stroke assignment state500J. In second ongoing stroke assignment state500J, the first, “T” character of character sequence display523is still highlighted and second stroke513B has been selected, indicating the second stroke should also be associated with the glyph sample for the “T” code point.

InFIG. 5K, handwriting ingestion UI500is illustrated in a third ongoing stroke assignment state500K. In third ongoing stroke assignment state500K, the second, “h” character of character sequence display523is now highlighted and first stroke segment520A has been selected, indicating the first stroke segment should be associated with a glyph sample for the “h” code point. This stroke assignment process may be continued until the user indicates all strokes and stroke segments have been assigned to the appropriate glyph samples.

Code Point Sample Stroke and Stroke Segment Assignment

FIG. 6illustrates the assignments of data corresponding to various strokes and stroke samples obtained via digitizing surface device218and digitizing marking device220during the exemplary handwriting sample ingestion process described above with respect toFIGS. 5A-Kare combined into glyph samples and assigned to code-points.

Data collected during stroke513A and stroke513B are combined to form glyph sample603A, which is assigned to the “T” code point605A.

Data collected during stroke513G forms glyph sample603B, which is assigned to the “a” code-point605B.

Data collected during stroke segment520G forms glyph sample603C, which is assigned to the “e” code-point605C.

Data collected during stroke segment520A forms glyph sample603D, which is assigned to the “h” code-point605D.

Data collected during stroke segment520B and stroke513D are combined to form glyph sample603E, which is assigned to the “i” code-point605E. Data collected during stroke segment520D and stroke513F are combined to form glyph sample603F, which is also assigned to the “i” code-point605E.

Data collected during stroke segment520F and stroke513J are combined to form glyph sample603J, which is assigned to the “t” code-point605F. Data collected during stroke513I and stroke513K are combined to form glyph sample603K, which is also assigned to the “t” code-point605F.

Data collected during stroke segment520C forms glyph sample603G, which is assigned to the “s” code-point605G. Data collected during stroke segment520E forms glyph sample603H, which is also assigned to the “s” code-point605G. Data collected during stroke segment520C forms glyph sample603G, which is assigned to the “s” code-point605G. Data collected during stroke segment520H forms glyph sample603I, which is also assigned to the “s” code-point605G.

Glyph Sample Data and Data Collection

FIGS. 7A-Billustrate respective graphical representations700A-B of stroke data collected by client handwriting ingestion application228during stroke513A and stroke513B (which combine to form a capital letter “T” in the English language) in the example described above and obtained via digitizing surface device218and digitizing marking device220.

Stroke data obtained from digitizing surface device218and digitizing marking device220corresponding to strokes, such as strokes513A-B, may take the form of a time-based series of sampled stroke dimensional data values. Stroke513A extends from an origin point703A, through a series of intermediate points703B-I, to a termination point703J. Stoke513B extends from an origin point705A, through a series of intermediate points705B-H, to a termination point7051.

An exemplary handwriting code point set data structure (hw_cp_set_DS) may include a character map data structure and other associated values corresponding to various aspects of the code point set, such as a user identifier value, an average character width value, an average character height value, a maximum character height value, and the like.

An exemplary character map data structure (char_map_DS) may include a plurality of code point data structures, e.g. a code point data structure for each character in the code point set (A, B, C, D, . . . T, . . . h, i, . . . s, . . . ).

An exemplary code point data structure may include one or more code point sample data structures, e.g. a code point sample data structure for each code point sample obtained for a given code point.

An exemplary code point sample data structure may include one or more stroke data structures, e.g. a stroke data structure corresponding to each stroke within a given code point sample. A code point sample data structure may also include values corresponding to the source of the sample (e.g. the word “This”) and the relative position within the source (e.g. 0 in the case of “T,” 1 in the case of “h,” etc.).

An exemplary stroke data structure may include of a time-based series of sampled stroke dimensional data values, e.g. data values corresponding to a time value (representing the sampling interval of the stroke), relative horizontal position or displacement values, relative vertical position or displacement values, pressure values, spin values, angle values, and the like. (Examples of the horizontal position values, vertical position values, and pressure values are visually depicted inFIGS. 7A-B(with pressure values being illustrated by data point diameter).)

Thus, an exemplary handwriting code point set data structure may be representable as:

Exemplary Series of Communications

FIG. 8illustrates a first exemplary series of communications800between client device200A, front-end server300A, and handwriting ingestion server300B in accordance with various embodiments of an exemplary client/server-based handwriting ingestion system, such as the exemplary client/server-based handwriting ingestion system illustrated inFIG. 1.

Client device200A may obtain803and process805a user session initiation command, e.g. via user input213in response to a prompt provided via display215.

Client device200A may provide front-end server300A with a corresponding user session initiation request808. User session initiation request808may include user identifying information corresponding to a user of client device200A, e.g. via an alphanumeric identifier associated with a user, and the like.

Front-end server300A may process810user session initiation request808. For example, front-end server300A may instantiate a user session associated with the user identifying information, obtain additional meta-data relating to the user identifying information, e.g. from administrative data store105and/or other sources, and the like.

Front-end server300A may provide handwriting ingestion server300B with a user data request813. User data request813may include data obtained from client device200A via user session initiation request808, such as a user identifier, and via processing810the user session initiation request by front-end server300A.

Handwriting ingestion server300B may process815user data request813. For example, handwriting ingestion server300B may obtain handwriting sample data associated with the user identifier, e.g. from handwriting sample set data store108, determine a measure of handwriting sample set completeness, and the like.

Handwriting ingestion server300B may then provide a user data response818to front-end server300A.

Front-end server300A may then process820user data response818, e.g. by parsing the user data response to extract data to pass on to client device200.

Front-end server300A may then provide a user session initiation response823to client device200A. User session initiation response823may include a user session identifier, handwriting sample set data, and/or the like.

Client device200A may then process825user session initiation response820, for example by rendering information provided in the user session initiation response via display215.

Client device200A may then obtain828and process830new handwriting sample data, such as in the manner described above in reference toFIGS. 5A-K, to obtain stroke dimension data for one or more glyph samples.

Client device200A may provide front-end server300A with a corresponding handwriting sample update request833. Handwriting sample update request833may include the user session identifier, code-point identifiers, stroke dimension data from one or more glyph samples, and/or the like.

Front-end server300A may process835handwriting sample update request833and provide handwriting ingestion server300B with an internal handwriting sample update request838. Internal handwriting sample update request838may include the user identifier associated with the user session, code-point identifiers, stroke dimension data from one or more glyph samples, and/or the like.

Handwriting ingestion server300B may process840internal handwriting sample update request838. For example, handwriting ingestion server300B may store the stroke dimension data in handwriting sample set data store108associated with the respective user identifier and code-point identifiers, determine an updated measure of handwriting sample set completeness, and the like.

Handwriting ingestion server300B may then provide an internal handwriting sample update response843to front-end server300A. For example, internal handwriting sample update response843may include an acknowledgement the handwriting sample set associated with the user identifier has been updated, the updated measure of handwriting sample set completeness, and the like.

Front-end server300A may then process845internal handwriting sample update response843and provide a handwriting sample update response848to client device200A. Handwriting sample update response848may include an acknowledgement the handwriting sample set associated with the user identifier has been updated, the updated measure of handwriting sample set completeness, and the like.

Client device200A may then process850handwriting sample update response848, for example by rendering information provided in the handwriting sample update response via display215, and the like.

Handwriting Sample Ingestion Routine

FIG. 9illustrates an exemplary handwriting sample ingestion routine900, which may be implemented by client handwriting ingestion application228operating on a client device, such as client devices200A-B.

Handwriting sample ingestion routine900may obtain a user identifier at execution block903. For example, client handwriting ingestion application228may obtain a user identifier via a user input213in response to a user's interaction with a user login interface, a user profile file stored in memory205, or the like.

Handwriting sample ingestion routine900may provide a handwriting code-point set request, including the user identifier, at execution block905. For example, client handwriting application may provide the handwriting code-point set request to front-end server300B via network103.

At decision block908, if a handwriting code-point set associated with the user identifier is obtained, e.g. from front-end server300B (or directly from handwriting ingestion server300A) via network103in response to the handwriting code point set request, then handwriting sample ingestion routine900may call a handwriting code-point set completeness sub-routine1100, described below in reference toFIG. 11; otherwise handwriting sample ingestion routine900may proceed to execution block910.

Handwriting sample ingestion routine900may then call an initial sample collection sub-routine1000, described below in reference toFIG. 10.

Handwriting sample ingestion routine900may then call handwriting code-point set completeness sub-routine1100.

At decision block913, if the result of handwriting code-point set completeness sub-routine1100indicates the handwriting code-point set in its current state is useable, handwriting sample ingestion routine900may proceed to decision block915; otherwise handwriting sample ingestion routine900may call a supplemental sample collection sub-routine1200, discussed below in reference toFIG. 12. After the completion of supplemental sample collection sub-routine1200, handwriting sample ingestion routine900may loop back to handwriting code point set completeness sub-routine block1100.

At decision block915, if the result of handwriting code-point set completeness sub-routine1100indicates the handwriting code-point set in its current state is complete, handwriting sample ingestion routine900may proceed to execution block918; otherwise,900may proceed to execution block920.

At execution block918, handwriting sample ingestion routine900may provide a handwriting code point data structure complete prompt, e.g. via display215, to inform the user of client device200that no additional handwriting samples are needed. Handwriting sample ingestion routine900may then proceed to termination block999.

At execution block920, handwriting sample ingestion routine900may provide a supplemental sample collection prompt, e.g. via display215, to inform the user of client device200that the handwriting code point set data structure is usable, but that additional samples may improve the results of handwriting generation services, and inviting the user to provide additional samples.

At decision block923, if handwriting sample ingestion routine900obtains an affirmative response to the supplemental sample collection prompt, then handwriting sample ingestion routine900may call supplemental sample collection sub-routine1200; otherwise, handwriting sample ingestion routine900may proceed to termination block999.

Handwriting sample ingestion routine900may end at termination block999.

Initial Sample Collection Sub-Routine

FIG. 10illustrates an exemplary initial sample collection sub-routine1000, which may be implemented by client handwriting ingestion application228, e.g. in response to a call from handwriting sample ingestion routine900.

Initial sample collection sub-routine1000may obtain an initial handwriting sample collection request at execution block1003. The initial handwriting sample collection request may include a user identifier.

Initial sample collection sub-routine1000may instantiate a handwriting code-point set data structure and associate the handwriting code-point set data structure with the user identifier at execution block1005.

Initial sample collection sub-routine1000may select an initial handwriting sample at execution block1008. For example, initial sample collection sub-routine1000may obtain a handwriting style identifier associated with the user identifier and then obtain a handwriting sample identifier associated with the handwriting style identifier.

Initial sample collection sub-routine1000may then call a handwriting sample collection sub-routine1300, described below in reference toFIGS. 13A-B, which may include providing the handwriting sample identifier to handwriting sample collection sub-routine1300.

Initial sample collection sub-routine1000may return, e.g. to handwriting sample ingestion routine900, at return block1099.

Handwriting Code Point Set Completeness Sub-Routine

FIGS. 11A-Billustrate an exemplary handwriting code point set completeness sub-routine1100, which may be implemented by client handwriting ingestion application228, e.g. in response to a call from handwriting sample ingestion routine900.

Referring toFIG. 11A, handwriting code point set completeness sub-routine1100may obtain a handwriting code point set completeness request at execution block1103. Handwriting code point set completeness request may include a user identifier, a handwriting code point set data structure identifier and a handwriting style identifier.

Handwriting code point set completeness sub-routine1100may determine one or more critical code point set requirements associated with the handwriting style identifier at execution block1105.

Handwriting code point set completeness sub-routine1100may set a critical pass flag value to true at execution block1108.

At starting loop block1110, handwriting code point set completeness sub-routine1100may process each critical code point set requirement in turn.

At decision block1113, if the handwriting code point set data structure associated with the handwriting code point set data structure identifier does not contain a minimum acceptable number of glyph samples for the current critical code point set requirement, handwriting code point set completeness sub-routine1100may proceed to execution block1115; otherwise handwriting code point set completeness sub-routine1100may proceed to ending loop block1118.

Handwriting code point set completeness sub-routine1100may set the critical pass flag value to false at execution block1115.

At ending loop block1118, handwriting code point set completeness sub-routine1100may loop back to starting loop block1110and process the next critical code point set requirement, if any.

At decision block1120, if the critical pass flag false is equal to true, handwriting code point set completeness sub-routine1100may proceed to execution block1123(FIG. 11B); otherwise handwriting code point set completeness sub-routine1100may proceed to return block1197.

Referring now toFIG. 11B, handwriting code point set completeness sub-routine1100may determine one or more complete code point set requirements associated with the handwriting style identifier at execution block1123.

Handwriting code point set completeness sub-routine1100may set a complete pass flag value to true at execution block1125.

At starting loop block1128, handwriting code point set completeness sub-routine1100may process each complete code point set requirement in turn.

At decision block1130, if the handwriting code point set data structure associated with the handwriting code point set data structure identifier does not contain a minimum acceptable number of glyph samples for the current complete code point set requirement, handwriting code point set completeness sub-routine1100may proceed to execution block1133; otherwise handwriting code point set completeness sub-routine1100may proceed to ending loop block1135.

Handwriting code point set completeness sub-routine1100may set the complete pass flag value to false at execution block1133.

At ending loop block1135, handwriting code point set completeness sub-routine1100may loop back to starting loop block1128and process the next complete code point set requirement, if any.

At decision block1138, if the complete pass flag false is equal to true, handwriting code point set completeness sub-routine1100may proceed to return block1198; otherwise handwriting code point set completeness sub-routine1100may proceed to return block1199.

At return block1197, handwriting code point set completeness sub-routine1100may return a handwriting code point set unusable message, e.g. to handwriting sample ingestion routine900.

At return block1198, handwriting code point set completeness sub-routine1100may return a handwriting code point set complete message, e.g. to handwriting sample ingestion routine900.

At return block1199, handwriting code point set completeness sub-routine1100may return a handwriting code point set usable message, e.g. to handwriting sample ingestion routine900.

Supplemental Sample Collection Sub-Routine

FIG. 12illustrates an exemplary supplemental sample collection sub-routine1200, which may be implemented by client handwriting ingestion application228, e.g. in response to a call from handwriting sample ingestion routine900.

Supplemental sample collection sub-routine1200may obtain a supplemental handwriting sample collection request at execution block1203. The supplemental handwriting sample collection request may include a user identifier and/or a handwriting code point set data structure identifier.

Supplemental sample collection sub-routine1200may parse a handwriting code point set data structure associated with the handwriting code point set data structure identifier at execution block1205. For example, supplemental sample collection sub-routine1200may determine one or more code-points for which the handwriting code point set data structure identifier does not contain a minimum acceptable number of glyph samples.

Supplemental sample collection sub-routine1200may select a supplemental handwriting sample target at execution block1208. For example, supplemental sample collection sub-routine1200may select a supplemental handwriting sample target based on the one or more code-points for which the handwriting code point set data structure identifier does not contain a minimum acceptable number of glyph samples.

Supplemental sample collection sub-routine1200may then call handwriting sample collection sub-routine1300, described below in reference toFIGS. 13A-B, which may include providing the supplemental handwriting sample target identifier to handwriting sample collection sub-routine1300.

Supplemental sample collection sub-routine1200may return, e.g. to handwriting sample ingestion routine900, at return block1299.

Handwriting Sample Collection Sub-Routine

FIGS. 13A-Billustrate an exemplary handwriting sample collection sub-routine1300, which may be implemented by client handwriting ingestion application228, e.g. in response to a call from initial sample collection sub-routine1000or supplemental sample collection supplemental sample collection sub-routine1200.

Referring toFIG. 13A, handwriting sample collection sub-routine1300may obtain a handwriting sample collection request at execution block1303.

Handwriting sample collection sub-routine1300may render a visual representation of the handwriting sample text, e.g. via display215, at execution block1305(see sample text display508,FIG. 5).

At decision block1308, if digitized handwriting sample data collection is complete, then handwriting sample collection sub-routine1300may proceed to execution block1311; otherwise handwriting sample collection sub-routine1300may proceed to decision block1309.

At decision block1309, if a sample collection period timeout or user cancellation interrupt has been received, handwriting sample collection sub-routine1300may proceed to return block1398.

At execution block1310, handwriting sample collection sub-routine1300may obtain digitized handwriting sample data, such as the digitized handwriting sample data described above with respect toFIGS. 7A-B, representative of the handwriting sample text and captured by digitizing marking surface218and digitizing marking device220,

Handwriting sample collection sub-routine1300may provide a digitized handwriting sample image representative of the handwriting sample data, e.g. via display215, at execution block1311.

Handwriting sample collection sub-routine1300may provide a stroke segmentation prompt, e.g. via display215, at execution block1312.

Handwriting sample collection sub-routine1300may capture stroke segmentation points, e.g. via user input213, at execution block1313.

Handwriting sample collection sub-routine1300may map any captured stroke segmentation points to points in the digitized handwriting sample data at execution block1314.

At decision block1315, if the stroke segmentation process is complete, handwriting sample collection sub-routine1300may proceed to starting loop block1318, described below in reference toFIG. 13B); otherwise handwriting sample collection sub-routine1300may continue to wait for the stroke segmentation process to complete.

Referring now toFIG. 13B, at starting loop block1316, handwriting sample collection sub-routine1300may process each code point sample in the handwriting sample text in turn.

Handwriting sample collection sub-routine1300may provide a stroke identification prompt at execution block1318, e.g. via display215.

At decision block1320, if a stroke identification response is obtained, e.g. via user input213, handwriting sample collection sub-routine1300may proceed to execution block1323; otherwise handwriting sample collection sub-routine1300may continue to wait for a stroke identification response.

Handwriting sample collection sub-routine1300may instantiate a new code point sample value for the current code point of the handwriting sample text in the handwriting code-point set data structure at execution block1323.

At decision block1325, if multiple stroke segments were identified for the current code point during stroke identification, then handwriting sample collection sub-routine1300may proceed to starting loop block1328; otherwise handwriting sample collection sub-routine1300may proceed to execution block1338.

At starting loop block1328, handwriting sample collection sub-routine1300may process each identified stroke segment in turn.

Handwriting sample collection sub-routine1300may instantiate a new segment value for the current sample value in the handwriting code point data structure at execution block1330.

Handwriting sample collection sub-routine1300may identify digitized handwriting sample data corresponding to the current stroke segment at execution block1331.

Handwriting sample collection sub-routine1300may associate the identified digitized handwriting sample data with the new segment value for the current code point at execution block1333.

At ending loop block1335, handwriting sample collection sub-routine1300may loop back to starting loop block1328to process the next identified stroke segment, if any.

At execution block1338, handwriting sample collection sub-routine1300may identify digitized handwriting sample data corresponding to the identified stroke.

Handwriting sample collection sub-routine1300may associate the identified digitized handwriting sample data with the new code point sample value for the current code point at execution block1340.

At ending loop block1343, handwriting sample collection sub-routine1300may loop back to starting loop block1316to process the next code point in the handwriting sample text, if any.

At termination block1398(seeFIG. 13A), handwriting sample collection sub-routine1300may return a collection successful message at termination block1399.

Handwriting sample collection sub-routine1300may return a collection successful message at termination block1399.

CONCLUSION

Although specific embodiments have been illustrated and described herein, a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present disclosure. This application is intended to cover any adaptations or variations of the embodiments discussed herein.

Certain aspects of the present methods and systems may focus on computer implemented methods of obtaining digitized hand-writing data corresponding to a sample of a needed code point of a set of code points. Such methods may include: obtaining a sample of digitized handwritten text, the sample of digitized handwritten text including glyph data corresponding to a first glyph, the first glyph corresponding to the needed code point of the set of code points; associating the first glyph with the needed code point; identifying stroke data in the glyph data, the stroke data corresponding to a stroke component of the first glyph, determining a plurality of dimensional values of the stroke component in the stroke data; and associating the plurality of dimensional values with a new code point sample of the needed code point in a code point set data structure.

Other, not-mutually exclusive, aspects of the present methods and systems may focus on computer implemented methods of supplementing an incomplete handwriting code point data structure corresponding to a set of code points. Such methods may include: identifying a plurality of code points from the set of code points, the plurality of code points corresponding to code points in need of dimensional values for at least one code point sample; selecting a preferred handwriting sample phrase from a plurality of predefined handwriting sample phrases, the preferred handwriting sample phrase including at least a first code point of the plurality of code points; obtaining a sample of digitized handwritten text corresponding to the preferred handwriting sample phrase, the sample of digitized handwritten text including glyph data corresponding to a first glyph, the first glyph corresponding to the first code point; associating the first glyph with the first code point; identifying stroke data in the glyph data, the stroke data corresponding to a stroke component of the first glyph, determining a plurality of dimensional values of the stroke component in the stroke data; and associating the plurality of dimensional values with a new code point sample of the needed code point in the code point set data structure.