Methods and systems of creating a confidence map for fillable forms

A method of creating a confidence map for an electronic fillable form may include identifying, by an electronic device, one or more fillable fields of an electronic fillable form. The method may include, for one or more of the identified fillable fields, assigning one or more first coordinates of the electronic fillable form to define a graphical zone associated with the fillable field, assigning one or more second coordinates of the electronic fillable form to define a fill zone, assigning one or more third coordinates of the electronic fillable form to define a click zone, and assigning, by the electronic device, one or more fourth coordinates of the electronic fillable form to define a start zone representing a most likely selection area for the fillable field. The method may include creating, a confidence map associated with the electronic fillable form.

BACKGROUND

Paper and pencil/pen are convenient tools to use to complete forms or to edit or augment structured documents, and are often easier to use than a mouse or other input devices. However, electronic representations of these types of documents are often cumbersome to work with because the precision of the paper and pencil/pen interaction is difficult to achieve through the finger-screen interaction. As such, users often print forms, fill them in manually, and then rescan the completed document.

SUMMARY

This disclosure is not limited to the particular systems, methodologies or protocols described, as these may vary. The terminology used in this description is for the purpose of describing the particular versions or embodiments only, and is not intended to limit the scope.

As used in this document, the singular forms “a,” “an,” and “the” include plural reference unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art. All publications mentioned in this document are incorporated by reference. All sizes recited in this document are by way of example only, and the invention is not limited to structures having the specific sizes or dimension recited below. As used herein, the term “comprising” means “including, but not limited to.”

In an embodiment, a method of creating a confidence map for an electronic fillable form may include identifying, by an electronic device, one or more fillable fields of an electronic fillable form. Each fillable field may include one or more dimensions, and the electronic fillable form may include content. The method may include, for one or more of the identified fillable fields, assigning, by the electronic device, one or more first coordinates of the electronic fillable form to define a graphical zone associated with the fillable field based on the one or more dimensions, where the graphical zone encompasses the one or more identified fillable fields, assigning, by the electronic device, one or more second coordinates of the electronic fillable form to define a fill zone, where the fill zone includes an area that is a subset of the graphical zone, and no content of the electronic fillable form is present in the fill zone, assigning, by the electronic device, one or more third coordinates of the electronic fillable form to define a click zone, where the click zone represents a superset of the graphical zone, and assigning, by the electronic device, one or more fourth coordinates of the electronic fillable form to define a start zone representing a most likely selection area for the fillable field. The method may include creating, by the electronic device, a confidence map associated with the fillable form including an indication of the graphical zone, the fill zone, the click zone and the start zone for the one or more identified fillable fields, and storing the confidence map in a database such that the confidence map is associated with the electronic fillable form.

In an embodiment, a system of creating a confidence map for an electronic fillable form may include an electronic device and a computer-readable storage medium. The computer-readable storage medium includes one or more programming instructions that are configured to instruct the electronic device to identify one or more fillable fields of an electronic fillable form, where each fillable field includes one or more dimensions, wherein the electronic fillable form includes content. The computer-readable storage medium includes one or more programming instructions that are configured to instruct the electronic device to, for one or more of the identified fillable fields, assign one or more first coordinates of the electronic fillable form to define a graphical zone associated with the fillable field based on the one or more dimensions, where the graphical zone encompasses the identified fillable field, assign one or more second coordinates of the electronic fillable form to define a fill zone, where the fill zone includes an area that is a subset of the graphical zone, and no content of the electronic fillable form is present in the fill zone, assign one or more third coordinates of the electronic fillable form to define a click zone, where the click zone represents a superset of the graphical zone, and assign one or more fourth coordinates of the electronic fillable form to define a start zone representing a most likely selection area for the fillable field. The computer-readable storage medium includes one or more programming instructions that are configured to instruct the electronic device to create a confidence map associated with the electronic fillable form including an indication of the graphical zone, the fill zone, the click zone and the start zone for the one or more identified fillable fields, and store the confidence map in a database such that the confidence map is associated with the electronic fillable form.

DETAILED DESCRIPTION

The following terms shall have, for purposes of this application, the respective meanings set forth below:

A “computing device” or “electronic device” refers to a device that includes a processor and non-transitory, computer-readable memory. The memory may contain programming instructions that, when executed by the processor, cause the computing device to perform one or more operations according to the programming instructions. As used in this description, a “computing device” or “electronic device” may be a single device, or any number of devices having one or more processors that communicate with each other and share data and/or instructions. Examples of computing devices or electronic devices include, without limitation, personal computers, servers, mainframes, gaming systems, televisions, and portable electronic devices such as smartphones, personal digital assistants, cameras, tablet computers, laptop computers, media players and the like.

A “confidence map” refers to information pertaining to the location of one or more zones relative to a fillable form.

A “fillable field” refers to a portion of an electronic document or file in which information or data may be entered, selected, added or modified. Examples of fillable fields include, without limitation, a check box, a text field, and/or the like.

A “fillable form” or an “electronic fillable form” refers to an electronic document or file having one or more fillable fields.

An “input indicator” refers to a movable visual indicator identifying a point that will be affected by input from a user. For example, an input indicator may be a cursor showing where typed text will be inserted.

A “zone” refers to a set of coordinates that define an area of a fillable form.

As the popularity of mobile electronic devices such as, for instance, mobile phones and tablets, increases, so does the number of fillable forms being completed using such devices. However, this use brings with it a change in the ergonomics of the user interaction. For instance, tablet input is often done via direct touch with either a finger or a stylus, which are both rather coarse input tools which often do not reach the precision of a mouse pointer. Applying a multi-layer zone map to a fillable form may make inputting information to the fillable form easier and more efficient.

FIG. 1illustrates an example system of creating a confidence map for a fillable form according to an embodiment. As illustrated byFIG. 1, the system100may include one or more client electronic devices102a-N, a form electronic device104and a computer-readable storage medium106. One or more of the client electronic devices102a-N may be in communication with the form electronic device104via a communication network108. A communication network108may be a local area network (LAN), a wide area network (WAN), a mobile or cellular communication network, an extranet, an intranet, the Internet and/or the like. In various embodiments, a client electronic device102a-N may be a mobile electronic device such as, for example, a smartphone, a tablet and/or the like.

FIG. 2illustrates a flow chart of an example method of creating a confidence map for a fillable form according to an embodiment. As illustrated byFIG. 2, a processor may identify200a fillable form. A processor may identify200a fillable form by recognizing when a fillable form is opened or accessed by a user. For instance, a processor may recognize when a user accesses a fillable webpage, portable document format (PDF) document, and/or the like.

In an embodiment, a processor may identify202one or more fillable fields of the fillable form. A processor may create202a list of fillable fields by performing connected component analysis. Connected component analysis may identify one or more fillable fields of a fillable document. A fillable field may be any portion of a form in which information may be selected, added, or modified. For example, a fillable field may be a check box, a free text field and/or the like. In certain embodiments, information may be entered into a fillable form using one or more input devices such as, for example, a finger or a stylus on a touch screen display. For instance, a processor may cause one or more fillable fields to be displayed to a user via a display device associated with a client electronic device. A user may enter, select and/or modify information of a fillable field by selecting the fillable field or an area in or around the fillable field. In the case of a touch screen display device, a user may use his or her finger to touch the fillable field or an area surrounding the fillable field.

FIGS. 3A and 3Billustrate example fillable fields according to various embodiments. As illustrated byFIG. 3A, the box300surrounding the text “NAME”302may be considered a fillable field, while, as illustrated byFIG. 3B, the line304next to the text “NAME”306may be considered a fillable field.

Table 1 illustrates a listing of example identified fillable fields according to an embodiment.

As illustrated by Table 1, one or more fillable fields may be identified by coordinates, position, dimensions and/or the like. For example, as shown by Table 1, a fillable field may be identified by a coordinate pair (x, y value) and a delta x (Δx) and delta y (Δy) value. In various embodiments, a processor may determine204a graphical zone identified with one or more identified fillable fields. A graphical field may indicate a type associated with a fillable field. Example fillable fields may include, without limitation, a line, a checkbox, a box and/or the like.

In an embodiment, a processor may determine204a graphical zone identified with one or more fillable fields based on one or more dimensions of the fillable field. For example, a fillable field with a large delta x and a small delta y may be determined204to be a line fillable field. As another example, a fillable field having a delta x that is much greater than its delta y may be determined204to be a box fillable field. As a further example, a fillable field having a delta x that is substantially similar to its delta y may be determined204to be a checkbox fillable field. In various embodiments, determining204a graphical zone may include determining one or more indications of an area, a location or position of a graphical zone on a fillable form and/or one or more dimensions of a graphical zone, such as for example, one or more coordinates, widths, heights, lengths and/or other measurements. Additional and/or alternate fillable field types and categorizations may be used within the scope of this disclosure.

FIGS. 4A and 4Billustrate example graphical zones corresponding to the fillable fields illustrated inFIGS. 3A and 3B, respectively. As illustrated byFIG. 4A, a graphical zone400may fit inside the fillable field300. As illustrated byFIG. 4B, a graphical zone402may sit on top of the line fillable field304. In certain embodiments, the size of a fillable field may be greater that a size of a graphical zone which, in turn, may be greater than a size of a fillable area.

In certain embodiments, a graphical zone may be associated with a probability of being an acceptable graphical zone. An acceptable graphical zone may be one that is of a particular size, has a certain location or placement, and/or other characteristics. For example, one or more characteristics of a graphical zone, such as, for instance, placement, size, or dimensions may be based, at least in part, on characteristics of a fillable field such as, for instance, the location of static content. For example,FIGS. 5A, 5B and 5Cillustrate example graphical zones according to various embodiments. The graphical zone500illustrated inFIG. 5Amay have a high probability of being an acceptable graphical zone because it has a reasonable size and is substantially centered within the associated fillable field. The graphical zone502illustrated inFIG. 5Bmay have a lower probability of being an acceptable graphical zone because the graphical zone has a smaller size and height due to the placement of the static text504. Likewise, the graphical zone506illustrated inFIG. 5Cmay have an even lower probability of being an acceptable graphical zone because it has a smaller size.

In an embodiment, a processor may determine a new graphical zone if an originally determined graphical zone is associated with a probability that is below a threshold value. For instance, a processor may determine a new graphical zone if a determined graphical zone is associated with a 20% or less probability of being an acceptable graphical zone. Additional and/or alternate probability values may be used within the scope of this disclosure. In certain embodiments, a process may determine a new graphical zone until one having a probability of being an acceptable graphical zone exceeds a threshold value.

In an embodiment, a processor may determine206a fill zone associated with one more fillable fields. A fill zone is a subset of a graphical zone and represents an area which can be filled with content because it does not have any content present. In various embodiments, the dimensions of a fill zone may be smaller than the dimensions of the corresponding graphical zone as the fill zone is a subset of the graphical zone. In various embodiments, a fill zone may determine a location of an input indicator such as, for example, pointer, cursor and/or the like when the fillable field is selected. For instance, the beginning of a fill zone may be where a user may expect to see an input indicator when the user selects the corresponding fillable field.

In some embodiments, the position or location of a fill zone may depend on whether a corresponding graphical zone includes static content, and if so, where the static content is positioned. Static content may refer to content of a form that cannot be changed, modified or altered by a user. Static content may be in any form such as, for example, text, images, graphics, pictures, icons, symbols and/or the like. An example of static content may be a text label associated with a fillable field explaining what information a user should enter into the fillable field. For example, if a graphical zone includes static content in the top of the graphical zone, a processor may determine that a fill zone includes an area below the static content. As another example, if a graphical zone includes static content in a bottom portion of the graphical zone, a processor may determine that a fill zone includes an area above the static content. If a graphical zone includes static content in a right portion of the graphical zone, a processor may determine that a fill zone includes an area to the left of the static content. In various embodiments, determining206a fill zone may include determining one or more indications of a location or position of a fill zone on a fillable form and/or one or more dimensions of a graphical zone, such as for example, one or more coordinates, widths, heights, lengths and/or other measurements. Additional and/or alternate positioning determinations may be made within the scope of this disclosure.

FIG. 6illustrates example fill zones and graphical zones for three example fillable fields according to an embodiment. As shown inFIG. 6, graphical zone600does not include any static content, so the fill zone602has a height of a line as estimated. Graphical zone604however, has static content toward the top left corner of the graphical zone, so the corresponding fillable field606is located below the static content. Lastly, graphical zone608has static content in the lower left, so the corresponding fillable field610is located to the right.

Referring back toFIG. 2, in various embodiments, a processor may identify208a click zone associated with one or more fillable fields. A click zone is a superset of a graphical zone, and may include an area within which a user may be likely to click or select in order to enter information into a fillable field. A click zone may be considered to include the graphical zone plus an additional area representative of the imprecision of pointing or selecting. In an embodiment, a processor may determine a click zone by increasing the size of the graphical zone by a certain percentage or amount. For instance, when an input device is expected to be a finger or another imprecise input device, a processor may determine a click zone by increasing the size of the graphical zone by, for example 20%. Similarly, for zones that are being displayed on electronic devices having smaller displays such as, for instance, a mobile phone, a processor may increase the size of the graphical zone by at least a certain amount, such as, for example 20%, to establish a minimum size. Additional and/or alternate size increases may be used within the scope of this disclosure.

For example, a click zone may be an area around a graphical zone that includes 20% more area than the graphical zone. In various embodiments, determining208a click zone may include determining one or more indications of a location or position of a click zone on a fillable form and/or one or more dimensions of a graphical zone, such as for example, one or more coordinates, widths, heights, lengths and/or other measurements.

In an embodiment, a processor may identify210a start zone associated with one or more fillable fields. A start zone represents a most likely click area associated with a fillable field. For example, for English forms, a start zone of a fillable field may include a leftmost portion of the graphical zone because users will most likely read and write left to right. As another example, forms in Arabic or Hebrew may have a start zone that includes a rightmost portion of a graphical zone because users will most likely read and write right to left.

In various embodiments, a start zone may include a portion of a graphical zone and/or a fill zone. A start zone may also include at least a portion of an area that is not within the graphical zone or the fill zone. In an embodiment, a processor may identify210a start zone that is centered or substantially centered around a fill zone as this is the area with the highest likelihood of being selected by a user. In various embodiments, identifying210a start zone may include determining one or more indications of a location or position of a graphical zone on a fillable form and/or one or more dimensions of a start zone, such as for example, one or more coordinates, widths, heights, lengths and/or other measurements.FIG. 7illustrates three examples of start zones according to an embodiment. As illustrated byFIG. 7, the start zone700of the top example coincides with the beginning of the graphical zone702.

In the middle example, the start zone704is centered around the fill zone706. In the bottom example, the line height influences the start zone. For graphical zones that cover multiple lines, users tend to indicate a starting area toward the top of the area, so the start zone708includes an area that includes the beginning of the lines in the fill zone710.

In certain embodiments, the size, dimensions and/or positioning of one or more graphical zones, fill zones, start zones and/or click zones may vary depending on the display device on which such zones are being displayed. For instance, one or more zones may be displayed proportionally larger on an electronic device having a smaller display, for example, a mobile phone, than on an electronic device having a larger display, for example, a tablet.

In various embodiments, an electronic device may create212a confidence map. An electronic device may create212a confidence map by creating an electronic file that includes an indication of a position or location of a graphical zone, a fill zone, a click zone and/or a start zone such as, for example, one or more coordinates associated with one or more of these zones.

In an embodiment, the combination of the graphical zone, fill zone, click zone and start zone associated with a fillable field may comprise a confidence map for the fillable field. The confidence map may represent a likelihood that a click at location (x, y) should lead to an input indicator being positioned at an input location (m, n). Referring back toFIG. 2, in various embodiments, an electronic device may store214confidence map information associated with a confidence map, such as, for example, position or location information or one or more dimensions associated with one or more graphical zones, fill zones, click zones and/or start zones. An electronic device may store214confidence map information in a table, a list, a database or other similar data structure such that the confidence map information is associated with the particular fillable form to which it pertains.

In various embodiments, an electronic device may automatically apply a stored confidence map for a fillable form when the form is opened, viewed or otherwise accessed. FIG.8illustrates a flow chart of an example method of applying a confidence map to a fillable form according to an embodiment.

As illustrated byFIG. 8, an electronic device may receive800one or more instructions to access a fillable form. An electronic device may receive800one or more instructions to access a fillable form from a client electronic device. For instance, an electronic device may receive800one or more instructions to access a fillable form when a client electronic device receives an instruction from a user of the electronic device and/or a process running on the electronic device to open, view, or otherwise access a fillable form.

An electronic device may identify802the fillable form to be accessed. For example, an electronic device may identify802a fillable form to be accessed by name, uniform resource locator (URL), or other unique identifier. An electronic device may retrieve804a confidence map for the identified fillable form from storage. For instance, an electronic device may use a unique identifier associated with a fillable form to locate a confidence map associated with the fillable form from a database.

In an embodiment, an electronic device may cause806the identified fillable form to be displayed at a client electronic device, such as, for example, on a display of a client electronic device. An electronic device may cause808a confidence map to be applied to the fillable form. In some embodiments, an electronic device may cause808the confidence map to be applied to the fillable form by applying at least a portion of the confidence map as an overlay to the fillable form. The overlay may be invisible so the experience is seamless to a user.

In an embodiment, an electronic device may receive810a selection of the fillable form. A selection may correspond to a particular location on the fillable form, and a selection may be associated with one or more coordinates. For instance, a selection of a fillable form displayed by a touch screen display device may be made by touching the screen with a finger, a stylus and/or other input device. In some embodiments, a fillable field may correspond to a touch-sensitive portion of a display device. For instance, with respect to a touch screen, only the portions of the touch screen on which one or more fillable fields are displayed may be touch-enabled. Other portions of the fillable form being displayed by the display device may not be receptive to input via the touch screen.

In an embodiment, an electronic device may receive810a selection from a client electronic device. For instance, a user of a client electronic device may use an input device to make a selection of a fillable form, and the client electronic device may send information associated with the selection such as, for instance, the coordinates associated with the selection, to the electronic device.

An electronic device may determine812to which portion of the confidence map the selection corresponds. For instance, an electronic device may determine whether the coordinates of a selection are within a graphic zone, a click zone, a fill zone or a start zone for a particular fillable field.

FIGS. 9A and 9Billustrate an example confidence map applied to a portion of a fillable form according to an embodiment. As illustrated byFIG. 9A, the portion includes two checkboxes900,902. The checkbox900on the left has a corresponding fill zone912, click zone904and start zone906. Likewise, the checkbox902on the right has a corresponding fill zone914, click zone908and start zone910. As shown byFIG. 9A, there are several points of overlap over different zones. For instance, click zone904and start zone906overlap, click zone904and start zone910.FIG. 9Billustrates the confidence map shown inFIG. 9Aalong with several example selections.

Referring back toFIG. 8, if an electronic device determines812that a selection corresponds to only one of a graphical zone, a click zone, a fill zone or a start zone for a fillable field, the electronic device may cause814an input indicator to be displayed in the fill zone corresponding to the fillable field. For example, inFIG. 9B, selection A916and selection B918only correspond to one zone (selection A corresponds to start zone606and selection B corresponds to click zone908), so an electronic device may cause an input indicator to be displayed in the corresponding fill zones912,914(as illustrated by the arrows inFIG. 9B).

Referring back toFIG. 8, in an embodiment, if an electronic device determines812that a selection corresponds to two or more overlapping zones, the electronic device may determine816in which fill zone the input indicator should be displayed. In certain embodiments, one or more zones may have precedence over one or more other zones such that one zone is given more weight than one or more other zones. For instance, in an embodiment, the following precedence may be followed:start zone>=fill zone>=graphical zone>=click zone

This means that if a selection falls within the start zone of a first fillable field and the fill zone, graphical zone or click zone of a second fillable field, the selection will be associated with the first fillable field. Additional and/or alternate precedences may be used within the scope of this disclosure.

For example, inFIG. 9B, selection C920falls within both the click zone904of checkbox 1900and the start zone910of checkbox 2902. To determine in which fill zone of which checkbox to place the input indicator, an electronic device may analyze the one or more options in light of the applicable hierarchy. For instance, using the precedence above, an electronic device may determine that a start zone has a higher weight than a click zone, so the electronic device may cause an input indicator to be displayed in a fill zone associated with the start zone—in this case, the fill zone914associated with checkbox 2.

FIG. 10illustrates an example confidence map according to an embodiment. As illustrated byFIG. 10, selection D1000falls within a fill zone1002of a first fillable field1004and a start zone1006of a second fillable field1008. Given the above precedence, an electronic device may cause an input indicator to be positioned in the fill zone1010associated with the second fillable field1008. However, if, in response to causing an input indicator to be positioned in the fill zone, a second selection is received in the same location or proximate to the same location as the first selection, an electronic device may cause the input indicator to be repositioned in a fill zone associated with the zone having the next highest precedence. For instance, if, after causing the input indicator to be positioned in the fill zone1010associated with the second fillable field1008, an electronic device again receives a selection at or near selection D1000, the electronic device may reposition the input indicator to be displayed in the fill zone1002associated with the first fillable field1004.

FIG. 11depicts a block diagram of hardware that may be used to contain or implement program instructions. A bus1100serves as the main information highway interconnecting the other illustrated components of the hardware. CPU1105is the central processing unit of the system, performing calculations and logic operations required to execute a program. CPU1105, alone or in conjunction with one or more of the other elements disclosed inFIG. 11, is an example of a production device, computing device or processor as such terms are used within this disclosure. Read only memory (ROM)1110and random access memory (RAM)1115constitute examples of non-transitory computer-readable storage media.

A controller1120interfaces with one or more optional non-transitory computer-readable storage media1125to the system bus1100. These storage media1125may include, for example, an external or internal DVD drive, a CD ROM drive, a hard drive, flash memory, a USB drive or the like. As indicated previously, these various drives and controllers are optional devices.

Program instructions, software or interactive modules for providing the interface and performing any querying or analysis associated with one or more data sets may be stored in the ROM1110and/or the RAM1115. Optionally, the program instructions may be stored on a tangible, non-transitory computer-readable medium such as a compact disk, a digital disk, flash memory, a memory card, a USB drive, an optical disc storage medium and/or other recording medium.

The hardware may also include an interface1145which allows for receipt of data from input devices such as a keyboard1150or other input device1155such as a mouse, a joystick, a touch screen, a remote control, a pointing device, a video input device and/or an audio input device.

It will be appreciated that the various above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems or applications or combinations of systems and applications. Also that various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.