Printer control automation from document annotation

Techniques to parse an electronic document to extract a comment and topological features of the electronic document to which the comment is attached, transform the comment into one or more printing command and one or more parameters of the printing command, generate a printed document from the electronic document, and apply the printing command and parameters of the printing command specifically to the topological features to which the comment is attached when generating the printed document.

BACKGROUND

Popular word processing and digital document manipulation tools, such as Portable Document Format (PDF) reader applications, enable users to insert comments into electronic documents. For instance, a user may select a section of text and right click to add a comment. In many cases the comments are “view only”, meaning they may not print on paper. Comments inserted into an electronic document may include instructions to be read by other users to make modifications to or process the electronic document. Current printing systems lack the capability to utilize comments to formulate machine control instructions and commands. Therefore a need exists for a printing system configured to transform and apply comments in electronic documents as printing commands.

BRIEF SUMMARY

Techniques for printer control automation from document annotations involves parsing an electronic document to extract a comment and topological features of the electronic document associated with the comment. The method analyzes the comment to identify a printing command. The method analyzes the comment to identify parameters of the printing command. The method operates a printer with the printing command and parameters of the printing command on topological features of the document attached to the comment.

A system for performing printer control automation from document annotations includes a parser and a processor. The parser extracts the comments and topological features of the electronic document associated with the comments. The processor is adapted to transform the comments into executable instructions for a print engine. The processor is adapted to determine parameters of the executable instructions from the comments and topological features of the electronic document to which the comments are attached. The processor applies the executable instructions and parameters to the print engine.

DETAILED DESCRIPTION

Techniques are described herein for printer control automation from document annotations. These techniques may extract comments and topological features from the electronic document. “Topological features” refers to any feature of a document lending the document visible structure. Topological features include document fields, headers, tables, blank areas (whitespace), horizontal and vertical lines, boxes, section markers, page markers, paragraphs, images, and so forth. “Comment” refers to any annotation added to a document to define operations to perform on content in the document. The comment may define operations on the document overall, or operations on specific topological features of the document. The comments may in some cases be analyzed to identify one or more of a file locator, a horizontal range, and a vertical range in a file identified by the file locator. A processing loop may then be iterated across discrete locations of the horizontal range and the vertical range. At each iteration a different version of the electronic document may be generated with a value or values from a different one or more of the discrete locations, at a location or locations associated with the topological features associated with the comment.

In some configurations, the file locator may be a uniform resource locator, and/or a document field to which the comment is attached. “Document field” refers to a defined topological area of a document. Document fields are often editable, meaning that they enable data input in the region by a user. Data entered in a document field is different than a comment; the former being actual content that is entered into the field, and the latter being an annotation or other meta-data describing the document or data in the document field and more particularly operations to perform on the document or document field. The file identified by the file locator may for example be a spreadsheet or delimiter-separated value file format that utilizes delimiters such as commas, tabs, and colons. “File” refers to a collection of stored signals (data values and/or instructions) containerized as a single manipulatable object in a computer system. Examples of these files types include comma-separated values (CSV) file and tab-separated values (TSV) file. The horizontal range may in some cases be a single column, while the vertical range may be a single row. The discrete locations may thus for example be cells of a spreadsheet.

In some configurations, the comment may be an annotation to the electronic document that is visually associated and/or linked with the topological features.

Systems are described for performing printer control automation from document annotations. Such systems may utilize a document annotator, a parser, and a processor. The document annotator (e.g., any document editing application) may be operated to add comments to an electronic document, and subsequently the parser operates to extract the comments and any topological features of the electronic document associated with the comments. The processor is adapted to perform several actions. In some instances, the processor analyzes the comments for a file locator, analyzes the comments for a horizontal range and a vertical range in a file identified by the file locator, and executes a loop command iterated across discrete locations of the horizontal range and the vertical range. In the loop, the processor generates at each iteration a different version of the electronic document and inserts into the different version a value (or values) from a different one of the discrete locations at a location associated with the topological features.

In other embodiments, printer control automation from document annotations is carried out by parsing an electronic document to extract a comment and topological features of the electronic document associated with the comment. The method analyzes the comment to identify a printing command. The method analyzes the comment to identify parameters of the printing command. The method operates a printer with the printing command and parameters of the printing command, specifically on content of the electronic document associated with the topological features.

The comment may in some instances include one or more images. The image may be an input to a neural network (e.g., a convolutional neural network classifier structure) to generate a classification of the comment. “Neural network” should be understood to have its conventional meaning in the arts as pertaining to any of a class of algorithms and structures for processing and classifying input vectors using multiple layers of artificial neurons. The comment may often include text. The text may be an input to a natural language processor (which itself may be and/or utilize a recurrent neural network, for example) to extract parameters of the printing command. “Natural language processor” should be understood to have its conventional meaning in the arts as pertaining to any of a class of algorithms and structures for manipulating and interpreting content encoded as natural human language. The printing command may be an executable instruction for a print engine of the printer. The printing command may encode or represent instructions to extract and email a portion of the electronic document associated with the topological features.

By way of example, the topological features may include a Tillable document field, and the method may be executed on a printer digital front end coupled to and controlling the printer. “Digital front end” refers to a computer system that operates to apply settings to a printer to carry out document processing. The digital front end and printer may be integral or may be separate devices.

In one use case, a sequential list of comments may be linked to (attached) or otherwise associated with one or more features of a document, e.g., a field, table, header, footer, title, and so on. The sequential list of comments may comprise a comment thread in one embodiment. A set of actions it performed for each comment of the comments, for example in an order from a beginning of the sequential list to an end of the sequential list. The comment may be analyzed to identify a printing command and parameters of the printing command. A printer is operated with the printing command and the parameters of the printing command to modify the feature of the document. In some cases the sequential list of comments may be re-sequenced to resolve dependencies or conflicts between printing commands specified in the comments. In some cases, the comments may be resolved (marked as resolved or closed) in the document, on condition that execution of the printing command is successful.

By way of example, a system for performing printer control automation from document annotations may include a parser and a processor. The parser (which may be executable instructions applied to the processor) is configured to extract the comments and topological features of the electronic document associated with the comments. The processor is further configured to transform the comments into executable instructions for a print engine and to determine parameters of the executable instructions from the comments and topological features of the electronic document to which the comments are attached or otherwise associated. The processor applies the executable instructions and parameters to the print engine.

Comment data from an electronic document such as a PDF may be interpreted using a machine learning system. The machine learning system may be trained utilizing user input data from a printer database. The user input data may be natural language strings. Samples of the training data provided to the machine learning system may embody the following instructions to the print engine or a human operator of the printer:Activate the use of comments as instructions for printing.Alert the operator of instructions for printing the document.Blank out a document feature/section with a horizontal line of a particular color.Execute a punch hole of the document.Add water mark to a location/feature/section linked to or associated with the comment.Download a document (e.g., from a local area network server or a web server) and perform VDP (variable data printing) using identified column(s) and/or row(s) from structured fields in the document.Send a copy of one or more document features/sections as text and/or image via email to a provided communication address.Replace or fill a document feature/field with date and time.Print associated feature/section using a particular color.Delete document feature/section.Replace document feature/section.

Henceforth, a reference to a document “feature” should be understood to include any sub-region of a document—field(s), section(s), page(s), table(s), header(s), footer(s), image(s), style(s), and so on. Comments should be understood to include text, images, emojis, and so on in any combination (any meta-layer of content linked/associated with document features, provided as a meta- or overlay layer to the document and distinct from the document content to print. The comments may be encoded as natural language, XJDF, Postscript, JSON, and other encodings known in the art.

FIG. 1depicts a printing system100in accordance with one embodiment. The printing system100comprises a printer102, a document annotator104, and an electronic document116. The printer102comprises a print engine108and a digital front end106. The digital front end106comprises a parser110and a processor112. The processor112may be configured to operate as a natural language processor114.

In the printing system100, the document annotator104adds comments118to the electronic document116. The parser110extracts the comments118and topological features122of the electronic document116associated with the comments118. The processor112is adapted to analyze the comments118for a file locator126. The processor112analyzes the comments118for a horizontal range130and a vertical range132in a file128identified by the file locator126. The processor112executes a loop160iterated across discrete locations (discrete location134, discrete location136, discrete location138) of the horizontal range130and the vertical range132. During each iteration, different versions154of the electronic document152are generated. During each iteration a value (value140, value142, and value144) from a different one of the discrete locations (discrete location134, discrete location136, and discrete location138), are inserted into a different versions154of the electronic document152at a location (document field124) associated with the topological features122associated with the comments118.

In some configurations, the file locator126is a uniform resource locator.

In some configurations, the topological features122may include a field to which the comments118is attached (linked or otherwise associated with the topological features122by the document annotator104).

In some configurations, the file128may be a spreadsheet and the horizontal range130may be a column or columns while the vertical range132may be a row or rows. In this configuration, the discrete locations (discrete location134, discrete location136, and discrete location138) are cells of the spreadsheet.

In some configurations, the comments118are annotations to the electronic document that are visually associated with the topological features122.

The printing system100may be operated such that, the electronic document116is parsed to extract at least one comment (comments118) and topological features122of the electronic document116associated with the at least one comment. The comments118are analyzed to identify a printing command146. The comments118are analyzed to identify parameters of the printing command148. The printer102is then operated with the printing command146and the parameters of the printing command148on content of the document associated with the topological features122.

In this operation of the printing system100, the parser110(e.g., the processor112configured with instructions to operate as a document parser) extracts the comments118and topological features122of the electronic document116associated with the comments118. The processor112may be adapted to transform the comments118into executable instructions150for the print engine108. The processor may determine parameters of the printing command148of the executable instructions150from the comments118and topological features122of the electronic document116to which the comments118are attached. The processor may apply the executable instructions150and parameters of the printing command148to the print engine108.

In some configurations, at least one comment (comments118) comprises an image120. The image120is input to a neural network156(e.g., a convolutional neural network trained (configured) as an image perceptron) to generate a classification of the comment (comments118).

In a common configuration, the comments118may include text. The processor may be configured as a natural language processor114that utilizes a semantic map158. Techniques for utilizing semantic maps with natural language processors are known in the art. The text may be an input to a natural language processor114to extract parameters of the printing command148.

The printing command146comprises one or more executable instructions150for a print engine108of the printer102. The printing command146may in some cases instruct a digital front end to extract and email a portion of the electronic document116associated with the topological features122. The topological features122may include a Tillable document field124.

In some configurations, the printing system100may operate on a sequential list of comments in a document. The printing system100may analyze each comment in the sequential list of comments to identify one or more printing command146. The printing system100may analyze each said comment to identify parameters of the printing command148. The printing system100may operate the printer102(via the print engine108) with the printing command146and parameters of the printing command148on the feature of the document to which the sequential list of comments is attached. This set of operations may be performed for each comment of the comments, in an order from a beginning of the sequential list to an end of the sequential list. The sequential list of comments may be re-sequenced to resolve dependencies or conflicts between printing command146specified in the comments118. That is, the printing system100may analyze the sequential list of comments as a whole to identify dependencies between commands encoded in the sequential list of comments, and may re-order the commands encoded in the sequential list of comments to satisfy these dependencies. The comments118may be resolved (marked as complete, deleted, or marked “resolved”) in the document on condition of successful execution of the printing command146.

The printing system100may be operated for example in accordance with the printing process200a, printing process200b, and electronic document generation process200cdescribed inFIG. 2A,FIG. 2B, andFIG. 2C, respectively.

FIG. 2Adepicts a printing process200ain accordance with one embodiment. In block202, the printing process200aparses an electronic document to extract a comment and topological features of the electronic document associated with the comment. In block204, the printing process200aanalyzes the comment to identify a printing command. In block206, the printing process200aanalyzes the comment to identify parameters of the printing command. In block208, the printing process200aoperates a printer with the printing command and parameters of the printing command on content of the document associated with the topological features, i.e., specifically to the topological features to which the comment is attached.

FIG. 2Bdepicts a printing process200bin accordance with another embodiment. In block210, the printing process200battaches a sequential list of comments to feature of a document. In opening loop block212, the printing process200bperforms the actions of block214and block216for each comment of the comments, in an order from a beginning of the sequential list to an end of the sequential list (potentially also re-ordering the commands encoded in the comments to satisfy dependencies).

In block214, the printing process200banalyzes the comment to identify a printing command. In block216, the printing process200banalyzes the comment to identify parameters of the printing command. In block218, the printing process200b(optionally) re-sequences the sequential list of comments to resolve dependencies or conflicts between printing commands specified in the comments.

In opening loop block220, the printing process200bperforms actions of block222and block224for each comment of the comments, in an order from a beginning of the sequential list to an end of the sequential list. In block222, the printing process200boperates a printer with the printing command and parameters of the printing command on the feature of the document. In block224, the printing process200b, on condition of successful execution of the printing command, resolves the comment(s) in the document. In closing loop block226, the printing process200bcompletes performing the actions of block222and block224when the iteration reaches the final comment/command in the sequential list.

FIG. 2Cdepicts an electronic document generation process200cin accordance with one embodiment. In block228, the electronic document generation process200cparses an electronic document to extract a comment and topological features of the electronic document associated with the comment. In block230, the electronic document generation process200canalyzes the comment to identify a file locator. In block232, the electronic document generation process200canalyzes the comment to identify a horizontal and vertical range of values in a file identified by the file locator. In opening loop block234, the electronic document generation process200cexecutes a loop iterated across discrete value locations of the horizontal and vertical range performing the actions of block236and block238. In block236, the electronic document generation process200cmay generate at each iteration a different version of the electronic document. In block238, the electronic document generation process200cinserts at each iteration a value from a different one of the discrete value locations into the different version of the electronic document at a location in the different version associated with the topological features of the electronic document associated with the comment.

FIG. 3Adepicts a comment-annotated document300ain accordance with the one embodiment. The comment-annotated document300aas depicted is an electronic document302opened in a document annotator304. The document annotator304identifies fields (document field314, document field316, and document field318) to which comments (comment308, comment310, and comment312) are attached. In the comment-annotated document300a, the electronic document302is a PDF document with the file name “KDDA 2020 Holiday Schedule.pdf”. The document annotator304has been operated to add four comments (comment306, comment308, comment310, and comment312).

The comment306in this example may be “Download the csv file from http://kyocera.com/employeeaddress.csv and use column 1, row 1 to row 100 for the replacement data” where the “http://kyocera.com/employeeaddress.csv” serves as the file locator for the file and the “use column 1, row 1 to row 100” are the range of discrete locations from the file to be used. The system recognizes that the phrasing “row 1 to row 100” identifies a printing command such that for each copy of the printed document, a discrete value from row 1 is to be inserted (the employee address, in this case) at the location to which comment306is attached, up to 100 distinct copies (e.g., one copy per employee).

The comment308on document field320in this example may be “Print in red”, which transforms into a printer command indicating the title text “Kyocera Document Solutions Development America”, the topological feature to which comment308is attached, is to be printed in red ink on all copies.

The comment310in this example may be “Download the csv file from http://kyocera.com/holiday2020.csv and use column 1 for the replacement data” where the “http://kyocera.com/holidayschedule.csv” serves as the file locator for the file and “use column 1 for the replacement data” transforms into a printer command to insert the entire contents of column 1 (the holiday dates) in the document at the location to which comment310is attached. The comment-annotated document300ais depicted as it would appear in printed form, with the holiday dates inserted.

The comment312in this example reads “Download the csv file from http://kyocera.com/holiday2020.csv and use column 2 for the replacement data” where the “http://kyocera.com/holidayschedule.csv” serves as the file locator for the file and the “use column 2 for the replacement data” transforms into a printer command to insert the entire contents of column 2 (the holiday names) in the document at the location to which comment312is attached. The comment-annotated document300ais depicted as it would appear in printed form, with the holiday names inserted.

When the electronic document302is sent to a printing system, the printer (or, for example, a digital front end of the printer) may identify the comments, apply parsing and natural language processing, and potentially also perceptron or other machine learning model processing (e.g., if the comments comprise images or emojis) to transform the comments into printing commands and parameters of the printing commands. The printer may parse one or more pages of the electronic document302to extract the comments (comment306, comment308, comment310, and comment312). The printer may access the employeeaddress.csv and holidayschedule.csv files, and determine that multiple copies of the document may be generated each customized with data from those files. An example of an employeeaddress.csv is the file with discrete locations for values300bdepicted inFIG. 3B, and an example of a holidayschedule.csv file with discrete locations for values300cis depicted inFIG. 3C.

FIG. 3Bdepicts a file with discrete locations for values300bin accordance with one embodiment. The file with discrete locations for values300bis a file titled “employeeaddress.csv” and comprises values in the first column comprising first address322, second address324, third address326and nth addresses that may be utilized by a printing system as replacement data inserted at a location or topological feature(s) of an attached comment comprising an identifier of the file.

FIG. 3Cdepicts a file with discrete locations for values300cin accordance with one embodiment. The file with discrete locations for values300cis a file titled “holiday2020.csv” and comprises discrete locations with values in the first and second column, e.g., value328and value330. The first column comprises holiday dates and the second column comprises holiday names. These may be utilized by a printing system as replacement data inserted at a location or topological feature(s) of an attached comment comprising an identifier of the file.

Machine Embodiments

FIG. 4depicts a diagrammatic representation of a digital front end400in the form of a computer system within which logic may be implemented to perform aspects of the techniques disclosed herein, according to an example embodiment.

Specifically,FIG. 4depicts a digital front end400comprising instructions408(e.g., a program, an application, an applet, an app, or other executable code) for causing the digital front end400to perform any one or more of the functions or methods discussed herein. For example the instructions408may cause the digital front end400to carry out aspects of the printing process200a, printing process200b, and electronic document generation process200c. The instructions408configure a general, non-programmed machine into a particular digital front end400programmed to carry out said functions and/or methods. It will be appreciated that a printer itself may be configured similarly to carry out the processes.

In alternative embodiments, the digital front end400operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the digital front end400may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The digital front end400may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions408, sequentially or otherwise, that specify actions to be taken by the digital front end400. Further, while only a single digital front end400is depicted, the term “machine” shall also be taken to include a collection of machines that individually or jointly execute the instructions408to perform any one or more of the methodologies or subsets thereof discussed herein.

The digital front end400may include processors402, memory404, and I/O components442, which may be configured to communicate with each other such as via one or more bus444. In an example embodiment, the processors402(e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, one or more processor (e.g., processor406and processor410) to execute the instructions408. The term “processor” is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. AlthoughFIG. 4depicts multiple processors402, the digital front end400may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

The memory404may include one or more of a main memory412, a static memory414, and a storage unit416, each accessible to the processors402such as via the bus444. The main memory412, the static memory414, and storage unit416may be utilized, individually or in combination, to store the instructions408embodying any one or more of the functionalities described herein. The instructions408may reside, completely or partially, within the main memory412, within the static memory414, within a machine-readable medium418within the storage unit416, within at least one of the processors402(e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the digital front end400.

Communication may be implemented using a wide variety of technologies. The I/O components442may include communication components440operable to couple the digital front end400to a network420or devices422via a coupling424and a coupling426, respectively. For example, the communication components440may include a network interface component or another suitable device to interface with the network420. In further examples, the communication components440may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), WiFi® components, and other communication components to provide communication via other modalities. The devices422may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

Instruction and Data Storage Medium Embodiments

The various memories (i.e., memory404, main memory412, static memory414, and/or memory of the processors402) and/or storage unit416may store one or more sets of instructions and data structures (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions408), when executed by processors402, cause various operations to implement the disclosed embodiments.

Communication Network Embodiments

The instructions408and/or data generated by or received and processed by the instructions408may be transmitted or received over the network420using a transmission medium via a network interface device (e.g., a network interface component included in the communication components440) and utilizing any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions408may be transmitted or received using a transmission medium via the coupling426(e.g., a peer-to-peer coupling) to the devices422. The terms “transmission medium” and “signal medium” mean the same thing and may be used interchangeably in this disclosure. The terms “transmission medium” and “signal medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions408for execution by the digital front end400, and/or data generated by execution of the instructions408, and/or data to be operated on during execution of the instructions408, and includes digital or analog communications signals or other intangible media to facilitate communication of such software. Hence, the terms “transmission medium” and “signal medium” shall be taken to include any form of modulated data signal, carrier wave, and so forth. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a matter as to encode information in the signal.

Various functional operations described herein may be implemented in logic that is referred to using a noun or noun phrase reflecting said operation or function. For example, an association operation may be carried out by an “associator” or “correlator”. Likewise, switching may be carried out by a “switch”, selection by a “selector”, and so on. “Logic” refers to any set of one or more components configured to implement functionality in a machine. Logic includes machine memories configured with instructions that when executed by a machine processor cause the machine to carry out specified functionality; discrete or integrated circuits configured to carry out the specified functionality; and machine/device/computer storage media configured with instructions that when executed by a machine processor cause the machine to carry out specified functionality. Logic specifically excludes software per se, signal media, and transmission media.

Having thus described illustrative embodiments in detail, it will be apparent that modifications and variations are possible without departing from the scope of the invention as claimed. The scope of inventive subject matter is not limited to the depicted embodiments but is rather set forth in the following Claims.