Patent Publication Number: US-2020301804-A1

Title: System Performance Monitor with Graphical User Interface

Description:
TECHNICAL FIELD 
     The inventions described herein are in the field of system performance monitors with graphical user interfaces. 
     BACKGROUND ART 
     There is a long felt need for a system performance monitor that graphically illustrates how quickly a given system, or subsystems within said given system, can perform certain tasks. The system can be a fully automated system, such as a computer-based system for routing messages on the Internet. The system may also comprise persons performing tasks, such as employees of a given company. Subsystems may comprise groups of automata, such as servers in a server farm. Subsystems may also comprise groups of persons, such as departments in an organization. Subsystems may include both automata and persons, such as groups of persons each working at a networked work station. 
     DISCLOSURE OF INVENTION 
     The disclosure of invention is provided as a guide to understanding the invention. It does not necessarily describe the most generic embodiment of the invention or the broadest range of alternative embodiments. 
       FIG. 1  is schematic of a system performance monitor  100  adapted to display the performance of a system  150  that carries out one more tasks (e.g. R 1 , R 2 , R 3 ). The system performance monitor may comprise:
         a) a computer implemented front end  110  comprising:
           i) a microprocessor;   ii) a permanent memory;   iii) an input device  111 ; and   iv) an output device  160 ;   
           b) a computer implemented back end  120 ; and   c) one or more computer implemented subsystem schema databases  140  each of which is associated with a subsystem schema.       

     The system  150  that carries out said one or more tasks is organized into one or more of said subsystem schemas  151  (e.g. A, B, and C). Each of the subsystem schemas comprises one or more members with a member ID (e.g. A 1 , A 2 , A 3 , B 1 , B 2 , B 3 , C 1 , C 2 , C 3 ). Each of the members carries out at least a portion of one or more of said tasks (e.g. R 1 , R z , R 3 ). 
     Each of the tasks has an associated task record  131  stored in a task database  130 . Each task record comprises:
         i) a task index  132 ;   ii) metadata  133  comprising one or more member IDs (e.g. A 1 , B 3 , C 2 ) of one or more members that carry out at least a portion of said task associated with said task record; and   iii) one or more event records (e.g.  134 ) each associated with an event wherein each event record comprises:
           1) an event type (e.g.  135 ) of said associated event; and   2) a time stamp (e.g.  136 ) of when said associated event occurred.   
               

     Each of the subsystem schema databases (e.g.  141 ) associated with a subsystem schema comprises:
         i) a member record (e.g.  147 ) associated with each member of said associated subsystem schema.       

     Each of the member records comprises:
         i) a member ID index (e.g.  142 ) associated with a member of said associated subsystem schema; and   ii) at least a portion (e.g.  143 ) of each task record associated with a task that was performed at least in part by said member associated with said member index.       

     The front end comprises computer readable instructions stored on said permanent memory of said front end, said computer readable instructions of said front end being operable to cause said microprocessor of said front end to physically carry out the steps:
         i) receive from a user via said input device, a selection  112  of one of said subsystems schemas (e.g.  115 ) and one of said members (e.g. A 3 ) of said selected subsystem schemas;   ii) call  114  said back end with said selected subsystem schema and selected member;   iii) receive via said back end from a subsystem schema database associated with said selected subsystem schema, a formatted member record  123  wherein said formatted member record comprises zero or more selected task records  144  associated with said selected member and zero or more non-selected task records  146  associated with said selected member; and   iv) format and display  161  on said output device, indicia (e.g.  164 ) of said selected task records.       

    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is schematic of a system performance monitor. 
         FIG. 2  is a time-cloud scatter plot displayed on an output device of said system performance monitor. 
     
    
    
     BEST MODE FOR CARRYING OUT THE INVENTION 
     The detailed description describes non-limiting exemplary embodiments. Any individual features may be combined with other features as required by different applications for at least the benefits described herein. As used herein, the term “about” means plus or minus 10% of a given value unless specifically indicated otherwise. 
     A portion of the disclosure of this patent document contains material to which a claim for copyright is made. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but reserves all other copyright rights whatsoever. 
     As used herein, a “computer-based system” comprises an input device (e.g. keyboard, touch screen, or electronic digital input from another device) for receiving data, an output device (e.g. printer, computer screen or digital connection to another device) for outputting data, a permanent digital memory for storing data, computer code or other digital instructions, and a digital processor for executing digital instructions wherein said digital instructions resident in said permanent memory will physically cause said digital processor to read-in data via said input device, process said data within said microprocessor and output said processed data via said output device. The digital processor may be a microprocessor. 
     As used herein, the term “shaped” means that an item has the overall appearance of a given shape even if there are minor variations from the pure form of said given shape. 
     As used herein, the term “generally” when referring to a shape means that an ordinary observer will perceive that an object has said shape even if there are minor variations from said shape. 
     As used herein, relative orientation terms, such as “up”, “down”, “top”, “bottom”, “left”, “right”, “vertical”, “horizontal”, “distal” and “proximal” are defined with respect to an initial presentation of an object and will continue to refer to the same portion of an object even if the object is subsequently presented with an alternative orientation, unless otherwise noted. 
     As used herein, disclosure of a singular implies disclosure of a plural and vice versa unless specifically indicated otherwise. 
     Referring again to  FIG. 1 , the computer readable instructions of the front end may further comprise the steps of:
         a) in response to receiving of a formatted member record, display a time-cloud scatter plot (e.g.  161 ) on the output device  160 . The time-cloud scatter plot may comprise:
           iv) an X-axis  162 ;   v) a Y-axis  163 ; and   vi) a data point indicium (e.g.  164 ) for each of the selected task records in the formatted member record.   
               

     The selected task records in said formatted member record may each comprise a start event with a time stamp and an end event with a time stamp. Each of the displayed data point indicia has an X-value based on said time stamp of said end event. Each of said displayed data point indicia has a Y-value based on said time stamp of said start event. 
     The X-axis may be horizontal. The Y-axis may be vertical. A surprising advantage of having the X-axis be horizontal is that a user can read the time-cloud graphic from left to right to see how the performance of the selected member has changed over time. Alternatively, the Y-axis may be horizontal and the X-axis may be vertical. An advantage of this orientation is that the horizontal axis now corresponds to when events started. 
     The displayed data point indicia may be semitransparent. Thus, when two datapoint indicia overlap (e.g.  166 ), the extent of the overlap will be visible. 
     The time-cloud scatter plot may have a first indicium line  165  that is located where Y equals X. The line represents the present. Any data point indicium on said line corresponds to a task that was completed immediately after it was initiated. 
     The time-cloud scatter plot may have a second indicium line  171  located where Y equals X plus an expected amount of time  173  between a start event and an end event for a given task. Thus, a user can view a time-cloud graphic and immediately perceive if tasks are being completed in the expected amount of time. If a data point indicium (e.g.  172 ) falls below the second indicium line, a user may want to investigate further to find out what the cause of the delay was. 
     The data point indicia may be interactive with a user. For example, each of said data point indicia may display metadata  167  associated with said associated task of a data point indicium upon activation of said associated data point indicium by said user. 
     The time-cloud scatter plot may further comprise a statistic  169  based on said selected task records of said selected member and said non-selected task records of said selected member. The statistic will help give context for the displayed data point indicia. For example, the displayed statistic may be the ratio of selected task records to total task records (i.e. selected plus non-selected). Thus, the user will be able to perceive if the number of displayed data point indicia is a lot or a little. 
     An indication  168  of the selected member may be presented in the time-cloud scatter plot so that the user will be able to show the time-cloud scatter plot to another viewer and the other viewer will know what member has been selected. 
     The member indication and statistics may also or alternatively be displayed outside of the time-cloud scatter plot, such as in a margin. 
     The system performance monitor may further comprise a computer implemented data munger  124  comprising:
         a) a microprocessor; and   b) a permanent memory comprising computer readable instructions to physically cause said microprocessor of said data munger to carry out the steps:
           i) read in one of said task records (e.g.  131 ) stored in said task database;   ii) create a member record for each of said members (e.g. A 1 , B 3 , C 2 ) in said metadata (e.g.  133 ) of said read-in task record; and   iii) store said created member records (e.g. A 1 :r 1 , B 3 :r 1 , C 2 :r 1 ) in said system schema databases.   
               

     The back end may be further configured to authenticate and route  121  the call  114  from the front end. The back end may be further configured to call  122  the appropriate subsystem schema database based on the routing request according to the selected subsystem schema. 
     The front end may be further configured to determine which subsystem schema (e.g.  115 ,  116 ,  117 ). 
     Example 1 
     The United States Patent and Trademark Office (USPTO) is a system that carries out one or more tasks. One of said tasks is the examination of a patent application. The examination is carried out at least in part by patent examiners. One of the subsystem schemas of the patent office is the technology class that is assigned to a patent application. The members of the patent class schema include the different classes (e.g. class  706 —Artificial Intelligence). Another subsystem schema is the art unit that a patent application is assigned to. The members of the art unit schema include the individual art units (e.g. AU  2121 ). Another subsystem schema is the patent examiners. The members of the patent examiner schema include the individual examiners. Other schemas may be contemplated such as a law firm schema, applicant schema or inventor schema. 
     Each patent application is assigned a serial number (e.g. Ser. No. 12/345,678). The USPTO keeps track of all events (e.g. the transaction history or file wrapper) that occur in the examination of a patent application. These events have an event type (e.g. “Non-Final Rejection”) and a time stamp (e.g. 05-05-2011 or mo-da-year). 
     The USPTO stores records for metadata and events associated with a patent application serial number in a publicly available database called the “Patent Examination Data System” (PEDS). The records in PEDS are indexed according to the serial number of the application. The metadata for each record includes subsystem labels such as a three-digit art class number for the class schema, a four-digit art unit number for the art unit schema and an alphabetic name for the examiner schema. The USPTO provides a search engine for selecting applications assigned to a particular class, art unit or examiner, but the elapsed time to retrieve a set of records for a given search might be quite long (e.g. 10 minutes or more for a large class). Furthermore, the time required to process said records after they are returned by the search engine to produce a time-cloud graphic can be very long. The data file for a single class might be 1 GB. The time required to process the data of a single class might be several hours on a single work station. There are about 700 members of the class schema. Processing the data for all 700 members has taken a matter of days on a large cloud-based data processing server. 
     To overcome this problem, a system performance monitor was developed according to the above description. The system performance monitor was generally cloud-based apart from a user&#39;s client device. The front end was hosted on a web server, Netlify.com. The user&#39;s client device connected to Netlify was considered part of the front end. The back end was hosted on an application server, amazon web services (aws.amazon.com). The subsystem schema databases were hosted on a cloud-based service, MongoDB Atlas (www.mongodb.com/cloud/atlas). A person of ordinary skill will recognize that alternative cloud-based services or dedicated resources such as mainframes or server farms could be used to host the front end, back end and subsystem schema databases. Thus, the inventions described herein are not limited to any particular computing platform. 
     A single input field was provided in the front end to receive the selection of a member of a subsystem schema from a user. A drop-down menu would display available members from all three schemas depending upon what the user typed in. The user could then select the desired member from the drop-down menu. Since the member names were unique to each subsystem schema, the selection of a member was enough for the back end to determine the selected subsystem schema and hence which subsystem schema database should be queried. In this example, if a selected member ID had an alphabetic character, then the selected subsystem schema was examiner. If the selected member ID was numeric with three digits, then the selected subsystem schema was class. If the selected member ID as numeric with four digits, then the selected subsystem schema was art unit. In this example, the front end merely forwarded the selected member ID to the back end and the back end determined the selected subsystem schema. 
     A data munger was written for the back end using Python programming language. The data munger read in the entire PEDS database (more than 1 TB). It then selected the metadata and events of interest for each application record, reformatted the data and stored the data in the subsystem schema databases. Three subsystem schema databases were set up. One for class, one for art unit, and one for examiner. A user, such as a registered US patent agent or attorney, would have interest in the performance of members of any of these schemas. Thus, a user could determine, for example, if a particular examiner&#39;s performance was in conformance to all of the examiners in a particular art unit. Similarly, the performance of different art units examining patent applications in the same class could be compared. A manager at the USPTO might be similarly interested, particularly if there were significant discrepancies between examiners, art units or classes that should otherwise have similar performance. 
     After filtering the PEDS data to include only the metadata (e.g. application serial number, title, class, art unit, examiner) and event data (e.g. non final rejection, final rejection, notice of allowance) of interest, the amount of data was reduced by 35×. However, since there were three subsystem schema databases, and each database had replicated data indexed according to the members of each subsystem schema, the net reduction of data was about 12×. Nonetheless, the delay between selecting a subsystem schema and selecting a member of said subsystem schema to when the time-cloud graphic was presented on the output device was only about 10 seconds or less, even for time cloud graphics that presented 100,000 data point indicia. 
       FIG. 2  is an exemplary time-cloud scatter plot  202  displayed on an output device  200  of said system performance monitor. The subsystem schema was class. The selected member of the class was class  706 —Artificial Intelligence. The selected records were the records of all applications in this class that had a notice of allowance. The non-selected records were the records for all applications that did not have a notice of allowance. Said non-selected records might be for patent applications that were abandoned or those that were still pending without a notice of allowance. If multiple notices of allowance were present in a record, both notices of allowance were selected and presented as individual data point indicia. 
     The Y-axis  231  was the filing date of a patent application. The X-axis  232  was the date of notice of allowance. Both axes start at Jan. 1, 2000 and end at the date the PEDS data was downloaded (i.e. Mar. 15, 2019). It is contemplated herein that after a first download of the entire PEDS database, subsequent downloads may only include records for applications that had new events since the prior download. The data from the subsequent downloads can be used to update the subsystem schema databases. 
     A first indicium line  223  was located at Y equals X. A second indicium line  224  was located at Y equals X plus three years. Three years is the maximum amount of time that it should take to get a notice of allowance according to US patent law assuming that the applicant has not otherwise delayed prosecution of the patent application. 
     A data point indicium (e.g.  211 ) is displayed on the time-cloud scatter plot for each of the selected records (i.e. all applications that had a notice of allowance). A total of about 11,000 data point indicia are presented. The data point indicia have a size of 3 points with a solid black fill and no border line. The transparency of the data points is 80%. Thus, single data point indicium (e.g.  212 ) are clearly visible while at the same time, high density areas (e.g.  213 ) can be discerned. 
     Upon selection of any data point indicium (e.g.  212 ) by the user, a popup  214  is displayed showing relevant metadata (e.g. serial number and title) of the application associated with the data point indicium. 
     The selected class (item  204 ) is displayed on the time-cloud scatter plot. In an alternative embodiment is was alternatively displayed in a margin of the time-cloud scatter plot. 
     The value (item  206 ) of a calculated statistic, APOA 12 , was also presented on the time-cloud scatter plot. APOA stands for “allowances per office action”. The “12” subscript indicates that the APOA was for office actions in the 12 months prior to the date the data was downloaded or last updated. The office actions included nonfinal rejections and final rejections from non-selected records, such as applications that did not have a notice of allowance. Thus APOA 12  gives a user an indication of how many office actions overall a user would have to respond to for all applications in a portfolio in order to get a single notice of allowance. An APOA 12  indicates that for the past 12 months, applicants had to reply to about 4 office actions in order to get a single notice of allowance. 
     Vertical grid lines  221  and horizontal grid lines  222  are presented on the time-cloud scatter plot so that a user can readily perceive when particular applicants where allowed and filed respectively. 
     A user viewed this time-cloud and noticed that the datapoint indicia thinned out  242  starting in the beginning of 2017. The user then created an alternative time-cloud graphic based on where applications with nonfinal or final office actions were represented by data point indicia. The user used the pop ups of several applications receiving rejections after 2017 to determine that many applications were receiving rejections based on 35 U.S.C. 101 when said applications had not received said rejections in the past. The user, therefore was discouraged from filing applications with an anticipated class  706  assigned to said applications. Upon more careful observation, however, the user observed that there was a sudden thickening  243  in the data point indicia of allowances in the first few months of 2019. In January 2019, the USPTO issued new guidance on the proper examination of patent applications under 35 U.S.C. 101. When the user examined the file wrappers of applications allowed in early 2019, said user observed that examiners in this class where allowing cases under the new guidance that they had previously rejected prior to January of 2019. Thus, user was encouraged to continue to file applications with this class. Users can observe both subtle and sudden changes on performance of different members of different subsystem schemas to more effectively draft and prosecute patent applications. 
     CONCLUSION 
     While the disclosure has been described with reference to one or more different exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the disclosure. In addition, many modifications may be made to adapt to a particular situation without departing from the essential scope or teachings thereof. Therefore, it is intended that the disclosure not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this invention.