Patent ID: 12248468

DETAILED DESCRIPTION OF THE INVENTION

A robotic software program runs autonomously and continuously in the background collecting invoice data as it is predicted to be available from a target set of sites. This software program has the necessary credentials to access the systems if they are password protected on behalf of the account holders. In the case of API access, the appropriate keys or authentication requirements are met to allow such access. This program can also be triggered and run manually.

Invoice data is captured and collected through various means including the interfacing with billing systems and web interfaces showing usage data for an account or other invoice management systems.

Availability of invoices is determined by a known schedule for a supplier. These intervals can be weekly, bi-weekly, monthly, first of the month, first Monday of the month, or any number of predictable intervals.

It is known and expected however that the interval will not be exact, and the system will attempt to retrieve the system at the predicted time and will retry until it has been able to receive the data.

Getting the invoice depends on the target system from which the data is obtained. In some cases, the system may allow querying through an API for the availability of the invoice for the period we are looking for. In other cases, a file may exist that can be downloaded.

In the case of screen scraping, one may need to look for the expected target date on the screen (example, June Invoice), or we may simply have to look for an invoice different from the one obtained last month (Example Invoice 008). Many such combinations may exist, and machine learning is used to determine how to approach each system learning from the historical successes and failures of attempts to access the data.

In order to prevent security concerns, various methods are employed to optimize the success rate of obtaining the necessary data while minimizing the risks of flagging security protocols.a. A careful selection of when to initially attempt retrieval of the data is selected and made. This may coincide with the typical billing cycle such as a monthly invoice being available the 1stof the month, or the first Monday of the month for example.b. A retry frequency count is set to determine how often to retry the system before manual intervention is required. This manual intervention may involve both verifying the access to the system or contacting the vendor if the invoice has not been made available at the expected time and location.c. A retry interval is set to select how often to query the system between retries so as not to trigger a lockout from the system or raise alarms such as DoS attack or other hacking attempts. It is also an undue burden on servers to access and retry or download and interact with such systems constantly. Of larger concern is overcoming any potential lockout scenarios which require manual intervention.d. Alternating access requests. In systems that limit access from a particular address or site, alternating sites can be used to increase the frequency of retries without flagging security protocols.e. Careful recording of failure cause allows the system to distinguish between failure modes that are inherently transient and those that are unlikely to clear over time. The system recognizes transient failure modes and assigns an appropriate retry cadence. The system also adjusts and tunes itself based on the results obtained. For example, to determine the optimal time to get the initial invoice, it may move the retry a little earlier in the subsequent month, such as by one retry frequency interval to see if it can get the invoice earlier. If it is unable to do so, it will still allow for the system to get the invoice at the same time on the second retry.

How to automate and optimize the handling of any error conditions so as not to introduce additional delays is also required. For example, if there are authentication issues, these must be flagged and brought for manual intervention as soon as possible. A test access is performed prior to the expected availability of the invoice to test access to the system. For example, some systems require the periodic update and change of credentials on a regular schedule. This knowledge may be updated in the system or can be learned by the system to proactively request the changing of such credentials to avoid such manual intervention in future.

In the event that an invoice is not available after the retry threshold has been retried, manual interventions must also be triggered. In this case, it could be an issue on the retrieval system unrelated to credentials. This may include interfacing with the existing system, newly changed IT settings on the system preventing access, or any number of issues. The system can trace steps as to how far it went and what results it obtained to help diagnose any errors. The error could also be due to changes on the supplier side, or in some cases a simple delay in the availability of invoice data.

In the case of a lockout, an alarm is also raised triggering manual actions such as a person calling a support line to validate and unblock access to the system for future attempts. This event is fed back to the system so that the retry interval can be increased or retry count decreased to a new threshold that will no longer trigger the lockout.

Since the company running the invoice gathering system may have other customers with accounts accessing the same vendor system, these other accounts are also updated accordingly to avoid multiple lockouts.

Further, when invoices are available from a particular system, this information is also correlated with other requests for invoices for other customers from the same system thereby fast tracking getting the data.

Similarly, if a retry was made and the invoice was not available for a given customer, getting invoices for other customers in the same time period is also postponed. The system will also alternate retry attempts across multiple accounts, so that no individual customer gets locked out for multiple accesses. This may not work with systems that look at the originating IP address and still penalize or flag the system attempting to do so. In these cases, the company can use alternate IP addresses to access the system by running tasks across multiple locations or servers with differing public Ips.

A set of behavioral analytics is built up over time for vendor billing or invoice systems based on the experience of repeated access and characterization tests performed by the system. A company that obtains invoices for multiple customers is likely to have numerous accounts with the same vendors. Thus, access to the vendor system is made multiple times in a billing cycle as the individual customer accounts are each accessed. This allows for multiple access attempts to the same system to obtain knowledge about the system's behavior that can benefit all customers.

While the system is primarily interested in obtaining the invoice data as soon as it is available, the access attempts and the knowledge of how the vendor system interacts with these attempts is memorialized in a knowledge base of behavioral analytics which can be used for machine learning and continuous performance improvement.

In some cases, to build up this knowledge database, the system may attempt to access the vendor system even when invoice data is not expected to be available, simply to learn more about the system behavior.

One such example may be a system that periodically requires a change of password. Rather than run into an authentication failure when we expect the invoice to be available, we may attempt to access the system earlier just to validate that our credentials are still valid.

Another such example may simply be a new vendor system or a recent change to a known system which has modified the access strategy and expected behavior. In such cases, we may attempt to access the system simply to build up or update our knowledge base of how the system behaves.

Before accessing a particular vendor system on behalf of a customer, an access strategy is developed based on the known or expected behavior of the system. This strategy can include optimal timing, expected authentication challenges, and other expected behaviors or safeguards we expect the vendor system to put up which can help us avoid lockout scenarios.

In some cases, the systems expected behavior can be gleaned by simply recognizing the system or the software components used by the system such as the browser, the operating system, and the versions of software that interact with us.

In other cases, the system may be proprietary, and application specific safeguards and behavior may be implemented that must be learned.

In other cases, as a customer representative, data about the system behavior may simply be obtained by interacting with the vendor's IT department and manual programming of the optimal behavior into the system can be done so that the access strategy is acceptable to the vendor. This may be particularly true when changes or upgrades are going to take place. A notification from the Vendor can help prepare the system for the next communication attempt

Accessing the system can divulge detailed information about the environment including underlying operating systems, browsers, and firewalls or switches used in this access. This information can sometimes be obtained simply by pinging or viewing data returned or can be gleaned from error messages or observed behavior. Once established, known behavior of the identified components and the versions of these can be used to build a more effective access strategy.

Data about the system behavior may be obtained from knowledge from publicly available system documents about the vendor or the components used in their system. Interaction with the suppliers of these components is another strategy to obtain information about expected behavior.

Systems are rarely static, and as vendor systems go through upgrades and updates, the behavior must also be tested and updated. In some cases, new security concerns also instigate safeguards that may be temporary or permanent in nature. For example, an ongoing DoS (Denial of Service) attack that is propagating on the web may cause IT departments to temporarily restrict or limit access. In other cases, a newly discovered vulnerability on components used in the vendors system may force the vendor to adopt more stringent safeguards to protect the system from hackers attempting to exploit these vulnerabilities.

Available public and private databases of known vulnerabilities and exploits are also available to IT departments and systems involved in establishing security. By recognizing current and possibly ongoing threats and the established norms or guidance on how to counter these the system behavior is modified to adapt. Vendors may employ enhanced security protocols which may include transient changes to access strategies. These enhanced security protocols may confuse our benign access attempts with ongoing threats attempting to leverage newly exposed vulnerabilities.

These changes and adaptations to protocols are rarely advertised, even to valid customers, due to the nature of security systems in general but also due to the immediacy of the actions that are required to protect these systems.

Due to the many variables and the quickly changing nature of these variables the system applies machine learning techniques to develop the behavioral analytics as well as the access strategy used on the attempts to retrieve invoices.

With the advantage of having many accounts with the same vendor, the system may have the luxury of repeated learning across accounts that a single account holder would not have. The knowledge obtained can be characterized and tied to the vendor in a way that all customers with that vendor will benefit. Further, should a security issue arise, the system can adjust access to the vendor system by other customers with that same vendor, thus not having the same security issues for all customers with a particular vendor.

While the system is intended to run automatically in the background and look for invoices automatically, it is also possible to access the system through a console and run the analysis manually. While in a typical automatic mode, the system will run on the expected invoice availability date.

A system consisting of a robotic software program running on one or more computers works autonomously and continuously in the background collecting invoice data from vendor invoice systems allowing for the payment of invoices on behalf of end customers using these services.

In one embodiment, the system accesses the companies account information through a software portal provided by the carrier or service provider that supplies invoice information as per the invoice cycle.

In another embodiment, the system can interact with a vendor billing system using available API (application Programming Interfaces) to obtain detailed billing data on behalf of customers using said services.

In yet another embodiment, the system accesses files from a vendor billing system obtaining these in an electronic format such as Electronic Data Exchange (EDI), Comma Separate Values (CSV), and extensible markup language (XML). In yet another embodiment the system performs screen scraping or other information gathering techniques to obtain the data from a website or portal supplied by the vendor.

In yet another embodiment, the system creates a set of behavioral analytics which can be used to build an access strategy to known vendor systems. This data set is built up from both known characteristics about the systems and their components as well as knowledge obtained over time through access attempts. This data set includes protocol level data in terms of timing, safeguards employed, and versions of software involved. The system uses sophisticated machine learning techniques to build and leverage this dataset when accessing vendor systems.

In one configuration a system and method of determining when to initially attempt retrieval of the data is selected and made that coincides with the billing cycle.

In another configuration a system and method of accessing and exploring how the vendor system interacts with access attempts is used to build up knowledge about vendor systems to optimize access attempts.

The method further comprises the development of an access strategy which includes a retry frequency count to determine how often to retry the system as well as a retry interval to select how often to query the system between retries as well as information about safeguards and timing used by the vendor system.

The method still further comprises the step of alternating access requests with systems that limit access from a particular address or site, alternating public addresses.

The method still further comprises the steps of handling exceptions such as failed authentication and errors in capturing the data that include triggering manual intervention where required.

Other aspects and features of the present invention will become apparent from consideration of the following description taken in conjunction with the accompanying drawings.

Referring now to the drawings, wherein like reference numerals designate corresponding structure throughout the views. The following examples are presented to further illustrate and explain the present invention and should not be taken as limiting in any regard.

FIG.1shows the service provider computer1000with software2000executing thereon. The computer/software has access to read/write to storage3000which contains a number of data records including invoice data3002, authentication keys3004, update timing data3006and historical data3008. The invoice data3002will contain past invoices collected from the vendors. Typically these invoices will be associated with a specific customer of the service provider. This customer will have their associated authentication key3004which could be e.g. a user name and password or could be other types of authentication information needed in order to access the invoice data5000associated with the vendor computer4000(e.g. as part of the computer or on an external storage). The software2000will try to gather invoice data5000based on when an update is expected to occur, which would be indicated by the update timing data3006. This update timing data may borrow from invoice data3002which may indicate that, for example, invoices are run on the 15thof every month. But the update timing data3006may go a step further and know specifically when invoices are usually available at the earliest. For example, if the invoice generation4100of the vendor results in invoices being available as invoice data5000typically at 1:00 AM on the 15th, the software2000would use this more specific update timing data3006to issue a data request2002at, for example, 1:10 AM, thus gathering the invoice data5000reasonably close to the earliest possible time. However, the update timing data may also vary by authentication key3004. The vendor computer4000and the invoice generation4100would often be expected to behave the same way across multiple different customers in that most authentication keys3004whose invoices are run on the 15thof a given month would be expected to have their updated data available at the same time.

However, as data requests2002are made to query the vendor computer4000, the availability of a data update2004related to a particular request or the indication that an update is not available2006would be seen by the software2000and used to adjust the update timing data3006. This indication of when requests were/were not successful can be stored as historical data3008so that the software2000can use this data in order to adjust when requests are generated in the future. Further, when requests are unsuccessful, a re-querying process would begin in which case the update timing data3006would be used to determine when to next query the vendor computer4000. Thus, when the software2000determines when to make a data request2002, expected update timing data is used and this expected update timing data can include, for example, invoice data3002, update timing data3006and historical data3008. The data request2002will typically include the appropriate authentication key3004based on the expected update timing data indicative on an update being available. The request uses the authentication key3004to access the remote access portal4200which may be a web portal, API, FTP or other way of gathering invoice data5000.

In connection with the data request, if data is available, the data update2004will be sent to the service provider computer1000and that data will typically be stored as part of the invoice data3002for further processing and action. For certain ones of the requests, it is possible that an update may not be available2006and this would also be returned to the service provider computer for update to timing data3006and/or historical data3008. The software2000would then re-query the vendor computer using another data request2002at a later time using a frequency or other type of timing or spacing as determined based on when an update is expected to occur and what types of security restrictions4302may exist. It is also understood that security restrictions4302may be detected as a result of a data update2004being made available, but that this would typically occur as part of the re-querying process as a result of a failed attempt to obtain update data.

The security restriction can be a number of different actions taken by the vendor computer security system4300. These include a challenge question, captcha or even a lockout as some examples. In the challenge question/captcha example, these would be considered security restrictions if not normally used by the vendor computer security system4300. This could be known based on historical data3008such that during prior runs of data requests, if the vendor computer did not utilize a challenge question but this time a challenge question was used, this could indicate that the security system4300has started to flag the querying as potentially illicit. Typically this would occur when prior data updates were received without need to re-query because of unavailable data. However, the occurrence of a security restriction could indicate that the re-query spacing/frequency/timing was too close together for the security system4300. However, the security restriction may also be the result of an update to the vendor computer4000security system4300which simply requires that restriction each time, regardless of the timing or frequency of re-query. Over time as the system adjusts and responds to and tests security restrictions, it can be determined if the restriction is implemented as a result of the re-querying or if the restriction was simply implemented because that is how the security system4300has been updated.

Depending on how restrictive the security restriction is, the additional challenge question may or may not be deemed to require that the re-query frequency/timing/spacing is updated. Typically, the answers to challenge questions could be stored as part of the authentication keys3004and that the security restriction4302could trigger transmission of the challenge question answer as part of the data request2002. However, if instead of a challenge question, a captcha is provided, this may be deemed more restrictive because the computer will often not be able to get past the captcha without human intervention to respond to the captcha. In this case, the software2000would be programmed to try and avoid these more challenging security restrictions4302so that human intervention to answer the captcha challenge is not needed for a large number of the data requests2002. Since the service provider will often have many customers, each data request2002will often be customer specific, usually at least in terms of the authentication key3004used. Thus, if all querying and re-querying results in a captcha that requires human intervention, the collection process will be slowed significantly. At the same time, there is a need to balance obtaining the update data as close as possible to first availability. In some cases, being early to request and requiring re-querying is useful for a few of the authentication keys3004because this information on the availability of data for one key can be relevant to other keys. For example, the update timing data3006and/or historical data3008can be used by the software to determine the expected update timing based on behavior of the vendor computer4000as a result of queries using different authentication keys3004(i.e. associated with different customers). As a result, the system can continually test and react to availability of data to try to minimize the amount of time the invoice data5000is available from the vendor computer4000but not actually stored as part of invoice data3002on the service provider system.

Thus, some of the data requests2002can actually be implemented early intentionally as a way to test the security system4300of the vendor computer and verify that the querying schedule is done as close as possible to the earliest reasonable time. If adjustments are needed as are result of this testing, the update timing data3006will now indicate this information which can be used by the software2000in order to adjust the query schedule. It is understood that the data3000may be related to specific vendor computers such that the schedule/timing of querying/re-querying is different for different ones of the vendor computers. Typically, the querying of the same vendor computer but with different authentication keys will be done using the same schedule logic, but adjusted for the dates the invoices are run in that the behavior of the system would be expected to be the same but that the availability of data would be typically predicated initially on when invoices are run in the first place. It is understood that different authentication keys used for the same vendor computer will normally be associated with different customers or accounts.

Turning to the drawings,FIG.2shows a high-level invoice collection process. The system has access to customer account information1which may be an authentication key including login information and credentials for the customers being served. This invoice collection process3obtains data4via the network5from the many vendors that provide services to the customer.

A given vendor7has a specific invoice format which is returned6over the network5to4the invoice collection process3. Similarly, another vendor9has another specific invoice format which is returned8over the network5to4the invoice collection process3.

Finally, any number of different formats for different vendors may be obtained via this process, and yet another vendor11with a specific format sends10this invoice over the network5to4the invoice collection process3in the system.

Turning now toFIG.3we see mode details about the various invoice collection processes from the vendors. Customer account information1is retrieved2by the invoice collection process3. This customer account information1is required to access the customer data5000at the various storages28,31,34for the different vendors and likely includes destination addresses as well as login credentials. Once the invoice collection process has requested2the Customer account information1,3004, it requests2002,4over the network/cloud5from the various vendors the data by various methods.

Customer Invoice Data28is obtained from one vendor through a web portal27where this data is obtained via screen scraping29or downloading via image or pdf format. Customer Invoice Data31is also obtained36from another vendor through an FTP site30where a vendor keeps invoices31in an FTP folder for collection. In this case the invoice collection process3can access this data through the ftp site30and access it in the FTP folder31where the data32may be downloaded in various formats which may be CSV, EDI, PDF or other formats as provided by the vendor.

Customer Invoice Data34is obtained from yet another vendor through an invoice management system33. This data can be obtained by the invoice management system3via Exporting it into a file, doing a screen gram or possibly through application programming interface (API) commands35. All of these various types/formats of invoice data once obtained are stored by the vendor computer, for example as part of the invoice data3002. The data28,31,34may be considered portions of the invoice data5000described inFIG.1.

Turning now toFIG.4we see various invoice collection processes from vendors and a single vendor is depicted as an example. Customer account information1,3004is obtained42as well as an expected availability date. In the case of monthly billing, this date42may correspond to the day of the month that the invoice is expected to be available43. As previously described this expected availability timing can be obtained through, for example, analysis of invoice data3002, update timing data3006and/or historical data3008. If the invoice is expected to be available43then additional Vendor accessibility metrics44are verified which include how often a system can be queried before it flags a security issue or other lockout mechanism. These metrics44may be stored as part of the historical data3008indicative of, e.g. when security restrictions4302have been imposed as a result of various querying/re-querying2002.

If the vendor accessibility metrics44allow for the retrieval, the system will attempt to obtain45through the network5the invoice. First the vendor system is accessed47using the credentials or methods obtained from the customer account information1database.

If the system has locked out the process48then a manual reinstatement49may be needed. Such a lockout would then be compared to prior querying to obtain update data and the spacing/frequency/timing of querying/re-querying would be adjusted to avoid these lockouts requiring manual intervention. This manual intervention may involve calling the vendor to reenable the IP address or the account which is being used to capture the data. When resolved, vendor accessibility metrics50are updated at the service provider storage3000. Additionally, if the lockout was due to frequent attempts, the vendor accessibility metrics are updated50to retry less often. If the system is not locked out48it will access the system and determine if the invoice is available51. It is understood that the system will normally try to avoid these lockouts that require manual intervention, but that sometimes they are useful as testing markers for purposes of updating the data3000available to the software2000.

If the invoice is available51then the invoice is obtained52and the current billing period invoice collection is marked as done53indicating that there is no longer a need to attempt to collect the invoice for the given billing period with the particular authentication key for which data collection was successful in that the data update has been obtained.

The vendor availability metrics are also updated to show this information and can be used by the system as intelligence for the collection of other invoices. In the case of errors with the invoice52or the inability to collect after retries, the invoice can be flagged for manual processing to retry collection, and this can be done by simply marking the current monthly collection period53as not done. In the case of the invoice not being available at the time of invoice collection, the vendor availability metrics55are also updated to reflect the non-availability. This information is used to train the system for subsequent billing periods as well as provide knowledge for other customer invoices which the system may attempt to retrieve45,46,52from the same vendor. The system will also then retry obtaining the invoice from the vendor after the vendor accessibility metrics56reflect that it is safe to retry again.

Turning now toFIG.5we see how the system accumulates a rich set of behavioral analytics about vendor systems through knowledge built up through historical download activities. Once again, the system starts60by retrieving62customer account information or authentication keys which is required to access the vendor system68/4000applicable to the current customer. This data1includes credentials and authentication data required to access the system68as well as billing cycle data that includes the expected availability of invoices that may be obtained from the service provider storage3000.

The system then retrieves64vendor behavioral analytics data63which has been accumulated through experience accessing the vendor system68for both the current customer as well as other customers on that system. This analytics data63may be encompassed by the invoice data3002, update timing data3006and historical data3008. Further, the vendor system68may have known safeguards that manifest expected behavior learned or known from other similar systems and can be thus characterized or confirmed71. In a number of cases, the underlying customer may be new with authentication keys3004used for the first time by the service provider computer1000, but analytics data63for the vendor4000may be applied to this new customer and possibly specific invoice data3002from a prior customer invoice that indicates the invoice run date. Thus, prior historical experience with a particular vendor4000can be useful for running data updates for new customers from a known vendor.

With this information63an access strategy65is built to attempt access to the vendor system. This strategy65may be as simple as telling the system not to access the system at this time if a lockout or other security restriction is expected. It may also be set to access the system68even if the invoice data is not expected to be available to obtain71valuable characterization data63about the system which is used to inform the access strategy65with a different authentication key but using the same vendor.

With the access strategy65defined, if such strategy includes an attempt to access, the system attempts to access67the vendor system68through the network5. This access67will obtain information about the systems expected behavior. This may include timing data, response and challenge data, or other authentication or communications or protocol level insights.

If the vendor system's behavior matches the known behavior set-out in the access strategy65derived from the behavioral analytics63, then we simply confirm70that the behavior is consistent with a time stamp and update72the behavioral analytics63to reflect the most recent access. If the vendor system behaves differently69the system/software may attempt to characterize the system further71through additional tests or accesses to obtain and update the characterization data. This may repeat69/71until the software develops a known behavior or knowledge that the queries may be introducing additional risk if additional characterization tests71are done. Typically, some of the more valuable information about the characterization tests71include detection of a lower level security restriction4302, especially when it is known that there are higher level security restrictions4302that can be implemented. Knowledge of the timing and spacing of requests/queries and the various security restrictions implemented can inform the access strategy further in order to obtain updated data as fast as possible but without triggering onerous security restrictions.

The system then update72the vendor systems behavioral analytics63with any new information gleaned from the latest characterization tests and continue with the next scheduled invoice download event73/60. This update is typically stored in the service provider storage3000as part of the various data categories described therein (or as part of additional data categories). It is understood that althoughFIG.1shows four different data categories that these categories can be part of a single database or separated into two or more databases. Through repeated accesses and continued characterization and learning the vendor behavioral analytics database grows a rich set of data that improves the systems behavior and odds of success at obtaining data updates and doing so as early as reasonably possible.

Turning now toFIG.6we see an exemplary process automation flow for a robot tasked with retrieving invoices from a target system. The system unpacks the workload81which described the system information to access and details of the operation to be undertaken. The process is updated82to reflect that the given job is now in process, and83that the job has started navigation and84is attempting login.

When the login85is attempted, the results may result in showing an invalid URL90upon which the system returns a login failure93. The system may also find that the credentials used were invalid91upon which it returns a login failure94. Finally, other failures may occur, and a login failed92event will trigger the returning of values showing login fail95.

If login85is successful, the system will log the end of login86event and then start the robot navigation87to obtain in invoice information. If Navigation errors occur88, a navigation fail89message is returned. This may be triggered by unexpected screens, fields, or return values among other things.

If the navigation87proceeds as expected, the start account96event is logged and the system will attempt to find the account97. If this account is not found98the status is updated accordingly and a return value of account failure99is returned. If finding the account97was successful, the account found100message is logged and the system also logs that it is now finding data101.

The system now looks to find the expected invoice date102. If the invoice is not yet available for this date or not found, an update with date not found103is logged and a status of date not found104is returned. If the system does find the invoice for the given date102then a log message for date found105is created and the system logs that it is now starting the download106process.

Should this download process107for the invoice fail108a Download failure109event is sent back. Otherwise the invoice file is uploaded110from the system and the status of download successful111is logged and the process is complete returning a value of download complete112.

While the invention is susceptible to various modifications, and alternative forms, specific examples thereof have been shown in the drawings and are herein described in detail. Is should be understood however that the invention is not to be limited to the particular forms or methods disclosed but to the contrary, the invention is to cover all modifications, equivalents and alternatives falling within the scope of the disclosures and/or claims.