Patent Description:
Remote access protocols allow a human operator to remotely interact with an application running on a remote server. Remote interaction is achieved by such protocols by sending the user interface (UI) of the application from the remote server to the human operator's computer. Using the received Ul, the human operator is able to interpret the actions that the application can perform and the controls that are available in the application. The human operator is then able to decide which actions should be performed by the application and input the appropriate control signals into their computer. These control signals are subsequently sent from the human operator's computer to the remote server, where the desired actions are performed.

There are several known remote access protocols. One example is the remote framebuffer (RFB) protocol, which sends rendered graphical user interface (GUI) images from the remote server to the human operator's computer. A second example of a remote access protocol is the remote desktop protocol (RDP) by Microsoft. RDP is similar to the RFB protocol except that, instead of sending the rendered GUI image from the remote server to the human operator's computer, data for rendering the GUI image are sent.

A problem with these remote access protocols is that the UI received from the remote server to the human operator's computer can include sensitive data (i.e. personal information, financial information, medical information). Sensitive data received at the human operator's computer are problematic if output to a human operator because they may not be authorised to view the sensitive data. Further, there is a risk of a data breach via the human operator. Data breaches of sensitive data can result in the accidental or unlawful destruction, loss, alternation, unauthorised disclosure of, or access to, sensitive data, leading to significant human consequences. Moreover, access by a human operator to the sensitive data may be unlawful under the regulations of various jurisdictions. Therefore, a method for preventing sensitive data from being output to the human operator is desirable.

Document <CIT> describes a customer support application that provides screen sharing of the user's computing device with a remote customer support agent, thereby enabling the customer support agent to view the content displayed on the user's device. Sensitive information that is displayed on a user's computing device is obfuscated from the computing device of the remote customer support agent, and a notification of that obfuscation is displayed on the user's computing device. Information can be determined to be sensitive based on a sensitive indicator tag or a heuristic.

The present invention is defined by the independent claims, with further optional features being defined by the dependent claims.

In a first aspect of the invention, there is provided a computer-implemented method of preventing sensitive data received via a remote access protocol from being output to a human operator, the method comprising: receiving, from a remote server via a remote access protocol, a user interface of an application executing on the remote server; determining, or receiving from the remote server, a text-based representation of the application; identifying sensitive data in the user interface and/or the text-based representation of the application using one or more preselected filters; modifying the user interface to remove the identified sensitive data; outputting the modified user interface to one or more output peripherals of a computer for receipt by the human operator. In this way, sensitive data is prevented from being output to a human operator using the UI of the application and/or a text-based representation of the application.

In one embodiment, the remote access protocol is a remote framebuffer protocol. For example, the remote server may be a virtual network computing (VNC) server, and the computer may be a VNC client. In this embodiment, the received user interface is in the form of an image of the user interface, the step of modifying comprising modifying the image of the user interface, and the step of outputting comprising outputting the modified image of the user interface. Thus, the method can be used with the remote framebuffer protocol.

In another embodiment, the remote access protocol is a remote desktop protocol. In this embodiment, the received user interface is in the form of data for rendering an image of the user interface, the step of modifying comprising modifying the data for rendering an image of the user interface, and the step of outputting comprises outputting the modified data to render an image of a modified user interface. Thus, the method can be used with the remote desktop protocol.

The text-based representation of the application may comprise text extracted from the user interface and/or the application. Additionally or alternatively, the text-based representation of the application may comprise a programmatic representation of the application. In some embodiments, the programmatic representation comprises a document object model of the application. The text-based representation of the application is used to identify additional sensitive data that may not have been identified using the user interface of the application. A particular advantage of using a text-based representation of the application in particular is that information underlying the application can be utilised for identifying sensitive data, meaning that sensitive data is identified more accurately.

When the sensitive data is in the form of text, the preselected filter comprises a rule, for example a regular expression matching rule. Alternatively, the preselected filter comprises a machine learning model that is configured to identify text. These alternatives both allow sensitive data in the form of text to be identified and therefore prevented from being output to the human operator.

When the sensitive data is in the form of an image, the preselected filter comprises a machine learning model that is configured to identify an image. This allows sensitive data in the form of an image to be identified and therefore prevented from being output to the human operator.

The method may further comprise, prior to the step of determining, or receiving from the remote server, preselecting one or more filters based on sensitive data in the application. The sensitive data may include personal information such as one or more of: a name, an address, a date of birth, a telephone number, an identification document image, a facial image. Additionally or alternatively, the sensitive data may include financial information such as one or more of: credit card numbers, bank numbers. Additionally or alternatively, the sensitive data may include medical information. This allows the one or more filters to be customised based on the type of sensitive data expected to be in the application.

The modified user interface may be configured for interaction with the human operator via one or more input peripherals of the computer. This is so that the human operator can input controls into the remote server via the remote access protocol. The unmodified user interface is not output to the to one or more output peripherals of the computer in order to ensure that the sensitive data cannot be leaked by the human operator.

Typically for the remote framebuffer protocol and the remote desktop protocol, the user interface is a graphical user interface. Moreover, the step of receiving a user interface comprises receiving a user interface of a desktop of the remote server, the user interface of the application forming a portion of the user interface of the desktop. The application may be a web browser, although any type of desktop application is suitable for use with the method.

In a second aspect of the invention, there is provided a computer program comprising instructions which, when the program is executed by a processor, cause the processor to carry out the method of the first aspect of the invention.

In a third aspect of the invention, there is provided a computer-readable medium comprising instructions which, when executed by a processor, cause the processor to carry out the method of the first aspect of the invention.

In a fourth aspect of the invention, there is provided a processor configured to perform the method of the first aspect of the invention.

In a fifth aspect of the invention, there is provided a computing system comprising: a processor configured to perform the method of the first aspect of the invention; a remote server communicatively coupled to the processor; and a computer communicatively coupled to the processor and the remote server, the computer having one or more output peripherals configured to output the modified user interface and one or more input peripherals for receipt by a human user, wherein the modified user interface is configured for interaction with the human operator via the one or more input peripherals.

In some embodiments, the computing system is a virtual network computing system, the remote server is a VNC server and the computer is a VNC client. The input peripherals may comprise a keyboard and/or a mouse. The output peripherals may comprise a display.

Embodiments of the invention are described below, by way of example, with reference to the following drawings, in which:.

<FIG> shows a computing system <NUM> in which, according to one embodiment, the method of the invention is implemented. Computing system <NUM> comprises one or more computers <NUM> which are physically accessible by one or more human operators <NUM>. Computing system <NUM> also comprises one or more remote servers <NUM>. Remote server <NUM> is "remote" in that it is located at a different location to computer <NUM> such that remote server <NUM> cannot be physically accessed by human operator <NUM>. In some instances, this may be because remote server <NUM> is a virtual remote server. Computer <NUM> and remote server <NUM> are communicatively coupled to one another via at least one communication network <NUM>. This communicative coupling enables data to be communicated between the computer <NUM> and remote server <NUM>. The at least one communication network <NUM> may include the internet (i.e., IP, IPv4, IPv6), a cellular network (i.e., <NUM>, <NUM> LTE, <NUM>), a local area network, a cloud network, a wireless network, or any other known communication network. Also present in computing system <NUM> is an administrator computer <NUM> which is accessible by a human administrator <NUM> of the computing system <NUM>. Administrator computer <NUM> is communicatively coupled to the computer <NUM> and remote server <NUM>, such that human administrator <NUM> can maintain and control policies for the interaction between computer <NUM> and remote server <NUM>. The described computing system <NUM> is only exemplary and modifications to it, including removing or adding of system components, are possible.

<FIG> shows selected aspects of computing system <NUM> shown in <FIG>. Specifically, <FIG> shows computer <NUM> in communication with remote server <NUM> over communication network <NUM>. Computer <NUM> comprises one or more software applications <NUM>, processor <NUM>, memory <NUM>, one or more input peripherals <NUM> and one or more output peripherals <NUM>. Processor <NUM> includes a central processing unit (CPU) and/or a graphical processing unit (GPU). Memory <NUM> comprises a data storage device and/or semiconductor memory. The data storage devices takes the form of a hard disks drive, solid state drive, external drive, removable optical disk, and/or memory card. Semiconductor memory takes the form of volatile memory for temporarily storing data, e.g. random-access memory (RAM), and non-volatile memory for storing data long-term, e.g. read-only memory (ROM), Flash memory.

One or more applications <NUM> are stored as computer programs in memory <NUM> and are executed on computer <NUM> via processor <NUM>. These applications, which facilitate direct interaction with human operator <NUM> via input peripherals <NUM> and output peripherals <NUM>, include the operating system (OS) and desktop applications. Examples of known operating systems include Microsoft Windows, MacOS, and Linux. Examples of known desktop applications for computer <NUM>' include web browsers such as Google Chrome, documentation applications such as Microsoft Word, and remote access applications, which are discussed further herein. It should be appreciated, however, that the invention is not limited to the specific applications mentioned here.

As mentioned, computer <NUM> comprises one or more input peripherals <NUM>. The purpose of input peripheral <NUM> is to enable human operator <NUM> to send instructions to computer <NUM>. Examples of input peripheral <NUM> include a mouse, a keyboard, a touch screen, an image scanner, a barcode reader, a game controller, a microphone, a digital camera, a webcam, and the like. Input peripheral <NUM> may be integrated with computer <NUM>, such as found in laptop computers, or may be external to computer <NUM>, as for desktop computers. Human operator <NUM> sends instructions to computer <NUM> using input peripheral <NUM> by interacting with application <NUM>. In particular, human operator <NUM> uses input peripheral <NUM> to interact with a user interface (UI) of application <NUM>. In the case of a graphical user interface (GUI), this interaction is achieved by, for example, pressing a button, clicking, dragging, scrolling, etc. on the GUI via the input peripheral <NUM>.

Computer <NUM> also comprises one or more output peripherals <NUM>. The purpose of output peripheral <NUM> is to enable human operator <NUM> to receive information from computer <NUM>. Examples of output peripheral <NUM> include a display device (for example, a computer monitor or a projector), a printer, headphones, and computer speakers. Similar to input peripheral <NUM>, output peripheral <NUM> may be integrated with computer <NUM> or may be external to computer <NUM>. Human operator <NUM> receives information from computer <NUM> using output peripheral <NUM> by interpreting the UI of application <NUM> using their senses, such as sight or hearing.

Other components (not shown in <FIG>) are present in computer <NUM>. For example, computer <NUM> comprises one or more of: a network adaptor card to enable communication across the communication network <NUM>, a power supply, a motherboard, a sound card, and the like.

<FIG> also shows remote server <NUM>. Remote server <NUM> comprises one or more applications <NUM>, processor <NUM>, memory <NUM> and machine interface <NUM>. The one or more applications <NUM> are stored as computer programs in memory <NUM> and are executed on remote server <NUM> via processor <NUM>. These applications do not have direct interaction with human operator <NUM> via input peripherals <NUM> and output peripherals <NUM> (although there is indirect interaction via a remote access protocol, as described below). Instead, the one or more applications <NUM> are applications which interact directly with computer <NUM> via communication network <NUM> and machine interface <NUM>. Example applications for remote server <NUM> include the applications mentioned above for computer <NUM>, and robotic process automation (RPA) applications such as those described in <CIT> and <CIT>.

Remote server <NUM> can take the form of a single server or multiple servers, or alternatively can take the form of a distributed server. Distributed servers operate by distributing processing and data across the constitute components.

Remote server <NUM> can be a physical remote server or a virtual remote server. When remote server <NUM> is a virtual remote server, the applications <NUM>, processor <NUM>, memory <NUM> and machine interface <NUM> are all virtual entities.

<FIG> also shows optional remote platform <NUM> comprised in computing system <NUM>. Remote platform <NUM> is a type of physical remote server that hosts one or more virtual remote servers <NUM>. In the same way as remote server <NUM>, remote platform <NUM> is "remote" in that it is located at a different location to computer <NUM> such that remote platform <NUM> cannot be physically accessed by human operator <NUM>. Computer <NUM> and remote platform <NUM> are communicatively coupled to one another via at least one communication network <NUM>. This communicative coupling enables data to be communicated between the computer <NUM> and remote platform <NUM> (and any of the one or more virtual remote servers <NUM> hosted thereon). In addition to the one or more virtual remote servers <NUM>, remote platform <NUM> comprises one or more applications, a processor, a memory and machine interface (not shown). Example applications for remote platform <NUM> include the applications mentioned above for computer <NUM> and remote server <NUM>, as well as virtual machine applications to provide the one or more virtual remote servers <NUM>.

Although, as mentioned above, human operator <NUM> cannot directly interact with remote server <NUM>, human operator <NUM> can indirectly interact with remote server <NUM> via computer <NUM> using a remote access protocol. In particular, the remote access protocol allows human operator <NUM> to remotely interact with application <NUM> on remote server <NUM>. This is achieved by receiving on computer <NUM>, in one form or another, the UI of application <NUM>, so that human operator <NUM> can interpret the controls that are available in application <NUM>. Human operator <NUM> then inputs control signals into computer <NUM> via input peripheral <NUM>, and these control signals are sent from computer <NUM> to remote server <NUM> via communication network <NUM>. Consequently, remote server <NUM> executes the control signals via processor <NUM> to cause an interaction with application <NUM> on remote server <NUM>.

There are several known remote access protocols which operate in the way described above that can be used with the invention. A first example of a remote access protocol for use with the invention is the remote framebuffer (RFB) protocol, which is depicted in <FIG>. The RFB protocol works on the framebuffer level, which corresponds to rendered GUI images, in this case the rendered GUI images of remote server <NUM>. As a consequence, the RFB protocol can be applied to any application <NUM> running on remote server <NUM>, including the operating system (i.e. Microsoft Windows, MacOS, etc.) and other applications such as web browsers.

The basic operation of the RFB protocol is as follows. As shown in <FIG>, computer <NUM> receives images of the framebuffer of remote server <NUM>, i.e. rendered GUI images of remote server <NUM>. By "image" it is meant that the data is pixel data. The format and encoding of this pixel data varies as they are negotiated by computer <NUM> and server <NUM> to suit the particular scenario in which the RFB protocol is being implemented. These GUI images are sent sequentially and form a 'video' of the GUI of remote server <NUM> for interpretation by human operator <NUM> via computer <NUM>. This means that once human operator <NUM> sends control signals to remote server <NUM> via input peripheral <NUM> and computer <NUM>, that human operator <NUM> can see the consequence of the control signals in subsequent GUI images received from remote server <NUM>. For instance, if the input peripheral <NUM> is a mouse, and the control signal sent by human operator <NUM> is to move the mouse, then the human operator <NUM> would see in subsequently received GUI images that the mouse cursor had been moved on the desktop of remote server <NUM>.

A particular example of a GUI image <NUM> sent by the RFB protocol is shown in <FIG>. In particular, <FIG> depicts the GUI image <NUM> of the desktop of server <NUM>, which shows a plurality of applications <NUM>, including an operating system <NUM> (Microsoft Windows), a web browser <NUM> (Google Chrome), an RPA application <NUM> (Blue Prism), and a remote access application <NUM> (Real VNC). The web browser <NUM> has an open window <NUM>, where an Electronic Health Record which includes sensitive data <NUM> is being viewed. GUI image <NUM> is typically sent in its entirety to computer <NUM> to be viewed by human operator <NUM>. Note that the grey boxes in window <NUM> of <FIG> are used for the simplicity of the figure, and would in the actual GUI image be filled with text.

In the RFB protocol, computer <NUM> is referred to as the 'client' and remote server <NUM> is referred to as the 'server'. Thus, on computer <NUM>, application <NUM> comprises a client remote access application, whilst on server <NUM>, application <NUM> comprises a server remote access application. Typically, the RFB protocol is implemented using virtual network computing (VNC) applications, such as Real VNC. Thus, remote server <NUM> may be a VNC server, whilst computer <NUM> may be a VNC client. Further explanation of the RFB protocol can be found at <NPL>.

A second example of a remote access protocol for use with the invention is the remote desktop protocol (RDP) by Microsoft, which is shown in <FIG>. RDP is similar to the RFB protocol except that, instead of sending the rendered GUI image from remote server <NUM> to computer <NUM>, data for rendering the GUI image is sent. In other words, in RDP, the data from remote server <NUM> are not pixel data. Once computer <NUM> receives the data for rendering the GUI image, the GUI is rendered by computer <NUM> and displayed on output peripheral <NUM> for receipt by human operator <NUM>. Additionally, RDP allows human operator <NUM> to interact with application <NUM> appearing in the rendered GUI image using operating system commands, including highlighting, copying, etc. In this example, application <NUM> on computer <NUM> includes Remote Desktop Connection in order to implement RDP.

Although in the two examples described here the data received by computer <NUM> relates to the GUI of remote server <NUM>, the invention is not limited to GUIs and may be applied to other types of UI. Moreover, other remote access protocols work with the invention, including the serial line internet protocol (SLIP), Point-to-Point Protocol (PPP), Point-to-Point Over Ethernet (PPPoE), Remote Access Services (RAS), Point-to-Point Tunnelling Protocol (PPTP), and others.

As shown in the example of <FIG>, the UI <NUM> received from remote server <NUM> at computer <NUM> via the remote access protocol can include sensitive data <NUM>. Sensitive data, as referred to herein, are a special type of data which requires heightened security considerations due to its cognitive content. Breaches of sensitive data can result in the accidental or unlawful destruction, loss, alternation, unauthorised disclosure of, or access to, sensitive data, which can have significant human consequences. For example, the permanent deletion of medical records of a person potentially has significant and long-lasting consequences for the health of said person. For this reason, in various jurisdictions, the storage and processing of sensitive data is regulated, for example via General Data Protection Regulation (GDPR) in the European Union, and the Data Protection Act <NUM> in the United Kingdom.

Sensitive data takes the form of text or images. Sensitive data may include personal information, i.e. information relating to an identified or identifiable natural person. For example, sensitive data may include a name, an address, a date of birth, a telephone number, an identification document image, a facial image. Other types of sensitive data include location data of a person, an online identifier or one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of the person. Additionally or alternatively, sensitive data may include financial information, such as credit card numbers, and bank numbers. As a further alternative, sensitive data may include medical information.

Referring back to <FIG>, an example UI <NUM> of remote server <NUM> is depicted which shows sensitive data <NUM>. In particular, <FIG> shows the window <NUM> of a web browser <NUM> which has open an Electronic Health Record web page containing sensitive data <NUM>. In this example, the sensitive data <NUM> takes the form of personal information, including an identification document (shown on the left of box <NUM>) which includes a facial image, and personal details such as name, address, date of birth and telephone number (shown on the right of box <NUM>).

Sensitive data received at computer <NUM> via the remote access protocol are problematic if output to human operator <NUM> because human operator <NUM> may not be authorised to view that sensitive data. Further, there is a risk of a data breach via human operator <NUM>. Moreover, access by human operator <NUM> to this sensitive data is potentially unlawful under the regulations of various jurisdictions. Therefore, a method for preventing sensitive data from being output to human operator <NUM> is desirable.

<FIG> shows the method employed by the invention to prevent sensitive data received via a remote access protocol from being output to human operator <NUM>. The method of <FIG> involves computer <NUM>, remote server <NUM> and processor <NUM>. Processor <NUM> may be one of processor <NUM> and processor <NUM>, or a different processor to processor <NUM> and processor <NUM>. It is preferable that processor <NUM> is processor <NUM> of remote server <NUM> (i.e. the method is performed server-side) to reduce possible security risks. Moreover, processor <NUM> being processor <NUM> reduces the computational load for computer <NUM>.

As shown in <FIG>, the method comprises the following steps which are performed at processor <NUM>:.

Thus, the method of the invention is centred around processor <NUM>, which sits between remote server <NUM> and human operator <NUM>, and controls what the human operator <NUM> receives of the UI of application <NUM>. In particular, processor <NUM> identifies sensitive data using the UI of application <NUM> and/or a text-based representation of application <NUM>, and consequently modifies the UI to remove the sensitive data in order to output to human operator <NUM> on computer <NUM> a UI which does not include sensitive data. In this way, human operator <NUM> does not have access to the sensitive data on remote server <NUM> via the remote access protocol, reducing the risk of a data breach.

In further detail, at step <NUM>, processor <NUM> receives from remote server <NUM> a UI of application <NUM> executing on remote server <NUM>. This step in typical for remote access protocols, as described above, except that the UI is received at processor <NUM> rather than necessarily at computer <NUM>. The form of the UI of application <NUM> depends on the application and the remote access protocol being used. Application <NUM> may be any of the applications mentioned herein, including an operating system or other applications such as a web browser, a virtual machine application, an RPA application, etc..

When either a RFB protocol or RDP is used, step <NUM> involves receiving a desktop GUI of remote server <NUM> at processor <NUM>. However, in some instances, the application <NUM> of interest for interaction with human operator <NUM> will be an application GUI residing on the desktop GUI, such as the web browser <NUM> in the example of <FIG>. In these instances, the GUI of the application <NUM> of interest forms a portion of the desktop GUI. This can be seen in <FIG>, where window <NUM> of web browser <NUM> forms only a portion of the desktop GUI <NUM>.

The next steps, <NUM> to <NUM> are discussed in detail in the sections below. In particular, step <NUM> is discussed in the section titled "text-based application representation". Step <NUM> is discussed under "filtering and filter customisation". Steps <NUM> and <NUM> are discussed in the section named "modified user interface".

At step <NUM>, processor <NUM> either determines a text-based representation of application <NUM>, or receives a text-based representation of application <NUM> from remote server <NUM>. A text-based representation of application <NUM> is a representation made up of alphanumeric characters, including letters, digits, punctuation, and other special characters. The letters may be from any writing system (e.g. Latin, Cyrillic, Hangeul, Arabic, Greek, etc.). The digits may be from any numeral system (e.g. Arabic numerals, Chinese numerals, Japanese numerals, Roman numerals). Punctuation and other special characters include, for example, [ ] ! " # $ % &' ( ) * + ,. / : ; < = > ? @ \ ^ _ ' | { } ~ -.

The purpose of the text-based representation of application <NUM> is to provide additional information about the UI of application <NUM>, which is useful for identifying and subsequently removing sensitive data in the UI of application <NUM>. For instance, the text-based representation of application <NUM> may reveal that a certain field in the UI of application <NUM> is a "name" field for a person (e.g. <name="BETTY DIXON">), whereas the UI itself may only reveal the person's actual name (e.g. "BETTY DIXON"), and this name may not be in a form that can be easily extracted (e.g. pixel data rather than text). By specifically referring to "name" and by providing the name text, the text-based representation of application <NUM> provides additional contextual information which makes the sensitive data easier to identify than using the UI of application <NUM> alone.

The text-based representation of application <NUM> can take a number of forms. In some instances, the text-based representation of application <NUM> comprises text extracted from the UI and/or from application <NUM>. For example, extracting text from GUI image <NUM> of <FIG>, would yield the following text:
"Electronic Health Record
My Day Patient
Patent Summary
Problems Medications Implanted Devices
Select Encounter. Today's Vitals
Health Reminders Due
Immunizations Due
Tasks
<NUM>:<NUM><NUM>/<NUM>/<NUM>".

In addition to the text in window <NUM> which appears in <FIG> as grey boxes.

The text is extracted using Optical Character Recognition (OCR). When using the RFB protocol, the text is extracted by processor <NUM> using OCR on the GUI image of application <NUM> sent from remote server <NUM>. When using RDP, text is extracted by processor <NUM> by identifying natural words and numbers in the data for rendering the GUI image of application <NUM> sent from remote server <NUM>. Alternatively, when using RDP, text is extracted by processor <NUM> using OCR on the rendered GUI image of application <NUM>.

In some instances, the text-based representation of application <NUM> comprises a programmatic representation of application <NUM>. A programmatic representation is a representation which is based on the underlying program of application <NUM>. For example, the programmatic representation of application <NUM> may be a document object model (DOM) of application <NUM> or a DOM of the UI of application <NUM>. DOM is an application programming interface for HTML- and XML-based applications. The DOM defines the logical structure of application <NUM>. In particular, tags (such as "name" in the <name="BETTY DIXON"> example above) become element nodes and form the structure of the DOM, whilst text (such as "BETTY DIXON" in the same example) becomes text nodes. The DOM is in the form of text such as JSON or XML, or the like. Further information about DOM is found at Hors, A. et al, <NUM>, "Document Object Model (DOM) Level <NUM> Core Specification", W3C Recommendation.

When using the RFB protocol, in order to receive the programmable representation of application <NUM>, processor <NUM> sends a message to remote server <NUM> in order to request the programmable representation. Alternatively, remote server <NUM> is programmed to send the programmable representation of application <NUM> whenever the remote access protocol is in use. As a further alternative, the programmable representation may be determined by processor <NUM> using the UI of application <NUM>. In the case of the programmable representation being a DOM, processor <NUM> determines the DOM from GUI image using a deep learning model. One example of a suitable deep learning model for this is "pix2code". When using RDP, the programmable representation of application <NUM> is determined by processor <NUM> based on the data for rendering the GUI image sent from remote server <NUM>.

The skilled person will appreciate that forms of text-based representation of application <NUM>, other than those mentioned here, are suitable for use with the invention.

Step <NUM> of <FIG> is to identify sensitive data in the UI of application <NUM> and/or the text-based representation of application <NUM> using one or more preselected filters. The purpose of step <NUM> is to determine particular portions the UI of application <NUM> received in step <NUM> that contain sensitive data so that these portions can be modified in step <NUM>. As mentioned above, sensitive data takes the form of text or images. The preselected filters used for application <NUM> thus depends on whether the sensitive data are text-based, image-based or both.

When sensitive data takes the form of text, the preselected filter comprises a rule. The text-based representation of application <NUM> will be parsed to determine if one or more portions of the UI of application <NUM> satisfy the rule, and thus contains sensitive data. For example, for an application <NUM> relating to personal data, the rule might determine the presence of country names (e.g. "United Kingdom", "United States of America", "Japan", "South Korea", etc.) as these country names are likely to be the birth country or country of residence of the person, and thus sensitive data. The list of country names may be stored in a database or the like.

DOMs are particularly good in combination with rules. This is because the element nodes of DOMs typically identify the type of information of the corresponding text node. For instance, in the <name="BETTY DIXON"> example mentioned above, the element node "name" identifies that the text node "BETTY DIXON" is a name, which is sensitive data. In this instance, the rule is that if the element node is "name", then the corresponding text node is sensitive data. In this way, sensitive data can easily be identified using the element nodes.

In some instances, the rule may be a regular expression (regex) matching rule. A regular expression, often called a pattern, is an expression used to specify a set of strings in a concise way. This conciseness is achieved using operations, such as "?" which indicates one or no occurrences (e.g. colou?r matches both "color" and "colour"), "{min,max}" which indicates the minimum and maximum number of matches, "[ ]" matches a single character that is contained within the brackets, "^" matches the starting position of the string, "$" matches the ending position of the string, and others. For example, UK postal codes are in the format "area-district sector-unit", for which: area is one or two letters; district is one digit, two digits, or a digit followed by a letter; sector is a single digit; and unit is two letters. For example, "WA2 0XP", "E7 0AQ" and "WC1B 5HA" are all valid UK postal codes. Searching for each possible combination of letters and digits would be cumbersome, so instead the following regular expression is used. A basic regular expression for UK postal codes is:
^[A-Z]{<NUM>,<NUM>}[<NUM>-<NUM>][A-Z0-<NUM>]? [<NUM>-<NUM>][A-Z]{<NUM>}$
where area is [A-Z]{<NUM>,<NUM>}, district is [<NUM>-<NUM>][A-Z <NUM>-<NUM>]?, sector is [<NUM>-<NUM>], and unit is [A-Z]{<NUM>}. As can be appreciated from this example, searching for a match to this regular expression in the text-based representation of application <NUM> is much more efficient than searching for each possible combination of letters and digits. Furthermore, regular expressions have a tendency to be suitable for sensitive data as the format of sensitive data is often known.

In some instances, the preselected filter comprises a machine learning model that is configured to identify text. An example machine learning model which is suitable for this purpose is a natural language processing (NLP). NLP models identify sensitive data using Named Entity Recognition (NEP), which seeks to locate and classify named entity mentions in unstructured text into pre-defined categories such as the person names, locations, medical codes, time expressions, quantities, monetary values, percentages, etc. For example, in the sentence "Betty Dixon has an appointment at East London Clinic on <NUM>/<NUM>/<NUM> at <NUM>:<NUM>", NEP produces an annotated block of text that highlights the names of entities as follows:
[Betty Dixon]Person has an appointment at [East London Clinic]Location on [<NUM>/<NUM>/<NUM>]Time at [<NUM>:<NUM>]Time.

In this example, a person name, a location name, and two temporal expressions have been detected and classified. NLP models are easy and efficient to train, as well as to run. Thus, where the sensitive data are in the form of text, the sensitive data are preferably identified using NLP models.

When sensitive data are in the form of an image, such as an identification document image or a person's facial image, or when not using the text-based representation of application <NUM>, the preselected filter comprises a machine learning model that is configured to identify an image from the GUI image of application <NUM>. Example machine learning models which are suitable for this purpose are computer vision models. Computer vision models detect the GUI elements (images, text boxes, buttons, icons, drop down lists etc.) that may contain sensitive data using contextual information. For example, a text box next to the label "Name" is considered sensitive data. These GUI elements are detected using object detection algorithms. Moreover, computer vision models are able to detect certain objects such as facial images or identification document images using object detection algorithms.

Once the sensitive data have been identified using the one or more preselected filters, then the location of the sensitive data in the GUI image has to be identified when using the RFB protocol. This is so that that in step <NUM> the GUI can be modified only at these locations of the GUI image (i.e. only at the pixels pertaining to sensitive data). Location identification of the identified sensitive data is based on the method used to identify the sensitive data. If the sensitive data are identified from the GUI image using a machine learning model, then the machine learning model can also extract the location of the corresponding GUI elements in the GUI image. Otherwise, one or more computer vision models are used to identify the location of GUI elements in the GUI image, the one or more computer vision models used being dependent on what kind GUI element that is being identified. In particular, if the GUI element corresponds to an image (e.g. an identification document image, a facial image), then template matching is used on the GUI image to identify the image location. If the GUI element corresponds to text, then the text box is detected in the GUI image (e.g. using OCR) and then matched against the GUI element. The location of the sensitive data typically takes the form of bounding boxes in the GUI image, which are each defined by the co-ordinates or other pixel reference to the four corners of the respective bounding box.

As mentioned above, one or more preselected filters are used. When more than one preselected filter (i.e. a plurality of preselected filters) is used, each of the plurality of preselected filters is different so that different sensitive data can be identified. When more than one preselected filter is present, there may be more than one type of preselected filter. For example, the preselected filters can include one regular expression matching rule which uses the text-based representation of application <NUM>, and one machine learning model which uses the GUI image of application <NUM>.

Typically, the likely content of application <NUM> is known before the remote access protocol is put into use, thus filters are preselected to identify the sensitive data that is expected in the content of application <NUM>. Put another way, the one or more preselected filters are customised according to the content of application <NUM>. For example, if application <NUM> is a web browser which often contains medical and personal information because remote server <NUM> is part of a medical facility, then filters are preselected to identify this medical and personal information. Conversely, if application <NUM> is a webcam application, then a filter is preselected to identify facial images. To this end, the method of the invention optionally comprises, prior to the step of receiving, preselecting one or more filters based on sensitive data in the application.

Administrative computer <NUM> and human administration <NUM> of <FIG> maintain and control policies for the interaction between computer <NUM> and remote server <NUM>, thus the one or more preselected filters may be preselected by human administrator <NUM> using administrative computer <NUM>. To this end, administrative computer <NUM> can contain a database of filters, to allow human administrator <NUM> to customise the one or more preselected filters for the content of application <NUM>. In order for human administrator <NUM> to select the appropriate one or more filters, administrative computer <NUM> comprises an UI to set the one or more filters. For example, the UI may be a GUI that can be used to add filters. The one or more preselected filters are then stored in a config file. Optionally, the config file can also dictate how to modify certain types of sensitive data in step <NUM>. For example, the config file may contain directs that specify "look for National Insurance Numbers and blur the field".

In steps <NUM> and <NUM> of <FIG>, the UI received in step <NUM> is modified to remove the sensitive data identified in step <NUM> to form a modified UI. The term "remove" in this context can mean either exorcise or obscure. This modified UI is output to one or more output peripherals <NUM> of computer <NUM> for receipt by human operator <NUM>.

In further detail, for step <NUM>, the method used for modifying the UI to remove sensitive data depends on the remote access protocol used. This is because the form of the UI received in step <NUM> from remote server <NUM> is dictated by the remote access protocol. For example, when the remote access protocol is the RFB protocol, the received UI is in the form of a GUI image, so step <NUM> comprises modifying a GUI image. In another example, when the remote access protocol is RDP, the received UI is in the form of data for rendering a GUI image, such that step <NUM> comprises modifying the data for rendering a GUI image.

There are several approaches that can be used for modifying a GUI image (i.e. when using the RFB protocol). One approach is to use one or more image transforms in order to manipulate the pixels pertaining to sensitive data such that the sensitive data can no longer interpreted by human operator <NUM>. For instance, the GUI image may be blurred at locations in the GUI image in which sensitive data have been identified. The amount of blurring has to be sufficient to make the sensitive data uninterpretable to human operator <NUM>. An example of GUI image blurring is shown in the GUI image <NUM> of <FIG>, which corresponds to the GUI image <NUM> of <FIG>. As shown in <FIG>, the box <NUM>, which corresponds to box <NUM> of <FIG> that contains personal information including an identification document and personal details, has been blurred such that the sensitive data are no longer interpretable by human operator <NUM>.

An alternative image transform to applying a blur is to apply a solid colour at locations in the GUI image in which sensitive data have been identified. An example of GUI image with solid colour applied is shown in the GUI image <NUM> of <FIG>. In particular, as shown in <FIG>, the solid colour white has been applied to box <NUM> such that the sensitive data are no longer interpretable by human operator <NUM>. Other solid colours may alternatively be used. In some instances, the solid colour chosen is based on the background colour of application <NUM> so that human operator <NUM> may not realise that the UI has been modified. A further alternative image transform is to decrease the image resolution (i.e. increasing pixelisation of the GUI image). The skilled person will appreciate that any image transform which means that the sensitive data can no longer be interpreted by human operator <NUM> is suitable for the method of the invention.

There are several approaches that can be used for modifying the data for rending a GUI image (i.e. when using RDP). In general, these approaches edit the data to remove the sensitive data, which is usually in the form of text or an image, such that the rendered GUI image does not contain the sensitive data. This is done so that human operator <NUM> cannot interact with application <NUM> appearing in the rendered GUI image using operating system commands (e.g. highlighting and copying) to extract the sensitive data. When the sensitive data are in the form of text, an example approach is to replace the characters of the text with random characters or asterisks. For example, if the sensitive data to be removed is a name, the name "BETTY DIXON" may become random characters such as "DLNN1 56OLP" or "***** *****". Alternatively, the characters may be replaced by blank spaces so that human operator <NUM> might not realise that the UI has been modified. When the sensitive data are in the form of an image, then the image file may be exorcised from the data. To do this, the image file would be identified in the data by the file extension (for example, tiff,. png, etc.), and then removed from the data. Alternatively, one or more of the image transforms previously mentioned may be used to obscure the sensitive data in the image.

In general, methods which are not computationally intensive are preferred for modifying the UI to remove sensitive data as this ensures that the time taken between processor <NUM> receiving the UI in step <NUM> and outputting the modified UI in step <NUM> is minimal. This ensures that the UI of remote server <NUM> responds quickly to inputs of human operator <NUM>, and thus does not have high latency.

The form of the modified UI also depends on the remote access protocol being used. This is because the modified UI takes the same form as the UI received in step <NUM> from remote server <NUM>. For the RFB protocol, the received UI is in the form of a GUI image, thus the output to computer <NUM> is modified UI is in the form of a GUI image. For RDP, the received UI is in the form of data for rendering a GUI image, such the output to computer <NUM> is modified data so as to render a modified GUI image.

Once the modified UI has been output to human operator <NUM> via output peripherals <NUM> of computer <NUM>, human operator <NUM> can interact with the modified UI in the same way as they would have interacted with the original, unmodified UI when not using the method of the invention. In other words, human operator <NUM> can interact with application <NUM> on the modified UI using input peripherals <NUM>. Since modifications made to the UI are specific to the type of remote access protocol, this ensures that human operator <NUM> cannot gain access to the sensitive data by interacting with the modified UI using input peripherals <NUM>. For example, in the case of an RFB protocol, the UI is a GUI image, and because this image has been altered, human operator <NUM> is not able to access the sensitive data. In the case of RDP, the data for rendering a GUI image are altered such that the sensitive data are removed from the rendering of application <NUM>, meaning that human operator <NUM> cannot highlight and copy sensitive data (e.g. using the copy command in the operating system).

For the purpose of completeness, it is noted that the original, unmodified UI is not output to the one or more output peripherals <NUM> of computer <NUM>. This would cause sensitive data to be revealed to human operator <NUM> and therefore defeat the purpose of the invention.

The disclosed method of preventing sensitive data received via a remote access protocol from being output to human operator <NUM> is useful for various implementations of remote access protocols. For example, for outsourced technical support it is often the case that human operator <NUM> (i.e. the technical support staff member) is not authorised to see sensitive data on remote server <NUM>. Nevertheless, human operator <NUM> must connect to remote server <NUM>, via their computer <NUM> and a remote access protocol, to provide technical support to remote server <NUM>. Advantageously, by implementing the disclosed method, human operator <NUM> would not see the sensitive data but would still be able to provide technical support.

One particular implementation of remote access protocols for use with the disclosed method involves Robotic Process Automation (RPA). RPA uses automated processes which mimic the interaction that a human operator would have had with the user interface (e.g. the GUI) of an application. In this way, the automated processes can be thought of as a virtual operator. Interaction with an application in this way is necessary for applications that cannot be interacted with using only a machine interface, such as applications that do not have APIs.

The automated processes in RPA are implemented using virtual machines, each virtual machine having their own virtual desktop for interaction with a virtual operator. Typically, there are a plurality of virtual machines running on one physical server in parallel which perform the same or similar automated processes on an application. This ensures that less physical hardware is required compared to using human operators (where one physical computer is required per human operator). RPA delivers secure and accurate interactions with application because there is no need for a human operator or human discretion. Thus, by using virtual operators instead of human operators to interact with applications, there is improved data security as the likelihood of a data breach is reduced. Further information about RPA can be found in <CIT> and <CIT>.

Sometimes in RPA it is useful for human operator <NUM> to check that the virtual machine is operating correctly in order to debug and the like. However, because virtual machines are usually headless in the sense that they do not have a connected display device, and because virtual machines tend to be remote from human operator <NUM>, the best way for human operator <NUM> to access a virtual machine is by a remote access protocol. However, this is problematic because the human operator <NUM> is able to see all of the data on the virtual desktop of the virtual machine, including sensitive data. This reduces the security of RPA dramatically. The method of the invention for preventing sensitive data received from being output to human operator <NUM> is therefore useful for RPA.

<FIG> shows an RPA implementation of the method of the invention. In <FIG>, remote server <NUM> is a virtual remote server in the form of a virtual machine. Thus, remote server <NUM> is capable of performing automatic processes on application <NUM>. In particular, the automated processes are performed by a virtual operator of the virtual machine using the user interface of application <NUM>. Typically, there is no involvement by human operator <NUM>, and it is not possible for human operator <NUM> to directly see the automated processes being performed by virtual operator as the virtual machine is headless and remote from human operator <NUM>.

When human operator <NUM> wishes to see the automated processes being performed by a virtual operator on the virtual machine (e.g. one of remote server 40C, 40B, 40C), they do so via their computer <NUM> and the remote access protocol. Typically, there is a plurality of virtual machines, as shown in <FIG> by remote servers 40A, 40B and 40C, hosted by remote platform <NUM>. Human operator <NUM> can select which of the virtual machines he or she wishes to interact with via computer <NUM> using RPA application <NUM>. The remote desktop application <NUM> is able to send to computer <NUM> the user interface of the virtual desktop of the selected virtual machine (including the user interface of an application <NUM>). However, due to the method of the invention, processor <NUM> (not shown in <FIG>) ensures that sensitive data is removed from the user interface before it is output to human operator <NUM> on computer <NUM>.

For example, as shown in <FIG>, human operator <NUM> has selected remote server 40B of remote platform <NUM> and can therefore see and interact with the user interface of remote server 40B via computer <NUM>. In particular, human operator <NUM> will see the automated processes interacting with application <NUM> on the virtual machine, which in this example is a web browser. However, because user interface of remote server 40B contains sensitive data, this sensitive data has been removed in the user interface output to human operator <NUM> via computer <NUM> (seen by comparing the respective user interfaces). Accordingly, the security level prior to involvement of human operator <NUM> is maintained.

When implemented in software, the invention can take the form of a computer program. The computer program may be embodied as a computer-readable medium having computer executable code for use by or in connection with a processor. A computer-readable medium is any tangible device that can contain, store, communicate, propagate, or transport the program for use by or in connection with a processor. Moreover, a computer-readable medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device or a propagation medium. Examples of a computer-readable medium include a semiconductor memory, a random access memory (RAM), a read-only memory (ROM), a Flash memory, a hard disk drive, a solid state drive, an optical disk, and memory cards. Current examples of optical disks include CDs, DVDs, and Blu-Rays. Current examples of memory cards include USB flash drives, SD cards, microSD cards, MMC cards, xD cards, and memory sticks.

When implemented in hardware, the invention is not limited to the specific hardware described herein. The skilled person would appreciate that the invention can be implemented on different hardware than computer <NUM> and remote server <NUM> and still function in the way described.

The flow diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the methods of the invention. In some alternative implementations, the steps noted in the figures may occur out of the order noted in the figures. For example, two steps shown in succession may, in fact, be performed substantially concurrently, or the blocks may sometimes be performed in the reverse order, depending upon the functionality involved.

Claim 1:
A computer-implemented method of preventing sensitive data (<NUM>) received via a remote access protocol from being output to a human operator (<NUM>), the method comprising:
receiving, from a virtual remote server (<NUM>) via a remote access protocol, a user interface of an application executing on the virtual remote server, wherein the virtual remote server is in the form of a virtual machine and wherein automated processes are performed on the application executing on the virtual remote server by a virtual operator of the virtual machine using the user interface of the application (<NUM>);
determining, or receiving from the virtual remote server, a text-based representation of the application;
identifying sensitive data in the user interface and/or the text-based representation of the application using one or more preselected filters;
modifying the user interface to remove the identified sensitive data;
outputting the modified user interface to one or more output peripherals of a computer for receipt by the human operator.