ELECTRONIC DEVICE AND METHOD FOR PROCESSING USER INTERACTION INFORMATION

A method for processing user interaction information by an electronic device, includes: executing an application in an unsecure area of the electronic device; instantiating an object of the application; recognizing a user interface of the application, converting a user reaction between a pseudo-event and the instantiated object into data, and transmitting the data to a secure area of the electronic device; mirroring the application to the secure area by using the data; based on a user input being detected, inferring an event to be recognized by a graphical user interface (GUI) framework of the electronic device; and interpreting, in the secure area, the user reaction to the instantiated object corresponding to the inferred event

BACKGROUND

The disclosure relates to a method for processing user interaction information and an electronic device including the method.

2. Description of Related Art

Modern operating systems may support rich graphical representations through an integrated graphical user interface (GUI) framework. In addition, the operating systems may provide programming interfaces and/or software development kits (SDKs) such that a related application may utilize the graphical representations in a consistent manner.

Applications based on a GUI may collect and/or process information on a user interaction with respect to a graphical object implemented on the applications.

A user may install various related applications in an electronic device and use the applications. However, security may be vulnerable in processing user interaction information collected by the applications.

SUMMARY

Provided are a method for processing user interaction information and an electronic device including the method, in which user interaction information on an object of an application is processed as privacy information.

In addition, provided are a method for processing user interaction information and an electronic device which may process user interaction information on an object of an application as privacy information, so as to enhance security of user interaction information of a user interaction.

In addition, provided are a method for processing user interaction information and an electronic device which may automatically convert definite interaction information collectable by an application into a random variable, so as to enhance security of user interaction information of a user interaction.

According to an aspect of the disclosure, a method for processing user interaction information by an electronic device, includes: executing an application in an unsecure area of the electronic device; instantiating an object of the application; recognizing a user interface of the application, converting a user reaction between a pseudo-event and the instantiated object into data, and transmitting the data to a secure area of the electronic device; mirroring the application to the secure area by using the data; based on a user input being detected, inferring an event to be recognized by a graphical user interface (GUI) framework of the electronic device; and interpreting, in the secure area, the user reaction to the instantiated object corresponding to the inferred event.

The method may further include generating input data based on the interpreted user reaction; and encrypting the input data in the secure area.

The encrypting the input data may include encrypting the input data by using a homomorphic encryption system.

The transmitting the data to the secure area may include recognizing the user interface and extracting the data from a physical interface corresponding to a valid event.

The transmitting the data to the secure area may further include generating key-data pair data of the physical interface corresponding to the user reaction corresponding to the instantiated object.

The mirroring the application may include at least one of: generating the pseudo-event while traversing a reaction list of the instantiated object; generating virtual user reactions to graphical reactive objects to generate a simulated response to the graphical reactive objects; and mirroring a rendered image generated to correspond to the pseudo-event to a secure buffer of the secure area.

The inferring of the event to be recognized by the GUI framework may include inferring the event to be recognized by the GUI framework by using a key-data pair in the secure area.

The application may be developed by using a software development kit (SDK) including an oblivious event receipt (OER) function.

According to an aspect of the disclosure, an electronic device includes: a display; a memory configured to store an application and an operating system; and a processor configured to execute the application and the operating system stored in the memory and operate while distinguishing between a secure area and a normal area, wherein the processor is further configured to: control to execute the application; instantiate an object of the application; recognize a user interface of the application, convert a user reaction between a pseudo-event and the instantiated object into data, and transmit the data to the secure area; mirror the application to the secure area by using the data; based on a user input being detected, infer an event to be recognized by a graphical user interface (GUI) framework; and interpret, in the secure area, the user reaction to the instantiated object corresponding to the inferred event.

The processor may be further configured to generate input data based on the interpreted user reaction; and encrypt the input data in the secure area.

The processor may be further configured to encrypt the input data by using a homomorphic encryption system.

The processor may be further configured to recognize the user interface and extract the data from a physical interface corresponding to a valid event.

The processor may be further configured to generate key-data pair data of the physical interface corresponding to the user reaction corresponding to the instantiated object.

The processor may be further configured to: generate the pseudo-event while traversing a reaction list of the instantiated object; generate virtual user reactions to graphical reactive objects to generate a simulated response to the graphical reactive objects; and mirror a rendered image generated to correspond to the pseudo-event to a secure buffer of the secure area.

The application may be developed by using a software development kit (SDK) including an oblivious event receipt (OER) function, and the processor may be further configured to infer the event to be recognized by the GUI framework by using a key-data pair in the secure area.

According to an aspect of the disclosure, a non-transitory computer-readable storage medium storing computer-executable instructions for processing user interaction information that, when executed by at least one processor of an electronic device, cause the electronic device to: execute an application in an unsecure area of the electronic device; instantiate an object of the application; recognize a user interface of the application, convert a user reaction between a pseudo-event and the instantiated object into data, and transmit the data to a secure area of the electronic device; mirror the application to the secure area by using the data; based on a user input being detected, infer an event to be recognized by a graphical user interface (GUI) framework of the electronic device; and interpret, in the secure area, the user reaction to the instantiated object corresponding to the inferred event.

The computer-executable instructions, when executed by the at least one processor, may further cause the electronic device to: generate input data based on the interpreted user reaction; and encrypt the input data in the secure area.

Encrypting the input data may include encrypting the input data by using a homomorphic encryption system.

Transmitting the data to the secure area may include recognizing the user interface and extracting the data from a physical interface corresponding to a valid event.

Transmitting the data to the secure area further may include generating key-data pair data of the physical interface corresponding to the user reaction corresponding to the instantiated object.

DETAILED DESCRIPTION

Hereinafter, various embodiments of this document may be described with reference to the accompanying drawings.

FIG.2illustrates user interaction information of an application based on a graphical user interface (GUI), according to various embodiments of the present disclosure.

A GUI-based application may include a plurality of graphical reactive objects (e.g., first graphical reactive object201, second graphical reactive object202, and third graphical reactive object203).

The plurality of graphical reactive objects201,202, and203may collect user reactions (e.g., first user reaction211, second user reaction212, and third user reaction213), respectively. For example, when the plurality of graphical reactive objects201,202, and203are user interface elements such as, but not limited to a button, the plurality of graphical reactive objects201,202, and203may collect a binary response (e.g., ri∈R:={r1, r2}). Alternatively or additionally, when the plurality of graphical reactive objects201,202, and203are user interfaces such as, but not limited to, an analog phone dial, the plurality of graphical reactive objects201,202, and203may collect a 10-ary response (e.g, ri∈R :={r1, r2, r3, r4, r5, r6, r7, r8, r9, r10}).

The user reactions211,212, and213collected by the application through the plurality of graphical reactive objects201,202, and203may be symbolized by ri∈πRj. For example, a symbolization operation may be defined in an application-specific method.

Continuing to refer toFIG.2, the application may reprocess the collected user reactions211,212, and213into input data231by using a corresponding function221.

In various embodiments, the input data231may be used as input data of a personal information service provided by the application.

The function221may structurize and/or process the user reactions211,212, and213, which may be symbolized by ri∈πRj, into the input data231corresponding to a personal information service input.

In various embodiments, the cardinality of a user reaction Rjmay be greater than or equal to 2 (e.g., j∈{1, . . . , N}, where N≥2). For example, since an action in which the application collects a user reaction by using a graphical reactive object may be the same as the process of “multiple choice questions and answers”, the cardinality of the user reaction Rjmay be greater than or equal to 2 (e.g., j∈{1, . . . , N}, where N≥2).

In various embodiments, a user reaction collected using a graphical reactive object may be a collection targeted as personal information, and/or may be a security target.

In various embodiments, a graphical object requiring a unary response may not be a collection target of personal information on a user reaction. For example, a graphical object requiring the unary response may include, but not be limited to, an object relating to confirmation (e.g., OK) for moving to the next screen, an arrow object for inducing a screen slide (e.g., stepping to a next slide).

The application may symbolize an ambiguous user intention (e.g., the user reactions211,212, and213) into ri∈R, which may represent a standardized action through user experiences and/or user interfaces (e.g., the plurality of graphical reactive objects201,202, and203) based on a GUI framework provided by an operating system. Alternatively or additionally, the application may process the user intentions (e.g., the user reactions211,212, and213) into the input data231corresponding to an input of a personal information service model through the function221.

FIG.3illustrates a user reaction to a reactive object, according to various embodiments of the present disclosure.

As shown inFIG.3, an application may include a plurality of graphical reactive objects (e.g., first graphical reactive object301, second graphical reactive object302, third graphical reactive object303, fourth graphical reactive object304, fifth graphical reactive object305, and sixth graphical reactive object306. The plurality of graphical reactive objects301to306ofFIG.3may include or may be similar in many respects to the plurality of graphical reactive objects201to203ofFIG.2, and may include additional features not mentioned above.

An operating system of the electronic device101may generate a process for an application and may execute a given program. For example, the operating system of the electronic device101may notify an application process of a time point to instantiate the plurality of graphical reactive objects301to306as an event. Such an event may be referred to as an oncreate event. Alternatively or additionally, the operating system of the electronic device101may inform an application process that a user has reacted to at least one of the plurality of graphical reactive objects301to306as an event. Such an event may be referred to as an onclick event.

WhileFIG.3depicts the operating system of the electronic device101having the plurality of graphical reactive objects301to306, the present disclosure is not limited in this regard. For example, the operating system of the electronic device101may include a smaller (less) amount of graphical reactive objects or a larger (more) amount of graphical reactive objects without deviating from the scope of the present disclosure. Alternatively or additionally, the operating system of the electronic device101may generate additional events that notify and/or inform an application process of the same and/or additional actions, reactions, and/or occurrences than those described above, without deviating from the scope of the present disclosure.

FIG.4is a block diagram illustrating an interaction among a user, an operating system, and an application of the electronic device101, according to various embodiments of the present disclosure.

The electronic device101may include an application401, an operating system402, and/or a secure area403.

The application401, according to various embodiments, may be software developed using a software development kit (SDK) including an oblivious event receipt (OER) function. For example, the application401may include an oblivious event receipt (OER) function (not shown).

The OER function may allow a method to be automatically added to an object Oj (not shown) of the application401. The OER function may allow recompiling the object Oj into a second object Oj′ (not shown) that may be obtained through modulation of the object Oj of the application401by automatic addition of a method thereto.

In various embodiments, the OER function may induce the operating system402to call back to the added method instead of an original method. The OER function may vary according to an application development language, a paradigm, and an operating system design manner.

The OER function may standardize a user reaction symbolization-interpretation process (e.g., UXA-FA) that may be required to be processed when the object Ojof the application401is called back by a user reaction by presenting an interface specification and/or an essential method to be overridden.

In various embodiments, the application401symbolizes an ambiguous user intention into ri∈R, which may be a standardized action through a user experience framework (UXA) based on a GUI framework provided by the operating system402. Alternatively or additionally, the application401may reprocess the user intentions into input data pi∈P, which may be a data form suitable as an input of a service model SA, through a function FA(e.g., the function221ofFIG.2). For example, the application401may acquire a user value VAthrough calculation of the service model SAand input data P input thereto.

In various embodiments, the OER function may randomize a signature and the number of additional methods. Alternatively or additionally, the OER function may dynamically and/or statically provide the signature and the number of the additional methods according to a development language of the application.

The OER function may communicate with a trusted kernel of the secure area403by using a signature. The trusted kernel of the secure area403may be needed to encrypt pi∈P corresponding to a user reaction into ENC (e.g., pi∈P) and transmit the ENC to a backend part of business logic (e.g., business logic of application401).

The operating system402, according to various embodiments, may be an operating system which operates in a normal area (e.g., normal or unsecured environment).

In various embodiments, the secure area403may be implemented in physical isolation (e.g., a trusted execution environment (TEE)) from the operating system402. Alternatively or additionally, the secure area403may be implemented in a framework of the normal area (e.g., normal or unsecured environment), which may provide a safety equivalent to the physical isolation.

The operating system402may include a GUI framework421, a graphic engine422, a pseudo-event generator423, a graphical behavior recorder424, and/or an event router425.

When the application401is executed, the application401may register the application in the operating system402. The application402may define symbolization-interpretation processing and/or data processing policies of a user reaction to a plurality of graphical reactive objects by using the operating system402. The electronic device101may perform preparation in the GUI framework421to use a graphical representation provided by the operating system402on the display module160. The electronic device101may display a visual interaction in response to a user input, on the display module160, by using the graphic engine422.

The event router425may process a user input collected through an input module (e.g., input module150of the electronic device101ofFIG.1). The electronic device101may interpret a user reaction, process data, and process life cycles of components according to an instruction (e.g., a program) of an application401by applying an application development template401provided by the operating system402.

In various embodiments, the operating system402may be configured, in terms of execution of a GUI-based application, to perform abstraction into a graphic engine black box which may repeatedly draw a picture (e.g., graphical user interface) according to an instruction of the application, and/or perform abstraction into an input processing black box which may translate a physical input signal into an event corresponding to a layout of the application.

The pseudo-event generator423may be a part (e.g., peer process) of the operating system402and may perform an operation of generating a predetermined pseudo-event intended by the application401on behalf of a user.

In various embodiments, the pseudo-event generator423may identify arrangement information of graphical reactive objects and a user interface layout of the application401with an authority equal to that of the GUI framework421, which may be provided by the operating system402.

Alternatively or additionally, the pseudo-event generator423may identify information on a type of user reaction to the graphical reactive objects and information on a user input signal from a physical interface corresponding to a user reaction from layout information of the application401. The user input signal from the physical interface may be, for example, information on an input coordinate when the input module150of the electronic device101is a touch screen. The pseudo-event generator423may generate a key-data pair with respect to a user reaction riand to a physical interface signal pi, which may refer to a combination of physical interface signals picorresponding to all user reactions and transmit the key-data pair to the secure area403.

In some embodiments, the pseudo-event generator423may generate virtual user reactions to graphical reactive objects included in the application401and transmit the generated user reactions to the event router425to trigger a dummy (e.g., simulated) response to the graphical reactive objects.

The graphical behavior recorder424may be a part (e.g., peer process) of the operating system402. When the graphic engine422of the operating system402renders graphic processing intended as a visual interaction by the application401with respect to a predetermined event, the graphical behavior recorder424may mirror a rendering result to a regular target frame buffer (not shown) and/or another memory area. For example, an information format of a graphic behavior may include, but not be limited to, a continuous frame of the rendering result of the graphic engine, and a script and/or a set of encoded codes indicated by the application401. However, the present disclosure is not limited in this regard. For example, the information format may vary according to the ability of an interpreter to interpret graphical behavior information.

The graphical behavior recorder424may participate in an operation of the graphic engine422with authority equal to that of the GUI framework. The graphical behavior recorder424may participate in allocation and compositing of a frame buffer for respective graphical reactive objects.

In some embodiments, the graphical behavior recorder424may transmit an image rendered by the graphic engine422of the operating system402to a separate auxiliary frame buffer (not shown), instead of the regular target frame buffer, according to graphic processing intended by the application401with respect to each user reaction which may be collected by a graphical reactive object.

The graphical behavior recorder424may mirror the image rendered in the auxiliary frame buffer to a trusted kernel residing in the secure area403via an interface of the secure area403.

The secure area403, according to various embodiments, may include a trusted kernel. The trusted kernel may be a secure process that may be executed within the secure area403and may interact with the pseudo-event generator423and/or the graphical behavior recorder424, which are components of the operating system402, and encrypt an input event. Alternatively or additionally, the trusted kernel of the secure area403may include a device driver which operates a physical interface which mediates an actual user's action in a secure mode.

The secure area403may refer to an area of which the processor120guarantees physical isolation and/or a program execution environment in the area. Alternatively or additionally, the secure area403may be include a sandbox process and/or a framework in which an independent address space may be implemented to have a sufficient security barrier within the operating system402. For example, visual interaction information to be mirrored to the secure area by the graphical behavior recorder424may be replaced with a set of encoded codes describing a rendering method, rather than a rendering frame buffer calculated by the graphic engine422of the operating system402.

The processor120ofFIG.1may support physical isolation (e.g., a TEE), and, as such, may implement the normal area402and the secure area403in independent operation modes, respectively, and ensure the integrity and confidentiality of a code and/or data stored in the secure area403. For example, in order to ensure the integrity and confidentiality of the code and/or data stored in the secure area403, the processor120may control information exchange and/or instructions between the normal area402and the secure area403. Alternatively or additionally, the processor120may limit the amount of types of resources required for program execution. For example, a library which may be included in a program may provide only a minimum amount of functions that may been verified to be error free (e.g., without errors such as buffer overflows). Due to these characteristics, it may not be possible to respond to complex visual processing requirements at a graphic engine level of the general operating system under the TEE. Therefore, a graphic reaction received from the graphical behavior recorder424may be a simple image stream in which complexity of a rendering process may have been reduced or eliminated.

FIG.5is a flowchart illustrating an operation of processing user interaction information, according to various embodiments of the present disclosure.

In operation501, under the control of the processor120, the electronic device101may perform an operation of executing the application401. The application401may be executed by a user input.

In operation503, under the control of the processor120, the electronic device101may perform an operation of instantiating an object of the application401.

In various embodiments, in operation503, under the control of the processor120, the electronic device101may perform, for example, an operation of generating a real object from an abstracted object in order to define a specific transformation with respect to the object of the application401.

In operation505, under the control of the processor120, the electronic device101may perform an operation of recognizing a user interface of the application401and converting a user reaction between a pseudo-event and the instantiated object into data.

In various embodiments, in operation505, under the control of the processor120, the electronic device101may extract data from a physical interface corresponding to the user reaction corresponding to the object of the application401and convert the same into data.

In various embodiments, in operation505, under the control of the processor120, the electronic device101may transmit data from a physical interface corresponding to the user reaction corresponding to the instantiated object as a key-data pair to a secure area (e.g., the secure area403ofFIG.4).

In various embodiments, in operation505, under the control of the processor120, the electronic device101may extract a coordinate on the display module160corresponding to an object and/or a layout of the user interface of the application401.

Under the control of the processor120, the electronic device101may transmit the data of the physical interface corresponding to the user reaction corresponding to the instantiated object as the key-data pair to the secure area (e.g., the secure area403).

In operation507, under the control of the processor120, the electronic device101may mirror the application401to the secure area (e.g., the secure area403) by using data on the user reaction between the pseudo-event and the instantiated object.

The operation of mirroring the application401may be, for example, an operation of copying a screen of the application401being executed.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may generate a pseudo-event while traversing a reaction list for the instantiated object.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may perform a pseudo-event generation operation so as to process only a graphical reactive object among instantiated objects.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may generate virtual user reactions to graphical reactive objects included in the application401, and trigger a dummy (e.g., simulated) response to the graphical reactive objects.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may generate rendering by performing graphic processing intended by the application401with respect to a predetermined event, and mirror a rendering result (or a rendered image) to a target frame buffer and/or another memory area.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may generate the pseudo-event by using the pseudo-event generator423, and mirror data on a visual interaction of the user interface of the application401to the secure area403by using the graphical behavior recorder424.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may request a secure buffer queue obtained by indexing an event into the secure area403.

In various embodiments, in operation507, under the control of the processor120, the electronic device101may mirror the rendered image to a secure buffer of the secure area403.

In operation509, under the control of the processor120, the electronic device101may wait for a user reaction from a physical interface operating in a secure mode.

In operation511, under the control of the processor120, the electronic device101may determine whether a user input has been received.

When the user input is received (YES on user input), the electronic device101may branch to operation511.

When the user input is not received (NO on user input), the electronic device101may branch to operation509.

In operation513, under the control of the processor120, the electronic device101may infer an event according to the user input received through the physical interface, and interpret a reaction to an instantiated object corresponding to the event.

In operation513, under the control of the processor120, the electronic device101may infer an event to be recognized by a GUI framework by using the key-data pair in the secure area.

In operation513, under the control of the processor120, the electronic device101may refer to (access) a secure buffer for each event mirrored in the secure area, and replicate a corresponding event to a secure frame buffer. The secure frame buffer may be referred to (accessed) by a display compositing driver.

In operation513, under the control of the processor120, the electronic device101may interpret a user reaction with respect to the instantiated object in the secure area403. For example, the electronic device may generate input data on the user reaction by using a function. The input data may be data input to a service model of the application401.

In various embodiments, in operation513, under the control of the processor120, the electronic device101may infer an event on the object and/or the layout of the application401corresponding thereto from key-data (e.g., a coordinate on the display module160). In operation513, under the control of the processor120, the electronic device101may provide, to the graphic engine422, information (e.g., an address of a secure frame buffer) of the data (e.g., a contiguous secure frame buffer) on the visual interaction of the user interface of the application mirrored to the secure area403from the inferred event, so that a trusted kernel (e.g., of secure area403) may copy the operation of the application401.

In various embodiments, in operation513, under the control of the processor120, the electronic device101may call back an event handler with respect to an object having a hashed signature, and interpret a user reaction and input data piwith respect to the object of the application401. The trusted kernel may call back the event handler under the control of the processor120. The hashed signature may be hashed by a SDK. In various embodiments, the application401included in the electronic device101may execute reaction interpretation for an instantiated object corresponding to an event by using a modified code instead of an original code, while maintaining business logic. For example, a method describing the operation of the object of the application401may be added and/or modified.

In operation513, under the control of the processor120, the electronic device101may encrypt the input data processed based on the user reaction interpreted in the secure area403.

In operation513, under the control of the processor120, the electronic device101may encrypt the input data processed based on the user reaction interpreted in the secure area403by using a homomorphic encryption system.

In operation513, under the control of the processor120, the electronic device101may encrypt the input data processed based on the reaction interpreted in the secure area403and transmit the encrypted input data to a server.

In operation513, under the control of the processor120, the electronic device101may transmit information obtained by encrypting the input data processed based on the user reaction interpreted in the secure area403to a backend part of the business logic of the application401.

FIG.6is a flowchart illustrating an operation of processing user interaction information, according to various embodiments of the present disclosure.

In operation601, under the control of the processor120, the electronic device101may perform an operation of executing the application401. The application401may be executed by a user input.

In operation603, under the control of the processor120, the electronic device101may prepare to execute a user interface element for an operation of the application401in the operating system403.

In various embodiments, in operation603, under the control of the processor120, the electronic device101may prepare to execute the application401on a GUI framework.

The user interface element may be, for example, a GUI, and the user interface element may be an object and/or a component used in the application401. The operation of preparing the execution of the user interface element for the operation of the application401in the operating system403may be, for example, an operation of calling and/or executing an object, a UI, and/or a GUI element of the application401.

In various embodiments, in operation605, the electronic device101may instantiate a second object recompiled based on a first object of the application401. The first object of the application401may refer to objects created and/or selected by an application developer. The second object may refer to objects obtained by recompiling the first object by a SDK.

In various embodiments, in operation605, under the control of the processor120, the electronic device101may perform the operation of instantiating the second object recompiled based on the first object of the application401in order to define a specific transformation with respect to the object of the application401, and may perform, for example, an operation of generating a real object from an abstracted object.

In various embodiments, in operation607, the electronic device101may extract data from a physical interface signal corresponding to a valid event.

In operation607, under the control of the processor120, the electronic device101may perform an operation of recognizing a user interface of the application401and converting a user reaction between a pseudo-event and the instantiated object into data.

In various embodiments, in operation607, under the control of the processor120, the electronic device101may extract data from a physical interface corresponding to the user reaction corresponding to the object of the application401and convert the same into data.

In various embodiments, in operation607, under the control of the processor120, the electronic device101may extract a coordinate on the display module160corresponding to an object and/or a layout of the user interface of the application.

In various embodiments, in operation609, under the control of the processor120, the electronic device101may transmit data of a physical interface corresponding to the user reaction corresponding to the instantiated object to a trusted kernel.

In various embodiments, in operation609, under the control of the processor120, the electronic device101may transmit the data of the physical interface corresponding to the user reaction corresponding to the instantiated object as a key-data pair to a secure area (e.g., the secure area403).

In various embodiments, in operation611, under the control of the processor120, the electronic device101may generate a pseudo-event while traversing a reaction list. In operation611, under the control of the processor120, the electronic device101may generate the pseudo-event by using the pseudo-event generator423, and mirror data on a visual interaction of the user interface of the application401to the secure area403by using the graphical behavior recorder424.

The reaction list may be a list of objects which can be interacted by an event (e.g., a touch event and/or a user input) among the data on the visual interaction of the user interface.

In various embodiments, in operation613, under the control of the processor120, the electronic device101may call back a pseudo-event handler. In various embodiments, in operation613, under the control of the processor120, the electronic device101may call back the pseudo-event handler in order to process a pseudo-event for the second object compiled to process only a graphic reaction part in the first object.

In operation615, under the control of the processor120, the electronic device101may request allocation of an event-indexed secure buffer queue from the trusted kernel.

In various embodiments, in operation615, under the control of the processor120, the electronic device101may request allocation of an event-indexed secure buffer queue from the trusted kernel by using the graphical behavior recorder424.

In operation615, under the control of the processor120, the electronic device101may request the trusted kernel to allocate and/or wait for a secure buffer having an index with respect to an event.

In operation617, under the control of the processor120, the electronic device101may render a graphic reaction (and/or a user reaction) of the second object to a dummy frame buffer, and mirror the rendered reaction to the secure buffer.

In various embodiments, in operation617, under the control of the processor120, the electronic device101may render a graphic reaction (and/or a user reaction) of the second object to the dummy frame buffer, and mirror the rendered reaction to the secure buffer by using the graphical behavior recorder424.

In various embodiments, in operation617, under the control of the processor120, the electronic device101may generate virtual user reactions to graphical reactive objects included in the application401, and trigger a dummy (e.g., simulated) response to all the graphical reactive objects.

In various embodiments, in operation617, under the control of the processor120, the electronic device101may generate rendering by performing graphic processing intended by the application401with respect to a predetermined event, and mirror a rendering result (or a rendered image) to a target frame buffer and/or another memory area.

In various embodiments, in operation617, under the control of the processor120, the electronic device101may generate a pseudo-event by using the pseudo-event generator423, and mirror data on a visual interaction of the user interface of the application401to the secure area403by using the graphical behavior recorder424.

In various embodiments, in operation619, under the control of the processor120, the electronic device101may determine whether a reaction list traversal has been completed.

In various embodiments, under the control of the processor120, the electronic device101may branch to operation621when it is determined that the reaction list traversal has been completed (YES at operation619).

In various embodiments, under the control of the processor120, the electronic device101may branch to operation611when the reaction list traversal has not been completed (NO at operation619).

In various embodiments, in operation621, under the control of the processor120, the electronic device101may wait for a user reaction.

In various embodiments, in operation623, under the control of the processor120, the electronic device101may determine whether a user input has been detected.

In various embodiments, under the control of the processor120, the electronic device101may branch to operation625when the user input is detected (YES at operation623).

In various embodiments, under the control of the processor120, the electronic device101may branch to operation621when there is no user input (NO at operation623).

In operation625, under the control of the processor120, the electronic device101may infer an event to be recognized by the GUI framework.

In various embodiments, in operation625, under the control of the processor120, the electronic device101may infer an event to be recognized by the GUI framework, based on the key-data pair for a valid event (or user reaction)-physical interface signal provided by the pseudo-event generator423by using the trusted kernel (e.g., of the secure area403).

In various embodiments, in operation625, under the control of the processor120, the electronic device101may infer an event according to the user input received through the physical interface by using the trusted kernel (e.g., of the secure area403), and interpret a reaction to an instantiated object corresponding to the event.

In various embodiments, in operation625, under the control of the processor120, the electronic device101may refer to (access) a secure buffer for each event mirrored in the secure area403, and replicate a corresponding event to a secure frame buffer. The secure frame buffer may be referred to (accessed) by an actual display compositing driver and/or the graphic engine422.

In various embodiments, in operation627, under the control of the processor120, the electronic device101may provide, to the graphic engine422, information (e.g., an address of a secure frame buffer) of the data (e.g., a contiguous secure frame buffer) on the visual interaction of the user interface of the application mirrored to the secure area403from the inferred event, so that the trusted kernel (e.g., of the secure area403) may copy the operation of the application401.

In operation629, under the control of the processor120, the electronic device101may call back an event handler.

In various embodiments, in operation629, under the control of the processor120, the electronic device101may call back an event handler with respect to an object having a hashed signature. The trusted kernel (e.g., of the secure area403) may call back the event handler under the control of the processor120. The hashed signature may be hashed by a SDK. In various embodiments, the application401included in the electronic device101may execute reaction interpretation for an instantiated object corresponding to an event by using a modified code instead of an original code, while maintaining business logic. For example, a method describing the operation of the object of the application401may be added and/or modified.

In operation631, under the control of the processor120, the electronic device101may transmit data obtained by interpreting, encrypting, and compiling a user reaction and input data piwith respect to the first object of the application401to a backend of the application401.

In various embodiments, in operation631, under the control of the processor120, the electronic device101may encrypt the input data processed based on the user reaction interpreted in the secure area403.

In operation631, under the control of the processor120, the electronic device101may encrypt the input data processed based on the user reaction interpreted in the secure area by using a homomorphic encryption system.

In operation631, under the control of the processor120, the electronic device101may encrypt the input data processed based on the reaction interpreted in the secure area403, and transmit the encrypted input data to a server.

In operation631, under the control of the processor120, the electronic device101may transmit information obtained by encrypting the input data processed based on the user reaction interpreted in the secure area403to a backend part of the business logic of the application401.

As used herein, the term “module” may include a unit implemented in hardware, software, or firmware, and may be interchangeably used with other terms, for example, “logic,” “logic block,” “component,” or “circuit”. The “module” may be a minimum unit of a single integrated component adapted to perform one or more functions, or a part thereof. For example, according to an embodiment, the “module” may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments, each element (e.g., a module or a program) of the above-described elements may include a single entity or multiple entities, and some of the multiple entities mat be separately disposed in any other element. According to various embodiments, one or more of the above-described elements may be omitted, or one or more other elements may be added. Alternatively or additionally, a plurality of elements (e.g., modules or programs) may be integrated into a single element. In such a case, according to various embodiments, the integrated element may still perform one or more functions of each of the plurality of elements in the same or similar manner as they are performed by a corresponding one of the plurality of elements before the integration. According to various embodiments, operations performed by the module, the program, or another element may be carried out sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be executed in a different order or omitted, or one or more other operations may be added.