Patent Publication Number: US-11050869-B1

Title: Privacy-based user behavior modification

Description:
TECHNICAL FIELD 
     The present disclosure is related generally to mobile electronic communications devices and, more particularly, to systems and methods for modifying user behavior based on privacy requirements in a mobile electronic communications device. 
     BACKGROUND 
     While the cellular phone was initially created to allow voice communications, it quickly grew into a device capable of handling entertainment tasks, financial transactions, social interactions and many other activities. As users interacted more frequently with their devices, user interface modalities were continually upgraded to keep up with the changes. For example, hardware push buttons gave way to virtual buttons displayed on the screen, and now even virtual buttons may be supplanted by voice interaction routines for most purposes. 
     However, voice interactions do not provide an ideal user interface modality. In particular, voice interactions are hampered by a number of concerns, including privacy and convenience. For example, voice assistant devices and routines today typically use loudspeakers and far-field microphones. 
     Before proceeding to the remainder of this disclosure, it should be appreciated that the disclosure may address some or all of the shortcomings listed or implicit in this Background section. However, any such benefit is not a limitation on the scope of the disclosed principles, or of the attached claims, except to the extent expressly noted in the claims. 
     Additionally, the discussion of technology in this Background section is reflective of the inventors&#39; own observations, considerations, and thoughts, and is in no way intended to be, to accurately catalog, or to comprehensively summarize any prior art reference or practice. As such, the inventors expressly disclaim this section as admitted or assumed prior art. Moreover, the identification or implication herein of one or more desirable courses of action reflects the inventors&#39; own observations and ideas, and should not be assumed to indicate an art-recognized desirability. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
       While the appended claims set forth the features of the present techniques with particularity, these techniques, together with their objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which: 
         FIG. 1  is a general schematic representation of a mobile electronic device in which various embodiments of the disclosed principles may be implemented; 
         FIG. 2  is a simplified side view of a mobile electronic device such as that described with respect to  FIG. 1 , usable in various embodiments of the disclosed principles; 
         FIG. 3  is a flow chart showing a process of privacy management in accordance with an embodiment of the disclosed principles; 
         FIG. 4  is a flow chart showing a process of privacy management in accordance with an embodiment of the disclosed principles within a context of assistant-based information retrieval; and 
         FIG. 5  is a flow chart showing an example of user privacy-based behavior modification instruction in accordance with an embodiment of the disclosed principles. 
     
    
    
     DETAILED DESCRIPTION 
     Before presenting a detailed discussion of embodiments of the disclosed principles, an overview of certain embodiments is given to aid the reader in understanding the later discussion. As noted above, voice interactions are useful for when users are trying to obtain information via their cellular device. However, existing voice interaction systems suffer from privacy issues, since such systems typically use loudspeakers and far-field microphones. 
     In overview, a wearable device such as a communications-enabled watch or other device analyzes user audio engagement with it, executes voice interpretation, assesses a user request or question, and analyzes background information such as environmental factors to determine whether a private or public setting is appropriate. The device then determines whether or not to alter user behavior before engaging to assure the determined privacy level. For example, if the is making a private inquiry from the wearable, e.g., a check account balance query, the wearable may assess a level of privacy from voice and background analysis and, if privacy is required, ask user to move closer (alter user interface/behavior) and/or adjusts volume levels accordingly before conveying the requested information to eth user. 
     Once determined, wearable then makes a determination whether or not to alter user behavior before engaging to assure privacy of communication. If user is making a private inquiry from wearable such as check account balance, wearable before conveying info to user assesses level of privacy from voice and background analysis and if privacy is determined to be required based on on-going assessment, device will ask user to move closer or alter voice levels (alter user interface/behavior) and adjusts volume levels accordingly. 
     With this overview in mind, and turning now to a more detailed discussion in conjunction with the attached figures, the techniques of the present disclosure are illustrated as being implemented in or via a suitable device environment. The following device description is based on embodiments and examples within which or via which the disclosed principles may be implemented, and should not be taken as limiting the claims with regard to alternative embodiments that are not explicitly described herein. 
     Thus, for example, while  FIG. 1  is a simplified electrical schematic drawing illustrating components of an example mobile electronic communications device with respect to which embodiments of the disclosed principles may be implemented, it will be appreciated that other device types may be used, including but not limited to laptop computers, tablet computers, and so on. It will be appreciated that additional or alternative components may be used in a given implementation depending upon user preference, component availability, price point and other considerations. 
     In the illustrated embodiment, the components of the user device  110  include a display screen  120 , applications (e.g., programs)  130 , a processor  140 , a memory  150 , one or more input components  160  such as RF input facilities or wired input facilities, including, for example one or more antennas and associated circuitry and logic. The antennas and associated circuitry may support any number of protocols, e.g., WiFi, Bluetooth, different generations of cellular service, e.g., 4G, 5G, etc. 
     The device  110  as illustrated also includes one or more output components  170  such as RF (radio frequency) or wired output facilities. The RF output facilities may similarly support any number of protocols, e.g., WiFi, Bluetooth, cellular including 5G, etc., and may be the same as or overlapping with the associated input facilities. It will be appreciated that a single physical input may serve for both transmission and receipt. 
     The processor  140  can be a microprocessor, microcomputer, application-specific integrated circuit, or other suitable integrated circuit. For example, the processor  140  can be implemented via one or more microprocessors or controllers from any desired family or manufacturer. Similarly, the memory  150  is a nontransitory media that may (but need not) reside on the same integrated circuit as the processor  140 . Additionally or alternatively, the memory  150  may be accessed via a network, e.g., via cloud-based storage. The memory  150  may include a random access memory (i.e., Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRM) or any other type of random access memory device or system). Additionally or alternatively, the memory  150  may include a read-only memory (i.e., a hard drive, flash memory or any other desired type of memory device). 
     The information that is stored by the memory  150  can include program code (e.g., applications  130 ) associated with one or more operating systems or applications as well as informational data, e.g., program parameters, process data, etc. The operating system and applications are typically implemented via executable instructions stored in a non-transitory computer readable medium (e.g., memory  150 ) to control basic functions of the electronic device  110 . Such functions may include, for example, interaction among various internal components and storage and retrieval of applications and data to and from the memory  150 . 
     Further with respect to the applications and modules, these typically utilize the operating system to provide more specific functionality, such as file system service and handling of protected and unprotected data stored in the memory  150 . In an embodiment, modules are software agents that include or interact with hardware components such as one or more sensors, and that manage the device  110 &#39;s operations and interactions with respect to the described embodiments. 
     With respect to informational data, e.g., program parameters and process data, this non-executable information can be referenced, manipulated, or written by the operating system or an application. Such informational data can include, for example, data that are preprogrammed into the device during manufacture, data that are created by the device or added by the user, or any of a variety of types of information that are uploaded to, downloaded from, or otherwise accessed at servers or other devices with which the device is in communication during its ongoing operation. In an embodiment, a privacy manager  180  executes functions associated with privacy assessment and privacy-based user behavior modification. 
     A power supply  190 , such as a battery or fuel cell, is included for providing power to the device  110  and its components. Additionally or alternatively, the device  110  may be externally powered, e.g., by a vehicle battery, wall socket or other power source. In the illustrated example, all or some of the internal components communicate with one another by way of one or more shared or dedicated internal communication links  195 , such as an internal bus. 
     In an embodiment, the device  110  is programmed such that the processor  140  and memory  150  interact with the other components of the device  110  to perform a variety of functions. The processor  140  may include or implement various modules and execute programs for initiating different activities such as launching an application, transferring data and toggling through various graphical user interface objects (e.g., toggling through various display icons that are linked to executable applications). As noted above, the device  110  may include one or more display screens  120 . These may include one or both of an integrated display and an external display. 
     In an embodiment, the input  160  and output  170  components include one or more speakers, e.g., one or more earpiece speakers and one or more loudspeakers, as well as one or more microphones. It will be appreciated that these components may be built into the device  110 , or alternatively, some or all may be separate from the device  110 . 
     Turning to  FIG. 2 , this figure shows a simplified side view of a flip-type device  210  within which various embodiments of the disclosed principles may be implemented. As can be seen, the device  210  includes a first part  201  hinged to a second part  203 . Although there is no requirement as to the features of each part  201 ,  203 , in the illustrated embodiment, the first part  201  may include a display screen  205 , while the second part  203  may include one or more physical keys  207  or another display screen. It will be appreciated that the display screen  205  of the first part  201  may also implement selectable keys, e.g., as touch sensitive displayed keys. 
     The device  210  as illustrated also includes a loudspeaker  209 , which is a speaker configured to transmit sound audible from a distance, e.g., within several feet of the device  210 . The device  210  also includes an earpiece speaker  211  which is placed so as to be adjacent to the user&#39;s ear when the device  210  is flipped open and is held to the user&#39;s head, e.g., during a phone call. The earpiece speaker  211  is configured to transmit sound that is clearly audible only from a very close range, e.g., within 12 inches. 
     One or more mics  213 ,  215  are also included, with one the mic  213  being usable while the device  210  is closed and the other mic  215  being usable while the device  210  is open. The second mic  215  is located so as to be adjacent the user&#39;s mouth when the device  210  is flipped open and is held to the user&#39;s head, e.g., during a phone call. Although not implicated in the present description, a camera  217  is also shown for reader context. 
     The primary longitudinal axis of the first part  201  is labelled by the identifier B while the primary longitudinal axis of the second part  203  is labelled by the identifier C. In operation of the device  210 , the device  210  may be open (e.g., the first part  201  rotated until it is essentially parallel and end-to-end with the second part  203  and axes B and C are collinear) or closed (e.g., with the first part  201  closed against the second part  203  such that the keys  207 , earpiece speaker  211  and mic  215  are not visible to, nor usable by, the user, and axes B and C are parallel but not collinear). In  FIG. 2 , the device  210  is shown partially open to illustrate the location and axis A of the hinge between parts  201  and  203 . 
     It will be appreciated that in the open state of the device  210 , the first part  201  may not be precisely parallel with the second part  203 , e.g., the first and second parts  201 ,  203  may meet at substantially more than or less than 180°. As such, the open state of the device  210  may be understood as the state in which the device cannot open substantially further without damage or strain. Similarly, in the closed state, whether or not the parts  201 ,  203  are precisely parallel, the device  210  cannot be closed substantially further without damage or strain. 
       FIGS. 3 and 4  include flowcharts showing methods of privacy management in accordance with embodiments of the disclosed principles. Referring first to  FIG. 3 , this figure is a flow chart showing a process of privacy management in accordance with an embodiment of the disclosed principles. The illustrated process  300  starts at stage  301  wherein the wearable device receives a request from the user. From stage  301 , the process flows to stage  303 , wherein the wearable device analyzes the ambient background (others present, background noisy, etc.) and user audio engagement (user close and quiet, user further and louder, etc.), and conducts voice interpretation to assigns a character (private or non-private) to the request result. For example, the weather is public information whereas a bank account balance is private information 
     At stage  305 , if the request result has been determined to be private and the environment has been determined non-private, the process  300  is routed to stage  307 , wherein the device alters its behavior to alter the user&#39;s behavior to assure privacy of communication. This alteration of device behavior may include one or both of (1) lowering speaker volume, thus forcing the user to come closer, and (2) displaying or uttering a request for the user to come closer to the device. If higher security is required, e.g., where moving closer and speaking at a lower volume is not sufficient, the device may request, audibly or via a screen, to defer responding until the context becomes more private. In an embodiment, the user is instructed to go to another location. 
     If instead one of the conditions analyzed at stage  305  fails (that is, either the request has been determined to be non-private or the environment has been determined to be private), the process  300  flows through stages  309  (request result has been determined to be private, environment has been determined to be private),  313  (request result has been determined to be non-private, environment has been determined to be non-private) and  315  (request result has been determined to be non-private, environment has been determined to be non-private) until a match is found, after which the process flows to stage  311 . 
     At stage  311 , the device behavior is left unaltered. Although a negative result at stage  305  could also feed directly to stage  311 , the additional stages are included not as required steps but to show the variety of conditions that may lead to stage the device behavior remaining unaltered. 
     It will be appreciated that a given device, wearable or otherwise, may include more than one path for voice assistance. For example, consider a mobile phone that is a flip-open device such as the device  210  shown in  FIG. 2 . A device-based path such as that discussed with respect to  FIG. 3  may apply when the device is open (private speaker enabled). However, an alternate assistant-based path (e.g., via ALEXA/GOOGLE) may apply when the device is closed (loudspeaker enabled). In this latter mode, the device may carry the appropriate process (e.g., Alexa/Google) to process user voice utterances when, for example, “Alexa” or “OK Google” is uttered. The results are then buffered by the device for processing based on content privacy assessment before being passed to the user. In this situation, background settings may determine whether the setting is private or non-private. 
     An example of the handling of such a situation is shown in the process  400  of  FIG. 4 . In particular,  FIG. 4  is a flow chart showing a process of privacy management in accordance with an embodiment of the disclosed principles within a context of assistant-based information retrieval. At stage  401  of the process  400 , while the flip device is closed, the user utters request. The device then sends the request to an assistant server associated with the assistant (e.g., an Alexa server). 
     The device receives the request result from the assistant server at stage  405  and buffers the request result at stage  407  to analyze the result for confidentiality or privacy. For example, if the result is a bank account balance or medical diagnosis, the result may be considered private (confidential), whereas if the result is a stock report or weather forecast, the result may be considered non-private. At stage  409 , the device compares the privacy of the request result with privacy evidenced in the device context (environment), and moves forward to stage  305  of the process  300  ( FIG. 3 ) via entry pint “A”. From there, as discussed above by reference to  FIG. 3 , device behavior is altered or not altered based on the two privacy levels (request result and environment). 
       FIG. 5  shows a flow chart of a process  500  that provides a more detailed view of potential initial steps prior to determining whether device or user behavior is to be altered to meet privacy requirements. At stage  501  of the process  500 , the user utters a request. The wearable device conducts a background privacy level assessment via the privacy manager  180  in conjunction with internet resources at stage  503 . The wearable also conducts an ASR assessment and analysis of the request at stage  505  to determine a required privacy level. 
     The wearable then syncs with a user privacy selection stored in a device menu setting governing engagement rules, and assigns a privacy level for the engagement accordingly at stage  507 . Finally, at stage  509 , the device compares the privacy of the request result with the privacy determined based on device context (environment) and moves forward to stage  305  of the process  300  ( FIG. 3 ) via entry pint “B”. From there, as discussed above by reference to  FIG. 3 , device behavior is altered or not altered based on the two privacy levels (request result and environment). 
     It will be appreciated that various systems and processes have been disclosed herein. However, in view of the many possible embodiments to which the principles of the present disclosure may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the claims. Therefore, the techniques as described herein contemplate all such embodiments as may come within the scope of the following claims and equivalents thereof.