Dynamic design of user interface elements

A user device may receive sensor data relating to one or more physical objects, transmit the sensor data to a server, receive a user profile relating to a user of the device, identify a user interface element corresponding to an application, modify the user interface element based on the user profile, and display the modified user interface element. A server may receive data relating to one or more physical objects, identify the one or more design patterns based on the one or more physical objects, create a user profile, update the user profile, receive application data relating to one or more applications running on a device remote from the server, identify a user interface element corresponding to the one or more applications running on the device, modify the user interface element based on the updated user profile, and transmit the modified user interface element to the device.

TECHNICAL FIELD

The present disclosure relates generally to methods and systems for dynamically designing user interface elements that may be displayed on a user device.

BACKGROUND

People have varying design preferences for everyday objects and digital user interface elements. One person may prefer one type of font in a document, such as a word document or e-mail, whereas another person may prefer a different type of font in the same type of document. Other examples of design preferences include a preference for round objects, or a preference for objects with sharp edges, or other different geometric shapes. In the design and implementation of digital products, designers and programmers must often make design choices before a digital product is released and made available to consumers. Designers and programmers can make informed decisions using for example marketing, sales, or other information, or can use personal experience in making a decision regarding the design of a digital product, but at some point the digital product must be prepared and finalized before being made available to a consumer through a display generated by an application, program, or other software.

SUMMARY OF THE DISCLOSURE

Systems and methods described herein may provide for dynamic design of user interface elements displayed on a device.

For example, a device configured to dynamically display user interface elements on the device may include a processor, a display in communication with the processor, a wireless transceiver in communication with the processor, and a non-transitory memory storing instructions that, when executed by the processor, cause the processor to perform processing including receiving sensor data relating to one or more physical objects, the data generated by a sensor; transmitting, by the wireless receiver, the sensor data to a server; receiving, by the processor, a user profile relating to a user of the device, the user profile comprising profile data describing one or more user preferences; identifying a user interface element corresponding to an application, the application running on the device, modifying the user interface element based on the user profile; and displaying, on the display of the device, the modified user interface element.

In some embodiments, the processing may further include receiving additional data relating to one or more second physical objects, the additional data being received from a second sensor. The processing may include transmitting the additional data to the server and receiving an updated user profile relating to the user of the device, the updated user profile comprising updated profile data.

In some embodiments, the user interface element may include a font. The modifying the user interface element may further include changing the font to a different font prior to display of the modified user interface element.

In some embodiments, the user interface element may further include one or more user interface windows, each of the one or more user interface windows having one or more first shapes. The processing for modifying the one or more user interface windows may further include changing each of the one or more user interface windows to a second shape different from the first shape.

In some embodiments, the device may further include a sensor. The sensor may be in communication with the processor, and the processor may receive the data relating to one or more physical objects from the sensor.

In some embodiments, the profile data of the user profile may be updated based on one or more probability deterministic scores before the processor receives the user profile. The probability deterministic scores may relate to the one or more user preferences.

As another example, a method for dynamically changing a user interface element on a device may include receiving, by a processor of the device, data relating to one or more physical objects, the data being generated by a sensor; transmitting, by a wireless transceiver of the device, the data to a server; receiving, by the processor, a user profile in response to transmitting the data to the server, and identifying, by the processor, a user interface element corresponding to an application running on the device; modifying, by the processor, the user interface element based on the updated user profile, and displaying the modified user interface element on a display of the device.

In some embodiments, the method may include the displayed modified user interface element relating to the application running on the device, and the one or more physical objects in the data describe one or more design patterns, and the user profile is based on the one or more design patterns.

In some embodiments, the method may include receiving, by a server, the data. The method may include identifying, by the server, the one or more design patterns in the data, and updating, by the server, the user profile based on the identified one or more design patterns.

In some embodiments, the method may include receiving, by the processor, additional data relating to one or more second physical objects. The method may include the additional data being received from a second sensor. The method may include identifying one or more second design patterns in the additional data, the one or more second design patterns based on the one or more second physical objects, and updating the design profile based on the identified one or more second design patterns.

In some embodiments, the user interface element may include a font. The method may include modifying the user interface element by changing the font to a different font prior to display of the modified user interface element.

In some embodiments, the user interface element can include one or more user interface windows with a first shape. The method may include modifying the user interface element further by changing the one or more user interface windows to a second shape.

In some embodiments, the method may include the sensor in the device, and the sensor further includes a camera.

In some embodiments, the method may include updating, by the server, the user profile based on one or more probability deterministic scores. The method may include the probability deterministic scores describing one or more user preferences.

In some embodiments, the method may include receiving, by the server, additional design data comprising one or more additional design patterns. The method may include associating, by the server, the additional design patterns with the user profile, and updating, by the server, the user profile based on associating the additional design patterns with the user profile.

In another example, a system for providing dynamic design of user interface elements may include a server comprising a non-transitory memory, the memory storing software instructions, and one or more processors configured to execute the software instructions. The software instructions may be configured to receive data relating to one or more physical objects, the data including one or more design patterns based on the one or more physical objects; identify the one or more design patterns based on the one or more physical objects; create a user profile, the user profile corresponding to a user; update the user profile, the updated user profile based on the one or more design patterns identified based on the one or more physical objects; receive application data relating to one or more applications running on a device remote from the server; identify a user interface element corresponding to the one or more applications running on the device; modify the user interface element based on the updated user profile; and transmit the modified user interface element to the device. The system may include instructions for causing the modified user interface element to be displayed on a screen of the device, the device running the one or more applications when the modified user interface element is displayed.

In some embodiments, the software instructions may be further configured to receive additional data relating to one or more additional physical objects, the additional data including one or more additional design patterns based on the one or more additional physical objects, and the one or more additional design patterns being different from the one or more design patterns. The software instructions may be configured to identify the one or more additional design patterns based on the one or more additional physical objects. The software instructions may be configured to update the user profile based on the one or more additional design patterns.

In some embodiments, the user interface element can include a font. The software instructions may be further configured to modify the user interface element to change the font to a different font.

In some embodiments, the user interface element can include one or more user interface windows with a first shape. The software instructions may be further configured to modify the user interface element to change the one or more user interface windows to a second shape.

In some embodiments, the software instructions may be further configured to receive additional application data relating to one or more applications running on a device remote from the server. The software instructions may be further configured to identify an additional user interface element corresponding to the one or more applications running on the device, and modify the additional user interface element based on the updated user profile, the updated user profile being updated based on the one or more additional design patterns. The software instructions may be further configured to transmit the modified additional user interface element to the device.

In some embodiments, the software instructions may be further configured to compare the updated user profile to a second user profile, the second user profile being different from the updated user profile. The software instructions may be further configured to update the updated user profile based on comparing the updated user profile to the second user profile.

DETAILED DESCRIPTION OF SEVERAL EMBODIMENTS

Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings and described herein. In the drawings, like reference numerals are sometimes used to designate like structural elements. It should also be appreciated that the depictions in the figures are diagrammatic and not to scale.

The present disclosure relates generally to methods and systems for dynamically designing user interface elements that may be displayed on a user device to reflect a user's personality or other design choices automatically, without a user having to select or configure particular design choices. The personalized design elements may be part of device interface (e.g., operating system specific) or one or more applications etc. executed on the device (e.g., application specific).

A user device can be for example a smartphone, and may include a dynamic design service for communicating with an operating system or an application running on the device and causing changes to user interface elements in the display of a user interface of the user device. In some embodiments, a server in communication with the user device may include the dynamic design service, such that the dynamic design service may send requests to the user device to change user interface elements displayed by an application or the operating system. In other embodiments, the user device may include the dynamic design service, and the service may cause changes to user interface elements and user interfaces of the user device.

A user profile may be created and stored on a user device or on a server. The user profile may be updated based on design preferences detected within data, for example an image of the user's home, vehicle, or clothing, and the image may come from various sources, such as a photo or video captured by the user device or by an image sent to a server from the user device. Thus, in some embodiments, a user may be able to visualize one or more personalized interface elements on his/her device that are determined based on the user's aesthetic design choices, personality, mood or on a whim, which may be determined based on historical image data or dynamically in real-time. For example, a user may request to have one or more interface elements dynamically updated based on an automatic and in real-time interpretation of a furniture style, car style, piece of art, etc. that may be captured in an image using the device or otherwise represented in an image or other sensory input. The user profile may include design preferences, and the design preferences may be updated based upon design preferences identified in data, for example image data. The user design preferences may be determined using one or more various image analysis or machine learning techniques. In this manner, a user does not need to manually customize or alter the appearance of a user interface element of their user device because the dynamic design service, as discussed herein, may implement dynamic design changes to the user interface of the user device. Thus, the present disclosure provides technological improvements to computer techniques that may require a user to select a particular interface design etc. The technological solutions discussed herein include dynamically (and automatically) designing a user interface and user interface elements based on computer analysis of received sensory input.

FIG. 1shows a network for creating and managing user profiles for purposes of generating a dynamic user interface according to an embodiment of the present disclosure. The network may exist in the context of user device102attempting to display an application on a user interface. User device102may be a smartphone, tablet, laptop computer, desktop computer, or other device configured to display information on a screen. User device102may send user data to server device104by network100, for example the Internet or another public network, private network, or combination thereof. Server device104may be a server configured to create a user account, where the user account is stored in memory, for example a user profile database108. The user account may have various attributes associated with the user profile, including for example a name, an application that runs on the user device102, and design attributes that can be modified over time based on data received from user device102or from other sources. Server device104also includes a dynamic design service106that contains logic for generating a dynamic user interface based on a user profile stored in user profile database108. The dynamic design service106can generate dynamic user interfaces based on attributes stored in the user profile. Server device104is depicted as a single server, including a single dynamic design service106and user profile database108for purposes of illustration, but those of ordinary skill in the art will appreciate that server device104may be embodied in different forms for different implementations. For example, server device104may include a plurality of servers.

User device102may send user data to server device104, and server device104may process the user data, which may be for example a request to display a web page, and/or to decide whether to process the request in a typical manner, or to process the request through dynamic design service106. The user data may include data describing the user interface that an application or other software running on user device102is attempting to display. For example, the user data may indicate that the user is attempting to access an application (“app”) running on user device102, or that the user is attempting to view a website via a browser app running on user device102.

While the network shown inFIG. 1shows the user device102in communication with server device104through network100, other arrangements are possible. For example, the logic for server device104, including dynamic design service106and user profile database108, can be stored on user device102itself, such that user device102does not need to access network100in order to obtain data regarding a user profile and/or to access dynamic design service106. In some embodiments, user device102can communicate with server device104over a local network, such as a home network, and in other embodiments, user device102can communicate with server device104over the Internet.

The dynamic design service106logic may differ depending on whether the logic is located on a user device102or on a server device104. For example, if the dynamic design service is located on a user device, the dynamic design service may include logic for communicating with an app, for example through an application programming interface (API), and send requests to modify user interface elements to the app. In other embodiments where the dynamic design service is located on a user device, the service may modify user interface elements based on request(s) received from a server. The dynamic design service may also be implemented to communicate with an operating system of the user device, or communicate with apps installed on the user device. In some embodiments, the dynamic design service may be implemented as part of an app that can be used to directly control the user interface of the app. In other embodiments, the dynamic design service may be implemented at the system level, such that an app installed and running on the user device can communicate with the service through, for example, system calls.

In some embodiments, user device102may include user data, for example transaction data, that may be used by server device104to update a user profile, including design preferences associated with a user profile, based on the user data received from user device102. For example, this data may include data identifying user device102and/or data describing the transaction (e.g., item purchased).

FIG. 2shows a block diagram of an example computing device, for example a computing device configured to function as user device102. For example, user device102may interact with server device104to gather data about a user, including a user profile and/or various attributes associated with the user profile, as described herein. The user device102may include a memory interface202, one or more data processors, image processors, and/or central processing units204(CPU), and a peripherals interface206. The memory interface202, the one or more processors204, and/or the peripherals interface206may be separate components or may be integrated in one or more integrated circuits. The various components in the user device102may be coupled by one or more communication buses or signal lines.

Sensors, devices, and subsystems may be coupled to the peripherals interface206to facilitate multiple functionalities. For example, a motion sensor210, a light sensor212, and a proximity sensor214may be coupled to the peripherals interface206to facilitate orientation, lighting, and proximity functions. Other sensors216may also be connected to the peripherals interface206, such as a global navigation satellite system (GNSS) (e.g., GPS receiver), a temperature sensor, a biometric sensor, magnetometer, or other sensing device, to facilitate related functionalities.

A camera subsystem220and an optical sensor222, for example a charged coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) optical sensor, may be utilized to facilitate camera functions, such as recording photographs and video clips. The camera subsystem220and the optical sensor222may be used to collect images of a user to be used during authentication of a user, e.g., by performing facial recognition analysis. The camera subsystem220and the optical sensor222may also be used to collect images of a user environment, for example the interior and/or exterior of a vehicle or a home, including objects located within such environments.

Communications may be facilitated through one or more wired and/or wireless communication subsystems224, which can include radio frequency receivers and transmitters and/or optical receivers and transmitters. For example, the Bluetooth (e.g., Bluetooth low energy (BTLE)) and/or WiFi communications described herein may be handled by wireless communication subsystems224. The specific design and implementation of the communication subsystems224may depend on the communication network(s) over which the user device102is intended to operate. For example, the user device102may include communication subsystems224designed to operate over a GSM network, a GPRS network, an EDGE network, a WiFi or WiMax network, and a Bluetooth network. For example, the wireless communication subsystems224may include hosting protocols such that the device120can be configured as a base station for other wireless devices and/or to provide a WiFi service.

An audio subsystem226may be coupled to a speaker228and a microphone230to facilitate voice-enabled functions, such as speaker recognition, voice replication, digital recording, and telephony functions. The audio subsystem226may be configured to facilitate processing voice commands, voiceprinting, and voice authentication, for example.

The I/O subsystem240may include a touch-surface controller242and/or other input controller(s)244. The touch-surface controller242may be coupled to a touch surface246. The touch surface246and touch-surface controller242may, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch surface246.

The other input controller(s)244may be coupled to other input/control devices248, such as one or more buttons, rocker switches, mouse wheel, infrared port, USB port, and/or a pointer device such as a stylus. The one or more buttons (not shown) may include a button for volume control of the speaker228and/or the microphone230.

In some implementations, a pressing of the button for a first duration may disengage a lock of the touch surface246; and a pressing of the button for a second duration that is longer than the first duration may turn power to the user device102on or off. Pressing the button for a third duration may activate a voice control, or voice command, module that enables the user to speak commands into the microphone230to cause the device to execute the spoken command. The user may customize a functionality of one or more of the buttons. The touch surface246can, for example, also be used to implement virtual or soft buttons and/or a keyboard.

In some implementations, the user device102may present recorded audio and/or video files, such as MP3, AAC, and MPEG files. In some implementations, the user device102may include the functionality of an MP3 player. The user device102may, therefore, include a 36-pin connector and/or 8-pin connector that is compatible with the iPod. Other input/output and control devices may also be used.

The memory interface202may be coupled to memory250. The memory250may include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and/or flash memory (e.g., NAND, NOR). The memory250may store an operating system252, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks. The operating system252may also be a mobile operating system, such as Android or iOS, when user device102is a mobile phone, tablet, or other device running operating system software designed for mobile applications.

The operating system252may include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, the operating system252may be a kernel (e.g., UNIX kernel). In some implementations, the operating system252may include instructions for performing voice authentication.

FIG. 3is a block diagram of an example server device104that may implement various features and processes as described herein. The server device104may be implemented on any electronic device that runs software applications derived from compiled instructions, including without limitation personal computers, servers, smartphones, media players, electronic tablets, game consoles, email devices, etc. In some implementations, the server device104may include one or more processors302, one or more input devices304, one or more display devices306, one or more network interfaces308, and one or more computer-readable mediums310. Each of these components may be coupled by bus312. While server device104is shown as a separate device from user device102, various elements shown inFIG. 3, for example operating system314, network communications316, dynamic design service318, and application(s)320, can all run on user device102, including the device shown inFIG. 2.

Display device306may be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology. Processor(s)302may use any known processor technology, including but not limited to graphics processors and multi-core processors. Input device304may be any known input device technology, including but not limited to a keyboard (including a virtual keyboard), mouse, track ball, and touch-sensitive pad or display. Bus312may be any known internal or external bus technology used for exchanging communications, for example ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATA and FireWire. Computer-readable medium310may be any medium that participates in providing instructions to processor(s)302for execution, including without limitation, non-volatile storage media, such as optical disks, magnetic disks, flash drives, etc., or volatile media, such as SDRAM and ROM.

Computer-readable medium310may include various instructions314for implementing an operating system (e.g., Mac OS, Windows, Linux). The operating system may be multi-user, multiprocessing, multitasking, multithreading, real-time, and the like. The operating system may perform basic tasks, including but not limited to recognizing input from input device304, sending output to display device306, keeping track of files and directories on computer-readable medium310, controlling peripheral devices (e.g., disk drives, printers, etc.) which can be controlled directly or through an I/O controller, and managing traffic on bus312. Network communications instructions316may establish and maintain network connections (e.g., software for implementing communication protocols, such as TCP/IP, HTTP, Ethernet, telephony, etc.).

Dynamic design service318can include instructions that evaluate user requests to display a user interface as described herein. For example, dynamic design service instructions may analyze a user request to display a user interface, access a user profile, and apply rules and/or logic that may modify a user interface to be displayed in a manner that is more preferable to a user, based at least in part on the user's profile.

Application(s)320may be an application that uses or implements the processes described herein and/or other processes. The processes may also be implemented in operating system314.

The described features may be implemented in one or more computer programs that may be executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program may be written in any form of programming language, for example Objective-C, Java, Swift, and C++, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

To provide for interaction with a user, the features may be implemented on a computer having a display device such as a LCD (liquid crystal display) monitor, or other display devices, such as a CRT (cathode ray tube), for displaying information to the user, and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer.

The features may be implemented in a computer system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination thereof. The components of the system may be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include, e.g., a telephone network, a LAN, a WAN, and the computers and networks forming the Internet. The computer system may include clients and servers. A client and server may generally be remote from each other and may typically interact through a network. The relationship of client and server may arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

One or more features or steps of the disclosed embodiments may be implemented using an API. An API may define one or more parameters that are passed between a calling application and other software code (e.g., an operating system, library routine, function) that provides a service, that provides data, or that performs an operation or a computation. The API may be implemented as one or more calls in program code that send or receive one or more parameters through a parameter list or other structure based on a call convention defined in an API specification document. A parameter may be a constant, a key, a data structure, an object, an object class, a variable, a data type, a pointer, an array, a list, or another call. API calls and parameters may be implemented in any programming language. The programming language may define the vocabulary and calling convention that a programmer will employ to access functions supporting the API. In some implementations, an API call may report to an application the capabilities of a device running the application, such as input capability, output capability, processing capability, power capability, communications capability, etc.

FIGS. 4A and 4Bshow a user device102and an application running on user device102. In both figures, the running application is an e-mail application, and the user device102is a mobile device, such as a smartphone. In other embodiments, the application running on user device102could be a word processor, browser, or other application, and the user device102could be a tablet, laptop computer, desktop computer, or other computing device.

As shown inFIG. 4A, an e-mail application user interface is displayed on the display of user device102. An e-mail application may allow a user to view an inbox (“Inbox”) containing e-mails sent to the user of user device102, as shown inFIG. 4A. In other embodiments, the e-mail application may present a “Sent” user interface containing e-mails that the user of user device102has sent to one or more intended recipients, or the e-mail application may present a “Draft” user interface containing e-mails that the user of user device102has begun drafting, but has not yet sent to one or more intended recipients. The Inbox, as shown in the user interface of device102, includes four messages, message402, message404, message406, and message408. Each of message402, message404, message406, and message408are displayed using a template that includes a particular font, font size, or font weight. Each of the messages shown are displayed using the same font, font size, and font weight, although the information specific to the e-mail may differ, including for example the date of receipt of the e-mail, the subject line of the e-mail, and the sender of the e-mail.

An embodiment of the user interface of the e-mail application is shown inFIG. 4B.FIG. 4Bshows a similar e-mail application, including an Inbox and various messages (message410, message412, message414, and message416). One difference shown inFIG. 4Bis the user interface of the e-mail application, including message410, message412, message414, and message416, are displayed using a different font than the font used to display Inbox and message402, message404, message406, and message408shown inFIG. 4A.

The dynamic design service106of server device104may be configured, for example, to cause a different font to be used for the Inbox text and message410, message412, message414, and message416. The dynamic design service106has access to a user profile stored in user profile database108, and analyzes the user profile to determine one or more attributes relating to the user profile. For example, the user profile may store one or more attributes relating to a design preference. In the context ofFIG. 4B, the design preference may relate to a type of font, including the type of font for applications involving e-mail. The server device104can generate a user interface using the design preference, such as the user interface shown inFIG. 4Bshowing text displayed using a type of font, or alternatively can send information relating to the design preference attribute(s) back to the user device102, which can then generate and present a user interface using the design preference, such as the user interface shown inFIG. 4Bshowing text displayed using a type of font.

One example of a dynamic change in a user interface element based on a user design preference is the use of a different type of font, as shown inFIGS. 4A and 4B. A person of ordinary skill in the art would appreciate that different types of dynamic changes in user interface elements are possible, including changing the font size and the font appearance (e.g., underlined, italicized, or bolded). Further, dynamic changes in user interface elements are not limited to a change relating to a font. Dynamic changes to user interface elements can for example relate to the appearance of a window itself, as shown in further detail inFIGS. 5A and 5B.

FIGS. 5A and 5Bshow a user device102that has multiple windows displayed. InFIG. 5A, window502, window504, and window506are displayed on the user interface of user device102, and inFIG. 5B, window508, window510, and window512are displayed on the user interface of user device102. The windows displayed in bothFIGS. 5A and 5Bcan all be related to the same application, or can relate to different applications. Some embodiments include each window relating to a separate instance of a browser running on the user device102, for example if a user were to open new windows while using the browser. In another embodiment, each window relates to a different application, for example a browser application, an e-mail application, and a game application. These are specific examples of multiple windows running in an environment of the user device102, and the disclosure should not be limited to any particular combination of running applications and windows. In bothFIGS. 5A and 5B, the user device102is a mobile device, such as a smartphone. In other embodiments, the user device102could be a tablet, laptop computer, desktop computer, or other computing device.

As shown inFIG. 5A, a user interface is displayed on the display of user device102, and three windows, window502, window504, and window506, are displayed. Each of window502, window504, and window506are displayed using a template that specifies a particular size and shape. In certain embodiments, the user of the user device102can re-size any of window502, window504, and window506, to have different dimensions, or may select the general area relating to window502or window504to cause the respectively selected window corresponding to the area selected to be brought to the front of the user interface. In the case ofFIG. 5A, each of window502, window504, and window506contain sharp edges.

An embodiment of the user interface is shown inFIG. 5B.FIG. 5Bshows a similar user interface, including multiple windows (window508, window510, and window512). One difference shown inFIG. 5Bis the user interface windows, including window508, window510, and window512, are displayed using soft edges, rather than the sharp edge used to display window502, window504, and window506inFIG. 5A.

The dynamic design service106of server device104may be configured, for example, to change the characteristics of each of window502, window504, and window506. The dynamic design service106has access to a user profile stored in user profile database108, and analyzes the user profile to determine one or more attributes relating to the user profile. For example, the user profile may store one or more attributes relating to a design preference. In the context ofFIG. 5B, the design preference may relate to a softening of the edges of user interface windows. The server device104can generate a user interface using the design preference, such as the user interface shown inFIG. 5Bshowing each of window508, window510, and window512having soft edges, or alternatively can send information relating to the design preference attribute(s) back to the user device102, which can then generate and present a user interface using the design preference, such as the user interface shown inFIG. 5Beach of window508, window510, and window512having soft edges.

One example of a dynamic change in a user interface element based on a user design preference is the use of a different shape for a user interface window, as shown inFIGS. 5A and 5B. A person of ordinary skill in the art would appreciate that different types of dynamic changes in user interface elements are possible, including changing the color scheme of a user interface window, or changing the shape of a user interface window in a manner different from the examples shown inFIGS. 5A and 5B. For example, the edges could be softened less than what is shown inFIG. 5B, or alternatively the edges could be softened more than what is shown inFIG. 5B.

The user device102shown inFIG. 6AandFIG. 6Bprovides an additional example of dynamically designing the user interface of a screen of the user device. InFIG. 6A, a user interface is displayed, and may include an application running on the user device102, such as a stopwatch application. The application's user interface may include a ‘Start’ soft button602and a ‘Stop’ soft button604, where selecting the ‘Start’ button may start the stopwatch (i.e., the time elapsed may start to display seconds, minutes, and hours), and selecting the ‘Stop’ button may stop the stopwatch (i.e., the time elapsed may no longer increase). In the user interface shown inFIG. 6A, the Start button602and Stop button604have sharp edges. In some embodiments, the application defaults to displaying button elements with sharp edges.

FIG. 6Bshows a similar user interface to that shown inFIG. 6A, except that Start button606and Stop button608may include soft, rounded edges. In some embodiments, the dynamic design service106may be configured to modify the user elements corresponding to the Start button606and Stop button608prior to those elements being displayed on the user interface shown inFIG. 6B.

FIG. 7shows a dynamic user interface modification process according to an embodiment of the present disclosure. User device102may perform certain steps of the process shown inFIG. 7, and server device104may perform other certain steps of the process shown inFIG. 7, for example.

At step702, the user device102receives data from a sensor. The sensor could be any of the sensors disclosed inFIG. 2, including motion sensor210, light sensor212, proximity sensor214, or other sensor(s)216, that are capable of capturing information. In some embodiments, the data can be media such as a photograph or a video. The media can be obtained from a sensor and/or from camera subsystem220and optical sensor222. In other embodiments, the user device102may receive data from another source, for example an image downloaded from the Internet, or an image sent by a contact through an e-mail or messages application. The user device102may receive data from other sources as well.

At step704, a device (e.g., user device102) transmits the sensor data to a server (e.g., server device104). In this process, the sensor data may be sent from a user device to a server device through a network, for example the Internet, and in a manner consistent with the structure ofFIG. 1. In other embodiments, this step may not be necessary, for example if the dynamic design service logic is stored on the user device102. In such an embodiment, the user device itself contains the necessary logic to generate or analyze a user profile and dynamically update a user interface element that is displayed on the screen of the user device, and communication with a remote server is not necessary. In some embodiments, a user can select data, including images, presented to the user by a server through the Internet. The server can present images representing different style preferences to the user, and based on the user's selection of one or more images, the server can process the selected images based upon methods and processes described herein to determine one or more aspects of user profile.

At step706, the dynamic design service106/318analyzes the sensor data to identify any design patterns in the data. In some embodiments, the sensor data comprises an image or images of objects found in an environment, such as a vehicle or a home. Using a home environment as an example, the sensor data (e.g., a photograph or video) may contain wall art that includes text displayed in a type of font. In some embodiments, the dynamic design service analyzes the sensor data and identifies the wall art, including the text displayed in a type of font. The dynamic design service can then use this design preference and create an association between the type of font identified from the wall art with a type of font that is used in user interfaces for a user device (e.g., user device102). As another example, the home environment may contain furniture, where the furniture has soft, rounded edges. The dynamic design service may determine this characteristic of the furniture, and may in response change a user interface object, for example a soft button, such that the button has soft, rounded edges. In other embodiments, the dynamic design service may determine that sensor data contains wall art, and that the wall art is associated with a preference for modern art. The dynamic design service may select a font that is associated with the modern art style of the wall art. A user may have various design preferences that can be exhibited in various forms and objects, for example a car, art, clothing, furniture, or other objects. Depending on the design characteristics of such objects, the dynamic design service may identify user interface objects that can be changed to include a characteristic or design preference associated with the design characteristics of such objects.

The dynamic design service can analyze the sensor data, including the furniture, and identify that the furniture has soft, round edges. The dynamic design service can then use this design preference and create an association between the soft, round edges identified from the furniture in the sensor data with user interface elements (e.g., an application window) that can be modified to include soft, round edges, for example. Many different types of objects can be analyzed in sensor data, for example the shape or color of a vehicle, the color scheme of a room, and the shape of furniture and other objects in the sensor data. Further, sensor data can be data relating to Internet activity, for example purchased furniture, clothing, or other consumer goods. In some embodiments, the dynamic design service can analyze such sensor data (e.g., sensor data obtained from Internet activity) and identify design preferences based on that sensor data. The dynamic design service can identify relationships between objects identified in the sensor data. For example, the dynamic design service can identify a consumer item, for example a designer dress, and can identify design preferences based on other user profiles where the user associated with the user profile purchased a similar or identical designer dress.

In some embodiments, the dynamic design service106/318is configured to generate or analyze a user profile using artificial intelligence (AI) or machine learning (ML) technology. The sensor data may be analyzed by AI/ML, and the AI/ML may parameterize or vectorize the detected information in the sensor data, for example. The AI/ML may apply the parameterized data through a model or other programming construct, and may take output relating to parameterized data and create a user profile or user preferences that may be stored in a user profile. In some embodiments, the AI/ML logic may be part of the dynamic design service, and in other embodiments, the AI/ML logic may be its own separate, logical unit.

At step708, the dynamic design service (e.g., dynamic design service106) may create or update the user profile that corresponds to, or is associated with, the user of the user device (e.g., user device102). For example, the dynamic design service can update a user profile stored in a database (e.g., user profile database108). As other example, the dynamic design service can execute on the user device and update a user profile stored in memory local to the user device. In still other embodiments, the user profile is stored in a memory local to a server (e.g., server device104), and the user profile stored in local memory on the server is updated by the dynamic design service. In some embodiments, the user profile is updated based on the sensor data, including the design pattern(s) identified in the sensor data.

At step710, the server (e.g., server device104) receives an updated user profile or user profile data indicative of one or more design preferences or design elements. In this example, the server obtains the most updated version of the user profile or profile data from a database (e.g., user profile database108). In other embodiments, the server maintains the data that is used to update the user profile, such that the server does not need to wait to receive the updated user profile, or alternatively receive confirmation that the user profile stored in the database has been updated. In an alternative embodiment, the user device receives the updated user profile or profile data. In such an embodiment, the dynamic design service (e.g., dynamic design service106) executes on the user device, rather than on the server device. In some embodiments, the user profile may include subprofiles, for example a work profile and a home profile. There also may be other types of subprofiles, including a weekend profile and a dynamic profile (e.g., a limited or experimental profile). For dynamic profiles, some embodiments may allow a user to experiment with different possible dynamic user interface elements by submitting image data and presenting a temporary user interface, where a user can either accept the changes displayed in the temporary user interface, or reject the changes displayed in the temporary user interface. Further, a subprofile may contain other information, including date and time related information. For example, the dynamic design service may present a different user interface on a Monday (e.g., the beginning of a work week), and a different user interface on a Friday (e.g., the end of the work week). As another example, on or near holidays, the dynamic design service may change user interface elements consistent with design preferences associated with a holiday. For example, near St. Patrick's day, the user interface may include green-colored elements.

At step712, the server (e.g., server device104) identifies user interface element(s) to update based on the updated user profile that was previously determined or received by the server. In some embodiments, the dynamic design service logic on server device104contains logic to identify user interface element(s) to modify based on the updated user profile. In other embodiments, the dynamic design service logic executes on user device102, and contains logic for identifying user interface element(s) to modify based on the updated user profile. As an example, the dynamic design service identifies the type of font used in an e-mail application, and based on the updated user profile, the dynamic design service changes the type of font used in the e-mail application to a different font. This example is further shown for example inFIGS. 4A and 4B. A person of ordinary skill in the art would appreciate that the dynamic design service could dynamically change other user interface elements, including changing other user interface elements based on the application running on the user device.

At step714, the identified user interface elements are modified. For example, the dynamic design service identifies the type of font used in an e-mail application, and based on the updated user profile, the dynamic design service changes the type of font used in the e-mail application to a different font. In some embodiments, the type of font may be dynamically determined, and thus may not necessarily include a change from a prior or a predetermined font. This example is further shown for example inFIGS. 4A and 4B. In other embodiments, the dynamic design interface can communicate with the application running on the user device to cause a change in the display of user interface elements. In some embodiments, for example where a user creates a new account, the dynamic design service may apply default user interface elements, and in some embodiments, the user may select a pre-defined user profile with defined design preferences. The pre-defined user profile may be used to create a dynamic user interface before the user profile is updated with data (e.g., image data) obtained from a source.

At step716, the modified user interface elements are displayed on the screen of the user device (e.g., user device102). The displayed modified user interface elements can be for example a change in font, as shown inFIGS. 4A and 4B, or a change in the appearance of windows corresponding to one or more applications running on the device, as shown inFIGS. 5A and 5B.

FIG. 8shows a user registration and dynamic user interface modification process according to an embodiment of the present disclosure. User device102can be used to perform certain steps of the process shown, and server device104can be used to perform other certain steps of the process shown, for example.

At step810, a user, through the user device, registers with the dynamic design service (e.g., dynamic design service106). In some embodiments, the user device is in communication with a server (e.g., server device104) and transmits user information to the server for purposes of registering the user to become eligible for the dynamic design service. In other embodiments, dynamic design service logic is installed on the user device in a manner such that registration is not required.

At step820, a user profile is created. In some embodiments, the user profile, which is also referred to herein as a user design profile, stores various attributes relating to the design preferences of a user. For example, the attributes associated with the user profile can relate to a type of font that the user prefers, or a window shape that the user likes and prefers, or more generally to a user's style or personality or mood that may be invoked in one or more design elements and combinations of design elements. In some embodiments, the dynamic design service (e.g., dynamic design service106) has access to the user profile, and can further modify the user profile based on additional data received from a user device (e.g., user device102).

At step830, the user profile (or user design profile) is linked with a registered user. In some embodiments, the registered user is the user of user device102.

At step840, dynamic design operations can be performed. In some embodiments, the dynamic design operations are performed in a manner consistent with the process shown inFIG. 7. In other embodiments, the dynamic design operations are performed by a dynamic design service (e.g., dynamic design service106). As discussed above, the dynamic design service logic may be stored on a user device (e.g., user device102), or on a server (e.g., server device104). In the embodiment shown inFIG. 8, when a user profile has been created, the dynamic design service can modify a user interface element that is to be displayed in the user interface of a user device (e.g., user device102).

FIG. 9shows a sensor data collection and dynamic user interface modification process900according to an embodiment of the present disclosure. User device102can be used to perform certain steps of the process shown inFIG. 9, and server device104can be used to perform other certain steps of the process shown, for example.

At step910, a device receives sensor data. In some embodiments, a user device (e.g., user device102) receives the sensor data, and in other embodiments, the server (e.g., server device104) receives the sensor data. The sensor data can include for example data relating to media, such as a photograph or video. The sensor data can be obtained from sensors and/or a camera subsystem (e.g., camera subsystem220). In some embodiments, the sensor data is obtained from sensors that are a part of user device102, and in other embodiments, user device102receives the sensor data via a transmission to user device102. In still other embodiments, server device104receives the sensor data via a transmission to server device104.

At step920, design pattern(s) in the sensor data are identified. In some embodiments, the dynamic design service (e.g., dynamic design service106) analyzes the sensor data to identify design pattern(s) in the sensor data. The design patterns can include for example a type of font or a shape of an object found in the sensor data, for example when the sensor data includes a photograph or video. In some embodiments, the dynamic design service is located on a user device (e.g., user device102). In other embodiments, the dynamic design service is located on a server device (e.g., server device104). A person of ordinary skill in the art would appreciate that the design patterns can be identified in different ways. For example, the dynamic device service could use a probability-based method for scoring one or more design patterns identified in the data and updating the user profile based on the profile data of the user profile is updated based on one or more probability deterministic scores before the processor receives the user profile, the probability deterministic scores relating to the one or more user preferences.

At step930, the sensor data, including the design pattern(s) identified in the sensor data, is associated with a user profile (or user design profile). In some embodiments, the sensor data is obtained from a location associated with a user of a user device (e.g., user device102), such that a relationship between the sensor data source and the user of the user device is known. In other embodiments, the dynamic design service logic executes on a server device (e.g., server device104), and the user device (e.g., user device102) in communication with the server device transmits the sensor data and a user device identifier to the server device. The user device identifier can for example be used to associate the design pattern(s) in the sensor data with a user profile. In other embodiments, if no user profile is found that is associated with a user of the user device, then dynamic design service logic requests creation of a new user profile (or new user design profile) that can be used to store attributes relating to the design pattern(s) found in the sensor data.

At step940, a user profile (or user design profile) is updated. In some embodiments, the user profile is updated based on the identified design pattern(s) found in the sensor data. In other embodiments, once the user profile is updated, the dynamic design service (e.g., dynamic design service106) performs logic for dynamically changing a user interface element for an application running on a user device (e.g., user device102) based on the updated user profile.

While various embodiments have been described above, it should be understood that they have been presented by way of example and not limitation. Various changes in form and detail can be made therein without departing from the spirit and scope. For example, other steps may be provided or steps may be eliminated from the described flows, and other components may be added to or removed from the described systems. Accordingly, other implementations are within the scope of the following claims.