REFRIGERATOR

Disclosed herein is a refrigerator including a touchscreen display configured to receive a touch input from a plurality of users and display content in response to the touch input, a storage configured to store content for each of the plurality of users, and a controller configured to sense any one of the plurality of users and display content for the user on the touchscreen display in response to the sensing of the user, wherein the controller displays a different piece of content on the touchscreen display in response to sensing of another user.

BACKGROUND

Embodiments of the present disclosure relate to a refrigerator, and more particularly, to a refrigerator capable of communicating with an external device.

2. Description of the Related Art

Nowadays, a refrigerator includes a display configured to display temperature of a storage compartment and an operation mode of the refrigerator.

Such a display not only uses a graphical user interface for a user to easily acquire image information but also uses a touch panel for the user to intuitively input a control command. In other words, nowadays, a display performs an information input function as well as an information display function.

Furthermore, nowadays, a refrigerator includes a communication module configured to be connected to an external device (for example, a server connected to the Internet).

Through the communication module, the refrigerator may connect to the Internet, acquire various pieces of information from various servers connected to the Internet, and provide various services on the basis of the acquired information. For example, the refrigerator may be linked to a user device and display various pieces of content such as a memo, a schedule, a sketch, and things to do.

In this way, the refrigerator offers wider variety of services to a user through the display and the communication module.

SUMMARY

Therefore, it is an aspect of the present disclosure to provide content customized for each of a plurality of users by using a communication device and a touchscreen display linked to user devices of the plurality of users.

It is another aspect of the present disclosure to provide a refrigerator capable of providing different pieces of content to different users by using a communication device and a touchscreen display.

In accordance with one aspect of the present disclosure, a refrigerator includes a touchscreen display configured to receive a touch input from a plurality of users and display content in response to the touch input, a storage configured to store content for each of the plurality of users, and a controller configured to sense any one of the plurality of users and, display content for the user on the touchscreen display in response to the sensing of the user, wherein the controller displays a different piece of content on the touchscreen display in response to sensing of another user.

The refrigerator may further include a microphone configured to receive a voice signal of the plurality of users and output an electrical signal corresponding to the voice signal, and the controller may identify any one of the plurality of users on the basis of the electrical signal from the microphone.

The content may include memos, recipients of the memos, and senders of the memos, and the controller may display a memo of which a recipient or sender matches the user on the touchscreen display in response to the sensing of the user.

The content may include things to do and performers of the things to do, and the controller may display things to do of which a performer matches the user on the touchscreen display in response to the sensing of the user.

The content may include sketches and writers of the sketches, and the controller may display a sketch of which a writer matches the user on the touchscreen display in response to the sensing of the user.

The content may include schedules, writers of the schedules, and participants in the schedules and the controller may display a schedule of which a writer or a participant matches the user on the touchscreen display in response to the sensing of the user.

The content may include at least one of weather information, schedule information, and news information, and the controller may display a piece of content selected by the user among the weather information, the schedule information, and the news information on the touchscreen display in response to the sensing of the user.

The refrigerator may further include an audio configured to output sound, and the controller may control the audio to output content selected by the user by sound.

The refrigerator may further include a communicator configured to communicate with a user device of each of the plurality of users, and the controller may transmit pieces of content for the plurality of users to the user devices of the plurality of users through the communicator.

The controller may transmit a different piece of content to a user device of another user through the communicator.

In accordance with another aspect of the present disclosure, a refrigerator includes a touchscreen display configured to display an image in response to a touch input of a first user and display content in response to a touch input of a second user, a storage configured to store user information of the first user and user information of the second user, and a controller configured to generate content related to the second user according to the touch input of the first user and display the user information of the second user and the content related to the second user on the touchscreen display.

The controller may further generate content related to the first user according to the touch input of the first user and display the content related to the second user and the content related to the first user on the touchscreen display in response to sensing of the first user or the second user.

The controller may generate content related to the first user according to the touch input of the first user, display the content related to the first user in response to sensing of the first user, and display content related to the second user in response to sensing of the second user.

The refrigerator may further include a communicator configured to communicate with a user device of the first user on the basis of the user information of the first user and communicate with a user device of the second user on the basis of the user information of the second user, and the controller may generate content related to the first user according to the touch input of the first user and transmit the content related to the first user and the content related to second user to the user device of the first user and the user device of the second user.

The refrigerator may further include a communicator configured to communicate with a user device of the first user on the basis of the user information of the first user and communicate with a user device of the second user on the basis of the user information of the second user, and the controller may generate content related to the first user according to the touch input of the first user, transmit the content related to the first user to the user device of the first user, and transmit the content related to the second user to the user device of the second user.

In accordance with still another aspect of the present disclosure, a method of controlling a refrigerator includes receiving a touch input from a first user, generating content related to a second user according to the touch input, and displaying a user image showing the first user, a user image showing the second user, and the content related to the second user.

The method may further include generating content related to the first user according to the touch input of the first user, and displaying the content related to the second user and the content related to the first user in response to sensing of the first user or the second user.

The method may further include generating content related to the first user according to the touch input of the first user, displaying the content related to the first user in response to sensing of the first user, and displaying the content related to the second user in response to sensing of the second user.

The method may further include generating content related to the first user according to the touch input of the first user, and transmitting the content related to the first user and the content related to the second user to a user device of the first user and a user device of the second user.

The method may further include generating content related to the first user according to the touch input of the first user, transmitting the content related to the first user to a user device of the first user, and transmitting the content related to the second user to a user device of the second user.

DETAILED DESCRIPTION

Like reference numerals refer to like elements throughout. Not all elements of embodiments are described herein, and general content in an art to which the disclosure pertains or overlapping content between embodiments will be omitted. Terms such as “part,” “module,” “member,” and “block,” when used herein, may be implemented by software or hardware. According to embodiments, a plurality of “parts,” “modules,” “members,” or “blocks” may be implemented as a single element, or a single “part,” “module,” “member,” or “block” may include a plurality of elements.

Throughout the specification, when a certain part is described as being “connected” to another part, both a case in which the certain part is indirectly connected to the other part as well as a case in which the certain part is directly connected to the other part are included therein, and the indirect connection includes a connection via a wireless communication network.

When a certain part is described as “including” a certain element, this signifies that the certain part may also include another element rather than excluding the other element unless particularly described otherwise.

Throughout the specification, when a certain member is described as being “on” another member, both a case in which still another member is present between the two members as well as a case in which the certain member is in contact with the other member are included therein.

Terms such as “first” and “second” are used to distinguish one element from another element, and an element is not limited by the above-mentioned terms.

A singular expression includes a plural expression unless context clearly indicates otherwise.

Reference numerals for steps are used for convenience of description and are not intended to describe an order of the steps. The steps may be performed in a different order from the stated order unless context clearly describes a specific order.

Hereinafter, the principles and embodiments of the present disclosure will be described with reference to the accompanying drawings.

FIG. 1illustrates an exterior of a refrigerator according to one embodiment.FIG. 2illustrates a front view of the refrigerator according to one embodiment.FIG. 3illustrates a configuration of the refrigerator according to one embodiment.FIG. 4illustrates a touchscreen display included in the refrigerator according to one embodiment.

Referring toFIGS. 1, 2, 3, and 4, a refrigerator1may include a main body10having an open front surface, a storage compartment20formed inside the main body10and configured to keep food refrigerated and/or frozen, and a door30configured to open and close the open front surface of the main body10.

The main body10may form an exterior of the refrigerator1. The main body10may include an inner case11forming the storage compartment20and an outer case12coupled to an outer side of the inner case11to form an exterior thereof. A portion between the inner case11and the outer case12of the main body10may be filled with an insulator (not illustrated) capable of preventing leakage of cold air from the storage compartment20.

The storage compartment20may be partitioned into a plurality of sections by a horizontal partition21and a vertical partition22. For example, as illustrated inFIG. 1, the storage compartment20may be partitioned into an upper storage compartment20a, a lower first storage compartment20b, and a lower second storage compartment20c. A shelf23on which food may be placed and an airtight container24configured to keep food airtight may be disposed in the storage compartment20.

The storage compartment20may be opened and closed by the door30. For example, as illustrated inFIG. 1, the upper storage compartment20amay be opened and closed by an upper first door30aaand an upper second door30ab, the lower first storage compartment20bmay be opened and closed by a lower first door30b, and the lower second storage compartment20cmay be opened and closed by a lower second door30c.

A handle31may be disposed at the door30for the door30to be easily opened and closed. The handle31may be longitudinally formed in a vertical direction along a portion between the upper first door30aaand the upper second door30aband a portion between the lower first door30band the lower second door30c. Because of this, when the door30is closed, the handle31may be seen as being integrally formed with the door30.

As illustrated inFIG. 3, the refrigerator1may include a touchscreen display120, a storage130, a communicator140, a dispenser150, a cooler160, a temperature sensor170, an audio180, a camera190, and a controller110.

The touchscreen display120may include a display panel121configured to display an image, a touch panel122configured to receive a touch input, and a touchscreen controller configured to operate/control the display panel121and the touch panel122.

The display panel121may convert image data received from the controller110through the touchscreen controller123into an optical signal that is visible by a user.

The display panel121may be implemented as a cathode ray tube (CRT) display panel, a liquid crystal display (LCD) panel, a light emitting diode (LED) panel, an organic LED (OLED) panel, a plasma display panel (PDP), a field emission display (FED) panel, and the like. However, the display panel121is not limited thereto, and the display panel121may be implemented as various displays capable of visually displaying an optical image corresponding to image data.

The touch panel122may receive a user's touch input on the touch panel122and output an electrical signal corresponding to the received touch input to the touchscreen controller123.

For example, the touch panel122may sense a user's touch on the touch panel122from a change in an electrical resistance value or a change in capacitance and output an electrical signal corresponding to coordinates of a user's touch point to the touchscreen controller123. The touchscreen controller123may determine the coordinates of the user's touch point on the basis of the electrical signal received from the touch panel122.

The touch panel122may be disposed in front of the display panel121. In other words, the touch panel122may be disposed on a surface on which an image is displayed. As a result, the touch panel122may be formed of a transparent material to prevent distortion of an image displayed on the display panel121.

The touch panel122may be implemented as a resistive film type touch panel or a capacitance type touch panel. However, the touch panel122is not limited thereto and may be implemented as various touch panels capable of sensing a touch or approach of a user and outputting an electrical signal corresponding to coordinates of a sensed touch point or coordinates of a sensed approach point.

The touchscreen controller123may control operations of the display panel121and the touch panel122. For example, as illustrated inFIG. 4, the touchscreen controller123may control the touch panel122to detect coordinates of the user's touch point and control the display panel121to display an optical image corresponding to image data received from the controller110.

According to an embodiment, the touchscreen controller123may determine coordinates of a user's touch point on the basis of an electrical signal output by the touch panel122and output the coordinates of the user's touch point to the controller110. According to an embodiment, the touchscreen controller123may transmit an electrical signal output by the touch panel122to the controller110for the controller110to determine coordinates of a user's touch point.

The touchscreen controller123may include a memory configured to store a program and data for controlling operations of the display panel121and the touch panel122, and a microprocessor configured to perform a computation for controlling the display panel121and the touch panel122according to the program and data stored in the memory. The memory and the processor of the touchscreen controller123may be implemented as separate chips or a single chip.

In this way, the touchscreen display120may receive a touch input from a user and display an image according to the user's touch input. In other words, the touchscreen display120may interact with the user.

The touchscreen display120may be installed at the door30for user convenience. For example, as illustrated inFIG. 2, the touchscreen display120may be installed at the upper second door30ab. Although the touchscreen display120installed at the upper second door30abwill be described below, the installation position of the touchscreen display120is not limited to the upper second door30ab. For example, the touchscreen display120may be installed anywhere such as the upper first door30aa, the lower first door30b, the lower second door30c, and the outer case12of the main body10as long as the touchscreen display120is visible by the user.

The touchscreen display120may have a wake-up function in which the touchscreen display120is automatically activated when a user approaches the touchscreen display120within a predetermined range. For example, when the user approaches the touchscreen display120within a predetermined range, the touchscreen display120may be activated. In other words, the touchscreen display120may be turned on. When a predetermined amount of time elapses after the user deviates from the predetermined range, the touchscreen display120may be deactivated. In other words, the touchscreen display120may be turned off.

The touchscreen display120may display various screens or images. The screens or images displayed on the touchscreen display120will be described in detail below.

The storage130may include a nonvolatile memory configured to store a program or include power so that the program or data stored therein is not lost even when power is cut off. For example, the storage130may include a high capacity flash memory, a solid state drive (SSD)131, or the like.

The storage130may store a control program and control data for controlling an operation of the refrigerator1and various application programs and application data that perform various functions according to a user's input. For example, the storage130may store an operating system (OS) program that manages configurations and resources (software and hardware) included in the refrigerator1, an album application through which an image is displayed, a memo application through which a memo is stored and displayed, a To-do application through which things to do are recorded, a sketch application through which sketching is possible, a scheduler application through which a schedule is managed, a recipe application through which a recipe of a dish is searched for and displayed, an online shopping application through which a product is purchased online, a shopping list application through which a product to be purchased online is managed, a view-inside application through which food stored in the refrigerator1is captured and the captured food is managed, and a morning briefing application through which news, schedules, weathers, and the like are displayed.

The storage130may provide a program and data to the controller110according to a storage control signal of the controller110or store data output from the storage110.

The communicator140may transmit and receive data to and from an external device according to a communication control signal of the controller110. For example, the communicator140may include a local wireless communication module141.

The local wireless communication module141may wirelessly transmit and receive data to and from an external device at a relatively short distance therefrom. For example, the local wireless communication module141may communicate with an external device using a communication standard such as wireless fidelity (WiFi), Bluetooth, and ZigBee.

The local wireless communication module141may include an antenna configured to transmit a wireless signal to a free space or receive a wireless signal from the free space, a modulator/demodulator configured to modulate data to be transmitted or demodulate a received wireless signal, and the like.

The dispenser150may discharge water or ice according to a user's input. In other words, the user may directly discharge water or ice to the outside through the dispenser150without opening the door30.

The dispenser150may include a dispenser lever151configured to receive a user's discharge command, a dispenser nozzle152configured to discharge water or ice, and a dispenser display panel153configured to display an operational state of the dispenser150.

The dispenser150may be installed at an outer side of the door30or the main body10. For example, as illustrated inFIG. 2, the dispenser150may be installed at the upper first door30aa. Although the dispenser150installed at the upper first door30aawill be described below, the installation position of the dispenser150is not limited to the upper first door30aa. The dispenser150may be installed anywhere such as the upper second door30ab, the lower first door30b, the lower second door30c, and the outer case12of the main body10as long as the user is able to discharge water or ice through the dispenser150.

A cavity150arecessed to form a discharge space of water or ice may be disposed at the door30, and the dispenser nozzle152and the dispenser lever151may be disposed in the cavity150a. Water or ice may be discharged from the dispenser nozzle152when the user presses the dispenser lever151.

The cooler160may supply cold air to the storage compartment20. Specifically, the cooler160may use evaporation of a refrigerant to maintain temperature of the storage compartment20within a predetermined range.

The cooler160may include a compressor161configured to compress a gaseous refrigerant, a condenser162configured to change a phase of the compressed gaseous refrigerant to a liquid state, an expander163configured to decompress the liquid refrigerant, and an evaporator164configured to change a phase of the decompressed liquid refrigerant to a gaseous phase.

The cooler160may use a phenomenon in which thermal energy is absorbed from surrounding air while the phase of the liquid refrigerant is changed to a gaseous state to cool air in the storage compartment20.

However, the cooler160is not limited to including the compressor161, the condenser162, the expander163, and the evaporator164.

For example, the cooler160may also include a Peltier element using the Peltier effect. The Peltier effect refers to a phenomenon in which, when a current is applied to a contact surface at which two different metals come into contact with each other, an exothermic phenomenon occurs at one metal and an endothermic phenomenon occurs at the other metal. The cooler160may user the Peltier element and cool the air in the storage compartment20.

As another example, the cooler160may include a magnetic cooling device using the magneto-caloric effect. The magneto-caloric effect refers to a phenomenon in which a specific material (magneto-caloric material) dissipates heat when the specific material (magneto-caloric material) is magnetized, and the specific material (magneto-caloric material) absorbs heat when the specific material (magneto-caloric material) is demagnetized. The cooler160may use the magnetic cooling device to cool air in the storage compartment20.

The temperature sensor170may include inner temperature sensors171disposed inside the storage compartment20and configured to sense temperature inside the storage compartment20.

The inner temperature sensors171may be respectively installed in a plurality of storage compartments20a,20b, and20c, sense temperatures inside the storage compartments20a,20b, and20c, and output electrical signals corresponding to the sensed temperatures to the controller110. Each of the inner temperature sensors171may include a thermistor whose electrical resistance changes according to temperature.

The audio180may include a speaker181configured to convert an electrical signal received from the controller110into an acoustic signal and output the acoustic signal, and a microphone182configured to convert an acoustic signal into an electrical signal and output the electrical signal to the controller110.

The camera190may include an imager191configured to capture an image and convert the captured image into an electrical signal, and an imager controller192configured to process an electrical signal output by the imager191and generate image data.

The imager191may include a plurality of photodiodes configured to convert an optical signal into an electrical signal, and the plurality of photodiodes may be two-dimensionally disposed. The imager191may include a charge-coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor in which the plurality of photodiodes are two-dimensionally disposed.

The imager controller192may control an operation of the imager191and generate image data from an electrical signal output by the imager191. The imager controller192may output the image data to the controller110.

The camera190may be installed inside the storage compartment20and acquire an image of food stored in the storage compartment20. The camera190may output the image of the food to the controller110.

The controller110may include a memory112configured to store a program and data for controlling an operation of the refrigerator1, and a processor111configured to generate a control signal for controlling the operation of the refrigerator1according to the program and data stored in the memory112. The processor111and the memory112may be implemented as separate chips or a single chip.

The memory112may temporarily store a user's touch input data input through the touchscreen display120, storage data stored in the storage130, communication data received through the communicator140, and/or temperature data sensed through the temperature sensor170. For example, the memory112may store a screen and/or an image to be displayed on the touchscreen display120and may store coordinates of a user's touch input through the touchscreen display120and a control command corresponding to the user's touch input.

The memory112may include a volatile memory such as a static random access memory (S-RAM), a dynamic RAM (D-RAM), and the like configured to temporarily store data. Also, the memory112may include a nonvolatile memory such as a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), and the like configured to store data for a long period.

The processor111may include various types of logic circuits and arithmetic circuits, process data according to a program provided from the memory112, and generate a control signal according to a result of the processing.

For example, the processor111may process a touch input that is input through the touchscreen display120and determine a user's control command according to coordinates of the touch input. Also, the processor111may control the communicator140to acquire information corresponding to the user's control command and control the touchscreen display120to display a screen and/or an image corresponding to the user's control command.

The processor111may process input data that is input through the touchscreen display120and transmit the input data to an external device through the communicator140. Also, the processor111may process communication data received through the communicator140and display an image indicated by the communication data on the touchscreen display120.

In this way, the controller110may control an operation of the refrigerator1according to a user's input received through the touchscreen display120and/or a program and data stored in the storage130.

Although the touchscreen display120, the storage130, the communicator140, the dispenser150, the cooler160, the temperature sensor170, the audio180, and the controller110included in the refrigerator1have been described above, a new configuration may be added or some of the above configurations may be omitted as necessary.

An operation of the refrigerator1which will be described below may be viewed as being performed by control of the controller110.

FIG. 5illustrates an example in which the refrigerator according to one embodiment communicates with external devices.

The refrigerator1may communicate with various external electronic devices through the communicator140.

For example, as illustrated inFIG. 5, the refrigerator1may connect to an access point (AP) through the communicator140. Specifically, the refrigerator1may connect to an AP using a wireless communication standard such as Wi-Fi™ (IEEE 802.11), Bluetooth™ (IEEE 802.15.1), ZigBee (IEEE 802.15.4), or the like.

The AP may be referred to by various terms such as “hub,” “router,” “switch,” and “gateway” and may be connected to a wide area network (WAN) such as the Internet.

Various home appliances, other than the refrigerator1, such as an air conditioner, a washing machine, an electric oven, a microwave oven, a robot cleaner, a security camera, an electric lamp, and a television may also be connected to the AP. The home appliances connected to the AP may form a local area network (LAN).

The AP may connect the LAN formed by the home appliances connected to the AP to the WAN such as the Internet.

A service server3configured to provide a predetermined service to the refrigerator1and user devices2(2aand2b) may be connected to the WAN.

The service server3may provide various services to the refrigerator1. For example, the service server3may manage account information of a user of the refrigerator1and store a picture, a video, a memo, a schedule, things to do, a sketch, and the like input by the user of the refrigerator1. The service server3may search for a recipe of a dish in response to a request from the refrigerator1, provide the recipe to the refrigerator1, and transmit a shopping list of items to be purchased online to an online shopping mall server.

The service server3may transmit a picture, a video, a memo, a schedule, things to do, a sketch, a recipe, a shopping list of items, and the like received from the refrigerator1to the user devices2(2aand2b).

The user devices2(2aand2b) may connect to the WAN through a mobile communication service provided by a mobile carrier or connect to the WAN through the AP.

The user devices2(2aand2b) may connect wirelessly or via a wire to the WAN and communicate with other devices such as the refrigerator1. The user devices2(2aand2b) may be portable devices that may be carried by a user or fixed devices that are difficult to carry. For example, the user devices2(2aand2b) may be referred to by various terms such as mobile phones, cellular phones, smartphones, personal communication services (PCS), personal digital assistants (PDAs), computers, notebooks, personal computers (PCs), laptops, and desktops.

Hereinafter, a configuration and an operation of the user devices2(2aand2b) will be described.

FIG. 6illustrates an exterior of a user device according to one embodiment.FIG. 7illustrates a configuration of the user device according to one embodiment.

The user device2may include a box-shaped main body200, a screen201disposed at a front surface of the main body200, and input buttons202,203, and204disposed at the front surface or side surfaces of the main body200.

The main body200may form an exterior of the user device2and accommodate and protect various configurations included in the user device2which will be described below.

The screen201may receive a user's input and display an image according to the user's input.

The input buttons202,203, and204may receive a predetermined command from a user. For example, the input buttons202,203, and204may include a power button202for turning the user device2on and off, a volume control button203for controlling a volume of sound being output by the user device2, and a home button204for changing an image displayed on the screen201of the user device2to a home screen.

The user device2may include a touchscreen display220, a storage230, a communicator240, a camera250, an audio260, and a controller210.

The touchscreen display220may form the screen201of the main body200. The touchscreen display220may include a display panel221configured to display an image, a touch panel222configured to receive a touch input, and a touchscreen controller223configured to operate/control the display panel221and the touch panel222.

The display panel221, the touch panel222, and the touchscreen controller223may be different from the display panel121, the touch panel122, and the touchscreen controller123, described with reference toFIGS. 3 and 4, in terms of sizes and installation positions but may have the same functions as the display panel121, the touch panel122, and the touchscreen controller123.

The storage230may include a nonvolatile memory configured to store a program or include power so that the program or data stored therein is not lost even when power is cut off. For example, the storage230may include a high capacity flash memory, an SSD231, or the like.

The storage230may store a control program and control data for controlling an operation of the user device2and various application programs and application data that perform various functions according to a user's input. For example, the storage230may store an OS program, an album application, a memo application, a To-do application, a sketch application, a scheduler application, an online shopping application, a shopping list application, a view-inside application, and the like.

The storage230may provide a program and data to the controller210according to a storage control signal of the controller210or store data output from the controller210.

The communicator240may transmit and receive data to and from an external device according to communication control of the controller210. For example, the communicator240may include a local wireless communication module241and a mobile wireless communication module242configured to wirelessly transmit communication data to an external device and wirelessly receive communication data from the external device.

The local wireless communication module241may wirelessly transmit and receive data to and from an external device at a relatively short distance therefrom. For example, the local wireless communication module241may communicate with an external device using a communication standard such as WiFi, Bluetooth, and ZigBee.

The mobile wireless communication module242may wirelessly transmit and receive data to and from a base station and connect to the WAN through the base station or communicate with an external device through the base station. For example, the mobile wireless communication module242may communicate using a communication standard such as time division multiple access (TDMA), code DMA (CDMA), wide CDMA (WCDMA), CDMA2000, a wireless broadband (Wibro), World Interoperability for Microwave Access (WiMAX), long term evolution (LTE), and/or wireless broadband evolution.

Each of the local area wireless communication module241and the mobile wireless communication module242may include an antenna configured to transmit a wireless signal to a free space or receive a wireless signal from the free space, a modulator/demodulator configured to modulate data to be transmitted or demodulate a received wireless signal, and the like.

The camera250may include an imager251configured to capture an image and convert the captured image into an electrical signal, and an imager controller252configured to process an electrical signal output by the imager251and generate image data.

The imager251may include a plurality of photodiodes configured to convert an optical signal into an electrical signal, and the plurality of photodiodes may be two-dimensionally disposed.

The imager controller252may control an operation of the imager251and generate image data from an electrical signal output by the imager251. The imager controller252may output the image data to the controller210.

The camera250may be installed outside the main body201, acquire an image of an external subject to be captured, and output the acquired image to the controller210.

The audio260may include a speaker261configured to convert an electrical signal received from the controller210into an acoustic signal and output the acoustic signal, and a microphone262configured to convert an acoustic signal into an electrical signal and output the electrical signal to the controller210.

The controller210may include a memory212configured to store a program and data for controlling an operation of the user device2, and a processor211configured to generate a control signal for controlling the operation of the refrigerator1according to the program and data stored in the memory212. The processor211and the memory212may be implemented as separate chips or a single chip.

The memory212may temporarily store a user's touch input data input through the touchscreen display220, storage data stored in the storage230, communication data received through the communicator240, and/or image data acquired through the camera250.

The memory212may include a volatile memory such as an S-RAM, a D-RAM, and the like configured to temporarily store data and a nonvolatile memory such as a ROM, an EPROM, an EEPROM, and the like configured to store data for a long period.

The processor211may include various types of logic circuits and arithmetic circuits, process data according to a program provided from the memory212, and generate a control signal according to a result of the processing.

For example, the processor211may process a touch input that is input through the touchscreen display220and control the touchscreen display220to display a screen and/or an image in response to the touch input. The processor211may also transmit data to an external device through the communicator240or receive data from the external device in response to the touch input.

In this way, the controller210may control an operation of the user device2according to a user's input received through the touchscreen display220and/or a program and data stored in the storage230.

The operation of the user device2, which will be described below, may be viewed as being performed by control of the controller210.

Although the touchscreen display220, the storage230, the communicator240, the camera250, the audio260, and the controller210included in the user device2have been described above, a new configuration may be added or some of the above configurations may be omitted as necessary.

Hereinafter, operations of the refrigerator1and the user device2will be described.

FIG. 8illustrates a home screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

When the touchscreen display120of the refrigerator1is activated, the controller110may display a home screen on the touchscreen display120. Various graphic user interface (GUI) elements may be displayed on the home screen. The GUI elements may include a widget capable of directly interacting with a user on a first home screen2010and an icon for executing an application stored in the storage130.

For example, the refrigerator1may display the first home screen2010illustrated inFIG. 8on the touchscreen display120.

A temperature setting GUI element2011may be displayed on the first home screen2010. The temperature setting GUI element2011may include a run button2011afor running a temperature setting application through which temperature of the upper storage compartment20a(seeFIG. 1), temperature of the lower first storage compartment20b(seeFIG. 1), and a function (refrigerating or freezing) of the lower second storage compartment20c(seeFIG. 1) are set, a refrigerating temperature region2011bconfigured to display the temperature of the upper storage compartment20a(seeFIG. 1), a freezing temperature region2011cconfigured to display the temperature of the lower first storage compartment20b(seeFIG. 1), and refrigerating/freezing setting region2011dconfigured to display a function (refrigerating or freezing) of the lower second storage compartment20c(seeFIG. 1).

A refrigerator setting GUI element2012may be displayed on the first home screen2010. In response to a touch input to the refrigerating setting GUI element2012, a refrigerator setting application for controlling operations of the touchscreen display120, the dispenser150, the cooler160, the audio180, and the like may be run. Particularly, the user may add or delete a user using the refrigerator1by using the refrigerator setting application.

A scheduler GUI element2013may be displayed on the first home screen2010. The scheduler GUI element2013may include a run button2013afor running a scheduler application that manages a schedule and a schedule region2013bconfigured to display schedules of users according to date and time.

A memo GUI element2014may be displayed on the first home screen2010. The memo GUI element2014may include a run button2014afor running a memo application through which a memo is stored and displayed and a memo region2014bconfigured to display a memo input by the users. Particularly, a plurality of memos input by different users may be separately displayed in the memo region2014b.

A To-do GUI element2015may be displayed on the first home screen2010. The To-do GUI element2015may include a run button2015afor running a To-do application through which things to do are recorded, a performer display region2015bconfigured to display a performer of things to do, and a things-to-do display region2015cconfigured to display things to do input by the users. Particularly, things to do of different users may be separately displayed in the things-to-do display region2015c.

A sketch GUI element2016may be displayed on the first home screen2010. The sketch GUI element2016may include a run button2016afor running a sketch application through which sketching is possible, a writer display region2016bconfigured to display a writer of a sketch, and a sketch display region2016cconfigured to display a sketch of a user. Particularly, sketches of different users may be separately displayed in the sketch display region2016c.

A view-inside GUI element2017may be displayed on the first home screen2010. In response to a touch input to the view-inside GUI element2017, a view-inside application through which food stored in the refrigerator1is captured and displayed may be run.

A shopping list GUI element2018may be displayed on the first home screen2010. In response to a touch input to the shopping list GUI element2018, a shopping list application though which products to be purchased online are managed may be run.

A morning briefing GUI element2019may be displayed on the first home screen2010. In response to a touch input to the morning briefing GUI element2019, a morning briefing application for morning briefing may be run.

As described above, GUI elements for displaying major information related to an operation of the refrigerator1or running various applications may be displayed on the home screen of the refrigerator1.

However, the first home screen2010illustrated inFIG. 8is merely an example of a home screen, and the refrigerator1may display various forms of home screens according to the user's settings and the like. The GUI elements displayed on the home screen are also not limited to those illustrated inFIG. 8.

Hereinafter, operations of the refrigerator1performed by the GUI elements on the home screen will be described.

FIG. 9illustrates a method of adding a user to the refrigerator according to one embodiment.FIGS. 10, 11, and 12illustrate screens for adding a user by using the method of adding a user illustrated inFIG. 9.

The refrigerator1may be used by a plurality of users. For example, when the refrigerator1is installed at home, all family members may use the refrigerator1. Consequently, several people (e.g., family members) may be registered to the refrigerator1as users.

According to an embodiment, all of the users may belong to the same class or the users may be classified into different classes. For example, the users may be classified as a main user and sub-users. The main user may represent the users of the refrigerator1and may use most of the functions of the refrigerator1. The main user's account may be registered to the service server3to register the refrigerator1to the service server3. The sub-users may user various functions of the refrigerator1together with the main user. An account of each of the sub-users may be registered to the service server3as a sub-user.

A user registration method (1000) in which the refrigerator1registers a new user will be described with reference toFIGS. 9, 10, 11, and 12.

The refrigerator1determines whether to add a user (1010).

The refrigerator1may determine whether to add a user according to a user's touch input through the touchscreen display120.

Users (a main user and/or sub-users) may run the refrigerator setting application to add a new user (sub-user). For example, the user may input a touch input to the refrigerator setting GUI element2012displayed on the first home screen2010.

Through the refrigerator setting application, the user may select a user adding menu from a screen displayed on the touchscreen display120. When the user adding menu is selected, the refrigerator1may determine to add a user.

When adding a user, the refrigerator1collects user information of the new user (1020).

The refrigerator1may collect the user information of the new user through the touchscreen display120.

For example, the refrigerator1may display a user information screen2110as illustrated inFIG. 10.

The user information screen2110may display pieces of information on a plurality of pre-registered users USER1, USER2, USER3, and USER4. The user information screen2110may include pieces of identification information2111of the plurality of pre-registered users USER1, USER2, USER3, and USER4, a user registration button2112for registering a new user, and a back button2113for returning to a previous screen.

The pieces of identification information2111of the plurality of users USER1to USER4may include an identification image2111ashowing each of the plurality of users USER1to USER4and an identification text2111b(a name, a nickname, an abbreviated name, etc.) indicating each of the plurality of users.

The identification image2111aand the identification text2111bmay indicate a corresponding user. The users of the refrigerator1may be differentiated from other users through the identification images2111aand the identification texts2111b.

When a touch input is made to the user registration button2112, the refrigerator1may display the user registration screen for registering a new user on the touchscreen display120.

For example, the refrigerator1may display a first user registration screen2120and a second user registration screen2130respectively illustrated inFIGS. 11 and 12.

The identification text and the identification image for identifying a user may be input to the first user registration screen2120. The first user registration screen2120may include an identification text input region2121for inputting a new user's identification text and an identification image selection region2122for selecting the new user's identification image. The user may input a name, a nickname, an abbreviated name, or the like indicating a new user in the identification text input region2121, and any one of a plurality of images2122adisplayed in the identification image selection region2122may be selected.

Pieces of contact information for contacting a new user may be input to the second user registration screen2130. The second user registration screen2130may include a mail address input region2131for inputting the new user's electronic mail address and a phone number input region2132for inputting the new user's phone number. The mail address input to the mail address input region2131may be used as an account identification (ID) that represents the new user. Also, the phone number input to the phone number input region2132may be used to identify a user device (e.g., a cellular phone) of the new user.

When the new user previously generated a user account using a mail address and a phone number in the service server3, the user device2of the new user and the refrigerator1may be linked to each other and be operated. For example, the new user may change a memo, things to do, a sketch, a schedule, and the like of the new user by using the refrigerator1as well as the user device2.

A mail address and/or a phone number input through the second user registration screen2130are not essentially required. For example, when the new user is a child, the new user may not have a mail address and/or a phone number, and a mail address and/or a phone number may not be input.

The refrigerator1which collected the user information of the user transmits the user information to the service server3(1030).

The refrigerator1may transmit user information including an identification image, an identification text, a mail address and/or a phone number of the user to the service server3through the communicator140.

The service server3which received the user information of the user transmits a registration request message for checking user registration to the user device2of the user (1040).

To check the user information received from the refrigerator1and prevent the user account from being stolen, the service server3may transmit the registration request message for checking user registration to the user device2.

For example, the service server3may transmit a text message (a short message service (SMS) message) to the user device2on the basis of the phone number included in the user information. The text message may display a message indicating that a user registration is requested through the refrigerator1or may request for an input of an authentication code for authenticating a user.

As another example, the service server3may transmit an electronic mail to the user device2on the basis of the mail address included in the user information. The electronic mail may display a message indicating that a user registration is requested through the refrigerator1or may request for an input of an authentication code for authenticating a user.

The user may check the user registration message (a text message and/or an electronic mail) received by the user device2and input an authentication code. The authentication code may be a preset password of the user account or a one-time password (OTP) arbitrarily generated by the service server3and transmitted to the user device2and the refrigerator1.

The user device2transmits a registration approval message that approves of the registration of the new account to the refrigerator1to the service server3(1050).

When the authentication code is input, the user device2may transmit the authentication code to the service server3, and the service server3may determine whether the authentication code input by the user matches a previously stored/generated authentication code. When the authentication code input by the user matches the previously stored/generated authentication code, user authentication is completed.

When the user authentication using the authentication code is completed, the user device2may transmit the registration approval message that approves of the registration of the new account to the refrigerator1to the service server3through the communicator240.

The above-described user authentication operations (1040and1050) by the user device2may be selective operations. For example, although the above user authentication operations (1040and1050) are performed when the user inputs a mail address and a phone number to the refrigerator1, the above user authentication operations (1040and1050) may not be performed when the user does not input a mail address and a phone number to the refrigerator1.

The service server3which received the registration approval message registers a new user (1060).

The service server3may register the new user on the basis of the user information received from the refrigerator1. For example, the service server3may additionally store user information of the new user in a list of users of the refrigerator1.

When the user does not input a mail address and a phone number to the refrigerator1, the service server3may register the new user in response to reception of the user information from the refrigerator1.

Then, the service server3transmits a registration completion message indicating that the user registration is completed to the refrigerator1(1070).

In response to reception of the registration completion message from the service server3, the refrigerator1registers the new user (1080).

The refrigerator1may register the new user on the basis of the user information input through the touchscreen display120. For example, the refrigerator1may additionally store the user information of the new user in the list of users.

As described above, the refrigerator1may collect user information of a new user and register the new user as the user of the refrigerator1on the basis of the collected user information. Specifically, the refrigerator1may store the user information of the new user in the list of users.

FIG. 13illustrates a method of displaying another screen with respect to another user by the refrigerator according to one embodiment.FIG. 14illustrates another home screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

As described above, the refrigerator1may be used by a plurality of users and register the plurality of users.

Further, the refrigerator1may display a different screen for each of the plurality of users. For example, a home screen displayed for a first user and a home screen displayed for a second user may be different from each other.

A method in which the refrigerator1displays a different screen for a different user (1100) will be described with reference toFIGS. 13 and 14.

The refrigerator1identifies a user (1110).

The refrigerator1may identify a user who is currently using the refrigerator1. For example, when the user approaches the refrigerator1, the refrigerator1may distinguish the user approaching the refrigerator1from a plurality of registered users on the basis of the user's voice, shape, motion, and the like.

The refrigerator1may identify the user approaching the refrigerator1by using various methods.

For example, the refrigerator1may identify a current user of the refrigerator1by using voice recognition or speaker recognition. The voice recognition or speaker recognition refers to identifying a user on the basis of features of an individual's voice.

The refrigerator1may analyze a spectrum of the user's voice during registration of the user and store the spectrum information. Specifically, the refrigerator1may acquire the user's voice using the audio180during the registration of the user and extract reference spectrum information of the user's voice. The spectrum information of the voice may indicate a magnitude of a signal according to a frequency of the voice. The refrigerator1may store the reference spectrum information of the user's voice together with the user's identification name, identification image, mail address, and/or phone number.

After the user is registered, the refrigerator1may acquire the user's voice from the surrounding using the audio180, extract spectrum information from the acquired voice, and compare the extracted spectrum information of the voice with a plurality of pieces of pre-stored reference spectrum information. When a piece of reference spectrum information matching the extracted spectrum information of the voice is found, the refrigerator1may determine the user having the found reference spectrum information as the current user. For example, when the extracted spectrum information of the voice matches reference spectrum information of the first user USER1, the refrigerator1may determine the first user USER1as the current user.

As another example, the refrigerator1may identify the current user of the refrigerator1using facial recognition. Facial recognition refers to identifying a user on the basis of an individual's facial features.

For facial recognition, the refrigerator1may further include a camera (not illustrated). The refrigerator1may capture a user's face during registration of the user, analyze a facial image, and store reference feature information. The facial feature information may include positions of the eyes, the nose, the mouth, and the like and/or distances between the eyes, the nose, and the mouth. The refrigerator1may store the reference facial feature information together with the user's identification name, identification image, mail address, and/or phone number.

After the user is registered, the refrigerator1may capture the user's face placed in front of the refrigerator1using the camera and extract facial feature information from the captured facial image. The refrigerator1may compare the extracted facial feature information with a plurality of pieces of previously stored reference feature information and determine a user having reference feature information that matches the extracted facial feature information as the current user.

As another example, the refrigerator1may identify the current user of the refrigerator1using fingerprint recognition, iris recognition, and/or vein recognition. Fingerprint recognition refers to identifying a user on the basis of the shape of a fingerprint formed at an individual's finger or palm, and iris recognition refers to identifying a user on the basis of the shape of wrinkles formed at an individual's iris. Vein recognition refers to identifying a user on the basis of formation of a vein in the back of an individual's hand or palm.

For fingerprint recognition, iris recognition, and vein recognition, the refrigerator1may further include a scanner (not illustrated) using infrared rays. The refrigerator1may acquire a fingerprint image, an iris image, and/or a vein image of the user during registration of the user and extract reference feature information from the fingerprint image, iris image, and/or vein image. Feature information of a fingerprint, an iris, and/or a vein may include whether the fingerprint, the iris, and/or the vein have distinguishing shapes and positions thereof.

After the user is registered, the refrigerator1may acquire the user's fingerprint image, iris image, and/or vein image using the scanner and extract feature information of the fingerprint, the iris, and/or the vein. The refrigerator1may compare the extracted feature information of the iris and/or vein with pre-stored reference feature information and determine a user having reference feature information matching the extracted feature information of the iris and/or vein as the current user of the refrigerator1.

Then, the refrigerator1displays a home screen of the identified user on the touchscreen display120(1120).

Each of the plurality of users may set his or her home screen, and, after a user is identified, the refrigerator1may display a home screen of the identified user.

For example, the first user USER1may set the above-described first home screen2010illustrated inFIG. 8as his or her home screen. When the first user USER1is determined as the current user, the refrigerator1may display the first home screen2010illustrated inFIG. 8on the touchscreen display120.

The second user USER2may set a second home screen2020illustrated inFIG. 14as his or her home screen. When the second user USER2is determined as the current user, the refrigerator1may display the second home screen2020on the touchscreen display120.

Although the second home screen2020may include a temperature setting GUI element2021, a refrigerator setting GUI element2022, a scheduler GUI element2023, a memo GUI element2024, a To-do GUI element2025, a sketch GUI element2026, a view-inside GUI element2027, a shopping list GUI element2028, and a morning briefing GUI element2029, the arrangement of the GUI elements2021to2029of the second home screen2020may be different from that of the GUI elements2100to2019of the first home screen2020.

In other words, the refrigerator1may display the first home screen2010for the first user USER1and display the second home screen2020, different from the first home screen2010, for the second use USER2, different from the first user USER1.

As described above, the refrigerator1may identify the current user and display a different home screen for a different user.

FIG. 15illustrates a method of generating and displaying a memo by the refrigerator according to one embodiment.FIGS. 16, 17, 18, and 19illustrate examples of screens for generating and displaying a memo according to the method of generating and displaying a memo illustrated inFIG. 15.

The users may input a memo using the memo application of the refrigerator1, and the refrigerator1may display the memo input by the users.

A method of generating/displaying a memo (1200) in which the refrigerator1generates and displays a memo will be described with reference toFIGS. 15, 16, 17, 18, and 19.

The refrigerator1determines whether to add a memo (1210).

The refrigerator1may determine whether to add a memo according to the user's touch input through the touchscreen display120. The users may run the memo application to add a memo. For example, the user may input a touch input for the memo GUI elements2014and2024displayed on the home screens2010and2020. When the touch inputs for the memo GUI elements2014and2024are made, the refrigerator1may determine to add a memo.

When adding a memo, the refrigerator1receives a new memo (1220) from a user and receives a recipient and a sender of the memo (1230) from the user.

The refrigerator1may receive a new memo from a user via the touchscreen display120. The user may generate a memo and transmit the memo to a specific user. For example, the first user USER1may generate a memo and transmit the memo to the fourth user USER4.

The refrigerator1may display a memo display screen2210as illustrated inFIG. 16. A first memo2211and a second memo2212input by the users may be displayed on the memo display screen2210. Each of the memos2211and2212may include content2211aof the memo, a recipient2211bof the memo, and a sender2211cof the memo.

A memo adding button2213for adding a memo and a memo editing button2214for editing previously input memos may be displayed on the memo display screen2210. The user may touch the memo adding button2213to add a new memo and touch the memo editing button2214for editing an existing memo. The user may touch the memos2211and2212displayed on the memo display screen2210to edit each of the memos.

When a touch input is made on the memo adding button2213, the refrigerator1may display a memo generation screen2220as illustrated inFIG. 17on the touchscreen display120.

The memo generation screen2220may include a memo display region2221in which a memo input by the user is displayed, a tool selection region2222for selecting a tool for generating a memo, and a tool display region2223in which a tool selected from the tool selection region2222is displayed.

A memo2221ainput by the user using a tool selected from the tool selection region2222may be displayed in the memo display region2221.

Tools2222ato2222gthat the user may user to generate a memo may be displayed in the tool selection region2222. The tools2222ato2222gmay include a recipient/sender input tool2222a, a text input tool2222b, a sound input tool2222c, a drawing tool2222d, a sticker input tool2222e, a picture input tool2222f, and a clearing tool2222g.

A tool selected from the tool selection region2222may be displayed in the tool display region2223. For example, when the text input tool2222bis selected, a keyboard for inputting a text, a number, and/or a symbol may be displayed in the tool display region2223, and when the sound input tool2222cis selected, a record button to start recording may be displayed in the tool display region2223. When the drawing tool2222dis selected, various drawing tools such as a pen, a brush, and a spray may be displayed in the tool display region2223, and when the sticker input tool2222eis selected, stickers that may be input to the tool display region2223may be displayed.

When the recipient/sender input tool2222ais selected, a recipient selection region2223afor selecting a recipient of a memo and a sender selection region2223bfor selecting a sender of the memo may be displayed in the tool display region2223. The plurality of users USER1to USER5may be displayed in each of the recipient selection region2223aand the sender selection region2223b, and the user may select a recipient of a memo and a sender of the memo from the plurality of users USER1to USER5.

When a memo is added, the refrigerator1displays the added memo (1240).

When the user adds a memo through the memo generation screen2220, the refrigerator1may display the added memo on the touchscreen display120. For example, the refrigerator1may display a memo display screen2230as illustrated inFIG. 18. Together with existing first memo2231and second memo2232, a third memo2233newly input from the user may be displayed on the memo display screen2230.

Then, the refrigerator1transmits memo data to the service server (1250), and the service server3transmits the received memo data to the user device2(1260).

When the memo data is received from the refrigerator1, the service server3may store the received memo data and transmit the memo data to the user device2. Particularly, the service server3may transmit the memo data to all of the users or transmit the memo data only to a sender and a recipient of a memo. For example, when the first user USER1transmits a memo to the fourth user USER4, the service server3may transmit memo data to the user device2of the first user USER1and the user device2of the fourth user USER4.

The user device2displays a memo indicated by the received memo data (1270).

The user device2may receive memo data from the service server3through the communicator240and store the received memo data in the storage230. The user device2may display a memo on the touchscreen display220according to the user's touch input.

For example, the user device2may display a memo display screen2240as illustrated inFIG. 19. In addition to an existing first memo2241and a second memo2242, a third memo2243received from the service server3may be displayed on the memo display screen2240.

As another example, the user device2may selectively display only a memo of which a recipient or a sender is the user of the user device2. For example, when the user of the user device2is the fourth user USER4, the user device2may selectively display only a memo of which a recipient or a sender is the fourth user USER4, that is, the third memo2243.

A memo adding button2244for adding a memo may be displayed on the memo display screen2240. The user may touch the memo adding button2244to add a new memo.

As described above, the refrigerator1may generate and display a memo according to a user's input and transmit the generated memo to the user device2. Here, the memo may include a recipient and a sender thereof as well as content thereof.

FIG. 20illustrates another example of a memo display screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

As described above, the refrigerator1may identify a user using speaker recognition, facial recognition, fingerprint recognition, iris recognition, vein recognition, and the like and display different home screens2010and2020according to an identified user.

The refrigerator1may identify a user and display a different memo display screen according to an identified user. Specifically, the refrigerator1may selectively display only a memo of which a recipient or a sender is the identified user.

For example, when the identified user is the fourth user USER4, the refrigerator1may selectively display only a memo of which a recipient or a sender is the fourth user USER4. As illustrated inFIG. 20, the refrigerator1may display a memo display screen2250including a third memo2253of which a recipient (or a sender) is the fourth user USER4.

As described above, the refrigerator1may identify the current user and display a different memo for a different user.

FIG. 21illustrates a method of generating and displaying things to do by the refrigerator according to one embodiment.FIGS. 22, 23, 24, and 25illustrate screens for generating and displaying things to do according to the method of generating and displaying things to do illustrated inFIG. 21.

The users may input things to do using the To-do application of the refrigerator1, and the refrigerator1may display the things to do input by the users.

A things-to-do generating/displaying method (1300) in which the refrigerator1generates and displays things to do will be described with reference toFIGS. 21, 22, 23, 24, and 25.

The refrigerator1determines whether to add things to do (1310).

The refrigerator1may determine whether to add things to do according to a user's touch input through the touchscreen display120. The users may run the To-do application to add things to do. For example, the user may make a touch input on the To-do GUI elements2015and2025displayed on the home screens2010and2020. When touch inputs are made on the To-do GUI elements2015and2025, the refrigerator1may determine to add things to do.

When adding things to do, the refrigerator1may receive selection on a performer of new things to do from the user (1320) and receive content of the new things to do (1330).

The refrigerator1may receive new things to do from the user through the touchscreen display120.

The refrigerator1may display a things-to-do display screen2310as illustrated inFIG. 22. Things-to-do lists2311to2315for all of the users USER1to USER5may be displayed on the things-to-do display screen2310. A first things-to-do list2311for the first user USER1, a second things-to-do list2312for the second user USER2, a third things-to-do list2313for the third user USER3, a fourth things-to-do list2314for the fourth user USER4, and a fifth things-to-do list2315for the fifth user USER5may be displayed on the things-to-do display screen2310.

Each of the things-to-do list may display a performer of the things to do and content of the things to do. For example, the first things-to-do list2311may include a performer indicator2311athat indicates that the things to do should be performed by the first user USER1and content2311bof the things to do of the first user USER1.

A things-to-do adding button2316for adding a new things-to-do list and a things-to-do editing button2317for editing the previously input things-to-do lists2311to2315may be displayed on the things-to-do display screen2310. The user may touch the things-to-do adding button2316to add a new things-to-do list and may touch the things-to-do editing button2317for editing the existing things-to-do lists2311to2315.

The user may touch the things-to-do lists2311to2315to add or edit the things to do of each of the users USER1to USER5. For example, when inputting things to do of the first user USER1, the user may touch the first things-to-do list2311in which things to do of the first user USER1are displayed.

When a touch input is made on the things-to-do adding button2316or the things-to-do lists2311to2315, the refrigerator1may display a things-to-do adding screen2320as illustrated inFIG. 23on the touchscreen display120.

The things-to-do adding screen2320may include a performer indicating region2321in which a user to perform the things to do is indicated, a things-to-do input region2322for inputting content of the things to do, a things-to-do display region2323in which content of already input things to do is displayed, and a keyboard2324for inputting things to do.

The user may use the keyboard2324to input new things to do to the things-to-do input region2322. The things to do input by the user may be displayed in the things-to-do display region2323.

When the things to do are added, the refrigerator1displays the added things to do on the touchscreen display120(1340).

When the user adds the things to do through the things-to-do adding screen2320, the refrigerator1may display the added things to do on the touchscreen display120. For example, the refrigerator1may display a things-to-do display screen2330as illustrated inFIG. 24. The things-to-do display screen2330may display a first things-to-do list2331, a second things-to-do list2332, a third things-to-do list2333, a fourth things-to-do list2334, and a fifth things-to-do list2335, and content2331bof newly input things to do may be displayed in the first things-to-do list2331.

Then, the refrigerator1transmits things-to-do data to the service server3(1350), and the service server3transmits the received things-to-do data to the user device2(1360).

When the things-to-do data is received from the refrigerator1, the service server3may store the received things-to-do data and transmits the things-to-do data to the user device2. Particularly, the service server3may transmit the things-to-do data to all of the users or transmit the things-to-do data only to a performer of the things to do. For example, when the performer of things to do is the first user USER1, the service server3may transmit the things-to-do data to the user device2of the first user USER1.

The user device2displays the things to do indicated by the received things-to-do data (1370).

The user device2may receive the things-to-do data from the service server3through the communicator240and store the received things-to-do data in the storage230. The user device2may display the things to do on the touchscreen display220according to the user's touch input.

For example, the user device2may display a things-to-do display screen2340as illustrated inFIG. 25. A first things-to-do list2341, a second things-to-do list2342, a third things-to-do list2343, a fourth things-to-do list2344, and a fifth things-to-do list2345respectively indicating things to do of the first user USER1, the second user USER2, the third user USER3, the fourth user USER4, and the fifth use USER5may be displayed on the things-to-do display screen2340. Particularly, content2341bof things to do received from the service server3may be displayed in the first things-to-do list2341.

As another example, the user device2may selectively display only things to do of which a performer is a user of the user device2. For example, when the user of the user device2is the first user USER1, the user device2may selectively display only the first things-to-do list2341of which a performer is the first user USER1.

A things-to-do adding button2346for adding things to do may be displayed on the things-to-do display screen2340. The user may touch the things-to-do adding button2346to add new things to do.

As described above, the refrigerator1may generate and display things to do according to a user's input and transmit the generated things to do to the user device2. Here, the things to do may include a performer thereof as well as content thereof.

FIG. 26illustrates another example of a things-to-do display screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

As described above, the refrigerator1may identify a user using speaker recognition, facial recognition, fingerprint recognition, iris recognition, vein recognition, and the like and display different home screens2010and2020according to an identified user.

The refrigerator1may identify a user and display a different things-to-do display screen according to an identified user. Specifically, the refrigerator1may selectively display only things to do of which a performer is the identified user.

For example, when the identified user is the first user USER1, the refrigerator1may selectively display only things to do of which a performer is the first user USER1. As illustrated inFIG. 26, the refrigerator1may display a things-to-do display screen2350, and a first things-to-do list2351of which a performer is the first user USER1may be displayed on the things-to-do display screen2350. When the things-to-do display screen2350is compared with the things-to-do display screen2330(seeFIG. 24) illustrated inFIG. 24, while all of the things-to-do lists2331to2335(seeFIG. 24) of all of the users USER1to USER5are displayed on the things-to-do screen2330(seeFIG. 24), only the first things-to-do list2351including things to do2351bof the first user USER1is selectively displayed on the things-to-do display screen2350ofFIG. 26.

As described above, the refrigerator1may identify the current user and display a different things-to-do list for a different user.

FIG. 27illustrates a method of generating and displaying a sketch by the refrigerator according to one embodiment.FIGS. 28, 29, 30, and 31illustrate screens for generating and displaying a sketch according to the method of generating and displaying a sketch illustrated inFIG. 27.

The users may input a sketch by using the sketch application of the refrigerator1, and the refrigerator1may display the sketch input by the users.

A sketch generating/displaying method (1400) in which the refrigerator1generates and displays a sketch will be described with reference toFIGS. 27, 28, 29, 30, and 31.

The refrigerator1determines whether to add a sketch (1410).

The refrigerator1may determine whether to add a sketch according to a user's touch input through the touchscreen display120. The users may run the sketch application to add a sketch. For example, the user may make touch inputs on the sketch GUI elements2016and2026displayed on the home screens2010and2020. When touch inputs are made on the sketch GUI elements2016and2026, the refrigerator1may determine to add a sketch.

When adding a sketch, the refrigerator1receives a new sketch from a user (1420) and receives a writer of the new sketch (1430).

The refrigerator1may receive a new sketch from a user through the touchscreen display120.

The refrigerator1may display a sketch display screen2410illustrated inFIG. 28. Sketches (a first sketch2411ofFIG. 28) input by the users may be displayed on the sketch display screen2410. Each of the sketches may include a sketched image2411aand a writer2411bof the sketch.

A sketch adding button2412for adding a sketch and a sketch editing button2413for editing previously input sketches may be displayed on the sketch display screen2410. The user may touch the sketch adding button2412to add a new sketch and touch the sketch editing button2413to edit an existing sketch. The user may touch a displayed sketch to edit the sketch.

When a touch input is made on the sketch adding button2412, the refrigerator1may display a sketch generation screen2420as illustrated inFIG. 29on the touchscreen display120.

The sketch generation screen2420may include a tool selection region2421for selecting a tool for generating a sketch, a sketch display region2422in which a sketch input by a user is displayed, and a tool display region2423in which a tool selected from the tool selection region2421is displayed.

Tools2421ato2421gthat the user may use to generate a sketch may be displayed in the tool selection region2421. The tools2421ato2421gmay include a writer input tool2421a, a drawing tool2421b, a text input tool2421c, a sticker input tool2421d, a picture input tool2421e, a video input tool2421f, and a discarding tool2421g.

A sketch2422ainput by the user using a tool selected from the tool selection region2421may be displayed in the sketch display region2422.

A tool selected from the tool selection region2421may be displayed in the tool display region2423. For example, when the drawing tool2421bis selected, various drawing tools such as a pen, a brush, and a spray may be displayed in the tool display region2423, and when the text input tool2421cis selected, a keyboard for inputting a text, a number, and/or a symbol may be displayed in the tool display region2423. When the sticker input tool2421dis selected, stickers that may be input to the tool display region2423may be displayed, and when the picture input tool2421eis selected, various pictures may be displayed in the tool display region2423.

When the writer input tool2421ais selected, a writer selection region2423afor selecting a writer of a sketch may be displayed in the tool display region2423. The plurality of users USER1to USER5may be displayed in the writer selection region2423a, and the user may select a writer of a sketch from the plurality of users USER1to USER5.

When a sketch is added, the refrigerator1displays the added sketch (1440).

When the user adds a sketch through the sketch generation screen2420, the refrigerator1may display the added sketch on the touchscreen display120. For example, the refrigerator1may display a sketch display screen2430illustrated inFIG. 30. Together with an existing first sketch2431, a second sketch2432newly input from the user may be displayed on the sketch display screen2430.

Then, the refrigerator1transmits sketch data to the service server3(1450), and the service server3transmits the received sketch data to the user device2(1460).

When the sketch data is received from the refrigerator1, the service server3may store the received sketch data and transmit the sketch data to the user device2. Particularly, the service server3may transmit the sketch data to all of the users or transmit the sketch data only to a writer of a sketch. For example, when the third user USER3is a writer of a sketch, the service server3may transmit sketch data to the user device2of the third user USER3.

The user device2displays a sketch indicated by the received sketch data (1470).

The user device2may receive sketch data from the service server3through the communicator240and store the received sketch data in the storage230. The user device2may display a sketch on the touchscreen display220according to the user's touch input.

For example, the user device2may display a sketch display screen2440as illustrated inFIG. 31. In addition to an existing first sketch2441, a second sketch2442received from the service server3may be displayed on the sketch display screen2440.

As another example, the user device2may selectively display only a sketch of which a writer is the user of the user device2. For example, when the user of the user device2is the third user USER3, the user device2may selectively display only a sketch of which a writer is the third user USER3, that is, the second sketch2442.

A sketch adding button2443for adding a sketch may be displayed on the sketch display screen2440. The user may touch the sketch adding button2443to add a new sketch.

As described above, the refrigerator1may generate and display a sketch according to a user's input and transmit the generated sketch to the user device2. Here, the sketch may include a writer thereof as well as a sketched image.

FIG. 32illustrates another example of a sketch display screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

As described above, the refrigerator1may identify a user using speaker recognition, facial recognition, fingerprint recognition, iris recognition, vein recognition, and the like and display different home screens2010and2020according to an identified user.

The refrigerator1may identify a user and display a different sketch display screen according to an identified user. Specifically, the refrigerator1may selectively display only a sketch of which a writer is the identified user.

For example, when the identified user is the third user USER3, the refrigerator1may selectively display only a sketch of which a writer is the third user USER3. As illustrated inFIG. 32, the refrigerator1may display a second sketch2452of which a writer is the third user USER3on a sketch display screen2450.

As described above, the refrigerator1may identify the current user and display a different sketch for a different user.

FIG. 33illustrates a method of generating and displaying a schedule by the refrigerator according to one embodiment.FIGS. 34, 35, and 36illustrate screens for generating and displaying a schedule according to the method of generating and displaying a schedule illustrated inFIG. 33.

The users may input a schedule by using the scheduler application of the refrigerator1, and the refrigerator1may display the schedule input by the users.

A schedule generating/displaying method (1500) in which the refrigerator1generates and displays a schedule will be described with reference toFIGS. 33, 34, 35, and 36.

The refrigerator1determines whether to add a schedule (1510).

The refrigerator1may determine whether to add a schedule according to a user's touch input through the touchscreen display120. The users may run the scheduler application to add a schedule. For example, the user may make touch inputs on the scheduler GUI elements2013and2023displayed on the home screens2010and2020. When touch inputs are made on the scheduler GUI elements2013and2023, the refrigerator1may determine to add a schedule.

When adding a schedule, the refrigerator1may receive a new schedule from a user (1520) and receive a participant in the schedule (1530) from the user.

The refrigerator1may receive a new schedule from a user through the touchscreen display120. The user may generate a schedule and designate a specific user as a participant in the schedule. For example, the first user USER1may generate a schedule and designate the second user USER2as a participant in the schedule.

The refrigerator1may display a schedule display screen2510as illustrated inFIG. 34. The schedule display screen2510may include a monthly schedule display region2511in which monthly schedules of the users are displayed and a weekly schedule display region2512in which weekly schedules of the users are displayed. A schedule of the user in a month to which the current date belongs may be displayed by date in the monthly schedule display region2511, and a schedule of the user in a week to which the current date belongs may be displayed by time in the weekly schedule display region2512.

A schedule adding button2513for adding a schedule may be displayed on the schedule display screen2510. The user may touch the schedule adding button2513to add a new schedule. The user may touch schedules displayed on the schedule display screen2510to edit the schedules.

When a touch input is made on the schedule adding button2513, the refrigerator1may display a schedule generation screen2520as illustrated inFIG. 35on the touchscreen display120.

The schedule generation screen2520may include a writer selection region2521for selecting a writer of a schedule, a title input region2522for inputting a title of a schedule, a date/time input region2523for inputting a date and time of a schedule, a location input region2524for inputting a location of a schedule, a note input region2525for inputting a brief description of a schedule, and a participant selection region2526for selecting a participant in a schedule.

The user may select a writer of a schedule in the writer selection region2521and input a title, a date/time, a location, and a note of the schedule in the title input region2522, the date/time input region2523, the location input region2524, and the note input region2525, respectively.

The user may select a participant in the schedule in the participant selection region2526. The plurality of users USER1to USER5may be displayed in the participant selection region2526, and the user may select a participant in the schedule among the plurality of users USER1to USER5displayed in the participant selection region2526.

When a schedule is added, the refrigerator1displays the added schedule (1540).

When the user adds a schedule through the schedule generation screen2520, the refrigerator1may display the added schedule on the touchscreen display120.

Then, the refrigerator1transmits schedule data to the service server3(1550), and the service server3transmits the received schedule data to the user device2(1560).

When the schedule data is received from the refrigerator1, the service server3may store the received schedule data and transmit the schedule data to the user device2. Particularly, the service server3may transmit the schedule data to all of the users or transmit the schedule data only to a writer of the schedule and a participant in the schedule. For example, when the third user USER3participates in a schedule of the first user USER1, the service server3may transmit the schedule data to the user device2of the first user USER1and the user device2of the third user USER3.

The user device2displays a schedule indicated by the received schedule data (1570).

The user device2may receive schedule data from the service server3through the communicator240and store the received schedule data in the storage230. The user device2may display the schedule on the touchscreen display220according to the user's touch input.

For example, the user device2may display a schedule display screen2530as illustrated inFIG. 36. The schedule display screen2530may include a monthly schedule display region2531in which monthly schedules of the users are displayed and a daily schedule display region2532in which daily schedules of the users are displayed.

As another example, the user device2may selectively display only a schedule of which a writer or a participant is the user of the user device2. For example, when the user of the user device2is the third user USER3, the user device2may selectively display only a schedule of which a writer or a participant is the third user USER3.

A schedule adding button2533for adding a schedule may be displayed on the schedule display screen2530. The user may touch the schedule adding button2533to add a new schedule.

As described above, the refrigerator1may generate and display a schedule according to a user's input and transmit the generated schedule to the user device2. Here, the schedule may include a writer of the schedule and a participant in the schedule as well as content thereof.

FIG. 37illustrates another example of a schedule display screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

As described above, the refrigerator1may identify a user using speaker recognition, facial recognition, fingerprint recognition, iris recognition, vein recognition, and the like and display different home screens2010and2020according to an identified user.

The refrigerator1may identify a user and display a different schedule display screen according to an identified user. Specifically, the refrigerator1may selectively display only a schedule of which a writer or a participant is the identified user.

For example, when the identified user is the third user USER3, the refrigerator1may selectively display only a schedule of which a writer or a participant is the third user USER3. As illustrated inFIG. 37, the refrigerator1may display a schedule of which a writer or a participant is the third user USER3in a monthly schedule display region2541and a weekly schedule display region2542of a schedule display screen2540.

As described above, the refrigerator1may identify the current user and display a different schedule for a different user.

FIG. 38illustrates a method of generating and displaying a shopping list by the refrigerator according to one embodiment.FIGS. 39, 40, 41, and 42illustrate screens for generating and displaying a shopping list according to the method of generating and displaying a shopping list illustrated inFIG. 38.

The users may input a shopping list by using the shopping list application of the refrigerator1, and the refrigerator1may display the shopping list input by the users.

A shopping list generating/displaying method (1600) in which the refrigerator1generates and displays a shopping list will be described with reference toFIGS. 38, 39, 40, 41, and 42.

The refrigerator1determines whether to add a shopping list (1610).

The refrigerator1may determine whether to add a shopping list according to a user's touch input through the touchscreen display120. The users may run the shopping list application to add a shopping list. For example, the user may make touch inputs on the shopping list GUI elements2018and2028displayed on the home screens2010and2020. When touch inputs are made on the shopping list GUI elements2018and2028, the refrigerator1may determine to add a shopping list.

When adding a shopping list, the refrigerator1may receive a new shopping list from a user (1620).

The refrigerator1may receive a new shopping list from a user through the touchscreen display120.

The refrigerator1may display a shopping list display screen2610as illustrated inFIG. 39. A first shopping list2611input by the users may be displayed on the shopping list display screen2610. Items (products)2611ato be shopped by the user may be indicated in the shopping list2611.

A shopping list adding button2612for adding a shopping list and a shopping list editing button2613for editing previously input shopping lists may be displayed on the shopping list display screen2610. The user may touch the shopping list adding button2612to add a new shopping list and touch the shopping list editing button2613to edit an existing shopping list. The user may touch the shopping list2611displayed on the shopping list display screen2610and edit the shopping list.

When a touch input is made on the shopping list adding button2612, the refrigerator1may display a shopping list generation screen2620as illustrated inFIG. 40on the touchscreen display120.

The shopping list generation screen2620may include a shopping item input region2621in which shopping items (products) that are being input by the user are displayed, and a shopping item display region2622in which the shopping items (products) input by the user are displayed. The user may input items (products) to be shopped in the shopping item input region2621, and the input shopping items (products) may be added to a shopping list and displayed in the shopping item display region2622.

The adding of the shopping list is not limited to being directly generated by a user through the shopping list generation screen2620. For example, a shopping list may be added through a recipe application. The user may search for a recipe using the recipe application. The recipe may include food ingredients as well as a cooking method. The refrigerator1may extract food ingredients included in the recipe from the recipe application and generate a shopping list including the extracted food ingredients.

When a shopping list is added, the refrigerator1displays the added shopping list (1630).

When the user adds a shopping list through the shopping list generation screen2620, the refrigerator1may display the added shopping list on the touchscreen display120. For example, the refrigerator1may display a shopping list display screen2630as illustrated inFIG. 41. Together with an existing first shopping list2631, a second shopping list2643newly input by the user may be displayed on the shopping list display screen2630.

Then, the refrigerator1transmits shopping list data to the service server3(1640), and the service server3transmits the received shopping list data to the user device2(1650).

When the shopping list data is received from the refrigerator1, the service server3may store the received shopping list data and transmit the shopping list data to the user device2.

The user device2displays a shopping list indicated by the received shopping list data (1660).

The user device2may receive the shopping list data from the service server3through the communicator240and store the received shopping list data in the storage230. The user device2may display a shopping list on the touchscreen display220according to the user's touch input.

For example, the user device2may display a shopping list display screen2640as illustrated inFIG. 42. In addition to an existing first shopping list2641, a second shopping list2642received from the service server3may be displayed on the shopping list display screen2640.

A shopping list adding button2643for adding a shopping list may be displayed on the shopping list display screen2640. The user may touch the shopping list adding button2643to add a new shopping list.

As described above, the refrigerator1may generate and display a shopping list according to a user's input and transmit the generated shopping list to the user device2.

FIG. 43illustrates a method of generating and displaying a food list by the refrigerator according to one embodiment.FIGS. 44, 45, 46, 47, and 48illustrate screens for generating and displaying a food list according to the method of generating and displaying a food list illustrated inFIG. 43.

The users may add food stored in the storage compartment20by using the view-inside application of the refrigerator1, and the refrigerator1may display a list of food stored in the storage compartment20.

A food list generating/displaying method (1700) in which the refrigerator1generates and displays a food list will be described with reference toFIGS. 43, 44, 45, 46, 47, and 48.

The refrigerator1determines whether food is additionally stored (1710).

The refrigerator1may monitor whether new food is additionally stored in the storage compartment20to manage foods stored in the storage compartment20.

The refrigerator1may use various methods to monitor whether new food is additionally stored. For example, the refrigerator1may determine that new food is additionally stored in the storage compartment20when the door30is opened and then closed.

As another example, when the view-inside GUI elements2017and2027of the home screens2010and2020are touched by the user, the refrigerator1may determine that new food is additionally stored. In response to touch inputs made on the view-inside GUI elements2017and2027, the refrigerator1may run the view-inside application. The refrigerator1may capture foods stored in the storage compartment20by running the view-inside application and manage the foods on the basis of the captured images of the foods.

The refrigerator1captures an inside of the storage compartment (1720) and displays the captured storage compartment image (1730).

When the door30is opened and then closed, the refrigerator1may run the view-inside application. Through the view-inside application, the refrigerator1may capture the inside of the storage compartment20and display the captured image of the storage compartment20.

The refrigerator1may use the camera190to capture the inside of the storage compartment20. The camera190may be installed in each of the plurality of storage compartments20a,20b, and20cand capture the inside of each of the plurality of storage compartments20a,20b, and20c. The plurality of cameras190may be installed in the upper storage compartment20ahaving a high storage capacity, and the plurality of cameras190may capture the entire upper storage compartment20a.

The refrigerator1may display the image of the inside of the storage compartment20captured using the camera190on the touchscreen display120. Because various types of foods are stored in the storage compartment20, the image of the inside of the storage compartment20may include images of various types of foods stored in the storage compartment20.

The refrigerator1may display a food management screen2710including an image of the inside of the storage compartment20as illustrated inFIG. 44.

The food management screen2710may include a display selection region2711for selecting a display of the image of the inside of the storage compartment20, an image display region2712in which the image of the inside of the storage compartment20is displayed, and a capture time display region2713in which the time at which the image of the inside of the storage compartment20is captured is displayed.

An image display tab2711afor displaying the image of the inside of the storage compartment20and a list display tab2711bfor displaying a list of foods stored in the storage compartment20may be displayed in the display selection region2711.

When the image display tab2711ais selected, images of foods2712a,2712b,2712c,2712d, and2712emay be displayed in the image display region2712of the food management screen2710. For example, the images of a first food2712a, a second food2712b, a third food2712c, a fourth food2712d, and a fifth food2712emay be displayed in the image display region2712.

The refrigerator1receives selection on new food from the user (1740).

The user may check the images of the foods2712a,2712b,2712c,2712d, and2712edisplayed on the food management screen2710and select food added to the storage compartment20among the displayed foods2712a,2712b,2712c,2712d, and2712e. For example, the user may touch the image of the first food2712aamong the images of the foods2712a,2712b,2712c,2712d, and2712edisplayed on the food management screen2710.

When a touch input is made on the image of the first food2712a, the refrigerator1may display a food adding screen2720as illustrated inFIG. 45.

A food image display region2721in which food selected by the user is displayed, a food list adding button2722for adding the food selected by the user to a food list, a shopping list adding button2723for adding the food selected by the user to a shopping list, and a memo adding button2724for generating a memo related to the food selected by the user may be displayed on the food adding screen2720.

For example, when the user touches the image of the first food2712a, the refrigerator1may display the image of the first food2712ain the food image display region2721. When the user touches the food list adding button2722, the refrigerator1may add the first food2712ato a food list. When the user touches the shopping list adding button2723, the refrigerator1may add the first food2712ato a shopping list. When the user touches the memo adding button2724, the refrigerator1may generate a memo using the image of the first food2712a.

When food is added, the refrigerator1displays a food list (1750).

When food is added to the food list through the food adding screen2720, the refrigerator1may display the food list including the added food.

When a touch input is made on the list display tab2711bof the food adding screen2720, the refrigerator1may display a food management screen2730as illustrated inFIG. 46.

The food management screen2730may include a display selection region2731for selecting a display of a list of foods stored in the storage compartment20and a list display region2732in which a list of foods stored in the storage compartment20is displayed.

An image display tab2731afor displaying an image of the inside of the storage compartment20and a list display tab2731bfor displaying a list of foods stored in the storage compartment20may be displayed in the display selection region2731.

When the list display tab2731bis selected, a list of foods stored in the storage compartment20may be displayed in the list display region2732of the food management screen2730. For example, an image2732aof a first food stored in the storage compartment20and a name2732bof the first food stored in the storage compartment20may be displayed in the list display region2732.

The refrigerator1may display food images and a food list for managing foods through the food management screens2710and2730. The user may manage foods stored in the refrigerator1by using the food management screens2710and2730of the refrigerator1.

Then, the refrigerator1transmits the food images and the food list to the service server3(1760), and the service server3transmits the received food images and food list to the user device2(1770).

When the food images and the food list are received from the refrigerator1, the service server3may store the received food images and food list and transmit the food images and the food list to the user device2.

The user device2displays the received food images (1780).

In response to reception of the food images from the service server3, the user device2may display an image of the inside of the refrigerator1on the touchscreen display220.

The user device2may display a food management screen2740including the image of the inside of the refrigerator1as illustrated inFIG. 47.

The food management screen2740may include a display selection region2741for selecting a display of the image of the inside of the refrigerator1, an image display region2742in which the image of the inside of the refrigerator1is displayed, and a capture time display region2743in which the time at which the image of the inside of the refrigerator1is captured is displayed.

When the image display tab2741ais selected, images of foods stored in the refrigerator1may be displayed in the image display region2742of the food management screen2740.

The user device2displays the received food list (1790).

In response to reception of the food list from the service server3, the user device2may display the list of foods stored in the refrigerator1on the touchscreen display220.

The user device2may display a food management screen2750including a list of foods stored in the refrigerator1as illustrated inFIG. 48.

The food management screen2750may include a display selection region2751for selecting a display of a list of foods stored in the refrigerator1and a list display region2752in which the list of foods stored in the refrigerator1is displayed.

When the list display tab2751bis selected, the list of foods stored in the refrigerator1may be displayed in the list display region2752of the food management screen2750.

A food adding button2753for adding food may be displayed on the food management screen2750. The user may touch the food adding button2753to add new food.

The user device2may display food images and a food list for managing foods through the food management screens2740and2750. The user may manage foods stored in the refrigerator1by using the food management screens2740and2750of the user device2.

FIG. 49illustrates a method of providing morning briefing by the refrigerator according to one embodiment.FIG. 50illustrates a morning briefing display screen displayed on the touchscreen display included in the refrigerator according to one embodiment.FIGS. 51, 52, 53, and 54illustrate screens for setting morning briefing of the refrigerator according to one embodiment.FIG. 55illustrates a morning briefing output screen for outputting morning briefing by the refrigerator according to one embodiment.

The refrigerator1may provide morning briefing including weather, schedule, and/or news according to a user's settings.

A morning briefing method (1800) in which the refrigerator1provides morning briefing to a user will be described with reference toFIGS. 49, 50, 51, 52, 53, 54, and 55.

The refrigerator1determines whether to provide morning briefing (1810).

When a briefing condition preset by the user is satisfied, the refrigerator1may provide morning briefing. For example, the refrigerator1may provide morning briefing at a time preset by the user.

The user may run the morning briefing application and set a briefing condition. For example, the user may make touch inputs on the morning briefing GUI elements2019and2029displayed on the home screens2010and2020. When the touch inputs are made on the morning briefing GUI elements2019and2029, the refrigerator1may display a morning briefing screen2810illustrated inFIG. 50.

The morning briefing screen2810may include a date/time display region2811in which a date and time are displayed, briefing regions2812,2813, and2814in which pieces of briefing content set by the user are displayed, a voice briefing start button2815for starting voice briefing, and a briefing setting button2816for setting briefing content.

Pieces of briefing content set by the user may be displayed in the briefing regions2812,2813, and2814. For example, as illustrated inFIG. 50, weather content2812, schedule content2813, and news content2814may be respectively displayed in the briefing regions2812,2813, and2814.

When the voice briefing start button2815of the morning briefing screen2810is touched, the refrigerator1may start voice briefing. For example, during the voice briefing, the refrigerator1may output text and/or numbers in the pieces of briefing contents respectively displayed in the briefing regions2812,2813, and2814by voice. In other words, during the voice briefing, the refrigerator1may read the pieces of briefing content out loud. When the voice briefing start button2815is touched, the refrigerator1may display a morning briefing screen2860, which will be described below, as illustrated inFIG. 55.

When the briefing setting button2816of the morning briefing screen2810is touched, the refrigerator1may display a screen for setting parameters related to morning briefing. For example, the refrigerator1may display a briefing setting screen2820as illustrated inFIG. 51.

The briefing setting screen2820may include an automatic briefing setting region2821for setting automatic briefing, a briefing start setting region2822for setting conditions for starting briefing, and a briefing content setting region2823for setting content to be briefed.

An automatic briefing activation button2821afor activating or deactivating the automatic briefing may be displayed in the automatic briefing setting region2821. The user may touch the automatic briefing activation button2821ato activate or deactivate automatic briefing.

When automatic briefing is activated, the refrigerator1may automatically provide morning briefing to the user in response to satisfaction of briefing conditions.

When automatic briefing is activated by the automatic briefing activation button2821a, the refrigerator1may display a voice briefing activation screen2830for setting whether voice briefing is performed at the start of briefing.

For example, as illustrated inFIG. 52, a voice briefing activation button2831for activating voice briefing and a voice briefing deactivation button2832for deactivating voice briefing may be displayed on the voice briefing activation screen2830. When voice briefing is activated, the refrigerator1may automatically start voice briefing upon the start of morning briefing. When voice briefing is deactivated, the refrigerator1may suspend voice briefing upon the start of morning briefing and display only the morning briefing screen.

A start time setting region2822afor setting/displaying a start time of morning briefing, and a weekly repetition setting region2822bfor setting/displaying the number of repetitions of morning briefing per week may be displayed in the briefing start setting region2822. The user may touch the start time setting region2822aand input the time (for example, 6 AM) for starting morning briefing. The user may touch the weekly repetition setting region2822band input the number of repetitions of morning briefing per week (for example, repeat everyday or repeat on Monday/Tuesday/Wednesday/Thursday/Friday).

Pieces of content2823aindicating content to be briefed by morning briefing may be displayed in the briefing content setting region2823. The user may touch the briefing content setting region2823to set content to be briefed.

When a touch input is made on the briefing content setting region2823, the refrigerator1may display a content selection screen2840for selecting briefing content.

For example, as illustrated inFIG. 53, pieces of content2841,2842, and2843that may be selected by the user may be displayed on the content selection screen2840. For example, the content selection screen2840may include a weather content checkbox2841for selecting weather content, a schedule content checkbox2842for selecting schedule content, and a news content checkbox2843for selecting news content. The user may check at least one of the plurality of checkboxes2841,2842, and2843displayed on the content selection screen2840.

When briefing content is selected through the content selection screen2840, the refrigerator1may display the selected content on the morning briefing screen2810.

When it is determined to provide morning briefing (YES to1810), the refrigerator1provides morning briefing to the user (1820).

When a briefing condition (for example, a briefing start time) set by the user through the briefing setting screen2820is satisfied, the refrigerator1may display a screen for morning briefing on the touchscreen display120.

For example, when voice briefing is deactivated by the voice briefing deactivation button2832of the voice briefing activation screen2830, the refrigerator1may display a morning briefing start screen2850as illustrated inFIG. 54.

The morning briefing start screen2850may include a voice briefing start button2851for starting voice briefing and a voice briefing skip button2852for skipping voice briefing.

When a touch input is made on the briefing start button2851, the refrigerator1may display content for morning briefing on the touchscreen display120and output sound related to briefing content through the audio180.

For example, the refrigerator1may display the morning briefing screen2860as illustrated inFIG. 55on the touchscreen display120. The morning briefing screen2860may include a date/time display region2861and briefing regions2862,2863, and2864. The refrigerator1may output pieces of content displayed in the briefing regions2862,2863, and2864by sound or voice through the audio180.

A volume control button2865for controlling volume of sound output through the audio180, a pause button2867for pausing an output of briefing content by sound, and a briefing resume button2866for resuming output of the briefing content by sound may be displayed on the morning briefing screen2860. The user may control volume of voice briefing using the volume control button2865. The user may pause voice briefing using the pause button2867and resume voice briefing using the briefing resume button2866.

As described above, the refrigerator1may provide morning briefing including content selected by the user.

FIGS. 56 and 57illustrate other examples of a briefing display screen displayed on the touchscreen display included in the refrigerator according to one embodiment.

As described above, the refrigerator1may identify a user using speaker recognition, facial recognition, fingerprint recognition, iris recognition, vein recognition, and the like and display different home screens2010and2020according to an identified user.

The refrigerator1may identify a user and provide different morning briefing according to an identified user. Specifically, the refrigerator1may provide morning briefing including content selected by the identified user.

For example, when the identified user is the second user USER2, the refrigerator1may provide morning briefing including content selected by the second user USER2. When the second user USER2selected weather content and schedule content through the content selection screen2840(seeFIG. 53), the refrigerator1may display a morning briefing screen2870as illustrated inFIG. 56. A date/time2871, weather content2872, and schedule content2873may be displayed on the morning briefing screen2870. The refrigerator1may output the weather content2872and the schedule content2873by sound through the audio180.

As another example, when the identified user is the third user USER3, the refrigerator1may provide morning briefing including content selected by the third user USER3. When the third user USER3selected weather content and news content, the refrigerator1may display a morning briefing screen2880as illustrated inFIG. 57. A date/time2881, weather content2882, and news content2884may be displayed on the morning briefing screen2880. The refrigerator1may output the weather content2882and the news content2884by sound through the audio180.

As described above, the refrigerator1may identify a current user and provide morning briefing having different pieces of content for different users.

As is apparent from the above description, according to an aspect of the disclosure, content customized for each of a plurality of users can be provided by using a communication device and a touchscreen display linked to user devices of the plurality of users.

According to another aspect of the disclosure, a refrigerator capable of providing different pieces of content to different users by using a communication device and a touchscreen display can be provided.

The embodiments disclosed herein may be implemented in the form of a recording medium that stores a computer executable command. The command may be stored in the form of a program code, and when executed by a processor, may generate a program module and perform operations of the embodiments disclosed herein. The recording medium may be implemented as a computer readable recording medium.

The computer readable recording medium includes all types of recording media in which computer readable commands are stored. Examples of the computer readable medium may include a ROM, a RAM, a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, and the like.

Embodiments disclosed herein have been described above with reference to the accompanying drawings. One of ordinary skill in the art to which the embodiments disclosed herein pertain should understand that the embodiments may be practiced in different forms from the above without changing the technical spirit or essential features of the embodiments. The embodiments are merely illustrative and should not be interpreted as limiting.