Electronic device for determining position of user based on image pixels, and method of controlling said device

This electronic device comprises: a sensor including a plurality of pixels disposed in a first direction; a processor electrically connected to the sensor; and a memory electrically connected to the processor. When implemented, the processor uses sensing data sensed by the sensor and determine user position information, and the memory may store instructions for carrying out a movement corresponding to the user position information.

PRIORITY

This application is a National Phase Entry of PCT International Application No. PCT/KR2016/011384, which was filed on Oct. 11, 2016, and claims priority to Korean Patent Application No. 10-2015-0155183, which was filed on Nov. 5, 2015, the contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to an electronic device for determining the position of a user and a method for controlling the same and, more particularly, to an electronic device for performing an operation corresponding to the determined position of a user and a method for controlling the same.

BACKGROUND ART

Conventionally, as a method for recognizing the position of a user indoors, a method using a camera and a method using a sensor have been proposed. In the method using the camera, a conventional electronic device receives images in real time and recognizes a user via an image processing technique. Further, the conventional electronic device may calculate precise information, such as the distance or angle between the electronic device and a user, using two cameras. The conventional electronic device, for example, a TV, may include a camera disposed at the center of an upper bezel and may determine user position information using images captured by the camera. In the method using the sensor, a conventional electronic device may employ a light level sensor or an ultrasonic sensor and may determine the presence or absence of a user or the distance from a user to the electronic device. The method using the sensor is advantageous in terms of cost.

DETAILED DESCRIPTION OF THE INVENTION

Technical Problem

A method using a camera requires relatively expensive hardware and a great amount of computation for image processing. In addition, since an electronic device needs to include a camera, an installation space is required, making it difficult to miniaturize the electronic device. Further, as hacking into electronic devices have frequently occurred in recent years, when electronic devices using a camera are hacked, the privacy of users may be breached. A method using a sensor can determine merely the presence or absence of a user due to low accuracy and resolution.

Therefore, to overcome the disadvantages of the two methods, it is required to develop an electronic device and a method for controlling the same that are capable of obtaining detailed user position information while reducing the possible invasion of privacy.

Technical Solution

According to various embodiments of the present disclosure, an electronic device may include: a sensor configured to include a plurality of pixels disposed in a first direction; a processor configured to be electrically connected to the sensor; and a memory configured to be electrically connected to the processor, wherein the memory may store an instruction that, when executed, enables the processor to determine position information on a user using sensing data sensed by the sensor and to perform an operation corresponding to the position information on the user.

According to various embodiments of the present disclosure, a control method of an electronic device including a plurality of pixels disposed in a first direction may include: determining position information on a user using sensing data sensed by the plurality of pixels; and performing an operation corresponding to the position information on the user.

Advantageous Effects

According to various embodiments of the present disclosure, there may be provided the development of an electronic device and a method for controlling the same that are capable of obtaining detailed user position information while reducing the possible invasion of privacy. In particular, a sensor array can be made small and thus has relatively less restriction in a position for disposition. Thus, the sensor array may be disposed at any position where user position information is required.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. In the present disclosure, the term “user” may indicate a person using an electronic device or a device (e.g., an artificial intelligence electronic device) using an electronic device.

FIG. 1illustrates an electronic device101in a network environment100according to various exemplary embodiments. The electronic device101may include a bus110, a processor120, a memory130, an input/output interface150, a display160, and a communication module170. In some embodiments, at least one of the components may be omitted, or additional components may be further included in the electronic device101.

The bus110may include, for example, a circuit that connects the components110to170to each other and delivers communications (for example, control messages and/or data) between the components.

The processor120may include one or more of a Central Processing Unit (CPU), an Application Processor (AP), and a Communication Processor (CP). The processor120may control, for example, at least one different component of the electronic device101, and/or may perform operations relating to communication or data processing.

The memory130may include a volatile and/or nonvolatile memory. The memory130may store, for example, a command or data related to at least one different component of the electronic device101. According to one exemplary embodiment, the memory130may store software and/or a program140. The program140may include, for example, a kernel141, middleware143, an Application Programming Interface (API)145, and/or an application (or “app”)147. At least part of the kernel141, the middleware143, and the API145may be designated as an Operating System (OS).

The kernel141may control or manage system resources (for example, the bus110, the processor120, the memory130, or the like) used to perform an operation or function implemented in other programs (for example, the middleware143, the API145, or the application147). Further, the kernel141may provide an interface that allows the middleware143, the API145, or the application147to access individual components of the electronic device101to thereby control or manage system resources.

The middleware143may serve as a relay so that, for example, the API145or the application147communicates with the kernel141to exchange data.

Further, the middleware143may process one or more requests for operations received from the application147according to the priority thereof. For example, the middleware143may assign at least one application147a priority for using a system resource of the electronic device101(for example, the bus110, the processor120, the memory130, or the like). For example, the middleware143may process the one or more requests for operations according to the priority assigned to the at least one application, thereby performing scheduling or load balancing for the one or more requests for operations.

The API145is, for example, an interface for the application147to control a function provided from the kernel141or the middleware143, and may include, for example, at least one interface or function (for example, a command) for file control, window control, image processing, or text control.

The input/output interface150may serve as an interface that delivers a command or data, which is input from, for example, a user or a different external device, to a different component(s) of the electronic device101. Further, the input/output interface150may output a command or data, which is received from a different component(s) of the electronic device101, to the user or to the different external device.

The display160may include, for example, a Liquid Crystal Display (LCD), a Light-Emitting Diode (LED) display, an Organic Light-Emitting Diode (OLED) display, a Micro-Electro-Mechanical Systems (MEMS) display, or an electronic paper display. The display160may display, for example, various types of content (for example, text, an image, a video, an icon, a symbol, or the like) for the user. The display160may include a touch screen and may receive touch, gesture, proximity, or hovering input using, for example, an electronic pen or a body part of a user.

The communication module170may establish communication, for example, between the electronic device101and an external device (for example, a first external electronic device102, a second external electronic device104, or a server106). For example, the communication module170may be connected to a network162via wireless communication or wired communication to communicate with the external device (for example, the second external electronic device104or the server106).

A sensor180may include a plurality of pixels arranged in a first direction, may sense sensing data corresponding to a one-dimensional image, and may output the sensing data to the processor120. The sensor180will be described in detail later.

The wireless communication may use, for example, a cellular communication protocol, which may be, for example, at least one of Long-Term Evolution (LTE), LTE-Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro), and Global System for Mobile Communications (GSM). Further, the wireless communication may include, for example, short-range communication164. The short-range communication164may include, for example, at least one of Wireless Fidelity (Wi-Fi), Bluetooth, Near-Field Communication (NFC), and Global Navigation Satellite System (GNSS). The GNSS may include, for example, at least one of a Global Positioning System (GPS), a Global Navigation Satellite System (GNSS), a BeiDou Navigation Satellite System (hereinafter, “BeiDou”), and Galileo, which is the European global satellite-based navigation system, depending on the use area or bandwidth. In the present document, “GPS” may be interchangeably used with “GNSS” hereinafter. The wired communication may include, for example, at least one of Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), Recommended Standard 232 (RS-232), and Plain Old Telephone Service (POTS). The network162may include a telecommunications network, which may be, for example, at least one of a computer network (for example, a Local Area Network (LAN) or Wide Area Network (WAN)), the Internet, and a telephone network.

Each of the first and second external electronic devices102and104may be a device of a type that is the same as, or different from, that of the electronic device101. According to one exemplary embodiment, the server106may include a group of one or more servers. According to various exemplary embodiments, all or some of the operations performed in the electronic device101may be performed in another electronic device or a plurality of electronic devices (for example, the electronic devices102and104or the server106). According to one exemplary embodiment, when the electronic device101needs to perform a function or service automatically or upon request, the electronic device101may request another electronic device (for example, the electronic device102or104, or the server106) to perform at least some functions related to the function or service, instead of, or in addition to, autonomously performing the function or service. The other electronic device (for example, the electronic device102or104, or the server106) may perform the requested functions or additional functions and may transmit the result thereof to the electronic device101. The electronic device101may provide the requested function or service using the same received result or after additionally processing the result. To this end, cloud-computing, distributed-computing, or client-server-computing technologies may be used.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to determine position information on a user using sensing data sensed by the sensor and to perform an operation corresponding to the position information on the user.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to divide the plurality of pixels into pixels corresponding to a background and pixels corresponding to a person using the sensing data and to determine the position information on the user on the basis of the pixels corresponding to the person.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to determine a feature point from the sensing data and to divide the plurality of pixels into the pixels corresponding to the background and the pixels corresponding to the person using the sensing data on the basis of the determined feature point.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to determine a horizontal position or a vertical position of the user on the basis of the pixels corresponding to the person.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to determine information on a distance between the user and the electronic device on the basis of a number of the pixels corresponding to the person.

According to various embodiments of the present disclosure, each of the plurality of pixels of the sensor180may include a plurality of sub-pixels, and the memory130may store an instruction that, when executed, enables the processor120to determine information on a distance between the user and the electronic device on the basis of a difference between pieces of sensing data sensed by a plurality of sub-pixels of each of the pixels corresponding to the person.

According to various embodiments of the present disclosure, the sensor180may include a first pixel array including a plurality of pixels disposed in the first direction and a second pixel array including a plurality of other pixels disposed in the first direction, and the memory130may store an instruction that, when executed, enables the processor120to determine information on a distance between the user and the electronic device on the basis of a difference between pixels corresponding to the person in the first pixel array and pixels corresponding to the person in the second pixel array.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to apply a stereo vision technique to the pixels corresponding to the person in the first pixel array and the pixels corresponding to the person in the second pixel array and to determine the information on the distance between the user and the electronic device on the basis of an application result.

According to various embodiments of the present disclosure, the memory130may store in advance a reference database associated with the sensing data, and may store an instruction that, when executed, enables the processor120to compare the sensing data with the reference database and to determine the position information on the user using a comparison result.

According to various embodiments of the present disclosure, the memory130may store an instruction that, when executed, enables the processor120to generate a difference image between the sensing data and the reference database, to divide a plurality of pixels of the difference image into pixels corresponding to the background and pixels corresponding to the person, and to determine the position information on the user on the basis of the pixels corresponding to the person.

According to various embodiments of the present disclosure, the sensor180may sense a plurality of pieces of sensing data over time, and the memory130may store an instruction that, when executed, enables the processor120to divide the plurality of pixels into pixels corresponding to the background and pixels corresponding to the person using each of the plurality of pieces of sensing data sensed over time, and to determine movement information on the user on the basis of a change in the pixels corresponding to the person over time.

FIG. 2is a block diagram of an electronic device201according to various exemplary embodiments. The electronic device201may include, for example, all or part of the electronic device101illustrated inFIG. 1. The electronic device201may include one or more processors (for example, Application Processors (APs))210, a communication module220, a Subscriber Identification Module (SIM)224, a memory230, a sensor module240, an input device250, a display260, an interface270, an audio module280, a camera module291, a power management module295, a battery296, an indicator297, and a motor298.

The processors210may run, for example, an operating system or an application to control a plurality of hardware or software components that are connected to the processors210, and may perform various kinds of data processing and operations. The processors210may be configured, for example, as a System on Chip (SoC). According to one exemplary embodiment, the processors210may further include a Graphic Processing Unit (GPU) and/or an image signal processor. The processors210may include at least some (for example, a cellular module221) of the components illustrated inFIG. 2. The processors210may load a command or data received from at least one of other components (for example, nonvolatile memory) into volatile memory to process the command or data, and may store various types of data in the nonvolatile memory.

The communication module220may have a configuration that is the same as, or similar to, that of the communication module170inFIG. 1. The communication module220may include, for example, a cellular module221, a Wi-Fi module223, a Bluetooth module225, a global navigation satellite system (GNSS) module227(for example, a global positioning system (GPS) module, a GLONASS module, a BeiDou module, or a Galileo module), a Near-Field Communication (NFC) module228, and a radio frequency (RF) module229.

The cellular module221may provide, for example, a voice call, a video call, a text messaging service, or an Internet service through a communication network. According to one exemplary embodiment, the cellular module221may perform identification and authentication of the electronic device201in a communication network using a Subscriber Identity Module (SIM, for example, a SIM card)224. According to one exemplary embodiment, the cellular module221may perform at least some of the functions provided by the processors210. According to one exemplary embodiment, the cellular module221may include a Communication Processor (CP).

The Wi-Fi module223, the Bluetooth module225, the GNSS module227, and the NFC module228may each include a processor to process data transmitted and received via the respective modules. According to one exemplary embodiment, at least some (for example, two or more) of the cellular module221, the Wi-Fi module223, the Bluetooth module225, the GNSS module227, and the NFC module228may be included in one Integrated Circuit (IC) or IC package.

The RF module229may transmit and receive, for example, a communication signal (for example, an RF signal). The RF module229may include, for example, a transceiver, a Power Amplifier (amp) Module (PAM), a frequency filter, a Low-Noise Amplifier (LNA), an antenna, or the like. According to another embodiment, at least one of the cellular module221, the Wi-Fi module223, the Bluetooth module225, the GNSS module227, and the NFC module228may transmit and receive an RF signal through a separate RF module.

The SIM224may include, for example, a card including a SIM and/or an embedded SIM, and may include unique identification information (for example, an Integrated Circuit Card Identifier (ICCID)) or subscriber information (for example, an International Mobile Subscriber Identity (IMSI)).

The memory230(for example, the memory130) may include, for example, an internal memory232or an external memory234. The internal memory232may include, for example, at least one of a volatile memory (for example, a Dynamic Random-Access Memory (DRAM), a Static RAM (SRAM), a Synchronous Dynamic RAM (SDRAM), or the like) and a nonvolatile memory (for example, a One-Time Programmable Read-Only Memory (OTPROM), a Programmable ROM (PROM), an Erasable and Programmable ROM (EPROM), an Electrically Erasable and Programmable ROM (EEPROM), a mask ROM, a flash ROM, a flash memory (for example, an NAND flash, an NOR flash, or the like), a hard drive, or a Solid-State Drive (SSD)).

The external memory234may further include a flash drive, for example, a Compact Flash (CF), a Secure Digital (SD), a micro Secure Digital (micro-SD), a mini Secure Digital (mini-SD), an extreme digital (xD), a Multi-Media Card (MMC), a memory stick, or the like. The external memory234may be functionally and/or physically connected to the electronic device201through any of various interfaces.

The sensor module240may measure, for example, physical quantities, or may detect the state of operation of the electronic device201and convert measured or detected information into an electrical signal. The sensor module240may include, for example, at least one of a gesture sensor240A, a gyro sensor240B, a barometric pressure sensor240C, a magnetic sensor240D, an accelerometer240E, a grip sensor240F, a proximity sensor240G, a color sensor240H (for example, a red, green, and blue (RGB) sensor), a biometric sensor240I, a temperature/humidity sensor240J, and an illumination sensor240K, and an ultraviolet (UV) sensor240M. Additionally or alternatively, the sensor module240may include, for example, an E-nose sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. The sensor module240may further include a control circuit to control at least one or more sensors belonging thereto. In one exemplary embodiment, the electronic device201may further include a processor configured, as a part of the processors210or separately from the processors210, to control the sensor module240, thereby controlling the sensor module240while the processors210are in a sleep state.

The input device250may include, for example, a touch panel252, a (digital) pen sensor254, a key256, or an ultrasonic input device258. The touch panel252may be, for example, at least one of an electrostatic type, a pressure-sensitive type, an infrared type, and an ultrasonic type. Further, the touch panel252may further include a control circuit. The touch panel252may further include a tactile layer to provide a user with a tactile response.

The (digital) pen sensor254may, for example, be part of the touch panel or may include a separate recognition sheet. The key256may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device258may detect ultrasonic waves generated in an input tool through a microphone (for example, a microphone288) and may identify data corresponding to the detected ultrasonic waves.

The display260(for example, the display160) may include a panel262, a hologram device264, or a projector266. The panel262may include a configuration that is the same as, or similar to, that of the display160ofFIG. 1. The panel262may be configured, for example, to be flexible, transparent, or wearable. The panel262may be formed with the touch panel252in a single module. The hologram device264may display a three-dimensional image in the air using light interference. The projector266may project light onto a screen to display an image. The screen may be disposed, for example, inside or outside the electronic device201. According to one exemplary embodiment, the display260may further include a control circuit to control the panel262, the hologram device264, or the projector266.

The audio module280may bidirectionally convert, for example, a sound and an electrical signal. At least some components of the audio module280may be included, for example, in the input/output interface150illustrated inFIG. 1. The audio module280may process sound information input or output, for example, through a speaker282, a receiver284, earphones286, or the microphone288.

The camera module291is a device that takes, for example, a still image and a video. According to one exemplary embodiment, the camera module291may include one or more image sensors (for example, a front sensor or a rear sensor), a lens, an Image Signal Processor (ISP), or a flash (for example, an LED, a xenon lamp, or the like).

The power management module295may manage, for example, the power of the electronic device201. According to one exemplary embodiment, the power management module295may include a Power Management Integrated Circuit (PMIC), a charger Integrated Circuit (IC), or a battery or fuel gauge. The PMIC may have wired and/or wireless charging methods. The wireless charging methods may include, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic wave method, and may further include an additional circuit for wireless charging, such as a coil loop, a resonance circuit, or a rectifier. The battery gauge may measure, for example, the remaining battery charge, the charging voltage, the current, or temperature of the battery296. The battery296may include, for example, a rechargeable battery and/or a solar battery.

The indicator297may display the specific state of the electronic device201or a component thereof (for example, the processors210), which may be, for example, a booting state, a message state, or a charging state. The motor298may convert an electrical signal into mechanical vibrations, and may generate vibrations or a haptic effect. Although not shown, the electronic device201may include a processing device for supporting a mobile TV (for example, a GPU). The processing device for supporting the mobile TV may process media data in accordance with Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting (DVB), or mediaFlo™ standards.

FIG. 3is a block diagram illustrating a program module according to various exemplary embodiments. According to one exemplary embodiment, the program module310(for example, the program140) may include an Operating System (OS) that controls resources related to an electronic device (for example, the electronic device101) and/or various applications (for example, the application147) that run on the OS. The OS may be, for example, Android, iOS, Windows, Symbian, Tizen, Bada, or the like.

The program module310may include a kernel320, middleware330, an API360, and/or an application370. At least part of the program module310may be preloaded onto the electronic device, or may be downloaded from an external electronic device (for example, the electronic device102or104, the server106, or the like).

The kernel320(for example, the kernel141) may include, for example, a system resource manager321and/or a device driver323. The system resource manager321may perform control, allocation, or recovery of system resources. According to one exemplary embodiment, the system resource manager321may include a process management unit, a memory management unit, or a file-system management unit. The device driver323may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a Wi-Fi driver, an audio driver, or an Inter-Process Communication (IPC) driver.

The middleware330may provide, for example, functions commonly needed for applications370, or may provide the applications370with various functions through the API360so that the applications370may efficiently use the limited systems resources in the electronic device. According to one exemplary embodiment, the middleware330(for example, the middleware143) may include at least one of a runtime library335, an application manager341, a window manager342, a multimedia manager343, a resource manager344, a power manager345, a database manager346, a package manager347, a connectivity manager348, a notification manager349, a location manager350, a graphic manager351, and a security manager352.

The runtime library355may include, for example, a library module used by a complier to add a new function through a programming language while the application370is running. The runtime library355may perform functions for input/output management, memory management, or arithmetic functions.

The application manager341may manage, for example, the life cycle of at least one application among the applications370. The window manager342may manage graphic user interface (GUI) resources used for a screen. The multimedia manager343may identify formats that are necessary to play various media files, and may encode or decode a media file using a codec suitable for a corresponding format. The resource manager344may manage resources, such as a source code, a memory, or a storage space, for at least one application among the applications370.

The power manager345may operate with, for example, a basic input/output system (BIOS) to manage a battery or power supply and may provide information on power necessary for operation of the electronic device. The database manager346may generate, retrieve, or change a database to be used for at least one application among the applications370. The package manager347may install or update an application distributed in the form of a package file.

The connectivity manager348may manage wireless connectivity via, for example, Wi-Fi or Bluetooth. The notification manager349may display or report an incoming message, an appointment, and an event including a proximity notification in a manner that does not disturb a user. The location manager350may manage position information on the electronic device. The graphic manager351may manage a graphic effect to be provided for the user or a user interface related to the graphic effect. The security manager352may provide overall security functions necessary for system security or user authentication. According to one exemplary embodiment, when the electronic device (for example, the electronic device101) has phone features, the middleware330may further include a telephony manager to manage a voice or video call function of the electronic device.

The middleware330may include a middleware module that forms combinations of various functions of the foregoing components. The middleware330may provide a specialized module for each type of OS in order to provide differentiated functions. Further, the middleware330may dynamically delete some of the existing components or add new components.

The API360(for example, the API145) is, for example, a set of API programming functions, and may be provided with a different configuration depending on the OS. For example, one API set for each platform may be provided in the case of Android or iOS, while two or more API sets for each platform may be provided in the case of Tizen.

The application370(for example, the application147) may include one or more applications that are capable of performing functions of, for example, a home screen371, a dialer372, SMS/MMS373, instant messaging (IM)374, a browser375, a camera376, an alarm377, an address book378, a voice dialer379, email380, a calendar381, a media player382, an album383, a clock384, or health care (for example, for measuring exercising or blood sugar), an environmental data application (for example, for providing atmospheric pressure, humidity, or temperature data), or the like.

According to one exemplary embodiment, the application370may include an application (hereinafter, “information exchange application” for convenience of description) that supports information exchange between the electronic device (for example, the electronic device101) and an external electronic device (for example, the electronic device102or104). The information exchange application may include, for example, a notification relay application for relaying specific information to the external electronic device or a device management application for managing the external electronic device.

For example, the notification relay application may include a function of relaying notification information, which is generated in another application (for example, an SMS/MMS application, an email application, a health care application, an environmental data application, or the like) of the electronic device, to the external electronic device (for example, the electronic device102or104). Additionally, the notification relay application may receive notification information, for example, from the external electronic device and may provide the notification information to the user.

The device management application may manage (for example, install, delete, or update), for example, at least one function (for example, a function of turning on/turning off the external electronic device itself (or some components) or adjusting the brightness (or resolution) of a display) of the external electronic device (for example, the electronic device102or104) communicating with the electronic device, an application operating in the external electronic device, or a service (for example, a call service or message service) provided by the external electronic device.

According to one exemplary embodiment, the application370may include an application (for example, a health care application of a mobile medical device) assigned according to the attributes of the external electronic device (for example, the electronic device102or104). According to one exemplary embodiment, the application370may include an application received from the external electronic device (for example, the server106or the electronic device102or104). According to one exemplary embodiment, the application370may include a preloaded application or a third-party application that may be downloaded from a server. The illustrated components of the program module310, according to the embodiments, may be termed differently depending on the OS.

According to various exemplary embodiments, at least part of the program module310may be implemented in software, firmware, hardware, or combinations of at least two or more thereof. At least part of the program module310may be implemented (for example, run) by, for example, a processor (for example, the processor210). At least part of the program module310may include, for example, a module, a program, a routine, sets of instructions, or a process to perform one or more functions.

FIGS. 4A and 4Bare the conceptual diagrams of an electronic device according to various embodiments of the present disclosure. The electronic device101ofFIGS. 4A and 4Bmay be configured in the form of a TV, but those skilled in the art would readily understand that there is no restriction on the configuration of the electronic device. The electronic device101may include a display160and a sensor180. The sensor180may be configured as an array including 1×N (N is a natural number) pixels181to187. Accordingly, the sensor180may obtain a one-dimensional image. A normal camera module may have A×B pixels. Here, A may be the number of pixels in the horizontal axis direction, B may be the number of pixels in the vertical axis direction, and A and B may be natural numbers The normal camera module may obtain a two-dimensional image of the external environment using sensing information output from each of the A×B pixels. In contrast, the sensor180according to various embodiments of the present disclosure includes a one-dimensional pixel array having one pixel either in the horizontal axis direction or in the vertical axis direction and thus may obtain a one-dimensional image of the external environment. A one-dimensional image obtained according to various embodiments of the present disclosure may have a significantly smaller amount of information than that of a two-dimensional image and thus may requires a reduced amount of computation. Particularly, since a one-dimensional image has a limited amount of information, using a one-dimensional image makes it possible to prevent an invasion of privacy. A one-dimensional image is merely one line of a two-dimensional image. Thus, even though a one-dimensional image is exposed, a user's privacy is less likely to be exposed.

In various embodiments of the present disclosure, the sensor180may include a linear CCD or a linear CMOS device. A linear CCD or linear CMOS has an extra-small thin and long shape and can be installed in a very thin space. Further, the linear CCD or linear CMOS consumes low power and is relatively inexpensive compared to a camera. In addition, the linear CCD or linear CMOS does not provide simple information, such as the presence or absence of an object or distance, like a light level sensor or an ultrasonic sensor, but may analyze input information in real time to detect a movement of a user.

The sensor180according to various embodiments of the present disclosure may photograph users401and402to obtain a one-dimensional image. For example, each of the pixels181to187of the sensor180may convert incident light into an electrical signal and may output the electrical signal, in which the characteristics (for example, voltage, current, or power) of the electrical signal may be associated with the amount of the incident light. Accordingly, the sensor180may obtain luminance information corresponding to each of the pixels181to187to obtain a one-dimensional image. That is, the one-dimensional image may include the luminance information. In various embodiments of the present disclosure, information reflected in the one-dimensional image is not limited to the luminance information but includes any type of information without restriction. The electronic device101may analyze the one-dimensional image including the obtained luminance information and may obtain position information on the users401and402. For example, the electronic device101may obtain user position information including at least one of the position of the users401and402, the distance from the users401and402to the electronic device101, and motion information on the users401and402. When determining the position of the users401and402, the electronic device101may also determine the number of users401and402.

In various embodiments of the present disclosure, the electronic device101may perform an operation corresponding to the position information on the users401and402. For example, when the electronic device101is configured as a TV, the electronic device101may adjust the orientation of the display160or may adjust the curvature of the display160according to the position information on the users401and402, which will be described in detail later.

The sensor180includes the one-dimensional pixel array and thus may be configured in a relatively thin and long form. Accordingly, the sensor180may be easily disposed in a bezel of the electronic device101or the like.

In the embodiment ofFIG. 4B, the electronic device101may include a plurality of sensors180aand180b. Specifically, each of the sensors180aand180bmay be configured as an array having 1×N pixels (N is a natural number). The number of sensors of each of the sensors180aand180bmay be the same or may be different. The distance between the sensors180aand180bmay be determined depending on the photographing angle of each of the sensors180aand180bbut is not limited thereto. The electronic device101may determine position information on users401,402, and403using a one-dimensional image obtained from each of the plurality of sensors180aand180b. In particular, the electronic device101may determine information on the distance from the users401,402, and403to the electronic device101by comparing one-dimensional images obtained respectively from the plurality of sensors180aand180b. In the present document, a sensor may be configured in the form of a pixel array and thus may be referred to as a sensor array.

As described above, the electronic device101according to various embodiments of the present disclosure may obtain and analyze a one-dimensional image, thereby determining position information on a user. As the electronic device101uses a one-dimensional image, it is possible to reduce the amount of computation and to prevent an invasion of privacy. In particular, a sensor for obtaining a one-dimensional image can be configured as an array of pixels arranged in one dimension and thus can be disposed in a relatively narrow space, which may advantageous for weight reduction and miniaturization of an electronic device.

FIG. 5is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

In operation510, an electronic device101may obtain sensing data from a pixel array. In various embodiments of the present disclosure, the sensing data may be an electrical signal or a characteristic (for example, voltage, current, or power) of an electrical signal related to the amount of light incident on each pixel. Alternatively, the sensing data may be converted information, such as luminance, obtained by converting an electrical signal.

In operation520, the electronic device101may determine user position information on the basis of the sensing data. The electronic device101may determine user position information on the basis of sensing data from each pixel of the one-dimensional array. For example, the electronic device101may obtain a one-dimensional image using the sensing data from each pixel of the one-dimensional array. The electronic device101may divide the one-dimensional image into a region corresponding to a user from a region corresponding to a background and may determine the position of the region corresponding to the user.

In operation530, the electronic device101may perform an operation corresponding to the determined user position information. For example, the electronic device101may store in advance information on a relationship between the position of the region corresponding to the user and the operation. Alternatively, the electronic device101may store in advance information on a relationship between the position of a pixel corresponding to the user and the operation. Table 1 illustrates relationship information according to various embodiments of the present disclosure.

TABLE 1User positionOperation(−100)Rotate the display by −30° relative to the front(0)Adjust the screen to the front(100)Rotate the display by 30° relative to the front

The relationship information according to the embodiment of Table 1 may be relationship information about the rotation of the display of the electronic device101according to the user's position. For example, when it is determined that the user is positioned at (−100, 0) as a result of analyzing the sensing data from the sensor, the electronic device101may rotate the display by −30 degrees relative to the front. The coordinates in Table 1 may be coordinates in a one-dimensional image or the coordinates of a pixel. In various embodiments of the present disclosure, the electronic device101may perform the operation corresponding to the user position information, for example, on the basis of the relationship information illustrated in Table 1. In another embodiment, however, the electronic device101may calculate a rotation angle using the user position information and may rotate the display using the calculation result.

FIGS. 6A to 6Care flowcharts illustrating a method for determining various pieces of user position information according to various embodiments of the present disclosure.

Referring toFIG. 6A, in operation610, an electronic device101may obtain sensing data sensed by each of a plurality of pixels forming a sensor, for example, a pixel array.

In operation620, the electronic device101may perform background-person separation using the obtained sensing data. As described above, the electronic device101may obtain a one-dimensional image and may divide the one-dimensional image into a region corresponding to a background and a region corresponding to a person. In various embodiments of the present disclosure, the electronic device101may determine a feature point in the one-dimensional image and may separate the region corresponding to the background and the region corresponding to the person using the determined feature. In one embodiment, the electronic device101may determine, as a feature point (for example, an edge), a pixel having a numerical value (for example, luminance) of sensing data that is different by greater than a predetermined threshold value from that of a neighboring pixel. The electronic device101may determine, as a boundary point, the pixel determined as the feature point, may determine a region on one side of the boundary point as the region corresponding to the background, and may determine a region on the other side of the boundary point as the region corresponding to the person. There is no restriction on a method for the electronic device101to perform background-person separation using sensing data. For example, the electronic device101may perform background-person separation using various image processing techniques, such as a Support Vector Machine (SVM) technique.

In operation630, the electronic device101may obtain user position information on the basis of the position of a pixel classified as a person. In operation640, the electronic device101may perform an operation corresponding to the determined user position information.

FIG. 6Bis a flowchart illustrating a method for determining the horizontal position of a user according to various embodiments of the present disclosure. The embodiment ofFIG. 6Bwill be described in detail with reference toFIGS. 7A and 7B.FIGS. 7A and 7Bare conceptual diagrams illustrating a method for determining the horizontal position of a user according to various embodiments of the present disclosure.

Operations610and620inFIG. 6Bhave been described above with reference toFIG. 6A, and thus a detailed description thereof is omitted herein.

In operation631, the electronic device101may obtain horizontal position information on a user on the basis of the position of a pixel classified as a person. For example, as illustrated inFIG. 7A, a user701may be positioned relatively on the left side relative to the center of gravity of the electronic device101. In this case, the electronic device101may obtain sensing data from a sensor710including a plurality of pixels711to721. The electronic device101may analyze the sensing data and may separate regions711to714classified as a person and regions715to721classified as a background. As the user701is positioned relatively on the left side, the user701may be photographed at the pixels711to714disposed relatively on the left side of the sensor. That is, the electronic device101may obtain a one-dimensional image with the user701positioned relatively on the left side. The electronic device101may determine the horizontal position of the user701in the one-dimensional image.

Referring toFIG. 7B, the user701may be positioned relatively on the right side relative to the center of gravity of the electronic device101. In this case, the electronic device101may obtain sensing data from the sensor710including the plurality of pixels711to721. The electronic device101may analyze the sensing data and may separate regions718to721classified as a person and regions711to717classified as a background. As the user701is positioned relatively on the right side, the user701may be photographed at the pixels718to721disposed relatively on the right side of the sensor. That is, the electronic device101may obtain a one-dimensional image with the user701positioned relatively on the right side. The electronic device101may determine the horizontal position of the user701in the one-dimensional image.

In operation641, the electronic device101may perform an operation corresponding to the determined horizontal position information on the user701. For example, when the electronic device101determines that the user701is positioned relatively on the left side, the electronic device101may rotate the display to the left so that the user701positioned on the left side can look straightly at the display.

Although it has been illustrated that the electronic device101determines the horizontal position information on the user701, the foregoing embodiment is merely an example. In various embodiments of the present disclosure, the electronic device101may include a sensor that includes a pixel array in a vertical direction, in which case the electronic device101may determine vertical position information on the user701.

FIG. 6Cis a flowchart illustrating a method for determining the horizontal position of a user according to various embodiments of the present disclosure. The embodiment ofFIG. 6Cwill be described in detail with reference toFIGS. 7A and 7C.FIGS. 7A and 7Care conceptual diagrams illustrating a method for determining the horizontal position of a user according to various embodiments of the present disclosure.

Operations610and620inFIG. 6Chave been described above with reference toFIG. 6A, and thus a detailed description thereof is omitted herein.

In operation632, the electronic device101may obtain information on the distance between a user and the electronic device on the basis of the position of a pixel classified as a person. For example, referring toFIG. 7A, the user701may be positioned a distance of d1 from the surface of the electronic device101. In this case, the electronic device101may obtain sensing data from the sensor710including the plurality of pixels711to721. The electronic device101may analyze the sensing data and may separate regions711to714classified as a person and regions715to721classified as a background. As the distance between the user and the electronic device is d1, the user701may be photographed at a relatively large number of pixels711to714. That is, the electronic device101may obtain a one-dimensional image in which the user701is photographed as a relatively large image. The electronic device101may determine the information on the distance between the user and the electronic device on the basis of the size of the user701or the number of regions711to714classified as a person.

For example, referring toFIG. 7C, the user701may be positioned a distance of d2 from the surface of the electronic device101. In this case, the electronic device101may obtain sensing data from the sensor710including the plurality of pixels711to721. The electronic device101may analyze the sensing data and may separate regions711and712classified as a person and regions713to721classified as a background. As the distance between the user and the electronic device is d2, the user701may be photographed at a relatively small number of pixels711and712. That is, the electronic device101may obtain a one-dimensional image in which the user701is photographed as a relatively small image. The electronic device101may determine the information on the distance between the user and the electronic device on the basis of the size of the user701or the number of regions711to712classified as a person.

The greater the distance between the user and the electronic device is, the smaller the size of the user701in the one-dimensional image or the number of regions711and712classified as a person may be, while the smaller the distance between the user and the electronic device is, the greater the size of the user701in the one-dimensional image or the number of regions711to714classified as a person may be.

The electronic device101may store in advance information on a relationship between the size of a user in a one-dimensional image or the number of pixels corresponding to the user and the distance between the user and the electronic device. For example, the electronic device101may store relationship information illustrated in Table 2.

TABLE 2Number of pixelsDistance between user andcorresponding to userelectronic devicea1d1a2d2a3d3

The electronic device101may determine the distance between the user and the electronic device using the relationship information illustrated in Table 2.

In operation642, the electronic device101may perform an operation corresponding to the determined information on the distance between the user and the electronic device. For example, the electronic device101may adjust the curvature of the display according to the information on the distance.

As described above, the electronic device101may determine various types of user position information, such as the horizontal position information on the user or the information on the distance between the user and the electronic device, using sensing data from one sensor including a pixel array.

In various embodiments of the present disclosure, each pixel of a pixel array may also be configured with a phase pixel, such as a photodiode (2PD). Specifically, one pixel of the pixel array may include two sub-pixels, and the amounts of light detected by the two sub-pixels may be different. The electronic device101may determine the distance between the user and the electronic device101on the basis of the difference between sensing data sensed by one sub-pixel and sensing data sensed by the other sub-pixel. The electronic device101may determine the horizontal position information on the user and the information on the distance between the user and the electronic device101using sensing data from the pixel array including the phase pixel.

FIG. 8is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure. The embodiment ofFIG. 8will be described in detail with reference toFIGS. 9A to 9H.FIGS. 9A to 9Hare conceptual diagrams illustrating a result of person-background separation by a pixel array according to various embodiments of the present disclosure.

In operation810, an electronic device101may obtain sensing data sensed by each of a plurality of pixels forming a pixel array at a first time. In operation820, the electronic device101may determine first user position information corresponding to the first time. In operation830, the electronic device101may obtain sensing data sensed by each of the plurality of pixels forming the pixel array at a second time. In operation840, the electronic device101may determine second user position information corresponding to the second time. That is, the electronic device101may obtain a plurality of one-dimensional images over time.

In operation850, the electronic device101may determine user movement information using the first user position information and the second user position information. In operation860, the electronic device101may perform an operation corresponding to the determined user movement information.

For example, the electronic device101may perform person-background separation using sensing data from a sensor including a plurality of pixels911to928at a first time. Referring toFIG. 9A, the electronic device101may separate all of the pixels911to928as the background at the first time. The electronic device101may perform person-background separation using sensing data from the sensor including the plurality of pixels911to928at a second time. Referring toFIG. 9B, the electronic device101may separate pixels912to914as a person region and may separate the remaining pixels911and915to928as a background region at the second time. The electronic device101may perform person-background separation using sensing data from the sensor including the plurality of pixels911to928at a third time. Referring toFIG. 9C, the electronic device101may separate pixels916to919as a person region and may separate the remaining pixels911to915and920to928as a background region at the third time. The electronic device101may perform person-background separation using sensing data from the sensor including the plurality of pixels911to928at a fourth time. Referring toFIG. 9D, the electronic device101may separate pixels919to923as a person region and may separate the remaining pixels911to918and924to928as a background region at the fourth time. That is, the electronic device101may determine that the person region, that is, the position of a user, is moved relatively to the right over time, and may determine that user movement information is a rightward movement. The electronic device101may perform an operation corresponding to rightward movement information. For example, the electronic device101may rotate a display in response to the user's position in real time. In various embodiments of the present disclosure, the electronic device101may obtain information on the user's moving speed by dividing the user movement information by the time spent. As described above, the electronic device101according to various embodiments of the present disclosure may determine information on a horizontal or vertical movement of the user.

The electronic device101may determine information on a movement of the user in a direction between the user and the electronic device. Referring toFIG. 9E, the electronic device101may separate pixels918to921as a person region and may separate the remaining pixels911to917and922to928as a background region at a first time. The electronic device101may perform person-background separation using sensing data from the sensor including the plurality of pixels911to928at a second time. Referring toFIG. 9F, the electronic device101may separate pixels918to923as a person region and may separate the remaining pixels911to917and924to928as a background region at the second time. The electronic device101may perform person-background separation using sensing data from the sensor including the plurality of pixels911to928at a third time. Referring toFIG. 9G, the electronic device101may separate pixels915to925as a person region and may separate the remaining pixels911to914and926to928as a background region at the third time. The electronic device101may perform person-background separation using sensing data from the sensor including the plurality of pixels911to928at a fourth time. Referring toFIG. 9H, the electronic device101may separate pixels914to928as a person region and may separate the remaining pixels911to913as a background region at the fourth time. That is, the electronic device101may determine that a relatively large number of pixels correspond to the user relating to the person region, that is, the position of the user, over time. The electronic device101may determine user movement information indicating that the user approaches the electronic device101over time. The electronic device101may perform an operation corresponding to the movement information of the approach. For example, the electronic device101may adjust the curvature of the display in response to the user's approach in real time. In various embodiments of the present disclosure, the electronic device101may obtain information on the user's moving speed by dividing the user movement information by the time spent.

FIG. 10is a flowchart illustrating a method for controlling an electronic device including a plurality of pixel arrays according to various embodiments of the present disclosure. The embodiment ofFIG. 10will be described in detail with reference toFIGS. 11A and 11B.FIGS. 11A and 11Bare conceptual diagrams illustrating the operation of an electronic device including a plurality of pixel arrays according to various embodiments of the present disclosure.

In operation1010, the electronic device101may obtain first sensing data sensed at each of a plurality of pixels forming a first pixel array. For example, referring toFIG. 11A, the electronic device101may include a first pixel array1110and a second pixel array1120. The first pixel array1110may include a plurality of pixels arranged in one dimension, and the second pixel array1120may include a plurality of pixels arranged in one dimension. The first pixel array1110and the second pixel array1120may be disposed in order to thoroughly take images in a particular direction of the electronic device101. For example, the distance between the first pixel array1110and the second pixel array1120may be set in view of the photographing angle of each of the first pixel array1110and the second pixel array1120. In the embodiment ofFIG. 11A, it is assumed that a user1130is spaced D3from the surface of the electronic device101and is spaced X1leftward from an axis passing through the center of gravity of the electronic device101. The electronic device101may obtain first sensing data1140illustrated inFIG. 11B. Although the first sensing data1140illustrated inFIG. 11Bis a result of person-background separation, those skilled in the art would readily understand that the electronic device101can obtain the first sensing data1140by first obtaining raw data, such as luminance or electrical characteristics, from each of the pixels1141to1151of the first array1110and applying person-background separation to the raw data.

In operation1020, the electronic device101may obtain second sensing data sensed at each of a plurality of pixels forming a second pixel array. The electronic device101may obtain, for example, second sensing data1160illustrated inFIG. 11B. Although the second sensing data1160illustrated inFIG. 11Bis a result of person-background separation, those skilled in the art would readily understand that the electronic device101can obtain the second sensing data1160by first obtaining raw data, such as luminance or electrical characteristics, from each of the pixels1161to1171of the second array1120and applying person-background separation to the raw data.

In operation1030, the electronic device101may obtain horizontal position information on a user using the first sensing data and the second sensing data. For example, the electronic device101may divide the first sensing data1140illustrated inFIG. 11Binto pixels1141to1147as a background region and pixels1148to1151as a person region. The electronic device101may divide the second sensing data1160illustrated inFIG. 11Binto pixels1163to1171as a background region and pixels1161and1162as a person region. The electronic device101may determine the position of the user1130relative to the first pixel array1110and determine the position of the user1130relative to the second pixel array1120. The electronic device101according to various embodiments of the present disclosure may determine the position of the user1130, for example, relative to an axis passing through the center of gravity, using the position of the user1130relative to the first pixel array1110and the position of the user1130relative to the second pixel array1120. In another embodiment, the electronic device101may determine the position of the user1130relative to an axis passing through the center of gravity directly using the first sensing data and the second sensing data.

In operation1040, the electronic device101may apply the first sensing data and the second sensing data to a stereo vision technique, thereby obtaining information the distance between the user and the electronic device. Referring to FIG.11A, since the first pixel array1110and the second pixel array1120are disposed at different positions in the electronic device101, the relative position of the user1130with respect to the second pixel array1120may be different from the relative position of the user1130with respect to the second pixel array1120. Accordingly, as illustrated inFIG. 11B, the first sensing data1140sensed from the first pixel array1110may be different from the second sensing data1160sensed from the second pixel array1120. In various embodiments of the present disclosure, the electronic device101may compare the first sensing data1140with the second sensing data1160and may determine that the distance from the user1130to the electronic device101is D3on the basis of the comparison result. The electronic device101may determine the distance between the user1130and the electronic device101using various image analysis techniques including an image analysis technique for analyzing distance using a plurality of images captured by a plurality of cameras corresponding to both eyes, for example, a stereo vision technique, but there is no restriction on image analysis techniques used for distance analysis.

In operation1050, the electronic device101may perform an operation corresponding to at least one of the determined horizontal position information on the user and the information on the distance between the user and the electronic device.

As described above, the electronic device101according to various embodiments of the present disclosure may determine more accurate user position information using sensing data from a plurality of pixel arrays.

FIG. 12is a flowchart illustrating a method for controlling an electronic device using a database according to various embodiments of the present disclosure. The embodiment ofFIG. 12will be described in detail with reference toFIGS. 13A to 13D.FIGS. 13A to 13Dare conceptual diagrams illustrating that an electronic device determines user position information using a database according to various embodiments of the present disclosure.

In operation1210, the electronic device101may obtain sensing data sensed by each of a plurality of pixels forming a pixel array. In operation1220, the electronic device101may perform background-person separation using the obtained sensing data and a pre-stored reference database. In operation1230, the electronic device101may obtain user position information on the basis of the position of a pixel classified as a person. In operation1240, the electronic device101may perform an operation corresponding to the determined user position information.

For example, referring toFIG. 13A, the electronic device101may include a pixel array1300. For example, the electronic device101may be configured as a TV and may be disposed in a home environment. Other pieces of furniture1301and1302included in the home environment may be disposed in front of the electronic device101. For example, a sofa1301may be disposed on the relatively front right side of the electronic device101, and a table1302may be disposed on the relatively front left side of the electronic device101. The electronic device101may photograph an environment where there is no user, that is, a usual environment. Specifically, the electronic device101may pre-store, as a database, a one-dimensional image obtained by photographing the usual environment using the pixel array1300.

FIG. 13Billustrates an example of the one-dimensional image1310obtained by the electronic device101photographing the usual environment. As illustrated inFIG. 13B, the one-dimensional image1310may include raw data, for example, a luminance values, for each of pixels1311to1324. The luminance value may be a natural number among values of 0 to 255. As illustrated inFIG. 13B, pixels1316to1318of the one-dimensional image1310may have a relatively smaller value than neighboring pixels, which may result from the sofa1301being included in the image. In addition, pixels1323and1324of the one-dimensional image1310may have a relatively smaller value than neighboring pixels, which may result from a table1302being included in the image.

FIG. 13Cis a conceptual diagram assuming that a user1303is positioned in front of the electronic device101. As illustrated inFIG. 13C, the user1303may be positioned between the sofa1301and the table1302. In this case, the electronic device101may obtain a one-dimensional image1340illustrated inFIG. 13Dor sensing data using the pixel array1300. The obtained one-dimensional image1340includes pixels1349to1352respectively having relatively low luminance values of 17, 14, 5, and 9, which may result from the user1303being included in the image. Referring toFIG. 13C, as the user1303is photographed at the positions of the pixels1349to1352, it may be difficult to distinguish the pixels1349to1352corresponding to the user1303from pixels1346to1348corresponding to the sofa1301. As described above, the electronic device101according to various embodiments of the present disclosure may separate a person from a background using a feature point as a boundary point in a person-background separation process. However, as illustrated inFIG. 13D, the difference in luminance value between the pixels1346to1348corresponding to the sofa1301and the pixels1349to1352corresponding to the user1303may be relatively small. In this case, the electronic device101cannot classify, for example, the pixel1348or pixel1349as a feature point and thus cannot determine the accurate position of the user1303.

The electronic device101may perform person-background separation on the basis of the result of comparing a previously stored database and the obtained sensing data. For example, as illustrated inFIG. 13D, the electronic device101may compare the one-dimensional image1310of the pre-stored usual environment with the obtained one-dimensional image1340. In various embodiments of the present disclosure, the electronic device101may obtain a difference image1360from the two images1310and1340. The difference image1360may be an image including a difference between the pixels of the two images1310and1340as a value. For example, a pixel1368of the difference image1360may have a value of 0 that is the difference between the value of the pixel1318of the one-dimensional image1310in the database, which is 5, and the value of the pixel1348of the obtained one-dimensional image1340, which is 5. A pixel1369of the difference image1360has a value of 77 that is the difference between the value of the pixel1319of the one-dimensional image1310in the database, which is 94, and the value of the pixel1349of the obtained one-dimensional image1340, which is 17. A pixel1370of the difference image1360has a value of 95 that is the difference between the value of the pixel1320of the one-dimensional image1310in the database, which is 109, and the value of the pixel1350of the obtained one-dimensional image1340, which is 14. A pixel1371of the difference image1360has a value of 130 that is the difference between the value of the pixel1321of the one-dimensional image1310in the database, which is 135, and the value of the pixel1351of the obtained one-dimensional image1340, which is 5. A pixel1372of the difference image1360has a value of 132 that is the difference between the value of the pixel1322of the one-dimensional image1310in the database, which is 141, and the value of the pixel1352of the obtained one-dimensional image1340, which is 9. A pixel1373of the difference image1360has a value of 0 that is the difference between the value of the pixel1323of the one-dimensional image1310in the database, which is 15, and the value of the pixel1353of the obtained one-dimensional image1340, which is 15. Accordingly, the pixel1369of the difference image1360may have a relatively significant difference from the neighboring pixel1368, and the pixel1372may have a relatively significant difference from the neighboring pixel1373. The electronic device101may classify the pixels1369and pixels1372of the difference image1360as feature points, may classify the pixels1369to1372as a region corresponding to a person, and may classify the pixels1361to1368,1373, and1374as a region corresponding to a background.

As described above, the electronic device101according to the various embodiments of the present disclosure may determine accurate position information on the user using the result of comparing the database, such as the difference image1360, and the obtained sensing data.

In various embodiments of the present disclosure, the electronic device101may monitor a surrounding environment. The electronic device101may periodically or aperiodically photograph the surrounding environment using the sensor array1300and may store a captured image as a database. That is, the electronic device101may update the database and accordingly may accurately determine the position information on the user by referring to changes in the surrounding environment.

FIGS. 14A and 14Bare conceptual diagrams illustrating the arrangement of an electronic device according to various embodiments of the present disclosure.

As illustrated inFIG. 14A, an electronic device1420may be configured separately from another electronic device1410. For example, the electronic device1420according to various embodiments of the present disclosure may include only a sensor array including a plurality of sensors and a communication module capable of transmitting data1430sensed by the sensor array. In this case, the electronic device1420may be driven with low power, including only the sensors configured in a one-dimensional array and the communication modules, and may be advantageous for establishing an Internet of Things (IoT) environment. In this case, the communication module (not shown) of the electronic device1420may include a stack for various modes of wireless communication, and there is no restriction on the wireless communication modes.

The electronic device1420may transmit the sensing data1430sensed by the pixel array to the electronic device1410. The electronic device1410may determine position information on a user1401based on the electronic device1420using the received sensing data1430. The electronic device1410may store in advance a positional relationship between the electronic device1410and the electronic device1420and may determine position information on the user1401based on the electronic device1410using the positional relationship and the received sensing data1430. The electronic device1410may store in advance information on a relationship the received sensing data1430and the position information on the user1401based on the electronic device1410and may determine the position information on the user1401based on the electronic device1410using the stored information on the relationship.

FIG. 14Bis a conceptual diagram illustrating a home network environment according to various embodiments of the present disclosure.

As illustrated inFIG. 14B, a home network1440may include at least one electronic device1441,1442, or1443. In addition, at least one pixel array1451,1452,1453, or1454may be disposed in the home network1440. The at least one electronic device1441,1442, or1443and the at least one pixel array1451,1452,1453, or1454may communicate with each other and may perform data transmission and reception. For example, each of the at least one pixel array1451,1452,1453, or1454may transmit sensing data to each of the at least one electronic device1441,1442, or1443. Each of the at least one electronic device1441,1442, or1443may determine position information on the user1401using the received sensing data. For example, each of the at least one electronic device1441,1442, or1443may determine that the user1401is moving. A TV1441may adjust a display in response to the movement of the user1401, an air conditioner1442may adjust an air discharge direction in response to the movement of the user1401, and a light1443may adjust a light emission direction in response to the movement of the user1401.

As described above, the at least one pixel array1451,1452,1453, or1454may include only a sensor array and a communication module and may be disposed at a plurality of portions of the home network1440as illustrated inFIG. 14B. The at least one pixel array1451,1452,1453, or1454may be detachable.

FIG. 15is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

In operation1510, a first electronic device101-1may obtain sensing data sensed by each of a plurality of pixels forming a pixel array. The first electronic device101-1may include a pixel array including a plurality of pixels. In addition, the first electronic device101-1may include a communication module capable of transmitting sensing data. In operation1520, the first electronic device101-1may transmit the sensing data.

In operation1530, a second electronic device101-2may obtain user position information using the received sensing data. The second electronic device101-2may store in advance information on a relationship between the received sensing data and user position information based on the second electronic device101-2. The second electronic device101-2may determine user position information based on the second electronic device101-2using the received sensing data and the information on the relationship. In operation1540, the second electronic device101-2may perform an operation corresponding to the determined user position information.

As described above, an electronic device according to various embodiments of the present disclosure may transmit sensing data sensed by a pixel array including a plurality of pixels to another electronic device. In addition, an electronic device according to various embodiments of the present disclosure may determine user position information based on the electronic device using sensing data, for example, a one-dimensional image, received from another electronic device.

FIG. 16is a conceptual diagram illustrating an electronic device according to various embodiments of the present disclosure. Referring toFIG. 16, the electronic device101according to various embodiments of the present disclosure may include a sensor180that includes a pixel array extending in a vertical direction. In this case, the electronic device101may determine vertical position information on a user and may perform an operation corresponding to the vertical position information on the user. For example, the electronic device101may determine the user's gaze using the sensing data from the sensor180including the vertical pixel array and may adjust a tilting level according to the height of the user's gaze.

Further, the electronic device101may include at least one of a horizontal pixel array and a vertical pixel array, in which case the electronic device101may recognize the user's motion by tracking the position of the user. When the electronic device101includes a horizontal pixel array, the electronic device101may recognize a horizontal motion of the user. When the electronic device101includes a vertical pixel array, the electronic device101may recognize a vertical motion of the user. When the electronic device101includes a horizontal pixel array and a vertical pixel array, the electronic device101may recognize a two-dimensional user motion. In addition, as described above, the electronic device101may also determine the distance between the user and the electronic device101and thus may recognize a three-dimensional user motion.

FIGS. 17 to 21are conceptual diagrams illustrating the operation of an electronic device in accordance with user position information according to various embodiments of the present disclosure.

Referring toFIG. 17, an electronic device101may include a display1701. Here, the display1701may be formed to be curved. The electronic device101may adjust the curvature of the display1701, and thus may adjust the display1701to have a first curvature, may adjust the display1702to have a second curvature, or may adjust the display1703to have a third curvature.

The electronic device101may determine position information on a user1711using sensing data sensed by a sensor including a one-dimensional pixel array. As described above, the electronic device101may determine information on the distance between the user1711and the electronic device101using sensing data from one pixel array. Specifically, the electronic device101may determine the information on the distance between the user1711and the electronic device101on the basis of the number of pixels classified as a person in the pixel array or the size of a person in a one-dimensional image. Alternatively, the electronic device101may determine the information on the distance between the user1711and the electronic device101using sensing data sensed by each of a plurality of pixel arrays. Specifically, the electronic device101may determine the information on the distance between the user1711and the electronic device101using the difference between pieces of sensing data sensed by each of the plurality of pixel arrays.

In the embodiment ofFIG. 17, the electronic device101may determine the approach of the user1711to the electronic device101, that is, movement information indicating a decrease in the distance between the user1711and the electronic device101. Accordingly, the electronic device101may adjust the curvature of the display. Further, the electronic device101may analyze a user behavior pattern using movement information on the user. The electronic device101may identify the user on the basis of the analyzed behavior pattern and may recommend or provide an advertisement using user-specific characteristics.

Further, the electronic device101may provide various services, such as adjusting the display brightness of a TV or controlling the turn on/off, on the basis of the distance between the user1711and the electronic device101.

Referring toFIG. 18, the electronic device101may include a display having the curvature adjustable. The electronic device101may adjust the display1810to have a first curvature and may adjust the display1811to have a second curvature.

The electronic device101may determine the number of users1821,1822, and1823. In various embodiments of the present disclosure, the electronic device101may perform person-background separation using sensing data. The electronic device101may determine the number of users according to the number of regions classified as a person. For example, as illustrated in the upper portion ofFIG. 18, the electronic device101may determine that there are three users1821,1822, and1823and may adjust the curvature of the display1810according to the number of users. Further, as illustrated in the lower portion ofFIG. 18, the electronic device101may determine that there is one user1821and may adjust the curvature of the display1811according to the number of users.

The electronic device101may also recommend a program on the basis of the number of users or may use the presence or absence of a viewer or the number of viewers for a poll.

Referring toFIG. 19, the electronic device101may include a rotatable display. The electronic device101may determine information on the movement1912of a user1911. For example, referring toFIG. 19, the electronic device101may determine that the user1911moves1912relatively to the left. The electronic device101may rotate the display1920based on the information on the movement1912of the user1911. The electronic device101may rotate the display1920such that the user1911can look straightly at the display1920.

Referring toFIG. 20, the electronic device101may include a pixel array2010. Further, the electronic device101may include a plurality of speakers2011and2012. The electronic device101may determine position information on a user2020using sensing data sensed by the pixel array2010. The electronic device101may determine the direction of beamforming2031and2032of the plurality of speakers2011and2010using the position information on the user2020. The electronic device101may adjust the audio output delay of the speakers2011and2010so that the beamforming2031and2032is formed to the position of the user2220.

In various embodiments of the present disclosure, the electronic device101may include a plurality of microphones instead of the speakers2011and2012. The electronic device101may determine the beamforming direction of the microphones using the determined position information on the user, thus obtaining a more accurate user's voice.

Referring toFIG. 21, the electronic device101may determine position information on a user2120using sensing data sensed by a pixel array. The electronic device101may adjust the discharge direction of air2130using the position information on the user2120.

FIGS. 17 to 21show various operations corresponding to user position information, which are merely illustrative, and those skilled in the art will readily understand that there is no restriction on the operations that can be performed on the basis of user position information.

According to various embodiments of the present disclosure, a control method of an electronic device including a plurality of pixels disposed in a first direction may include: determining position information on a user using sensing data sensed by the plurality of pixels; and performing an operation corresponding to the position information on the user.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: dividing the plurality of pixels into pixels corresponding to a background and pixels corresponding to a person using the sensing data; and determining the position information on the user on the basis of the pixels corresponding to the person.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: determining a feature point from the sensing data; and dividing the plurality of pixels into the pixels corresponding to the background and the pixels corresponding to the person using the sensing data on the basis of the determined feature point.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: determining a horizontal position or a vertical position of the user on the basis of the positions of the pixels corresponding to the person.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: determining information on the distance between the user and the electronic device on the basis of the number of the pixels corresponding to the person.

According to various embodiments of the present disclosure, each of the plurality of pixels may include a plurality of sub-pixels, and the determining of the position information on the user may include determining information on a distance between the user and the electronic device on the basis of a difference between pieces of sensing data sensed by a plurality of sub-pixels of each of the pixels corresponding to the person

According to various embodiments of the present disclosure, the electronic device may include a first pixel array including a plurality of pixels disposed in the first direction and a second pixel array including a plurality of other pixels disposed in the first direction, and the determining of the position information on the user may include determining information on a distance between the user and the electronic device on the basis of a difference between pixels corresponding to the person in the first pixel array and pixels corresponding to the person in the second pixel array.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: applying a stereo vision technique to the pixels corresponding to the person in the first pixel array and the pixels corresponding to the person in the second pixel array; and determining the information on the distance between the user and the electronic device on the basis of an application result.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: comparing the sensing data with a reference database; and determining the position information on the user using a comparison result.

According to various embodiments of the present disclosure, the determining of the position information on the user may include: generating a difference image between the sensing data and the reference database; dividing a plurality of pixels of the difference image into pixels corresponding to the background and pixels corresponding to the person; and determining the position information on the user on the basis of the pixels corresponding to the person.

According to various embodiments of the present disclosure, each of the plurality of pixels may sense a plurality of pieces of sensing data over time, and the determining of the position information on the user may include: dividing the plurality of pixels into pixels corresponding to the background and pixels corresponding to the person using each of the plurality of pieces of sensing data sensed over time; and determining movement information on the user on the basis of a change in the pixels corresponding to the person over time.

Each of the components of the electronic device according to the present disclosure may be implemented by one or more components and the name of the corresponding component may vary depending on a type of the electronic device. In various embodiments, the inspection apparatus may include at least one of the above-described elements. Some of the above-described elements may be omitted from the electronic device, or the inspection apparatus may further include additional elements. Further, some of the components of the electronic device according to the various embodiments of the present disclosure may be combined to form a single entity, and thus, may equivalently execute functions of the corresponding elements prior to the combination.

The term “module” as used herein may, for example, mean a unit including one of hardware, software, and firmware or a combination of two or more of them. The “module” may be interchangeably used with, for example, the term “unit”, “logic”, “logical block”, “component”, or “circuit”. The “module” may be the smallest unit of an integrated component or a part thereof. The “module” may be the smallest unit that performs one or more functions or a part thereof. The “module” may be mechanically or electronically implemented. For example, the “module” according to the present disclosure may include at least one of an Application-Specific Integrated Circuit (ASIC) chip, a Field-Programmable Gate Arrays (FPGA), and a programmable-logic device for performing operations which has been known or are to be developed hereinafter.

According to various embodiments, at least some of the devices (for example, modules or functions thereof) or the method (for example, operations) according to the present disclosure may be implemented by a command stored in a computer-readable storage medium in a programming module form. When the command is executed by one or more processors (for example, the processor120), the one or more processors may execute a function corresponding to the command. The computer-readable storage medium may, for example, be the memory130.

The programming module according to the present disclosure may include one or more of the aforementioned components or may further include other additional components, or some of the aforementioned components may be omitted. Operations executed by a module, a programming module, or other component elements according to various embodiments of the present disclosure may be executed sequentially, in parallel, repeatedly, or in a heuristic manner. Furthermore, some operations may be executed in a different order or may be omitted, or other operations may be added.

According to various embodiments of the present disclosure, a storage medium may store instructions, wherein the instructions may be configured for at least one processor to perform at least one operation when executed by the at least one processor, the at least one operation including: determining position information on a user using sensing data sensed by a plurality of pixels; and performing an operation corresponding to the position information on the user.

Various embodiments disclosed herein are provided merely to easily describe technical details of the present disclosure and to help the understanding of the present disclosure, and are not intended to limit the scope of the present disclosure. Therefore, it should be construed that all modifications and changes or modified and changed forms based on the technical idea of the present disclosure fall within the scope of the present disclosure.