SYSTEMS AND METHODS FOR IMAGE ARCHIVING

The present disclosure relates to systems and methods for retrieving image data. The systems may obtain a search request from a user device, the search request including at least one keyword. The systems may identify, in an image database, one or more target image archives associated with one or more target tags, respectively. The systems may retrieve, from the image database, the one or more target image archives, each of the one or more target image archives including a plurality of target images. The systems may transmit the one or more target image archives to the user device to be displayed via a user interface of the user device.

TECHNICAL FIELD

The present disclosure generally relates to systems and methods for image archiving, and in particular, to systems and methods for image archiving based on feature information of images.

BACKGROUND

Image archiving is an important hot spot in various fields. For image archiving in family scenario (e.g., a daily life record), an image archiving system may capture and store a large amount of images every day. The capturing of the images may be triggered by various factors such as a detected object or an event. However, it is difficult for a user to identify a target image from the large amount of images or to flexibly group the images into different categories or folders. Therefore, it is desirable to provide systems and methods for automatically archiving images based on different features of the images and retrieving target images based on keywords included in search requests from a user.

SUMMARY

An aspect of the present disclosure relates to a system for retrieving image data. The system may include at least one storage medium including a set of instructions and at least one processor in communication with the at least one storage medium. When executing the set of instructions, the at least one processor may be configured to cause the system to perform one or more of the following operations. The at least one processor may obtain a search request from a user device, the search request including at least one keyword. The at least one processor may identify, in an image database, one or more target image archives associated with one or more target tags, respectively. The at least one processor may retrieve, from the image database, the one or more target image archives, each of the one or more target image archives including a plurality of target images. The at least one processor may transmit the one or more target image archives to the user device to be displayed via a user interface of the user device.

Another aspect of the present disclosure relates to a method implemented on a computing device having at least one processor, at least one storage medium, and a communication platform connected to a network. The method may include one or more of the following operations. The at least one processor may obtain a search request from a user device, the search request including at least one keyword. The at least one processor may identify, in an image database, one or more target image archives associated with one or more target tags, respectively. The at least one processor may retrieve, from the image database, the one or more target image archives, each of the one or more target image archives including a plurality of target images. The at least one processor may transmit the one or more target image archives to the user device to be displayed via a user interface of the user device.

A further aspect of the present disclosure relates to a non-transitory computer readable medium including executable instructions. When the executable instructions are executed by at least one processor, the executable instructions may direct the at least one processor to perform a method. The method may include one or more of the following operations. The at least one processor may obtain a search request from a user device, the search request including at least one keyword. The at least one processor may identify, in an image database, one or more target image archives associated with one or more target tags, respectively. The at least one processor may retrieve, from the image database, the one or more target image archives, each of the one or more target image archives including a plurality of target images. The at least one processor may transmit the one or more target image archives to the user device to be displayed via a user interface of the user device.

In some embodiments, the at least one processor may obtain a plurality of images from a camera device. For each of the plurality of images, the at least one processor may determine feature information of the image and determine one or more tags associated with the image based on the feature information. The at least one processor may generate one or more image archives based on the plurality of images and a plurality of tags associated with the plurality of images.

In some embodiments, the feature information may include at least one of object information, expression information, event information, time information, and/or location information.

In some embodiments, for at least two images of the plurality of images, the at least one processor may determine that the at least two images direct to a same tag and correlate the at least two images based on the same tag.

In some embodiments, the at least one processor may obtain an image captured by a camera device. The at least one processor may determine feature information of the image. The at least one processor may determine one or more tags associated with the image based on the feature information. The at least one processor may identify one or more image archives associated with the one or more tags, respectively. The at least one processor may archive, in the image database, the image to the one or more image archives, respectively.

In some embodiments, the search request may be associated with at least one of a text input, an audio input, an image input, and/or a video input.

DETAILED DESCRIPTION

An aspect of the present disclosure relates to systems and methods for establishing an image database. The systems and methods may obtain a plurality of images (e.g., a still image, a video) from a camera device. For each of the plurality of images, the systems and methods may determine feature information (e.g., object information, expression information, event information) of the image and determine one or more tags (e.g., a description associated with the feature information) associated with the image based on the feature information. The systems and methods may further determine one or more image archives based on the plurality of images and a plurality of tags associated with the plurality of images. Accordingly, the systems and methods may establish the image database based on the one or more image archives.

Another aspect of the present disclosure relates to systems and methods for retrieving target image archives from the image database based on a search request from a user. The systems and methods may obtain a search request including one or more keywords from a user device (e.g., a smart phone). The systems and methods may determine one or more target tags based on the one or more keywords and identify one or more target image archives associated with the one or more target tags in the image database. Further, the systems and methods may retrieve the one or more target image archives from the image database and transmit the one or more target image archives to the user device to be displayed via a user interface of the user device.

The systems and methods may automatically and efficiently determine tags for the captured images, and identify target image archives in response to a search request based on the tags. Accordingly, the systems and methods provide a solution for a user to search for any target image based on a search request including one or more keywords.

FIG. 1is a schematic diagram illustrating an exemplary image auto-archiving system according to some embodiments of the present disclosure. The image auto-archiving system100may include a server110, a network120, a camera device130, a user device140, and a storage150.

In some embodiments, the server110may be a single server, or a server group. The server group may be centralized, or distributed (e.g., server110may be a distributed system). In some embodiments, the server110may be local or remote. For example, the server110may access information and/or data stored in the camera device130, the user device140, and/or the storage150via the network120. As another example, the server110may be directly connected to the camera device130, the user device140, and/or the storage150to access stored information and/or data. In some embodiments, the server110may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof. In some embodiments, the server110may be implemented on a computing device200having one or more components illustrated inFIG. 2in the present disclosure.

In some embodiments, the server110may include a processing engine112. The processing engine112may process information and/or data to perform one or more functions described in the present disclosure. For example, the processing engine112may obtain one or more images from the camera device130and determine one or more tags associated with each of the one or more images based on feature information of the one or more images. In some embodiments, the processing engine112may include one or more processing engines (e.g., signal-core processing engine(s) or multi-core processor(s)). Merely by way of example, the processing engine112may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction-set processor (ASIP), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction-set computer (RISC), a microprocessor, or the like, or any combination thereof.

The network120may facilitate exchange of information and/or data. In some embodiments, one or more components of the auto-archiving system100(e.g., the server110, the camera device130, the user device140, or the storage150) may transmit information and/data to other component(s) of the image auto-archiving system100via the network120. For example, the server110may receive a plurality of images from the camera device130via the network120. In some embodiments, the network120may be any type of wired or wireless network, or any combination thereof. Merely by way of example, the network120may include a cable network, a wireline network, an optical fiber network, a tele communications network, an intranet, an internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public telephone switched network (PTSN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or the like, or any combination thereof. In some embodiments, the network120may include one or more network access points. For example, the network120may include wired or wireless network access points such as base stations and/or internet exchange points120-1,120-2, . . . , through which one or more components of the image auto-archiving system100may be connected to the network120to exchange data and/or information between them.

The camera device130may be configured to capture images associated with one or more objects. As used herein, the “image” may refer to a static image, a series of image frames, a video, etc. The object may include a person, an animal, a facility, a furniture, a decoration, etc. In some embodiments, the camera device130may include a spherical camera, a hemispherical camera, a rifle camera, etc. In some embodiments, the camera device130may include a black-white camera, a color camera, an infrared camera, an X-ray camera, etc. In some embodiments, the camera device130may include a digital camera, an analog camera, etc. In some embodiments, the camera device130may include a monocular camera, a binocular camera, a multi-camera, etc. In some embodiments, the camera device130may be an IP camera which can transmit the captured images to any component (e.g., the server110, the user device140, the storage150) of the image auto-archiving system100via the network120.

In some embodiments, the camera device130may automatically start a capture process in response to a first triggering event (e.g., a detection of a specific object) and stop the capture process in response to a second triggering event (e.g., a disappearance of the specific object). In some embodiments, the camera device130may automatically determine capture parameters (e.g., a brightness, a focal length, a capture time interval, an exposure time, a contrast, a saturation) based on characteristics of a scene or a target object. For example, the camera device130may automatically adjust the brightness based on a light (e.g., sunlight, lamplight, low light) of the scene. As another example, the camera device130may automatically adjust the capture time interval based on a type of the target object. For a moving target object, the capture time interval may be relatively short for an ordinary person in the art; whereas for a still target object, the capture time interval may be relatively long for an ordinary person in the art. As a further example, the camera device130may automatically adjust the exposure time based on a type of the scene. For a landscape photography, the exposure time may be relatively long for an ordinary person in the art; whereas for a portrait photography, the exposure time may be relatively short for an ordinary person in the art.

In some embodiments, the camera device130may include a built-in clock or communicate with an external clock which can record time information associated with images captured by the camera device130. In some embodiments, the camera device130may include a positioning module with positioning technology for locating the position of the camera device130. The positioning technology may be based on a global positioning system (GPS), a global navigation satellite system (GLONASS), a compass navigation system (COMPASS), a Galileo positioning system, a quasi-zenith satellite system (QZSS), a wireless fidelity (WiFi) positioning technology, or the like, or any combination thereof.

In some embodiments, the images captured by the camera device130may be stored in a local storage device (not shown) in the camera device130or a remote storage device (e.g., the storage150, a cloud storage) associated with the image auto-archiving system100. In some embodiments, both the local storage device and the remote storage device may be encrypted based on an encryption algorithm (e.g., Advanced Encryption Standard Algorithm, Digital Signature Algorithm). One or more components of the image auto-archiving system100may access the storage device via a secret key. In some embodiments, one or more components of the image auto-archiving system100may have permissions to access the storage device. For example, a registered user device can access the storage device via registration information (e.g., a user name, a password, a face recognition, a voice recognition, a fingerprint).

The user device140may be connected to and/or communicate with the camera device130, the processing engine112, and/or the storage150. The user device140include a mobile device, a tablet computer, a laptop computer, or the like, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home device may include a smart lighting device, a control device of an intelligent electrical apparatus, a smart monitoring device, a smart television, a smart video camera, an interphone, or the like, or combination thereof. In some embodiments, the wearable device may include a smart bracelet, a smart footgear, a smart glass, a smart helmet, a smart watch, a smart clothing, a smart backpack, a smart accessory, or the like, or any combination thereof. In some embodiments, the smart mobile device may include a smartphone, a personal digital assistance (PDA), a gaming device, a navigation device, a point of sale (POS) device, or the like, or any combination. In some embodiments, the virtual reality device and/or the augmented reality device may include a virtual reality helmet, a virtual reality glass, a virtual reality patch, an augmented reality helmet, an augmented reality glass, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device and/or the augmented reality device may include a Google Glass™, an Oculus Rift™, a Hololens™, a Gear VR™, etc. In some embodiments, some functions of the server110may be implemented by the user device140.

The storage150may store data and/or instructions. In some embodiments, the storage150may store data obtained from the camera device130, the user device140, and/or the processing engine112. For example, the storage150may store a plurality of images obtained from the camera device130. As another example, the storage150may store one or more tags associated with an image determined by the processing engine112. In some embodiments, the storage150may store data and/or instructions that the server110may execute or use to perform exemplary methods described in the present disclosure. For example, the storage150may store instructions that the processing engine112may execute or use to determine feature information of an image and one or more tags associated with the image. In some embodiments, the storage150may include a mass storage, a removable storage, a volatile read-and-write memory, a read-only memory (ROM), or the like, or any combination thereof. Exemplary mass storage may include a magnetic disk, an optical disk, a solid-state drives, etc. Exemplary removable storage may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. Exemplary volatile read-and-write memory may include a random access memory (RAM). Exemplary RAM may include a dynamic RAM (DRAM), a double date rate synchronous dynamic RAM (DDR SDRAM), a static RAM (SRAM), a thyristor RAM (T-RAM), and a zero-capacitor RAM (Z-RAM), etc. Exemplary ROM may include a mask ROM (MROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a compact disk ROM (CD-ROM), and a digital versatile disk ROM, etc. In some embodiments, the storage150may be implemented on a cloud platform. Merely by way of example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, or the like, or any combination thereof.

In some embodiments, the storage150may be connected to the network120to communicate with one or more components of the image auto-archiving system100(e.g., the server110, the camera device130, or the user device140). One or more components of the image auto-archiving system100may access the data and/or instructions stored in the storage150via the network120. In some embodiments, the storage150may be directly connected to or communicate with one or more components (e.g., the server110, the camera device130, the user device140) of the image auto-archiving system100. In some embodiments, the storage150may be part of the server110.

This description is intended to be illustrative, and not to limit the scope of the present disclosure. Many alternatives, modifications, and variations will be apparent to those skilled in the art. The features, structures, methods, and other characteristics of the exemplary embodiments described herein may be combined in various ways to obtain additional and/or alternative exemplary embodiments. However, those variations and modifications do not depart the scope of the present disclosure.

FIG. 2is a schematic diagram illustrating exemplary hardware and software components of a computing device200on which the server110, the user device140may be implemented according to some embodiments of the present disclosure. For example, the processing engine112may be implemented on the computing device200and configured to perform functions of the processing engine112disclosed in this disclosure.

The computing device200may be used to implement any component of the image auto-archiving system100as described herein. For example, the processing engine112may be implemented on the computing device200, via its hardware, software program, firmware, or a combination thereof. Although only one such computer is shown, for convenience, the computer functions relating to the auto-archiving service as described herein may be implemented in a distributed fashion on a number of similar platforms to distribute the processing load.

The computing device200, for example, may include COM ports250connected to and from a network connected thereto to facilitate data communications. The computing device200may also include a processor220, in the form of one or more processors (e.g., logic circuits), for executing program instructions. For example, the processor220may include interface circuits and processing circuits therein. The interface circuits may be configured to receive electronic signals from a bus210, wherein the electronic signals encode structured data and/or instructions for the processing circuits to process. The processing circuits may conduct logic calculations, and then determine a conclusion, a result, and/or an instruction encoded as electronic signals. Then the interface circuits may send out the electronic signals from the processing circuits via the bus210.

The computing device200may further include program storage and data storage of different forms including, for example, a disk270, and a read only memory (ROM)230, or a random access memory (RAM)240, for various data files to be processed and/or transmitted by the computing device. The exemplary computer platform may also include program instructions stored in the ROM230, RAM240, and/or other type of non-transitory storage medium to be executed by the processor220. The methods and/or processes of the present disclosure may be implemented as the program instructions. The computing device200also includes an I/O component260, supporting input/output between the computer and other components. The computing device200may also receive programming and data via network communications.

Merely for illustration, only one processor is described inFIG. 2. Multiple processors are also contemplated, thus operations and/or method steps performed by one processor as described in the present disclosure may also be jointly or separately performed by the multiple processors. For example, if in the present disclosure the processor of the computing device200executes both step A and step B, it should be understood that step A and step B may also be performed by two different CPUs and/or processors jointly or separately in the computing device200(e.g., the first processor executes step A and the second processor executes step B, or the first and second processors jointly execute steps A and B).

FIG. 3is a schematic diagram illustrating exemplary hardware and/or software components of a mobile device300on which the user device140may be implemented according to some embodiments of the present disclosure. As illustrated inFIG. 3, the mobile device300may include a communication unit310, a display320, a graphic processing unit (GPU)330, a central processing unit (CPU)340, an I/O350, a memory360, a mobile operating system (OS)370, and a storage390. The CPU340may include interface circuits and processing circuits similar to the processor220. In some embodiments, any other suitable component, including but not limited to a system bus or a controller (not shown), may also be included in the mobile device300.

In some embodiments, the mobile operating system (OS)370(e.g., iOS™ Android™, Windows Phone™, etc.) and one or more application(s)380may be loaded into the memory360from the storage390in order to be executed by the CPU340. The application(s)380may include a browser or any other suitable mobile apps for receiving and rendering information relating to a service request or other information from the location based service providing system on the mobile device300. User interactions with the information stream may be achieved via the I/O devices350and provided to the processing engine112and/or other components of the system100via the network120.

FIG. 4is a block diagram illustrating an exemplary processing engine according to some embodiments of the present disclosure. The processing engine112may include a data obtaining module402, a feature determination module404, a tag determination module406, an archive module408, a storage module410, and an identification module412.

The data obtaining module402may be configured to obtain a plurality of images (e.g.,420illustrated inFIG. 4). As used herein, the “image” may refer to a static image, a series of image frames, a video, etc. The image may be a two-dimensional image, a three-dimensional image, a four-dimensional image, etc. The image may further include voice information associated with the image. The data obtaining module402may obtain the plurality of images from a camera device (e.g., the camera device130) periodically (e.g., per second, per 2 seconds, per 5 seconds, per 10 seconds) or in real time. In some embodiments, during the capturing of the images, the camera device130may transmit the images to a storage device (e.g., the storage150) periodically (e.g., per second, per 2 seconds, per 5 seconds, per 10 seconds) or in real time via the network120. Further, the data obtaining module402may access the storage device and retrieve the images.

The feature determination module404may be configured to determine feature information of each of the plurality of images. The feature determination module404may determine the feature information of the plurality of images via a plurality of threads or a plurality of computing nodes. The feature information may include object information, expression information, event information, time information, location information, or the like, or any combination thereof. In some embodiments, before determining the feature information of the image, the feature determination module404may preprocess the image. For example, the feature determination module404may enlarge a specific region in the image to identify a region of interest (ROI). As another example, the feature determination module404may reduce noise(s) in the image based on a noise reduction technique (e.g., a wavelet noise reduction technique). As a further example, the feature determination module404may compress the image to increase the processing speed.

The tag determination module406may be configured to determine one or more tags associated with each of the plurality of images based on the feature information. As used herein, a “tag” may refer to a description (e.g., a keyword, a term, a sentence) associated with the feature information of the image. Take a family scenario as an example, assuming that the feature information of an image includes object information of a family member A and a family member B, expression information (e.g., “happiness”), event information (e.g., “wedding anniversary”), the tag determination module406may determine tags “A,” “B,” “happiness,” “wedding anniversary” associated with the image. As another example, assuming that the feature information of an image includes object information (e.g., clothing) of a family member C, the tag determination module406may determine tags “C,” “white long dress” associated with the image.

The archive module408may be configured to determine one or more image archives based on the plurality of images and the one or more tags associated with each of the plurality of images. As used herein, an “image archive” may refer to a collection of one or more images corresponding to a same tag.

In some embodiments, after determining the one or more image archives, the archive module408may further establish an image database in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure based on the one or more image archives. In some embodiments, the archive module408may identify the one or more image archives from an existing image database in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure based on the one or more tags associated with each of the plurality of images and archive the plurality of images to the one or more image archives.

The storage module410may be configured to store any information and/or data associated with the modules in the processing engine112. For example, the storage module410may store the feature information of the plurality of images, the tag of the images, and/or the image archives. As described elsewhere in the present disclosure, the storage module410may include any suitable storage device which may perform storage function.

The identification module412may be configured to obtain a search request from the user device140, identify one or more target image archives associated with one or more target tags based on the search request in an image database, retrieve the one or more target image archives from the image database, and transmit the one or more target image archives to the user device140via the network120. More descriptions may be found elsewhere in the present disclosure (e.g.,FIG. 7and the description thereof).

The modules in the processing engine112may be connected to or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, or the like, or any combination thereof. The wireless connection may include a Local Area Network (LAN), a Wide Area Network (WAN), a Bluetooth, a ZigBee, a Near Field Communication (NFC), or the like, or any combination thereof. Two or more of the modules may be combined into a single module, and any one of the modules may be divided into two or more units. For example, the storage module410may be unnecessary and the processing engine112may store the information and/or data in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure.

FIG. 5is a block diagram illustrating an exemplary feature determination module and an exemplary tag determination module according to some embodiments of the present disclosure. The feature determination module404may include an object detection unit502, an expression determination unit504, and an event determination unit506. The tag determination module406may include a correlation unit508.

The object detection unit502may be configured to detect object information of the plurality of images. The object information may indicate general information associated with object(s) in the images. As described elsewhere in the present disclosure, the object in the image may include a person, an animal, a facility, a furniture, a decoration, etc. Take the “person” as an example, the object information may include identification information (e.g., a name, an ID number), a height, a weight, an age, a gender, clothing (e.g., color, style), etc. Also take the “furniture (e.g., a dining-table)” as an example, the object information may include purchase information (e.g., price, purchase time), a length, a width, a height, a weight, a color, etc. In some embodiments, the object detection unit502may first detect the object(s) in the images and further extract the object information. The object detection unit502may detect the object(s) in the images based on an object detection algorithm (e.g., an inter-frame difference algorithm, a background difference algorithm, an optical flow algorithm). After detecting the object(s) in the images, the object detection unit502may extract the object information corresponding to the object(s).

The expression determination unit504may be configured to detect expression information of the plurality of images. The expression information may indicate emotional expression and/or action expression of the object(s) in the images. The emotional expression may include happiness, anger, disgust, fear, sadness, etc. The action expression may include sleep, watching TV, work, reading, cooking, cleaning, etc. Take the “emotional expression” as an example, the expression determination unit504may identify a facial feature (e.g., a wrinkle, a muscle change, a shape of an organ (e.g., eyes)) of the object (e.g., a person) and determine the emotional expression based on the facial feature. In some embodiments, the expression determination unit504may identify the facial feature of the object based on a feature extraction algorithm. Exemplary feature extraction algorithm may include a principal component analysis (PCA), a linear discriminant analysis (LDA), an independent component analysis (ICA), a multi-dimensional scaling (MDS) algorithm, a discrete cosine transform (DCT) algorithm, etc. As described elsewhere in the present disclosure, the expression determination unit504may further determine the emotional expression based on the facial feature according to a machine learning model (e.g., a neural network model, a regression model, a classification tree model).

The event determination unit506may be configured to detect event information of the plurality of images. The event information may indicate a date section (e.g., a birthday) and/or an activity (e.g., a birthday party) associated with the image. For example, the event information may include a New Year's Day, a Spring Festival, a Christmas Day, a Halloween, a wedding anniversary, a Christmas dinner, a home party, etc. In some embodiments, the event determination unit506may obtain the event information from a built-in calendar system in the auto-archiving system100. For example, a user may manually set a special date (e.g., a birthday, an anniversary) in the calendar system and the event determination unit506may access the calendar system to retrieve the event information. In some embodiments, the event determination unit506may obtain the event information from an external resource (e.g., a news platform, an online calendar) via the network120. In some embodiments, the event determination unit506may determine the event information (e.g., a birthday party) by identifying and/or analyzing a typical characteristic (e.g., a cake, a candle, a bubbly) in the image.

The correlation unit508may be configured to correlate two or more images of the plurality of images based on the tags associated with the images. For example, assuming that both a first image and a second image correspond to a same tag “wedding anniversary,” the correlation unit508may correlate the two images via a correlation word “wedding anniversary.”

The units in the feature determination module404and the tag determination module406may be connected to or communicated with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, or the like, or any combination thereof. The wireless connection may include a Local Area Network (LAN), a Wide Area Network (WAN), a Bluetooth, a ZigBee, a Near Field Communication (NFC), or the like, or any combination thereof. Two or more of the units may be combined into a single units, and any one of the units may be divided into two or more sub-units. For example, the feature determination module404may further include other units (e.g., a time determination unit) configured to determine other feature information of the plurality of images.

FIG. 6is a flowchart illustrating an exemplary process for determining one or more image archives according to some embodiments of the present disclosure. The process600may be executed by the image auto-archiving system100. For example, the process600may be implemented as a set of instructions stored in the storage ROM230or RAM240. The processor220and/or the modules inFIG. 4may execute the set of instructions, and when executing the instructions, the processor220and/or the modules may be configured to perform the process600. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process600may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process600as illustrated inFIG. 6and described below is not intended to be limiting.

In602, the processing engine112(e.g., the data obtaining module402) (e.g., the interface circuits of the processor220) may obtain a plurality of images. As used herein, the “image” may refer to a static image, a series of image frames, a video, etc. The image may be a two-dimensional image, a three-dimensional image, a four-dimensional image, etc. The image may further include voice information associated with the image.

In some embodiments, the processing engine112may obtain the plurality of images from a camera device (e.g., the camera device130) periodically (e.g., per second, per 2 seconds, per 5 seconds, per 10 seconds) or in real time. In some embodiments, during the capturing of the images, the camera device130may transmit the images to a storage device (e.g., the storage150) periodically (e.g., per second, per 2 seconds, per 5 seconds, per 10 seconds) or in real time via the network120. Further, the processing engine112may access the storage device and retrieve the images.

In604, the processing engine112(e.g., the feature determination module404) (e.g., the processing circuits of the processor220) may determine feature information of each of the plurality of images. The processing engine112may determine the feature information of the plurality of images via a plurality of threads or a plurality of computing nodes. The feature information may include object information, expression information, event information, time information, location information, or the like, or any combination thereof.

The object information may indicate general information associated with object(s) in the images. As described elsewhere in the present disclosure, the object in the image may include a person, an animal, a facility, a furniture, a decoration, etc. Take the “person” as an example, the object information may include identification information (e.g., a name, an ID number), a height, a weight, an age, a gender, clothing (e.g., color, style), etc. Also take the “furniture (e.g., a dining-table)” as an example, the object information may include purchase information (e.g., price, purchase time), a length, a width, a height, a weight, a color, etc.

The processing engine112may first detect the object(s) in the images and further extract the object information. The processing engine112may detect the object(s) in the images based on an object detection algorithm (e.g., an inter-frame difference algorithm, a background difference algorithm, an optical flow algorithm). After detecting the object(s) in the images, the processing engine112may extract the object information corresponding to the object(s). Take a specific object as an example, some of the object information (e.g., identification information, gender) may be directly obtained from registration information of the specific object. In this situation, the processing engine112may access a storage device (e.g., the storage150) associated with the image auto-archiving system100to obtain the registration information. Otherwise, some of the object information may be obtained indirectly by analyzing the detected object. For example, the processing engine112may determine the object information such as the weight, the height, the color of clothing, based on a machine learning model (e.g., a neural network model, a regression model, a classification tree model).

The expression information may indicate emotional expression and/or action expression of the object(s) in the images. The emotional expression may include happiness, anger, disgust, fear, sadness, etc. The action expression may include sleep, watching TV, work, reading, cooking, cleaning, etc. Take the “emotional expression” as an example, the processing engine112may identify a facial feature (e.g., a wrinkle, a muscle change, a shape of an organ (e.g., eyes)) of the object (e.g., a person) and determine the emotional expression based on the facial feature. In some embodiments, the processing engine112may identify the facial feature of the object based on a feature extraction algorithm. Exemplary feature extraction algorithm may include a principal component analysis (PCA), a linear discriminant analysis (LDA), an independent component analysis (ICA), a multi-dimensional scaling (MDS) algorithm, a discrete cosine transform (DCT) algorithm, etc. As described elsewhere in the present disclosure, the processing engine112may further determine the emotional expression based on the facial feature according to a machine learning model (e.g., a neural network model, a regression model, a classification tree model).

The event information may indicate a date section (e.g., a birthday) and/or an activity (e.g., a birthday party) associated with the image. For example, the event information may include a New Year's Day, a Spring Festival, a Christmas Day, a Halloween, a wedding anniversary, a Christmas dinner, a home party, etc. In some embodiments, the processing engine112may obtain the event information from a built-in calendar system in the auto-archiving system100. For example, a user may manually set a special date (e.g., a birthday, an anniversary) in the calendar system and the processing engine112may access the calendar system to retrieve the event information. In some embodiments, the processing engine112may obtain the event information from an external resource (e.g., a news platform, an online calendar) via the network120. In some embodiments, the processing engine112may determine the event information (e.g., a birthday party) by identifying and/or analyzing a typical characteristic (e.g., a cake, a candle, a bubbly) in the image.

The time information may indicate a time (e.g., a data, a time period, a time point) when the image was captured. The location information may indicate a location where the image was captured. Take a family scenario as an example, the location information may include a living room, a bedroom, a kitchen, a dining room, a study, etc. As described elsewhere in the present disclosure, the camera device130may include a positioning module which can determine the location of the camera device130.

In some embodiments, before determining the feature information of the image, the processing engine112may preprocess the image. For example, the processing engine112may enlarge a specific region in the image to identify a region of interest (ROI). As another example, the processing engine112may reduce noise(s) in the image based on a noise reduction technique (e.g., a wavelet noise reduction technique). As a further example, the processing engine112may compress the image to increase the processing speed.

In606, the processing engine112(e.g., the tag determination module406) (e.g., the processing circuits of the processor220) may determine one or more tags associated with each of the plurality of images based on the feature information. As used herein, a “tag” may refer to a description (e.g., a keyword, a term, a sentence) associated with the feature information of the image. Take a family scenario as an example, assuming that the feature information of an image includes object information of a family member A and a family member B, expression information (e.g., “happiness”), event information (e.g., “wedding anniversary”), the processing engine112may determine tags “A,” “B,” “happiness,” “wedding anniversary” associated with the image. As another example, assuming that the feature information of an image includes object information (e.g., clothing) of a family member C, the processing engine112may determine tags “C,” “white long dress” associated with the image.

In some embodiments, the processing engine112may correlate two or more images of the plurality of images based on the tags associated with the images. For example, assuming that both a first image and a second image correspond to a same tag “wedding anniversary,” the processing engine112may correlate the two images via a correlation word “wedding anniversary.”

In608, the processing engine112(e.g., the archive module408) (e.g., the processing circuits of the processor220) may determine one or more image archives based on the plurality of images and the one or more tags associated with each of the plurality of images. As used herein, an “image archive” may refer to a collection of one or more images corresponding to a same tag.

In some embodiments, after determining the one or more image archives, the processing engine112may further establish an image database in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure based on the one or more image archives. In some embodiments, the processing engine112may identify the one or more image archives from an existing image database in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure based on the one or more tags associated with each of the plurality of images and archive the plurality of images to the one or more image archives.

For illustration purposes, the present disclosure takes “the plurality of images” as an example, it should be noted that the processing engine112may obtain images one by one in real time from the camera device130. The processing engine112may process the images in real time via a plurality of threads or a plurality of computing nodes. For example, assuming that the processing engine112obtains an image A1from the camera device130at a time point T1, the processing engine112may process the image A1via a thread X1; the processing engine112further obtains an image A2from the camera device130at a time point T2(the image A1is still under processing at the moment), the processing may process the image A2via a thread X2; and so on. When the processing of an image is completed, the corresponding thread may be idle and the thread may be assigned a next task. In this way, the processing engine112may obtain and process images simultaneously.

It should be noted that the above description is merely provided for the purpose of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. For example, one or more steps (e.g., a storing step) may be added elsewhere in the process600. In the storing step, the processing engine112may store information and/or data associated with the images (e.g., the feature information of the images, the one or more tags associated with the images) in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure. As another example, step604and step606may be combined as a single step in which the processing engine112may both determine the feature information of each of the plurality of images and determine the one or more tags associated with each of the plurality of images based on the feature information.

FIG. 7is a flowchart illustrating an exemplary process for determining one or more target image archives based on a search request according to some embodiments of the present disclosure. The process700may be executed by the image auto-archiving system100. For example, the process700may be implemented as a set of instructions stored in the storage ROM230or RAM240. The processor220and/or the modules inFIG. 4may execute the set of instructions, and when executing the instructions, the processor220and/or the modules may be configured to perform the process700. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process700may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order in which the operations of the process700as illustrated inFIG. 7and described below is not intended to be limiting.

In702, the processing engine112(e.g., the identification module412) (e.g., the interface circuits of the processor220) may obtain a search request from the user device140. The processing engine112may obtain the search request from the user device140via the network120. The search request may be associated with a text input, an image input, a video input, a voice input, etc. The search request may include one or more keywords. For example, the processing engine112may obtain a voice input from a user and identify the one or more keywords from the voice input based on a voice recognition technique (e.g., a voice model, a language model).

In some embodiments, the user device140may establish a communication (e.g., a wireless communication) with the processing engine112through an application (e.g., the application380) associated with the image auto-archiving system100installed in the user device140. Further, a user can initiate the search request via a user interface (e.g., a user interface800illustrated inFIG. 8) of the application. For example, the user may input a search query including one or more keywords via a search box820illustrated inFIG. 8. As another example, the user may initiate the search request by selecting one or more options via one or more option sections (e.g.,830and840illustrated inFIG. 8). More descriptions of the user interface may be found elsewhere in the present disclosure (e.g.,FIG. 8and the description thereof).

In704, the processing engine112(e.g., the identification module412) (e.g., the processing circuits of the processor220) may identify one or more target image archives associated with one or more target tags based on the search request in an image database (e.g., the image database described inFIG. 6).

In some embodiments, the processing engine112may identify the one or more target tags based on the one or more keywords included in the search request and identify the one or more target image archives based on the one or more target tags. For example, assuming that the search request includes one keyword inputted by the user, the processing engine112may identify a target tag that is the same as or substantially similar to the keyword. As used herein, “substantially similar” refers to that a similarity between the keyword and the target tag is larger than a threshold (e.g., 98%, 95%, 90%, 85%) or the keyword is a synonym of the target tag. Further, the processing engine112may identify the target image archive corresponding to the target tag in the image database. As another example, the user may select one or more target tags via one or more option sections of the user interface (e.g.,830and840illustrated inFIG. 8). Further, the processing engine112may identify the one or more target image archives corresponding to the one or more target tags in the image database.

In706, the processing engine112(e.g., the identification module412) (e.g., the processing circuits of the processor220) may retrieve the one or more target image archives from the image database.

In708, the processing engine112(e.g., the identification module412) (e.g., the interface circuits of the processor220) may transmit the one or more target image archives to the user device140via the network120. The processing engine112may transmit the one or more target image archives via any suitable communication protocol (e.g., Address Resolution Protocol (ARP), Dynamic Host Configuration Protocol (DHCP), File Transfer Protocol (FTP), Hypertext Transfer Protocol (HTTP)). After receiving the one or more target image archives, the user device140may display the target image archives via a user interface (e.g., a user interface900illustrated inFIG. 9) of the application. The user device140may display the target image archives in the form of graph, audio, video, or the like, or a combination thereof.

It should be noted that the above description is merely provided for the purpose of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure. For example, one or more other optional steps (e.g., a storing step) may be added elsewhere in the process700. In the storing step, the processing engine112may store information and/or data associated with the target image archives in a storage device (e.g., the storage150) disclosed elsewhere in the present disclosure.

FIG. 8is a schematic diagram illustrating an exemplary user interface for an image auto-archiving system according to some embodiments of the present disclosure. The user interface800may include one or more user interface elements (also referred to as “UI elements”) for presenting information associated with the image auto-archiving system. The UI elements may include one or more buttons, icons, checkboxes, message boxes, text fields, data fields, search fields, etc.

As illustrated, the user interface800may include a menu bar810for presenting operations (e.g., “edit,” “view,” “search,” “print,” “help,” “feedback”) associated with the system. The user interface800may also include a search box820for presenting a search query inputted by a user. The user can input one or more keywords in the search box820to initiate a search request. The user interface800may further include a type option section830for presenting various types (e.g., “graph,” “video,” “audio”) of the target image archives to be searched for. The user interface800may further include a tag option section840for presenting various types (e.g., “time,” “object,” “location,” “event,” “expression”) of tags that can be selected by the user to initiate a search request. For each type (e.g., “expression”), the tag option section840may include a pull-down list (“happy,” “angry,” “surprised,” “sad”), a text box, etc. The user can initiate a search request by selecting a plurality of tags and defining a relationship (e.g., “and,” “or”) among the plurality of selected tags via the tag option section840. The user interface800may further include a button850for presenting a confirmation by the user.

FIG. 9is a schematic diagram illustrating an exemplary user interface for displaying target image archives in response to a search request according to some embodiments of the present disclosure. The user interface900may include one or more UI elements for presenting selection options for a user to view and edit the target image archive(s).

As illustrated, the user interface900may include a display screen910for displaying the target image archive(s). The user interface900may also include a tag section920for presenting various tags associated with the target image archive(s). The user interface900may further include a function button section930for presenting operations (e.g., “fast forward,” “play,” “fast backward,” “stop,” “share,” “back”) associated with the target image archive(s).