Display apparatus and control method thereof

An apparatus includes: a communication interface configured to communicate with an external device which is configured to provide video data; a user interface configured to control based on a user input; a processor configured to process the video data, which is received in the communication interface, to be displayed as an image on the display; a controller configured to perform control to extract a text for a plurality of respective processing regions which are set up to have a plurality of same coordinates as a designated region with regard to each video frame based on the processed video data after a point of time in response to a partial region being designated by the user interface within a displayed video frame on the display, and apply a preset process to the extracted text in sync with each video frame.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Korean Patent Application No. 10-2013-0116276, filed on Sep. 30, 2013 in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference in its entirety.

BACKGROUND

Field

Apparatuses and methods consistent with the exemplary embodiments relate to a display apparatus for processing the video data received from an external device and displaying an image and a control method thereof. In particular, exemplary embodiments relate to a display apparatus and a control method thereof, in which a text such as a subtitle or a similar text contained in an image is processed while displaying the video data reproduced and input from an external device as the corresponding image.

Description of the Related Art

A related art image processing apparatus for processing video data to be displayed as an image is classified into a first type that includes a display panel and displays the processed video signal by itself, and a second type that includes no display panel and outputs the processed video signal to another device. In particular, the first type is called a display apparatus and may include a television as an example.

The video data to be processed for the display apparatus may be stored in the display apparatus or transmitted to the display apparatus from an exterior device. As an example of transmitting the video data from the exterior device to the display apparatus, there may be a case where the display apparatus is locally connected to an external device such as a digital versatile disc (DVD)/Blu-ray player and the video data reproduced in the external device is transmitted to the display apparatus.

A text subtitle may be displayed within a scene of an image displayed on the display apparatus. The text subtitle is not transmitted to the display apparatus as additional information separate from the image, but forms a data portion of the image itself. In this case, the subtitle is not text data but video data. Thus, the subtitle is difficult to process when a user wants to apply a text-related process to the subtitle separately from the image.

With recent trends in information technology (IT) and electronics, a lot of additional services which use moving images have been offered due to an increase in communication bandwidth and communication speed. Therefore, demands for various user services which are convenient to a user have increased with regard to the moving images. In this regard, since an image includes a subtitle (i.e., the text data related to the subtitle is not separated from the subtitle) transmitted to the display apparatus, if the subtitle is extracted and processed separately from the image, it may improve user convenience through an additional user service.

SUMMARY

According to an aspect of an exemplary embodiment there is provided a display apparatus which includes: a display; a communication interface configured to communicate with an external device which is configured to provide video data; a user interface configured to control based on a user input; a processor configured to process the video data, which is received in the communication interface, to be displayed as an image on the display; and a controller configured to perform control to extract a text for a plurality of processing regions which are set up to have a plurality of same coordinates as a designated region with regard to each video frame based on the processed video data after a point of time in response to a partial region being designated by the user interface within a displayed video frame on the display, and apply a preset process to the extracted text in sync with each video frame.

The controller may be further configured to control the processor to apply optical character recognition (OCR) to the processing regions and extract a plurality of texts from the respective processing regions.

The user interface may include a motion sensor configured to sense a user motion, and the controller may be further configured to designate the partial region in accordance with the sensed user motion sensed by the motion sensor.

The controller may be further configured to designate the partial region in accordance with a cursor movement through the user interface.

The preset process may include a translation service for the text, and the controller may be further configured to transmit the text to a server and may receive and display a translation of the text from the server.

The controller may be further configured to display a processing region to be overlaid with the translation in response to the translation being displayed in the video frame.

In response to determining that a difference between a first area value of the partial region first designated by the user interface and a second area value occupied with the text within a processing region is greater than a preset threshold, the controller may be further configured to adjust an area of the processing region so that a difference between the first area value and the second area value can be smaller than the preset threshold.

The controller may be further configured to delay a point of time for displaying the video frame so as to synchronize the text processed by the preset process with the video frame, based on a time taken in extracting the text and performing the preset process with regard to a processing region of one video frame.

In response to a first text, a second text, and a third text extracted from a first video frame, a second video frame, and a third video frame, respectively, which are sequenced in time, the controller may be further configured to determine that an error occurs while the second text is extracted from the second video frame, in response to the first text and the third text having the same contents and the second text being different in contents from the first text and the third text.

In response to determining that the second text has the error, the controller may be further configured to process the second text to have the same contents as those of the first text and the third text.

According to another aspect of an exemplary embodiment, there is provided a method of controlling a display apparatus, the method includes: receiving video data from an external device; displaying the video data as an image; receiving input for designating a partial region by a user within a video frame of the displayed image at a point of time; performing control to extract a text for a plurality of respective processing regions which are set up to have a plurality of same coordinates as those of the designated region with regard to each video frame based on a processed video data after a point of time; and applying a preset process to the extracted text in sync with each video frame.

The performing control to extract the text for the respective processing regions may include applying optical character recognition (OCR) to the processing regions and extracting a plurality of texts from the respective processing regions

The receiving the input for designating the partial region by a user may include designating the partial region in accordance with a sensed user motion sensed by a motion sensor of the display apparatus.

The receiving the input for designating the partial region by a user may include designating the partial region in accordance with a cursor movement through a user interface of the display apparatus.

The preset process may include a translation service for the text, and the applying the preset process to the extracted text in sync with each video frame may include: transmitting the text to a server; and receiving and displaying a translation of the text from the server.

The receiving and displaying the translation of the text from the server may include displaying a processing region to be overlaid with the translation.

The method may further include in response to determining that a difference between a first area value of the partial region first designated by the user and a second area value occupied with the text within a processing region is greater than a preset threshold, adjusting an area of the processing region so that a difference between the first area value and the second area value can be smaller than the preset threshold.

The applying the preset process to the extracted text in sync with each video frame may include delaying a point of time for displaying the video frame so as to synchronize the text processed by the preset process with the video frame, based on a time taken in extracting the text and performing the preset process with regard to a processing region of one video frame.

The method may further include: in response to a first text, a second text, and a third text extracted from a first video frame, a second video frame, and a third video frame, respectively, which are sequenced in time, determining that an error occurs while the second text is extracted from the second video frame, in response to the first text and the third text having the same contents and the second text being different in contents from the first text and the third text.

The determining that an error occurs while the second text is extracted from the second video frame may include processing the second text to have the same contents as those of the first text and the third text.

According to another aspect of an exemplary embodiment, a method of controlling a display apparatus may be provided including: pausing a displayed image in response to an event occurring which designates a processing region; storing position information of the processing region in response to a plurality of coordinates being input and the delayed image being paused; resuming the displayed image in response to the stored position information; extracting a text from the processing region; applying a preset process to the extracted text; and displaying the processed texts.

The displaying the processed text may occur in sync with a video frame which includes the displayed image.

The preset process may include a translation service for the text.

The displaying the processed text may further include: displaying the processing region overlaid with the translation.

The apparatus may further include the display.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Below, exemplary embodiments will be described in detail with reference to accompanying drawings so as to be easily realized by a person having ordinary knowledge in the art. The exemplary embodiments may be embodied in various forms without being limited to the exemplary embodiments set forth herein. Descriptions of well-known parts are omitted for clarity, but this does not mean that the omitted parts are unnecessary for realization of apparatuses or systems to which the exemplary embodiments are applied. Like reference numerals refer to like elements throughout.

FIG. 1is a block diagram of a display apparatus100involved in a system according to a first exemplary embodiment. The display apparatus100may be achieved by any electronic device that can process and display an image. In this exemplary embodiment, a television (TV) is employed as the display apparatus100, but the display apparatus is not limited thereto. Alternatively, a portable multimedia reproducer, a mobile phone, etc., may be used.

As shown inFIG. 1, the display apparatus100in this exemplary embodiment is connected to an external server or an external device20through a network or a local connection. In the case where the external device20is an image supply source, the display apparatus100may receive video data from the external device20. In this exemplary embodiment, the external device20is an image reproducer which is locally connected to the display apparatus100, and which transmits the reproduced video data reproduced by itself to the display apparatus100.

The display apparatus100includes a communication interface110which performs communication with an exterior device to transmit or receive data or a signal, a processor120which processes data received in the communication interface110in accordance with preset processes, a display130which displays video data as an image if data processed in the processor120is the video data, a user interface140which is for a user input, a storage150which stores data of information, a buffer160which temporarily stores the data or the information to be processed, and a controller170which controls general operations of the display apparatus100.

The communication interface110transmits or receives data so that interactive communication can be performed between the display apparatus100and the server10or the external device20. The communication interface110accesses the server10or the external device20through wide area networks, local area networks, or local connections in accordance with preset communication protocols.

The communication interface110may be achieved by connection ports according to devices or an assembly of connection modules, in which the protocol for connection or the external device20for connection is not limited to one kind or type. The communication interface110may be a built-in device of the display apparatus100, or the entire or a part thereof may be added to the display apparatus100in the form of an add-on or dongle type.

The communication interface110transmits or receives a signal in accordance with protocols designated according to the connected devices, in which the signals can be transmitted or received based on individual connection protocols with regard to the connected devices. In the case of video data, the communication interface110may transmit or receive the signal bases on various standards such as a radio frequency (RF) signal, composite/component video, super video, Syndicat des Constructeurs des Appareils Radiorécepteurs et Téléviseurs (SCART), high definition multimedia interface (HDMI), display port, unified display interface (UDI), wireless HD, etc.

The processor120performs various processes with regard to data or a signal received in the communication interface110. If the communication interface110receives the video data, the processor120applies an imaging process to the video data and the video data processed by this process is output the display130. Therefore, the display130may display an image based on the corresponding video data. If the signal received in the communication interface110is a broadcasting signal, the processor120extracts video, audio, and appended data from the broadcasting signal tuned to a certain channel, and adjusts an image to have a preset resolution, so that the image can be displayed on the display130.

There is no limit to the kind of imaging processes to be performed by the processor120. For example, image processes to be performed may include decoding corresponding to an image format of the video data, de-interlacing for converting the video data from an interlace type into a progressive type, scaling for adjusting the video data to have a preset resolution, noise reduction for improving image qualities, detail enhancement, frame refresh rate conversion, etc.

The processor120may perform various processes in accordance with the kinds and attributes of data. Thus, the process to be implemented in the processor120is not limited to the imaging process. Also, the data processed in the processor120is not limited to only data received in the communication interface110. For example, the processor120also processes a user utterance through a preset voicing process when the user interface140receives the corresponding utterance, and processes a sensing result through a preset gesture process when the user interface140senses a user gesture.

The processor120may be achieved by an image processing board (not shown) on a system-on-chip where various functions are integrated or an individual chip-set capable of independently performing each process is mounted on a printed circuit board. The processor120may be built-in the display apparatus100.

The display130displays the video signal or the video data processed by the processor120as an image. The display130may be achieved by various display types such as liquid crystal, plasma, a light-emitting diode, an organic light-diode, a surface-conduction electron-emitter, a carbon nano-tube and a nano-crystal. However, the display is not limited thereto.

The display130may additionally include an appended element in accordance with the display types. For example, in the case of the liquid crystal type, the display130may include a liquid crystal display (LCD) panel (not shown), a backlight unit (not shown) which emits light to the LCD panel, a panel driving substrate (not shown) which drives the panel (not shown), etc.

The user interface140transmits various preset control commands or information to the controller170in accordance with a user control or input. The user interface140generates information corresponding to various events that occur in accordance with a user intention and transmits it to the controller170. The events that occur by a user may have various forms, and may, for example, include a user control, utterance, gesture, etc.

In accordance with methods by which a user inputs information, the user interface140is provided to detect the information input by the corresponding method. For example, the user interface140may be achieved by a remote controller remotely separated from the display apparatus100, a menu key or an input panel provided outside the display apparatus100, a touch screen provided in the display130, a microphone for receiving a user utterance, a camera or motion sensor for sensing a user motion, etc.

The storage150stores various data under control of the controller170. The storage150is achieved by a nonvolatile memory such as a flash memory, a hard disk drive, etc., so as to retain data regardless of power on/off of the system. The storage150is accessed by the controller170so that previously stored data can be read, recorded, modified, deleted, updated, and so on.

The buffer160temporarily stores data scheduled to be processed or being processed by the processor120and the controller170. For example, there is a need for temporarily storing the video data received in the communication interface110from the external device20because the video data has to be temporarily stored before or while being processed by the processor120. To achieve this temporary storage, the buffer160is provided. The buffer160only has to retain data while the display apparatus100operates. Therefore, the buffer160is achieved by a volatile memory, since a nonvolatile memory is not required (wherein the storage150requires a nonvolatile memory).

The controller170is achieved by a central processing unit (CPU), and controls operations of general elements of the display apparatus100, such as the processor120, in response to occurrence of a predetermined event. For example, if the communication interface110receives the video data from the external device20, the controller170controls the processor120to process the video data to be displayed as an image on the display130. Also, the controller170controls the elements such as the processor120to implement a preset operation corresponding to a user input event when the corresponding event occurs through the user interface140.

There are two cases where the text subtitle is shown within an image displayed on the display apparatus100.

One case is where the display apparatus100receives video data and the text data from the external device20. In this case, the display apparatus100not only displays an image based on the video data, but also displays the text data in sync with the image in such a manner that the image is overlaid with the text data. Thus, the display apparatus displays both the image and the subtitle.

The other case is where the external device20does not offer the text data to the display apparatus100and the video data involves the contents of the subtitle. In this case, the display apparatus100processes only the video data and displays it as an image, since there is no separately received text data. In this case, the image is not overlaid with the text data on the display apparatus100, and the text data is displayed as a part of the image.

Regarding the two cases where the subtitle is displayed, suppose that the subtitle displayed on the image is foreign to a user language. For example, an English subtitle may be displayed while a Korean user (who is not good at English) views. In this case, a user may want to view a Korean subtitle translated from the English subtitle, instead of the original English subtitle.

If the text data for the subtitle is separately offered, the display apparatus100processes the text data to be translated and displays the subtitle based on the translated text data.

In the other case of the two cases where the subtitle is displayed, if the text data is not separately offered and the subtitle is involved in the video data, the display apparatus100cannot process the contents of the subtitle to be translated. Processing of the contents of the subtitle cannot occur because the subtitle is part of the video data (and not a separate text data).

Therefore, to process the subtitle to be translated in the other case, the display apparatus100applies an optical character reading/recognition (OCR) process to a region occupied with a subtitle within a video frame.

FIG. 2shows an example of a video frame210displayed on the display apparatus ofFIG. 1.

As shown inFIG. 2, the display apparatus100displays an image based on video data received from the external device20. In a case where the video data involves the content of the subtitle, a video frame210of an image according to this embodiment includes an image region211where the image is displayed, and a subtitle region212where a subtitle is displayed.

While the display apparatus100displays an image as time goes by, the position of the subtitle region212is set up in a specific position within the video frame210. The subtitle region212is in a specific position because the subtitle region has to be placed in front of the image region211without interfering with the image region211(if possible). In a related art, the subtitle region212is placed in lower or upper edges of the image region211.

If a sentence displayed in the subtitle region212is foreign to a user language, a user may want to view a translation of the sentence displayed on the subtitle region212. Therefore, the display apparatus100applies the OCR process to the video frame210and extracts the text from the subtitle region212. Thus, the extracted text is processed for translation.

The subtitle region212and the image region211are divided in this exemplary embodiment, i.e., both regions211and212constitute an image forming the video frame210. However, in a related art, the display apparatus100cannot distinguish the image region211and the subtitle region212within the video frame210. Thus, in the related art, the OCR process is applied to the entire video frame210.

However, the OCR process involves a scanning process with regard to respective pixels of a region to be processed. Thus, the time taken in the OCR process and system load of the display apparatus100are increased as the area of the region to be processed becomes larger. Also, if the time taken in the OCR process is increased, the amount of video data to be stored in the buffer160is increased. Therefore a storage amount of the buffer160is increased.

Also, due to the characteristics of the OCR process, if the area of the region to be processed is increased, the accuracy of the text is deteriorated when the text is extracted from the region to be processed.

In other words, in order to decrease the time taken in the OCR process, reduce the system load, and improve the accuracy of the extracted text, the subtitle region212has to be specified on the video frame210. Then, the OCR process has to be applied to only the specified subtitle region212.

However, the position of the subtitle region212within the video frame210may be set to one position with respect to one video content, and may be varied depending on a plurality of video contents. For example, the position of the subtitle region212in a predetermined first video content is placed at the lower edge of the video frame210in the entire first video contents, but the position of the subtitle region212in a second video content may be placed at an upper edge of the video frame210. Therefore, a problem may arise if the position of the subtitle region212for processing is fixed by the display apparatus100.

Accordingly, this exemplary embodiment proposes the following method to overcome the varied position of the subtitle region212.

FIG. 3shows an example of a method of designating an optical character recognition (OCR) processing region222in the video frame220.

As shown inFIG. 3, if predetermined processing region222is designated by a user within the video frame220displayed on the display130at a first point of time, the display apparatus100designates respective regions, which have the same coordinates as this designated region with regard to respective video frames based on the video data processed after the first point of time, as the processing region, and applies the OCR process to each processing region.

The display apparatus100performs the preset process with regard to the text extracted by the OCR process from the processing region222of each video frame220, and displays the processed text in sync with each video frame220.

Further, while an image is displayed in the display apparatus100, a user may set up the processing region222within the video frame220in a state that the corresponding image is paused.

The processing region222within the video frame220may be set up by any user input through the user interface140(refer to ‘user interface140’ inFIG. 1). For example, if a user designates predetermined points P01 and P02 within respective spaces by both hands, the display apparatus100derives a spatial rectangle having points P01 and P02 as vertexes, and sets up the processing region222within the video frame220in accordance with the derived rectangle. In this case, the user interface140includes a motion sensor or a camera to detect a hand motion of a user within the space.

Also, if the user interface140includes a remote controller, a user uses the remote controller to move a cursor (not shown) on the video frame220, thereby setting up the processing region222.

Thus, if the processing region222is set up within the video frame220at the first point of time, the display apparatus100memorizes the coordinates of the processing region222. The display apparatus100sets up the processing region222at the same position based on the memorized coordinates, with regard to each video frame220of the image processed after the first point of time. In other words, the display apparatus100sets up each processing region222at the position of the same coordinates within each video frame220after the first point of time.

The display apparatus100applies the OCR process to each processing region222, and extracts the text from the corresponding processing region222. The extracted text may undergo various processes. For example, the display apparatus100may translate the extracted text into another language and display it on the video frame220.

FIG. 4shows an example of a translation of an extracted text displayed on the video frame230.

As shown inFIG. 4, if a text extracting process is applied to a certain region of the video frame230, and the text is extracted from the corresponding region, the display apparatus100carries out translation of the extracted text.

In accordance with the kinds of process with regard to the text, the corresponding process may be performed by the display apparatus100or by the server10. If the process is simple and lays a small button on the display apparatus100, the display apparatus100may perform the corresponding process. However, if the process is a high-level analysis operation such as the translation, it is more efficient that the process is performed by an external server10than that the process is directly performed by the display apparatus100.

The display apparatus100sends the extracted text to the server10, and requests the extracted text to be translated. The server10returns a translation of the corresponding text to the display apparatus100.

The display apparatus100displays the video frame230to be overlaid with the translation232received from the server10. The position of the translation232within the video frame230may be different from the processing region for extracting the text, or may be overlaid on the corresponding processing region.

In one video content, the position of the subtitle is generally defined within the video frame. Thus if a user designates a certain region (as described above), the subtitle content of all the video frames within the video contents may be processed.

Meanwhile, the accuracy of the OCR process increases as the processing region to be processed becomes smaller. As described in the foregoing embodiments, if the processing region designated by a user is called the first processing region, the first processing region designated by the user may be excessively large as compared with the area actually occupied with each video frame.

If the first processing region designated by the user is excessively large as compared to the area actually occupied with the text, it is possible to improve the accuracy of the OCR process by reducing the area of the first processing region.

FIG. 5shows an example where a first processing region241is adjusted by a user according to a second exemplary embodiment.

As shown inFIG. 5, if the first processing region241is designated at the first point of time when the video content is reproduced and displayed, the display apparatus100records the coordinates and area of the first processing region241within the video frame240and sets up the processing region having the same coordinates and area as the first processing region241in each video frame240after the first point of time.

The display apparatus100samples processing regions according to respective points of time after the first point of time, extracts the text from the sampled processing regions, and calculates the area occupied with the text within each processing region.

For example, if there are five samples, let the area of the first processing region241be M1; an area occupied with the text within the processing region at a second point of time be M2; an area occupied with the text within the processing region at a third point of time be M3; an area occupied with the text within the processing region at a fourth point of time be M4; an area occupied with the test within the processing region at the fifth point of time be M5; and an area occupied with the text within the processing region at a sixth point of time be M6.

The display apparatus100determines whether the greatest one among M2, M3, M4, M5 and M6 is smaller than M1 and a difference therebetween is smaller than a preset threshold V1. For example, if the greatest one among M2, M3, M4, M5 and M6 is M3, ‘M1−M3<V1’ shows that the area of the first processing region241is appropriate. In this case, there is no need of adjusting the first processing region241.

Meanwhile, if the greatest one among M2, M3, M4, M5 and M6 is smaller than M1 and a difference therebetween is greater than the preset threshold V1, there is a need of adjusting the first processing region241. For example, if the greatest one among M2, M3, M4, M5 and M6 is M3, ‘M1-M3V1’ shows that the first processing region241designated by the user is excessively large as compared with the area actually needed for processing the text, the display apparatus100readjusts the first processing region241into the second processing region242satisfying ‘M1−M3<V1’.

The display apparatus100sets up a processing region corresponding to the coordinates and area of the second processing region242instead of the first processing region241, with respect to all the subsequent video frames240, and applies the OCR process to each setup processing region.

There may be various methods of determining the area occupied with the text within the processing region. As one example, the area occupied with the text may be determined based on the number of text lines within each sampled processing region. Under the conditions that the first processing region241is designated to correspond to five lines and V1 is set up as 1.5, a case where M3 greatest among M2, M3, M4, M5 and M6 is 4 and a case where M3 is 3 will be described below.

If M3 is 4, ‘M1−M3’ is 1 and thus smaller than 1.5, ‘M1−M3<V1’ is satisfied. This scenario shows that the first processing region241is set up to have an appropriate area, and therefore the display apparatus100does not have to adjust the first processing region241.

On the other hand, if M3 is 3, ‘M1-M3’ is 2 and thus greater than 1.5, ‘M1−M3V1’ is satisfied. This scenario shows that the first processing region241is set up to have an excessively large area, and therefore the display apparatus100reduces the first processing region241until it satisfies ‘M1−M3<V1’. For example, the display apparatus100may change the designation of the first processing region241again from five lines to four lines.

If the greatest one among M2, M3, M4, M5 and M6 is greater than M1, the first processing region241is set up to be smaller than an actually required region. In this case, contents of a subtitle are not normally displayed within the video frame240. Therefore, a user recognizes that the contents of the subtitle are not normally displayed within the video frame240, and the user sets up the first processing region241again through a method similar to the foregoing first exemplary embodiment.

As shown in the first exemplary embodiment, translations of texts extracted from the video frames according to respective points of time has to be synchronized with the respective video frames and then displayed on the video frame from which an original text is extracted. This will be described below.

FIG. 6shows a concept of displaying a translation252of a text extracted from a predetermined video frame250in sync with the corresponding video frame250according to a third exemplary embodiment.

As shown inFIG. 6, the display apparatus100extracts a processing region251from a video frame250at a point of time t0. The method of setting up the processing region251from the video frame250and extracting a text from processing region251may be based on the foregoing exemplary embodiments. Thus, detailed descriptions thereof will be omitted.

The video frame250temporarily stored in the buffer (refer to buffer160inFIG. 1) at the point of time t0 is processed by the processor (refer to processor120inFIG. 1) and displayed on the display (refer to display130inFIG. 1) at a point of time t1 as long as specific control is not performed by the controller170.

The display apparatus100extracts the text from the processing region251by applying the OCR process to the processing region251extracted from the video frame250at the point of time t0, and requests the server (refer to server10inFIG. 1) to translate the text. Since time is taken in extracting and translating the text, a final translation252for the processing region251at the point of time t0 is displayable at the point of time t2.

However, the time taken in the process makes the point of time t2 not be the point of time t1, but the subsequent point of time. If the display apparatus100displays the translation252at the point of time t2, a problem arises in that the video frame250is not corresponding to the original text, but another video frame after the point of time for the original video frame250is overlaid with the translation252.

Thus, the display apparatus100solves the foregoing problem by the following method. The display apparatus100tabulates a time table with a value of the point of time t0 for extracting the processing region251, and a text extracted from the processing region251. Further, the display apparatus100calculates a time to be taken in extracting the text from the processing region251and completely translating the text. Then, the display apparatus100compensates t0 of the time table with this calculated time so that the point of time for displaying the translation252can be coincident with the point of time for displaying the video frame250. The display apparatus100displays the video frame250and the translation252based on the compensated time table.

In other words, the display apparatus100delays the display of the video frame250as much as the time taken in extracting the text from the processing region251of the video frame250and translating the corresponding text, thereby displaying the video frame250.

Thus, the display apparatus100can display the translation252of the text extracted from the video frame250in sync with the corresponding video frame250.

In terms of successively applying the OCR process to respective processing regions, different contents may be extracted from the text due to an error in the OCR process even though the contents are actually the same, which will be described below.

FIG. 7shows an extracted state of a text261,262,263,264sampled by a predetermined number of times according to a fourth exemplary embodiment. In this exemplary embodiment, a word will be described for simplicity, but not limited thereto. Alternatively, a unit of sentence may be applied to this exemplary embodiment.

As shown inFIG. 7, the display apparatus100samples the texts261,262,263, and264extracted from the video frames sequenced in time by a predetermined number of times. Let successive points of time be t3, t4, t5 and t6, respectively. Further, let the texts extracted from the video frames at the respective points of time be a first text261, a second text262, a third text263and a fourth text264, respectively.

For example, the contents of the first text261, the second text262, and the fourth text264correspond to a word “feeding”, and the content of the third text263corresponds to a word “reading”. In this case, the contents of the second text262and the fourth text264before and after the point of time for the third text263are the same. Therefore, the display apparatus100determines that the third text263is not actually different in contents from the first text261, the second text262and the fourth text264and an error occurs in the OCR process. Thus, the display apparatus100determines the extracted third text263as dummy data and ignores the content of the third text263, thereby processing the third text263to have the same contents as the second text262.

In other words, the display apparatus100samples the texts extracted from the video frames sequenced in time by a preset number of times, and compares the text of the third video frame corresponding to the subsequent point of time with the text of the first video frame if the text of the second video frame is different in contents from that of first video frame corresponding to the previous point of time. As a result of comparison, if the text of the first video frame has the same contents as the text of the third video frame, the display apparatus100determines that the OCR error occurs while extracting the text from the second video frame, and determines that the text of the second video frame has the same contents as the texts of the first video frame or the third video frame.

Accordingly, the display apparatus100can compensate for an error in the OCR process, which may occur while extracting the successive texts.

Meanwhile, the foregoing exemplary embodiments describe the translation service as an example of the preset process for the text extracted from the video frame. However, the service for processing the extracted text is not limited to the translation. Alternatively, various service environments may be offered to a user as long as it can be realized in the display apparatus100.

FIG. 8shows an example where a video frame270is overlaid with a text271according to a fifth exemplary embodiment.

As shown inFIG. 8, the display apparatus100extracts the text271from the video frame270, and displays the video frame270to be overlaid with the extracted text271. The display apparatus100allows each text271to be selected by a cursor272.

A user moves the cursor272to a word of each text271and selects it. If a user selects one among the words of the text271, the display apparatus100may pop up the meaning and explanation of the selected word, and output the pronunciation of the corresponding word. To this end, the display apparatus100may be internally provided with a dictionary database of words, or may access the server (not shown) that offers a search service for this database.

The display apparatus100may reflect various services on the text extracted from the video frame, and provide it to a user.

A method of controlling the display apparatus100will be described.

FIG. 9is a flowchart showing a method of controlling a display apparatus according to a sixth exemplary embodiment.

As shown inFIG. 9, at operation5100, the display apparatus100pauses display of an image if an event occurs for designating the processing region. At operation5110, the display apparatus100sets up the processing region in the video frame based on coordinates input by a user. At operation S120, the display apparatus100stores position information of the setup processing region.

At operation5130, the display apparatus100resumes the display of the image. At operation5140, the display apparatus100sets up the processing region in accordance with the position information previously stored for the respective video frames to be displayed. At operation5150, the display apparatus100extracts the text with respect to the respectively setup processing region. At operation5160, the display apparatus100applies the preset process to the text.

At operation5170, the display apparatus100displays the processed texts in sync with the video frames, respectively.

FIG. 10is a flowchart showing a method of controlling a display apparatus according to a seventh exemplary embodiment. In this exemplary embodiment, an initial state refers to a state that the processing region is set up in accordance with the foregoing method shown inFIG. 9.

As shown inFIG. 10, at operation5200, the display apparatus100extracts and processes the text with respect to the setup processing region.

At operation5210, the display apparatus100calculates a difference value between the first area of the processing region designated at first by a user and the second area occupied with the text within the processing region of the next video frame.

At operation5220, the display apparatus100determines whether the calculated difference value is greater than a preset threshold.

If the calculated difference value is not greater than the threshold, at operation5230the display apparatus100does not adjust but maintains the processing region as it is.

On the other hand, if the calculated difference value is greater than the threshold, at operation5240the display apparatus100reduces the area of the processing region so that the difference value can be smaller than the threshold.

Further, at operation5250, the display apparatus100extracts and processes the texts with respect to the respective processing regions of the following video frames.