User terminal apparatus and controlling method thereof

A user terminal apparatus is provided. The apparatus includes a display which displays a screen, a storage which stores at least one template for defining a layout according to which a plurality of application windows is arranged on the screen, a user interface which receives a user command, and a controller which controls the display to arrange and display the plurality of application windows based on a template selected by the user command from among the at least one template on the screen.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. § 119(a) of a Korean patent application filed on Aug. 28, 2012 in the Korean Intellectual Property Office and assigned Serial No. 10-2012-0094508, the entire disclosure of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention The present invention relates to a user terminal apparatus and a controlling method thereof. More particularly, the present invention relates to a user terminal apparatus which provides a plurality of application windows in multi-tasking environment and a controlling method thereof.

2. Description of the Related Art

Support of electronic technology development has enabled development of various types of display apparatuses. Specifically, display apparatuses such as a TV, a Personal Computer (PC), a lap-top computer, a tablet PC, a mobile phone, and a MP3 player are widely distributed and used.

In order to meet user needs for newer and more various functions, mobile terminals such as tablet PCs and mobile phones provide a multi-tasking environment to simultaneously implement a plurality of applications.

Thus, a method of managing a plurality of application windows in multi-tasking environment is necessary.

Therefore, a need exists for an apparatus and method for arranging and displaying a plurality of application windows on a multi-tasking environment.

SUMMARY OF THE INVENTION

Aspects of the present invention are to address the above-mentioned problems and/or disadvantages and to provide at least the advantages described below. However, the present invention is not required to overcome the disadvantages described above, and an exemplary embodiment of the present invention concept may not overcome any of the problems described above.

An aspect of the present invention, a technical objective is to provide a user terminal apparatus capable of arranging a plurality of application windows on multi-tasking environment and displaying the same, and a controlling method is provided.

In accordance with an aspect of the present invention, a user terminal apparatus is provided. The apparatus includes a display which displays a screen, a storage which stores at least one template for defining a layout according to which a plurality of application windows is arranged on the screen, a user interface which receives a user command, and a controller which controls the display to arrange and display the plurality of application windows based on a template selected by the user command from among the at least one template on the screen.

In accordance with an aspect of the present invention, the plurality of application windows may include at least one of currently-executed application windows and application windows to be executed after the template is selected.

In accordance with an aspect of the present invention, the controller may arrange the plurality of application windows on window areas within the selected template according to a predetermined arranging order.

In accordance with an aspect of the present invention, the controller may consecutively arrange the plurality of executed application windows on window areas within the template based on an order of arranging the plurality of executed application windows on a Z-axis of the screen.

In accordance with an aspect of the present invention, the plurality of executed application windows may be consecutively arranged on window areas within the template based on a priority of a higher Z-order for arranging the plurality of executed application windows on the Z-axis.

In accordance with an aspect of the present invention, if the number of the plurality of executed application windows is more than the number of the window areas within the template, the controller may overlap and display the executed applications windows on the window areas within the template, and if the number of the plurality of executed application windows is less than the number of the window areas within the template, the controller may display the window areas such that application windows are not arranged as blank among the window areas within the template.

In accordance with an aspect of the present invention, if one executed application window is arranged on one window area within the template and if another executed application window is arranged to be overlapped on the one window area within the template, the controller may overlap and display the another application window on a lower portion of the one application window.

In accordance with an aspect of the present invention, the controller may consecutively arrange one or more application windows executed after selecting of the template on the window areas within the template according to an order of execution.

In accordance with an aspect of the present invention, if a new application window is executed in a state in which the plurality of executed application windows are arranged and displayed on all the window areas within the template, the controller may overlap and display the new application window on an upper portion of one of the plurality of application windows.

In accordance with an aspect of the present invention, the controller may provide, according to a predetermined user command, at least one of: a window edit mode in which a plurality of executed application windows are arranged within the template according to user selection; and a template edit mode in which the user performs at least one of editing prestored templates and creating a new template.

In accordance with an aspect of the present invention, if a predetermined menu item is selected, the controller may display at least one template on an area of the screen.

In accordance with an aspect of the present invention, the user terminal apparatus includes an input unit into which a touch input is entered.

In accordance with an aspect of the present invention, a method of controlling a user terminal apparatus is provided. The method includes receiving a user command to select a template for defining a layout according to which of a plurality of application windows is arranged on a screen, and arranging and displaying a plurality of application windows on the screen based on the selected template according to the user command.

In accordance with an aspect of the present invention, the plurality of application windows may include at least one of currently-executed application windows and application windows to be executed after the template is selected.

In accordance with an aspect of the present invention, the displaying of the plurality of application windows may include arranging the plurality of application windows on window areas within the selected template according to predetermined arranging order.

In accordance with an aspect of the present invention, the displaying of the plurality of application windows may include consecutively arranging the plurality of executed application windows on window areas within the template based on an order of arranging the plurality of executed application windows on a Z-axis of the screen.

In accordance with an aspect of the present invention, the displaying of the plurality of application windows may include, if the number of the plurality of executed application windows is more than the number of the window areas within the template, overlapping and displaying the executed applications windows on the window areas within the template, and if the number of the plurality of executed application windows is less than the number of the window areas within the template, displaying the window areas such that application windows are not arranged as blank among the window areas within the template.

In accordance with an aspect of the present invention, the displaying of the plurality of application windows may include, if one executed application window is arranged on one window area within the template and if another executed application window is arranged to be overlapped on the one window area within the template, overlapping and displaying the another application window on a lower portion of the one application window.

In accordance with an aspect of the present invention, the displaying may include consecutively arranging one or more application windows executed after selecting of the template on the window areas within the template according to an order of execution.

In accordance with an aspect of the present invention, if a new application window is executed in a state in which the plurality of executed application windows are arranged and displayed on all the window areas within the template, the method may include overlapping and displaying the new application window on an upper portion of one of the plurality of application windows.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

In the following description, same drawing reference numerals are used for the same elements even in different drawings. The matters defined in the description, such as detailed construction and elements, are provided to assist in a comprehensive understanding of the present invention. Accordingly, it is apparent that the exemplary embodiments of the present invention can be carried out without those specifically defined matters. Also, well-known functions or constructions are not described in detail because such a description would obscure the invention with unnecessary detail.

FIGS. 1A and 1Bare diagrams illustrating a user terminal apparatus according to an exemplary embodiment of the present invention.

Referring toFIG. 1A, a user terminal apparatus100may be implemented to be a tablet Personal Computer (PC)100A. However, the user terminal apparatus100is not limited to a tablet PC. For example, the user terminal apparatus100may be implemented as various types of apparatuses such as a mobile phone100B including a smart phone, a Personal Media Player (PMP), a Personal Digital Assistant (PDA), or a Global Positioning System (GPS) device, as long as the apparatuses have a display function and are portable. Further, the user terminal apparatus100may include a touch screen to allow a user to input commands for executing programs with a finger, a pen102, and/or the like.

More specifically, the tablet PC as one of examples of the user terminal apparatus100according to an exemplary embodiment of the present invention combines mobility of the PDA and functions of the laptop computer. Thus, while having functions of the desktop, wireless internet may be used. Although a main means for input is a touch screen, a conventional keyboard, a mouse, or the like may be connected and used.

Meanwhile, the user terminal apparatus100may provide a multi-tasking environment to simultaneously run a plurality of applications and carry out jobs. Hereinbelow, a method of providing a plurality of application windows in multi-tasking environment according to exemplary embodiments of the present invention will be described.

Referring toFIG. 1B, the user terminal apparatus100may include a display110, a user interface120, a controller130, and a storage140.

The display110may display a screen. In an exemplary embodiment of the present invention, the screen may include an application executing screen having various objects such as image, video image, text, a Graphic User Interface (GUI) screen, and the like.

More specifically, the display110may simultaneously display a plurality of application executing screens (e.g., may simultaneously display a plurality of application windows on a screen). This specific mode of displaying screen will be herein referred to as ‘multi-window mode’.

An application window provided on multi-window mode may be provided in a form that allows implementation of functions including moving position, adjusting size, or pin-up. Thus, the application window may include a title area having menu items to provide the corresponding functions and may be implemented to have a predetermined size. More specifically, the title area may include a maximization button, a finish button, or a pin-up button. According to manipulating on each button, an instruction of maximizing window, finishing window, or a pin-up window may be inputted. Meanwhile, the application window provided on multi-window mode may be implemented in a smaller size to occupy part of the entire screen area on the display110so as to allow convenient implementation of moving position and adjusting size. Such window mode will be herein referred to as a ‘mini mode’ or ‘free style mode’. In an exemplary embodiment of the present invention, the application window may be displayed to cover the entire screen size according to size adjustment by a user.

Further, the display110may display one application window in a maximized size on the whole screen area. The above screen displaying mode will be referred to herein as a ‘normal mode’. The application window provided on the normal mode may be provided in a form that does not have a title area so as to provide a user with as large an application executing screen as possible. If the application window provided on the normal mode has a maximized size to cover the whole screen area on the display110, the mode will be referred to as a ‘maximization mode’ or ‘split mode’ herein. If a plurality of application windows are displayed on the split mode, screen area may be divided and the plurality of application windows may be displayed in each divided area.

Meanwhile, the display110may be implemented as Liquid Crystal Display (LCD) panel, Organic Light Emitting Diodes (OLED), or the like, although the display is not limited thereto. Specifically, the display110may be implemented as a touch screen having inter-layer structure with a touch pad. In this case, the display110may be used as the user interface120as well as an output device, which will be described below. A touch screen may be configured to detect a touch input pressure, a touch input position, a touch input area, and the like.

The user interface120may receive various user commands. For example, the user interface120may receive a user command to display a title area on an application window.

According to an exemplary embodiment of the present invention, the user interface120may receive a user command to display the title area on application window when application window is displayed on maximization mode to cover whole screen area. The ‘user command’ may be user manipulation corresponding to touching a predetermined area of application window, such as a one-tap manipulation, and the like, although the user manipulation is not limited thereto. Accordingly, the user command may be a flick manipulation or a drag manipulation. According to an exemplary embodiment of the present invention, a certain area may be preset to be an information display area or a menu display area provided on, for example, the top of the application window, although not limited thereto.

When the maximized application window is displayed, the title area, displayed according to the user command, may provide a restore button which restores the window to mini mode or a finish button which closes the window. For example, in addition to the touch manipulation directed to the area excluding the area having buttons on the title area, a user command to restore maximization mode to mini mode may be received by a touch manipulation directed to the restore button on the title area, as described above.

According to exemplary embodiments of the present invention, the user interface120may receive a user command to restore the mini mode by reducing the application window by a predetermined size while the title area is displayed on the application window. The user command may be touch manipulation directed to the area excluding the area having buttons on the title area, or a touch manipulation directed to the window restore button provided on the title area.

The user interface120may receive a user command to restore window to mini mode while the application window is in maximized state and while the title area is not displayed. The user command may be a two-tap manipulation, flick-down manipulation, or drag-down manipulation directed to the information display area.

The user interface120may receive a user command to move application window position, a user command to adjust application window size, or a user command to pin-up application window on multi-window mode. The user command to pin-up may be implemented as a pin-up on/off by a tap manipulation directed to the pin-up button. Thus, when pin-up is on, the application window may be displayed in a fixed form. If pin-up is off, the application window may be implemented in a form in which moving position and adjusting size are possible.

The controller130may control the overall operation of the user terminal apparatus100.

Displaying a Plurality of Application Windows on Multi-Window Mode

The controller130may control the display110to arrange and display a plurality of application windows on a screen based on the template selected by a user command inputted through the user interface120from among one or more templates stored in the storage140. The plurality of application windows may include at least one of the application windows executed following selection of a currently-executing application window and template.

If a predetermined menu item is selected, the controller130may display at least one of the templates stored in the storage140on one area of a screen. For example, menu items to select templates may be installed on a screen, and if a menu item may be selected, at least one of previously stored templates may be arranged and displayed so that a user can select a template according to the user's preferences. In this case, the menu item may be displayed as an icon shape showing specific template format, although such a display configuration is not limited thereto.

The controller130may arrange a plurality of application windows on windows within the selected template according to a predetermined arrangement order. The ‘predetermined arrangement order’ as used herein may be left-to-right direction (from − to + on X axis of screen), top-to-bottom direction (from − to + on Y axis of screen), or top-to-bottom direction on Z axis of screen (from − to + on Z axis of screen) based on the windows within a template.

For example, if a template dividing a screen into four equal parts is selected and if there are five application windows to be arranged, a first application window is placed on left top area, a second application window is placed on right top area, a third application window is placed on left bottom area, a fourth application window is placed on right bottom area, and a fifth application window is placed to overlap a lower portion of the first application window placed on the left top area. Thus, if a first application window closes, a fifth application window may be displayed on the left top area.

Further, the controller130may consecutively arrange a plurality application windows which is currently executed on the window within a template, based on an order of arranging the plurality of executed application windows toward Z direction of the screen.

According to an exemplary embodiment of the present invention, the first to fifth applications may be application windows displayed in sequence on a current screen from + direction of Z axis to − direction of Z axis. In other words, if a plurality of application windows is executed, the plurality of application windows may be overlapped and displayed on each other in the Z axis direction of the screen before the application window is arranged based on template. In this case, a most recently operated application may be displayed on the highest portion. The application windows within template may thus be arranged by considering an operating order according to an exemplary embodiment of the present invention.

According to an exemplary embodiment of the present invention, application windows may be arranged within a template according to an application executing order, a frequency of operating executed application windows, an operating hour (or time), and the like. For example, a plurality of currently-operating application windows may be arranged and displayed according to a predetermined arrangement order that gives priority to higher operating frequency.

The controller130may properly arrange the application windows within template according to the number of executed application windows and the number of windows within the selected template.

The controller130may arrange executed application windows to overlap other windows within a template, if the number of executed application windows is more than the number of windows within the selected template. If a first executed application window is arranged on a first window within the template and a second executed application window is arranged to overlap the first window within the template, the controller130may arrange the second application window to overlap on the lower portion of the first application window. Because the example is already described above, this will not be further explained below.

Further, if the number of executed application windows is less than the number of windows within template, the controller130may display a blank area for a window area in which the application window is not arranged, among the windows within the selected template. If new application window is executed, the new application window may be automatically displayed on the blank area.

Further, the controller130may perform arranging of the application windows executed following selecting of a template, on the window area within the template according to the same arrangement order.

In this case, if the new application window is executed while a plurality of executed application windows are arranged and displayed on every window within the selected template, the controller130may overlap and display the new application window on the upper portion of one of the plurality of application windows. For example, if the most currently arranged application window is a left window of the windows within template dividing a screen into two equal parts, the new application window may be overlapped and displayed on the next order (e.g., on the right window).

According to an exemplary embodiment of the present invention, a template may automatically change based on the number of executed applications. For example, if a template dividing a screen to two equal parts is selected, and if new application window is executed while there are two application windows arranged on the two windows within the template respectively, the template may automatically change to a template that can arrange three application windows on a screen, and thus, three executed application windows may be consecutively arranged on a screen.

According to an exemplary embodiment of the present invention, a template may be recommended according to the number of executed applications. For example, if a template having format dividing a screen into two equal parts is selected, and if a new application window is executed while there are two application windows arranged on two windows within the template respectively, another template that can arrange three application windows on a screen may be recommended to the user.

The above methods of automatically changing and recommending may be applied by considering operating frequency or numbers of application windows. For example, the methods of automatically changing and recommending may be applied only if repeated operations are performed more than a predetermined frequency on both of the two executed application windows in the above exemplary embodiment.

Window Edit Mode

If a predetermined user command is inputted, the controller130may provide a window edit mode screen so as to arrange a plurality of executed application windows according to a user selection on a window within the selected template. The predetermined user command may be a pinch manipulation on a screen. Specifically, the predetermined user command may be a pinch-in manipulation that uses multi-touch. Further, the window edit mode screen may close according to a pinch-out manipulation. The ‘pinch-in manipulation’ as used herein refers to manipulation of two fingers joining closer while ‘pinch-out manipulation’ as used herein refers to manipulation of the two fingers moving apart from each other. According to exemplary embodiments of the present invention, another user manipulation such as flick-up and flick-down may be used, or another menu time to enter and exit window edit mode screen may be provided.

On window edit mode, the application window position may be adjusted within template according to a drag-and-drop user manipulation. In this case, the size of the moved application window may be automatically adjusted to correspond to the window size within template.

Further, on the window edit mode, currently executed applications may be displayed on the lower portion. Thus, a user may select an application and arrange the application on a desired area. In this case, an image corresponding to the application may be displayed on part of the corresponding application window area so that a user can recognize the application.

Further, a new application window may also be executed on the window edit mode, and the corresponding application window may be added to the lower area showing therein currently executed applications. Other detailed functions of the window edit mode will be described below by referring to the drawings.

Template Edit Mode

If a predetermined user command is inputted, the controller130may provide a template edit mode so that a user can edit previously stored templates or create new template. The ‘predetermined user command’ as used herein may be a multi-touch user manipulation including, for example, pinch-in, pinch-out, flick-up, flick-down, and the like. According to exemplary embodiments of the present invention, depending on need and/or user preferences, another menu item to enter and exit template edit mode screen may be provided.

On the template edit mode, a user may edit previously stored templates according to the user's needs and/or preferences and create a new template. For example, the user may adjust and edit window size within template divided into two equal parts from 1:1 ratio to 2:1 ratio. Other detailed functions of the template edit mode will be described below by referring to the drawings.

According to an exemplary embodiment of the present invention, the template editing may be performed on the window edit mode.

Method of Controlling Screen on Other Multi-Window Mode

The controller130may control the user terminal apparatus100so that the application window is reduced to a predetermined size (e.g., displayed on mini mode on the multi-window mode). In this case, a title area may be displayed in a fixed manner on one part of mini window area. Therefore, on the maximization mode in which the application window is displayed on the whole screen area, the title area may not be displayed basically and may be displayed according to a user command. However, on the multi-window mode, the title area may be displayed basically in a fixed manner. According to an exemplary embodiment of the present invention, the title area may be displayed according to a user command even on the mini mode.

Further, the controller130may control the user terminal apparatus100so that a guide GUI is displayed to provide guide about position movement and size adjustment of the application windows on the multi-window mode according to a user command. In this case, the controller130may control the guide GUI to provide haptic feedback on an area on which the guide GUI is displayed.

Further, the controller130may provide various menu items to implement the multi-window mode and the normal mode.

The storage140may perform a function of storing at least one or more templates defining layout to arrange a plurality of application windows on a screen. For example, the storage140may store various formats of templates provided by the number of windows. Further, templates edited by using the templates stored in the storage140or newly created templates may be additionally stored in the storage140.

FIG. 2is a detailed block diagram of the user terminal apparatus according to an exemplary embodiment of the present invention.

Referring toFIG. 2, the user terminal apparatus100may include a display110, a user interface120, a controller130, a storage140, an application driver150, a feedback provider160, a communicator170, an audio processor180, a video processor185, a speaker190, a button191, a USB port192, a camera193, and a microphone194. Among the components illustrated inFIG. 2, the components described above with reference toFIG. 1will not be further explained as such a description would be redundant.

The operation of the controller130may be performed by programs stored in the storage140. The storage140may store various data such as an Operating System (OS) software module to drive the user terminal apparatus100, various applications, various data inputted or established while executing applications, or content. Particularly, depending on the characteristics thereof, the applications stored in the storage140may be classified into applications that support multi-window mode and other applications that do not support the multi-window mode.

Further, the storage140may store various types of templates predefining layout to arrange a plurality of application windows on the multi-window mode.

Other various software modules stored in the storage140will be described below by referring toFIG. 3.

The application driver150may drive and implement applications that the user terminal apparatus100can provide. The ‘application’ as used herein may refer to the programs that can automatically run which may include various multimedia content. The term, ‘multimedia content’ as used herein includes text, audio, still images, animation, video, interactivity content, Electronic Program Guide (EPG) content from content provider, electronic message received from users, or information regarding current events, although not limited thereto.

The feedback provider160may perform a function of providing various feedbacks according to functions implemented in the user terminal apparatus100.

For example, the feedback provider160may provide haptic feedback regarding a GUI displayed on a screen. The haptic feedback technology, or computer touch technology enables users to feel touch, by generating vibration, power or impact in the user terminal apparatus100. The feedback provider160may provide haptic feedback regarding guide a GUI, if the guide GUI is displayed to provide guide about position movement and size adjustment of application windows according to user command on the multi-window mode.

In this case, the feedback provider160may provide various feedbacks by differently applying vibrating conditions such as a vibrating frequency, a vibrating length, a vibrating shape, a vibrating position, and the like according to controlling of the controller130. A method of generating various haptic feedback by differently applying vibration corresponds to technology according to the related art, and will not be further explained for the sake of brevity.

Although the feedback provider160may use a vibration sensor and provide haptic feedback, according to exemplary embodiments of the present invention haptic feedback may be provided by using a piezo sensor.

The communicator170may configured to communicate with various types of external apparatuses according to various types of communication methods. The communicator170may include various communication chips such as a Wi-Fi chip171, a Bluetooth chip172, a wireless communication chip173, and the like.

The Wi-Fi chip171and the Bluetooth chip172may communicate according to Wi-Fi method and Bluetooth method, respectively. The wireless communication chip173may communicate according to various communication protocols such as Institute of Electrical and Electronics Engineers (IEEE) standards, ZigBee, 3rd Generation (3G), 3rd Generation Partnership Project (3GPP), or Long Term Evolution (LTE). The communicator170may further include a Near Field Communication (NFC) chip operating according to NFC method using 13.56 MHz from the various RF-ID frequency bandwidths such as 135 kHz, 13.56 MHz, 433 MHz, 860˜960 MHz, and 2.45 GHz.

The audio processor180is configured to process audio data. In the audio processor180, various audio data processing such as decoding, amplifying, or noise filtering may be implemented.

The video processor185is configured to process video data. In the video processor185, various video data processing such as decoding, scaling, noise filtering, converting frame rate, or converting resolution may be implemented.

The speaker190is configured to output various alarm sounds and voice messages as well as various audio data processed in the audio processor180.

The button191may correspond to various types of buttons such as a mechanical button, a touch pad, or a wheel installed on a random area of front face, side face, or back face of a body of the user terminal apparatus100. For example, a button to turn electrical power on/off may be installed on the user terminal apparatus100.

The USB port192may perform communication with various external apparatuses through USB cables or charge electric power.

The camera193is configured to photograph still image or video image according to controlling of a user. The camera193may be implemented with a plurality of cameras including a front face camera and a back face camera.

The microphone194is configured to receive user voices or other sounds and to convert the received sound into audio data. The controller130may use user voices inputted through the microphone194during a call. The controller130may convert user voices into audio data and store the audio data in the storage140.

If the camera193and the microphone194are provided, the controller130may perform controlling operation according to user voices inputted through the microphone194or according to user motions recognized by the camera193. For example, the user terminal apparatus100may operate on motion controlling mode and voice controlling mode. If operating on motion controlling mode, the controller130may activate the camera193to photograph a user, trace changes in user motions and perform corresponding controlling operations. If operating on voice controlling mode, the controller130may analyze user voices inputted through the microphone and perform controlling operations according to the analyzed user voices (e.g., may operate on voice recognizing mode).

According to exemplary embodiments of the present invention, the user terminal apparatus100may include various external inputting ports to connect various external components such as a headset, a mouse, a LAN, and the like.

The controller130may control the overall operation of the user terminal apparatus100by using various programs stored in the storage140.

For example, the controller130may implement applications stored in the storage140, create an implementing screen, and display the screen. Further, the controller130may play various content stored in the storage140. The controller130may also communicate with external apparatuses through the communicator170.

According to exemplary embodiments of the present invention, the controller130may include a Random Access Memory (RAM)131, a Read Only Memory (ROM)132, a main Central Processing Unit (CPU)133, a graphic processor134, first to (n)th interfaces135-1˜135-n, a bus136, and the like.

The RAM131, the ROM132, the main CPU133, the graphic processor134, and the first to (n)th interfaces135-1˜135-nmay connect to each other through the bus136.

The first to (n)th interfaces135-1˜135-nmay connect to the sorts of components as described above. According to exemplary embodiments of the present invention, one of the interfaces may be a network interface connecting to external apparatuses through network.

The main CPU133may access the storage140, and perform booting by using the Operating System (OS) stored in the storage140. The main CPU133may perform various operations by using various programs, content, and data stored in the storage140.

The ROM132may store therein a set of commands to boot the system. If a command to turn-on is inputted and if electrical power is provided, the main CPU133may load the OS stored in the storage140to the RAM131according to commands stored in the ROM132, run the OS, and boot the system. If booting completes, the main CPU133may paste various application programs stored in the storage140to the RAM131, implement the application programs pasted on the RAM131, and perform various operations.

The graphic processor134may create a screen including various objects such as icons, images, or text by using a calculator (not illustrated) and a renderer (not illustrated). The calculator may calculate attribute values such as coordinates, shapes, sizes, or colors of each object to be displayed according to screen layout by using a control command received from an inputter134. The renderer may create a screen of various layouts including objects based on the attribute values calculated by the calculator. The screen created by the renderer may be displayed within a display area of the display110.

According to exemplary embodiments of the present invention, the user terminal apparatus100may additionally include a detector (not illustrated). The detector (not illustrated) may detect various manipulations of the user terminal apparatus100such as touching, rotating, sliding, pressing, and zooming.

According to exemplary embodiments of the present invention, the detector (not illustrated) may include a touch sensor for detecting touch. A touch sensor may be implemented as a capacitive type or a resistive type. The capacitive type sensor uses coated conductive material on the surface of the display150, detects micro electricity excited by movement of a user when a user touches the surface of the display110(e.g., with the user's hand, a pen, or the like), and calculates touch coordinate. The resistive type sensor includes two electrode panels, detects the electrical current flow caused by contacting upper panel and lower panel on a touch point if a user touches the screen, and calculates touch coordinate. The touch sensor may be implemented with various types as explained above. The detector may further include a geomagnetic sensor to detect rotating situation and moving direction of the user terminal apparatus100, an acceleration sensor to detect sliding degree of the user terminal apparatus100, and the like.

FIG. 2shows an example of the detailed configuration of the user terminal apparatus100. According to exemplary embodiments of the present invention, certain component(s) ofFIG. 2may be excluded or modified, or other component(s) may be added. For example, a Global Positioning System (GPS) receiver (not illustrated) for receiving GPS signals from the GPS satellite and for calculating current position of the user terminal apparatus100, or a Digital Multimedia Broadcasting (DMB) receiver (not illustrated) for receiving and processing DMB signals may be further included.

FIGS. 3A and 3Bare diagrams illustrating a software structure stored in a storage.

Referring toFIG. 3A, the storage140may store software having a base module141, a sensing module142, a communicating module143, a presentation module144, a web browser module145, and a service module146.

The base module141is a basic module which processes signals transmitted from hardware included in the user terminal apparatus100and delivers the processed signals to a higher layer module. The base module141includes a storage module141-1, a security module141-2, a network module141-3, and/or the like. The storage module141-1is a program module which manages a Database (DB) or registry. The main CPU133may access the DB in the storage140by using storage module141-1and detect various data. The security module141-2is a program module which supports hardware certification, request permission, or secure storage. The network module141-3supports network connection, and includes a Device Network (DNET) module or a Universal Plug and Play (UPnP) module.

The sensing module142is collects information from various sensors, analyzes, and manages the collected information. The sensing module142may include a face recognizing module, a voice recognizing module, a motion recognizing module, a NFC recognizing module, and the like.

The communicating module143is provided for external communication. The communicating module143may include a messaging module143-1such as a messenger program. Short Message Service (SNS) and Multimedia Message Service (MMS) program, an e-mail program, and the like. The communicating module143may include a calling module143-2such as a call info aggregator program module and a VoIP module.

The presentation module144creates a display screen. The presentation module144includes a multimedia module144-1to play and output multimedia content, and a UI rendering module144-2to process a UI and a graphic. The multimedia module144-1may include a player module, a camcorder module, a sound processing module, and the like. Thus, the multimedia module144-1plays various multimedia content, and creates and plays screens and sounds. The UI rendering module144-2may include a coordinate combining module to combine and create coordinates of the screen in which images may be displayed, a X11 module to receive sorts of events from hardware, and a Two Dimensional 2D/Three Dimensional (3D) UI toolkit to provide tools for creating a UI of 2D or 3D.

The web browser module145performs web browsing and accesses web server. The web browser module145may include various modules such as a web view module to create a web page, a download agent module to perform downloading, a bookmark module, a webkit module, and the like.

The service module146includes various applications to provide various services. Specifically, the service module146may include various program modules such as a navigation program, a content play program, a game program, an electronic book program, a calendar program, an alarm manage program, or various other widgets.

AlthoughFIG. 3Aillustrates various program modules, according to exemplary embodiments of the present invention, certain program module(s) may be excluded, modified, or added according to types and features of the user terminal apparatus100. For example, a position base module for operating in association with hardware such as a GPS chip and for supporting position base service may be further included.

Referring toFIG. 3B, an example of a multi-window framework architecture on system (ex. Android® system) used by the user terminal apparatus100according to an exemplary embodiment of the present invention in order to display a plurality of application windows on a screen is provided. The framework architecture ofFIG. 3Bmay be one component of the software illustrated inFIG. 3A; however, this will be explained by referring to a separate drawing for convenient explanation.

As illustrated inFIG. 3B, the multi-window framework architecture may include an application framework310and a multi-window framework320. In this case, the multi-window framework320may operate separately from the application framework310.

The application framework310may include an activity manager311, a window manager312, and a view system313. The multi-window framework320may include a multi-window manager321.

As applications are executed, the activity manager311may call information corresponding to an executing window of the executed application to the multi-window framework. The activity manager311may receive information regarding display mode, size and position of an application executing window based on a life cycle of the application executing window from the multi-window framework. The activity manager311may call information regarding display mode, size and position of the application executing window during creating step of the life cycle of the application executing window.

Further, the window manager312may confirm the application executing window corresponding to a touch inputted by a user. The window manager312may provide position information on a display corresponding to a touch input of a user to the multi-window framework, and receive information of the application executing window corresponding to the touch input determined by the multi-window framework from the multi-window framework.

According to exemplary embodiments of the present invention, in response to a touch input of a user, the window manager312may receive information regarding position and size of application executing window from the multi-window framework, and determine application executing window corresponding to touch input of a user based on the received position and size of application executing window.

The view system313may confirm positions and sizes of widget window and pop-up window. In this case, the multi-window framework320may determine sizes and positions of widget window and pop-up window, and the view system313may receive information regarding sizes and positions of widget window and pop-up window from the window framework.

The multi-window manager321included in the multi-window framework320manages various operations regarding multi-window functions provided from the user terminal apparatus100, and provides various Application Programming Interfaces (APIs) regarding multi-window functions. Further, multi-window service may store various APIs regarding multi-window functions. An API regarding common functions of single window and multi-window may be embodied as common class, and an API regarding functions only applied in multi-window may be implemented so as to be divided according to display mode. The application framework310may further include a content provider314, a package manager315, a telephone manager316, a resource manager317, a position manager318, a notice manager319, and the like, but they are not related to the present invention, and thus further explanations will be omitted.

The multi-window framework320may further include a multi-window service322, which will not be further explained because service to be provided is described above.

FIGS. 4A to 4Dare diagrams illustrating a method of displaying application supporting multi-window mode according to an exemplary embodiment of the present invention.

Referring toFIGS. 4A-4D, various menu items may be displayed as icon interface format on an initial screen according to an exemplary embodiment of the present invention. For example, a first menu item10may be displayed on a lower center area of a screen, and a second menu item20may be displayed on an upper right area of a screen. The first menu item10operates to display applications that support multi-window mode on a specific screen area, and the second menu item20operates to arrange and display all applications which may be provided by the user terminal apparatus100according to user manipulation on an entire screen area.

If the first menu item10is selected, referring toFIG. 4B, application icons (hereinbelow, ‘applications’) referring to applications that support multi-window mode are displayed in a line on a lower portion of the screen. For convenient explanation, the area displaying the applications in a line is referred to as a ‘mini tray’420.

Further, a third menu item30and a fourth menu item40may be displayed on left and right sides of the first menu item10. The third menu item30operates to provide various formats of templates defining layouts to arrange a plurality of application windows on a screen in multi-window mode. The fourth menu item40operates to provide a list of applications that are currently implemented on the multi-window mode.

If a user manipulation of touching the mini tray420and dragging the mini tray420toward a specific direction is inputted, applications move toward dragging direction as inFIG. 4C.

If the user touch manipulation ceases, referring toFIG. 4D, movement of application stops with the display state being maintained at the moment the touch manipulation is ceased.

For example, in a state in which the first to fifth applications421to425are displayed on the mini tray420, if a user touches a specific application424and drags the mini tray420to the left direction, the application424moves by a distance as long as a user drags, and other icons on the mini tray420moves toward the same direction. Further, in a state in which third to seventh applications423to427are displayed on the mini tray420, if user touch manipulation ceases, movement of applications on the mini tray420stops in the current displaying state.

FIGS. 5A and 5Bare diagrams illustrating display formats of an application window when an application is executed on multi-window mode and normal mode according to an exemplary embodiment of the present invention.

Referring toFIG. 5A, if the first menu item10is selected, applications that support multi-window mode may be displayed on a lower portion of the screen. If a specific application501is selected, a corresponding application window510may be displayed in mini mode format. In this case, the title area511may be displayed on the application window510and various menu items to support multi-window mode may be displayed on the title area511. For example, menu items may include a maximization button511-1, a complete button511-2, and a pin-up button511-3, and the like.

Referring toFIG. 5B, if the second menu item20is selected, all the applications that the user terminal apparatus100can provide may be displayed on the entire screen area. If a specific application501is selected, a corresponding application window520may be displayed on maximization mode. According to exemplary embodiments of the present invention, the title area may not be displayed on the application window520.

Thus, even if the same application is selected, different corresponding application windows may be displayed depending on a user intention and/or preferences. If a user intends to execute an application on multi-window mode, a window may be displayed in mini mode which enables implementation of the functions such as moving a window position, adjusting size, pin-up, and the like. If a user intends to execute an application on normal mode, the window may be displayed in maximization mode. Thus, the window displaying can be implemented by considering a user intention according to the user's convenience and/or preferences.

FIGS. 6A to 6Dare diagrams illustrating a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIG. 6A, a first menu item10is selected in a state in which a plurality of application windows610to630are displayed on foreground or on background of a screen on a user terminal apparatus100according to an order of operations.

Referring toFIG. 6B, applications that support multi-window mode are displayed on mini tray area on a lower screen. Further, a third menu item30and a fourth menu item40to support multi-window mode may be displayed on left and right sides of the first menu item10. The plurality of application windows610to630may be displayed on a background of the screen and may be displayed in a dimmed state, a dark state, or the like.

Referring toFIG. 6C, if the third menu item30is selected, the template providing area650having various types of templates651to655to arrange a plurality of application windows on a multi-window mode may be displayed.

Referring toFIG. 6D, if a template653of at least one provided templates is selected, a plurality of executed application windows610to630may be arranged and displayed based on the selected template. In this case, on the screen ofFIG. 6A, application windows may be arranged consecutively from left to right direction, or from upper to lower direction according to a sequence of: an application window610displayed on a highest portion, an application window620displayed therebelow, and an application window630displayed on a lowest portion.

FIGS. 7A to 7Eare diagrams illustrating a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIG. 7A, one template752is selected in a state when a template providing area750having a plurality of templates751to755is displayed.

Referring toFIGS. 7B and 7C, if one application704is selected according to user manipulation in a state in which applications701to705that support multi-window mode are displayed, a screen may be provided, in which the window710of the selected application corresponding to the selected template is arranged.

Referring toFIGS. 7D and 7E, if another application703is additionally selected according to user manipulation in a state in which applications701to706that support multi-window mode are displayed, a screen may be provided by adding and arranging window720of the selected application corresponding to the selected template.

According to exemplary embodiments of the present invention, if one application is selected on a mini tray having applications that support multi-window mode, a screen may be displayed by arranging the selected application corresponding to the selected template, and another application may be additionally selected to arrange new application on the selected template.

FIGS. 8A to 8Care diagrams illustrating a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIG. 8A, one template852is selected in a state in which a template providing area850having a plurality of templates851to855is displayed.

Referring toFIGS. 8B and 8C, if a plurality of applications803and804are selected in a state in which applications801to806that support multi-window mode are displayed according to user manipulation, a screen arranging windows810and820of the selected a plurality of applications corresponding to the selected template may be provided. According to exemplary embodiments of the present invention, the plurality of applications801,805may be selected simultaneously or consecutively as illustrated inFIG. 8B.

According to exemplary embodiments of the present invention, if a plurality of applications is selected on a mini tray having applications that multi-window mode can support, a screen arranging the selected a plurality of applications corresponding to the selected template may be displayed.

FIGS. 9A to 9Care diagrams illustrating explain a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIGS. 9A and 9B, in a state in which a plurality of currently-executed application windows901to903are arranged based on the selected template, if user manipulation on a screen (e.g., if a pinch-in manipulation is inputted with respect to the screen), the window edit mode screen may be provided as illustrated inFIG. 9B. Meanwhile, user manipulation to enter the window edit mode is not limited to the above, and various gesture manipulations are available.

As illustrated inFIG. 9B, on the window edit mode, an application arranging area910including the currently-executed applications911to913may be provided on a lower screen. According to exemplary embodiments of the present invention, an image corresponding to the application may be indicated on part of the window area so that a user can recognize the application.

On the window edit mode, the selected template format920may be displayed on a center area of the screen, and a template area950having templates that are available may be displayed on an upper right area of the screen.

According to exemplary embodiments of the present invention, a user may drag-and-drop the applications displayed on a lower area of the windows921to923included in the displayed template920to a desired window. For example, a first application911may be placed on a first window921, a second application912on a second window922, and a third application913on a third window923.

Referring toFIG. 9C, the applications911to913placed on a template920may disappear from the application arranging area910. Window size of moved application may be automatically adjusted to correspond to the window size of the template.

If there is no application arranging manipulation by drag-and-drop manipulation, it is available that an application displayed on a lower area of a window according to a predetermined order is automatically arranged on windows912to923.

According to exemplary embodiments of the present invention, a template area, a position and a size of an arranging application display area may be variously modified.

FIGS. 10A and 10Bare diagrams illustrating a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIGS. 10A and 10B, if user manipulation (e.g., a pinch-out manipulation, or the like) is inputted on the window edit mode screen, the window edit mode may finish. For example, as illustrated inFIG. 10B, a screen in which position of application window is adjusted to a format edited on the window edit mode may be displayed.

FIGS. 11A and 11Bare diagrams illustrating a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIGS. 11A and 11B, a template may be edited according to a manipulation of moving window boundary within template on the window edit mode screen. For example, if a user manipulation is inputted by touching a boundary line1110of a first window921, a second window922, and a third window923within the template920and dragging the boundary line110to the left side, the template920may be modified to a format in which the boundary line1110moves to the left side. When the template is modified to a new format, the new format includes a first window921-1, a second window922-1, and a third window923-1with a boundary line1110.

According to exemplary embodiments of the present invention, a template edit mode screen may also be provided, separately from the window edit mode screen.

FIGS. 12A and 12Bare diagrams illustrating a method of displaying screen according to an exemplary embodiment of the present invention.

Referring toFIGS. 12A and 12B, in a state in which first to third application windows1210to1230are arranged and displayed based on the selected template, the title area1231of the third application window1230may be touched and dragged (or dragged and dropped) in the direction of the first application window1210.

According to exemplary embodiments of the present invention, the third application window1230may be displayed on position of the first application window1210and the first application window1210may be displayed on position of the third application window1230. For example, the first application window1210and the third application window1230may be interchanged as illustrated inFIGS. 12A and 12B. The first and third application windows1210and1230moved as illustrated may be adjusted and displayed to a size corresponding to each area, respectively.

Although not illustrated in the drawing, in a state in which the first to third application windows1210to1230are arranged and displayed based on the selected template, a window size can be adjusted according to user manipulation. For example, a window size may be adjusted by touching and dragging boundary area of application window. In this case, information on the modified template state may be stored, and the template may be stored in the same manner as the modified template state.

FIGS. 13A and 13Bare diagrams illustrating template formats according to various exemplary embodiments of the present invention.

Referring toFIG. 13A, various templates1311to1318may be provided by the number of windows and previously stored. Although the template format with four windows is provided for illustrative purposes, exemplary embodiments of the present invention are not so limiting.

Referring toFIG. 13B, templates may be edited or created in various formats1319to1321according to a user needs. For example, it is possible to create a template in which window size is adjusted by moving position of window boundary line in a prestored template1312as illustrated in the left-most drawing.

Further, a new format of template can be created without using prestored templates as illustrated in the middle and right-most drawings. In addition, the corresponding template may be created by using prestored templates. For example, in the prestored template1312, one window of two windows may be divided into two and boundary line may be adjusted, creating new template format in which three windows are arranged horizontally as the one illustrated in the middle drawing.

FIGS. 14A and 14Bare diagrams illustrating a method of arranging application windows according to an exemplary embodiment of the present invention.

Referring toFIG. 14A, if a specific template is selected in a state in which two application windows (e.g., application window A1410and B application window B1420) are simultaneously executed and displayed, the executed application window A1410and application window B1420may be arranged and displayed based on the selected template.

Referring toFIG. 14B, if a new application window (e.g., application window C1430) is executed, application window C1430may be overlapped and displayed on the previously-executed application window A1410.

If another new application window (e.g., application window D1440) is executed, application window D1440may be overlapped and displayed on the previously-executed application window B1420.

According to exemplary embodiments of the present invention, if a new application window is executed, the new application window may be overlapped and displayed on a previously displayed application window according to a predetermined arranging order.

FIGS. 15A to 15Care diagrams illustrating a method of arranging application windows according to an exemplary embodiment of the present invention.

Referring toFIG. 15A, the plurality of currently-executed application windows1510to1560may have a Z-order corresponding to the order of displaying the windows on Z-axis of a screen. In other words, application window A1510to application window F1560may have a Z-order of 1 to 6.

Referring toFIG. 15B, the currently-executed application windows corresponding to application window A1510to application window F1560according to the predetermined arranging order on a selected template1570may be consecutively arranged based on the priority of higher Z-order. Thus, application window A1510having a 1 Z-order may be placed on upper left area and application window B1520having a 2 Z-order may put on upper right area. On lower left and right areas, application window C1530and application window D1540having 3 and 4 Z orders may be respectively placed. Application window E1550and application window F1560having 5 and 6 Z-orders respectively may be overlapped and displayed on the areas in which application window A1510and application window B1520having 1 and 2 Z-orders respectively are already arranged.

Referring toFIG. 15C, if application windows corresponding to application window G1591to application window L1596are executed after selecting a template1580, the application windows may be consecutively arranged on the template1580according to an execution order. Thus, the first executed application window G1591may be arranged on upper left area, and the next executed application window H1592may be arranged on upper right area. On lower left and right areas, the next executed application windows corresponding to application window11593and application window J1594may be consecutively arranged. The next executed application windows corresponding to application window K1595and application window L1596may be overlapped and arranged consecutively on upper left and right areas.

FIG. 16is a flowchart illustrating a method of controlling a user terminal apparatus according to an exemplary embodiment of the present invention.

Referring toFIG. 16, at S1610, a user command to select a template defining layout is received, so that a plurality of application windows may be arranged on a screen.

At S1620, a plurality of application windows are arranged and displayed on a screen based on the selected template by the received user command.

According to exemplary embodiments of the present invention, the plurality of application windows may include at least one of application windows which are currently executed and application windows which are to be executed after selecting a template.

According to exemplary embodiments of the present invention, at S1620, a plurality of application windows may be arranged on window area within the selected template according to predetermined arranging order.

According to exemplary embodiments of the present invention, at S1620, the plurality of executed application windows may be consecutively arranged on window area within the template based on an order of arranging the plurality of executed application windows on Z-axis of a screen.

According to exemplary embodiments of the present invention, at S1620, if the number of executed application windows is more than the number of window areas within a template, executed application windows may be overlapped and arranged on window area within a template. If the number of executed application windows is less than the number of window areas within a template, window areas in which application windows are not arranged may be displayed as blank among window areas within a template.

If the first currently-executed application window is arranged on the first window area within a template and if the second currently-executed application window is overlapped and arranged on the first window area within the template, the second application window may be overlapped and arranged on a lower portion of the first application window.

According to exemplary embodiments of the present invention, at S1620, at least one of application windows executed after selecting a template may be consecutively arranged on the window areas within a template based on the order of execution.

According to exemplary embodiments of the present invention, at S1620, if new application window is executed in a state in which the plurality of currently-executed application windows are arranged and displayed on all window areas within a template, the new application window may be overlapped and displayed on an upper portion of one of the plurality of executed application windows.

Further, according to a predetermined user command, at least one of window edit mode to arrange a plurality of executed application windows within a template by a user selection, and a template edit mode for a user to edit prestored templates or to create new templates also may be provided.

According to exemplary embodiments of the present invention, the controlling method may be implemented to be a computer program and provided to a user terminal apparatus. For example, if a template for defining layout to arrange a plurality of application windows on a screen is selected, non-transitory computer readable medium storing program may be provided, in which the program arranges a plurality of application windows on a screen based on the selected template.

The ‘non-transitory computer readable medium’ as used herein refers to medium that can store data semi-permanently and can be read by apparatuses, rather than a medium that can store data temporarily such as a register, a cache, and a memory. Specifically, non-transitory computer readable medium such as the CD, the DVD, a hard disk, the Blu-ray disk, the USB, a memory card, and the ROM may store and provide the above various applications or programs.