Abstract:
An interface for searching and playing recorded video comprises a filmstrip snapshot sequence, a selection window, a first input device, a second input device, and a third input device. The filmstrip snapshot sequence comprises chronologically ordered snapshots associated with video segments of the recorded video. The snapshots are taken at regular intervals that may be equal to the length of the video segments. The selection window highlights consecutive snapshots from the filmstrip snapshot sequence. The first input device advances the filmstrip snapshot sequence, causing the selection window to highlight a chronologically later set of snapshots. The second input device zooms in on the filmstrip snapshot sequence as a function of time, causing a new set of snapshots to be retrieved at a smaller regular interval. The third input device causes the video segments associated with the highlighted snapshots to be played.

Description:
BACKGROUND 
       [0001]    The present invention relates generally to video searching, and more particularly to a search interface for locating a transition event in recorded video. 
         [0002]    Video surveillance commonly produces a large volume of recorded video, of which only a few minutes or a few seconds may be of interest in the event of a theft or incident. A camera in an art museum, for instance, might capture several hours of footage of normal activity on a night a painting is stolen, while the theft itself might only appear on recorded video for a minute or less. Many events—herein referred to as transition events—cause a persistent change in an environment under surveillance. Playing through large volumes of video to find such events can be time consuming and imprecise. 
       SUMMARY 
       [0003]    The present invention is directed toward a user interface for searching and playing recorded video, a network comprising a client device which runs the user interface, and a method for searching and playing recorded video using the user interface. The user interface comprises a filmstrip snapshots sequence, a selection window, a first input device, a second input device, and a third input device. The filmstrip snapshot sequence comprises a series of chronologically ordered snapshots of the recorded video, each snapshot having an associated video segment of the recorded video from which the snapshot was taken. Each of the snapshots is taken at a regular interval equal to the length of the video segments. The selection window highlights a set of at least two consecutive snapshots from the filmstrip snapshot sequence. The first input device advances the filmstrip snapshot sequence when activated, causing the selection window to highlight a chronologically later set of snapshots. The second input device zooms in on the filmstrip snapshot sequence as a function of time when activated, causing a new set of snapshots to be retrieved at a smaller regular interval. The third input device plays the video segments associated with the highlighted snapshots, in chronological order, when activated. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0004]      FIG. 1  is a block diagram of a video capture and replay network. 
           [0005]      FIG. 2  is a simulated screenshot of a graphical user interface used to search video in the video capture and replay network of  FIG. 1 . 
           [0006]      FIG. 3A and 3B  are timelines of video segments represented by the graphical user interface of  FIG. 2  at a low and a high zoom level, respectively. 
           [0007]      FIG. 4  is a flowchart of a method for searching video using the graphical user interface of  FIG. 2 . 
       
    
    
     DETAILED DESCRIPTION 
       [0008]      FIG. 1  is a block diagram of video capture and playback network  10 , comprising source  12 , recorder  14 , local server  16 , and client device  18  with interface device  20  and playback monitor  22 . Source  12  is a video source such as a digital camera. Although only one source  12  is shown, many video sources may be included in video capture and playback network  10 . Recorder  14  is a video recorder which encodes real time video from source  12 , and stores encoded video in a machine-readable format. In some embodiments source  12  and recorder  14  may communicate directly, while in other embodiments recorder may receive video from source  12  only through video capture and playback network  10 . Although only one recorder  12  is shown, many video recorders may be included in video capture and replay network  10 , potentially including multiple recorders which encode video from source  12 , as well as multiple recorders which encode video from other sources. Local server  16  is a video management server which may catalogue, retrieve, or process video from recorder  14  for playback at client device  18 . Alternatively, local server  16  may catalogue video from recorder  14  and provide configuration information enabling client device  18  to directly retrieve and play back video from recorder  14 . Client device  18  is a logic-capable user-side device such as a personal computer, through which a user may search, manipulate, or play back video from recorder  14 . Client device  18  includes at least one interface device  20  to allow user input, and at least one playback monitor  22  to display video from recorder  14 . Local server  16  and client device  18  are computers with processors and hardware memory, and may be either specialized hardware computers or general purpose computers running specialized software for video management and processing. In some embodiments, local server  16 , recorder  14 , and client device  18 , or some combination thereof, may be logically separable parts of a single hardware computer. 
         [0009]    Users at client device  18  can review video collected by source  12  and stored at recorder  14 . Client device  18  runs graphical user interface (GUI)  100  on local memory, as depicted and described below with respect to  FIG. 2 . GUI  100  facilitates rapidly and easily searching, retrieving, and playing back recorded video from the period of interest, as described below. 
         [0010]      FIG. 2  is a simulated screenshot of graphical user interface  100  for client device  18 . Graphical user interface (GUI)  100  features filmstrip panel  102  displaying filmstrip snapshot sequence  104  (including filmstrip snapshots  104   a,    104   b,    104   c,    104   d,    104   e,  and  104   f ), selection window  106  (enclosing first selected image  108  and second selected image  110 ), zoom-in input device  112 , zoom-out input device  114 , play input device  116 , forward input device  118 , and reverse input device  120 . Filmstrip panel  102  is a region of graphical user interface  100  devoted to source  12 , and displays filmstrip snapshot sequence  104 . Filmstrip snapshots  104   a - 104   f  are chronologically arranged images taken at regular time intervals from recorded video originated at source  12 , and stored at recorder  14 . Filmstrip snapshots  104   a - 104   f  are retrieved from recorder  14  by client device  18 , over video capture and playback network  10 . In some embodiments client device  18  retrieves filmstrip snapshots  104   a - 104   f  from recorder  14  without input from local server  16  (see  FIG. 1 ). In other embodiments, client device  18  requests filmstrip snapshots  104   a - 104   f  from video local server  16 , which may either retrieve and forward filmstrip snapshots  104   a - 104   f  to client device  18 , or provide instructions to client device  18  which enable client device  18  to retrieve filmstrip snapshots  104   a - 104   f  directly from recorder  14 . 
         [0011]    In some embodiments the input devices described herein are buttons activated by pressing or clicking on a pre-defined area. Such buttons may include zones on a touch screen, GUI regions which react to mouse clicks, or physical keys. In other embodiments these input devices are cursor movements or cursor swipes. Although buttons  112 ,  114 ,  116 ,  118 , and  120  are depicted as GUI buttons situated on filmstrip panel  102 , alternative embodiments may use other input means well known in the art, such as keyboard hotkeys or drop-down menus. The terms “input device” or “button” refer herein to any such mouse click, mouse swipe, touch screen zone, physical keyboard hotkey, drop-down menu, or other conventional input device. 
         [0012]    In the depicted embodiment, filmstrip snapshot sequence  104  is arranged such that earlier images appear to the left of later images, forming a filmstrip which extends in chronological order from left to right. Filmstrip snapshot sequence  104  may alternatively be positioned in other arrangements which preserve the order of filmstrip snapshots  104   a - 104   f , such as chronologically from top to bottom, or chronologically from right to left. Although filmstrip snapshot sequence  104  is shown in  FIG. 2  as forming a single row extending across filmstrip panel  102 , filmstrip snapshot sequence  104  may in some embodiments be arranged in multiple rows or columns. 
         [0013]    Filmstrip snapshots  104   a - 104   f  originate from source  12 . In some embodiments, further filmstrip panels containing filmstrip snapshot sequences associated with another source may be arranged adjacent to filmstrip panel  102 . In some embodiments, graphical user interface  100  may include a menu, button, drag-and-drop list, or other selection means (not shown) for controlling which source is represented in filmstrip panel  102 . 
         [0014]    First selection image  108  and second selection image  110  are adjacent images enclosed by selection window  106 . In  FIG. 2 , first selection image  108  is filmstrip snapshot  104   c,  while second selection image  110  is selection image  104   d.  Although only two selection images are shown in  FIG. 2 , some embodiments may enclose additional images within selection window  106 . Selection window  106  may be a frame surrounding selected images, a tint applied to selected or unselected images, or any other means of visually highlighting selected images. Each filmstrip snapshot  104 N (i.e.  104   a,    104   b,  . . . or  104   f ) corresponds to a video segment which begins with, ends with, or otherwise includes corresponding filmstrip snapshot  104 N, and has a duration equal to the interval between filmstrip snapshots  104 N. Where filmstrip snapshots  104   a - 104   f  are taken at one hour intervals from security camera footage, for instance, each filmstrip snapshot  104 N will be associated with an hour-long video segment. Client device  18  retrieves video segments from recorder  14  via video capture and playback network  10 . In some embodiments, client device  18  may retrieve video segments corresponding to each filmstrip snapshot  104   a - 104   f  when each filmstrip snapshot is retrieved; in such embodiments, filmstrip snapshots  104   a - 104   f  may be extracted from corresponding video segments by client device  18 . Alternatively, client device  18  may only retrieve video segments corresponding to filmstrip snapshots in selection window  106  (i.e. first selected image  108  and second selected image  110 ) when play input device  116  is pressed (as described below), thereby conserving bandwidth. As with filmstrip snapshots  104   a - 104   f,  video segments may be retrieved directly from recorder  14  without input from local server  16 , may be retrieved via local server  16 , or may be retrieved directly from recorder  14  using instructions provided by local server  16 . Play input device  116  plays back video segments associated with first selected image  108  and second selected image  110 , as is explained in further detail below with respect to  FIGS. 3A and 3B . 
         [0015]    Filmstrip snapshots  104  are drawn at regular intervals from recorded video stored on at least one recorder  14 . Each filmstrip snapshot  104  is separated from adjacent filmstrip snapshots by a time interval determined by a zoom level of filmstrip panel  102 , which may be adjusted with zoom-in input device  112  and zoom-out input device  114 . Pressing zoom-in input device  114  causes client device  18  to retrieve and display a new set filmstrip snapshots  104  separated by a shorter time interval. Conversely, pressing zoom-out input device  116  causes client device  18  to retrieve and display a new set of filmstrip snapshots  104  separated by a longer time interval. In some embodiments, filmstrip intervals at every zoom level are “even” or “neat” time periods, such as one hour, fifteen minute, or one minute. Graphical user interface  100  may support any number of zoom levels, although only two to five levels will be useful for most video searching applications. 
         [0016]    Forward input device  118  and reverse input device  120  allow a user to shift filmstrip snapshot sequence  104  as if spooling through a filmstrip. Activating forward input device  118  advances the sequence of filmstrip snapshots  104   a - 104   f  by one, such that filmstrip snapshot  104   d  becomes first selection image  108 , and filmstrip snapshot  104   e  becomes second selection image  110 . Analogously, pressing reverse input device  120  retreats the sequence of filmstrip snapshots  104   a - 104   f  by one, such that filmstrip snapshot  104   b  becomes first selection image  108 , and filmstrip snapshot  104   c  becomes second filmstrip snapshot  110 . In some embodiments, forward input device  118  and reverse input device  120  are mouse swipes, such that dragging or scrolling across filmstrip snapshot sequence  104  advances or retreats chronologically through filmstrip snapshot sequence  104 . Alternatively, filmstrip panel  102  may include separate mechanisms for advancing or retreating filmstrip snapshot sequence  104  incrementally or via a scan. In still other embodiments, forward input device  118  and reverse input device  120  may be scan buttons that cause filmstrip snapshot sequence  104  to advance or retreat automatically at a moderate rate until stopped. Some embodiments of graphical user interface  100  may provide more than one of these options, e.g. both an automatic advancement button and the capacity to advance and retreat filmstrip snapshot sequence  104  with a mouse swipe. 
         [0017]    Graphical user interface  100  can be used to play back recorded video, as described above, and to search recorded video, as described below with respect to  FIG. 4 . In addition the elements enumerated above, graphical user interface  100  may include such secondary elements as a camera information display (indicating which source  12  video comes from), a time indicator (indicating the timestamp for each filmstrip snapshot  104 N), and a quality monitor (indicating the encoded video frame rate and/or resolution). 
         [0018]      FIGS. 3A and 3B  are timelines advancing chronologically from left to right, depicting video segment sequences  200  and  300 , respectively. Video segment sequence  200  includes video segments vs 1 , vs 2 , vs 3 , and vs 4 , while video segment sequence  300  includes video segments vs 5 , vs 6 , vs 7 , vs 8 , vs 9 , and vs 10 . Each video segment vs 1 , vs 2 , . . . vs 10  correspond to some displayed or potential filmstrip snapshots  104 N described above with respect to  FIG. 2 .  FIG. 3A  depicts a first zoom level, while  FIG. 3B  depicts a second, higher zoom level; in particular,  FIG. 3B  depicts one possible timeline of video segments which could be obtained from the timeline of  FIG. 3A  by pressing zoom-in input device  112 . Each video segment vsN has a start time stN and an end time etN separated by a regular time interval T. All video segments in  FIG. 3A  have a duration defined by time interval T 1 , while all video segments in  FIG. 3B  have a duration defined by shorter time interval T 2 , representing in increase in zoom between  FIG. 3A  and  FIG. 3B . End time etN of each video segment vsN within video sequences  200  or  300  substantially matches start time stN+1 of subsequent video segment vsN+1. Slight variations in the length of each video segment may occur where time interval T is not a perfect multiple of a recording frame rate of video encoded by recorder  14 . 
         [0019]    Selection S 1  includes video segments vs 2  and vs 3 , which correspond to first selected image  108  and second selected image  110 , respectively. When a user presses play input device  116  (see  FIG. 2 , above), client device  18  plays back the entirety of selection S 1 , beginning at start time st 2  and ending at end time et 3 . For systems wherein selection window  106  encloses more than two filmstrip snapshots  104 , selection S 1  will correspondingly include more than two video segments, all of which will be played back, in order, when play input device  116  is pressed. Selection S 2  is a higher-zoom analogue of selection S 1 , and accordingly spans a shorter time. Selection S 2  includes video segments vs 7  and vs 8 , starts at start time st 7 , and ends at end time et 8 . In some embodiments, only the current selection (S 1  or S 2 ) will be played in when play input device  116  is perssed 
         [0020]    Video segment sequences  200  and  300  and filmstrip snapshot sequence  104  are centered on time t 0 . Accordingly, time t 0  represents the midpoint of both selection S 1  and selection S 2 , such that t 0 =et 2 =st 3 =et 7 =st 8  in the depicted embodiment. Pressing zoom-in input device  112  or zoom-out input device  114  causes user interface  100  to zoom in or out about time t 0 , such that time t 0  remains the midpoint time of the video sequence corresponding to post-zoom filmstrip snapshot sequence  104 . 
         [0021]    Graphical user interface  100  provides allows a user at client device  18  to easily recognize, select, and play a desired selection S by positioning appropriate filmstrip snapshots  104   a - 104   f  within selection window  106  using forward input device  118  and reverse input device  120 , and pressing play input device  116 . Graphical user interface  100  can also be used to search video for transition events, as described below with respect to  FIG. 4 . 
         [0022]      FIG. 4  is a flow chart of a method for locating and viewing transition events using graphical user interface  100 . Some events result in a lasting change to recorded video area, such that a first state before the event differs visibly from a second state after the event; these events are referred to herein as “transition” events. The theft of a painting or the breaking of a window, for instance, will result in lasting change to the environment, viz. the absence of the painting or window. Transition events can be recognized using graphical user interface  100  by identifying a difference between a before-state visible in an earlier filmstrip snapshot  104 N, and an after-state visible in a later filmstrip snapshot  104 M (where N&lt;M). (Step S 1 ). At a low zoom level corresponding to a long time interval T, a user can detect at a glance whether a transition event has occurred during the long time period corresponding to filmstrip snapshot sequence  104 . Similarly, a user can locate a known transition event by advancing through filmstrip snapshot sequence  104  with forward input device  118  and reverse input device  120 , until first selected image  108  differs from second selected image  110  in the expected way (e.g. a painting that is present in first selected image  108  is missing from second selected image  110 ). (Step S 2 ).Once the transition has been located within selection window  106 , a user determines whether the time span included in the selection window  106  is sufficiently short. (Step S 3 ). The smashing of a car windshield, for instance, might take place in a matter of seconds, making it inefficient for a user to play back an entire two hour selected video clip comprised of two selected one hour video segments. Accordingly, the user can zoom in as described above with respect to  FIGS. 2 ,  3 A, and  3 B, shortening time interval T to a more manageable value. (Step S 4 ). After shortening the time interval, the user can repeat this process, locating the transition event progressively more precisely in time (Step S 2 ), and continuing to zoom in (Step S 4 ) until selection window  106  encloses a sufficiently brief clip encompassing the transition event. Longer playback intervals may be appropriate for lengthier events. Once the selected video is sufficiently short, a user can press play input device  116  to play back the selected clip, as described above with respect to  FIG. 2 . (Step S 5 ). 
         [0023]    The present invention allows a user to quickly locate transition events in recorded video without playing through a large volume of irrelevant video. Once such a transition event has been located, a user can quickly and easily select an appropriate video clip for playback, and play that video clip. 
         [0024]    While the invention has been described with reference to an exemplary embodiment(s), it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment(s) disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.