Information processing apparatus and non-transitory computer readable medium storing program

An information processing apparatus includes a reception section that receives an editing instruction with respect to an arrangement configuration of a still image including plural pages, and a generation section that generates a moving image which corresponds to the arrangement configuration of the still image edited due to the editing instruction and which is acquired through combination with a moving image including playback sections corresponding to the still image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2018-166231 filed Sep. 5, 2018.

BACKGROUND

(i) Technical Field

The present invention relates to an information processing apparatus and a non-transitory computer readable medium storing a program.

(ii) Related Art

JP4292902B includes a content editing device that performs processing of receiving designation of a part, which is removed, of video data, determining material image data associated with the part, which is designated to be removed, of the video data, and a time location of addition image data which is associated with the material image, recoupling the video data after removing the part, which is designated to be removed, of the video data, and updating the material image data corresponding to the part, which is removed, and the associated time location of the addition image data in response to the part, which is removed, thereby editing content which includes the video data and the material image data associated with a time location of the video data while maintaining a reasonable relation in a case where a correspondence relation therebetween is read as the content.

SUMMARY

There is a case where a still image, which is described in a moving image, or a still image, which is used to describe the moving image, is associated with the moving image. Here, in a case where the still image is switched over or removed, it may be desired to edit the moving image in synchronization with the switching over or the removal of the still image.

Aspects of non-limiting embodiments of the present disclosure relate to an information processing apparatus which is capable of acquiring the moving image in synchronization with editing which is performed with respect to an arrangement configuration of the still image including a plurality of pages.

According to an aspect of the present disclosure, there is provided an information processing apparatus including a reception section that receives an editing instruction with respect to an arrangement configuration of a still image including a plurality of pages, and a generation section that generates a moving image which corresponds to the arrangement configuration of the still image edited due to the editing instruction and which is acquired through combination with a moving image including playback sections corresponding to the still image.

DETAILED DESCRIPTION

FIG. 1is a system diagram illustrating a configuration of a multimedia content editing system according to the exemplary embodiment of the present invention. As illustrated inFIG. 1, the multimedia content editing system according to the exemplary embodiment of the present invention is configured with an editing processing server10, a terminal device20, such as a personal computer, which are connected to each other through a network30, and a video camera40.

In a case where video data, which is acquired by recording a conference, a lecture, and a briefing session (in the exemplary embodiment, these are referred to as presentations) that are performed by exhibiting material images, such as the conference, the lecture, the briefing session, and presentations, is delivered to audiences as content, the multimedia content editing system according to the exemplary embodiment generates and edits multimedia content which is acquired by combining various types of content, such as a moving image, a still image, a sound, and text. Specifically, according to the multimedia content editing system of the exemplary embodiment, in a case where a presentation moving image, in which a still image including a plurality of pages is displayed on a monitor or a screen as a presentation material, is captured and editing is separately performed with respect to an arrangement configuration of the still image which is the presentation material, it is possible to generate a moving image which is synchronized with the editing, that is, which corresponds to the arrangement configuration of the still image.

The editing processing server10is an information processing apparatus installed with editing software used to generate the moving image synchronized with an arrangement configuration of the pages of the still image. Furthermore, the terminal device20acquires the moving image (moving image for a presentation) captured by the video camera40, and outputs the moving image to the editing processing server10. Meanwhile, it is possible to directly install the editing software in the terminal device20, such as the personal computer, without installing the editing software in the editing processing server10.

Subsequently,FIG. 2illustrates a hardware configuration of the editing processing server10in an image forming system according to the exemplary embodiment. As illustrated inFIG. 2, the editing processing server10includes a CPU111, a memory112, a storage device113, such as a Hard Disk Drive (HDD), a communication interface114that transmits and receives data to and from an external device, such as the terminal device20, or the like through the network30, and a user interface115that includes a touch panel, a liquid crystal display, and a keyboard. The components are connected to each other through a control bus116.

The CPU111controls an operation of the editing processing server10by executing prescribed processing based on a control program which is stored in the memory112or the storage device113. Meanwhile, in the exemplary embodiment, description is performed such that the CPU111reads and executes the control program which is stored in the memory112or the storage device113. However, it is possible to store the program in a storage medium, such as a CD-ROM, and to provide the program to the CPU111.

FIG. 3is a block diagram illustrating a functional configuration of the editing processing server10which is realized in such a way that the control program is executed. As illustrated inFIG. 3, the editing processing server10according to the exemplary embodiment includes a data communication unit120, a control unit130, and a multimedia data storage unit140.

The data communication unit120performs data communication with the terminal device20through the network30.

The control unit130operates as a moving image reception unit131, an association unit132, an editing instruction reception unit133, a still image editing unit134, a moving image generation unit135, and a display control unit136by controlling the operation of the editing processing server10.

The multimedia data storage unit140stores moving image data and still image data on which it is desired to perform editing processing.

The moving image reception unit131receives the moving image relevant to the still image from the terminal device20. Specifically, although being described later, a file of the moving image designated by the user is acquired from the terminal device20according to an instruction of the user.

The association unit132associates the respective pages included in the still image with respective playback sections included in the moving image which is received by the moving image reception unit131. Specifically, in a case where images of specific areas designated by the user in the moving image coincide with images of the respective pages of the still image, the association unit132associates the respective pages of the still image with the respective playback sections included in the moving image. Meanwhile, here, it is assumed that the playback sections correspond to one playback period of the moving image in a period in which the images of the specific area included in the moving image do not change.

The editing instruction reception unit133receives an editing instruction with respect to the arrangement configuration of the still image including the plurality of pages from the user. Specifically, the editing instruction reception unit133receives a removal instruction with respect to the still image in page units, a duplication instruction to perform duplication and addition with respect to the still image in page units, and a sorting instruction to sort display order with respect to the still image in page units.

The still image editing unit134edits the arrangement configuration of the pages of the still image according to the editing instruction which is received by the editing instruction reception unit133. The editing includes modification with respect to the arrangement configuration of the pages of the still image, that is, removal of the still image in page units, duplication in page units, and sorting in page units.

The moving image generation unit135generates the moving image which corresponds to the arrangement configuration of the still image that is edited by the still image editing unit134according to the editing instruction and which is acquired through combination with the moving image that includes the playback sections corresponding to the still image. Specifically, in a case where the arrangement configuration of the still image is edited, that is, modified due to the editing instruction, the moving image generation unit135generates the moving image corresponding to the arrangement of the edited still image by performing editing, that is, modification such that the arrangement configuration of the respective playback sections of the moving image corresponds to the arrangement configuration of the still image.

For example, in a case where a certain page of the still image is removed due to the removal instruction, the moving image generation unit135generates a moving image that does not include a playback section corresponding to the page, which is removed due to the removal instruction, of the still image, or removes the playback section corresponding to the page, which is removed due to the removal instruction, of the still image, from the moving image. In this case, the moving image generation unit135may add special effects to joint parts of front and back playback sections of the removed playback section of the moving image.

In a case where the certain page of the still image is duplicated and added due to the duplication instruction, the moving image generation unit135duplicates a playback section corresponding to the page, which is duplicated due to the duplication instruction, of the still image and adds the duplicated playback section to a designated location of the moving image.

In addition, in a case where the display order of the plurality of pages of the still image are sorted due to the sorting instruction, the moving image generation unit135sorts the order of the playback sections of the moving image so as to be playback order according to the display order of the still image which is sorted due to the sorting instruction.

Furthermore, in a case where a correspondence relation between the pages of the still image and the playback sections of the moving image is listed in time series and any of portions, in which modification is desired to be performed with respect to the moving image, is designated in the listed image, the moving image generation unit135restores the modified playback sections of the moving image to a state before the modification.

The display control unit136generates a moving image reception screen used to designate an editing target moving image, a still image editing instruction reception screen used to receive the editing instruction with respect to the still image, and a screen used to list the correspondence relation between the pages of the still image and the playback sections of the moving image in time series, and displays the screens on the user interface115.

Subsequently, the operation of the editing processing server10in the multimedia content editing system according to the exemplary embodiment will be described in detail with reference toFIGS. 4 to 10. Meanwhile,FIG. 4is a flowchart illustrating a flow of editing processing in the editing processing server10according to the exemplary embodiment of the present invention. Meanwhile, in an example below, it is assumed that the moving image for the presentation is captured in advance by the video camera40, data of the captured moving image is preserved as a moving image file by the terminal device20, and a state is made in which acquisition by the editing processing server10is possible in a case where the instruction from the user exists. Meanwhile, it is assumed that the moving image for the presentation is captured by the video camera40which is fixed using a tripod or the like, and the presentation material, which is acquired by displaying the still image including the plurality of pages that will be described later on the screen or the monitor, is reflected in the captured moving image.

First, in step S401, the editing instruction reception unit133of the editing processing server10receives the editing processing with respect to the still image. For example, as illustrated inFIG. 5A, the display control unit136generates an editing screen500, in which a list of the respective pages of the still image including the plurality of pages is displayed using thumbnails, and displays the editing screen500on the user interface115. For example, in the editing screen ofFIG. 5A, first to twenty pages of the still image including the plurality of pages are sequentially displayed using the thumbnails.

The user manipulates the user interface115while visually recognizing a displayed still image list, and performs editing on the arrangement configuration of the pages of the still image. For example, the removal instruction to remove a page which is considered to be not necessary in page units is input.FIG. 5Aillustrates a state in which an instruction to remove fifth, seventh to ninth, fourteenth, and fifteenth is input by the user. In step S402, the still image editing unit134removes a removal target page from the still image according to the removal instruction. At the same time, the display control unit136removes the page to which the removal instruction is provided from the editing screen500which is displayed using the thumbnails, generates a screen510, which is acquired after editing is performed and in which the list of the respective pages of the still image acquired after the editing is displayed using the thumbnails as illustrated inFIG. 5B, and displays the generated screen on the user interface115.FIG. 5Billustrates a state in which the initial fifth, seventh to ninth, fourteenth, and fifteenth pages are removed and subsequent pages sequentially exist subsequent to the pages which are not removed.

In step S403, the moving image reception unit131receives the editing target moving image relevant to the still image on which the editing is performed. Specifically, the display control unit136generates a moving image file acquisition screen600as illustrated inFIG. 6, and displays the moving image file acquisition screen600on the user interface115. The moving image file acquisition screen600is configured with an explanation610“An editing target moving image file is selected.”, a selected file name display field620, a folder display button630, a selected file information display field640, and an execution button650, and a cancellation button660.

The user manipulates the user interface115while visually recognizing the displayed moving image file acquisition screen600, and designates the editing target moving image file. Specifically, the user displays a folder of the terminal device20or the editing processing server10, in which the editing target moving image is preserved, by pressing the folder display button630, and designates the editing target moving image file from a moving image file list which is preserved in the folder. In the selected file information display field640, detailed information relevant to the editing target moving image file which is selected at time, for example, moving image playback time, resolution, a writer, a preparation date and time of the moving image file, and the like are displayed. In addition, one frame, which is included in the moving image of the moving image file, may be displayed using the thumbnails.

In a case where the user visually recognizes the selected file information display field640and selects the execution button650, the moving image file is designated as the editing target moving image file. In a case where the user selects the cancellation button660, the processing is stopped and ends.

In a case where the editing target moving image file is designated, the association unit132instructs the display control unit136to display one frame, which is included in the moving image of the designated moving image file, in step S404. The display control unit136generates an area designation reception screen700, which includes one frame of the moving image of the designated moving image file and receives designation of an area of the presentation material on the moving image, as illustrated inFIG. 7, and displays the area designation reception screen700on the user interface115.

The area designation reception screen illustrated inFIG. 7is configured with an explanation710“designation of a presentation material display location” and “designate the presentation material location, please”, a moving image display part720, an execution button740, and a cancellation button750. In the moving image display part720, a presentation material730included in the moving image is displayed. Meanwhile, the presentation material indicates a state in which the still image is displayed on the screen or the monitor.

The user designates an area of the presentation material730in the moving image display part720by manipulating the user interface115. For example, in a case where the user specifies a rectangular area by designating an upper left corner and a lower right corner (or an upper right corner and a lower left corner) of the area of the presentation material, the association unit132stores coordinate information of the rectangular area in the memory112. Meanwhile, the association unit132may automatically specify the area of the presentation material730.

In step S405, the association unit132specifies a portion, in which an image of the area of the presentation material designated by the user significantly changes, as timing at which the presentation material is switched over. In addition, the association unit132specifies sections, in which the image of the area of the presentation material does not change, as respective playback sections included in the moving image, generates representative images for the respective playback sections, and generates an image list in which the representative images are sequentially arranged. A conceptual diagram at this time is illustrated inFIG. 8.

FIG. 8is a diagram illustrating a concept that the moving image is divided into the plurality of playback sections according to a change in an image of the presentation material. It is assumed that an image list810includes three representative images811,812, and813. The respective representative images811,812, and813correspond to playback sections821,822, and823in the moving image, respectively. That is, in the playback section821of the moving image, a presentation material corresponding to the representative image811is displayed in the area of the presentation material included in the moving image, and the image of the presentation material is switched over to an image corresponding to the representative image812at time t1. Similarly, in the playback section822of the moving image, a presentation material corresponding to the representative image812is displayed in the area of the presentation material included in the moving image, and the image of the presentation material is switched over to an image corresponding to the representative image813at time t2.

In step S406, the association unit132compares the arrangement configuration of the still image, which is edited in steps S401and S402, with the image list of the representative images generated based on the moving image. Furthermore, in a case where the representative images generated based on the moving image coincide with the images of the respective pages of the still image, the respective pages of the still image are associated with respective playback sections specified using the representative images generated based on the moving image, and the respective pages of the still image and the respective playback sections are stored in the memory202. In contrast, in a case where there is an image, which does not correspond to the page of the still image edited in steps S401and S402, in the image list generated based on the moving image, the moving image generation unit135removes the playback section, which corresponds to the representative image, of the moving image from the moving image. Furthermore, a subsequent part of the removed playback section is connected to a last part of a front playback section of the removed part such that a blank does not occur in the removed part. Therefore, the moving image, which does not include the playback section corresponding to the page, which is removed due to the removal instruction, of the still image, is generated.

Meanwhile, a conceptual diagram illustrating the case will be described with reference toFIG. 9.FIG. 9is a conceptual diagram illustrating a case where a specific playback section of the moving image is removed in response to the editing performed with respect to the arrangement configuration of the still image including the plurality of pages. InFIG. 9, it is assumed that the moving image includes three playback sections901,902, and903expressed by representative images911,912, and913, respectively. The edited still image corresponding to the playback sections901,902, and903includes a first page921corresponding to the representative image911, a second page923corresponding to the representative image913, and a third page924corresponding to another representative image which is not illustrated in the drawing. A page, which corresponds to the representative image912, of the still image is removed. Therefore, the moving image is generated in which the second playback section902corresponding to the representative image912is removed by the moving image generation unit135and the third playback section903is continued immediately after the first playback section901.

Meanwhile, the moving image generation unit135may insert the special effects between the first playback section901and the third playback section903. Any of dissolve, wipe out/in, and fade out/in is used as the special effects. The special effects are performed such that a transition part between the first playback section901and the third playback section903does not become unnatural.

Subsequently, in step S407, the display control unit136generates a moving image editing screen1000used to list the correspondence relation between the pages of the still image and the playback sections of the moving image in time series as illustrated inFIG. 10, displays the moving image editing screen1000on the user interface115, and ends the processing.

The moving image editing screen1000includes a preview field1010, a time line1020of the moving image acquired before the editing is performed, a time line1030of the moving image acquired after the editing is performed, and a time line1040of the still image. The preview field1010is used to selectively reproduce any of the moving image acquired before the editing is performed, the moving image acquired after the editing is performed, and the edited still image based on selection performed by the user. Horizontal axes of the time line1020of the moving image acquired before the editing is performed, the time line1030of the moving image acquired after the editing is performed, and the time line1040of the still image indicate time (elapse of time). In the time line1020of the moving image acquired before the editing is performed, the moving image acquired before the editing is performed is arranged and displayed for the respective playback sections. In the time line1030of the moving image acquired after the editing is performed, the moving image acquired after the editing is performed is arranged and displayed for the respective playback sections.

Meanwhile, as being understood in a case where the time line1020of the moving image acquired before the editing is compared with the time line1030of the moving image acquired after the editing is performed, the removed parts are displayed as blanks in the time line1030of the moving image acquired after the editing is performed but the removed parts are displayed using different colors through grayout (displayed using hashing in an opposite direction) in the time line1020of the moving image acquired before the editing is performed. That is, in a case where the editing is performed with respect to the playback sections of the moving image, the edited playback sections are displayed using a different display method from other playback sections in the listed images. In addition, in a case where any of the playback sections of the moving image is designated in the listed image, the display control unit136generates a representative image corresponding to the designated playback section and displays the representative image in a form of pop out1050.

In the time line1040of the still image, display time of the respective pages, which are included in the still image acquired after the editing, is listed in time series. Meanwhile, parts, which correspond to the still image removed through the editing, are displayed as blanks.

Meanwhile, in a case where any of portions of modification performed with respect to the moving image is selected and designated in the listed image (time line1020of the moving image acquired before the editing is performed), the moving image generation unit135may restore the modified playback section of the moving image to a state acquired before the modification is performed. Specifically, in a case where a grayout part of the time line1020of the moving image acquired before the editing is performed, that is, a part corresponding to the removed part in the moving image acquired after the editing is performed is designated, and drag and drop is performed at a blank part of the time line1030of the moving image acquired after the editing is performed, the moving image generation unit135may restore to the state acquired before the modification is performed by inserting the playback section of the designated part into a relevant location of the moving image acquired after the editing is performed.

Meanwhile, in the above-described example, a case is described where the arrangement configuration of the respective pages of the still image is previously edited, the moving image relevant to the still image is subsequently received, and the arrangement configuration of the respective playback sections of the moving image is edited so as to correspond to the arrangement configuration of the still image. However, the exemplary embodiment of the present invention is not limited to the example. In a case where the plurality of playback sections included in the moving image are previously associated with the respective pages of the still image and the arrangement configuration of the still image is edited due to the editing instruction from the user, the arrangement configuration of the respective playback sections of the moving image is edited in response to the editing, and the moving image corresponding to the edited arrangement configuration of the still image may be generated. Meanwhile, the association may be performed in the editing processing server10or may be performed in the terminal device20.

In addition, in the examples described with reference toFIGS. 4 to 10, the editing performed with respect to the arrangement configuration of the pages included in the still image is removal of the certain page included in the still image. However, the exemplary embodiment of the present invention is not limited to the removal, and the editing performed with respect to the arrangement configuration of the pages included in the still image may include addition through duplication in page units and sorting of the display order in page units, as will be described later.

For example, it is assumed that there is a still image which includes first to fifteenth pages with which the playback sections of the moving image are associated, respectively, as illustrated inFIG. 11A. It is assumed that the editing instruction reception unit133receives a removal instruction to remove the third page and the sixth page of the still image. The still image editing unit134removes the third page and the sixth page, which are targets of the removal instruction, from the still image. As illustrated inFIG. 11B, the moving image generation unit135removes playback sections, which correspond to the third page and the sixth page of the still image, of the moving image, that is, a third playback section and a sixth playback section, respectively, and connects playback sections, which are subsequent to the third playback section and the sixth playback section, to immediate before playback sections.

As a result, the moving image that does not include the playback sections corresponding to the third page and the sixth page, which are removed due to the removal instruction, of the still image, that is, the moving image, in which the playback sections are arranged in order of the playback sections1-2-4-5-7-8-9. . . , is generated.

In addition, as another example, it is assumed that there is a still image which includes first to twelfth pages with which the playback sections of the moving image are previously associated, respectively, as illustrated inFIG. 12A. It is assumed that the editing instruction reception unit133receives the duplication instruction to duplicate the fifth page, the ninth page, and the eleventh page of the still image and inserts the pages to a head of the arrangement configuration of the pages of the still image. If so, the still image editing unit134duplicates the fifth page, the ninth page, and the eleventh page, which are targets of the duplication instruction, of the still image, and inserts the pages to the head of the arrangement configuration of the pages of the still image. As illustrated inFIG. 12B, the moving image generation unit135duplicates the playback sections, which correspond to the fifth page, the ninth page, and the eleventh page of the still image, of the moving images, that is, a fifth playback section, a ninth playback section, and an eleventh playback section, and inserts the playback sections to a head of the moving image.

As a result, the moving image in which the playback sections that corresponds to the fifth page, the ninth page, and the eleventh page, which are duplicated due to the duplication instruction, of the still image are added to a head part of the moving image acquired before the editing is performed, that is, the moving image, in which the playback sections are arranged in order of the playback sections5-9-11-1-2-3-4-5-6-7-8-9-10-11-12, is generated.

In addition, as another example, it is assumed that there is a still image which includes first to fifteenth pages with which the playback sections of the moving image are previously associated, respectively, as illustrated inFIG. 13A. It is assumed that the editing instruction reception unit133receives the sorting instruction to move the thirteenth, fourteenth, and fifteenth pages of the still image without duplication, and inserts the pages to the head of the arrangement configuration of the pages of the still image. If so, the still image editing unit134executes sorting in which the thirteenth, fourteenth, and fifteenth pages, which are targets of the sorting instruction, of the still image are moved without duplication and the pages are inserted to the head of the arrangement configuration of the pages of the still image. As illustrated inFIG. 13B, the moving image generation unit135moves the playback sections, which correspond to the thirteenth, fourteenth, and fifteenth pages of the still image, of the moving image, that is, the thirteenth, fourteenth, and the fifteenth playback sections to the head of the moving image.

As a result, the moving image in which the playback sections that corresponds to the thirteenth, fourteenth, and fifteenth pages, which are moved due to the sorting instruction, of the still image are moved to the head part of the moving image acquired before the editing is performed, that is, the moving image, in which the playback sections are arranged in order of the playback sections13-14-15-1-2-3-4- . . . (omission) . . . -12, is generated.

Meanwhile, in the exemplary embodiment, the generated and edited moving image is preserved in the editing processing server10, and becomes a state which can be visibly recognized by a plurality of users who use the terminal devices (not illustrated in the drawing) connected to the network30or a state which can be delivered to the plurality of users.

Meanwhile, in a case where secret information or individual information is included in a certain page of the still image and the page of the still image is removed, a playback section, in which the secret information and the individual information are included, may be removed from the generated and edited moving image.