An information processing apparatus configured to generate metadata corresponding to content comprising: a data output section configured to output reproduced content; a data input section configured to capture at least one of image data and voice data of a user viewing the reproduced content; a control section configured to determine interaction data for prompting said user on the basis of a user status predicted from a result of processing of the at least one of image data and voice data of said user, captured by said data input section, and on the basis of information extracted from the reproduced content; an interaction execution section configured to execute an interaction with said user viewing the reproduced content on the basis of said interaction data determined by said control section; a metadata generation section configured to analyze at least one of a captured user's image and a captured user's voice including a result of said interaction, thereby generating interaction-based metadata corresponding to said reproduced content; and to generate content-based metadata based on image data and voice data forming the reproduced content by matching one of image data and voice data forming the reproduced content with data registered in a respective identification dictionary or recognition dictionary, said matching including selecting an applicable identification dictionary or recognition dictionary on the basis of the information obtained from said user as a result of said interaction; and a content control section configured to record said interaction-based metadata and said content-based metadata as metadata that corresponds to said reproduced content, wherein the metadata generation section is configured to switch between generation of said interaction-based metadata and said content-based metadata; and the metadata generation section is configured to use the same technique of at least one of identification and recognition processing operations of face identification processing, general-object identification processing, smiling face identification processing, speaker identification processing, voice recognition processing, laughing voice identification processing, screaming voice identification processing, and crying voice identification processing, for generation of said interaction-based metadata and said content-based metadata to extract said interaction-based metadata and said content-based metadata on the basis of these identification and recognition processing operations. a data output section configured to output reproduced content; a data input section configured to capture at least one of image data and voice data of a user viewing the reproduced content; a control section configured to determine interaction data for prompting said user on the basis of a user status predicted from a result of processing of the at least one of image data and voice data of said user, captured by said data input section, and on the basis of information extracted from the reproduced content; an interaction execution section configured to execute an interaction with said user viewing the reproduced content on the basis of said interaction data determined by said control section; a metadata generation section configured to analyze at least one of a captured user's image and a captured user's voice including a result of said interaction, thereby generating interaction-based metadata corresponding to said reproduced content; and to generate content-based metadata based on image data and voice data forming the reproduced content by matching one of image data and voice data forming the reproduced content with data registered in a respective identification dictionary or recognition dictionary, said matching including selecting an applicable identification dictionary or recognition dictionary on the basis of the information obtained from said user as a result of said interaction; and to analyze at least one of a captured user's image and a captured user's voice including a result of said interaction, thereby generating interaction-based metadata corresponding to said reproduced content; and to generate content-based metadata based on image data and voice data forming the reproduced content by matching one of image data and voice data forming the reproduced content with data registered in a respective identification dictionary or recognition dictionary, said matching including selecting an applicable identification dictionary or recognition dictionary on the basis of the information obtained from said user as a result of said interaction; and a content control section configured to record said interaction-based metadata and said content-based metadata as metadata that corresponds to said reproduced content, wherein the metadata generation section is configured to switch between generation of said interaction-based metadata and said content-based metadata; and the metadata generation section is configured to use the same technique of at least one of identification and recognition processing operations of face identification processing, general-object identification processing, smiling face identification processing, speaker identification processing, voice recognition processing, laughing voice identification processing, screaming voice identification processing, and crying voice identification processing, for generation of said interaction-based metadata and said content-based metadata to extract said interaction-based metadata and said content-based metadata on the basis of these identification and recognition processing operations.