A video multiplexing apparatus for multiplexing video data with first media data other than video comprising: a control unit which, when a decoding unit of the video data is a picture, the decoding unit being a unit for which a decoding process for the video data is performed, determines first decode/display time information representing a decode time and display time for each picture in the video data and for each decoding unit of the first media data when the decoding unit of the video data is the picture, while, when the decoding unit of the video data is a sub-picture obtained by dividing the picture into a plurality of sub-pictures, determines the first decode/display time information and second decode/display time information representing a decode time and display time for each picture in the video data and for each decoding unit of the first media data when the decoding unit of the video data is the sub-picture; a video packet generating unit which divides encoded data of each picture in the video data into a plurality of video packets, and appends the first decode/display time information for the picture to a video packet that carries encoded data of the picture when the decoding unit of the video data is the picture, while appending the first decode/display time information and second decode/display time information for the picture to a video packet that carries encoded data of the picture when the decoding unit of the video data is the sub-picture; a media packet generating unit which appends the first decode/display time information for the decoding unit to a media packet that carries encoded data of the first media data when the decoding unit of the video data is the picture, while appending the first decode/display time information and second decode/display time information for the decoding unit to the media packet when the decoding unit of the video data is the sub-picture; a system packet generating unit which generates a transport stream packet which includes program specific information and information indicating whether the decoding unit of the video data is the picture or sub-picture; and a packet multiplexing unit which generates a data stream by multiplexing the video packet, the media packet and the transport stream packet and outputs the data stream, wherein the first decode/display time information includes information indicating a time at which to display the video data and the first media data in synchronized fashion when the decoding unit of the video data is the picture, the second decode/display time information includes information indicating a time at which to display the video data and the first media data in synchronized fashion when the decoding unit of the video data is the sub-picture, and the decode time and display time of the picture determined based on the second decode/display time information are earlier than the decode time and display time of the picture determined based on the first decode/display time information. a control unit which, when a decoding unit of the video data is a picture, the decoding unit being a unit for which a decoding process for the video data is performed, determines first decode/display time information representing a decode time and display time for each picture in the video data and for each decoding unit of the first media data when the decoding unit of the video data is the picture, while, when the decoding unit of the video data is a sub-picture obtained by dividing the picture into a plurality of sub-pictures, determines the first decode/display time information and second decode/display time information representing a decode time and display time for each picture in the video data and for each decoding unit of the first media data when the decoding unit of the video data is the sub-picture; a video packet generating unit which divides encoded data of each picture in the video data into a plurality of video packets, and appends the first decode/display time information for the picture to a video packet that carries encoded data of the picture when the decoding unit of the video data is the picture, while appending the first decode/display time information and second decode/display time information for the picture to a video packet that carries encoded data of the picture when the decoding unit of the video data is the sub-picture; a media packet generating unit which appends the first decode/display time information for the decoding unit to a media packet that carries encoded data of the first media data when the decoding unit of the video data is the picture, while appending the first decode/display time information and second decode/display time information for the decoding unit to the media packet when the decoding unit of the video data is the sub-picture; a system packet generating unit which generates a transport stream packet which includes program specific information and information indicating whether the decoding unit of the video data is the picture or sub-picture; and a packet multiplexing unit which generates a data stream by multiplexing the video packet, the media packet and the transport stream packet and outputs the data stream, wherein