This will most often be wrong. Note that pts is not in decode order. B frames will (and must come first). Don't try to set duration from anything but the demuxer packet.