Dividir (scene) para conquistar!
Tags: ffmpeg
Published: 2021-11-28 15:08:58

Voltei rs! 

No último post, falamos sobre alguns conceitos do FFmpeg, hoje aplicaremos em algumas situações. Neste mundo, ninguém gosta mais de 'CPU' do que encoding de vídeo, isso é fato.  

Sabendo disso, hoje realizaremos o processamento da mídia "full" e outro exemplo realizando o split da mídia com "scenes", utilizando um recurso da fantástica ferramenta **PySceneDetect**.

Antes de começar, só lembrando que existe 'n' formas de chegar no mesmo objetivo, como estou aprofundando na ferramenta **PySceneDetect**, utilizarei um recurso da mesmo. 

Partindo do pressuposto que já tenha instalado ffmpeg, mediainfo e PySceneDetect, vamos iniciar! 


Detalhes do vídeo que utilizaremos no lab:
```
bash-3.2$ mediainfo all.mp4 
General
Complete name                            : all.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 15.1 MiB
Duration                                 : 1 min 22 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 533 kb/s
Encoded date                             : UTC 2016-02-03 04:12:42
Tagged date                              : UTC 2016-02-03 04:12:42
gsst                                     : 0
gstd                                     : 82638

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.1
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 22 s
Bit rate                                 : 1 339 kb/s
Maximum bit rate                         : 2 751 kb/s
Width                                    : 1 280 pixels
Height                                   : 544 pixels
Display aspect ratio                     : 2.35:1
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.080
Stream size                              : 13.2 MiB (87%)
Tagged date                              : UTC 2016-02-03 04:12:42
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 min 22 s
Bit rate mode                            : Variable
Bit rate                                 : 192 kb/s
Maximum bit rate                         : 203 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 1.89 MiB (13%)
Title                                    : IsoMedia File Produced by Google, 5-11-2011
Encoded date                             : UTC 2016-02-03 04:12:42
Tagged date                              : UTC 2016-02-03 04:12:42
``` 

Alguns campos que podemos comparar durante o laboratório: *File size,Duration,Encoded date,Format,Codec ID/Info,Bit rate,Maximum bit rate,Width e Height,Frame rate,Stream size*.

Antes de realizar o 'Splitting' do vídeo realizaremos o Encoding com a mesma configuração que será aplicada nos próximos passos, assim podemos comparar o mediainfo de ambos: 
```
ffmpeg -i all.mp4 -threads 0 -pass 1 -s 640x360 -preset faster -profile:v high -level:v 3.0 -vcodec libx264 -r 29.97 -pix_fmt yuv420p -x264opts no-mbtree:bframes=1:keyint=30:min-keyint=30:scenecut=-1  -b:v 756k -bt 226k -flags +psnr -y -f mp4  output_no_split.mp4
```
Agora o mediainfo do output_no_split.mp4:
```
bash-3.2$ mediainfo output_no_split.mp4
General
Complete name                            : output_no_split.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 9.04 MiB
Duration                                 : 1 min 22 s
Overall bit rate                         : 918 kb/s
Writing application                      : Lavf58.76.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L3
Format settings                          : CABAC / 2 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 2 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 22 s
Bit rate                                 : 756 kb/s
Maximum bit rate                         : 781 kb/s
Width                                    : 640 pixels
Height                                   : 360 pixels
Display aspect ratio                     : 2.35:1
Original display aspect ratio            : 2.35:1
Frame rate mode                          : Constant
Frame rate                               : 29.970 (29970/1000) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.109
Stream size                              : 7.68 MiB (85%)
Writing library                          : x264 core 163 r3060 5db6aa6
Encoding settings                        : cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x1:0 / me=dia / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=11 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=1 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=30 / keyint_min=16 / scenecut=0 / intra_refresh=0 / rc=abr / mbtree=0 / bitrate=756 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 min 22 s
Source duration                          : 1 min 22 s
Bit rate mode                            : Constant
Bit rate                                 : 129 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 1.27 MiB (14%)
Source stream size                       : 1.27 MiB (14%)
Title                                    : IsoMedia File Produced by Google, 5-11-2011
Default                                  : Yes
Alternate group                          : 1
mdhd_Duration                            : 82593
```

O primeiro passo é adicionar timecode a mídia, assim podemos ter um 'controle' ao manipular o vídeo em algum ‘player’ e verificar se este processo 'comeu algum frame rs'.Podemos fazer da seguinte forma: 
```
ffmpeg -i all.mp4 -vf "drawtext=timecode='00\:00\:00\:00':rate=25:x=10:y=10" split.mp4
```

Agora realizaremos o 'Splitting' no vídeo utilizando PySceneDetect:
```
scenedetect -i split.mp4 detect-content  -t 27 list-scenes split-video -o output
```
Podem reparar que no final do comando passamos a pasta 'output' como saída para os arquivos gerados. Agora realizaremos o mesmo procedimento de Encoding realizado com o vídeo antes do Splitting e comparar o tempo de processamento/qualidade da mídia gerada. Comando: 
```
cd output && ls | grep split |sed  's/^/file '/ > files.txt && ffmpeg -f concat -safe 0 -i files.txt -threads 0 -pass 1 -s 640x360 -preset faster -profile:v high -level:v 3.0 -vcodec libx264 -r 29.97 -pix_fmt yuv420p -x264opts no-mbtree:bframes=1:keyint=30:min-keyint=30:scenecut=-1  -b:v 756k -bt 226k -flags +psnr -y -f mp4  output.mp4
```
Geramos o arquivo output.mp4, agora podemos comparar o mediainfo de ambos os arquivos: 
```
bash-3.2$ mediainfo output.mp4 
General
Complete name                            : output.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 9.08 MiB
Duration                                 : 1 min 22 s
Overall bit rate                         : 921 kb/s
Writing application                      : Lavf58.76.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L3
Format settings                          : CABAC / 2 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 2 frames
Format settings, GOP                     : M=2, N=30
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 22 s
Bit rate                                 : 756 kb/s
Maximum bit rate                         : 783 kb/s
Width                                    : 640 pixels
Height                                   : 360 pixels
Display aspect ratio                     : 2.35:1
Original display aspect ratio            : 2.35:1
Frame rate mode                          : Constant
Frame rate                               : 29.970 (29970/1000) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.109
Stream size                              : 7.71 MiB (85%)
Writing library                          : x264 core 163 r3060 5db6aa6
Encoding settings                        : cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x1:0 / me=dia / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=11 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=1 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=30 / keyint_min=16 / scenecut=0 / intra_refresh=0 / rc=abr / mbtree=0 / bitrate=756 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 1 min 22 s
Bit rate mode                            : Constant
Bit rate                                 : 129 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 1.27 MiB (14%)
Title                                    : IsoMedia File Produced by Google, 5-11-2011
Default                                  : Yes
Alternate group                          : 1
```

Porque realizar o Splitting? A ídeia é utilizar o Jobs do Kubernetes para processar cada requisição de Encoding. Se eu tenho 'n' vídeos com tamanho reduzido, teoricamente consigo aproveitar do *Pod Lifecycle* de uma forma mais conciente(Pods Kubernetes são efêmeros). 

E para você, neste contexto, faz sentido dividir para conquistar? 


Sobre [PySceneDetect](https://pyscenedetect.readthedocs.io/projects/Manual/en/latest/index.html#)
Sobre [Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/)
Sobre [Pod Lifecycle](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/) 
Um pouco mais sobre [Timecode](http://andrewduncan.net/timecodes/)

*Tudo aqui é teoria e/ou experimentação