Package: wandarr
wandarr - the Distributed Transcoding ToolkitThis CLI tool is a transcoding workflow manager that makes transcoding video files easier and optionally across multiple
machines in parallel, using ffmpeg. It is the successor to pytranscoder and is based on the fundamental codebase  of that project.Features:Sequential or concurrent transcoding.Concurrent mode allows you to make maximum use of your
nVidia CUDA-enabled graphics card or Intel accelerated video (QSV)Preserves all streams but allows for filtering by audio and subtitle language.Configurable transcoding templatesTranscode from a list of files (queue) or all on the command lineClustering allows use of other machines.On-the-fly compression monitoring and optional early job termination if not compressing as expected.RequirementsLinux or MacOS, Windows.latestffmpeg(3.4.3-2 or higher, lower versions may still work)nVidia graphics card with latest nVidia CUDA drivers (optional)Intel CPU with QSV enabled (optional)Python 3 (3.10 or higher)Installationpip3installwandarrUpgradingWhatever method above for installing works for you, just use the --upgrade option to update, ie:pip3install--upgradewandarrSupportPlease log issues or questions via the github home page for now.ConfigurationThere is a supplied samplewandarr.ymlconfig file, or you can download it from the git home.  This can be customized all you like, however be
sure to preserve the YAML formatting. Either specify this file on the commandline with the-yoption
or copy it to your home directory as.wandarr.yml(default)In short, you will define your machine(s) used for transcoding, the transcoding capabilities of those machines, and template to tie it all together.There are 4 sections in the file:Section 1 - config - Global configuration informationSampleconfig:ffmpeg:'/opt/homebrew/bin/ffmpeg'# path to ffmpeg for this configrich:yes# use rich text library for nicer outputSection 2 - host definition(s)Thecluster:section is where you define all the machines in your network you intend to use for asynchronous transcoding jobs.
This is optional - you need only define a single machine if that's all you are using.Single-host sample - you can just start out like this:cluster:mbp:os:macostype:local# the host you will run wandarr on.working_dir:/tmp# best if this is an SSDffmpeg:'/opt/homebrew/bin/ffmpeg'engines:-vtstatus:enabledMulti-host Sample:All fields in these samples are required for their respective host type and there are no optional ones.
This sample cluster defines 3 hosts that will participate in a transcode job. The video files reside on a NAS.The first host, "homeserver", is an Ubuntu machine with an NFS mount to the NAS (type=mounted). It has both a QSV-enabled
Intel CPU and a discrete nVidia graphics card with CUDA support.The second host, "winpc", is a Windows 11 machine running wandarr as an agent (type=agent).  It also has an nVidia card
installed but not QSV-enabled.The final host, "mbp", is a MacBook Pro (M1 Pro) where this process is running from (type=local).  It will also handle
transcoding using the Apple VideoToolbox hardware acceleration feature.cluster:# sample linux machine with an Intel i5 CPU/iGPU and nVidia graphics cardhomeserver:os:linuxtype:mounted# files to transcode are available via a network mountworking_dir:/tmp# dir to use for temp files during transcode, best to place on an SSDip:192.168.2.64# ip or hostname of the machineuser:marshall# your ssh user login name. You must be able to ssh into this host w/o a passwordffmpeg:'/usr/bin/ffmpeg'# location of ffmpeg on the hostengines:# list one of more of the video "engines" defined in the next section.  This identifies# the video handling capabilities of the host. In this sample, this host can do QSV and CUDA-qsv-cuda-cpupath-substitutions:# for mounted types only - maps the local drive path to remote mounted one (more later)-'/Volumes/USB11//mnt/media/'-'/Volumes/media//mnt/media/'-'/mnt/downloads//mnt/downloads/'status:enabled# enabled or disabled# sample Windows 11 machine with an nVidia graphics cardwinpc:os:win10type:agent# this host is running wandarr in agent mode (more later). No mounts or ssh used.ip:192.168.2.61ffmpeg:'c:\ffmpeg\bin\ffmpeg.exe'engines:-cuda-cpuworking_dir:'c:/temp'path-substitutions:-'/Volumes/mediam:'-'/Volumes/USB11/mediam:'-'/mnt/merger/media/video/n:video\'-'/mnt/downloads/z:\'status:enabled# sample MacBook Pro (M1 Pro silicon) using VideoToolkit hw accelerationmbp:os:macostype:local# the host you will run wandarr on. Not required if you will not transcode on this machineworking_dir:/tmpffmpeg:'/opt/homebrew/bin/ffmpeg'engines:-vt-cpustatus:enabledHost Types:localThe machine running wandarr.  It's either your only single transcoding machine, or a member of a cluster. If you have multiple machines and will not use your "local" machine to also transcode them you need no local host defined.mountedThis machine can access the media via a network mount. Besides local, this is the next fasted option as the files can be immediately accessed.  You must have an ssh password-less login to this host. You can use the ssh-copy_id tool to establish trust between machines.streamingThis machine has no network mount, so copy the file to it over first, transcode, then copy the resulting file back.  This still requires ssh access like mounted.agentThis machine is running as a remote wandarr agent and requires no ssh or mounted filesystem. The tool must be installed there and started withwandarr --agent.  It will use port 9567 to communicate with wandarr on your local machine to transfer files and perform transcoding. Note that this is insecure - this should only be used on your private network where you have control.Section 3 - enginesThis section defines the video transcoding capabilities of your host(s).  The labels and values can be anything you like.
Each section underenginesdefines a hardware capability.  Here you see 3 - vt, qsv, and cuda representing the 3 types of hardware transcoding available to the sample hosts.
Each section underqualitydefines as many named configurations as you need.  In the sample here you see low, medium, high, and copy.
These are ffmpeg options that control how your video is transformed.  These areonlythe video options as they are the ones that can vary between hardware.
While these are usable samples, you may alter then to suit your needs or add many more.
Also keep in mind you do not need to use hardware acceleration, but it does makes things go faster.Sample:engines:vt:# videotoolkit hardware accelerationquality:low:"-c:vhevc_videotoolbox-qp25-b:v5000K-fmatroska"medium:"-c:vhevc_videotoolbox-presetmedium-qp23-b:v6000K-fmatroska-max_muxing_queue_size1024"high:"-c:vhevc_videotoolbox-presetmedium-qp21-b:v8000K-fmatroska-max_muxing_queue_size1024"qsv:# qsv hardware accelerationquality:medium:"-c:vhevc_qsv-presetmedium-qp23-b:v7000K-fmatroska-max_muxing_queue_size1024"high:"-c:vhevc_qsv-presetmedium-qp21-b:v7000K-fmatroska-max_muxing_queue_size1024"cuda:# nvidia hardware accelerationquality:high:"-c:vhevc_nvenc-cq:v21-rcvbr-rc-lookahead20-bufsize3M-b:v6M-profile:vmain-maxrate:v6M-presetmedium-b_ref_mode0-fmatroska-max_muxing_queue_size1024"medium:"-c:vhevc_nvenc-cq:v23-rcvbr-rc-lookahead20-bufsize3M-b:v7M-profile:vmain-maxrate:v7M-presetmedium-b_ref_mode0-fmatroska-max_muxing_queue_size1024"cpu:# transcode using only the CPU (ick!)quality:high-hevc:"-c:vhevc-cq:v21-rcvbr-rc-lookahead20-bufsize3M-b:v6M-profile:vmain-maxrate:v6M-presetmedium-b_ref_mode0-fmatroska-max_muxing_queue_size1024"medium-hevc:"-c:vhevc-cq:v23-rcvbr-rc-lookahead20-bufsize3M-b:v7M-profile:vmain-maxrate:v7M-presetmedium-b_ref_mode0-fmatroska-max_muxing_queue_size1024"high-264:"-c:vx264-cq:v21-rcvbr-rc-lookahead20-bufsize3M-b:v6M-profile:vmain-maxrate:v6M-presetmedium-b_ref_mode0-fmatroska-max_muxing_queue_size1024"medium-264:"-c:vx264-cq:v23-rcvbr-rc-lookahead20-bufsize3M-b:v7M-profile:vmain-maxrate:v7M-presetmedium-b_ref_mode0-fmatroska-max_muxing_queue_size1024"copy:"-c:vcopy-fmatroska"Section 4 - templatesTemplates define overall how to handle a video.  As with engine definitions above, theclisection here defines the ffmpeg options for handling audio and subtitles.In the samples below you will see examples of video-only transcoding (preserve audio as-is), full audio and video transcoding,
and special case template used just to scrub out unwanted languages.Note thevideo-qualityelement.  This is used to complete the linkage from template to engine to hosts.Sample:templates:vid-only:# name of the template - you will use this on the commandlinecli:# section for non-video ffmpeg commandline optionsaudio:"-c:acopy"subtitles:"-c:scopy"video-quality:medium# match this template to the "medium" quality defined in *engines*audio-lang:eng# preserve only English audio tracks (opt).subtitle-lang:eng# preserve only English subtitle tracks (opt).threshold:15# minimum required compression is %15, or terminate transcode (opt)threshold_check:20# start checking for minimum threshold at 20% (opt)extension:'.mkv'# use this file extensionvid-only-anime:cli:audio:"-c:acopy"subtitles:"-c:scopy"video-quality:mediumaudio-lang:"engjpn"# preserve English and Japanese audiosubtitle-lang:engthreshold:15threshold_check:20extension:'.mkv'best-medium:cli:audio:"-c:aac3-b:a768k"subtitles:"-c:scopy"video-quality:mediumaudio-lang:engsubtitle-lang:engthreshold:15threshold_check:20extension:'.mkv'best-medium-anime:cli:audio:"-c:aac3-b:a768k"subtitles:"-c:scopy"video-quality:mediumaudio-lang:"engjpn"subtitle-lang:engthreshold:15threshold_check:30extension:'.mkv'scrub:# this template used only to scrub out undesired audio and subtitle tracks. no transcoding done.cli:audio:"-c:acopy"subtitles:"-c:scopy"video-quality:copyaudio-lang:engsubtitle-lang:engextension:'.mkv'scrub-anime:# this template used only to scrub out undesired audio and subtitle tracks. no transcoding done.cli:audio:"-c:acopy"subtitles:"-c:scopy"video-quality:copyaudio-lang:"engjpn"subtitle-lang:engextension:'.mkv'Putting it all togetherHere's how to read the samples above in their entirety.In thevid-onlytemplate above, we defined the ffmpeg options to handle audio and subtitles.  In this case, we're just copying with no changes.
Now we need to know the ffmpeg options for handling video.
The video-quality value ofmediummatches allmediumdefinitions in engines.  This tells wandarr that any hosts that match an
engine definition containing amediumwill be a eligible match for this template.  So for medium we have told wandarr how to transcode
video for video toolbox, intel qsv, and nvidia cuda.  Whichever hosts are associated to those engines may be selected to do the job.So, a template relates to an engine quality. A quality defines ffmpeg options for all supported hardware to accomplish the same result.
A host relates to one or more engines. This is how a host is selected for transcoding, and how to do it.You can be as basic or complex as you need.  The typical user only needs 2 or 3 templates.RunningThe default behavior is to remove the original video file after encodingand replace it with the new version.
If you want to keep the sourcebe sure to use the -kparameter.  The work file will be placed in the same
folder as the source with the same name and a .tmp extension while being encoded.usage: main.py [-h] [-v] [-i] [-k] [--dry-run] [-y CONFIGFILE_NAME] [--agent] [-t TEMPLATE] [--hosts HOST_OVERRIDE] [--from-file FROM_FILE] [filename ...]

wandarr (ver 1.0.0)

positional arguments:
  filename

options:
  -h, --help            show this help message and exit
  -v                    verbose mode
  -i                    show technical info on files and stop
  -k                    keep source (do not replace)
  --dry-run             Test run, show steps but don't change anything
  -y CONFIGFILE_NAME    Full path to configuration file. Default is ~/.wandarr.yml
  --agent               Start in agent mode on a host and listen for transcode requests from other wandarr.
  -t TEMPLATE           Template name to use for transcode jobs
  --hosts HOST_OVERRIDE
                        Only run transcode on given host(s), comma-separated
  --from-file FROM_FILE
                        Filename that contains list of full paths of files to transcodeExamples:To get help and version number:wandarr-hShow me the technical goods on some files:wandarr-i*.mkvTo transcode 2 files using a specific template:wandarr-tmy_fave_x264/tmp/video1.mp4/tmp/video2.mp4To auto transcode everything listed in a specific file:wandarr-ttv--from-file/tmp/queue.txtTo do a test run without transcoding:wandarr-tmovie-high--dry-runatestvideo.mp4To use a specific yml file:wandarr-y/home/me/etc/quickstart.yml-ttv*.mp4To transcode on a specific host only:wandarr-ttv--hostworkstation*.mp4