Package: ytdl-server
ytdl-serverytdl-server is a program that allows you to download videos onto a remote server
using youtube-dl.Using youtube-dl to download videos onto a network file share is inefficient
since the video file has to pass through your workstation. Additionally, many of
the post-processing options that youtube-dl provides will create a temporary
copy of the file, which means that the amount of data uploaded over the network
can be 3-4x the size of the actual file.ytdl-server solves this by allowing you to download videos directly to the
remote server via a REST API.ytdl-server also supports high-availability, which means that you can perform
maintenance without downtime.[TOC]How it worksytdl-server has two main components: the Flask REST API and the Celery worker.The REST API communicates with the user in order to create jobs. The jobs are
then run by the worker in order to download the desired videos.The complete structure of ytdl-server is shown in the below diagram:Included in ytdl-servermeans that the component is part of ytdl-server.Included in the Docker imagesmeans that the component is built into the Docker images. You'll need to install it separately if you're not using them.Not includedmeans that the component is not part of ytdl-server and needs to
be installed separately.Reverse proxy / Load balancer (not included)Used to forward user-requests to the WSGI server(s).Some available options areNGINXandCaddy.Load balancing is only needed if you're running multiple WSGI server
instances.WSGI server (included in the Docker images)Used to run the Flask REST API(s).Some available options areGunicornandgevent.If you're using the Docker images, Gunicorn is included.If you're running multiple REST API instances, each one needs its own WSGI
server instance.Flask REST API (included in ytdl-server)Used to create and manage jobs.You can run multiple instances if you want high-availability. Otherwise, a
single instance is fine.Celery broker (not included)Used to dispatch jobs created by the REST API to a Celery worker.You can use any brokersupported by Celery, includingRabbitMQandRedis.Celery worker (included in ytdl-server)Used to run jobs (download videos).You can run multiple instances if you want high-availability. Otherwise, a
single instance is fine.PostgreSQL database (not included)Used to store information about jobs, and also functions as the Celery result
backend.InstallationDocker (recommended)Prebuilt Docker images are provided. SeeRegistryfor a list of
supported tags.An example Docker Compose file is providedhere.
To use it, copy the Compose file to a directory, and run the following command
to start it:docker-composeupThe REST API will be accessible at port 8000. SeeUsagefor how to use
it.If you want to use a custom config file, overwrite the existing config file at/config/ytdl-server.yml. SeeConfigurationfor available
options.Manualytdl-server requiresPython3.9+.Install fromPyPI:pip3install'ytdl-server[dbapi]'Install from source:gitclone'https://gitlab.com/adralioh/ytdl-server.git'pip3install'./ytdl-server[dbapi]'The[dbapi]extra installsPsycopg2, which is
used to connect to the PostgreSQL database. You must install a DBAPI for
ytdl-server to work. You can alternatively installpsycopg2-binaryif you
don't want to compile it from source. SeeInstallation - Psycopg 2for
details.If you're usingRedisorAmazon SQSas
the broker, you must also install the following extra dependencies:# Redispip3install'celery[redis]'# Amazon SQSpip3install'celery[sqs]'Run the REST API using a WSGI server such asGunicorn:gunicorn-w4'ytdl_server.flask:with_logging()'Run the Celery worker using Celery:celery-A'ytdl_server.run_celery'workerYou also need to set up the reverse proxy, Celery broker, and PostgreSQL
database. SeeHow it worksfor information, and seeConfigurationfor how to configure ytdl-server to use the
database and broker.UpdatingWhen updating ytdl-server, refer to theversioning schemeand thechangelogfor information about what has changed.ConfigurationSeeConfiguration.UsageVideos are downloaded via jobs. You create a new job whenever you want to
download one or more videos.The following examples access the ytdl-server directly using curl. SeeFrontendsfor more user-friendly ways of accessing it.Create a job:curlytdl.example.com/jobs/\-H'Content-Type: application/json'\-d'{"urls": [ "https://www.youtube.com/watch?v=AAAAAAAAAAA","https://www.youtube.com/watch?v=BBBBBBBBBBB" ],"ytdl_opts": { "format": "best","outtmpl": "%(id)s.%(ext)s" }}'Jobs are created asynchronously, which means that the videos will be downloaded
in the background.You can check the status of a job as follows:curlytdl.example.com/jobs/ffedda2d-7fa4-4839-a705-88c893e3f942ffedda2d-7fa4-4839-a705-88c893e3f942is the job ID. This is obtained from the
output of the previous command where you created the job.You can also cancel a running job:curlytdl.example.com/jobs/ffedda2d-7fa4-4839-a705-88c893e3f942\-XPATCH\-H'Content-Type: application/json'\-d'{"status": "cancelled"}'For complete documentation about the REST API, seeREST API.RegistryPrebuit Docker images are provided atregistry.gitlab.com/adralioh/ytdl-serverfor the REST API and the Celery worker.The supported tags are rebuilt monthly in order to provide the latest security
updates.REST APIImages for the REST API are available atregistry.gitlab.com/adralioh/ytdl-server/api.The images includeGunicorn, which is used as the WSGI server.Supported tags:latest,1,1.1,1.1.8Celery workerImages for the Celery worker are available atregistry.gitlab.com/adralioh/ytdl-server/worker.Theyt_dlptags useyt-dlpinstead of
youtube-dl. You also need to set theYTDL_MODULEenv-var toyt_dlpon the
REST API when using these.Theffmpegtags includeFFmpeg, which is required
for many of youtube-dl's post-processing options.Supported tags:latest,1,1.1,1.1.8yt_dlp,1-yt_dlp,1.1-yt_dlp,1.1.8-yt_dlpffmpeg,1-ffmpeg,1.1-ffmpeg,1.1.8-ffmpegyt_dlp-ffmpeg,1-yt_dlp-ffmpeg,1.1-yt_dlp-ffmpeg,1.1.8-yt_dlp-ffmpegTesting and developmentSeetestsFrontendsytcl- Command-line interface with syntax
similar to youtube-dlFrequently asked questionsSeeFrequently asked questions.