Package: dtlapse
DtLapse - Create smooth timelapse videos withdarktableTOCIntroductionLicensingMotivational ExampleKeyframesWorkflowTl;DrUsage ExamplesAdding more modulesTipps & TricksInstallationDevelopersReferenceIntroductionWhile timelapse videos can be created from regular video footage a more
professional approach would be to take a series of still photos. This allows to
control parameters like exposure length, interval length and thus theshutter
angle. I won't dig
any deeper here, because this is not the objective of this document and there
is plenty of information out there.Now, imagine you have hundreds or even thousands of RAW pictures. All of them
need some sort of processing and in the non-free world people would quickly turn
to Lightroom and LRTimelapse which are undoubtly great tools. But not
everybody's running Windows or MacOS and there are some great open source
alternatives likedarktable. What's missing though
is a tool which, just like LRTimelapse, helps with the editing of all those
pictures based on keyframes. That's wheredtLapsecomes into play!A word of warning: This tool does not replace a solid knowledge ofdarktableand it's editing workflows. For a starting point I recommend thedarktable
manual, thedarktable-user mailing
listand thepixls.us (inofficial)website.LicensingAll source files in this repository are distributed under the terms of the
Gnu General Public License Version 3, or laterunlessotherwise noted.Please see the fileLICENSEfor the full license text.ContributorsFiles in theiopsfolder can carry their own license information,
chosen by the author. Since json does not allow comments please use this format:{ "operation": "exposure"
, "iopdata": [ { "modversion": 5
               , "iop_order": 21.0
               , "smooth": [false, true, true, true, true]
               , "cformat": "iffff"
               }
             ]
, "help": "Smoothing for the exposure module. The parameters are mode (auto/manual), black point, exposure, deflicker percentile and deflicker target level."
, "license": "GPL-3.0-or-later"
, "copyright": ["2020 Jochen Keil <jochen.keil@gmail.com>"]
}DonatingIf you like this work, find it useful and would like to donate please use Paypal
(jochen.keil@gmail.com) or contact me for other means. Thank you very much in
advance!Commercial useFor commercial use, please contact me (jochen.keil@gmail.com) for custom license
terms.This does not affect the use of this software for commercial video or photo
projects or monetization on YouTube, etc. I would appreciate proper attribution
and a donation though.Motivational ExampleLet's assume you've taken a (very short) timelapse consisting of 10 frames. If
you'd take your pictures as they were using an automatic algorithm (e.g. your
camera's jpeg routine) you might notice that brightness and color flicker. This
does not happen always and depends heavily on how well your camera's light meter
and white balance algorithm works but it happens.Back to your frames. You start working on your first frame in darktable and once
you've got a result you're happy with you just copy the entire history stack
over to your other pictures.But NO, what just happened? The highlights are blown out and the colors look
totally off, even stronger for every picture. The lighting conditions changed
rapidly during your shot and your edit of the first frame doesn't fit your last
frame. How can you fix that without tediously working on every single frame?It's actually pretty easy! You just declare frames 1,4,7 and 10 as keyframes, do
your editing there and interpolate the module parameters for 2,3,5,6,8 and 9.
So, instead of editing 10 frames, you're now down to 4.Key-what? (Keyframes)Akeyframeis a special frame (photo, picture) which has been picked in
advance to represent a set of similar pictures. This sounds complicated at first
but actually it's pretty simple. When you open updarktableand load your
stills you're presented with thumbnails for all of them. Now you can select
every Nth frame and tag them. For exampleKeyframe, which is the defaultdtLapsewill look for, but you can also use a different tag. The amount of
keyframes depends on the amount of work you're willing to put into your
timelapse and most importantly how rapidly the light changes. For around 1300
pictures I've had good success with 30 to 40 keyframes. You can also add more
keyframes for time spans where the light changes quickly, e.g. sunrise or sunset
and fewer for very constant footage. Always ensure to include the very first and
the last frame!WorkflowNow that you've got a bunch of keyframes you can start working on them
individually. Use thecollect imagemodule to narrow down to your keyframes
only.First thing you might want to fix is thewhite balance. Color temperature
usually only changes for day-to-night or night-to-day shots, so be sure to put
an emphasis here if you captured that. In addition, sometimes the auto white
balance mode of (at least my) camera fails which can result in ugly color
flicker. To fix this, adjust the color balance for all your keyframes to get
a consistent look across them.Next thing would beexposure. Here, too, flicker is the main problem, e.g.
when your camera adjust exposure time or ISO too rapidly or even randomly.
I recommend to usedarktable's auto exposure mode and start from there. Again,
aim for a consistent look throughout your keyframes.Now that the basics are fixed you can start to usefilmicrgbto adjust shadows
and highlights. After that I usually carry on withlocal contrast,rgb curve,color balance,vibranceandvelvia. For extreme cases of bright
and dark scenesgraduatedndmight help, but usuallyfilmicrgbdoes a better
job. As above, aim for a consistent look across your keyframes.Once you are happy with the look of your keyframes, make sure to save all the
XMPs (write sidecar files). Now you can closedarktable(or minimize it) and
open a terminal. Navigate to the directory which holds your XMP files and rundtLapsewith the--plotoption:$ dtlapse temperature --xmps ${my_xmps} --plot(This requires your keyframes to be tagged withKeyframe. If that's not the
case you can use either the--keyframe-tagoption to specify a different
keyframe or specify the keyframes manually with the--keyframesoption.)Now you should see a window with a plotted curve of the interpolated and
smoothed values between your keyframes. If you're happy with the result, close
the window and rundtLapseagain without the--plotoption:$ dtlapse temperature --xmps ${my_xmps}DtLapsewill now write the calculated values to the XMP files. If you did not
specify the--no-backupoption then every XMP will be copied with a.bkpsuffix.Repeat this for every module you've made modifications with and which is
supported bydtLapse. For a list of supported modules rundtlapse --helpor
look into theiopsdirectory.Finally, you need to reload the XMP files indarktable. Just re-import all
pictures from your timelapse anddarktableshould pick up the changes. If that
does not work you closedarktablebefore runningdtlapse. If you still
experience problems, make sure thatdarktabledoes not give precedence to it's
database rather than the XMP files.Tl;DrLoad your timelapse pictures indarktable.Pick some keyframes. Always include the very first and the last frame.Tag you keyframes, e.g. withKeyframe.Edit your keyframes using the modules supported bydtLapse.Save the XMP sidecar files (write sidecar files) or closedarktable.RundtLapsewith the--plotswitch, check the curve and (if necessary)
tweak it using the--interpolation,--smoothand--window&--orderparameters.Once you're happy with the graph, rundtLapsewithout the--plotswitch
to apply the modifications to your XMP files.Reload your pictures (importmodule) or open updarktableagain.Check if you like the result.Export your pictures or tweak your keyframes and rundtLapseagain.Usage examples$ dtlapse temperature --xmps ${my_xmps} --plotPlot a graph of the interpolated values for the temperature module.$ dtlapse temperature --xmps ${my_xmps} --interpolation cubic --plotUse a different interpolation algorithm instead of the defaultquadratic.$ dtlapse temperature --xmps ${my_xmps} --smooth --plotAdd smoothing to the interpolated values and plot the graph again.$ dtlapse temperature --xmps ${my_xmps} --smooth --window 101 --order 5 --plotAdjust smoothing parameters and plot the graph.$ dtlapse temperature --xmps ${my_xmps} --keyframe-tag MyKeyframeTag --dry-runUse a different tag instead of the defaultKeyframe.$ dtlapse temperature --xmps ${my_xmps} --keyframes ${my_keyframe_xmps} --dry-runSpecify keyframes manually without the use of a tag.Adding more modulesAdding modules is actually pretty easy. Let's assume there's a module you'd like
to add. Go todarktable's git repository, find the module in thesrc/iopdirectory and open the file, for exampleexposure.c. There you will find
a data structure calleddt_iop_exposure_params_t. For thetemperaturemodule
it's calleddt_iop_temperature_params_t. You get the idea. Now, when you look
at theiops/exposure.jsonin this repository you'll see a data entry calledsmoothandcformat. Thesmoothfield contains a list of boolean values
which telldtLapsewhether the parameter should be interpolated and smoothed
or not. For example in theexposuremodule the first data value is a switch
for the module mode which we do not want to be interpolated. Hence it's set to
false. Thecformatfield species the data value types for the struct.  For theexposuremodule it'sint,float,float,floatandfloat(iffff).
Check pythonsstructmodule fordocumentationon how to specify
types. Finally you'll need the proper name of the module for theoperationdata field, themodversionand theiop_order. Themodversionfor a certain
darktable release can be found in a macro calledDT_MODULE_INTROSPECTIONin
the module source file. Themodversionis especially important to distinguish
between different module version with different parameter sets. New versions can
be easily added to theiopdatalist. For theiop_orderI have not found
a better way than to look at an XMP file. Finally, please write a somewhat
meaningful help text.  Drop your now module.json file into theiopsfolder and
it should be available.DT_MODULE_INTROSPECTION(5, dt_iop_exposure_params_t)For more complex modules and parameters python's eval mode can be used to
specifysmoothandcformat. Just prefix the string witheval:. For an
example, take a look atfilmicrgb.json.typedef struct dt_iop_exposure_params_t
{
  dt_iop_exposure_mode_t mode; // $DEFAULT: EXPOSURE_MODE_MANUAL
  float black;    // $MIN: -1.0 $MAX: 1.0 $DEFAULT: 0.0 $DESCRIPTION: "black level correction"
  float exposure; // $MIN: -18.0 $MAX: 18.0 $DEFAULT: 0.0
  float deflicker_percentile;   // $MIN: 0.0 $MAX: 100.0 $DEFAULT: 50.0 $DESCRIPTION: "percentile"
  float deflicker_target_level; // $MIN: -18.0 $MAX: 18.0 $DEFAULT: -4.0 $DESCRIPTION: "target level"
  int compensate_exposure_bias; // $DEFAULT: FALSE
} dt_iop_exposure_params_t;<rdf:li
 darktable:num="6"
 darktable:operation="exposure"
 darktable:enabled="0"
 darktable:modversion="5"
 darktable:params="0100000000000000000000000000484248e18ac0"
 darktable:multi_name=""
 darktable:multi_priority="0"darktable:iop_order="21.0000000000000"darktable:blendop_version="9"
 darktable:blendop_params="gz11eJxjYGBgkGAAgRNODGiAEV0AJ2iwh+CRyscOAAdeGQQ="/>{ "operation": "exposure"
, "iopdata": [ { "modversion": 5
               , "iop_order": 21.0
               , "smooth": [false, true, true, true, true]
               , "cformat": "iffff"
               }
             ]
, "help": "Smoothing for the exposure module. The parameters are mode (auto/manual), black point, exposure, deflicker percentile and deflicker target level."
}Tipps & TricksEvery image must have an existing XMP file. Rundarktablefirst, thendtLapse.Include the first and the last frame in your keyframes!Every keyframe must have the module enabled you're planing to use, even if you
leave it at default settings. For the rest of the images it's not necessary.If you want to use multiple modules, simply use aforloop:foropinexposuretemperaturefilmicrgb;dodtlapse${op}--xmps${xmps}doneThe--xmpparameter does only accept XMP files. You can use shell globbing
to match all your files. Either copy all pictures to particular folder and use--xmps *.xmp, or (like I do) leave a your camera's sequence number in the
filename and use something like--xmps *{00013..00314}.xmp.If darktable is acting strangly or overwrites your XMPs, run it without the
database: darktable --library :memory:Suppose you have all your timelapse pictures (and nothing else) in one folder,
e.g. /path/to/my/timelapse.  Then you just do this:$cd/path/to/my/timelapse
$darktable.# or$darktable*.arw# *.nef *.cr2 or what you're using# edit edit edit$dtlapse${operation}--xmps*.xmp# --plot --smooth --window --order etc$darktable.# or ..InstallationInstall it fromPyPIviapip!pip install --user --upgrade dtlapseorpip3 install --user --upgrade dtlapseorpython3 -m pip install --user --upgrade dtlapseDevelopersSetting up a virtual environment for pythonMake sure that you havevenvinstalled. Either usingpip:$ pip install venvOr with your favorite package manager, e.g.:$ apt install python3-venv$ python3 -m venv env
$ source env/bin/activate
$ pip install -r requirements.txtBuilding and Uploading a package to PyPIhttps://packaging.python.org/tutorials/packaging-projects/python3 -m pip3 install setuptools wheel
python3 setup.py sdist bdist_wheel --universal
python3 -m twine upload --repository testpypi dist/*ReferenceModulesModuleDescriptiontemperatureSmoothing for the temperature (white balance) module. The parameters temperature coefficients.rgbcurveSmoothing for the rgbcurve module. The parameters are the actual nodes for each curve (120 floats), number of nodes per curve (3 ints), curve type (3 ints), autoscale, middle grey scaling and color preservation.colorbalanceSmoothing for the colorbalance module. The parameters are mode, lift[4], gamma[4], gain[4], saturation, contrast, grey and saturation_out.vibranceSmoothing for the vibrance module. The only parameter is the amount.velviaSmoothing for the velvia module. The parameters are strength and bias.bilatSmoothing for the bilat (local contrast) module. The parameters are mode, sigma_r, sigma_s, detail and midtone.exposureSmoothing for the exposure module. The parameters are mode (auto/manual), black point, exposure, deflicker percentile and deflicker target level.filmicrgbSmoothing for the filmicrgb module. The parameters are grey point source, black point source, white point source, security factor, grey point target, black point target, white point target, output power, latitude, contrast, saturation, balance and color preservationgraduatedndSmoothing for the graduatednd module. The parameters are density, hardness, rotation, offset, hue and saturation.toneequalSmoothing for the tone equalizer module. The 9 exposure parameters are smoothed, masking parameters are copied.hotpixelsCopy hot pixels module settings.hazeremovalCopy haze removal module settings. The parameters are strength and distancedemosaicCopy demosaic module settings.highlightsCopy highlight module settings.lensCopy lens correction module settings.ParametersParameterDescription--dry-runDo not modify XMP files.--no-backupBy default all XMP files will be copied to a new file with the suffix "*.bkp".  Use this flag to disable this behaviour.--plotPlot a graph for fine tuning parameters. XMPs will not be modified.--xmpsXMP files. Mandatory.--keyframesA list of XMP files which serve as keyframes. If this is used, keyframe tags will be ignored, even if specified with the --keyframe-tag switch.--keyframe-tagTag used for selecting Keyframes. Default: "Keyframe".--interpolationInterpolation method. One of linear, nearest, zero, slinear, quadratic, cubic, previous, next. See scipy.interpolate.interp1d documentation for details.  Default: quadratic.--smoothUse a Savitzky-Golay filter to smooth the data points. Use --window and --order to fine tune the result.--windowWindow size for the smoothing filter. Must be odd. Default: length of input values. Greater values result in more smoothing.--orderThe order of the polynom for the filter function. Smaller values result in more smoothing. Default value: 3.