File size: 11,320 Bytes

baa8e90

# Dream Project Animation Nodes for ComfyUI

This repository contains various nodes for supporting Deforum-style animation generation with ComfyUI. I created these
for my own use (producing videos for my "Alt Key Project" music - 
[youtube channel](https://www.youtube.com/channel/UC4cKvJ4hia7zULxeCc-7OcQ)), but I think they should be generic enough 
and useful to many ComfyUI users.

I have demonstrated the use of these custom nodes in this [youtube video](https://youtu.be/pZ6Li3qF-Kk).

# Notice!

This custom node pack is currently not being updated. Stable Diffusion video generation is moving towards a different 
workflow with AnimateDiff and Stable Video Diffusion. I decided to not try to update this node pack, but I am instead 
creating a separate custom node pack here:

[github](https://github.com/alt-key-project/comfyui-dream-video-batches)

This new node pack will be getting my attention from now on (at least as long as stable diffusion video generation is done mostly 
in batches).

## Installation

### Simple option

You can install Dream Project Animation Nodes using the ComfyUI Manager.

### Manual option

Run within (ComfyUI)/custom_nodes/ folder:

* git clone https://github.com/alt-key-project/comfyui-dream-project.git
* cd comfyui-dream-project

Then, if you are using the python embedded in ComfyUI:
* (ComfyUI)/python_embedded/python.exe -s -m pip install -r requirements.txt

With your system-wide python:
*  pip install -r requirements.txt

Finally:
* Start ComfyUI.

After startup, a configuration file 'config.json' should have been created in the 'comfyui-dream-project' directory. 
Specifically check that the path of ffmpeg works in your system (add full path to the command if needed).

## Upgrade

When upgrading, it is good to re-run the pip install command as specified in the install section. This will install any 
new dependencies.

## Configuration

### debug

Setting this to true will enable some trace-level logging.

### ffmpeg.file_extension

Sets the output file extension and with that the envelope used.

### ffmpeg.path 

Path to the ffmpeg executable or just the command if ffmpeg is in PATH.

### ffmpeg.arguments

The arguments sent to FFMPEG. A few of the values are provided by the node:

* %FPS% the target framerate
* %FRAMES% a frame ionput file
* %OUTPUT% output video file path

### encoding.jpeg__quality

Sets the encoding quality of jpeg images.

### ui.top_category

Sets the name of the top level category on the menu. Set to empty string "" to remove the top level. If the top level 
is removed you may also want to disable the category icons to get nodes into existing category folders.

### prepend_icon_to_category / append_icon_to_category

Flags to add a icon before and/or after the category name at each level.

### prepend_icon_icon_to_node / append_icon_icon_to_node

Flags to add an icon before and/or after the node name.

### ui.category_icons

Each key defines a unicode symbol as an icon used for the specified category.

### mpeg_coder.bitrate_factor

This factor allows changing the bitrate to better fit the required quality and codec. A value of 1 is typically 
suitable for H.265.

### mpeg_coder.codec_name

Codec names as specified by ffmpeg. Some common options include "libx264", "libx264" and "mpeg2video".

### mpeg_coder.encoding_threads

Increasing the number of encoding threads in mpegCoder will generally reduce the overall encoding time, but it will also 
increase the load on the computer.

### mpeg_coder.file_extension

Sets the output file extension and with that the envelope used.

### mpeg_coder.max_b_frame

Sets the max-b-frames parameter for as specified in ffmpeg.

## Concepts used

These are some concepts used in nodes:

### Frame Counter

The frame counter is an abstraction that keeps track of where we are in the animation - what frame is rendered 
and how does the current frame fit into the current animation.

### Curves

A curve is simply a node that produces a value based on the frame counter (changing over time).

### Palette

A palette is a collection of color values.

### Sequence

A sequence is a full set of animation frames and a corresponding timeline for these frames. The sequence is
created by the 'Image Sequence Saver' node and it may be used to trigger post processing tasks such as generating the 
video file using ffmpeg. These nodes should be seen as a convenience and they are severely limited. Never put sequence 
nodes in parallel - they will not work as intended!

## The nodes
### Analyze Palette [Dream]
Output brightness, red, green and blue averages of a palette. Useful to control other processing.

### Beat Curve [Dream]
Beat pattern curve with impulses at specified beats of a measure.

### Big *** Switch [Dream]
Switch nodes for different type for up to ten inputs.

### Boolean To Float/Int [Dream]
Converts a boolean value to two different numeric values.

### Build Prompt [Dream] (and Finalize Prompt [Dream])
Weighted text prompt builder utility. Chain any number of these nodes and terminate with 'Finalize Prompt'.

### Calculation [Dream]
Mathematical calculation node. Exposes most of the mathematical functions in the python 
[math module](https://docs.python.org/3/library/math.html), mathematical operators as well as round, abs, int, 
float, max and min.

### Compare Palettes [Dream]
Analyses two palettes and produces the quotient for each individual channel (b/a) and brightness.

### CSV Curve [Dream]
CSV input curve where first column is frame or second and second column is value.

### CSV Generator [Dream]
CSV output, mainly for debugging purposes. First column is frame number and second is value.
Recreates file at frame 0 (removing and existing content in the file).

### Common Frame Dimensions [Dream]
Utility for calculating good width/height based on common video dimensions.

### Video Encoder (FFMPEG) [Dream]
Post processing for animation sequences calling FFMPEG to generate video files.

### File Count [Dream]
Finds the number of files in a directory matching specified patterns.

### Float/Int/string to Log Entry [Dream]
Logging for float/int/string values.

### Frame Count Calculator [Dream]
Simple utility to calculate number of frames based on time and framerate.

### Frame Counter (Directory) [Dream]
Directory backed frame counter, for output directories.

### Frame Counter (Simple) [Dream]
Integer value used as frame counter. Useful for testing or if an auto-incrementing primitive is used as a frame 
counter.

### Frame Counter Info [Dream]
Extracts information from the frame counter.

### Frame Counter Offset [Dream]
Adds an offset (in frames) to a frame counter.

### Frame Counter Time Offset [Dream]
Adds an offset in seconds to a frame counter.

### Image Brightness Adjustment [Dream]
Adjusts the brightness of an image by a factor.

### Image Color Shift [Dream]
Allows changing the colors of an image with a multiplier for each channel (RGB).

### Image Contrast Adjustment [Dream]
Adjusts the contrast of an image by a factor.

### Image Motion [Dream]
Node supporting zooming in/out and translating an image.

### Image Sequence Blend [Dream]
Post processing for animation sequences blending frame for a smoother blurred effect.

### Image Sequence Loader [Dream]
Loads a frame from a directory of images.

### Image Sequence Saver [Dream]
Saves a frame to a directory.

### Image Sequence Tweening [Dream]
Post processing for animation sequences generating blended in-between frames.

### Laboratory [Dream]
Super-charged number generator for experimenting with ComfyUI.

### Log Entry Joiner [Dream]
Merges multiple log entries (reduces noodling).

### Log File [Dream]
The text logging facility for the Dream Project Animation nodes.

### Linear Curve [Dream]
Linear interpolation between two values over the full animation.

### Noise from Area Palettes [Dream]
Generates noise based on the colors of up to nine different palettes, each connected to position/area of the 
image. Although the palettes are optional, at least one palette should be provided.

### Noise from Palette [Dream]
Generates noise based on the colors in a palette.

### Palette Color Align [Dream]
Shifts the colors of one palette towards another target palette. If the alignment factor 
is 0.5 the result is nearly an average of the two palettes. At 0 no alignment is done and at 1 we get a close 
alignment to the target. Above one we will overshoot the alignment.

### Palette Color Shift [Dream]
Multiplies the color values in a palette to shift the color balance or brightness.

### Sample Image Area as Palette [Dream]
Randomly samples a palette from an image based on pre-defined areas. The image is separated into nine rectangular areas 
of equal size and each node may sample one of these.

### Sample Image as Palette [Dream]
Randomly samples pixels from a source image to build a palette from it.

### Saw Curve [Dream]
Saw wave curve.

### Sine Curve [Dream]
Simple sine wave curve.

### Smooth Event Curve [Dream]
Single event/peak curve with a slight bell-shape. 

### String Tokenizer [Dream]
Splits a text into tokens by a separator and returns one of the tokens based on a given index.

### Triangle Curve [Dream]
Triangle wave curve. 

### Triangle Event Curve [Dream]
Single event/peak curve with triangular shape.

### WAV Curve [Dream]
Use an uncompressed WAV audio file as a curve.

### Other custom nodes

Many of the nodes found in 'WAS Node Suite' are useful the Dream Project Animation nodes - I suggest you install those 
custom nodes as well!

## Examples

### Image Motion with Curves

This example should be a starting point for anyone wanting to build with the Dream Project Animation nodes.

[motion-workflow-example](examples/motion-workflow-example.json)

### Image Motion with Color Coherence

Same as above but with added color coherence through palettes.

[motion-workflow-with-color-coherence](examples/motion-workflow-with-color-coherence.json)

### Area Sampled Noise

This flow demonstrates sampling image areas into palettes and generating noise for these areas.

[area-sampled-noise](examples/area-sampled-noise.json)

### Prompt Morphing

This flow demonstrates prompt building with weights based on curves and brightness and contrast control.

[prompt-morphing](examples/prompt-morphing.json)

### Laboratory

This flow demonstrates use of the Laboratory and Logging nodes.

[laboratory](examples/laboratory.json)

## Known issues

### FFMPEG

The call to FFMPEG currently in the default configuration (in config.json) does not seem to work for everyone. The good 
news is that you can change the arguments to whatever works for you - the node-supplied parameters (that probably all need to be in the call)
are:

* -i %FRAMES% (the input file listing frames)
* -r %FPS% (sets the frame rate)
* %OUTPUT% (the path to the video file)

If possible, I will change the default configuration to one that more versions/builds of ffmpeg will accept. Do let me 
know what arguments are causing issues for you!

### Framerate is not always right with mpegCoder encoding node

The mpegCoder library will always use variable frame rate encoding if it is available in the output format. With most 
outputs this means that your actual framerate will differ slightly from the requested one.