PipelineState and BlockState

In Modular Diffusers, PipelineState and BlockState are the core data structures that enable blocks to communicate and share data. The concept is fundamental to understand how blocks interact with each other and the pipeline system.

In the modular diffusers system, PipelineState acts as the global state container that all pipeline blocks operate on. It maintains the complete runtime state of the pipeline and provides a structured way for blocks to read from and write to shared data.

A PipelineState consists of two distinct states:

• The immutable state (i.e. the inputs dict) contains a copy of values provided by users. Once a value is added to the immutable state, it cannot be changed. Blocks can read from the immutable state but cannot write to it.

• The mutable state (i.e. the intermediates dict) contains variables that are passed between blocks and can be modified by them.

Here’s an example of what a PipelineState looks like:

PipelineState(
  inputs={
    'prompt': 'a cat'
    'guidance_scale': 7.0
    'num_inference_steps': 25
  },
  intermediates={
    'prompt_embeds': Tensor(dtype=torch.float32, shape=torch.Size([1, 1, 1, 1]))
    'negative_prompt_embeds': None
  },
)

Each pipeline blocks define what parts of that state they can read from and write to through their inputs, intermediate_inputs, and intermediate_outputs properties. At run time, they gets a local view (BlockState) of the relevant variables it needs from PipelineState, performs its operations, and then updates PipelineState with any changes.

For example, if a block defines an input image, inside the block’s __call__ method, the BlockState would contain:

BlockState(
    image: <PIL.Image.Image image mode=RGB size=512x512 at 0x7F3ECC494640>
)

You can access the variables directly as attributes: block_state.image.

We will explore more on how blocks interact with pipeline state through their inputs, intermediate_inputs, and intermediate_outputs properties, see the PipelineBlock guide.