dgenerate module

exception dgenerate.BatchProcessError

Bases: Exception

Thrown by BatchProcessor.run_file() and BatchProcessor.run_string() when an error in a batch processing script is encountered.

exception dgenerate.DgenerateHelpException

Bases: Exception

Raised by parse_args() and parse_known_args() when --help is encountered and help_raises=True

exception dgenerate.DgenerateUsageError

Bases: Exception

Raised by parse_args() and parse_known_args() on argument usage errors.

exception dgenerate.FrameStartOutOfBounds

Bases: ValueError

Raised by MultiMediaReader when the provided frame_start frame slicing value is calculated to be out of bounds.

exception dgenerate.ImageProcessorArgumentError

Bases: PluginArgumentError, ImageProcessorError

Raised when an image processor receives invalid arguments.

exception dgenerate.ImageProcessorError

Bases: Exception

Generic image processor error base exception.

exception dgenerate.ImageProcessorImageModeError

Bases: ImageProcessorError

Raised when an image processor cannot support a PIL images reported mode.

A mode being a mode string such as RGB, BGR, etc.

exception dgenerate.ImageProcessorNotFoundError

Bases: PluginNotFoundError, ImageProcessorError

Raised when a reference to an unknown image processor name is made.

exception dgenerate.ImageSeedError

Bases: Exception

Raised on image seed parsing and loading errors.

exception dgenerate.InvalidModelFileError

Bases: Exception

Raised when a file is loaded from disk that is an invalid diffusers model format.

This indicates that was a problem loading the primary diffusion model, This could also refer to an SDXL refiner model or Stable Cascade decoder model which are considered primary models.

exception dgenerate.InvalidModelUriError

Bases: Exception

Thrown on model path syntax or logical usage error

exception dgenerate.InvalidSchedulerNameError

Bases: SchedulerLoadError

Unknown scheduler name used.

exception dgenerate.LatentsProcessorArgumentError

Bases: LatentsProcessorError, PluginArgumentError

Raised when invalid arguments are passed to a latents processor.

exception dgenerate.LatentsProcessorError

Bases: Exception

Base exception for all latents processor related errors.

exception dgenerate.LatentsProcessorNotFoundError

Bases: PluginNotFoundError, LatentsProcessorError

Raised when a reference to an unknown latents processor name is made.

exception dgenerate.MediaIdentificationError

Bases: Exception

Raised when a media file is being loaded and it fails to load due to containing invalid or unexpected data.

exception dgenerate.ModelNotFoundError

Bases: Exception

Raised when a specified model can not be located either locally or remotely

exception dgenerate.ModelUriLoadError

Bases: Exception

Thrown when model fails to load from a URI for a reason other than not being found, such as being unsupported.

This exception refers to loadable sub models such as VAEs, LoRAs, ControlNets, Textual Inversions etc.

exception dgenerate.ModuleFileNotFoundError

Bases: FileNotFoundError

Raised by load_modules() if a module could not be found on disk.

exception dgenerate.NonHFConfigDownloadError

Bases: NonHFDownloadError

Raised when a non-Hugging Face config download fails.

exception dgenerate.NonHFDownloadError

Bases: Exception

Raised when a non-Hugging Face download fails.

exception dgenerate.NonHFModelDownloadError

Bases: NonHFDownloadError

Raised when a non-Hugging Face model download fails.

exception dgenerate.OutOfMemoryError(message)

Bases: Exception

Raised when a GPU or processing device runs out of memory.

__init__(message)

exception dgenerate.PluginArgumentError

Bases: Exception

Raised when a plugin encounters an error in the arguments it is loaded by.

Or errors in arguments used for execution.

exception dgenerate.PluginNotFoundError

Bases: Exception

Raised when a plugin could not be located by a name.

exception dgenerate.PromptEmbeddedArgumentError

Bases: Exception

Error involving a prompt embedded argument other than weighter

exception dgenerate.PromptUpscalerProcessingError

Bases: PromptUpscalerError

Thrown when a dgenerate.promptupscalers.PromptUpscaler implementation runs into an issue processing a prompt.

exception dgenerate.PromptWeightingUnsupported

Bases: PromptWeighterError

Thrown when a dgenerate.promptweighters.PromptWeighter implementation cannot handle a specific pipeline or combination of pipeline arguments.

exception dgenerate.RenderLoopConfigError

Bases: Exception

Raised by RenderLoopConfig.check() on configuration errors.

exception dgenerate.SchedulerArgumentError

Bases: SchedulerLoadError

Scheduler URI argument error.

exception dgenerate.SchedulerLoadError

Bases: Exception

Base class for scheduler loading exceptions.

exception dgenerate.SpacyModelNotFoundError

Bases: ModelNotFoundError

Raised when a spacy model cannot be loaded, due to being unable to locate it either online or in the cache.

exception dgenerate.UnknownMimetypeError

Bases: Exception

Raised when an unsupported mimetype is encountered

exception dgenerate.UnsupportedPipelineConfigError

Bases: Exception

Occurs when a diffusers pipeline is requested to be configured in a way that is unsupported by that pipeline.

exception dgenerate.WebFileCacheOfflineModeException

Bases: Exception

Exception raised when the web cache is in offline mode and a file is not found in the cache.

class dgenerate.AnimationETAEvent(origin, frame_index: int, total_frames: int, eta: timedelta)

Bases: Event

Common event stream object produced by the events() event stream of a render loop.

Occurs when there is an update about the estimated finish time of an animation being generated.

__init__(origin, frame_index: int, total_frames: int, eta: timedelta)

eta: timedelta

Current estimated time to complete the animation.

frame_index: int

Frame index at which the ETA was calculated.

total_frames: int

Total frames needed for the animation to complete.

class dgenerate.AnimationFileFinishedEvent(origin: RenderLoop, path: str, config_filename: str, starting_event: StartingAnimationFileEvent)

Bases: Event

Generated in the event stream of RenderLoop.events()

Occurs when an animation (video or animated image) has finished being written to disk.

__init__(origin: RenderLoop, path: str, config_filename: str, starting_event: StartingAnimationFileEvent)

config_filename: str | None

Path to a dgenerate config file if output_configs is enabled.

path: str

Path on disk where the video/animated image was saved.

starting_event: StartingAnimationFileEvent

Animation StartingAnimationFileEvent related to this file finished event.

class dgenerate.AnimationFinishedEvent(origin, starting_event: StartingAnimationEvent)

Bases: Event

Common event stream object produced by the events() event stream of a render loop.

Occurs when a sequence of images that belong to an animation are done generating.

This occurs whether an animation was written to disk or not.

__init__(origin, starting_event: StartingAnimationEvent)

starting_event: StartingAnimationEvent

Animation StartingAnimationEvent related to this file finished event.

class dgenerate.ConfigRunner(injected_args: Sequence[str] | None = None, render_loop: RenderLoop | None = None, plugin_loader: ConfigRunnerPluginLoader = None, version: tuple[int, int, int] | str = '5.0.0', throw: bool = False)

Bases: BatchProcessor

A BatchProcessor that can run dgenerate batch processing configs from a string or file.

__init__(injected_args: Sequence[str] | None = None, render_loop: RenderLoop | None = None, plugin_loader: ConfigRunnerPluginLoader = None, version: tuple[int, int, int] | str = '5.0.0', throw: bool = False)

Raises:

dgenerate.plugin.ModuleFileNotFoundError – If a module path parsed from --plugin-modules in injected_args could not be found on disk.

Parameters:

• injected_args – dgenerate command line arguments in the form of a list, see: shlex module, or sys.argv. These arguments will be injected at the end of every dgenerate invocation in the config. --plugin-modules are parsed from injected_args and added to plugin_loader. If -v/--verbose is present in injected_args debugging output will be enabled globally while the config runs, and not just for invocations. Passing -v/--verbose also enables printing stack traces for all unhandled directive exceptions to stderr. If -ofm/--offline-mode is present in injected_args, plugins will be instructed to only look for resources such as models in cache / on disk and never attempt to download from the internet.

• render_loop – RenderLoop instance, if None is provided one will be created.

• plugin_loader – Batch processor plugin loader, if one is not provided one will be created.

• version – Config version for #! dgenerate x.x.x version checks, defaults to dgenerate.__version__

• throw – Whether to throw exceptions from dgenerate.invoker.invoke_dgenerate() or handle them. If you set this to True exceptions will propagate out of dgenerate invocations instead of a dgenerate.batchprocess.BatchProcessError being raised by the created dgenerate.batchprocess.BatchProcessor. A line number where the error occurred can be obtained using dgenerate.batchprocess.BatchProcessor.current_line.

generate_directives_help(directive_names: Collection[str] | None = None, help_wrap_width: int | None = None)

Generate the help string for --directives-help

Parameters:

• directive_names – Display help for specific directives, if None or [] is specified, display all.

• help_wrap_width – Wrap documentation strings by this amount, if None use dgenerate.textprocessing.long_text_wrap_width()

Raises:

ValueError – if given directive names could not be found

Returns:

help string

generate_functions_help(function_names: Collection[str] | None = None, help_wrap_width: int | None = None)

Generate the help string for --functions-help

Parameters:

• function_names – Display help for specific functions, if None or [] is specified, display all.

• help_wrap_width – Wrap documentation strings by this amount, if None use dgenerate.textprocessing.long_text_wrap_width()

Raises:

ValueError – if given directive names could not be found

Returns:

help string

generate_template_variables_help(variable_names: Collection[str] | None = None, show_values: bool = True)

Generate a help string describing available template variables, their types, and values for use in batch processing.

This is used for --templates-help

Parameters:

• variable_names – Display help for specific variables, if None or [] is specified, display all.

• show_values – Show the value of the template variable or just the name?

Raises:

ValueError – if given variable names could not be found

Returns:

a human-readable description of all template variables

property local_files_only: bool

Is this config runner only going to look for resources such as models in cache / on disk?

This will be True if -ofm/--offline-mode was parsed from injected_args

property plugin_module_paths: frozenset[str]

Set of plugin module paths if they were injected into the config runner by --plugin-modules or used in a \import_plugins statement in a config.

Returns:

a set of paths, may be empty but not None

class dgenerate.ConfigRunnerPlugin(loaded_by_name: str, config_runner: ConfigRunner | None = None, render_loop: RenderLoop | None = None, local_files_only: bool = False, **kwargs)

Bases: Plugin

Abstract base class for config runner plugin implementations.

__init__(loaded_by_name: str, config_runner: ConfigRunner | None = None, render_loop: RenderLoop | None = None, local_files_only: bool = False, **kwargs)

Parameters:

• loaded_by_name – The name the plugin was loaded by, will be passed by the loader. If None is passed, the first name mentioned by the plugin implementation will be used. This can simplify using some plugin classes directly without loading them through a loader implementation.

• argument_error_type – This exception type will be raised upon argument errors (invalid arguments) when loading a plugin using a PluginLoader implementation. It should match the argument_error_type given to the PluginLoader implementation being used to load the inheritor of this class.

• kwargs – Additional arguments that may arise when using an ARGS static signature definition with multiple NAMES in your implementation.

register_directive(name, implementation: Callable[[Sequence[str]], int])

Register a config directive implementation on the dgenerate.batchprocess.ConfigRunner instance.

Your directive should return a return code, 0 for success and anything else for failure.

Returning non zero will cause BatchProcessError to be raised from the runner, halting execution of the config.

Any non-exiting exception will be eaten and rethrown as BatchProcessError, also halting execution of the config.

Raises:

RuntimeError – if a config directive with the same name already exists

Parameters:

• name – directive name

• implementation – implementation callable

register_template_function(name, implementation: Callable)

Register a config template function implementation on the dgenerate.batchprocess.ConfigRunner instance.

Raises:

RuntimeError – if a template function with the same name already exists

Parameters:

• name – function name

• implementation – implementation callable

set_template_variable(name, value)

Set a template variable on the dgenerate.batchprocess.ConfigRunner instance.

Parameters:

• name – variable name

• value – variable value

update_template_variables(values)

Update multiple template variable values on the dgenerate.batchprocess.ConfigRunner instance.

Parameters:

values – variable values, dictionary of names to values

property config_runner: ConfigRunner | None

Provides access to the currently instantiated dgenerate.batchprocess.ConfigRunner object running the config file that this directive is being invoked in.

property injected_args: Sequence[str]

Return any arguments injected into the config from the command line.

If none were injected an empty sequence will be returned.

Returns:

command line arguments

property local_files_only: bool

Is this plugin only going to look for resources such as models in cache / on disk?

property plugin_module_paths: frozenset[str]

Set of plugin module paths if they were injected into the config runner by --plugin-modules or used in a \import_plugins statement in a config.

Returns:

a set of paths, may be empty but not None

property render_loop: RenderLoop | None

Provides access to the currently instantiated dgenerate.renderloop.RenderLoop object.

This object will have been used for any previous invocation of dgenerate in a config file.

class dgenerate.ConfigRunnerPluginLoader

Bases: PluginLoader

Loads dgenerate.batchprocess.ConfigRunnerPlugin plugins.

__init__()

load(uri: str, **kwargs) → ConfigRunnerPlugin

Load an plugin using a URI string containing its name and arguments.

Parameters:

• uri – The URI string

• kwargs – default argument values, will be override by arguments specified in the URI

Raises:

• ValueError – If uri is None

• RuntimeError – If a plugin is discovered to be using a reserved argument name upon loading it.

• dgenerate.plugin.PluginArgumentError – If there is an error in the loading arguments for the plugin.

• dgenerate.plugin.PluginNotFoundError – If the plugin name mentioned in the URI could not be found.

Returns:

plugin instance

class dgenerate.DataType(value)

Bases: Enum

Represents model precision

AUTO = 0

Auto selection.

BFLOAT16 = 3

16 bit brain floating point.

FLOAT16 = 1

16 bit floating point.

FLOAT32 = 2

32 bit floating point.

class dgenerate.DgenerateArguments

Bases: RenderLoopConfig

Represents dgenerate’s parsed command line arguments, can be used as a configuration object for dgenerate.renderloop.RenderLoop.

__init__()

global_config: str | None = None

global config file path.

plugin_module_paths: Sequence[str]

Plugin module paths --plugin-modules

verbose: bool = False

Enable debug output? -v/--verbose

class dgenerate.DiffusionArguments

Bases: SetFromMixin

Represents all possible arguments for a DiffusionPipelineWrapper call.

static prompt_embedded_arg_checker(name: str, value: Any)

Checks if a class member / value is forbidden to use with a prompt embedded argument specification.

Parameters:

• name – the argument name

• value – the argument value

describe_pipeline_wrapper_args() → str

Describe the pipeline wrapper arguments in a pretty, human-readable way, with word wrapping depending on console size or a maximum length depending on what stdout currently is.

Returns:

description string.

determine_pipeline_type()

Determine the dgenerate.pipelinewrapper.PipelineType needed to utilize these arguments.

Returns:

dgenerate.pipelinewrapper.PipelineType

get_pipeline_wrapper_kwargs()

Get the arguments dictionary needed to call DiffusionPipelineWrapper

Returns:

dictionary of argument names with values

adetailer_class_filter: Collection[int | str] | None = None

A collection of class IDs and/or class names that indicates what YOLO detection classes to keep. This filter is applied before index-filter. Detections that don’t match any of the specified classes will be ignored. Integers are treated as ID’s, strings are treated as names.

adetailer_detector_padding: int | tuple[int, int] | tuple[int, int, int, int] | None = None

This value specifies the amount of padding that will be added to the detection rectangle which is used to generate a masked area. The default is 0, you can make the mask area around the detected feature larger with positive padding and smaller with negative padding.

Example:

32 (32px Uniform, all sides)

(10, 20) (10px Horizontal, 20px Vertical)

(10, 20, 30, 40) (10px Left, 20px Top, 30px Right, 40px Bottom)

Defaults to 0.

adetailer_index_filter: Collection[int] | None = None

A list index values that indicates what YOLO detection indices to keep, the index values start at zero. Detections are sorted by their top left bounding box coordinate from left to right, top to bottom, by (confidence descending). The order of detections in the image is identical to the reading order of words on a page (english). Inpainting will only be performed on the specified detection indices, if no indices are specified, then inpainting will be performed on all detections. This filter is applied after class-filter.

adetailer_mask_blur: int | None = None

Indicates the level of gaussian blur to apply to the inpaint mask generated by adetailer, which can help with smooth blending of the inpainted feature. Defaults to 4.

adetailer_mask_dilation: int | None = None

Indicates the amount of dilation applied to the generated adetailer inpaint mask, see: cv2.dilate. Defaults to 4.

adetailer_mask_padding: int | tuple[int, int] | tuple[int, int, int, int] | None = None

This value indicates how much padding to place around the masked area when cropping out the image to be inpainted, this value must be large enough to accommodate any feathering on the edge of the mask caused by DiffusionArguments.mask_blur or :py:attr:`.DiffusionArguments.mask_dilation for the best result.

Example:

32 (32px Uniform, all sides)

(10, 20) (10px Horizontal, 20px Vertical)

(10, 20, 30, 40) (10px Left, 20px Top, 30px Right, 40px Bottom)

Defaults to 32.

adetailer_mask_shape: str | None = None

This indicates what mask shape adetailer should attempt to draw around a detected feature, the default value is “rectangle”. You may also specify “circle” to generate an ellipsoid shaped mask, which might be helpful for achieving better blending.

adetailer_model_masks: bool | None = None

Indicates that masks generated by the model itself should be preferred over masks generated from the detection bounding box. If this is True, and the model itself returns mask data, DiffusionArguments.adetailer_mask_shape, DiffusionArguments.adetailer_mask_padding, and DiffusionArguments.adetailer_detector_padding will all be ignored.

adetailer_size: int | None = None

Target size for processing detected areas. When specified, detected areas will always be scaled to this target size (with aspect ratio preserved) for processing, then scaled back to the original size for compositing. This can significantly improve detail quality for small detected features like faces or hands, or reduce processing time for overly large detected areas. The scaling is based on the larger dimension (width or height) of the detected area. The optimal resampling method is automatically selected for both upscaling and downscaling. Must be an integer greater than 1. Defaults to none (process at native resolution).

aspect_correct: bool = False

When resizing input images according to DiffusionArguments.width and DiffusionArguments.height, should the resize be aspect correct?

batch_size: int | None = None

Number of images to produce in a single generation step on the same GPU.

clip_skip: int | None = None

Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Only supported for model_type values sd and sdxl, including with controlnet_uris defined.

control_images: Sequence[Image] | None = None

ControlNet guidance images to use if controlnet_uris were given to the constructor of DiffusionPipelineWrapper.

Note: Control images must be PIL Images, tensors are not supported since ControlNet/T2I-Adapter operate in pixel space.

All input images involved in a generation must match in dimension.

All incoming ControlNet images will be aligned by 8 automatically, if they need to be aligned by a value higher than this, a warning will be issued to stdout via dgenerate.messages.

decoded_latents_image_processor_uris: Sequence[str] | None = None

One or more image processor URI strings for processing images decoded from incoming latents.

These processors are applied to images that are decoded from latent tensors provided through the DiffusionArguments.images argument when doing img2img with tensor inputs. The processors are applied in sequence after VAE decoding but before the images are used in the pipeline.

The processing flow is: decoded images → pre-resize processing → resize to user dimensions → post-resize processing. This allows both preprocessing the raw decoded images and postprocessing after they are resized to the target dimensions.

If no processors are specified, images are simply resized to user dimensions without additional processing.

deep_cache: bool = False

Enable DeepCache acceleration for the main model? DeepCache caches the intermediate attention layer outputs to speed up the diffusion process. This is beneficial for higher inference steps.

This is supported for Stable Diffusion, Stable Diffusion XL, Stable Diffusion Upscaler X4, Kolors, and Pix2Pix variants.

deep_cache_branch_id: int | None = None

Controls which branch ID DeepCache should operate on in the UNet for the main model.

This value must be greater than or equal to 0.

This is supported for Stable Diffusion, Stable Diffusion XL, Stable Diffusion Upscaler X4, Kolors, and Pix2Pix variants.

Supplying any value implies that DiffusionArguments.deep_cache is enabled.

Defaults to 1.

deep_cache_interval: int | None = None

Controls the frequency of caching intermediate outputs in DeepCache for the main model.

This value must be greater than zero.

This is supported for Stable Diffusion, Stable Diffusion XL, Stable Diffusion Upscaler X4, Kolors, and Pix2Pix variants.

Supplying any value implies that DiffusionArguments.deep_cache is enabled.

Defaults to 5.

denoising_end: float | None = None

Denoising should end at this fraction of total timesteps (0.0 to 1.0).

This is useful for generating noisy latents that can be saved and passed to other models.

Scheduler Compatibility:

• SD 1.5 models: Only stateless schedulers are supported (EulerDiscreteScheduler, LMSDiscreteScheduler, EDMEulerScheduler, DPMSolverMultistepScheduler, DDIMScheduler, DDPMScheduler, PNDMScheduler)

• SDXL models: All schedulers supported via native denoising_start/denoising_end

• SD3/Flux models: FlowMatchEulerDiscreteScheduler and standard schedulers supported

denoising_start: float | None = None

Denoising should start at this fraction of total timesteps (0.0 to 1.0).

This is useful continuing denoising on noisy latents generated with DiffusionArguments.denoising_end

Scheduler Compatibility:

• SD 1.5 models: Only stateless schedulers are supported (EulerDiscreteScheduler, LMSDiscreteScheduler, EDMEulerScheduler, DPMSolverMultistepScheduler, DDIMScheduler, DDPMScheduler, PNDMScheduler)

• SDXL models: All schedulers supported via native denoising_start/denoising_end

• SD3/Flux models: FlowMatchEulerDiscreteScheduler and standard schedulers supported

floyd_image: Image | Tensor | None = None

The output image or tensor of the last stage when performing img2img or inpainting generation with Deep Floyd. When performing txt2img generation DiffusionArguments.image is used.

When a tensor is provided, it represents latent space data from a previous Floyd stage.

Incoming floyd images will be automatically aligned by 8.

freeu_params: tuple[float, float, float, float] | None = None

FreeU is a technique for improving image quality by re-balancing the contributions from the UNet’s skip connections and backbone feature maps.

This can be used with no cost to performance, to potentially improve image quality.

This argument can be used to specify The FreeU parameters: s1, s2, b1, and b2 in that order.

This argument only applies to models that utilize a UNet: SD1.5/2, SDXL, and Kolors

See: https://huggingface.co/docs/diffusers/main/en/using-diffusers/freeu

And: https://github.com/ChenyangSi/FreeU

guidance_rescale: float | None = None

This value is only supported for certain dgenerate.pipelinewrapper.DiffusionPipelineWrapper configurations, an error will be produced when it is unsupported.

Guidance rescale factor proposed by [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://arxiv.org/pdf/2305.08891.pdf) guidance_scale is defined as φ in equation 16. of [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://arxiv.org/pdf/2305.08891.pdf). Guidance rescale factor should fix overexposure when using zero terminal SNR.

guidance_scale: float | None = None

A higher guidance scale value encourages the model to generate images closely linked to the text DiffusionArguments.prompt at the expense of lower image quality. Guidance scale is enabled when DiffusionArguments.guidance_scale > 1

height: int | None = None

Output image height.

Will be automatically aligned by 8.

If alignments of more than 8 need to be forced, a warning will be issued to stdout via dgenerate.messages.

hi_diffusion: bool = False

Activate HiDiffusion for the primary model?

This can increase the resolution at which the model can output images while retaining quality with no overhead, and possibly improved performance.

See: https://github.com/megvii-research/HiDiffusion

This is supported for: --model-type sd, sdxl, and kolors.

hi_diffusion_no_raunet: bool | None = None

Disable RAU-Net when using HiDiffusion for the primary model?

This disables the Resolution-Aware U-Net component of HiDiffusion.

See: https://github.com/megvii-research/HiDiffusion

This is supported for: --model-type sd, sdxl, and kolors.

hi_diffusion_no_win_attn: bool | None = None

Disable window attention when using HiDiffusion for the primary model?

This disables the MSW-MSA (Multi-Scale Window Multi-Head Self-Attention) component of HiDiffusion.

See: https://github.com/megvii-research/HiDiffusion

This is supported for: --model-type sd, sdxl, and kolors.

image_guidance_scale: float | None = None

This value is only relevant for pix2pix dgenerate.pipelinewrapper.ModelType.

Image guidance scale is to push the generated image towards the initial image DiffusionArguments.image. Image guidance scale is enabled by setting DiffusionArguments.image_guidance_scale > 1. Higher image guidance scale encourages to generate images that are closely linked to the source image DiffusionArguments.image, usually at the expense of lower image quality.

image_seed_strength: float | None = None

Image seed strength, which relates to how much an img2img source (image attribute) is used during generation. Between 0.001 (close to zero but not 0) and 1.0, the closer to 1.0 the less the image is used for generation, IE. the more creative freedom the AI has.

images: Sequence[Image] | Sequence[Tensor] | None = None

Images or tensors for img2img operations, or the base for inpainting operations.

All inputs must be either PIL Images or torch Tensors - mixing both types in the same sequence is not supported.

When tensors are provided, they represent latent space data and bypass VAE encoding. Tensor inputs cannot be resized or processed with image processors.

All input images involved in a generation except for ip_adapter_images must match in dimension, except in the case of Stable Cascade, which can accept multiple images of any dimension for the purpose of image based prompting similar to IP Adapters.

All incoming images will be aligned by 8 automatically, if they need to be aligned by a value higher than this, a warning will be issued.

All other pipelines interpret multiple image inputs as a batching request.

img2img_latents_processors: Sequence[str] | None = None

One or more latents processor URI strings for processing img2img latents before pipeline execution.

These processors are applied to latent tensors provided through the DiffusionArguments.images argument when doing img2img with tensor inputs. The processors are applied in sequence and may occur before VAE decoding (for models that decode img2img latents) or before direct pipeline usage.

inference_steps: int | None = None

The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.

inpaint_crop: bool = False

Enable cropping to mask bounds for inpainting. When enabled, input images will be automatically cropped to the bounds of their masks (plus any padding) before processing, then the generated result will be pasted back onto the original uncropped image. This allows inpainting at higher effective resolutions for better quality results.

Batching Behavior:

• Cannot be used with multiple input images/masks in the same call

• Each image/mask pair must be processed individually as different masks may have different crop bounds

• However, batch_size > 1 is supported for generating multiple variations of a single crop

• Multiple images require separate pipeline calls, not batch processing

Auto-enabling:

This is automatically enabled when DiffusionArguments.inpaint_crop_padding, DiffusionArguments.inpaint_crop_feather, or DiffusionArguments.inpaint_crop_masked are specified.

inpaint_crop_feather: int | None = None

Feather value to use when pasting the generated result back onto the original image when DiffusionArguments.inpaint_crop is enabled. Feathering creates smooth transitions from opaque to transparent. Cannot be used together with DiffusionArguments.inpaint_crop_masked.

Specifying this value automatically enables DiffusionArguments.inpaint_crop if it is not already enabled.

Note: Inpaint crop cannot be used with multiple input images. See DiffusionArguments.inpaint_crop for batching details.

inpaint_crop_masked: bool = False

When inpaint_crop is enabled, use the mask when pasting the generated result back onto the original image. This means only the masked areas will be replaced. Cannot be used together with DiffusionArguments.inpaint_crop_feather.

Specifying this value automatically enables DiffusionArguments.inpaint_crop if it is not already enabled.

Note: Inpaint crop cannot be used with multiple input images. See DiffusionArguments.inpaint_crop for batching details.

inpaint_crop_padding: int | tuple[int, int] | tuple[int, int, int, int] | None = None

Padding values to use around mask bounds when inpaint_crop is enabled.

Supported formats:

• int: Same padding on all sides

• tuple[int, int]: (horizontal, vertical) padding

• tuple[int, int, int, int]: (left, top, right, bottom) padding

Specifying this value automatically enables DiffusionArguments.inpaint_crop if it is not already enabled. Default value when DiffusionArguments.inpaint_crop is enabled but no padding is specified: 32 pixels.

Note: Inpaint crop cannot be used with multiple input images. See DiffusionArguments.inpaint_crop for batching details.

ip_adapter_images: Sequence[Sequence[Image]] | None = None

IP Adapter images to use if ip_adapter_uris were given to the constructor of DiffusionPipelineWrapper.

Note: IP Adapter images must be PIL Images, tensors are not supported since IP-Adapter operates in pixel space.

This should be a list of Sequence[PIL.Image]

Each list entry corresponds to an IP adapter URI.

Multiple IP Adapter URIs can be provided, each IP Adapter can get its own set of images.

All incoming IP Adapter images will be aligned by 8 automatically, if they need to be aligned by a value higher than this, a warning will be issued to stdout via dgenerate.messages.

latents: Sequence[Tensor] | None = None

Noisy latents to serve as a starting point for generation, this should be a list of tensors in the format [C, H, W] or [B, C, H, W], A list of tensors with a batch dimension will be concatenated intelligently.

latents_post_processors: Sequence[str] | None = None

One or more latents processor URI strings for processing output latents when outputting to latents.

These processors are applied to latents when DiffusionArguments.output_latents is True. The processors are applied in sequence after the diffusion pipeline generates the latents but before they are returned in the result.

latents_processors: Sequence[str] | None = None

One or more latents processor URI strings for processing raw input latents before pipeline execution.

These processors are applied to latents provided through the DiffusionArguments.latents argument (raw latents used as noise initialization). The processors are applied in sequence before the latents are passed to the diffusion pipeline.

mask_images: Sequence[Image] | None = None

Mask images for inpainting operations.

The amount of img2img images must be equal to the amount of mask_images supplied.

Note: Mask images are always PIL Images, tensor masks are not supported.

All input images involved in a generation except for ip_adapter_images must match in dimension.

All incoming mask images will be aligned by 8 automatically, if they need to be aligned by a value higher than this, a warning will be issued to stdout via dgenerate.messages.

max_sequence_length: int | None = None

Max number of prompt tokens that the T5EncoderModel (text encoder 3) of Stable Diffusion 3 or Flux can handle.

This defaults to 256 for SD3 when not specified, and 512 for Flux.

The maximum value is 512 and the minimum value is 1.

High values result in more resource usage and processing time.

output_latents: bool = False

Whether to output raw latent tensors instead of decoded PIL Images.

When True, the pipeline will return raw latent tensors instead of decoded images. This is useful for saving latent representations or for chaining multiple pipeline operations.

Defaults to False (outputs PIL Images).

pag_adaptive_scale: float | None = None

Adaptive perturbed attention guidance scale.

pag_scale: float | None = None

Perturbed attention guidance scale.

prompt: Prompt | None = None

Primary prompt

prompt_weighter_uri: str | None = None

Default prompt weighter plugin to use for all models.

ras: bool = False

Activate RAS (Region-Adaptive Sampling) for the primary model?

This can increase inference speed with SD3.

See: https://github.com/microsoft/ras

This is supported for: --model-type sd3.

ras_end_step: int | None = None

Ending step for RAS (Region-Adaptive Sampling).

This controls when RAS stops applying its sampling strategy. Must be greater than or equal to 1.

Defaults to the number of inference steps if not specified.

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

ras_error_reset_steps: Sequence[int] | None = None

Dense sampling steps to reset accumulated error in RAS.

The dense sampling steps inserted between the RAS steps to reset the accumulated error. A list of step numbers, e.g. [12, 22].

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

ras_high_ratio: float | None = None

Ratio of high value tokens to be cached in RAS.

Based on the metric selected, the ratio of the high value chosen to be cached. Default value is 1.0, but can be set between 0.0 and 1.0 to balance the sample ratio between the main subject and the background.

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

ras_index_fusion: bool | None = None

Enable index fusion in RAS (Region-Adaptive Sampling) for the primary model?

This can improve attention computation in RAS for SD3.

See: https://github.com/microsoft/ras

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3, (but not for SD3.5 models)

ras_metric: str | None = None

Metric to use for RAS (Region-Adaptive Sampling).

This controls how RAS measures the importance of tokens for caching. Valid values are “std” (standard deviation) or “l2norm” (L2 norm). Defaults to “std”.

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

ras_sample_ratio: float | None = None

Average sample ratio for each RAS step.

For instance, setting this to 0.5 on a sequence of 4096 tokens will result in the noise of averagely 2048 tokens to be updated during each RAS step. Must be between 0.0 and 1.0.

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

ras_skip_num_step: int | None = None

Skip steps for RAS (Region-Adaptive Sampling). Controls the number of steps to skip between RAS steps. The actual number of tokens skipped will be rounded down to the nearest multiple of 64 to ensure efficient memory access patterns for attention computation.

When used with DiffusionArguments.ras_skip_num_step_length greater than 0, this value determines how the number of skipped tokens changes over time.

Positive values will increase the number of skipped tokens over time, while negative values will decrease it.

Each value will be tried in turn.

Supplying any values implies DiffusionArguments.ras.

This is supported for: --model-type sd3.

ras_skip_num_step_length: int | None = None

Skip step lengths for RAS (Region-Adaptive Sampling). Controls the length of steps to skip between RAS steps. When set to 0, static dropping is used where the number of skipped tokens remains constant throughout the generation process.

When greater than 0, dynamic dropping is enabled where the number of skipped tokens varies over time based on DiffusionArguments.ras_skip_num_step.

The pattern of skipping will repeat every DiffusionArguments.ras_skip_num_step_length steps.

Each value will be tried in turn.

Supplying any values implies DiffusionArguments.ras.

This is supported for: --model-type sd3.

ras_start_step: int | None = None

Starting step for RAS (Region-Adaptive Sampling).

This controls when RAS begins applying its sampling strategy. Must be greater than or equal to 1.

Defaults to 4 if not specified.

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

ras_starvation_scale: float | None = None

Starvation scale for RAS patch selection.

RAS tracks how often a token is dropped and incorporates this count as a scaling factor in the metric for selecting tokens. This scale factor prevents excessive blurring or noise in the final generated image. Larger scaling factor will result in more uniform sampling. Usually set between 0.0 and 1.0.

Supplying any value implies that DiffusionArguments.ras is enabled.

This is supported for: --model-type sd3.

sada: bool = False

Enable SADA (Stability-guided Adaptive Diffusion Acceleration) with default parameters for the primary model.

This is equivalent to setting all SADA parameters to their default values.

See: https://github.com/Ting-Justin-Jiang/sada-icml

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_acc_range: tuple[int, int] | None = None

SADA acceleration range start / end step for the primary model.

Defines the starting step for SADA acceleration.

Starting step must be at least 3 as SADA leverages third-order dynamics.

Defaults to [10,47].

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_lagrange_int: int | None = None

SADA Lagrangian interpolation interval for the primary model.

Interval for Lagrangian interpolation. Must be compatible with sada_lagrange_step (lagrange_step % lagrange_int == 0).

Model-specific defaults:

• SD/SD2: 4

• SDXL/Kolors: 4

• Flux: 4

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_lagrange_step: int | None = None

SADA Lagrangian interpolation step for the primary model.

Step value for Lagrangian interpolation. Must be compatible with sada_lagrange_int (lagrange_step % lagrange_int == 0).

Model-specific defaults:

• SD/SD2: 24

• SDXL/Kolors: 24

• Flux: 20

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_lagrange_term: int | None = None

SADA Lagrangian interpolation terms for the primary model.

Number of terms to use in Lagrangian interpolation. Set to 0 to disable Lagrangian interpolation.

Model-specific defaults:

• SD/SD2: 4

• SDXL/Kolors: 4

• Flux: 3

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_max_downsample: int | None = None

SADA maximum downsample factor for the primary model.

Controls the maximum downsample factor in the SADA algorithm. Lower values can improve quality but may reduce speedup.

Model-specific defaults:

• SD/SD2: 1

• SDXL/Kolors: 2

• Flux: 0

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

SADA is not compatible with HiDiffusion, DeepCache, or TeaCache.

sada_max_fix: int | None = None

SADA maximum fixed memory for the primary model.

Maximum amount of fixed memory to use in SADA optimization.

Model-specific defaults:

• SD/SD2: 5120 (5 * 1024)

• SDXL/Kolors: 10240 (10 * 1024)

• Flux: 0

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_max_interval: int | None = None

SADA maximum interval for optimization for the primary model.

Maximum interval between optimizations in the SADA algorithm.

Defaults to 4.

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_sx: int | None = None

SADA spatial downsample factor X for the primary model.

Controls the spatial downsample factor in the X dimension. Higher values can increase speedup but may affect quality.

Model-specific defaults:

• SD/SD2: 3

• SDXL/Kolors: 3

• Flux: 0 (not used)

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

sada_sy: int | None = None

SADA spatial downsample factor Y for the primary model.

Controls the spatial downsample factor in the Y dimension. Higher values can increase speedup but may affect quality.

Model-specific defaults:

• SD/SD2: 3

• SDXL/Kolors: 3

• Flux: 0 (not used)

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

scheduler_uri: str | None = None

Primary model scheduler URI

sdxl_aesthetic_score: float | None = None

Optional, defaults to 6.0. This argument is used for img2img and inpainting operations only Used to simulate an aesthetic score of the generated image by influencing the positive text condition. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

sdxl_crops_coords_top_left: tuple[int, int] | None = None

Optional SDXL conditioning parameter. DiffusionArguments.sdxl_crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position DiffusionArguments.sdxl_crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting DiffusionArguments.sdxl_crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

sdxl_high_noise_fraction: float | None = None

SDXL high noise fraction. This proportion of timesteps/inference steps are handled by the primary model, while the inverse proportion is handled by the refiner model when an SDXL model_type value.

When the refiner is operating in edit mode the number of total inference steps for the refiner will be calculated in a different manner, currently the refiner operates in edit mode during generations involving ControlNets as well as inpainting.

In edit mode, the refiner uses img2img with an image seed strength to add details to the image instead of cooperative denoising, this image seed strength is calculated as (1.0 - DiffusionArguments.sdxl_high_noise_fraction), and the number of inference steps for the refiner is then calculated as (image_seed_strength * inference_steps).

sdxl_negative_aesthetic_score: float | None = None

Negative influence version of DiffusionArguments.sdxl_aesthetic_score

sdxl_negative_crops_coords_top_left: tuple[int, int] | None = None

Negative influence version of DiffusionArguments.sdxl_crops_coords_top_left

sdxl_negative_original_size: tuple[int, int] | None = None

This value is only supported for certain dgenerate.pipelinewrapper.DiffusionPipelineWrapper configurations, an error will be produced when it is unsupported. It is not known to be supported by pix2pix.

Optional SDXL conditioning parameter. To negatively condition the generation process based on a specific image resolution. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.

sdxl_negative_target_size: tuple[int, int] | None = None

This value is only supported for certain dgenerate.pipelinewrapper.DiffusionPipelineWrapper configurations, an error will be produced when it is unsupported. It is not known to be supported by pix2pix.

Optional SDXL conditioning parameter. To negatively condition the generation process based on a target image resolution. It should be as same as the DiffusionArguments.target_size for most cases. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.

sdxl_original_size: tuple[int, int] | None = None

Optional SDXL conditioning parameter. If DiffusionArguments.sdxl_original_size is not the same as DiffusionArguments.sdxl_target_size the image will appear to be down- or up-sampled. DiffusionArguments.sdxl_original_size defaults to (width, height) if not specified or the size of any input images provided. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

sdxl_refiner_aesthetic_score: float | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_aesthetic_score

sdxl_refiner_clip_skip: int | None = None

Clip skip override value for the SDXL refiner, which normally defaults to that of DiffusionArguments.clip_skip when it is defined.

sdxl_refiner_crops_coords_top_left: tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_crops_coords_top_left

sdxl_refiner_deep_cache: bool | None = None

Enable DeepCache acceleration for the SDXL Refiner?

This is supported for Stable Diffusion XL and Kolors based models.

sdxl_refiner_deep_cache_branch_id: int | None = None

Controls which branch ID DeepCache should operate on in the UNet for the SDXL Refiner.

This value must be greater than or equal to 0.

This is supported for Stable Diffusion XL and Kolors based models.

Supplying any value implies that DiffusionArguments.sdxl_refiner_deep_cache is enabled.

Defaults to 1.

sdxl_refiner_deep_cache_interval: int | None = None

Controls the frequency of caching intermediate outputs in DeepCache for the SDXL Refiner.

This value must be greater than zero.

This is supported for Stable Diffusion XL and Kolors based models.

Supplying any value implies that DiffusionArguments.sdxl_refiner_deep_cache is enabled.

Defaults to 5.

sdxl_refiner_edit: bool | None = None

Force the SDXL refiner to operate in edit mode instead of cooperative denoising mode.

sdxl_refiner_freeu_params: tuple[float, float, float, float] | None = None

FreeU parameters for the SDXL refiner

See: DiffusionArguments.freeu_params for clarification.

sdxl_refiner_guidance_rescale: float | None = None

Override the guidance rescale value used by the SDXL refiner, which is normally set to the value of DiffusionArguments.guidance_rescale.

sdxl_refiner_negative_aesthetic_score: float | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_aesthetic_score

sdxl_refiner_negative_crops_coords_top_left: tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_crops_coords_top_left

sdxl_refiner_negative_original_size: tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_original_size

sdxl_refiner_negative_target_size: tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_target_size

sdxl_refiner_original_size: tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_original_size

sdxl_refiner_pag_adaptive_scale: float | None = None

Adaptive perturbed attention guidance scale for the SDXL refiner.

sdxl_refiner_pag_scale: float | None = None

Perturbed attention guidance scale for the SDXL refiner.

sdxl_refiner_sigmas: Sequence[float] | str | None = None

Sigma values, this is supported when using a DiffusionArguments.second_model_scheduler_uri that supports setting sigmas.

These sigma values control the noise schedule specifically for the SDXL refiner’s diffusion process, allowing for customized denoising behavior during the refinement stage. This can be particularly useful for fine-tuning the level of detail and quality in the refined image.

Format: A list of floating point values in descending order, typically ranging from higher values (more noise) to lower values (less noise).

Or: a string expression involving sigmas from the selected scheduler such as sigmas * 0.95, sigmas will be represented as a numpy array, numpy is available through the namespace np, this uses asteval.

sdxl_refiner_target_size: tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_target_size

sdxl_t2i_adapter_factor: float | None = None

SDXL specific T2I adapter factor, this controls the amount of time-steps for which a T2I adapter applies guidance to an image, this is a value between 0.0 and 1.0. A value of 0.5 for example indicates that the T2I adapter is only active for half the amount of time-steps it takes to completely render an image.

sdxl_target_size: tuple[int, int] | None = None

Optional SDXL conditioning parameter. For most cases, DiffusionArguments.sdxl_target_size should be set to the desired height and width of the generated image. If not specified it will default to (width, height) or the size of any input images provided. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

second_model_guidance_scale: float | None = None

Override the guidance scale used by the SDXL refiner or Stable Cascade decoder.

second_model_inference_steps: int | None = None

Override the default amount of inference steps performed by the SDXL refiner or Stable Cascade decoder.

second_model_prompt: Prompt | None = None

Primary prompt for SDXL Refiner or Stable Cascade Decoder.

second_model_prompt_weighter_uri: str | None = None

The URI of a prompt-weighter implementation supported by dgenerate to use with the SDXL refiner or Stable Cascade Decoder.

Defaults to DiffusionArguments.prompt_weighter_uri if not specified.

This corresponds to the --second-model-prompt-weighter argument of the dgenerate command line tool.

second_model_scheduler_uri: str | None = None

SDXL refiner scheduler / Stable Cascade Decoder URI, if not specified, defaults to DiffusionArguments.scheduler

second_model_second_prompt: Prompt | None = None

Secondary Prompt for SDXL Refiner, the Stable Cascade Decoder does not support this argument.

second_prompt: Prompt | None = None

Secondary Prompt for SDXL, SD3, Flux.

seed: int | None = None

An integer to serve as an RNG seed.

sigmas: Sequence[float] | str | None = None

Sigma values, this is supported when using a when using a DiffusionArguments.scheduler_uri that supports setting sigmas.

Sigma values control the noise schedule in the diffusion process, allowing for fine-grained control over how noise is added and removed during image generation. Custom sigma values can be used to achieve specific artistic effects or to optimize the generation process for particular types of images.

Format: A list of floating point values in descending order, typically ranging from higher values (more noise) to lower values (less noise).

Or: a string expression involving sigmas from the selected scheduler such as sigmas * 0.95, sigmas will be represented as a numpy array, numpy is available through the namespace np, this uses asteval.

tea_cache: bool = False

Activate TeaCache for the primary model?

This is supported for Flux, TeaCache uses a novel caching mechanism in the forward pass of the flux transformer to reduce the amount of computation needed to generate an image, this can speed up inference with small amounts of quality loss.

See: https://github.com/ali-vilab/TeaCache

Also see: DiffusionArguments.tea_cache_rel_l1_threshold

This is supported for: --model-type flux*.

tea_cache_rel_l1_threshold: float | None = None

TeaCache relative L1 threshold when DiffusionArguments.tea_cache is enabled.

Higher values mean more speedup.

Defaults to 0.6 (2.0x speedup). 0.25 for 1.5x speedup, 0.4 for 1.8x speedup, 0.6 for 2.0x speedup, 0.8 for 2.25x speedup

See: https://github.com/ali-vilab/TeaCache

Supplying any value implies that DiffusionArguments.tea_cache is enabled.

This is supported for: --model-type flux*.

third_prompt: Prompt | None = None

Tertiary Prompt for SD3.

upscaler_noise_level: int | None = None

Upscaler noise level for the dgenerate.pipelinewrapper.ModelType.UPSCALER_X4 model type only.

vae_slicing: bool = False

Enable VAE slicing?

vae_tiling: bool = False

Enable VAE tiling?

width: int | None = None

Output image width.

Will be automatically aligned by 8.

If alignments of more than 8 need to be forced, a warning will be issued to stdout via dgenerate.messages.

class dgenerate.DiffusionPipelineWrapper(model_path: str, model_type: ModelType | str = ModelType.SD, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataType | str = DataType.AUTO, unet_uri: str | None = None, second_model_unet_uri: str | None = None, transformer_uri: str | None = None, vae_uri: str | None = None, lora_uris: Sequence[str] | None = None, lora_fuse_scale: float | None = None, image_encoder_uri: str | None = None, ip_adapter_uris: Sequence[str] | None = None, textual_inversion_uris: Sequence[str] | None = None, text_encoder_uris: Sequence[str] | None = None, second_model_text_encoder_uris: Sequence[str] | None = None, controlnet_uris: Sequence[str] | None = None, t2i_adapter_uris: Sequence[str] | None = None, sdxl_refiner_uri: str | None = None, s_cascade_decoder_uri: str | None = None, quantizer_uri: str | None = None, quantizer_map: Sequence[str] | None = None, second_model_quantizer_uri: str | None = None, second_model_quantizer_map: Sequence[str] | None = None, device: str = 'cpu', safety_checker: bool = False, original_config: str | None = None, second_model_original_config: str | None = None, auth_token: str | None = None, local_files_only: bool = False, model_extra_modules: dict[str, Any] = None, second_model_extra_modules: dict[str, Any] = None, model_cpu_offload: bool = False, model_sequential_offload: bool = False, second_model_cpu_offload: bool = False, second_model_sequential_offload: bool = False, prompt_weighter_loader: PromptWeighterLoader | None = None, latents_processor_loader: LatentsProcessorLoader | None = None, decoded_latents_image_processor_loader: ImageProcessorLoader | None = None, adetailer_detector_uris: Sequence[str] | None = None, adetailer_crop_control_image: bool = False)

Bases: object

Monolithic diffusion pipelines wrapper.

static recall_last_used_main_pipeline() → PipelineCreationResult | None

Return a reference to the last dgenerate.pipelinewrapper.pipelines.TorchPipelineCreationResult for the pipeline that successfully executed an image generation.

This may recreate the pipeline if it is not cached.

If no image generation has occurred, this will return None.

Returns:

dgenerate.pipelinewrapper.pipelines.TorchPipelineCreationResult or None

static recall_last_used_secondary_pipeline() → PipelineCreationResult | None

Return a reference to the last dgenerate.pipelinewrapper.pipelines.TorchPipelineCreationResult for the secondary pipeline (refiner / stable cascade decoder) that successfully executed an image generation.

This may recreate the pipeline if it is not cached.

If no image generation has occurred or no secondary pipeline has been called, this will return None.

Returns:

dgenerate.pipelinewrapper.pipelines.TorchPipelineCreationResult or None

__call__(args: DiffusionArguments | None = None, **kwargs) → PipelineWrapperResult

Call the pipeline and generate a result.

Parameters:

• args – Optional DiffusionArguments

• kwargs – See DiffusionArguments.get_pipeline_wrapper_kwargs(), any keyword arguments given here will override values derived from the DiffusionArguments object given to the args parameter.

Raises:

• InvalidModelFileError –

• InvalidModelUriError –

• InvalidSchedulerNameError –

• dgenerate.OutOfMemoryError –

• UnsupportedPipelineConfigError –

Returns:

PipelineWrapperResult

__init__(model_path: str, model_type: ModelType | str = ModelType.SD, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataType | str = DataType.AUTO, unet_uri: str | None = None, second_model_unet_uri: str | None = None, transformer_uri: str | None = None, vae_uri: str | None = None, lora_uris: Sequence[str] | None = None, lora_fuse_scale: float | None = None, image_encoder_uri: str | None = None, ip_adapter_uris: Sequence[str] | None = None, textual_inversion_uris: Sequence[str] | None = None, text_encoder_uris: Sequence[str] | None = None, second_model_text_encoder_uris: Sequence[str] | None = None, controlnet_uris: Sequence[str] | None = None, t2i_adapter_uris: Sequence[str] | None = None, sdxl_refiner_uri: str | None = None, s_cascade_decoder_uri: str | None = None, quantizer_uri: str | None = None, quantizer_map: Sequence[str] | None = None, second_model_quantizer_uri: str | None = None, second_model_quantizer_map: Sequence[str] | None = None, device: str = 'cpu', safety_checker: bool = False, original_config: str | None = None, second_model_original_config: str | None = None, auth_token: str | None = None, local_files_only: bool = False, model_extra_modules: dict[str, Any] = None, second_model_extra_modules: dict[str, Any] = None, model_cpu_offload: bool = False, model_sequential_offload: bool = False, second_model_cpu_offload: bool = False, second_model_sequential_offload: bool = False, prompt_weighter_loader: PromptWeighterLoader | None = None, latents_processor_loader: LatentsProcessorLoader | None = None, decoded_latents_image_processor_loader: ImageProcessorLoader | None = None, adetailer_detector_uris: Sequence[str] | None = None, adetailer_crop_control_image: bool = False)

This is a monolithic wrapper around all supported diffusion pipelines which handles txt2img, img2img, and inpainting on demand. It spins up the correct pipelines as needed in order to handle provided pipeline arguments using lazy initialization.

Pipelines and user specified sub models are memoized and their lifetimes are managed via heuristics based on system memory and available resources.

All arguments to this constructor should be provided as keyword arguments, using this constructor in any other fashion could result in breakage inbetween semver compatible versions.

Parameters:

• model_path – main model path

• model_type – main model type

• revision – main model revision

• variant – main model variant

• subfolder – main model subfolder (huggingface or disk)

• dtype – main model dtype

• unet_uri – main model UNet URI string

• second_model_unet_uri – secondary model unet uri (SDXL Refiner, Stable Cascade decoder)

• transformer_uri – Optional transformer URI string for specifying a specific Transformer, currently this is only supported for Stable Diffusion 3 models.

• vae_uri – main model VAE URI string

• lora_uris – One or more LoRA URI strings

• lora_fuse_scale – Optional global LoRA fuse scale value. Once all LoRAs are merged with their individual scales, the merged weights will be fused into the pipeline at this scale. The default value is 1.0.

• image_encoder_uri – One or more Image Encoder URI strings, Image Encoders are used with IP Adapters and Stable Cascade

• ip_adapter_uris – One or more IP Adapter URI strings

• textual_inversion_uris – One or more Textual Inversion URI strings

• text_encoder_uris – One or more Text Encoder URIs (“+”, or None for default. Or “null” indicating do not load) for the main model

• second_model_text_encoder_uris – One or more Text Encoder URIs (“+”, or None for default. Or “null” indicating do not load) for the secondary model (SDXL Refiner or Stable Cascade decoder)

• controlnet_uris – One or more ControlNet URI strings

• t2i_adapter_uris – One or more T2IAdapter URI strings

• sdxl_refiner_uri – SDXL Refiner model URI string

• s_cascade_decoder_uri – Stable Cascade decoder URI string

• quantizer_uri – Global –quantizer URI value

• quantizer_map – Collection of pipeline submodule names to which quantization should be applied when quantizer_uri is provided. Valid values include: unet, transformer, text_encoder, text_encoder_2, text_encoder_3. If None, all supported modules will be quantized.

• second_model_quantizer_uri – Global –second-model-quantizer URI value

• second_model_quantizer_map – Collection of pipeline submodule names to which quantization should be applied when second_model_quantizer_uri is provided. Valid values include: unet, transformer, text_encoder, text_encoder_2, text_encoder_3. If None, all supported modules will be quantized.

• device – Rendering device string, example: cuda:0 or cuda

• safety_checker – Use safety checker model if available? (antiquated, for SD 1/2, Deep Floyd etc.)

• original_config – Optional original LDM config .yaml file path when loading a single file checkpoint.

• second_model_original_config – Optional original LDM config .yaml file path when loading a single file checkpoint for the secondary model (SDXL Refiner, Stable Cascade Decoder).

• auth_token – huggingface authentication token.

• local_files_only – Do not attempt to download files from huggingface?

• model_extra_modules – Raw extra diffusers modules for the main pipeline

• second_model_extra_modules – Raw extra diffusers modules for the secondary pipeline (SDXL Refiner, Stable Cascade decoder)

• model_cpu_offload – Use model CPU offloading for the main pipeline via the accelerate module?

• model_sequential_offload – Use sequential CPU offloading for the main pipeline via the accelerate module?

• second_model_cpu_offload – Use CPU offloading for the SDXL Refiner or Stable Cascade Decoder via the accelerate module?

• second_model_sequential_offload – Use sequential CPU offloading for the SDXL Refiner or Stable Cascade Decoder via the accelerate module?

• prompt_weighter_loader – Plugin loader for prompt weighter implementations, if you pass None a default instance will be created.

• latents_processor_loader – Plugin loader for latents processor implementations, if you pass None a default instance will be created.

• decoded_latents_image_processor_loader – Plugin loader for image processor implementations that process images decoded from incoming latents, if you pass None a default instance will be created.

• adetailer_detector_uris – adetailer subject detection model URIs, specifying this argument indicates img2img mode implicitly, the pipeline wrapper will accept a single image and perform the adetailer inpainting algorithm on it using the provided detector URIs.

• adetailer_crop_control_image – Should adetailer crop any provided ControlNet control image in the same way that it crops the generated mask to the detection area? Otherwise, use the full control image resized down to the size of the detection area. If you enable this and your control image is not the same size as your input image, a warning will be issued and resizing will be used instead of cropping.

Raises:

• UnsupportedPipelineConfigError –

• InvalidModelUriError –

decode_latents(latents: Sequence[Tensor] | Tensor) → list[Image]

Decode latents using the main pipeline’s VAE.

A generation must have occurred at least once for this method to be usable.

You must be using a model type that utilizes a VAE, Stable Cascade and Deep Floyd model types are not supported by this method.

Parameters:

latents – Latents to decode, can be a sequence of tensors (batched), or a single tensor. A single tensor with a batch dimension [B, C, H, W] will be assumed to be a batch of latents and batched if the batch dimension is > 1, [C, H, W] will be assumed to be a single latent tensor. For Flux models, latents should be in unpacked format [B, C, H, W] where C=16.

Raises:

dgenerate.pipelinewrapper.UnsupportedPipelineConfigError – If the decoding the latents is not supported.

gen_dgenerate_command(args: DiffusionArguments | None = None, extra_opts: Sequence[tuple[str] | tuple[str, Any]] | None = None, omit_device: bool = False, overrides: dict[str, Any] = None)

Generate a valid dgenerate command line invocation that reproduces the arguments associated with DiffusionArguments.

This does not reproduce --image-seeds, you must include that value in extra_opts, this is because there is not enough information in DiffusionArguments to accurately reproduce it.

Parameters:

• args – DiffusionArguments object to take values from

• extra_opts – Extra option pairs to be added to the end of reconstructed options of the dgenerate invocation, this should be a sequence of tuples of length 1 (switch only) or length 2 (switch with args)

• omit_device – Omit the --device option? For a shareable configuration it might not make sense to include the device specification. And instead simply fallback to whatever the default device is, which is generally cuda

• overrides – pipeline wrapper keyword arguments, these will override values derived from any DiffusionArguments object given to the args argument. See: DiffusionArguments.get_pipeline_wrapper_kwargs

Returns:

A string containing the dgenerate command line needed to reproduce this result.

gen_dgenerate_config(args: DiffusionArguments | None = None, extra_opts: Sequence[tuple[str] | tuple[str, Any]] | None = None, extra_comments: Iterable[str] | None = None, omit_device: bool = False, overrides: dict[str, Any] = None)

Generate a valid dgenerate config file with a single invocation that reproduces the arguments associated with DiffusionArguments.

This does not reproduce --image-seeds, you must include that value in extra_opts, this is because there is not enough information in DiffusionArguments to accurately reproduce it.

Parameters:

• args – DiffusionArguments object to take values from

• extra_opts – Extra option pairs to be added to the end of reconstructed options of the dgenerate invocation, this should be a sequence of tuples of length 1 (switch only) or length 2 (switch with args)

• extra_comments – Extra strings to use as comments after the initial version check directive

• omit_device – Omit the --device option? For a shareable configuration it might not make sense to include the device specification. And instead simply fallback to whatever the default device is, which is generally cuda

• overrides – pipeline wrapper keyword arguments, these will override values derived from any DiffusionArguments object given to the args argument. See: DiffusionArguments.get_pipeline_wrapper_kwargs

Returns:

The configuration as a string

get_decoded_latents_size(latents: Tensor) → tuple[int, int]

Given a latent tensor return the expected decoded image (width, height) in pixels.

Parameters:

latents – Latent tensor of shape [B, C, H, W] or [C, H, W].

Returns:

width, height

recall_main_pipeline() → PipelineCreationResult

Fetch the last used main pipeline creation result, possibly the pipeline will be recreated if no longer in the in memory cache. If there is no pipeline currently created, which will be the case if an image was never generated yet, RuntimeError will be raised.

Raises:

RuntimeError –

Returns:

dgenerate.pipelinewrapper.PipelineCreationResult

recall_secondary_pipeline() → PipelineCreationResult

Fetch the last used refiner / stable cascade decoder pipeline creation result, possibly the pipeline will be recreated if no longer in the in memory cache. If there is no refiner / decoder pipeline currently created, which will be the case if an image was never generated yet or a refiner / decoder model was not specified, RuntimeError will be raised.

Raises:

RuntimeError –

Returns:

dgenerate.pipelinewrapper.PipelineCreationResult

reconstruct_dgenerate_opts(args: DiffusionArguments | None = None, extra_opts: Sequence[tuple[str] | tuple[str, Any]] | None = None, omit_device: bool = False, shell_quote: bool = True, overrides: dict[str, Any] = None) → list[tuple[str] | tuple[str, Any]]

Reconstruct dgenerate’s command line arguments from a particular set of pipeline wrapper call arguments.

This does not reproduce --image-seeds, you must include that value in extra_opts, this is because there is not enough information in DiffusionArguments to accurately reproduce it.

Parameters:

• args – DiffusionArguments object to take values from

• extra_opts – Extra option pairs to be added to the end of reconstructed options, this should be a sequence of tuples of length 1 (switch only) or length 2 (switch with args)

• omit_device – Omit the --device option? For a shareable configuration it might not make sense to include the device specification. And instead simply fallback to whatever the default device is, which is generally cuda

• shell_quote – Shell quote and format the argument values? or return them raw.

• overrides – pipeline wrapper keyword arguments, these will override values derived from any DiffusionArguments object given to the args argument. See: DiffusionArguments.get_pipeline_wrapper_kwargs

Returns:

List of tuples of length 1 or 2 representing the option

property adetailer_crop_control_image: bool

Should adetailer crop any provided control image in the same way that it crops the generated mask to the detection area? Otherwise, use the full control image resized down to the size of the detection area.

property adetailer_detector_uris: Sequence[str] | None

List of supplied --adetailer-detectors URI strings or an empty list.

property auth_token: str | None

Current --auth-token value or None.

property controlnet_uris: Sequence[str] | None

List of supplied --control-nets URI strings or an empty list.

property decoded_latents_image_processor_loader: ImageProcessorLoader

Current decoded latents image processor loader.

property device: str

Currently set --device string.

property dtype: DataType

Currently set --dtype enum value for the main model.

property dtype_string: str

Currently set --dtype string value for the main model.

property image_encoder_uri: str | None

Selected --image-encoder uri for the main model or None.

property ip_adapter_uris: Sequence[str] | None

List of supplied --ip-adapters URI strings or an empty list.

property latents_processor_loader: LatentsProcessorLoader

Current latents processor loader.

property local_files_only: bool

Currently set value for local_files_only.

property lora_fuse_scale: float

Supplied --lora-fuse-scale value.

property lora_uris: Sequence[str] | None

List of supplied --loras uri strings or an empty list.

property model_cpu_offload: bool

Current --model-cpu-offload value.

property model_path: str

Model path for the main model.

property model_sequential_offload: bool

Current --model-sequential-offload value.

property model_type: ModelType

Currently set --model-type enum value.

property model_type_string: str

Currently set --model-type string value.

property original_config: str | None

Current --original-config value.

property prompt_weighter_loader: PromptWeighterLoader

Current prompt weighter loader.

property quantizer_map: Sequence[str] | None

Current --quantizer-map value.

property quantizer_uri: str | None

Current --quantizer value.

property revision: str | None

Currently set --revision for the main model or None.

property s_cascade_decoder_uri: str | None

Model URI for the Stable Cascade decoder or None.

property safety_checker: bool

Safety checker enabled status.

property sdxl_refiner_uri: str | None

Model URI for the SDXL refiner or None.

property second_model_cpu_offload: bool

Current --second-model-cpu-offload value.

property second_model_original_config: str | None

Current --second-model-original-config value.

property second_model_quantizer_map: Sequence[str] | None

Current --second-model-quantizer-map value.

property second_model_quantizer_uri: str | None

Current --second-model-quantizer value.

property second_model_sequential_offload: bool

Current --second-model-sequential-offload value.

property second_model_text_encoder_uris: Sequence[str] | None

List of supplied --second-model-text-encoders URI strings or an empty list.

property second_model_unet_uri: str | None

Selected --second-model-unet uri for the SDXL refiner or Stable Cascade decoder model or None.

property subfolder: str | None

Selected model --subfolder for the main model, (remote repo subfolder or local) or None.

property t2i_adapter_uris: Sequence[str] | None

List of supplied --t2i-adapters URI strings or an empty list.

property text_encoder_uris: Sequence[str] | None

List of supplied --text-encoders URI strings or an empty list.

property textual_inversion_uris: Sequence[str] | None

List of supplied --textual-inversions URI strings or an empty list.

property transformer_uri: str | None

Model URI for the SD3 Transformer or None.

property unet_uri: str | None

Selected --unet uri for the main model or None.

property vae_uri: str | None

Selected --vae uri for the main model or None.

property variant: str | None

Currently set --variant for the main model or None.

class dgenerate.ImageFileSavedEvent(origin: RenderLoop, generated_event: ImageGeneratedEvent, path: str, config_filename: str | None = None)

Bases: Event

Generated in the event stream of RenderLoop.events()

Occurs when an image file is written to disk.

__init__(origin: RenderLoop, generated_event: ImageGeneratedEvent, path: str, config_filename: str | None = None)

config_filename: str | None = None

Path to a dgenerate config file if output_configs is enabled.

generated_event: ImageGeneratedEvent

The ImageGeneratedEvent for the image that was saved.

path: str

Path to the saved image.

class dgenerate.ImageGeneratedEvent(origin: RenderLoop, image: Image | None, latents: Tensor | None, generation_step: int, batch_index: int, suggested_directory: str, suggested_filename: str, diffusion_args: DiffusionArguments, image_seed: ImageSeed, command_string: str, config_string: str)

Bases: Event

Generated in the event stream of RenderLoop.events()

Occurs when an image is generated (but not saved yet).

__init__(origin: RenderLoop, image: Image | None, latents: Tensor | None, generation_step: int, batch_index: int, suggested_directory: str, suggested_filename: str, diffusion_args: DiffusionArguments, image_seed: ImageSeed, command_string: str, config_string: str)

batch_index: int

The index in the image batch for this image. Will only every be greater than zero if RenderLoopConfig.batch_size > 1 and RenderLoopConfig.batch_grid_size is None.

command_string: str

Reproduction of a command line that can be used to reproduce this image.

This does not include the --device argument.

config_string: str

Reproduction of a dgenerate config file that can be used to reproduce this image.

This does not include the --device argument.

diffusion_args: DiffusionArguments

Diffusion argument object, contains dgenerate.pipelinewrapper.DiffusionPipelineWrapper arguments used to produce this image.

property frame_index: int | None

The frame index if this is an animation frame. Also available through image_seed.frame_index, though here for convenience.

generation_step: int

The current generation step. (zero indexed)

image: Image | None

The generated image. Will be None if latent output is being used.

image_seed: ImageSeed | None

If an --image-seeds specification was used in the generation of this image, this object represents that image seed and contains the images that contributed to the generation of this image.

property is_animation_frame: bool

Is this image a frame in an animation?

property is_image_output: bool

Is this event representing image output?

property is_latents: bool

Is this event representing latents tensor output?

latents: Tensor | None

The generated latents tensor. Will be None if image output is being used.

suggested_directory: str

A suggested directory path for saving this image in.

A value of '.' may be present, this indicates the current working directory.

suggested_filename: str

A suggested filename for saving this image as. This filename will be unique to the render loop run / configuration. This filename will not contain RenderLoopConfig.output_path, it is the suggested filename by itself.

class dgenerate.ModelType(value)

Bases: Enum

Enum representation of --model-type

FLUX = 13

Flux pipeline

FLUX_FILL = 14

Flux infill / outfill pipeline

FLUX_KONTEXT = 15

Flux Kontext pipeline

IF = 3

Deep Floyd IF stage 1

IFS = 4

Deep Floyd IF superscaler (stage 2)

IFS_IMG2IMG = 5

Deep Floyd IF superscaler (stage 2) image to image / variation mode.

KOLORS = 16

Kolors (SDXL + ChatGLM)

PIX2PIX = 1

Stable Diffusion pix2pix prompt guided editing.

SD = 0

Stable Diffusion, such as SD 1.0 - 2.x

SD3 = 11

Stable Diffusion 3

SD3_PIX2PIX = 12

Stable Diffusion 3 pix2pix prompt guided editing.

SDXL = 2

Stable Diffusion XL

SDXL_PIX2PIX = 6

Stable Diffusion XL pix2pix prompt guided editing.

S_CASCADE = 9

Stable Cascade prior

S_CASCADE_DECODER = 10

Stable Cascade decoder

UPSCALER_X2 = 7

Stable Diffusion X2 upscaler

UPSCALER_X4 = 8

Stable Diffusion X4 upscaler

class dgenerate.PipelineType(value)

Bases: Enum

Represents possible diffusers pipeline types.

IMG2IMG = 2

Image to image mode. Generation seeded / controlled with an image in some fashion.

INPAINT = 3

Inpainting mode. Generation seeded / controlled with an image and a mask in some fashion.

TXT2IMG = 1

Text to image mode. Prompt only generation.

class dgenerate.Prompt(positive: str | None = None, negative: str | None = None, delimiter: str = ';', weighter: str | None = None, upscaler: str | Sequence[str] | None = None, embedded_args: dict[str, str] | None = None)

Bases: object

Represents a combined positive and optional negative prompt split by a delimiter character.

static copy(prompt: Prompt)

Return a copy of another prompt.

Parameters:

prompt – The prompt to copy.

Returns:

A copy of the provided prompt.

static parse(value: str, delimiter=';', parse_embedded_args: bool = True, embedded_arg_names: list[str] | None = None) → Prompt

Parse the positive and negative prompt from a string and return a prompt object.

Parameters:

• value – the string

• delimiter – The prompt delimiter character

• parse_embedded_args – parse embedded args? < arg: value >

• embedded_arg_names – list of embedded argument names to parse, if None, all are parsed.

Raises:

ValueError – if value is None

Returns:

Prompt (returns self)

__init__(positive: str | None = None, negative: str | None = None, delimiter: str = ';', weighter: str | None = None, upscaler: str | Sequence[str] | None = None, embedded_args: dict[str, str] | None = None)

Parameters:

• positive – positive prompt component.

• negative – negative prompt component.

• delimiter – delimiter for stringification.

• weighter – --prompt-weighter plugin URI.

• upscaler – --prompt-upscaler plugin URI.

• embedded_args – embedded prompt arguments parsed from <argument: value_text>.

copy_embedded_args_from(prompt: Prompt)

set_embedded_args_on(on_object: Any, forbidden_checker: Callable[[str, Any], bool] | None = None, validate_only: bool = False)

Set the other embedded arguments parsed out of a prompt on to an object.

The object should be type hinted using types from dgenerate.types

Specifically, any of:

• dgenerate.types.Size

• dgenerate.types.Sizes

• dgenerate.types.OptionalSize

• dgenerate.types.OptionalSizes

• dgenerate.types.Padding

• dgenerate.types.Paddings

• dgenerate.types.OptionalPadding

• dgenerate.types.OptionalPaddings

• dgenerate.types.Boolean

• dgenerate.types.OptionalBoolean

• dgenerate.types.Float

• dgenerate.types.Floats

• dgenerate.types.OptionalFloat

• dgenerate.types.OptionalFloats

• dgenerate.types.Integer

• dgenerate.types.Integers

• dgenerate.types.OptionalInteger

• dgenerate.types.OptionalIntegers

• dgenerate.types.String

• dgenerate.types.Strings

• dgenerate.types.OptionalString

• dgenerate.types.OptionalStrings

• dgenerate.types.Name

• dgenerate.types.Names

• dgenerate.types.OptionalName

• dgenerate.types.OptionalNames

• dgenerate.types.Uri

• dgenerate.types.Uris

• dgenerate.types.OptionalUri

• dgenerate.types.OptionalUris

Raises:

PromptEmbeddedArgumentError – If there was a problem applying the embedded arguments to the object.

Parameters:

• on_object – The object to set values on.

• forbidden_checker – This is a function that should return True if an argument name / value is forbidden to use.

• validate_only – Only run validation and do not set any values?

property delimiter: str

Positive / Negative delimiter for this prompt, for example “;”

property embedded_args: list[tuple[str, str]]

Other embedded arguments parsed out of the prompt.

property negative: str | None

Negative prompt value.

property positive: str | None

Positive prompt value.

property upscaler: str | Sequence[str] | None

Embedded prompt upscaler URI argument for this prompt if any.

property weighter: str | None

Embedded prompt weighter URI argument for this prompt if any.

class dgenerate.RenderLoop(config: RenderLoopConfig | None = None, image_processor_loader: ImageProcessorLoader | None = None, prompt_weighter_loader: PromptWeighterLoader | None = None, model_extra_modules: dict[str, Any] = None, second_model_extra_modules: dict[str, Any] = None, disable_writes: bool = False)

Bases: object

Render loop which implements the bulk of dgenerate’s rendering capability.

This object handles the scatter gun iteration over requested diffusion parameters, the generation of animations, and writing images and media to disk or providing those to library users through callbacks.

__init__(config: RenderLoopConfig | None = None, image_processor_loader: ImageProcessorLoader | None = None, prompt_weighter_loader: PromptWeighterLoader | None = None, model_extra_modules: dict[str, Any] = None, second_model_extra_modules: dict[str, Any] = None, disable_writes: bool = False)

Parameters:

• config – RenderLoopConfig or dgenerate.arguments.DgenerateArguments. If None is provided, a RenderLoopConfig instance will be created and assigned to RenderLoop.config.

• image_processor_loader – dgenerate.imageprocessors.ImageProcessorLoader. If None is provided, an instance will be created and assigned to RenderLoop.image_processor_loader.

• prompt_weighter_loader – dgenerate.promptweighters.PromptWeighterLoader. If None is provided, an instance will be created and assigned to RenderLoop.prompt_weighter_loader.

• model_extra_modules – Extra raw diffusers modules to use in the creation of the main model pipeline.

• second_model_extra_modules – Extra raw diffusers modules to use in the creation of any refiner or stable cascade decoder model pipeline.

• disable_writes – Disable or enable all writes to disk, if you intend to only ever use the event stream of the render loop when using dgenerate as a library, this is a useful option. RenderLoop.written_images and RenderLoop.written_animations will not be available if writes to disk are disabled.

events() → Generator[ImageGeneratedEvent | StartingAnimationEvent | StartingAnimationFileEvent | AnimationFileFinishedEvent | ImageFileSavedEvent | AnimationFinishedEvent | StartingGenerationStepEvent | AnimationETAEvent, None, None]

Run the render loop, and iterate over a stream of event objects produced by the render loop.

This calls RenderLoopConfig.check() on a copy of your configuration prior to running.

Event objects are of the union type RenderLoopEvent

The exceptions mentioned here are those you may encounter upon iterating, they will not occur upon simple acquisition of the event stream iterator.

Raises:

• RenderLoopConfigError –

• dgenerate.ModelNotFoundError –

• dgenerate.OutOfMemoryError –

Returns:

RenderLoopEventStream

run()

Run the diffusion loop, this calls RenderLoopConfig.check() on a copy of your configuration prior to running.

Raises:

• RenderLoopConfigError –

• dgenerate.ModelNotFoundError –

• dgenerate.OutOfMemoryError –

config: RenderLoopConfig

Render loops generation related configuration.

disable_writes: bool = False

Disable or enable all writes to disk, if you intend to only ever use the event stream of the render loop when using dgenerate as a library, this is a useful option.

RenderLoop.written_images and RenderLoop.written_animations will not be available if writes to disk are disabled.

property generation_step

Returns the current generation step, (zero indexed)

image_processor_loader: ImageProcessorLoader

Responsible for loading any image processors referenced in the render loop configuration.

model_extra_modules: dict[str, Any] = None

Extra raw diffusers modules to use in the creation of the main model pipeline.

property pipeline_wrapper: DiffusionPipelineWrapper

Get the last used dgenerate.pipelinewrapper.DiffusionPipelineWrapper instance.

Will be None if RenderLoop.run() has never been called.

Returns:

dgenerate.pipelinewrapper.DiffusionPipelineWrapper or None

prompt_weighter_loader: PromptWeighterLoader

Responsible for loading any prompt weighters referenced in the render loop configuration.

second_model_extra_modules: dict[str, Any] = None

Extra raw diffusers modules to use in the creation of any refiner or stable cascade decoder model pipeline.

property written_animations: Iterable[str]

Iterable over animation filenames written by the last run

property written_images: Iterable[str]

Iterable over image filenames written by the last run

class dgenerate.RenderLoopConfig

Bases: SetFromMixin

This object represents configuration for RenderLoop.

It nearly directly maps to dgenerate’s command line arguments.

See subclass dgenerate.arguments.DgenerateArguments

__init__()

apply_prompt_upscalers()

Apply requested prompt upscaling operations to all prompts in the configuration.

This potentially modifies the configuration in place, specifically the prompt arguments.

Raises:

• dgenerate.promptupscalers.PromptUpscalerNotFoundError –

• dgenerate.promptupscalers.PromptUpscalerArgumentError –

• dgenerate.promptupscalers.PromptUpscalerProcessingError –

calculate_generation_steps() → int

Calculate the number of generation steps that this configuration results in.

This factors in diffusion parameter combinations as well as scheduler combinations.

Returns:

int

check(attribute_namer: Callable[[str], str] | None = None)

Check the configuration for type and logical usage errors, set defaults for certain values when needed and not specified.

This may modify the configuration.

Parameters:

attribute_namer – Callable for naming attributes mentioned in exception messages

copy() → RenderLoopConfig

Create a deep copy of this RenderLoopConfig instance.

Returns:

RenderLoopConfig instance that is a deep copy of this instance.

is_output_latents() → bool

Check if the current image_format results in outputting latents.

Returns:

True if the image output format indicates to output latents.

iterate_diffusion_args(**overrides) → Iterator[DiffusionArguments]

Iterate over dgenerate.pipelinewrapper.DiffusionArguments argument objects using every combination of argument values provided for that object by this configuration.

Parameters:

overrides – use key word arguments to override specific attributes of this object with a new list value.

Returns:

an iterator over dgenerate.pipelinewrapper.DiffusionArguments

adetailer_class_filter: Collection[int | str] | None = None

A collection of class IDs and/or class names that indicates what YOLO detection classes to keep. This filter is applied before index-filter. Detections that don’t match any of the specified classes will be ignored. Integers are treated as ID’s, strings are treated as names.

adetailer_crop_control_image: bool | None = None

Should adetailer crop any control image the same way that it crops the mask?

This is only relevant when using adetailer with ControlNet models.

When enabled, control images will be cropped to match the detected region before being passed to the inpainting pipeline. This can help ensure that the control guidance is properly aligned with the area being inpainted.

When disabled (default), control images will be resized to match the cropped region size without cropping.

This corresponds to the --adetailer-crop-control-image argument of the dgenerate command line tool.

adetailer_detector_paddings: Sequence[int | tuple[int, int] | tuple[int, int, int, int]] | None = None

One or more adetailer detector padding values.

This value specifies the amount of padding that will be added to the detection rectangle which is used to generate a masked area. The default is 0, you can make the mask area around the detected feature larger with positive padding and smaller with negative padding.

Example:

32 (32px Uniform, all sides)

(10, 20) (10px Horizontal, 20px Vertical)

(10, 20, 30, 40) (10px Left, 20px Top, 30px Right, 40px Bottom)

Defaults to [0].

adetailer_detector_uris: Sequence[str] | None = None

One or more adetailer YOLO detector model URIs. Corresponds directly to –adetailer-detectors.

Specification of this argument enables the adetailer inpainting algorithm and requires the use of RenderLoopConfig.image_seeds

adetailer_index_filter: Collection[int] | None = None

A list index values that indicates what YOLO detection indices to keep, the index values start at zero. Detections are sorted by their top left bounding box coordinate from left to right, top to bottom, by (confidence descending). The order of detections in the image is identical to the reading order of words on a page (english). Inpainting will only be performed on the specified detection indices, if no indices are specified, then inpainting will be performed on all detections. This filter is applied after class-filter.

adetailer_mask_blurs: Sequence[int] | None = None

Indicates the level of gaussian blur to apply to the inpaint mask generated by adetailer, which can help with smooth blending of the inpainted feature. Defaults to [4].

adetailer_mask_dilations: Sequence[int] | None = None

Indicates the amount of dilation applied to the generated adetailer inpaint mask, see: cv2.dilate. Defaults to [4].

adetailer_mask_paddings: Sequence[int | tuple[int, int] | tuple[int, int, int, int]] | None = None

One or more adetailer mask padding values.

This value indicates how much padding to place around the masked area when cropping out the image to be inpainted, this value must be large enough to accommodate any feathering on the edge of the mask caused by RenderLoopConfig.adetailer_mask_blurs or RenderLoopConfig.adetailer_mask_dilations for the best result.

Example:

32 (32px Uniform, all sides)

(10, 20) (10px Horizontal, 20px Vertical)

(10, 20, 30, 40) (10px Left, 20px Top, 30px Right, 40px Bottom)

Defaults to [32].

adetailer_mask_shapes: Sequence[str] | None = None

One or more adetailer mask shapes to try.

This indicates what mask shape adetailer should attempt to draw around a detected feature, the default value is “rectangle”. You may also specify “circle” to generate an ellipsoid shaped mask, which might be helpful for achieving better blending.

Valid values are: (“r”, “rect”, “rectangle”), or (“c”, “circle”, “ellipse”)

adetailer_model_masks: bool | None = None

Indicates that masks generated by the model itself should be preferred over masks generated from the detection bounding box. If this is True, and the model itself returns mask data, RenderLoopConfig.adetailer_mask_shape, RenderLoopConfig.adetailer_mask_padding, and RenderLoopConfig.adetailer_detector_padding will all be ignored.

adetailer_sizes: Sequence[int] | None = None

One or more target sizes for processing detected areas. When specified, detected areas will always be scaled to this target size (with aspect ratio preserved) for processing, then scaled back to the original size for compositing. This can significantly improve detail quality for small detected features like faces or hands, or reduce processing time for overly large detected areas. The scaling is based on the larger dimension (width or height) of the detected area. The optimal resampling method is automatically selected for both upscaling and downscaling. Each value must be an integer greater than 1. Defaults to none (process at native resolution).

animation_format: str = 'mp4'

Format for any rendered animations, see: dgenerate.mediaoutput.supported_animation_writer_formats(). This value may also be set to ‘frames’ to indicate that only individual frames should be output and no animation file coalesced. This corresponds to the --animation-format argument of the dgenerate command line tool.

auth_token: str | None = None

Optional huggingface API token which will allow the download of restricted repositories that your huggingface account has been granted access to. This corresponds to the --auth-token argument of the dgenerate command line tool.

batch_grid_size: tuple[int, int] | None = None

Optional image grid size specification for when batch_size is greater than one. This is the --batch-grid-size argument of the dgenerate command line tool.

batch_size: int | None = None

Image generation batch size, --batch-size argument of dgenerate command line tool.

clip_skips: Sequence[int] | None = None

List of clip skip values. Clip skip is the number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Only supported for model_type values sd and sdxl, including with controlnet_uris defined.

control_image_processors: Sequence[str] | None = None

Corresponds to the --control-image-processors argument of the dgenerate command line tool verbatim, including the grouping syntax using the “+” symbol, the plus symbol should be used as its own list element, IE: it is a token.

controlnet_uris: Sequence[str] | None = None

Optional user specified ControlNet URIs, this corresponds to the --control-nets argument of the dgenerate command line tool.

deep_cache: bool = False

Activate DeepCache for the main model?

DeepCache caches intermediate attention layer outputs to speed up the diffusion process. This is beneficial for higher inference steps.

See: https://github.com/horseee/DeepCache

This is supported for Stable Diffusion, Stable Diffusion XL, Stable Diffusion Upscaler X4, Kolors, and Pix2Pix variants.

deep_cache_branch_ids: Sequence[int] | None = None

Branch IDs to try for DeepCache for the main model.

Controls which branches of the UNet attention blocks the caching is applied to. Advanced usage only.

This value must be greater than or equal to 0.

Each value supplied will be tried in turn.

This is supported for Stable Diffusion, Stable Diffusion XL, Stable Diffusion Upscaler X4, Kolors, and Pix2Pix variants.

Supplying any value implies that RenderLoopConfig.deep_cache is enabled.

(default: 1)

deep_cache_intervals: Sequence[int] | None = None

Cache intervals to try for DeepCache for the main model.

Controls how frequently the attention layers are cached during the diffusion process. Lower values cache more frequently, potentially resulting in more speedup but using more memory.

This value must be greater than zero.

Each value supplied will be tried in turn.

This is supported for Stable Diffusion, Stable Diffusion XL, Stable Diffusion Upscaler X4, Kolors, and Pix2Pix variants.

Supplying any value implies that RenderLoopConfig.deep_cache is enabled.

(default: 5)

denoising_end: float | None = None

Denoising should end at this fraction of total timesteps (0.0 to 1.0).

This is useful for generating noisy latents that can be saved and passed to other models.

Scheduler Compatibility:

• SD 1.5 models: Only stateless schedulers are supported (EulerDiscreteScheduler, LMSDiscreteScheduler, EDMEulerScheduler, DPMSolverMultistepScheduler, DDIMScheduler, DDPMScheduler, PNDMScheduler)

• SDXL models: All schedulers supported via native denoising_start/denoising_end

• SD3/Flux models: FlowMatchEulerDiscreteScheduler and standard schedulers supported

This corresponds to the --denoising-end argument of the dgenerate command line tool.

denoising_start: float | None = None

Denoising should start at this fraction of total timesteps (0.0 to 1.0).

This is useful continuing denoising on noisy latents generated with RenderLoopConfig.denoising_end

Scheduler Compatibility:

• SD 1.5 models: Only stateless schedulers are supported (EulerDiscreteScheduler, LMSDiscreteScheduler, EDMEulerScheduler, DPMSolverMultistepScheduler, DDIMScheduler, DDPMScheduler, PNDMScheduler)

• SDXL models: All schedulers supported via native denoising_start/denoising_end

• SD3/Flux models: FlowMatchEulerDiscreteScheduler and standard schedulers supported

This corresponds to the --denoising-start argument of the dgenerate command line tool.

device: str = 'cpu'

Processing device specification, for example “cuda” or “cuda:N” where N is an alternate GPU id as reported by nvidia-smi if you want to specify a specific GPU. This corresponds to the --device argument of the dgenerate command line tool.

The default device on MacOS is “mps”.

“xpu” is an option for intel GPUs, for which device indices are also supported.

dtype: DataType = 0

Primary model data type specification, IE: integer precision. Default is auto selection. Lower precision datatypes result in less GPU memory usage. This corresponds to the --dtype argument of the dgenerate command line tool.

frame_end: int | None = None

Optional end frame inclusive frame slice for any rendered animations. This corresponds to the --frame-end argument of the dgenerate command line tool.

frame_start: int = 0

Start frame inclusive frame slice for any rendered animations. This corresponds to the --frame-start argument of the dgenerate command line tool.

freeu_params: Sequence[tuple[float, float, float, float]] | None = None

FreeU is a technique for improving image quality by re-balancing the contributions from the UNet’s skip connections and backbone feature maps.

This can be used with no cost to performance, to potentially improve image quality.

This argument can be used to specify The FreeU parameters: s1, s2, b1, and b2 in that order.

You can specify the FreeU parameters as a list / sequence of tuples that will be tried in turn for generation.

This argument only applies to models that utilize a UNet: SD1.5/2, SDXL, and Kolors

See: https://huggingface.co/docs/diffusers/main/en/using-diffusers/freeu

And: https://github.com/ChenyangSi/FreeU

guidance_rescales: Sequence[float] | None = None

List of floating point guidance rescale values which are supported by some pipelines, (there will be an error if it is unsupported upon running), this corresponds to the --guidance-rescales argument of the dgenerate command line tool.

guidance_scales: Sequence[float]

List of floating point guidance scales, this corresponds to the --guidance-scales argument of the dgenerate command line tool.

hi_diffusion: bool = False

Activate HiDiffusion for the primary model?

This can increase the resolution at which the model can output images while retaining quality with no overhead, and possibly improved performance.

See: https://github.com/megvii-research/HiDiffusion

This is supported for: --model-type sd, sdxl, and kolors.

hi_diffusion_no_raunet: bool | None = None

Disable RAU-Net when using HiDiffusion for the primary model?

This disables the Resolution-Aware U-Net component of HiDiffusion.

See: https://github.com/megvii-research/HiDiffusion

This is supported for: --model-type sd, sdxl, and kolors.

hi_diffusion_no_win_attn: bool | None = None

Disable window attention when using HiDiffusion for the primary model?

This disables the MSW-MSA (Multi-Scale Window Multi-Head Self-Attention) component of HiDiffusion.

See: https://github.com/megvii-research/HiDiffusion

This is supported for: --model-type sd, sdxl, and kolors.

image_encoder_uri: str | None = None

Optional user specified Image Encoder URI when using IP Adapter models or Stable Cascade. This corresponds to the --image-encoder argument of the dgenerate command line tool.

If none of your specified --ip-adapters URIs point to a model which contains an Image Encoder model, you will need to specify one manually using this argument.

image_format: str = 'png'

Format for any images that are written including animation frames.

Anything other than “png”, “jpg”, or “jpeg” is not compatible with output_metadata=True and a RenderLoopConfigError will be raised upon running the render loop if output_metadata=True and this value is not one of those mentioned formats.

image_guidance_scales: Sequence[float] | None = None

Optional list of floating point image guidance scales, used for pix2pix model types, this corresponds to the --image-guidance-scales argument of the dgenerate command line tool.

image_seed_strengths: Sequence[float] | None = None

Optional list of floating point image seed strengths, this corresponds to the --image-seed-strengths argument of the dgenerate command line tool.

image_seeds: Sequence[str] | None = None

List of --image-seeds URI strings.

img2img_latents_processors: Sequence[str] | None = None

One or more latents processor URI strings for processing img2img latents before pipeline execution.

These processors are applied to latent tensors provided through the RenderLoopConfig.image_seeds argument when doing img2img with tensor inputs. The processors are applied in sequence and may occur before VAE decoding (for models that decode img2img latents) or before direct pipeline usage.

This corresponds to the --img2img-latents-processors argument of the dgenerate command line tool.

inference_steps: Sequence[int]

List of inference steps values, this corresponds to the --inference-steps argument of the dgenerate command line tool.

inpaint_crop: bool = False

Enable cropping to mask bounds for inpainting. When enabled, input images will be automatically cropped to the bounds of their masks (plus any padding) before processing, then the generated result will be pasted back onto the original uncropped image. This allows inpainting at higher effective resolutions for better quality results.

Note: Inpaint crop cannot be used with multiple input images. Each image/mask pair must be processed individually for optimal cropping, as different masks may have different bounds. However, batch_size > 1 is supported for generating multiple variations of a single crop.

This corresponds to the --inpaint-crop argument of the dgenerate command line tool.

inpaint_crop_feathers: Sequence[int] | None = None

One or more feather values to use when pasting the generated result back onto the original image for inpaint cropping. Specifying this automatically enables RenderLoopConfig.inpaint_crop. Each value will be tried in turn (combinatorial). Feathering creates smooth transitions from opaque to transparent. Cannot be used together with RenderLoopConfig.inpaint_crop_masked.

This corresponds to the --inpaint-crop-feathers argument of the dgenerate command line tool.

inpaint_crop_masked: bool = False

Use the mask when pasting the generated result back onto the original image for inpaint cropping. Specifying this automatically enables RenderLoopConfig.inpaint_crop. This means only the masked areas will be replaced. Cannot be used together with RenderLoopConfig.inpaint_crop_feathers.

This corresponds to the --inpaint-crop-masked argument of the dgenerate command line tool.

inpaint_crop_paddings: Sequence[int | tuple[int, int] | tuple[int, int, int, int]] | None = None

One or more padding values to use around mask bounds for inpaint cropping. Each value will be tried in turn (combinatorial). Specifying this automatically enables RenderLoopConfig.inpaint_crop.

Padding can be specified as: - A single integer (e.g., 32) for uniform padding on all sides - “WIDTHxHEIGHT” format (e.g., “10x20”) for horizontal and vertical padding - “LEFTxTOPxRIGHTxBOTTOM” format (e.g., “5x10x5x15”) for specific side padding

This corresponds to the --inpaint-crop-paddings argument of the dgenerate command line tool.

ip_adapter_uris: Sequence[str] | None = None

Optional user specified IP Adapter URIs, this corresponds to the --ip-adapters argument of the dgenerate command line tool.

latents: Sequence[Tensor] | None = None

Optional list of tensors containing noisy latents to use as starting points for diffusion. These latents can be generated by using –denoising-end with –image-format pt/pth/safetensors to save intermediate noisy latents from a previous generation. This allows for advanced workflows where you can pass partially denoised latents between different models or generation stages.

latents_post_processors: Sequence[str] | None = None

One or more latents processor URI strings for processing output latents when outputting to latents.

These processors are applied to latents when RenderLoopConfig.image_format is set to a tensor format (pt, pth, safetensors). The processors are applied in sequence after the diffusion pipeline generates the latents but before they are returned in the result.

This corresponds to the --latents-post-processors argument of the dgenerate command line tool.

latents_processors: Sequence[str] | None = None

One or more latents processor URI strings for processing raw input latents before pipeline execution.

These processors are applied to latents provided through the RenderLoopConfig.latents argument (raw latents used as noise initialization). The processors are applied in sequence before the latents are passed to the diffusion pipeline.

This corresponds to the --latents-processors argument of the dgenerate command line tool.

lora_fuse_scale: float | None = None

Optional global LoRA fuse scale, this corresponds to the --lora-fuse-scale argument of the dgenerate command line tool.

LoRA weights are merged into the main model at this scale.

When specifying multiple LoRA models, they are fused together into one set of weights using their individual scale values, after which they are fused into the main model at this scale value.

The default value when None is specified is 1.0.

lora_uris: Sequence[str] | None = None

Optional user specified LoRA URIs, this corresponds to the --loras argument of the dgenerate command line tool.

mask_image_processors: Sequence[str] | None = None

Corresponds to the --mask-image-processors argument of the dgenerate command line tool verbatim.

max_sequence_length: int | None = None

Max number of prompt tokens that the T5EncoderModel (text encoder 3) of Stable Diffusion 3 or Flux can handle.

This defaults to 256 for SD3 when not specified, and 512 for Flux.

The maximum value is 512 and the minimum value is 1.

High values result in more resource usage and processing time.

model_cpu_offload: bool = False

Force model cpu offloading for the main pipeline, this may reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be slower. Mutually exclusive with RenderLoopConfig.model_sequential_offload

model_path: str | None = None

Primary diffusion model path, model_path argument of dgenerate command line tool.

model_sequential_offload: bool = False

Force sequential model offloading for the main pipeline, this may drastically reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be much slower. Mutually exclusive with RenderLoopConfig.model_cpu_offload

model_type: ModelType = 0

Corresponds to the --model-type argument of the dgenerate command line tool.

no_aspect: bool = False

Should Seed, Mask, and Control guidance images specified in RenderLoopConfig.image_seeds definitions (--image-seeds) have their aspect ratio ignored when being resized to RenderLoopConfig.output_size (--output-size) ?

no_frames: bool = False

Should individual frames not be output when rendering an animation? defaults to False. This corresponds to the --no-frames argument of the dgenerate command line tool. Using this option when RenderLoopConfig.animation_format is equal to "frames" will cause a RenderLoopConfigError be raised.

offline_mode: bool = False

Avoid ever connecting to the internet to download anything? this can be used if all your models / media are cached or if you are only ever using resources that exist on disk already. Corresponds to the --offline-mode argument of the dgenerate command line tool.

original_config: str | None = None

This option can be used to supply an original LDM config .yaml file that was provided with a single file checkpoint.

output_auto1111_metadata: bool = False

Write Automatic1111 compatible metadata to the metadata of all written images? this data is not written to animated files, only PNGs and JPEGs. This corresponds to the --output-metadata argument of the dgenerate command line tool.

output_configs: bool = False

Output a config text file next to every generated image or animation? this file will contain configuration that is pipeable to dgenerate stdin, which will reproduce said file. This corresponds to the --output-configs argument of the dgenerate command line tool.

output_metadata: bool = False

Write config text to the metadata of all written images? this data is not written to animated files, only PNGs and JPEGs. This corresponds to the --output-metadata argument of the dgenerate command line tool.

output_overwrite: bool = False

Allow overwrites of files? or avoid this with a file suffix in a multiprocess safe manner? This corresponds to the --output-overwrite argument of the dgenerate command line tool.

output_path: str = 'output'

Render loop write folder, where images and animations will be written. This corresponds to the --output-path argument of the dgenerate command line tool.

output_prefix: str | None = None

Output filename prefix, add an optional prefix string to all written files. This corresponds to the --output-prefix argument of the dgenerate command line tool.

output_size: tuple[int, int] | None = None

Desired output size, sizes not aligned by 8 pixels will result in an error message. This corresponds to the --output-size argument of the dgenerate command line tool.

pag: bool = False

Use perturbed attention guidance?

pag_adaptive_scales: Sequence[float] | None = None

List of floating point adaptive perturbed attention guidance scales, this corresponds to the --pag-adaptive-scales argument of the dgenerate command line tool.

pag_scales: Sequence[float] | None = None

List of floating point perturbed attention guidance scales, this corresponds to the --pag-scales argument of the dgenerate command line tool.

parsed_image_seeds: Sequence[ImageSeedParseResult] | None = None

The results of parsing URIs mentioned in RenderLoopConfig.image_seeds, will only be available if RenderLoopConfig.check() has been called.

post_processors: Sequence[str] | None = None

Corresponds to the --post-processors argument of the dgenerate command line tool verbatim.

prompt_upscaler_uri: str | Sequence[str] | None = None

The URI of a prompt-upscaler implementation supported by dgenerate.

This may also be a list of URIs, the prompt upscalers will be chained together sequentially.

This corresponds to the --prompt-upscaler argument of the dgenerate command line tool.

prompt_weighter_uri: str | None = None

The URI of a prompt-weighter implementation supported by dgenerate.

This corresponds to the --prompt-weighter argument of the dgenerate command line tool.

prompts: Sequence[Prompt]

List of prompt objects, this corresponds to the --prompts argument of the dgenerate command line tool.

quantizer_map: Sequence[str] | None = None

Collection of pipeline submodule names to which quantization should be applied when :py:attr`RenderLoopConfig.quantizer_uri` is provided. Valid values include: unet, transformer, text_encoder, text_encoder_2, text_encoder_3. If None, all supported modules will be quantized.

quantizer_uri: str | None = None

Global quantizer URI for main pipline, this corresponds to the --quantizer argument of the dgenerate command line tool.

The quantization backend and settings specified by this URI will be used globally on the the most appropriate models associated with the main diffusion pipeline.

ras: bool = False

Activate RAS (Region-Adaptive Sampling) for the primary model?

This can increase inference speed with SD3.

See: https://github.com/microsoft/ras

This is supported for: --model-type sd3.

ras_end_steps: Sequence[int] | None = None

Ending steps to try for RAS (Region-Adaptive Sampling).

This controls when RAS stops applying its sampling strategy. Must be greater than or equal to 1. Defaults to the number of inference steps if not specified.

Each value will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

ras_error_reset_steps: Sequence[Sequence[int]] | None = None

Error reset step patterns to try for RAS (Region-Adaptive Sampling).

The dense sampling steps inserted between the RAS steps to reset the accumulated error. Should be a list of lists of step numbers, e.g. [[12, 22], …].

Each list will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

ras_high_ratios: Sequence[float] | None = None

High ratios to try for RAS (Region-Adaptive Sampling).

Based on the metric selected, the ratio of the high value chosen to be cached. Default value is 1.0, but can be set between 0 and 1 to balance the sample ratio between the main subject and the background.

Each value will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

ras_index_fusion: bool | None = None

Enable index fusion in RAS (Region-Adaptive Sampling) for the primary model?

This can improve attention computation in RAS for SD3.

See: https://github.com/microsoft/ras

Setting to True implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3, (but not for SD3.5 models)

ras_metrics: Sequence[str] | None = None

One or more RAS metrics to try.

This controls how RAS measures the importance of tokens for caching. Valid values are “std” (standard deviation) or “l2norm” (L2 norm). Defaults to “std”.

Each value will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

ras_sample_ratios: Sequence[float] | None = None

Sample ratios to try for RAS (Region-Adaptive Sampling).

For instance, setting this to 0.5 on a sequence of 4096 tokens will result in the noise of averagely 2048 tokens to be updated during each RAS step. Must be between 0 and 1.

Each value will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

ras_skip_num_step_lengths: Sequence[int] | None = None

Skip step lengths to try for RAS (Region-Adaptive Sampling).

This controls the length of steps to skip between RAS steps. Must be greater than or equal to 0.

When set to 0, static dropping is used where the same number of tokens are skipped at each step (except for error reset steps and steps before RenderLoopConfig.ras_start_steps).

When greater than 0, dynamic dropping is used where the number of skipped tokens varies over time based on RenderLoopConfig.ras_skip_num_steps. The pattern repeats every RenderLoopConfig.ras_skip_num_step_lengths steps.

Each value will be tried in turn.

Supplying any value implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

(default: 0)

ras_skip_num_steps: Sequence[int] | None = None

Skip steps to try for RAS (Region-Adaptive Sampling).

This controls the number of steps to skip between RAS steps.

The actual number of tokens skipped will be rounded down to the nearest multiple of 64. This ensures efficient memory access patterns for the attention computation.

When used with RenderLoopConfig.ras_skip_num_step_lengths > 0, this value determines how much to increase/decrease the number of skipped tokens over time. A positive value will increase the number of skipped tokens, while a negative value will decrease it.

Each value will be tried in turn.

Supplying any value implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

(default: 0)

ras_start_steps: Sequence[int] | None = None

Starting steps to try for RAS (Region-Adaptive Sampling).

This controls when RAS begins applying its sampling strategy. Must be greater than or equal to 1. Defaults to 4 if not specified.

Each value will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

ras_starvation_scales: Sequence[float] | None = None

Starvation scales to try for RAS (Region-Adaptive Sampling).

RAS tracks how often a token is dropped and incorporates this count as a scaling factor in the metric for selecting tokens. This scale factor prevents excessive blurring or noise in the final generated image. Larger scaling factor will result in more uniform sampling. Usually set between 0.0 and 1.0.

Each value will be tried in turn.

Supplying any values implies that RenderLoopConfig.ras is enabled.

This is supported for: --model-type sd3.

revision: str = 'main'

Repo revision selector for the main model when loading from a huggingface repository. This corresponds to the --revision argument of the dgenerate command line tool.

s_cascade_decoder_uri: str | None = None

Stable Cascade model URI, --s-cascade-decoder argument of dgenerate command line tool.

sada: bool = False

Enable SADA (Stability-guided Adaptive Diffusion Acceleration) with default parameters for the primary model.

Specifying this alone is equivalent to setting all SADA parameters to their model-specific default values:

• SD/SD2: sada_max_downsamples=1, sada_sxs=3, sada_sys=3, sada_lagrange_terms=4, sada_lagrange_ints=4, sada_lagrange_steps=24, sada_max_fixes=5120

• SDXL/Kolors: sada_max_downsamples=2, sada_sxs=3, sada_sys=3, sada_lagrange_terms=4, sada_lagrange_ints=4, sada_lagrange_steps=24, sada_max_fixes=10240

• Flux: sada_max_downsamples=0, sada_lagrange_terms=3, sada_lagrange_ints=4, sada_lagrange_steps=20, sada_max_fixes=0

SADA is not compatible with HiDiffusion, DeepCache, or TeaCache.

sada_acc_ranges: Sequence[tuple[int, int]] | None = None

SADA acceleration range start / end steps for the primary model.

Defines the starting / ending step for SADA acceleration.

Starting step must be at least 3 as SADA leverages third-order dynamics.

Defaults to [[10,47]].

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_lagrange_ints: Sequence[int] | None = None

SADA Lagrangian interpolation interval for the primary model.

Interval for Lagrangian interpolation. Must be compatible with sada_lagrange_step (lagrange_step % lagrange_int == 0).

Model-specific defaults:

• SD/SD2: 4

• SDXL/Kolors: 4

• Flux: 4

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_lagrange_steps: Sequence[int] | None = None

SADA Lagrangian interpolation step for the primary model.

Step value for Lagrangian interpolation. Must be compatible with sada_lagrange_int (lagrange_step % lagrange_int == 0).

Model-specific defaults:

• SD/SD2: 24

• SDXL/Kolors: 24

• Flux: 20

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_lagrange_terms: Sequence[int] | None = None

SADA Lagrangian interpolation terms for the primary model.

Number of terms to use in Lagrangian interpolation. Set to 0 to disable Lagrangian interpolation.

Model-specific defaults:

• SD/SD2: 4

• SDXL/Kolors: 4

• Flux: 3

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_max_downsamples: Sequence[int] | None = None

SADA maximum downsample factor for the primary model.

Controls the maximum downsample factor in the SADA algorithm. Lower values can improve quality but may reduce speedup.

Model-specific defaults:

• SD/SD2: 1

• SDXL/Kolors: 2

• Flux: 0

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_max_fixes: Sequence[int] | None = None

SADA maximum fixed memory for the primary model.

Maximum amount of fixed memory to use in SADA optimization.

Model-specific defaults:

• SD/SD2: 5120 (5 * 1024)

• SDXL/Kolors: 10240 (10 * 1024)

• Flux: 0

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_max_intervals: Sequence[int] | None = None

SADA maximum interval for optimization for the primary model.

Maximum interval between optimizations in the SADA algorithm.

Defaults to 4.

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_sxs: Sequence[int] | None = None

SADA spatial downsample factor X for the primary model.

Controls the spatial downsample factor in the X dimension. Higher values can increase speedup but may affect quality.

Model-specific defaults:

• SD/SD2: 3

• SDXL/Kolors: 3

• Flux: 0 (not used)

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

sada_sys: Sequence[int] | None = None

SADA spatial downsample factor Y for the primary model.

Controls the spatial downsample factor in the Y dimension. Higher values can increase speedup but may affect quality.

Model-specific defaults:

• SD/SD2: 3

• SDXL/Kolors: 3

• Flux: 0 (not used)

See: https://github.com/Ting-Justin-Jiang/sada-icml

Supplying any SADA parameter implies that SADA is enabled.

This is supported for: --model-type sd, sdxl, kolors, flux*.

Each value supplied will be tried in turn.

safety_checker: bool = False

Enable safety checker? --safety-checker

scheduler_uri: str | Sequence[str] | None = None

Optional primary model scheduler/sampler class name specification, this corresponds to the --scheduler argument of the dgenerate command line tool. Setting this to ‘help’ will yield a help message to stdout describing scheduler names compatible with the current configuration upon running. Passing ‘helpargs’ will yield a help message with a list of overridable arguments for each scheduler and their typical defaults.

This may be a list of schedulers, indicating to try each scheduler in turn.

sdxl_aesthetic_scores: Sequence[float] | None = None

Optional list of SDXL aesthetic-score conditioning values, this corresponds to the --sdxl-aesthetic-scores argument of the dgenerate command line tool.

sdxl_crops_coords_top_left: Sequence[tuple[int, int]] | None = None

Optional list of SDXL top-left-crop-coords micro-conditioning parameters, this corresponds to the --sdxl-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_high_noise_fractions: Sequence[float] | None = None

Optional list of SDXL refiner high noise fractions (floats), this value is the fraction of inference steps that the base model handles, the inverse proportion of the provided fraction is handled by the refiner model. This corresponds to the --sdxl-high-noise-fractions argument of the dgenerate command line tool.

sdxl_negative_aesthetic_scores: Sequence[float] | None = None

Optional list of negative influence SDXL aesthetic-score conditioning values, this corresponds to the --sdxl-negative-aesthetic-scores argument of the dgenerate command line tool.

sdxl_negative_crops_coords_top_left: Sequence[tuple[int, int]] | None = None

Optional list of negative influence SDXL top-left crop coords micro-conditioning parameters, this corresponds to the --sdxl-negative-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_negative_original_sizes: Sequence[tuple[int, int]] | None = None

Optional list of negative influence SDXL original-size micro-conditioning parameters, this corresponds to the --sdxl-negative-original-sizes argument of the dgenerate command line tool.

sdxl_negative_target_sizes: Sequence[tuple[int, int]] | None = None

Optional list of negative influence SDXL target-size micro-conditioning parameters, this corresponds to the --sdxl-negative-target-sizes argument of the dgenerate command line tool.

sdxl_original_sizes: Sequence[tuple[int, int]] | None = None

Optional list of SDXL original-size micro-conditioning parameters, this corresponds to the --sdxl-original-sizes argument of the dgenerate command line tool.

sdxl_refiner_aesthetic_scores: Sequence[float] | None = None

Optional list of SDXL-refiner override aesthetic-score conditioning values, this corresponds to the --sdxl-refiner-aesthetic-scores argument of the dgenerate command line tool.

sdxl_refiner_clip_skips: Sequence[int] | None = None

Clip skip override values for the SDXL refiner, which normally defaults to the clip skip value for the main model when it is defined.

sdxl_refiner_crops_coords_top_left: Sequence[tuple[int, int]] | None = None

Optional list of SDXL-refiner override top-left-crop-coords micro-conditioning parameters, this corresponds to the --sdxl-refiner-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_refiner_deep_cache: bool | None = None

Activate DeepCache for the SDXL Refiner?

See: RenderLoopConfig.deep_cache

This is supported for Stable Diffusion XL and Kolors based models.

sdxl_refiner_deep_cache_branch_ids: Sequence[int] | None = None

Branch IDs to try for DeepCache for the SDXL Refiner.

Controls which branches of the UNet attention blocks the caching is applied to. Advanced usage only.

This value must be greater than or equal to 0.

Each value supplied will be tried in turn.

This is supported for Stable Diffusion XL and Kolors based models.

Supplying any value implies that RenderLoopConfig.sdxl_refiner_deep_cache is enabled.

(default: 1)

sdxl_refiner_deep_cache_intervals: Sequence[int] | None = None

Cache intervals to try for DeepCache for the SDXL Refiner.

Controls how frequently the attention layers are cached during the diffusion process. Lower values cache more frequently, potentially resulting in more speedup but using more memory.

This value must be greater than zero.

Each value supplied will be tried in turn.

This is supported for Stable Diffusion XL and Kolors based models.

Supplying any value implies that RenderLoopConfig.sdxl_refiner_deep_cache is enabled.

(default: 5)

sdxl_refiner_edit: bool | None = None

Force the SDXL refiner to operate in edit mode instead of cooperative denoising mode.

sdxl_refiner_freeu_params: Sequence[tuple[float, float, float, float]] | None = None

List / sequence of FreeU parameters to try for the SDXL refiner.

See: RenderLoopConfig.freeu_params for clarification.

sdxl_refiner_guidance_rescales: Sequence[float] | None = None

Optional list of guidance rescale value overrides for the SDXL refiner or Stable Cascade decoder, this corresponds to the --sdxl-refiner-guidance-rescales argument of the dgenerate command line tool.

sdxl_refiner_negative_aesthetic_scores: Sequence[float] | None = None

Optional list of negative influence SDXL-refiner override aesthetic-score conditioning values, this corresponds to the --sdxl-refiner-negative-aesthetic-scores argument of the dgenerate command line tool.

sdxl_refiner_negative_crops_coords_top_left: Sequence[tuple[int, int]] | None = None

Optional list of negative influence SDXL-refiner top-left crop coords micro-conditioning parameters, this corresponds to the --sdxl-refiner-negative-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_refiner_negative_original_sizes: Sequence[tuple[int, int]] | None = None

Optional list of negative influence SDXL-refiner override original-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-negative-original-sizes argument of the dgenerate command line tool.

sdxl_refiner_negative_target_sizes: Sequence[tuple[int, int]] | None = None

Optional list of negative influence SDXL-refiner override target-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-negative-target-sizes argument of the dgenerate command line tool.

sdxl_refiner_original_sizes: Sequence[tuple[int, int]] | None = None

Optional list of SDXL-refiner override original-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-original-sizes argument of the dgenerate command line tool.

sdxl_refiner_pag: bool | None = None

Use perturbed attention guidance in the SDXL refiner?

sdxl_refiner_pag_adaptive_scales: Sequence[float] | None = None

List of floating point adaptive perturbed attention guidance scales to try with the SDXL refiner, this corresponds to the --sdxl-refiner-pag-adaptive-scales argument of the dgenerate command line tool.

sdxl_refiner_pag_scales: Sequence[float] | None = None

List of floating point perturbed attention guidance scales to try with the SDXL refiner, this corresponds to the --sdxl-refiner-pag-scales argument of the dgenerate command line tool.

sdxl_refiner_sigmas: Sequence[Sequence[float] | str] | None = None

One or more lists of sigma values to try with the SDXL refiner. This is supported when using a RenderLoopConfig.second_model_scheduler_uri that supports setting sigmas.

Or: string expressions involving sigmas from the selected scheduler such as sigmas * 0.95, sigmas will be represented as a numpy array, numpy is available through the namespace np, this uses asteval.

Lists of floats and strings representing expressions can be intermixed.

Sigma values control the noise schedule in the diffusion process, allowing for fine-grained control over how noise is added and removed during image generation.

This corresponds to the --sdxl-refiner-sigmas command line argument, which accepts multiple comma-separated lists of floating point values, or singular values, or expressions denoted with: expr: ....

You do not need to specify expr: when passing this value in the library, simply pass a string instead of a list of floats.

Example: [[1.0,2.0,3.0], 'sigmas * 0.95']

sdxl_refiner_target_sizes: Sequence[tuple[int, int]] | None = None

Optional list of SDXL-refiner override target-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-target-sizes argument of the dgenerate command line tool.

sdxl_refiner_uri: str | None = None

SDXL Refiner model URI, --sdxl-refiner argument of dgenerate command line tool.

sdxl_t2i_adapter_factors: Sequence[float] | None = None

Optional list of SDXL specific T2I adapter factors to try, this controls the amount of time-steps for which a T2I adapter applies guidance to an image, this is a value between 0.0 and 1.0. A value of 0.5 for example indicates that the T2I adapter is only active for half the amount of time-steps it takes to completely render an image.

sdxl_target_sizes: Sequence[tuple[int, int]] | None = None

Optional list of SDXL target-size micro-conditioning parameters, this corresponds to the --sdxl-target-sizes argument of the dgenerate command line tool.

second_model_cpu_offload: bool | None = None

Force model cpu offloading for the SDXL refiner or Stable Cascade decoder pipeline, this may reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be slower. Mutually exclusive with RenderLoopConfig.second_model_sequential_offload

second_model_guidance_scales: Sequence[float] | None = None

Optional list of guidance scale value overrides for the SDXL refiner or Stable Cascade decoder, this corresponds to the --second-model-guidance-scales argument of the dgenerate command line tool.

second_model_inference_steps: Sequence[int] | None = None

Optional list of inference steps value overrides for the SDXL refiner, this corresponds to the --second-model-inference-steps argument of the dgenerate command line tool.

second_model_original_config: str | None = None

This option can be used to supply an original LDM config .yaml file that was provided with a single file checkpoint for the secondary model, i.e. the SDXL Refiner or Stable Cascade Decoder.

second_model_prompt_upscaler_uri: str | Sequence[str] | None = None

The URI of a prompt-upscaler implementation supported by dgenerate to use with the SDXL refiner or Stable Cascade decoder.

Defaults to RenderLoopConfig.prompt_upscaler_uri if not specified.

This may also be a list of URIs, the prompt upscalers will be chained together sequentially.

This corresponds to the --second-model-prompt-upscaler argument of the dgenerate command line tool.

second_model_prompt_weighter_uri: str | None = None

The URI of a prompt-weighter implementation supported by dgenerate to use with the SDXL refiner or Stable Cascade decoder.

Defaults to RenderLoopConfig.prompt_weighter_uri if not specified.

This corresponds to the --second-model-prompt-weighter argument of the dgenerate command line tool.

second_model_prompts: Sequence[Prompt] | None = None

Optional list of SDXL refiner or Stable Cascade decoder prompt overrides, this corresponds to the --second-model-prompts argument of the dgenerate command line tool.

second_model_quantizer_map: Sequence[str] | None = None

Collection of secondary pipeline submodule names to which quantization should be applied when :py:attr`RenderLoopConfig.second_model_quantizer_uri` is provided. Valid values include: unet, transformer, text_encoder, text_encoder_2, text_encoder_3. If None, all supported modules will be quantized.

second_model_quantizer_uri: str | None = None

Global quantizer URI for secondary pipeline (SDXL Refiner or Stable Cascade decoder), this corresponds to the --second-model-quantizer argument of the dgenerate command line tool.

The quantization backend and settings specified by this URI will be used globally on the the most appropriate models associated with the secondary diffusion pipeline (SDXL Refiner, Stable Cascade Decoder).

second_model_scheduler_uri: str | Sequence[str] | None = None

Optional SDXL Refiner / Stable Cascade Decoder model scheduler/sampler class name specification, this corresponds to the --second-model-scheduler argument of the dgenerate command line tool. Setting this to ‘help’ will yield a help message to stdout describing scheduler names compatible with the current configuration upon running. Passing ‘helpargs’ will yield a help message with a list of overridable arguments for each scheduler and their typical defaults.

This may be a list of schedulers, indicating to try each scheduler in turn.

second_model_second_prompt_upscaler_uri: str | Sequence[str] | None = None

The URI of a prompt-upscaler implementation supported by dgenerate to use with the SDXL refiner --second-prompts value.

Or rather RenderLoopConfig.second_model_second_prompts

Defaults to RenderLoopConfig.prompt_upscaler_uri if not specified.

This may also be a list of URIs, the prompt upscalers will be chained together sequentially.

This corresponds to the --second-model-second-prompt-upscaler argument of the dgenerate command line tool.

second_model_second_prompts: Sequence[Prompt] | None = None

Optional list of SDXL refiner secondary prompt overrides, this corresponds to the --second-model-second-prompts argument of the dgenerate command line tool. The Stable Cascade Decoder does not support this argument.

second_model_sequential_offload: bool | None = None

Force sequential model offloading for the SDXL refiner or Stable Cascade decoder pipeline, this may drastically reduce memory consumption and allow large models to run when they would otherwise not fit in your GPUs VRAM. Inference will be much slower. Mutually exclusive with RenderLoopConfig.second_model_cpu_offload

second_model_text_encoder_uris: Sequence[str] | None = None

Optional user specified Text Encoder URIs, this corresponds to the --second-model-text-encoders argument of the dgenerate command line tool. This specifies text encoders for the SDXL refiner or Stable Cascade decoder.

second_model_unet_uri: str | None = None

Optional user specified second UNet URI, this corresponds to the --second-model-unet argument of the dgenerate command line tool. This UNet uri will be used for the SDXL refiner or Stable Cascade decoder model.

second_prompt_upscaler_uri: str | Sequence[str] | None = None

The URI of a prompt-upscaler implementation supported by dgenerate that applies to RenderLoopConfig.second_prompts

Defaults to RenderLoopConfig.prompt_upscaler_uri if not specified.

This may also be a list of URIs, the prompt upscalers will be chained together sequentially.

This corresponds to the --second-prompt-upscaler argument of the dgenerate command line tool.

second_prompts: Sequence[Prompt] | None = None

Optional list of SD3 / Flux secondary prompts, this corresponds to the --second-prompts argument of the dgenerate command line tool.

seed_image_processors: Sequence[str] | None = None

Corresponds to the --seed-image-processors argument of the dgenerate command line tool verbatim.

seeds: Sequence[int]

List of integer seeds, this corresponds to the --seeds argument of the dgenerate command line tool.

seeds_to_images: bool = False

Should RenderLoopConfig.seeds be interpreted as seeds for each image input instead of combinatorial? this includes control images.

sigmas: Sequence[Sequence[float] | str] | None = None

One or more lists of sigma values to try. This is supported when using a RenderLoopConfig.scheduler_uri that supports setting sigmas.

Or: string expressions involving sigmas from the selected scheduler such as sigmas * 0.95, sigmas will be represented as a numpy array, numpy is available through the namespace np, this uses asteval.

Lists of floats and strings representing expressions can be intermixed.

Sigma values control the noise schedule in the diffusion process, allowing for fine-grained control over how noise is added and removed during image generation.

This corresponds to the --sigmas command line argument, which accepts multiple comma-separated lists of floating point values, or singular values, or expressions denoted with: expr: ....

You do not need to specify expr: when passing this value in the library, simply pass a string instead of a list of floats.

Example: [[1.0,2.0,3.0], 'sigmas * 0.95']

subfolder: str | None = None

Primary model subfolder argument, --subfolder argument of dgenerate command line tool.

t2i_adapter_uris: Sequence[str] | None = None

Optional user specified T2IAdapter URIs, this corresponds to the --t2i-adapters argument of the dgenerate command line tool.

tea_cache: bool = False

Activate TeaCache for the primary model?

This is supported for Flux, teacache uses a novel caching mechanism in the forward pass of the flux transformer to reduce the amount of computation needed to generate an image, this can speed up inference with small amounts of quality loss.

See: https://github.com/ali-vilab/TeaCache

Also see: RenderLoopConfig.tea_cache_rel_l1_thresholds

This is supported for: --model-type flux*.

tea_cache_rel_l1_thresholds: Sequence[float] | None = None

TeaCache relative L1 thresholds to try when RenderLoopConfig.tea_cache is enabled.

This should be one or more float values between 0.0 and 1.0, each value will be tried in turn. Higher values mean more speedup.

Defaults to 0.6 (2.0x speedup). 0.25 for 1.5x speedup, 0.4 for 1.8x speedup, 0.6 for 2.0x speedup, 0.8 for 2.25x speedup

See: https://github.com/ali-vilab/TeaCache

Supplying any value implies that RenderLoopConfig.tea_cache is enabled.

This is supported for: --model-type flux*.

text_encoder_uris: Sequence[str] | None = None

Optional user specified Text Encoder URIs, this corresponds to the --text-encoders argument of the dgenerate command line tool.

textual_inversion_uris: Sequence[str] | None = None

Optional user specified Textual Inversion URIs, this corresponds to the --textual-inversions argument of the dgenerate command line tool.

third_prompt_upscaler_uri: str | Sequence[str] | None = None

The URI of a prompt-upscaler implementation supported by dgenerate that applies to RenderLoopConfig.third_prompts

Defaults to RenderLoopConfig.prompt_upscaler_uri if not specified.

This may also be a list of URIs, the prompt upscalers will be chained together sequentially.

This corresponds to the --third-prompt-upscaler argument of the dgenerate command line tool.

third_prompts: Sequence[Prompt] | None = None

Optional list of SD3 tertiary prompts, this corresponds to the --third-prompts argument of the dgenerate command line tool. Flux does not support this argument.

transformer_uri: str | None = None

Optional user specified Transformer URI, this corresponds to the --transformer argument of the dgenerate command line tool.

This is currently only supported for Stable Diffusion 3 and Flux models.

unet_uri: str | None = None

Optional user specified UNet URI, this corresponds to the --unet argument of the dgenerate command line tool.

upscaler_noise_levels: Sequence[int] | None = None

Optional list of integer upscaler noise levels, this corresponds to the --upscaler-noise-levels argument of the dgenerate command line tool that is used for the dgenerate.pipelinewrapper.ModelType.UPSCALER_X4 model type only.

vae_slicing: bool = False

Enable VAE slicing? --vae-slicing

vae_tiling: bool = False

Enable VAE tiling? --vae-tiling

vae_uri: str | None = None

Optional user specified VAE URI, this corresponds to the --vae argument of the dgenerate command line tool.

variant: str | None = None

Primary model weights variant string. This corresponds to the --variant argument of the dgenerate command line tool.

class dgenerate.StartingAnimationEvent(origin, total_frames: int, fps: float, frame_duration: float)

Bases: Event

Common event stream object produced by the events() event stream of a render loop.

Occurs when a sequence of images that belong to an animation are about to start being generated.

This occurs whether an animation is going to be written to disk or not.

__init__(origin, total_frames: int, fps: float, frame_duration: float)

fps: float

FPS of the generated file.

frame_duration: float

Frame duration of the generated file, (the time a frame is visible in milliseconds)

total_frames: int

Number of frames written.

class dgenerate.StartingAnimationFileEvent(origin, path: str, total_frames: int, fps: float, frame_duration: float)

Bases: Event

Common event stream object produced by the events() event stream of a render loop.

Occurs when a sequence of images that belong to an animation are about to start being written to a file.

__init__(origin, path: str, total_frames: int, fps: float, frame_duration: float)

fps: float

FPS of the generated file.

frame_duration: float

Frame duration of the generated file, (the time a frame is visible in milliseconds)

path: str

File path where the animation will reside.

total_frames: int

Number of frames written.

class dgenerate.StartingGenerationStepEvent(origin, generation_step: int, total_steps: int)

Bases: Event

Common event stream object produced by the events() event stream of a render loop.

Occurs when a generation step is starting, a generation step may produce multiple images and or an animation.

__init__(origin, generation_step: int, total_steps: int)

generation_step: int

The generation step number.

total_steps: int

The total number of steps that are needed to complete the render loop.

dgenerate.clear_device_cache(device: device | str | None = None)

Clear every object cached by dgenerate in its GPU side cache for a specific device.

Parameters:

device – The device the objects are cached on, specifying None clears all devices.

dgenerate.default_device() → str

Return a string representing the systems default accelerator device.

Possible Values:

• "cuda"

• "mps"

• "xpu"

• "cpu"

Returns:

"cuda", "mps", "xpu", etc.

dgenerate.disable_offline_mode()

Disable offline mode for dgenerate.

This will allow network requests to be made again.

dgenerate.enable_offline_mode()

Enable global offline mode for dgenerate.

This will prevent any network requests from being made.

dgenerate.format_image_seed_uri(seed_images: str | Iterable[str] | None = None, mask_images: str | Iterable[str] | None = None, control_images: str | Iterable[str] | None = None, adapter_images: str | Iterable[str] | None = None, latents: str | Iterable[str] | None = None, floyd_image: str | None = None, resize: str | tuple[int | str, int | str] | None = None, aspect: bool = True, frame_start: int | None = None, frame_end: int | None = None) → str

Formats a --image-seeds URI to its shortest possible string form.

Raises:

ValueError – if inpaint_image is specified without seed_image. if keyword arguments are present without seed_image or control_images. if frame_start or frame_end are negative values. if frame_start is greater than frame_end. if adapter_images are used with floyd_image. if latents are used with floyd_image. if both control_images and floyd_image are specified. if resize is specified when only latents are provided. if aspect=False is specified when only latents are provided. if frame_start or frame_end is specified when only latents are provided. if too many mask images are provided. if too few mask images are provided. if no arguments are provided.

Parameters:

• seed_images – Seed image path(s)

• mask_images – Inpaint image path(s)

• control_images – Control image path(s)

• adapter_images – Adapter image path(s)

• latents – Raw latent tensor path(s) (.pt, .pth, .safetensors files)

• floyd_image – Path to a Floyd image

• resize – Optional resize dimension (WxH string)

• aspect – Preserve aspect ratio?

• frame_start – Optional frame start index

• frame_end – Optional frame end index

Returns:

The generated --image-seeds URI string

dgenerate.gen_seeds(n: int) → list[int]

Generate a list of N random seed integers

Parameters:

n – number of seeds to generate

Returns:

list of integer seeds

dgenerate.invoke_dgenerate(args: Sequence[str], render_loop: RenderLoop | None = None, config_overrides: dict[str, Any] | None = None, throw: bool = False, log_error: bool = True, help_raises: bool = False) → int

Invoke dgenerate using its command line arguments and return a return code.

dgenerate is invoked in the current process, this method does not spawn a subprocess.

Meta arguments such as --file, --shell, --no-stdin, and --console are not supported

Parameters:

• args – dgenerate command line arguments in the form of a list, see: shlex module, or sys.argv

• render_loop – dgenerate.renderloop.RenderLoop instance, if None is provided one will be created. Note that the config object generated by argument parsing will completely overwrite the render loop config.

• config_overrides – Optional dictionary of configuration overrides to apply to the render loop config object after argument parsing, this should consist of attribute names with values, the config object generated by argument parsing is of type dgenerate.arguments.DgenerateArguments.

• throw – Whether to throw known exceptions or handle them.

• log_error – Write ERROR diagnostics with dgenerate.messages?

• help_raises – --help raises dgenerate.arguments.DgenerateHelpException ? When True, this will occur even if throw=False

Raises:

• dgenerate.DgenerateUsageError –

• dgenerate.DgenerateHelpException –

• dgenerate.ImageSeedError –

• dgenerate.UnknownMimetypeError –

• dgenerate.MediaIdentificationError –

• dgenerate.FrameStartOutOfBounds –

• dgenerate.InvalidModelFileError –

• dgenerate.InvalidModelUriError –

• dgenerate.ModelUriLoadError –

• dgenerate.InvalidSchedulerNameError –

• dgenerate.OutOfMemoryError –

• dgenerate.ModelNotFoundError –

• dgenerate.NonHFModelDownloadError –

• dgenerate.UnsupportedPipelineConfigError –

• dgenerate.PromptWeightingUnsupported –

• dgenerate.PluginNotFoundError –

• dgenerate.PluginArgumentError –

• dgenerate.ModuleFileNotFoundError –

• dgenerate.SpacyModelNotFoundException –

• dgenerate.WebFileCacheOfflineModeException –

• OSError –

Returns:

integer return-code, anything other than 0 is failure

dgenerate.invoke_dgenerate_events(args: Sequence[str], render_loop: RenderLoop | None = None, config_overrides: dict[str, Any] | None = None, throw: bool = False, log_error: bool = True, help_raises: bool = False) → Generator[ImageGeneratedEvent | StartingAnimationEvent | StartingAnimationFileEvent | AnimationFileFinishedEvent | ImageFileSavedEvent | AnimationFinishedEvent | StartingGenerationStepEvent | AnimationETAEvent | DgenerateExitEvent, None, None]

Invoke dgenerate using its command line arguments and return a stream of events, you must iterate over this stream of events to progress through the execution of dgenerate.

dgenerate is invoked in the current process, this method does not spawn a subprocess.

Meta arguments such as --file, --shell, --no-stdin, and --console are not supported

The exceptions mentioned here are those you may encounter upon iterating, they will not occur upon simple acquisition of the event stream iterator.

Parameters:

• args – dgenerate command line arguments in the form of a list, see: shlex module, or sys.argv

• render_loop – dgenerate.renderloop.RenderLoop instance, if None is provided one will be created. Note that the config object generated by argument parsing will completely overwrite the render loop config.

• config_overrides – Optional dictionary of configuration overrides to apply to the render loop config object after argument parsing, this should consist of attribute names with values, the config object generated by argument parsing is of type dgenerate.arguments.DgenerateArguments.

• throw – Whether to throw known exceptions or handle them.

• log_error – Write ERROR diagnostics with dgenerate.messages?

• help_raises – --help raises dgenerate.arguments.DgenerateHelpException ? When True, this will occur even if throw=False

Raises:

• dgenerate.DgenerateUsageError –

• dgenerate.DgenerateHelpException –

• dgenerate.ImageSeedError –

• dgenerate.UnknownMimetypeError –

• dgenerate.MediaIdentificationError –

• dgenerate.FrameStartOutOfBounds –

• dgenerate.InvalidModelFileError –

• dgenerate.InvalidModelUriError –

• dgenerate.ModelUriLoadError –

• dgenerate.InvalidSchedulerNameError –

• dgenerate.OutOfMemoryError –

• dgenerate.ModelNotFoundError –

• dgenerate.NonHFModelDownloadError –

• dgenerate.UnsupportedPipelineConfigError –

• dgenerate.PromptWeightingUnsupported –

• dgenerate.PromptEmbeddedArgumentError –

• dgenerate.PluginNotFoundError –

• dgenerate.PluginArgumentError –

• dgenerate.ModuleFileNotFoundError –

• dgenerate.SpacyModelNotFoundException –

• dgenerate.WebFileCacheOfflineModeException –

• OSError –

Returns:

InvokeDgenerateEventStream

dgenerate.is_offline_mode() → bool

Check if dgenerate is in global offline mode.

Returns:

True if dgenerate is in offline mode, False otherwise.

dgenerate.main(args: Sequence[str] | None = None)

Entry point for the dgenerate command line tool.

Parameters:

args – program arguments, if None is provided they will be taken from sys.argv

dgenerate.offline_mode_context(enabled=True)

Context manager to temporarily enable or disable offline mode for dgenerate.

Parameters:

enabled – If True, enables offline mode. If False, disables it.

dgenerate.parse_args(args: Sequence[str] | None = None, overrides: dict[str, str] | None = None, throw: bool = True, log_error: bool = True, help_raises: bool = False) → DgenerateArguments | None

Parse dgenerate’s command line arguments and return a configuration object.

Parameters:

• args – arguments list, as in args taken from sys.argv, or in that format

• overrides – Optional dictionary of overrides to apply to the DgenerateArguments object after parsing but before validation, this should consist of attribute names with values.

• throw – throw DgenerateUsageError on error? defaults to True

• log_error – Write ERROR diagnostics with dgenerate.messages?

• help_raises – --help raises dgenerate.arguments.DgenerateHelpException ? When True, this will occur even if throw=False

Raises:

• DgenerateUsageError –

• DgenerateHelpException –

Returns:

DgenerateArguments. If throw=False then None will be returned on errors.