dgenerate submodules

dgenerate.arguments module

exception dgenerate.arguments.DgenerateUsageError

Bases: Exception

class dgenerate.arguments.DgenerateArguments

Bases: RenderLoopConfig

Represents dgenerates parsed command line arguments, can be used as a configuration object for dgenerate.renderloop.RenderLoop.

__init__()

cache_memory_constraints: List[str] | None = None

control_net_cache_memory_constraints: List[str] | None = None

pipeline_cache_memory_constraints: List[str] | None = None

plugin_module_paths: List[str]

vae_cache_memory_constraints: List[str] | None = None

verbose: bool = False

dgenerate.arguments.config_attribute_name_to_option(name)

Convert an attribute name of DgenerateArguments into its command line option name.

Parameters:

name – the attribute name

Returns:

the command line argument name as a string

dgenerate.arguments.parse_args(args: Sequence[str], throw: bool = True) → DgenerateArguments | None

Parse dgenerates command line arguments and return a configuration object.

Parameters:

• args – arguments list, as in args taken from sys.argv, or in that format

• throw – throw DgenerateUsageError on error? defaults to True

Raises:

DgenerateUsageError –

Returns:

DgenerateArguments. If throw=False then None will be returned on errors.

dgenerate.batchprocess module

exception dgenerate.batchprocess.BatchProcessError

Bases: Exception

Thrown on errors encountered within a batch processing script including non-zero return codes from the invoker.

class dgenerate.batchprocess.BatchProcessor(invoker: Callable[[list], int], name: str, version: Tuple[int, int, int] | str, template_variable_generator: Callable[[], dict] | None = None, template_variables: Dict[str, Any] | None = None, template_functions: Dict[str, Callable[[Any], Any]] | None = None, directives: Dict[str, Callable[[list], None]] | None = None, injected_args: List[str] | None = None)

Bases: object

Implements dgenerates batch processing scripts in a generified manner.

__init__(invoker: Callable[[list], int], name: str, version: Tuple[int, int, int] | str, template_variable_generator: Callable[[], dict] | None = None, template_variables: Dict[str, Any] | None = None, template_functions: Dict[str, Callable[[Any], Any]] | None = None, directives: Dict[str, Callable[[list], None]] | None = None, injected_args: List[str] | None = None)

Parameters:

• invoker – A function for invoking lines recognized as shell commands, should return a return code.

• template_variable_generator – A function that generates template variables for templating after an invocation, should return a dictionary.

• name – The name of this batch processor, currently used in the version check directive and messages

• version – Version for version check hash bang directive.

• template_variables – Live template variables, the initial environment, this dictionary will be modified during runtime.

• template_functions – Functions available to Jinja2

• directives – batch processing directive handlers, for: \directives. This is a dictionary of names to functions which accept a single parameter, a list of directive arguments.

• injected_args – Arguments to be injected at the end of user specified arguments for every shell invocation

render_template(string: str)

Render a template from a string

Parameters:

string – the string containing the Jinja2 template.

Returns:

rendered string

run_file(stream: TextIO)

Process a batch processing script from a file stream

Raises:

BatchProcessError –

Parameters:

stream – A filestream in text read mode

run_string(string: str)

Process a batch processing script from a string

Raises:

BatchProcessError –

Parameters:

string – a string containing the script

property current_line: int

The current line in the file being processed.

directives: Dict[str, Callable[[list], None]]

Batch process directive handlers by name.

injected_args: List[str]

Shell arguments to inject at the end of every invocation.

invoker: Callable[[list], int]

Invoker function, responsible for executing lines recognized as shell commands.

name: str

Name of this batch processor, currently used in the hash bang version check directive and messages.

template_functions: Dict[str, Callable[[Any], Any]]

Functions available when templating is occurring.

template_variable_generator: Callable[[], dict]

Function that generates template variables after an invocation occurs.

template_variables: Dict[str, Any]

Live template variables.

version: Tuple[int, int, int]

Version tuple for the version check hash bang directive.

class dgenerate.batchprocess.PeekReader(file: TextIO)

Bases: object

Read from a TextIO file object while peeking at the next line in the file.

This is an iterable reader wrapper that yields the tuple (current_line, next_line)

next_line will be None if the next line is the end of the file.

__init__(file: TextIO)

Parameters:

file – The TextIO reader to wrap.

dgenerate.batchprocess.create_config_runner(injected_args: Sequence[str] | None = None, render_loop: RenderLoop | None = None, version: Tuple[int, int, int] | str = '2.1.0', throw: bool = False)

Create a BatchProcessor that can run dgenerate batch processing configs from a string or file.

Parameters:

• injected_args – dgenerate command line arguments in the form of a list, see: shlex module, or sys.argv. These arguments will be injected at the end of every dgenerate invocation in the config file.

• render_loop – RenderLoop instance, if None is provided one will be created.

• version – Config version for #! dgenerate x.x.x version checks, defaults to dgenerate.__version__

• throw – Whether to throw exceptions from dgenerate.invoker.invoke_dgenerate() or handle them, if you set this to True exceptions will propagate out of dgenerate invocations instead of a BatchProcessError being raised, a line number where the error occurred can be obtained using BatchProcessor.current_line.

Returns:

integer return-code, anything other than 0 is failure

dgenerate.renderloop module

exception dgenerate.renderloop.RenderLoopConfigError

Bases: Exception

Raised by RenderLoopConfig.check() on configuration errors.

class dgenerate.renderloop.ImageGeneratedCallbackArgument

Bases: object

This argument object gets passed to callbacks registered to RenderLoop.image_generated_callbacks.

batch_index: int = 0

The index in the image batch for this image. Will only every be greater than zero if RenderLoopConfig.batch_size > 1 and RenderLoopConfig.batch_grid_size is None.

command_string: str = None

Reproduction of a command line that can be used to reproduce this image.

config_string: str = None

Reproduction of a dgenerate config file that can be used to reproduce this image.

diffusion_args: DiffusionArguments = None

Diffusion argument object, contains dgenerate.pipelinewrapper.DiffusionPipelineWrapper arguments used to produce this image.

property frame_index: int | None

The frame index if this is an animation frame. Also available through image_seed.frame_index, though here for convenience.

generation_step: int = 0

The current generation step. (zero indexed)

image: Image = None

The generated image.

image_seed: ImageSeed | None = None

If an --image-seeds specification was used in the generation of this image, this object represents that image seed and contains the images that contributed to the generation of this image.

property is_animation_frame: bool

Is this image a frame in an animation?

suggested_filename: str = None

A suggested filename for saving this image as. This filename will be unique to the render loop run / configuration.

class dgenerate.renderloop.RenderLoop(config=None, preprocessor_loader=None)

Bases: object

Render loop which implements the bulk of dgenerates rendering capability.

This object handles the scatter gun iteration over requested diffusion parameters, the generation of animations, and writing images and media to disk or providing those to library users through callbacks.

__init__(config=None, preprocessor_loader=None)

Parameters:

• config – RenderLoopConfig or dgenerate.arguments.DgenerateArguments. If None is provided, a RenderLoopConfig instance will be created and assigned to RenderLoop.config.

• preprocessor_loader – dgenerate.preprocessors.loader.Loader. If None is provided, an instance will be created and assigned to RenderLoop.preprocessor_loader.

generate_template_variables() → Dict[str, Any]

Generate a dictionary from the render loop that describes its current / last used configuration.

This is consumed by the dgenerate.batchprocess.BatchProcessor that is created by dgenerate.batchprocess.create_config_runner() for use in Jinja2 templating.

Returns:

a dictionary of attribute names to values

generate_template_variables_help(values: Dict[str, Tuple[Type, Any]] | None = None, show_values: bool = True, header=None)

Generate a help string describing available template variables, their types, and values for use in batch processing.

This is used to implement --templates-help in dgenerate.invoker.invoke_dgenerate()

Parameters:

• show_values – Show the value of the template variable or just the name?

• header – Override the default help message header.

Returns:

a human-readable description of all template variables

generate_template_variables_with_types() → Dict[str, Tuple[Type, Any]]

Generate a dictionary from the render loop that describes its current / last used configuration with type hints.

Returns:

a dictionary of attribute names to tuple(type_hint_type, value)

run()

Run the diffusion loop, this calls RenderLoopConfig.check() prior to running.

Raises:

• dgenerate.pipelinewrapper.ModelNotFoundError –

• dgenerate.pipelinewrapper.OutOfMemoryError –

disable_writes: bool = False

Disable or enable all writes to disk, if you intend to only ever use the callbacks of the render loop when using dgenerate as a library, this is a useful option.

last_images and last_animations will not be available as template variables in batch processing scripts with this enabled, they will be empty lists

property generation_step

Returns the current generation step, (zero indexed)

image_generated_callbacks: List[Callable[[ImageGeneratedCallbackArgument], None]]

Optional callbacks for handling individual images that have been generated.

The callback has a single argument: ImageGeneratedCallbackArgument

model_extra_modules: Dict[str, Any] = None

Extra raw diffusers modules to use in the creation of the main model pipeline.

property pipeline_wrapper: DiffusionPipelineWrapper

Get the last used dgenerate.pipelinewrapper.DiffusionPipelineWrapper instance.

Will be None if RenderLoop.run() has never been called.

Returns:

dgenerate.pipelinewrapper.DiffusionPipelineWrapper or None

refiner_extra_modules: Dict[str, Any] = None

Extra raw diffusers modules to use in the creation of any refiner model pipeline.

property written_animations: Iterator[str]

Iterator over animation filenames written by the last run

property written_images: Iterator[str]

Iterator over image filenames written by the last run

class dgenerate.renderloop.RenderLoopConfig

Bases: SetFromMixin

This object represents configuration for RenderLoop.

It nearly directly maps to dgenerates command line arguments.

See subclass dgenerate.arguments.DgenerateArguments

__init__()

calculate_generation_steps()

Calculate the number of generation steps that this configuration results in.

Returns:

int

check(attribute_namer: Callable[[str], str] = None)

Check the configuration for type and logical usage errors, set defaults for certain values when needed and not specified.

Raises:

RenderLoopConfigError – on errors

Parameters:

attribute_namer – Callable for naming attributes mentioned in exception messages

generate_template_variables(variable_prefix: str | None = None) → Dict[str, Any]

Generate a dictionary from this configuration object that is suitable for using as Jinja2 environmental variables.

Parameters:

variable_prefix – Prefix every variable name with this prefix if specified

Returns:

a dictionary of attribute names to values

generate_template_variables_with_types(variable_prefix: str | None = None) → Dict[str, Tuple[Type, Any]]

Generate a dictionary from this configuration object that maps attribute names to a tuple containing (type_hint_type, value)

Parameters:

variable_prefix – Prefix every variable name with this prefix if specified

Returns:

a dictionary of attribute names to tuple(type_hint_type, value)

iterate_diffusion_args(**overrides) → Iterator[DiffusionArguments]

Iterate over dgenerate.pipelinewrapper.DiffusionArguments argument objects using every combination of argument values provided for that object by this configuration.

Parameters:

overrides – use key word arguments to override specific attributes of this object with a new list value.

Returns:

an iterator over dgenerate.pipelinewrapper.DiffusionArguments

animation_format: str = 'mp4'

Format for any rendered animations, see: dgenerate.mediaoutput.supported_animation_writer_formats() This corresponds to the --animation-format argument of the dgenerate command line tool.

auth_token: str | None = None

Optional huggingface API token which will allow the download of restricted repositories that your huggingface account has been granted access to. This corresponds to the --auth-token argument of the dgenerate command line tool.

batch_grid_size: Tuple[int, int] | None = None

Optional image grid size specification for when batch_size is greater than one. This is the --batch-grid-size argument of the dgenerate command line tool.

batch_size: int | None = None

Image generation batch size, --batch-size argument of dgenerate command line tool.

clip_skips: List[int] | None = None

List of clip skip values. Clip skip is the number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Only supported for model_type values torch and torch-sdxl, including with control_net_uris defined.

control_image_preprocessors: List[str] | None = None

Corresponds to the --control-image-preprocessors argument of the dgenerate command line tool verbatim, including the grouping syntax using the “+” symbol, the plus symbol should be used as its own list element, IE: it is a token.

control_net_uris: List[str] | None = None

Optional user specified ControlNet URIs, this corresponds to the --control-nets argument of the dgenerate command line tool.

device: str = 'cuda'

Processing device specification, for example “cuda” or “cuda:N” where N is an alternate GPU id as reported by nvidia-smi if you want to specify a specific GPU. This corresponds to the --device argument of the dgenerate command line tool.

dtype: DataTypes = 0

Primary model data type specification, IE: integer precision. Default is auto selection. Lower precision datatypes result in less GPU memory usage. This corresponds to the --dtype argument of the dgenerate command line tool.

frame_end: int | None = None

Optional end frame inclusive frame slice for any rendered animations. This corresponds to the --frame-end argument of the dgenerate command line tool.

frame_start: int = 0

Start frame inclusive frame slice for any rendered animations. This corresponds to the --frame-start argument of the dgenerate command line tool.

guidance_rescales: List[float] | None = None

List of floating point guidance rescale values which are supported by some pipelines, (there will be an error if it is unsupported upon running), this corresponds to the --guidance-rescales argument of the dgenerate command line tool.

guidance_scales: List[float]

List of floating point guidance scales, this corresponds to the --guidance-scales argument of the dgenerate command line tool.

image_guidance_scales: List[float] | None = None

Optional list of floating point image guidance scales, used for pix2pix model types, this corresponds to the --image-guidance-scales argument of the dgenerate command line tool.

image_seed_strengths: List[float] | None = None

Optional list of floating point image seed strengths, this corresponds to the --image-seed-strengths argument of the dgenerate command line tool.

image_seeds: List[str] | None = None

List of --image-seeds URI strings.

inference_steps: List[int]

List of inference steps values, this corresponds to the --inference-steps argument of the dgenerate command line tool.

lora_uris: List[str] | None = None

Optional user specified LoRA URIs, this corresponds to the --lora/--loras argument of the dgenerate command line tool.

mask_image_preprocessors: List[str] | None = None

Corresponds to the --mask-image-preprocessors argument of the dgenerate command line tool verbatim.

model_path: str | None = None

Primary diffusion model path, model_path argument of dgenerate command line tool.

model_type: ModelTypes = 0

Corresponds to the --model-type argument of the dgenerate command line tool.

no_aspect: bool = False

Should Seed, Mask, and Control guidance images specified in RenderLoopConfig.image_seeds definitions (--image-seeds) have their aspect ratio ignored when being resized to RenderLoopConfig.output_size (--output-size) ?

offline_mode: bool = False

Avoid ever connecting to the huggingface API? this can be used if all your models are cached or if you are only ever using models that exist on disk. Corresponds to the --offline-mode argument of the dgenerate command line tool.

output_configs: bool = False

Output a config text file next to every generated image or animation? this file will contain configuration that is pipeable to dgenerate stdin, which will reproduce said file. This corresponds to the --output-configs argument of the dgenerate command line tool.

output_metadata: bool = False

Write config text to the metadata of all written images? this data is not written to animated files, only PNGs. This corresponds to the --output-metadata argument of the dgenerate command line tool.

output_overwrite: bool = False

Allow overwrites of files? or avoid this with a file suffix in a multiprocess safe manner? This corresponds to the --output-overwrite argument of the dgenerate command line tool.

output_path: str = 'output'

Render loop write folder, where images and animations will be written. This corresponds to the --output-path argument of the dgenerate command line tool.

output_prefix: str | None = None

Output filename prefix, add an optional prefix string to all written files. This corresponds to the --output-prefix argument of the dgenerate command line tool.

output_size: Tuple[int, int] | None = None

Desired output size, sizes not aligned by 8 pixels will result in an error message. This corresponds to the --output-size argument of the dgenerate command line tool.

parsed_image_seeds: List[ImageSeedParseResult] | None = None

The results of parsing URIs mentioned in RenderLoopConfig.image_seeds, will only be available if RenderLoopConfig.check() has been called.

prompts: List[Prompt]

List of prompt objects, this corresponds to the --prompts argument of the dgenerate command line tool.

revision: str = 'main'

Repo revision selector for the main model when loading from a huggingface repository. This corresponds to the --revision argument of the dgenerate command line tool.

safety_checker: bool = False

Enable safety checker? --safety-checker

scheduler: str | None = None

Optional primary model scheduler/sampler class name specification, this corresponds to the --scheduler argument of the dgenerate command line tool. Setting this to ‘help’ will yield a help message to stdout describing scheduler names compatible with the current configuration upon running.

sdxl_aesthetic_scores: List[float] | None = None

Optional list of SDXL aesthetic-score conditioning values, this corresponds to the --sdxl-aesthetic-scores argument of the dgenerate command line tool.

sdxl_crops_coords_top_left: List[Tuple[int, int]] | None = None

Optional list of SDXL top-left-crop-coords micro-conditioning parameters, this corresponds to the --sdxl-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_high_noise_fractions: List[float] | None = None

Optional list of SDXL refiner high noise fractions (floats), this value is the fraction of inference steps that the base model handles, the inverse proportion of the provided fraction is handled by the refiner model. This corresponds to the --sdxl-high-noise-fractions argument of the dgenerate command line tool.

sdxl_negative_aesthetic_scores: List[float] | None = None

Optional list of negative influence SDXL aesthetic-score conditioning values, this corresponds to the --sdxl-negative-aesthetic-scores argument of the dgenerate command line tool.

sdxl_negative_crops_coords_top_left: List[Tuple[int, int]] | None = None

Optional list of negative influence SDXL top-left crop coords micro-conditioning parameters, this corresponds to the --sdxl-negative-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_negative_original_sizes: List[Tuple[int, int]] | None = None

Optional list of negative influence SDXL original-size micro-conditioning parameters, this corresponds to the --sdxl-negative-original-sizes argument of the dgenerate command line tool.

sdxl_negative_target_sizes: List[Tuple[int, int]] | None = None

Optional list of negative influence SDXL target-size micro-conditioning parameters, this corresponds to the --sdxl-negative-target-sizes argument of the dgenerate command line tool.

sdxl_original_sizes: List[Tuple[int, int]] | None = None

Optional list of SDXL original-size micro-conditioning parameters, this corresponds to the --sdxl-original-sizes argument of the dgenerate command line tool.

sdxl_refiner_aesthetic_scores: List[float] | None = None

Optional list of SDXL-refiner override aesthetic-score conditioning values, this corresponds to the --sdxl-refiner-aesthetic-scores argument of the dgenerate command line tool.

sdxl_refiner_clip_skips: List[int] | None = None

Clip skip override values for the SDXL refiner, which normally defaults to the clip skip value for the main model when it is defined.

sdxl_refiner_crops_coords_top_left: List[Tuple[int, int]] | None = None

Optional list of SDXL-refiner override top-left-crop-coords micro-conditioning parameters, this corresponds to the --sdxl-refiner-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_refiner_guidance_rescales: List[float] | None = None

Optional list of guidance rescale value overrides for the SDXL refiner, this corresponds to the --sdxl-refiner-guidance-rescales argument of the dgenerate command line tool.

sdxl_refiner_guidance_scales: List[float] | None = None

Optional list of guidance scale value overrides for the SDXL refiner, this corresponds to the --sdxl-refiner-guidance-scales argument of the dgenerate command line tool.

sdxl_refiner_inference_steps: List[int] | None = None

Optional list of inference steps value overrides for the SDXL refiner, this corresponds to the --sdxl-refiner-inference-steps argument of the dgenerate command line tool.

sdxl_refiner_negative_aesthetic_scores: List[float] | None = None

Optional list of negative influence SDXL-refiner override aesthetic-score conditioning values, this corresponds to the --sdxl-refiner-negative-aesthetic-scores argument of the dgenerate command line tool.

sdxl_refiner_negative_crops_coords_top_left: List[Tuple[int, int]] | None = None

Optional list of negative influence SDXL-refiner top-left crop coords micro-conditioning parameters, this corresponds to the --sdxl-refiner-negative-crops-coords-top-left argument of the dgenerate command line tool.

sdxl_refiner_negative_original_sizes: List[Tuple[int, int]] | None = None

Optional list of negative influence SDXL-refiner override original-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-negative-original-sizes argument of the dgenerate command line tool.

sdxl_refiner_negative_target_sizes: List[Tuple[int, int]] | None = None

Optional list of negative influence SDXL-refiner override target-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-negative-target-sizes argument of the dgenerate command line tool.

sdxl_refiner_original_sizes: List[Tuple[int, int]] | None = None

Optional list of SDXL-refiner override original-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-original-sizes argument of the dgenerate command line tool.

sdxl_refiner_prompts: List[Prompt] | None = None

Optional list of SDXL refiner prompt overrides, this corresponds to the --sdxl-refiner-prompts argument of the dgenerate command line tool.

sdxl_refiner_scheduler: str | None = None

Optional SDXL refiner model scheduler/sampler class name specification, this corresponds to the --sdxl-refiner-scheduler argument of the dgenerate command line tool. Setting this to ‘help’ will yield a help message to stdout describing scheduler names compatible with the current configuration upon running.

sdxl_refiner_second_prompts: List[Prompt] | None = None

Optional list of SDXL refiner secondary prompt overrides, this corresponds to the --sdxl-refiner-second-prompts argument of the dgenerate command line tool.

sdxl_refiner_target_sizes: List[Tuple[int, int]] | None = None

Optional list of SDXL-refiner override target-size micro-conditioning parameters, this corresponds to the --sdxl-refiner-target-sizes argument of the dgenerate command line tool.

sdxl_refiner_uri: str | None = None

SDXL Refiner model URI, --sdxl-refiner argument of dgenerate command line tool.

sdxl_second_prompts: List[Prompt] | None = None

Optional list of SDXL secondary prompts, this corresponds to the --sdxl-secondary-prompts argument of the dgenerate command line tool.

sdxl_target_sizes: List[Tuple[int, int]] | None = None

Optional list of SDXL target-size micro-conditioning parameters, this corresponds to the --sdxl-target-sizes argument of the dgenerate command line tool.

seed_image_preprocessors: List[str] | None = None

Corresponds to the --seed-image-preprocessors argument of the dgenerate command line tool verbatim.

seeds: List[int]

List of integer seeds, this corresponds to the --seeds argument of the dgenerate command line tool.

seeds_to_images: bool = False

Should RenderLoopConfig.seeds be interpreted as seeds for each image input instead of combinatorial? this includes control images.

subfolder: str | None = None

Primary model subfolder argument, --subfolder argument of dgenerate command line tool.

textual_inversion_uris: List[str] | None = None

Optional user specified Textual Inversion URIs, this corresponds to the --textual-inversions argument of the dgenerate command line tool.

upscaler_noise_levels: List[int] | None = None

Optional list of integer upscaler noise levels, this corresponds to the --upscaler-noise-levels argument of the dgenerate command line tool that is used for the dgenerate.pipelinewrapper.ModelTypes.TORCH_UPSCALER_X4 model type only.

vae_slicing: bool = False

Enable VAE slicing? --vae-slicing

vae_tiling: bool = False

Enable VAE tiling? --vae-tiling

vae_uri: str | None = None

Optional user specified VAE URI, this corresponds to the --vae argument of the dgenerate command line tool.

variant: str | None = None

Primary model weights variant string. This corresponds to the --variant argument of the dgenerate command line tool.

dgenerate.renderloop.gen_seeds(n)

Generate a list of N random seed integers

Parameters:

n – number of seeds to generate

Returns:

list of integer seeds

dgenerate.renderloop.iterate_attribute_combinations(attribute_defs: List[Tuple[str, List]], my_class: Type) → Iterator

Iterate over every combination of attributes in a given class using a list of tuples mapping attribute names to a list of possible values.

Parameters:

• attribute_defs – list of tuple (attribute_name, [list of values])

• my_class – Construct this class and assign attribute values to it

Returns:

an iterator over instances of the type mentioned in the my_class argument

dgenerate.filelock module

dgenerate.filelock.suffix_path_maker(filenames, suffix)

To be used with touch_avoid_duplicate(), a pathmaker implementation that appends a suffix and a number to a filename or list of files when a duplicate is detected for any of them in the directory.

Parameters:

• filenames – Original filename, or a list of filenames

• suffix – Suffix to append if needed, a trailing number will be appended

Returns:

dgenerate.filelock.temp_file_lock(path)

Multiprocess synchronization utility.

Get a lock on an empty file as a context manager, delete the lock file if possible when done.

Parameters:

path – Path where the lock file will be created.

Returns:

Lock as a context manager

dgenerate.filelock.touch_avoid_duplicate(directory: str, path_maker: Callable[[str | None, int | None], str | List[str]], lock_name: str = '.lock', return_list=False)

Generate a filename in a directory and avoid duplicates using a file lock in that directory with a known name. Use to ensure duplicate checking in a directory is multiprocess safe, at least for processes using this function to write to the same directory.

Parameters:

• return_list – Always return a list even if generated paths is only of length 1, defaults to False, which means that a single string will be returned if only one path was generated by the pathmaker

• directory – The directory to create the lockfile in

• path_maker – Callback that generates paths until a non-existent path is found, first argument is the base filename and the second is attempt number. On the first attempt to create the files both arguments will be none, in which case the callback should return a single filename or list of filenames to touch with duplicate avoidance. Calls to the callback thereafter will have non None values for both arguments and the callback should take the passed base filename and apply a suffix using the attempt number.

• lock_name – Name of the lock file to be used as a mutex

Returns:

Unique path that has been touched (created but empty), or a tuple of paths if the path maker requested duplicate checks on multiple files

dgenerate.image module

dgenerate.image.align_by_8(x, y) → Tuple[int, int]

Align x and y by 8 and return a tuple

Parameters:

• x – x

• y – y

Returns:

tuple(x, y)

dgenerate.image.copy_img(img: Image)

Copy a PIL.Image.Image while preserving its filename attribute.

Parameters:

img – the image

Returns:

a copy of the image

dgenerate.image.get_filename(img: Image)

Get the PIL.Image.Image.filename attribute or “NO_FILENAME” if it does not exist.

Parameters:

img – PIL.Image.Image

Returns:

filename string or “NO_FILENAME”

dgenerate.image.is_aligned_by_8(x, y) → bool

Check if x and y are aligned by 8

Parameters:

• x – x

• y – y

Returns:

bool

dgenerate.image.resize_image(img: Image, size: Tuple[int, int] | None, aspect_correct: bool = True)

Resize a PIL.Image.Image and return a copy, resize is aligned to 8 pixels.

The filename attribute is preserved.

Parameters:

• img – the image

• size – the new size for the image

• aspect_correct – preserve aspect ratio?

Returns:

the resized image

dgenerate.image.resize_image_calc(old_size: Tuple[int, int], new_size: Tuple[int, int] | None, aspect_correct: bool = True)

Calculate the new dimensions for a requested resize of an image, align by 8 pixels.

Parameters:

• old_size – The old image size

• new_size – The new image size

• aspect_correct – preserve aspect ratio?

Returns:

calculated new size

dgenerate.image.to_rgb(img: Image)

Convert a PIL.Image.Image to RGB format while preserving its filename attribute.

Parameters:

img – the image

Returns:

a converted copy of the image

dgenerate.invoker module

dgenerate.invoker.invoke_dgenerate(args: Sequence[str], render_loop: RenderLoop | None = None, throw: bool = False)

Invoke dgenerate using its command line arguments and return a return code.

dgenerate is invoked in the current process, this method does not spawn a subprocess.

Parameters:

• args – dgenerate command line arguments in the form of a list, see: shlex module, or sys.argv

• render_loop – dgenerate.renderloop.RenderLoop instance, if None is provided one will be created.

• throw – Whether to throw exceptions or handle them.

Raises:

• dgenerate.arguments.DgenerateUsageError –

• dgenerate.mediainput.ImageSeedError –

• dgenerate.mediainput.UnknownMimetypeError –

• dgenerate.preprocessors.ImagePreprocessorArgumentError –

• dgenerate.preprocessors.ImagePreprocessorNotFoundError –

• dgenerate.pipelinewrapper.InvalidModelUriError –

• dgenerate.pipelinewrapper.InvalidSchedulerName –

• dgenerate.pipelinewrapper.OutOfMemoryError –

• dgenerate.pipelinewrapper.ModelNotFoundError –

• NotImplementedError –

• EnvironmentError –

Returns:

integer return-code, anything other than 0 is failure

dgenerate.mediainput module

exception dgenerate.mediainput.ImageSeedError

Bases: Exception

Raised on image seed parsing and loading errors.

exception dgenerate.mediainput.ImageSeedSizeMismatchError

Bases: ImageSeedError

Raised when the constituent image sources of an image seed specification are mismatched in dimension.

exception dgenerate.mediainput.UnknownMimetypeError

Bases: Exception

class dgenerate.mediainput.AnimatedImageReader(file: str | BinaryIO, file_source: str, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, preprocessor: ImagePreprocessor = None)

Bases: ImagePreprocessorMixin, AnimationReader

Implementation of AnimationReader that reads animated image formats using Pillow.

All frames from this animation reader will be aligned by 8 pixels.

__init__(file: str | BinaryIO, file_source: str, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, preprocessor: ImagePreprocessor = None)

Parameters:

• file – a file path or binary file stream

• file_source – the source filename for the animated image, should be the filename. this is for informational purpose when reading from a stream or a cached file and should be provided in every case even if it is a symbolic value only. It should possess a file extension. PIL.Image.Image objects produced by the reader will have this value set to their filename attribute.

• resize_resolution – Progressively resize each frame to this resolution while reading. The provided resolution will be aligned by 8 pixels.

• aspect_correct – Should resize operations be aspect correct?

• preprocessor – optionally preprocess every frame with this image preprocessor

class dgenerate.mediainput.AnimationReader(width: int, height: int, anim_fps: int, anim_frame_duration: float, total_frames: int, **kwargs)

Bases: object

Abstract base class for animation readers.

__init__(width: int, height: int, anim_fps: int, anim_frame_duration: float, total_frames: int, **kwargs)

Parameters:

• width – width of the animation, X dimension

• height – height of the animation, Y dimension

• anim_fps – frames per second

• anim_frame_duration – frame duration in milliseconds

• total_frames – total frames in the animation

• kwargs – for mixins

property anim_fps: int

Frame per second.

Returns:

float or integer

property anim_frame_duration: float

Duration of each frame in milliseconds.

Returns:

duration

property height: int

Height dimension, (Y dimension).

Returns:

height

property size: Tuple[int, int]

returns (width, height) as a tuple.

Returns:

(width, height)

property total_frames: int

Total number of frames that can be read.

Returns:

count

property width: int

Width dimension, (X dimension).

Returns:

width

class dgenerate.mediainput.AnimationReaderSpec(path: str, preprocessor: ImagePreprocessor | None = None, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True)

Bases: object

Used by MultiAnimationReader to define resource paths.

__init__(path: str, preprocessor: ImagePreprocessor | None = None, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True)

Parameters:

• path – File path or URL

• resize_resolution – Resize resolution

• aspect_correct – Aspect correct resize enabled?

• preprocessor – Optional image preprocessor associated with the file

aspect_correct: bool = True

Aspect correct resize enabled?

path: str

File path (or HTTP/HTTPS URL with default path_opener)

preprocessor: ImagePreprocessor | None = None

Optional image preprocessor associated with the file

resize_resolution: Tuple[int, int] | None = None

Optional resize resolution.

class dgenerate.mediainput.ImageSeed(image: Image | None = None, mask_image: Image | None = None, control_images: List[Image] | None = None, floyd_image: Image | None = None)

Bases: object

An ImageSeed with attached image data

__init__(image: Image | None = None, mask_image: Image | None = None, control_images: List[Image] | None = None, floyd_image: Image | None = None)

anim_fps: int | float | None = None

Frames per second in the case that ImageSeed.is_animation_frame is True

anim_frame_duration: float | None = None

Duration of a frame in milliseconds in the case that ImageSeed.is_animation_frame is True

control_images: List[Image] | None

List of control guidance images, or None.

floyd_image: Image | None

An optional image from a Deep Floyd IF stage, used for disambiguation in the case of using Deep Floyd for img2img and inpainting, where the un-varied input image is needed as a parameter for both stages. This image is used to define the image that was generated by Deep Floyd in a previous stage and to be used in the next stage, where ImageSeed.image defines the img2img image that you want a variation of.

This image will never be assigned a value when ImageSeed.control_image has a a value. As that is considered incorrect –image-seeds

frame_index: int | None = None

Frame index in the case that ImageSeed.is_animation_frame is True

image: Image | None

An optional image used for img2img mode, or inpainting mode in combination with ImageSeed.mask

is_animation_frame: bool

Is this part of an animation?

mask_image: Image | None

An optional inpaint mask image, may be None

total_frames: int | None = None

Total frame count in the case that ImageSeed.is_animation_frame is True

class dgenerate.mediainput.ImageSeedInfo(is_animation: bool, total_frames: int | None, anim_fps: int | None, anim_frame_duration: float | None)

Bases: object

Information acquired about an --image-seeds uri

__init__(is_animation: bool, total_frames: int | None, anim_fps: int | None, anim_frame_duration: float | None)

anim_fps: int | None

Animation frames per second in the case that ImageSeedInfo.is_animation is True

anim_frame_duration: float | None

Animation frame duration in milliseconds in the case that ImageSeedInfo.is_animation is True

is_animation: bool

Does this image seed specification result in an animation?

total_frames: int | None

Animation frame count in the case that ImageSeedInfo.is_animation is True

class dgenerate.mediainput.ImageSeedParseResult

Bases: object

The result of parsing an --image-seeds uri

get_control_image_paths() → List[str] | None

Split ImageSeed.seed_path by ‘,’ if ImageSeed.is_single_spec is True and return the result.

If the image seed is not a single specification, split ImageSeed.control_path and return the result.

If ImageSeed.control_path is not set and the image seed is not a single specification, return None.

Returns:

list of resource paths, or None

aspect_correct: bool | None = None

Aspect correct resize setting override from the aspect image seed keyword argument, if this is None it was not specified. This value if defined should override any globally defined aspect correct resize setting.

control_path: str | None = None

Optional controlnet guidance path, or comma seperated list of controlnet guidance paths. This field is only used when the secondary syntax of --image-seeds is encountered.

In parses such as:

• --image-seeds "img2img.png;control=control.png"

• --image-seeds "img2img.png;control=control1.png, control2.png"

• --image-seeds "img2img.png;mask=mask.png;control=control.png"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png, control2.png"

floyd_path: str | None = None

Optional path to a result from a Deep Floyd IF stage, used only for img2img and inpainting mode with Deep Floyd. This is the only way to specify the image that was output by a stage in that case.

the arguments floyd and control are mutually exclusive.

In parses such as:

• --image-seeds "img2img.png;floyd=stage1-output.png"

• --image-seeds "img2img.png;mask=mask.png;floyd=stage1-output.png"

frame_end: int | None = None

Optional end frame specification for per-image seed slicing.

frame_start: int | None = None

Optional start frame specification for per-image seed slicing.

property is_single_spec: bool

Is this --image-seeds uri a single resource or resource group specification existing within the seed_path attribute of this object?

For instance could it be a single img2img image definition, or a controlnet guidance image or list of controlnet guidance images?

This requires that mask_path, control_path, and floyd_path are all undefined.

Possible parses which trigger this condition are:

• --image-seeds "img2img.png"

• --image-seeds "control-image.png"

• --image-seeds "control-image1.png, control-image2.png"

Since this is an ambiguous parse, it must be resolved later with the help of other specified arguments. Such as by the specification of --control-nets, which makes the intention unambiguous.

Returns:

bool

mask_path: str | None = None

Optional path to an inpaint mask, may be an HTTP/HTTPS URL or file path.

resize_resolution: Tuple[int, int] | None = None

Per image user specified resize resolution for all components of the --image-seed specification.

This field available in parses such as:

• --image-seeds "img2img.png;512x512"

• --image-seeds "img2img.png;mask.png;512x512"

• --image-seeds "img2img.png;control=control.png;resize=512x512"

• --image-seeds "img2img.png;control=control1.png, control2.png;resize=512x512"

• --image-seeds "img2img.png;mask=mask.png;control=control.png;resize=512x512"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png, control2.png;resize=512x512"

This should override any globally defined resize value.

seed_path: str

The seed path, contains an image path that will be used for img2img operations or the base image in inpaint operations. Or a controlnet guidance path, or a comma seperated list of controlnet guidance paths. A path being a file path, or an HTTP/HTTPS URL.

class dgenerate.mediainput.MockImageAnimationReader(img: Image, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, image_repetitions: int = 1, preprocessor: ImagePreprocessor = None)

Bases: ImagePreprocessorMixin, AnimationReader

Implementation of AnimationReader that repeats a single PIL image as many times as desired in order to mock/emulate an animation.

All frame images from this animation reader will be aligned by 8 pixels.

__init__(img: Image, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, image_repetitions: int = 1, preprocessor: ImagePreprocessor = None)

Parameters:

• img – source image to copy for each frame, the image is immediately copied once upon construction of the mock reader, and then once per frame thereafter. Your copy of the image can be disposed of after the construction of this object.

• resize_resolution – the source image will be resized to this dimension with a maintained aspect ratio. This occurs once upon construction, a copy is then yielded for each frame that is read. The provided resolution will be aligned by 8 pixels.

• aspect_correct – Should resize operations be aspect correct?

• image_repetitions – number of frames that this mock reader provides using a copy of the source image.

• preprocessor – optionally preprocess the initial image with this image preprocessor, this occurs once.

property total_frames: int

Settable total_frames property.

Returns:

frame count

class dgenerate.mediainput.MultiAnimationReader(specs: ~typing.List[~dgenerate.mediainput.AnimationReaderSpec], frame_start: int = 0, frame_end: int | None = None, path_opener: ~typing.Callable[[str], ~typing.BinaryIO] = <function fetch_media_data_stream>)

Bases: object

Zips together multiple automatically created AnimationReader implementations and allows enumeration over their reads, which are collected into a list of a defined order.

Images when zipped together with animated files will be repeated over the total amount of frames.

The animation with the lowest amount of frames determines the total amount of frames that can be read when animations are involved.

If all paths point to images, then MultiAnimationReader.total_frames will be 1.

All images read by this reader are aligned by 8 pixels, there is no guarantee that images read from the individual readers are the same size and you must handle that condition.

__init__(specs: ~typing.List[~dgenerate.mediainput.AnimationReaderSpec], frame_start: int = 0, frame_end: int | None = None, path_opener: ~typing.Callable[[str], ~typing.BinaryIO] = <function fetch_media_data_stream>)

Parameters:

• specs – list of AnimationReaderSpec

• frame_start – inclusive frame slice start frame

• frame_end – inclusive frame slice end frame

• path_opener – opens a binary file stream from paths mentioned by AnimationReaderSpec

property anim_fps: int | None

Frames per second, this will be None if there is only a single frame

property anim_frame_duration: float | None

Duration of a frame in milliseconds, this will be None if there is only a single frame

property frame_end: int

Frame slice end value (inclusive)

property frame_index: int

Current frame index while reading.

property frame_start: int

Frame slice start value (inclusive)

property total_frames: int

Total number of frames readable from this reader.

class dgenerate.mediainput.VideoReader(file: str | BinaryIO, file_source: str, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, preprocessor: ImagePreprocessor = None)

Bases: ImagePreprocessorMixin, AnimationReader

Implementation AnimationReader that reads Video files with PyAV.

All frame images from this animation reader will be aligned by 8 pixels.

__init__(file: str | BinaryIO, file_source: str, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, preprocessor: ImagePreprocessor = None)

Parameters:

• file – a file path or binary file stream

• file_source – the source filename for the video data, should be the filename. this is for informational purpose when reading from a stream or a cached file and should be provided in every case even if it is a symbolic value only. It should possess a file extension as it is used to determine file format when reading from a byte stream.:py:class:PIL.Image.Image objects produced by the reader will have this value set to their filename attribute.

• resize_resolution – Progressively resize each frame to this resolution while reading. The provided resolution will be aligned by 8 pixels.

• aspect_correct – Should resize operations be aspect correct?

• preprocessor – optionally preprocess every frame with this image preprocessor

dgenerate.mediainput.create_animation_reader(mimetype: str, file_source: str, file: BinaryIO, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, preprocessor: ImagePreprocessor | None = None) → AnimationReader

Create an animation reader object from mimetype specification and binary file stream.

Images will return a MockImageAnimationReader with a total_frames value of 1, which can then be adjusted by you.

VideoReader or AnimatedImageReader will be returned for Video files and Animated Images respectively.

Parameters:

• mimetype – one of get_supported_mimetypes()

• file – the binary file stream

• file_source – the source filename for videos/animated images, should be the filename. this is for informational purpose and should be provided in every case even if it is a symbolic value only. It should possess a file extension. PIL.Image.Image objects produced by the reader will have this value set to their filename attribute.

• resize_resolution – Progressively resize each frame to this resolution while reading. The provided resolution will be aligned by 8 pixels.

• aspect_correct – Should resize operations be aspect correct?

• preprocessor – optionally preprocess every frame with this image preprocessor

Returns:

AnimationReader

dgenerate.mediainput.create_image(path_or_file: BinaryIO | str, file_source: str, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True) → Image

Create an RGB format PIL image from a file path or binary file stream. The image is oriented according to any EXIF directives. Image is aligned to 8 pixels in every case.

Parameters:

• path_or_file – file path or binary IO object

• file_source – PIL.Image.Image.filename is set to this value

• resize_resolution – Optional resize resolution

• aspect_correct – preserve aspect ratio when resize_resolution is specified?

Returns:

PIL.Image.Image

dgenerate.mediainput.fetch_media_data_stream(uri: str) → Tuple[str, BinaryIO]

Get an open stream to a local file, or file at an HTTP or HTTPS URL, with caching for web files.

Caching for downloaded files is multiprocess safe, multiple processes using this module can share the cache simultaneously, the last process alive clears the cache when it exits.

Parameters:

uri – Local file path or URL

Raises:

UnknownMimetypeError –

Return type:

(mime-type string, BinaryIO)

dgenerate.mediainput.frame_slice_count(total_frames: int, frame_start: int, frame_end: int | None = None) → int

Calculate the number of frames resulting from frame slicing.

Parameters:

• total_frames – Total frames being sliced from

• frame_start – The start frame

• frame_end – The end frame

Returns:

int

dgenerate.mediainput.get_control_image_info(uri: str | ImageSeedParseResult, frame_start: int = 0, frame_end: int | None = None) → ImageSeedInfo

Get an informational object from a dgenerate --image-seeds uri that is known to be a control image/video specification.

This can consist of a single resource path or a list of comma seperated image and video resource paths, which may be files on disk or remote files (http / https).

This method is to be used when it is known that there is only a control image/video specification in the path, and it handles this specification syntax:

• --image-seeds "control1.png"

• --image-seeds "control1.png, control2.png"

Parameters:

• uri – The path string or ImageSeedParseResult

• frame_start – slice start

• frame_end – slice end

Returns:

ImageSeedInfo

dgenerate.mediainput.get_image_seed_info(uri: str | ImageSeedParseResult, frame_start: int = 0, frame_end: int | None = None) → ImageSeedInfo

Get an informational object from a dgenerate --image-seeds uri.

This method is used to obtain information about an --image-seeds uri in the case that the image source mentioned is to be used for img2img / inpaint operations, and handles this syntax:

• --image-seeds "img2img.png;mask.png"

• --image-seeds "img2img.png;mask.png;512x512"

Additionally control net image sources are handled via the secondary syntax:

• --image-seeds "img2img.png;control=control1.png, control2.png"

• --image-seeds "img2img.png;control=control1.png, control2.png;resize=512x512"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png, control2.png"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png, control2.png;resize=512x512"

Parameters:

• uri – The uri string or ImageSeedParseResult

• frame_start – slice start

• frame_end – slice end

Returns:

ImageSeedInfo

dgenerate.mediainput.get_supported_animated_image_mimetypes() → List[str]

dgenerate.mediainput.get_supported_image_mimetypes() → List[str]

Get all supported --image-seeds image mimetypes, including animated image mimetypes

Returns:

list of strings

dgenerate.mediainput.get_supported_mimetypes() → List[str]

Get all supported --image-seeds mimetypes, video mimetype may contain a wildcard.

Returns:

list of strings

dgenerate.mediainput.get_supported_static_image_mimetypes() → List[str]

dgenerate.mediainput.get_supported_video_mimetypes() → List[str]

Get all supported --image-seeds video mimetypes, may contain a wildcard

Returns:

list of strings

dgenerate.mediainput.get_web_cache_directory() → str

Get the default web cache directory or the value of the environmental variable DGENERATE_WEB_CACHE

Returns:

string (directory path)

dgenerate.mediainput.iterate_control_image(uri: str | ImageSeedParseResult, frame_start: int = 0, frame_end: int | None = None, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, preprocessor: ImagePreprocessor | List[ImagePreprocessor] | None = None) → Iterator[ImageSeed]

Parse and load a control image/video in an --image-seeds uri and return an iterator that produces ImageSeed objects while progressively reading that file.

One or more ImageSeed objects may be yielded depending on whether an animation is being read.

This can consist of a single resource path or a list of comma seperated image and video resource paths, which may be files on disk or remote files (http / https).

This method is to be used when it is known that there is only a controlnet guidance resource specification in the path, and it handles this specification syntax:

• --image-seeds "control1.png"

• --image-seeds "control1.png, control2.png"

• --image-seeds "control1.png, control2.png;512x512"

• --image-seeds "control1.png, control2.png;resize=512x512"

• --image-seeds "control1.png, control2.png;frame-start=2"

• --image-seeds "control1.png, control2.png;frame-start=2;frame-end=10"

• --image-seeds "control1.png, control2.png;resize=512x512;frame-start=2;frame-end=10"

The image or images read will be available from the ImageSeed.control_images attribute.

Parameters:

• uri – --image-seeds uri or ImageSeedParseResult

• frame_start – starting frame, inclusive value

• frame_end – optional end frame, inclusive value

• resize_resolution – optional global resize resolution. The URI syntax of image seeds allows for overriding this value.

• aspect_correct – should the global resize operation be aspect correct by default? The URI syntax for image seeds allows for overriding this value with the aspect keyword argument.

• preprocessor – optional dgenerate.preprocessors.ImagePreprocessor or list of them. A list is used to specify preprocessors for individual images in a multi guidance image specification such as uri = “img1.png, img2.png”. In the case that a multi guidance image specification is used and only one preprocessor is given, that preprocessor will be used on only the first image / video in the specification. Images in a guidance specification with no corresponding preprocessor value will have their preprocessor set to None, specifying extra preprocessors as compared to control guidance image sources will cause ValueError to be raised.

Raises:

ValueError – if there are more preprocessor values than there are control guidance image sources in the URI.

Returns:

an iterator over ImageSeed objects

dgenerate.mediainput.iterate_image_seed(uri: str | ImageSeedParseResult, frame_start: int = 0, frame_end: int | None = None, resize_resolution: Tuple[int, int] | None = None, aspect_correct: bool = True, seed_image_preprocessor: ImagePreprocessor | None = None, mask_image_preprocessor: ImagePreprocessor | None = None, control_image_preprocessor: ImagePreprocessor | List[ImagePreprocessor] | None = None) → Iterator[ImageSeed]

Parse and load images/videos in an --image-seeds uri and return an iterator that produces ImageSeed objects while progressively reading those files.

This method is used to iterate over an --image-seeds uri in the case that the image source mentioned is to be used for img2img / inpaint operations, and handles this syntax:

• --image-seeds "img2img.png"

• --image-seeds "img2img.png;mask.png"

• --image-seeds "img2img.png;mask.png;512x512"

Additionally controlnet guidance resources are handled via the secondary syntax:

• --image-seeds "img2img.png;control=control1.png, control2.png"

• --image-seeds "img2img.png;control=control1.png, control2.png;resize=512x512"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png, control2.png"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png, control2.png;resize=512x512"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png;frame-start=2"

• --image-seeds "img2img.png;mask=mask.png;control=control1.png;frame-start=2;frame-end=5"

Deep Floyd img2img and inpainting mode are handled via a tertiary syntax:

• --image-seeds "img2img.png;floyd=stage1-image.png"

• --image-seeds "img2img.png;mask=mask.png;floyd=stage2-image.png"

Note that all keyword arguments mentioned above can be used together, with exception of “control” and “floyd” which are mutually exclusive arguments.

One or more ImageSeed objects may be yielded depending on whether an animation is being read.

Parameters:

• uri – --image-seeds uri or ImageSeedParseResult

• frame_start – starting frame, inclusive value

• frame_end – optional end frame, inclusive value

• resize_resolution – optional global resize resolution. The URI syntax of image seeds allows for overriding this value.

• aspect_correct – should the global resize operation be aspect correct by default? The URI syntax for image seeds allows for overriding this value with the aspect keyword argument.

• seed_image_preprocessor – optional dgenerate.preprocessors.ImagePreprocessor

• mask_image_preprocessor – optional dgenerate.preprocessors.ImagePreprocessor

• control_image_preprocessor – optional dgenerate.preprocessors.ImagePreprocessor or list of them. A list is used to specify preprocessors for individual images in a multi guidance image specification such as uri = “img2img.png;control=img1.png, img2.png”. In the case that a multi guidance image specification is used and only one preprocessor is given, that preprocessor will be used on only the first image / video in the specification. Images in a guidance specification with no corresponding preprocessor value will have their preprocessor set to None, specifying extra preprocessors as compared to control guidance image sources will cause ValueError to be raised.

Raises:

ValueError – if there are more control_image_preprocessor values than there are control guidance image sources in the URI.

Returns:

an iterator over ImageSeed objects

dgenerate.mediainput.mime_type_is_supported(mimetype: str) → bool

Check if dgenerate supports a given input mimetype

Parameters:

mimetype – The mimetype string

Returns:

bool

dgenerate.mediainput.mimetype_is_animated_image(mimetype: str) → bool

Check if a mimetype is one that dgenerate considers an animated image

Parameters:

mimetype – The mimetype string

Returns:

bool

dgenerate.mediainput.mimetype_is_static_image(mimetype: str) → bool

Check if a mimetype is one that dgenerate considers a static image

Parameters:

mimetype – The mimetype string

Returns:

bool

dgenerate.mediainput.mimetype_is_video(mimetype: str) → bool

Check if a mimetype is a video mimetype supported by dgenerate

Parameters:

mimetype – The mimetype string

Returns:

bool

dgenerate.mediainput.parse_image_seed_uri(uri: str) → ImageSeedParseResult

Parse an --image-seeds uri into its constituents

Raises:

ImageSeedError – on parse errors

Parameters:

uri – --image-seeds uri

Returns:

ImageSeedParseResult

dgenerate.mediaoutput module

class dgenerate.mediaoutput.AnimatedImageWriter(filename: str, duration: float)

Bases: AnimationWriter

Animation writer for animated images such as GIFs and webp

__init__(filename: str, duration: float)

Parameters:

• filename – Filename to write to.

• duration – Frame duration, (duration of a single frame) in milliseconds.

end(new_file: str = None)

write(img: Image)

class dgenerate.mediaoutput.AnimationWriter

Bases: object

Interface for animation writers

__init__()

end(new_file: str = None)

write(pil_img_rgb: Image)

class dgenerate.mediaoutput.MultiAnimationWriter(animation_format: str, filename: str, fps: float | int, allow_overwrites=False)

Bases: AnimationWriter

Splits writes between N files with generated filename suffixes if necessary depending on how many images were written on the first write.

__init__(animation_format: str, filename: str, fps: float | int, allow_overwrites=False)

Parameters:

• animation_format – One of supported_animation_writer_formats()

• filename – The desired filename, if multiple images are written a suffix _animation_N will be appended for each file

• fps – Frames per second

• allow_overwrites – Allow overwrites of existing files? or append _duplicate_N, The overwrite dis-allowance is multiprocess safe between instances of this library.

end(new_file=None)

write(img: Image | List[Image])

class dgenerate.mediaoutput.VideoWriter(filename, fps: float | int)

Bases: AnimationWriter

Animation writer for MP4 h264 format video

__init__(filename, fps: float | int)

Parameters:

• filename – Filename to write to.

• fps – Frame rate, in frames per second.

end(new_file=None)

write(img: Image)

dgenerate.mediaoutput.create_animation_writer(animation_format: str, out_filename: str, fps: float | int)

Create an animation writer of a given format.

Parameters:

• animation_format – The animation format, see supported_animation_writer_formats()

• out_filename – the output file name

• fps – FPS

Returns:

AnimationWriter

dgenerate.mediaoutput.supported_animation_writer_formats()

Supported animation writer formats, file extensions with no period.

Returns:

list of file extensions.

dgenerate.memoize module

dgenerate.memoize.args_cache_key(args_dict: Dict[str, Any], custom_hashes: Dict[str, Callable[[Any], str]] = None)

Generate a cache key for a functions arguments to use for memoization.

Parameters:

• args_dict – The args dictionary of the function

• custom_hashes – Custom hash functions for specific argument names if needed

Returns:

string

dgenerate.memoize.memoize(cache: ~typing.Dict[str, ~typing.Any], exceptions: ~typing.Set[str] = None, hasher: ~typing.Callable[[~typing.Dict[str, ~typing.Any]], str] = <function args_cache_key>, on_hit: ~typing.Callable[[str, ~typing.Any], None] = None, on_create: ~typing.Callable[[str, ~typing.Any], None] = None)

Decorator used to Memoize a function using a dictionary as a value cache.

Parameters:

• cache – The dictionary to serve as a cache

• exceptions – Function arguments to ignore

• hasher – Responsible for hashing arguments and argument values

• on_hit – Called on cache hit for the wrapped function

• on_create – Called on cache miss for the wrapped function

Returns:

decorator

dgenerate.memoize.simple_cache_hit_debug(title: str, cache_key: str, cache_hit: Any)

Basic cache hit debug message for memoize() decorator on_hit parameter.

Messages are printed using dgenerate.messages.debug_log()

Example:

on_hit=lambda key, hit: simple_cache_hit_debug("My Object", key, hit)

Debug Prints:

Cache Hit, Loaded My Object: (fully qualified name of hit object), Cache Key: (key)

Parameters:

• title – Object Title

• cache_key – cache key

• cache_hit – cached object

dgenerate.memoize.simple_cache_miss_debug(title: str, cache_key: str, new: Any)

Basic cache hit debug message for memoize() decorator on_create parameter.

Messages are printed using dgenerate.messages.debug_log()

Example:

on_create=lambda key, hit: simple_cache_miss_debug("My Object", key, hit)

Debug Prints:

Cache Miss, Created My Object: (fully qualified name of new object), Cache Key: (key)

Parameters:

• title – Object Title

• cache_key – cache key

• new – newly created object

dgenerate.memoize.struct_hasher(obj: Any, custom_hashes: Dict[str, Callable[[Any], str]] = None) → str

Create a hash string from a simple objects public attributes.

Parameters:

• obj – the object

• custom_hashes – Custom hash functions for specific attribute names if needed

Returns:

string

dgenerate.memory module

exception dgenerate.memory.MemoryConstraintSyntaxError

Bases: Exception

Thrown by memory_constraints() on syntax errors or if an expression returns a non-boolean value

dgenerate.memory.bytes_best_human_unit(byte_count: int, delimiter='') → str

Return a string for humans from a byte count using an appropriate unit: IE 1KB, 1MB, 1GB etc.

Parameters:

• delimiter – add this string between the value and the unit

• byte_count – the byte count

Returns:

formatted string

dgenerate.memory.get_available_memory(measure='b')

Get the available memory remaining on the system in a selectable unit.

Parameters:

measure – one of (case insensitive): b (bytes), kb (kilobytes), mb (megabytes), gb (gigabytes), kib (kibibytes), mib (mebibytes), gib (gibibytes)

Returns:

Requested value.

dgenerate.memory.get_total_memory(measure='b')

Get the total physical memory on the system.

Parameters:

measure – one of (case insensitive): b (bytes), kb (kilobytes), mb (megabytes), gb (gigabytes), kib (kibibytes), mib (mebibytes), gib (gibibytes)

Returns:

Requested value.

dgenerate.memory.get_used_memory(measure='b', pid: int | None = None) → int

Get the memory used by a process in a selectable unit.

Parameters:

• measure – one of (case insensitive): b (bytes), kb (kilobytes), mb (megabytes), gb (gigabytes), kib (kibibytes), mib (mebibytes), gib (gibibytes)

• pid – The process PID to retrieve this information from, defaults to the current process.

Returns:

Requested value.

dgenerate.memory.get_used_memory_percent(pid: int | None = None) → float

Get the percentage of memory used by a process as a percentage of already used memory plus available virtual memory.

Parameters:

pid – PID of the process, defaults to the current process.

Returns:

A whole percentage, for example: 25.4

dgenerate.memory.get_used_total_memory_percent(pid: int | None = None) → float

Get the percentage of memory used by a process as a percentage of total system memory.

Parameters:

pid – PID of the process, defaults to the current process.

Returns:

A whole percentage, for example: 25.4

dgenerate.memory.memory_constraint_syntax_check(expression)

Syntax check an expression given to memory_constraints()

Parameters:

expression – the expression string

Raises:

MemoryConstraintSyntaxError – on syntax errors.

dgenerate.memory.memory_constraints(expressions: str | list | None, extra_vars: ~typing.Dict[str, int | float] | None = None, mode=<built-in function any>, pid: int | None = None) → bool

Evaluate a user boolean expression involving the processes used memory in bytes, used memory percent, and available system memory in bytes.

Available functions are:

• kb(bytes to kilobytes)

• mb(bytes to megabytes)

• gb(bytes to gigabytes)

• kib(bytes to kibibytes)

• mib(bytes to mebibytes)

• gib(bytes to gibibytes)

Available values are:

• used / u (memory currently used by the process in bytes)

• used_total_percent / utp (memory used by the process, as percent of total system memory, example: 25.4)

• used_percent / up (memory used by the process, as a percent of used + available memory, example 75.4)

• available / a (available memory remaining on the system in bytes that can be used without going to the swap)

• total / t (total memory on the system in bytes)

Example expressions:

• used > gb(1) (when the process has used more than 1GB of memory)

• used_total_percent > 25 (when the process has used more than 25 percent of system memory)

• used_percent > 25 (when the process has used more than 25 percent of virtual memory available to it)

• available < gb(2) (when the available memory on the system is less than 2GB)

Expressions may not be longer than 128 characters. However multiple expressions may be provided.

Raises:

• ValueError – if extra_vars overwrites a reserved variable name

• MemoryConstraintSyntaxError – on syntax errors or if the return value of an expression is not a boolean value.

Parameters:

• expressions – a string containing an expression or a list of expressions, If expressions is None or empty this function will return False.

• extra_vars – extra integer or float variables

• mode – the standard library function ‘any’ (equating to OR all expressions) or the standard library function ‘all’ (equating to AND all expressions). The default is ‘any’ which ORs all expressions.

• pid – PID of the process from which to acquire the ‘used’ and ‘used_percent’ variable values from, defaults to the current process.

Returns:

Boolean result of the expression

dgenerate.memory.memory_use_debug_string(pid=None)

Return a debug string using describing the memory consumption of a process and also available system memory.

Example:

“Used Memory: 465.25MB, Available Memory: 50.94GB, Used Percent: 0.91%, Total Memory: 68.64GB, Used Total Percent: 0.68%”

Where:

• Used Memory = get_used_memory()

• Available Memory = get_available_memory()

• Used Percent = get_used_memory_percent()

• Total Memory = get_total_memory()

• Used Percent Total = get_used_total_memory_percent()

Parameters:

pid – PID of the process to describe, defaults to the current process.

Returns:

formatted string

dgenerate.messages module

dgenerate.messages.add_logging_handler(callback: Callable[[ParamSpecArgs, int, bool, str], None])

Add your own logging handler callback.

Parameters:

callback – Callback accepting (*args, LEVEL, underline (bool), underline_char)

dgenerate.messages.debug_log(*func_or_str: Callable[[], Any] | Any, underline=False, underline_char='=')

Conditionally log strings or possibly expensive functions if LEVEL is set to DEBUG.

Parameters:

• func_or_str – objects to be stringified and printed or callables that return said objects

• underline – Underline this message?

• underline_char – Underline character.

dgenerate.messages.errors_to_null()

Force dgenerates error output to a null file.

dgenerate.messages.log(*args: Any, level=0, underline=False, underline_char='=')

Write a message to dgenerates log

Parameters:

• args – args, objects that will be stringified and joined with a space

• level – Log level, one of: INFO, WARNING, ERROR, DEBUG

• underline – Underline this message?

• underline_char – Underline character

Returns:

dgenerate.messages.messages_to_null()

Force dgenerates normal output to a null file.

dgenerate.messages.remove_logging_handler(callback: Callable[[ParamSpecArgs, int, bool, str], None])

Remove a logging handler callback by reference.

Parameters:

callback – The previously registered callback

dgenerate.messages.set_error_file(file: TextIO)

Set a file stream or file like object for dgenerates error output.

Parameters:

file – The file stream

dgenerate.messages.set_message_file(file: TextIO)

Set a file stream or file like object for dgenerates normal (non error) output.

Parameters:

file – The file stream

dgenerate.messages.AUTO_FLUSH_MESSAGES = True

Whether to auto flush the output stream when printing to stdout or the output file assigned with set_message_file().

Errors are printed to stderr which is unbuffered by default.

dgenerate.messages.DEBUG = 3

Log Level DEBUG

dgenerate.messages.ERROR = 2

Log Level ERROR

dgenerate.messages.INFO = 0

Log level INFO

dgenerate.messages.LEVEL = 0

Current Log Level (set-able)

dgenerate.messages.WARNING = 1

Log Level WARNING

dgenerate.pipelinewrapper module

dgenerate.pipelinewrapper.CACHE_MEMORY_CONSTRAINTS = ['used_percent > 70']

Cache constraint expressions for when to clear all model caches (DiffusionPipeline, VAE, and ControlNet), syntax provided via dgenerate.memory.memory_constraints()

If any of these constraints are met, a call to dgenerate.pipelinewrapper.enforce_cache_constraints() will call dgenerate.pipelinewrapper.clear_model_cache() and force a garbage collection.

dgenerate.pipelinewrapper.PIPELINE_CACHE_MEMORY_CONSTRAINTS = ['pipeline_size > (available * 0.75)']

Cache constraint expressions for when to clear the DiffusionPipeline cache, syntax provided via dgenerate.memory.memory_constraints()

If any of these constraints are met, a call to dgenerate.pipelinewrapper.enforce_pipeline_cache_constraints() will call dgenerate.pipelinewrapper.clear_pipeline_cache() and force a garbage collection.

Extra variables include: cache_size (the current estimated cache size in bytes), and pipeline_size (the estimated size of the new pipeline before it is brought into memory, in bytes)

dgenerate.pipelinewrapper.VAE_CACHE_MEMORY_CONSTRAINTS = ['control_net_size > (available * 0.75)']

Cache constraint expressions for when to clear the ControlNet cache, syntax provided via dgenerate.memory.memory_constraints()

If any of these constraints are met, a call to dgenerate.pipelinewrapper.enforce_control_net_cache_constraints() will call dgenerate.pipelinewrapper.clear_control_net_cache() and force a garbage collection.

Extra variables include: cache_size (the current estimated cache size in bytes), and control_net_size (the estimated size of the new ControlNet before it is brought into memory, in bytes)

dgenerate.pipelinewrapper.CONTROL_NET_CACHE_MEMORY_CONSTRAINTS = ['vae_size > (available * 0.75)']

Cache constraint expressions for when to clear VAE cache, syntax provided via dgenerate.memory.memory_constraints()

If any of these constraints are met, a call to dgenerate.pipelinewrapper.enforce_vae_cache_constraints() will call dgenerate.pipelinewrapper.clear_vae_cache() and force a garbage collection.

Extra variables include: cache_size (the current estimated cache size in bytes), and vae_size (the estimated size of the new VAE before it is brought into memory, in bytes)

DEFAULT_INFERENCE_STEPS = 30

Default value for inference steps.

dgenerate.pipelinewrapper.DEFAULT_GUIDANCE_SCALE = 5

Default value for guidance scale.

dgenerate.pipelinewrapper.DEFAULT_IMAGE_SEED_STRENGTH = 0.8

Default image seed strength for img2img.

dgenerate.pipelinewrapper.DEFAULT_IMAGE_GUIDANCE_SCALE = 1.5

Default image guidance scale for pix2pix.

dgenerate.pipelinewrapper.DEFAULT_SDXL_HIGH_NOISE_FRACTION = 0.8

Default SDXL high noise fraction.

dgenerate.pipelinewrapper.DEFAULT_X4_UPSCALER_NOISE_LEVEL = 20

Default x4 upscaler noise level.

dgenerate.pipelinewrapper.DEFAULT_OUTPUT_WIDTH = 512

Default output width for txt2img.

dgenerate.pipelinewrapper.DEFAULT_OUTPUT_HEIGHT = 512

Default output height for txt2img.

dgenerate.pipelinewrapper.DEFAULT_SDXL_OUTPUT_WIDTH = 1024

Default output width for SDXL txt2img.

dgenerate.pipelinewrapper.DEFAULT_SDXL_OUTPUT_HEIGHT = 1024

Default output height for SDXL txt2img.

dgenerate.pipelinewrapper.DEFAULT_FLOYD_IF_OUTPUT_WIDTH = 64

Default output width for Deep Floyd IF txt2img first stage.

dgenerate.pipelinewrapper.DEFAULT_FLOYD_IF_OUTPUT_HEIGHT = 64

Default output height for Deep Floyd IF txt2img first stage.

dgenerate.pipelinewrapper.DEFAULT_SEED = 0

Default RNG seed.

exception dgenerate.pipelinewrapper.InvalidControlNetUriError

Bases: InvalidModelUriError

Error in --control-nets uri

exception dgenerate.pipelinewrapper.InvalidDeviceOrdinalException

Bases: Exception

GPU in device specification (cuda:N) does not exist

exception dgenerate.pipelinewrapper.InvalidLoRAUriError

Bases: InvalidModelUriError

Error in --loras uri

exception dgenerate.pipelinewrapper.InvalidModelUriError

Bases: Exception

Thrown on model path syntax or logical usage error

exception dgenerate.pipelinewrapper.InvalidSDXLRefinerUriError

Bases: InvalidModelUriError

Error in --sdxl-refiner uri

exception dgenerate.pipelinewrapper.InvalidSchedulerName

Bases: Exception

Unknown scheduler name used

exception dgenerate.pipelinewrapper.InvalidTextualInversionUriError

Bases: InvalidModelUriError

Error in --textual-inversions uri

exception dgenerate.pipelinewrapper.InvalidVaeUriError

Bases: InvalidModelUriError

Error in --vae uri

exception dgenerate.pipelinewrapper.ModelNotFoundError

Bases: Exception

Raised when a specified model can not be located either locally or remotely

exception dgenerate.pipelinewrapper.OutOfMemoryError(message)

Bases: Exception

Raised when a GPU or processing device runs out of memory.

__init__(message)

exception dgenerate.pipelinewrapper.SchedulerHelpException

Bases: Exception

Not an error, runtime scheduler help was requested by passing “help” to a scheduler name argument of DiffusionPipelineWrapper.__init__() such as scheduler or sdxl_refiner_scheduler. Upon calling DiffusionPipelineWrapper.__call__() info was printed using dgenerate.messages.log(), then this exception raised to get out of the call stack.

class dgenerate.pipelinewrapper.DataTypes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Represents model precision

AUTO = 0

Auto selection.

FLOAT16 = 1

16 bit floating point.

FLOAT32 = 2

32 bit floating point.

class dgenerate.pipelinewrapper.DiffusionArguments

Bases: SetFromMixin

Represents all possible arguments for a DiffusionPipelineWrapper call.

describe_pipeline_wrapper_args() → str

Describe the pipeline wrapper arguments in a pretty, human-readable way, with word wrapping depending on console size or a maximum length depending on what stdout currently is.

Returns:

description string.

determine_pipeline_type()

Determine the dgenerate.pipelinewrapper.PipelineTypes needed to utilize these arguments.

Returns:

dgenerate.pipelinewrapper.PipelineTypes

get_pipeline_wrapper_kwargs()

Get the arguments dictionary needed to call DiffusionPipelineWrapper

Returns:

dictionary of argument names with values

batch_size: int | None = None

Number of images to produce in a single generation step on the same GPU.

Invalid to use with FLAX ModeTypes.

clip_skip: int | None = None

Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Only supported for model_type values torch and torch-sdxl, including with control_net_uris defined.

control_images: List[Image] | None = None

ControlNet guidance images to use if control_net_uris were given to the constructor of DiffusionPipelineWrapper.

All input images involved in a generation must match in dimension and be aligned by 8 pixels.

floyd_image: Image | None = None

The output image of the last stage when preforming img2img or inpainting generation with Deep Floyd. When preforming txt2img generation DiffusionArguments.image is used.

guidance_rescale: float | None = None

This value is only supported for certain dgenerate.pipelinewrapper.DiffusionPipelineWrapper configurations, an error will be produced when it is unsupported.

Guidance rescale factor proposed by [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://arxiv.org/pdf/2305.08891.pdf) guidance_scale is defined as φ in equation 16. of [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://arxiv.org/pdf/2305.08891.pdf). Guidance rescale factor should fix overexposure when using zero terminal SNR.

guidance_scale: float | None = None

A higher guidance scale value encourages the model to generate images closely linked to the text DiffusionArguments.prompt at the expense of lower image quality. Guidance scale is enabled when DiffusionArguments.guidance_scale > 1

height: Tuple[int, int] | None = None

Output image height.

Ignored when img2img, inpainting, or controlnet guidance images are involved.

Width will be the width of the input image in those cases.

Output image width, must be aligned by 8

image: Image | None = None

Image for img2img operations, or the base for inpainting operations.

All input images involved in a generation must match in dimension and be aligned by 8 pixels.

image_guidance_scale: float | None = None

This value is only relevant for pix2pix dgenerate.pipelinewrapper.ModelTypes.

Image guidance scale is to push the generated image towards the initial image DiffusionArguments.image. Image guidance scale is enabled by setting DiffusionArguments.image_guidance_scale > 1. Higher image guidance scale encourages to generate images that are closely linked to the source image DiffusionArguments.image, usually at the expense of lower image quality.

image_seed_strength: float | None = None

Image seed strength, which relates to how much an img2img source (image attribute) is used during generation. Between 0.001 (close to zero but not 0) and 1.0, the closer to 1.0 the less the image is used for generation, IE. the more creative freedom the AI has.

inference_steps: int | None = None

The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.

mask_image: Image | None = None

Mask image for inpainting operations.

All input images involved in a generation must match in dimension and be aligned by 8 pixels.

prompt: Prompt | None = None

Primary prompt

sdxl_aesthetic_score: float | None = None

Optional, defaults to 6.0. This argument is used for img2img and inpainting operations only Used to simulate an aesthetic score of the generated image by influencing the positive text condition. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

sdxl_crops_coords_top_left: Tuple[int, int] | None = None

Optional SDXL conditioning parameter. DiffusionArguments.sdxl_crops_coords_top_left can be used to generate an image that appears to be “cropped” from the position DiffusionArguments.sdxl_crops_coords_top_left downwards. Favorable, well-centered images are usually achieved by setting DiffusionArguments.sdxl_crops_coords_top_left to (0, 0). Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

sdxl_high_noise_fraction: float | None = None

SDXL high noise fraction. This proportion of timesteps/inference steps are handled by the primary model, while the inverse proportion is handled by the refiner model when an SDXL model_type value.

When the refiner is operating in edit mode the number of total inference steps for the refiner will be calculated in a different manner, currently the refiner operates in edit mode during generations involving ControlNets as well as inpainting.

In edit mode, the refiner uses img2img with an image seed strength to add details to the image instead of cooperative denoising, this image seed strength is calculated as (1.0 - DiffusionArguments.sdxl_high_noise_fraction), and the number of inference steps for the refiner is then calculated as (image_seed_strength * inference_steps).

sdxl_negative_aesthetic_score: float | None = None

Negative influence version of DiffusionArguments.sdxl_aesthetic_score

sdxl_negative_crops_coords_top_left: Tuple[int, int] | None = None

Negative influence version of DiffusionArguments.sdxl_crops_coords_top_left

sdxl_negative_original_size: Tuple[int, int] | None = None

This value is only supported for certain dgenerate.pipelinewrapper.DiffusionPipelineWrapper configurations, an error will be produced when it is unsupported. It is not known to be supported by pix2pix.

Optional SDXL conditioning parameter. To negatively condition the generation process based on a specific image resolution. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.

sdxl_negative_target_size: Tuple[int, int] | None = None

This value is only supported for certain dgenerate.pipelinewrapper.DiffusionPipelineWrapper configurations, an error will be produced when it is unsupported. It is not known to be supported by pix2pix.

Optional SDXL conditioning parameter. To negatively condition the generation process based on a target image resolution. It should be as same as the DiffusionArguments.target_size for most cases. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.

sdxl_original_size: Tuple[int, int] | None = None

Optional SDXL conditioning parameter. If DiffusionArguments.sdxl_original_size is not the same as DiffusionArguments.sdxl_target_size the image will appear to be down- or up-sampled. DiffusionArguments.sdxl_original_size defaults to (width, height) if not specified or the size of any input images provided. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

sdxl_refiner_aesthetic_score: float | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_aesthetic_score

sdxl_refiner_clip_skip: int | None = None

Clip skip override value for the SDXL refiner, which normally defaults to that of DiffusionArguments.clip_skip when it is defined.

sdxl_refiner_crops_coords_top_left: Tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_crops_coords_top_left

sdxl_refiner_guidance_rescale: float | None = None

Override the guidance rescale value used by the SDXL refiner, which is normally set to the value of DiffusionArguments.guidance_rescale.

sdxl_refiner_guidance_scale: float | None = None

Override the guidance scale used by the SDXL refiner, which is normally set to the value of DiffusionArguments.guidance_scale.

sdxl_refiner_inference_steps: int | None = None

Override the default amount of inference steps preformed by the SDXL refiner. Which is normally set to the value for the primary model.

The attribute DiffusionArguments.sdxl_high_noise_fraction still factors in to the actual amount of inference steps preformed.

sdxl_refiner_negative_aesthetic_score: float | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_aesthetic_score

sdxl_refiner_negative_crops_coords_top_left: Tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_crops_coords_top_left

sdxl_refiner_negative_original_size: Tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_original_size

sdxl_refiner_negative_target_size: Tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_negative_target_size

sdxl_refiner_original_size: Tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_original_size

sdxl_refiner_prompt: Prompt | None = None

Primary prompt for the SDXL refiner when a refiner URI is specified in the constructor of DiffusionPipelineWrapper. Usually the prompt attribute of this object is used, unless you override it by giving this attribute a value.

sdxl_refiner_second_prompt: Prompt | None = None

Secondary prompt for the SDXL refiner when a refiner URI is specified in the constructor of DiffusionPipelineWrapper. Usually the sdxl_refiner_prompt attribute of this object is used, unless you override it by giving this attribute a value.

sdxl_refiner_target_size: Tuple[int, int] | None = None

Override the refiner value usually taken from DiffusionArguments.sdxl_target_size

sdxl_second_prompt: Prompt | None = None

Secondary prompt for the SDXL main pipeline when a refiner URI is specified in the constructor of DiffusionPipelineWrapper. Usually the prompt attribute of this object is used, unless you override it by giving this attribute a value.

sdxl_target_size: Tuple[int, int] | None = None

Optional SDXL conditioning parameter. For most cases, DiffusionArguments.sdxl_target_size should be set to the desired height and width of the generated image. If not specified it will default to (width, height) or the size of any input images provided. Part of SDXL’s micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).

seed: int | None = None

An integer to serve as an RNG seed.

upscaler_noise_level: int | None = None

Upscaler noise level for the dgenerate.pipelinewrapper.ModelTypes.TORCH_UPSCALER_X4 model type only.

width: Tuple[int, int] | None = None

Output image width.

Ignored when img2img, inpainting, or controlnet guidance images are involved.

Width will be the width of the input image in those cases.

Output image width, must be aligned by 8

class dgenerate.pipelinewrapper.DiffusionPipelineWrapper(model_path: str, model_type: ModelTypes | str = ModelTypes.TORCH, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes | str = DataTypes.AUTO, vae_uri: str | None = None, vae_tiling: bool = False, vae_slicing: bool = False, lora_uris: str | List[str] | None = None, textual_inversion_uris: str | List[str] | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, sdxl_refiner_uri: str | None = None, sdxl_refiner_scheduler: str | None = None, device: str = 'cuda', safety_checker: bool = False, auth_token: str | None = None, local_files_only: bool = False, model_extra_modules=None, refiner_extra_modules=None)

Bases: object

Monolithic diffusion pipelines wrapper.

__call__(args: DiffusionArguments | None = None, **kwargs) → PipelineWrapperResult

Call the pipeline and generate a result.

Parameters:

• args – Optional DiffusionArguments

• kwargs – See DiffusionArguments.get_pipeline_wrapper_kwargs(), any keyword arguments given here will override values derived from the DiffusionArguments object given to the args parameter.

Raises:

• InvalidModelUriError –

• InvalidSchedulerName –

• OutOfMemoryError –

• NotImplementedError –

Returns:

PipelineWrapperResult

__init__(model_path: str, model_type: ModelTypes | str = ModelTypes.TORCH, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes | str = DataTypes.AUTO, vae_uri: str | None = None, vae_tiling: bool = False, vae_slicing: bool = False, lora_uris: str | List[str] | None = None, textual_inversion_uris: str | List[str] | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, sdxl_refiner_uri: str | None = None, sdxl_refiner_scheduler: str | None = None, device: str = 'cuda', safety_checker: bool = False, auth_token: str | None = None, local_files_only: bool = False, model_extra_modules=None, refiner_extra_modules=None)

gen_dgenerate_command(args: DiffusionArguments | None = None, extra_opts: List[Tuple[str] | Tuple[str, Any]] | None = None, **kwargs)

Generate a valid dgenerate command line invocation that reproduces this result.

Parameters:

• args – DiffusionArguments object to take values from

• extra_opts – Extra option pairs to be added to the end of reconstructed options of the dgenerate invocation, this should be a list of tuples of length 1 (switch only) or length 2 (switch with args)

• kwargs – pipeline wrapper keyword arguments, these will override values derived from any DiffusionArguments object given to the args argument. See: DiffusionArguments.get_pipeline_wrapper_kwargs

Returns:

A string containing the dgenerate command line needed to reproduce this result.

gen_dgenerate_config(args: DiffusionArguments | None = None, extra_opts: List[Tuple[str] | Tuple[str, Any]] | None = None, extra_comments: Sequence[str] | None = None, **kwargs)

Generate a valid dgenerate config file with a single invocation that reproduces this result.

Parameters:

• args – DiffusionArguments object to take values from

• extra_comments – Extra strings to use as comments after the initial version check directive

• extra_opts – Extra option pairs to be added to the end of reconstructed options of the dgenerate invocation, this should be a list of tuples of length 1 (switch only) or length 2 (switch with args)

• kwargs – pipeline wrapper keyword arguments, these will override values derived from any DiffusionArguments object given to the args argument. See: DiffusionArguments.get_pipeline_wrapper_kwargs

Returns:

The configuration as a string

recall_main_pipeline() → PipelineCreationResult

Fetch the last used main pipeline creation result, possibly the pipeline will be recreated if no longer in the in memory cache. If there is no pipeline currently created, which will be the case if an image was never generated yet, RuntimeError will be raised.

Raises:

RuntimeError –

Returns:

dgenerate.pipelinewrapper.PipelineCreationResult

recall_refiner_pipeline() → PipelineCreationResult

Fetch the last used refiner pipeline creation result, possibly the pipeline will be recreated if no longer in the in memory cache. If there is no refiner pipeline currently created, which will be the case if an image was never generated yet or a refiner model was not specified, RuntimeError will be raised.

Raises:

RuntimeError –

Returns:

dgenerate.pipelinewrapper.PipelineCreationResult

reconstruct_dgenerate_opts(args: DiffusionArguments | None = None, extra_opts: List[Tuple[str] | Tuple[str, Any]] | None = None, shell_quote=True, **kwargs) → List[Tuple[str] | Tuple[str, Any]]

Reconstruct dgenerates command line arguments from a particular set of pipeline wrapper call arguments.

Parameters:

• args – DiffusionArguments object to take values from

• extra_opts – Extra option pairs to be added to the end of reconstructed options, this should be a list of tuples of length 1 (switch only) or length 2 (switch with args)

• shell_quote – Shell quote and format the argument values? or return them raw.

• kwargs – pipeline wrapper keyword arguments, these will override values derived from any DiffusionArguments object given to the args argument. See: DiffusionArguments.get_pipeline_wrapper_kwargs

Returns:

List of tuples of length 1 or 2 representing the option

property auth_token: str | None

Current –auth-token value or None

property control_net_uris: List[str] | None

List of supplied –control-nets uri strings or None

property device: str

Currently set –device string

property dtype: DataTypes

Currently set dtype for the main model

property dtype_enum: DataTypes

Currently set –dtype enum value

property dtype_string: str

Currently set –dtype string value

property local_files_only: bool

Currently set value for local_files_only

Returns:

property lora_uris: List[str] | None

List of supplied –loras uri strings or None

property model_path: str

Model path for the main model

property model_type_enum: ModelTypes

Currently set --model-type enum value

property model_type_string: str

Currently set --model-type string value

property revision: str | None

Currently set revision for the main model or None

property safety_checker: bool

Safety checker enabled status

property scheduler: str | None

Selected scheduler name for the main model or None

property sdxl_refiner_scheduler: str | None

Selected scheduler name for the SDXL refiner or None

property sdxl_refiner_uri: str | None

Model path for the SDXL refiner or None

property subfolder: str | None

Selected model subfolder for the main model, (remote repo subfolder or local) or None

property textual_inversion_uris: List[str] | None

List of supplied –textual-inversions uri strings or None

property vae_slicing: bool

Current –vae-slicing status

property vae_tiling: bool

Current –vae-tiling status

property vae_uri: str | None

Selected –vae uri for the main model or None

property variant: str | None

Currently set variant for the main model or None

class dgenerate.pipelinewrapper.FlaxControlNetUri(model: str, revision: str | None = None, subfolder: str | None = None, dtype: DataTypes | str | None = None, scale: float = 1.0, from_torch: bool = False)

Bases: object

Representation of --control-nets uri when --model-type flax*

__init__(model: str, revision: str | None = None, subfolder: str | None = None, dtype: DataTypes | str | None = None, scale: float = 1.0, from_torch: bool = False)

load(dtype_fallback: DataTypes = DataTypes.AUTO, use_auth_token: str | None = None, local_files_only: bool = False) → Tuple[FlaxControlNetModel, Any]

Load a diffusers.FlaxControlNetModel from this URI.

Parameters:

• dtype_fallback – Fallback datatype if dtype was not specified in the URI.

• use_auth_token – Optional huggingface API auth token, used for downloading restricted repos that your account has access to.

• local_files_only – Avoid connecting to huggingface to download models and only use cached models?

Returns:

tuple (diffusers.FlaxControlNetModel, flax_control_net_params)

static parse(uri: str) → FlaxControlNetUri

Parse a --model-type flax* --control-nets uri specification and return an object representing its constituents

Parameters:

uri – string with --control-nets uri syntax

Raises:

InvalidControlNetUriError –

Returns:

FlaxControlNetPath

dtype: DataTypes | None

Model dtype (precision)

from_torch: bool

Load from a model format meant for torch?

model: str

Model path, huggingface slug

revision: str | None

Model repo revision

scale: float

ControlNet guidance scale

subfolder: str | None

Model repo subfolder

class dgenerate.pipelinewrapper.FlaxPipelineCreationResult(pipeline: FlaxDiffusionPipeline, flax_params: Dict[str, Any], parsed_vae_uri: FlaxVAEUri | None, flax_vae_params: Dict[str, Any] | None, parsed_control_net_uris: List[FlaxControlNetUri], flax_control_net_params: Dict[str, Any] | None)

Bases: PipelineCreationResult

__init__(pipeline: FlaxDiffusionPipeline, flax_params: Dict[str, Any], parsed_vae_uri: FlaxVAEUri | None, flax_vae_params: Dict[str, Any] | None, parsed_control_net_uris: List[FlaxControlNetUri], flax_control_net_params: Dict[str, Any] | None)

call(*args, **kwargs) → BaseOutput

Call pipeline

Parameters:

• args – forward args to pipeline

• kwargs – forward kwargs to pipeline

Returns:

A subclass of diffusers.utils.BaseOutput

flax_control_net_params: Dict[str, Any] | None

Flax specific ControlNet params object

flax_params: Dict[str, Any]

Flax specific Pipeline params object

flax_vae_params: Dict[str, Any] | None

Flax specific VAE params object

parsed_control_net_uris: List[FlaxControlNetUri]

Parsed ControlNet URIs if any were present

parsed_vae_uri: FlaxVAEUri | None

Parsed VAE URI if one was present

property pipeline: FlaxDiffusionPipeline

A created subclass of diffusers.FlaxDiffusionPipeline

class dgenerate.pipelinewrapper.FlaxPipelineFactory(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes = ModelTypes.FLAX, revision: str | None = None, subfolder: str | None = None, dtype: DataTypes = DataTypes.AUTO, vae_uri: str | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, safety_checker: bool = False, auth_token: str | None = None, extra_modules: Dict[str, Any] | None = None, local_files_only: bool = False)

Bases: object

Turns create_flax_diffusion_pipeline() into a factory that can recreate the same Flax pipeline over again, possibly from cache.

__call__() → FlaxPipelineCreationResult

Raises:

• ModelNotFoundError –

• InvalidModelUriError –

• InvalidSchedulerName –

• NotImplementedError –

Returns:

FlaxPipelineCreationResult

__init__(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes = ModelTypes.FLAX, revision: str | None = None, subfolder: str | None = None, dtype: DataTypes = DataTypes.AUTO, vae_uri: str | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, safety_checker: bool = False, auth_token: str | None = None, extra_modules: Dict[str, Any] | None = None, local_files_only: bool = False)

class dgenerate.pipelinewrapper.FlaxVAEUri(encoder: str, model: str, revision: str | None, subfolder: str | None, dtype: DataTypes | None)

Bases: object

Representation of --vae uri when --model-type flax*

__init__(encoder: str, model: str, revision: str | None, subfolder: str | None, dtype: DataTypes | None)

load(dtype_fallback: DataTypes = DataTypes.AUTO, use_auth_token: str | None = None, local_files_only=False) → Tuple[FlaxAutoencoderKL, Any]

Load a diffusers.FlaxAutoencoderKL VAE and its flax_params from this URI

Parameters:

• dtype_fallback – If the URI does not specify a dtype, use this dtype.

• use_auth_token – optional huggingface auth token.

• local_files_only – avoid downloading files and only look for cached files when the model path is a huggingface slug or blob link

Returns:

tuple (diffusers.FlaxAutoencoderKL, flax_vae_params)

static parse(uri: str) → FlaxVAEUri

Parse a --model-type flax* --vae uri and return an object representing its constituents

Parameters:

uri – string with --vae uri syntax

Raises:

InvalidVaeUriError –

Returns:

FlaxVAEPath

dtype: DataTypes | None

Model dtype (precision)

encoder: str

Encoder class name such as “FlaxAutoencoderKL”

model: str

Model path, huggingface slug, file path, or blob link

revision: str | None

Model repo revision

subfolder: str | None

Model repo subfolder

class dgenerate.pipelinewrapper.LoRAUri(model: str, revision: str | None = None, subfolder: str | None = None, weight_name: str | None = None, scale: float = 1.0)

Bases: object

Representation of a --loras uri

__init__(model: str, revision: str | None = None, subfolder: str | None = None, weight_name: str | None = None, scale: float = 1.0)

load_on_pipeline(pipeline: DiffusionPipeline, use_auth_token: str | None = None, local_files_only=False)

Load LoRA weights on to a pipeline using this URI

Parameters:

• pipeline – diffusers.DiffusionPipeline

• use_auth_token – optional huggingface auth token.

• local_files_only – avoid downloading files and only look for cached files when the model path is a huggingface slug

static parse(uri: str) → LoRAUri

Parse a --loras uri and return an object representing its constituents

Parameters:

uri – string with --loras uri syntax

Raises:

InvalidLoRAUriError –

Returns:

LoRAPath

model: str

Model path, huggingface slug, file path

revision: str | None

Model repo revision

scale: float

LoRA scale

subfolder: str | None

Model repo subfolder

weight_name: str | None

Model weight-name

class dgenerate.pipelinewrapper.ModelTypes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Enum representation of --model-type

FLAX = 9

Stable Diffusion, such as SD 1.0 - 2.x, with Flax / Jax parallelization.

TORCH = 0

Stable Diffusion, such as SD 1.0 - 2.x

TORCH_IF = 3

Deep Floyd IF stage 1

TORCH_IFS = 4

Deep Floyd IF superscaler (stage 2)

TORCH_IFS_IMG2IMG = 5

Deep Floyd IF superscaler (stage 2) image to image / variation mode.

TORCH_PIX2PIX = 1

Stable Diffusion pix2pix prompt guided editing.

TORCH_SDXL = 2

Stable Diffusion XL

TORCH_SDXL_PIX2PIX = 6

Stable Diffusion XL pix2pix prompt guided editing.

TORCH_UPSCALER_X2 = 7

Stable Diffusion X2 upscaler

TORCH_UPSCALER_X4 = 8

Stable Diffusion X4 upscaler

class dgenerate.pipelinewrapper.PipelineCreationResult(pipeline)

Bases: object

__init__(pipeline)

get_pipeline_modules(names=typing.Iterable[str])

Get associated pipeline module such as vae etc, in a dictionary mapped from name to module value.

Possible Module Names:

• vae

• text_encoder

• text_encoder_2

• tokenizer

• tokenizer_2

• safety_checker

• feature_extractor

• controlnet

• scheduler

• unet

If the module is not present or a recognized name, a ValueError will be thrown describing the module that is not part of the pipeline.

Raises:

ValueError –

Parameters:

names – module names, such as vae, text_encoder

Returns:

dictionary

property pipeline

class dgenerate.pipelinewrapper.PipelineTypes(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Represents possible diffusers pipeline types.

IMG2IMG = 2

Image to image mode. Generation seeded / controlled with an image in some fashion.

INPAINT = 3

Inpainting mode. Generation seeded / controlled with an image and a mask in some fashion.

TXT2IMG = 1

Text to image mode. Prompt only generation.

class dgenerate.pipelinewrapper.PipelineWrapperResult(images: List[Image] | None)

Bases: object

The result of calling DiffusionPipelineWrapper

__init__(images: List[Image] | None)

image_grid(cols_rows: Tuple[int, int])

Render an image grid from the images in this result.

Raises:

ValueError – if no images are present on this object. This is impossible if this object was produced by DiffusionPipelineWrapper.

Parameters:

cols_rows – columns and rows (WxH) desired as a tuple

Returns:

PIL.Image.Image

property image

The first image in the batch of requested batch size.

Returns:

PIL.Image.Image

property image_count

The number of images produced.

Returns:

int

images: List[Image] | None

class dgenerate.pipelinewrapper.SDXLRefinerUri(model: str, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes | str | None = None)

Bases: object

Representation of --sdxl-refiner uri

__init__(model: str, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes | str | None = None)

static parse(uri: str) → SDXLRefinerUri

Parse an --sdxl-refiner uri and return an object representing its constituents

Parameters:

uri – string with --sdxl-refiner uri syntax

Raises:

InvalidSDXLRefinerUriError –

Returns:

SDXLRefinerPath

dtype: DataTypes | None

Model dtype (precision)

model: str

Model path, huggingface slug

revision: str | None

Model repo revision

subfolder: str | None

Model repo subfolder

variant: str | None

Model repo revision

class dgenerate.pipelinewrapper.TextualInversionUri(model: str, revision: str | None = None, subfolder: str | None = None, weight_name: str | None = None)

Bases: object

Representation of --textual-inversions uri

__init__(model: str, revision: str | None = None, subfolder: str | None = None, weight_name: str | None = None)

load_on_pipeline(pipeline: DiffusionPipeline, use_auth_token: str | None = None, local_files_only=False)

Load Textual Inversion weights on to a pipeline using this URI

Parameters:

• pipeline – diffusers.DiffusionPipeline

• use_auth_token – optional huggingface auth token.

• local_files_only – avoid downloading files and only look for cached files when the model path is a huggingface slug

static parse(uri: str) → TextualInversionUri

Parse a --textual-inversions uri and return an object representing its constituents

Parameters:

uri – string with --textual-inversions uri syntax

Raises:

InvalidTextualInversionUriError –

Returns:

TextualInversionPath

model: str

Model path, huggingface slug, file path

revision: str | None

Model repo revision

subfolder: str | None

Model repo subfolder

weight_name: str | None

Model weight-name

class dgenerate.pipelinewrapper.TorchControlNetUri(model: str, revision: str | None, variant: str | None, subfolder: str | None, dtype: DataTypes | str | None = None, scale: float = 1.0, start: float = 0.0, end: float = 1.0)

Bases: object

Representation of --control-nets uri when --model-type torch*

__init__(model: str, revision: str | None, variant: str | None, subfolder: str | None, dtype: DataTypes | str | None = None, scale: float = 1.0, start: float = 0.0, end: float = 1.0)

load(dtype_fallback: DataTypes = DataTypes.AUTO, use_auth_token: str | None = None, local_files_only: bool = False) → ControlNetModel

Load a diffusers.ControlNetModel from this URI.

Parameters:

• dtype_fallback – Fallback datatype if dtype was not specified in the URI.

• use_auth_token – Optional huggingface API auth token, used for downloading restricted repos that your account has access to.

• local_files_only – Avoid connecting to huggingface to download models and only use cached models?

Returns:

diffusers.ControlNetModel

static parse(uri: str) → TorchControlNetUri

Parse a --model-type torch* --control-nets uri specification and return an object representing its constituents

Parameters:

uri – string with --control-nets uri syntax

Raises:

InvalidControlNetUriError –

Returns:

TorchControlNetPath

dtype: DataTypes | None

Model dtype (precision)

end: float

ControlNet guidance end point, fraction of inference / timesteps.

model: str

Model path, huggingface slug

revision: str | None

Model repo revision

scale: float

ControlNet guidance scale

start: float

ControlNet guidance start point, fraction of inference / timesteps.

subfolder: str | None

Model repo subfolder

variant: str | None

Model repo revision

class dgenerate.pipelinewrapper.TorchPipelineCreationResult(pipeline: DiffusionPipeline, parsed_vae_uri: TorchVAEUri | None, parsed_lora_uris: List[LoRAUri], parsed_textual_inversion_uris: List[TextualInversionUri], parsed_control_net_uris: List[TorchControlNetUri])

Bases: PipelineCreationResult

__init__(pipeline: DiffusionPipeline, parsed_vae_uri: TorchVAEUri | None, parsed_lora_uris: List[LoRAUri], parsed_textual_inversion_uris: List[TextualInversionUri], parsed_control_net_uris: List[TorchControlNetUri])

call(*args, **kwargs) → BaseOutput

Call pipeline

Parameters:

• args – forward args to pipeline

• kwargs – forward kwargs to pipeline

Returns:

A subclass of diffusers.utils.BaseOutput

parsed_control_net_uris: List[TorchControlNetUri]

Parsed ControlNet URIs if any were present

parsed_lora_uris: List[LoRAUri]

Parsed LoRA URIs if any were present

parsed_textual_inversion_uris: List[TextualInversionUri]

Parsed Textual Inversion URIs if any were present

parsed_vae_uri: TorchVAEUri | None

Parsed VAE URI if one was present

property pipeline: DiffusionPipeline

A created subclass of diffusers.DiffusionPipeline

class dgenerate.pipelinewrapper.TorchPipelineFactory(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes = ModelTypes.TORCH, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes = DataTypes.AUTO, vae_uri: str | None = None, lora_uris: str | List[str] | None = None, textual_inversion_uris: str | List[str] | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, safety_checker: bool = False, auth_token: str | None = None, device: str = 'cuda', extra_modules: Dict[str, Any] | None = None, model_cpu_offload: bool = False, sequential_cpu_offload: bool = False, local_files_only: bool = False, vae_tiling=False, vae_slicing=False)

Bases: object

Combines create_torch_diffusion_pipeline() and set_vae_slicing_tiling() into a factory that can recreate the same Torch pipeline over again, possibly from cache.

__call__() → TorchPipelineCreationResult

Raises:

• ModelNotFoundError –

• InvalidModelUriError –

• InvalidSchedulerName –

• NotImplementedError –

Returns:

TorchPipelineCreationResult

__init__(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes = ModelTypes.TORCH, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes = DataTypes.AUTO, vae_uri: str | None = None, lora_uris: str | List[str] | None = None, textual_inversion_uris: str | List[str] | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, safety_checker: bool = False, auth_token: str | None = None, device: str = 'cuda', extra_modules: Dict[str, Any] | None = None, model_cpu_offload: bool = False, sequential_cpu_offload: bool = False, local_files_only: bool = False, vae_tiling=False, vae_slicing=False)

class dgenerate.pipelinewrapper.TorchVAEUri(encoder: str, model: str, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes | str | None = None)

Bases: object

Representation of --vae uri when --model-type torch*

__init__(encoder: str, model: str, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes | str | None = None)

load(dtype_fallback: DataTypes = DataTypes.AUTO, use_auth_token: str | None = None, local_files_only=False) → AutoencoderKL | AsymmetricAutoencoderKL | AutoencoderTiny

Load a VAE of type diffusers.AutoencoderKL, diffusers.AsymmetricAutoencoderKL,

or diffusers.AutoencoderTiny from this URI

Parameters:

• dtype_fallback – If the URI does not specify a dtype, use this dtype.

• use_auth_token – optional huggingface auth token.

• local_files_only – avoid downloading files and only look for cached files when the model path is a huggingface slug or blob link

Returns:

diffusers.AutoencoderKL, diffusers.AsymmetricAutoencoderKL, or diffusers.AutoencoderTiny

static parse(uri: str) → TorchVAEUri

Parse a --model-type torch* --vae uri and return an object representing its constituents

Parameters:

uri – string with --vae uri syntax

Raises:

InvalidVaeUriError –

Returns:

TorchVAEPath

dtype: DataTypes | None

Model dtype (precision)

encoder: str

Encoder class name such as “AutoencoderKL”

model: str

Model path, huggingface slug, file path, or blob link

revision: str | None

Model repo revision

subfolder: str | None

Model repo subfolder

variant: str | None

Model repo revision

dgenerate.pipelinewrapper.clear_control_net_cache(collect=True)

Clear ControlNet cache and then garbage collect.

Parameters:

collect – Call gc.collect() ?

dgenerate.pipelinewrapper.clear_model_cache(collect=True)

Clear all in memory model caches and garbage collect.

Parameters:

collect – Call gc.collect() ?

dgenerate.pipelinewrapper.clear_pipeline_cache(collect=True)

Clear DiffusionPipeline cache and then garbage collect.

Parameters:

collect – Call gc.collect() ?

dgenerate.pipelinewrapper.clear_vae_cache(collect=True)

Clear VAE cache and then garbage collect.

Parameters:

collect – Call gc.collect() ?

dgenerate.pipelinewrapper.control_net_cache_size() → int

Return the estimated memory usage in bytes of all user specified ControlNet models currently cached in memory.

Returns:

memory usage in bytes.

dgenerate.pipelinewrapper.controlnet_create_update_cache_info(controlnet, estimated_size: int)

Add additional information about the size of a newly created ControlNet model to the cache.

Tag the object with an internal tag.

Parameters:

• controlnet – the ControlNet object

• estimated_size – size bytes

dgenerate.pipelinewrapper.create_flax_diffusion_pipeline(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes = ModelTypes.FLAX, revision: str | None = None, subfolder: str | None = None, dtype: DataTypes = DataTypes.AUTO, vae_uri: str | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, safety_checker: bool = False, auth_token: str | None = None, extra_modules: Dict[str, Any] | None = None, local_files_only: bool = False) → FlaxPipelineCreationResult

Create a diffusers.FlaxDiffusionPipeline in dgenerates in memory cacheing system.

Parameters:

• pipeline_type – py:class:dgenerate.pipelinewrapper.PipelineTypes enum value

• model_path – huggingface slug, huggingface blob link, path to folder on disk, path to file on disk

• model_type – Currently only accepts dgenerate.pipelinewrapper.ModelTypes.FLAX

• revision – huggingface repo revision (branch)

• subfolder – huggingface repo subfolder if applicable

• dtype – Optional py:class:dgenerate.pipelinewrapper.DataTypes enum value

• vae_uri – Optional Flax specific --vae URI string for specifying a specific VAE

• control_net_uris – Optional --control-nets URI strings for specifying ControlNet models

• scheduler – Optional scheduler (sampler) class name, unqualified, or “help” to print supported values to STDOUT and raise dgenerate.pipelinewrapper.SchedulerHelpException

• safety_checker – Safety checker enabled? default is false

• auth_token – Optional huggingface API token for accessing repositories that are restricted to your account

• extra_modules – Extra module arguments to pass directly into diffusers.FlaxDiffusionPipeline.from_pretrained()

• local_files_only – Only look in the huggingface cache and do not connect to download models?

Raises:

• ModelNotFoundError –

• InvalidModelUriError –

• InvalidSchedulerName –

• NotImplementedError –

Returns:

FlaxPipelineCreationResult

dgenerate.pipelinewrapper.create_torch_diffusion_pipeline(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes = ModelTypes.TORCH, revision: str | None = None, variant: str | None = None, subfolder: str | None = None, dtype: DataTypes = DataTypes.AUTO, vae_uri: str | None = None, lora_uris: str | List[str] | None = None, textual_inversion_uris: str | List[str] | None = None, control_net_uris: str | List[str] | None = None, scheduler: str | None = None, safety_checker: bool = False, auth_token: str | None = None, device: str = 'cuda', extra_modules: Dict[str, Any] | None = None, model_cpu_offload: bool = False, sequential_cpu_offload: bool = False, local_files_only: bool = False) → TorchPipelineCreationResult

Create a diffusers.DiffusionPipeline in dgenerates in memory cacheing system.

Parameters:

• pipeline_type – py:class:dgenerate.pipelinewrapper.PipelineTypes enum value

• model_type – py:class:dgenerate.pipelinewrapper.ModelTypes enum value

• model_path – huggingface slug, huggingface blob link, path to folder on disk, path to file on disk

• revision – huggingface repo revision (branch)

• variant – model weights name variant, for example ‘fp16’

• subfolder – huggingface repo subfolder if applicable

• dtype – Optional py:class:dgenerate.pipelinewrapper.DataTypes enum value

• vae_uri – Optional --vae URI string for specifying a specific VAE

• lora_uris – Optional --loras URI strings for specifying LoRA weights

• textual_inversion_uris – Optional --textual-inversions URI strings for specifying Textual Inversion weights

• control_net_uris – Optional --control-nets URI strings for specifying ControlNet models

• scheduler – Optional scheduler (sampler) class name, unqualified, or “help” to print supported values to STDOUT and raise dgenerate.pipelinewrapper.SchedulerHelpException

• safety_checker – Safety checker enabled? default is false

• auth_token – Optional huggingface API token for accessing repositories that are restricted to your account

• device – Optional --device string, defaults to “cuda”

• extra_modules – Extra module arguments to pass directly into diffusers.DiffusionPipeline.from_single_file() or diffusers.DiffusionPipeline.from_pretrained()

• model_cpu_offload – This pipeline has model_cpu_offloading enabled?

• sequential_cpu_offload – This pipeline has sequential_cpu_offloading enabled?

• local_files_only – Only look in the huggingface cache and do not connect to download models?

Raises:

• ModelNotFoundError –

• InvalidModelUriError –

• InvalidSchedulerName –

• NotImplementedError –

Returns:

TorchPipelineCreationResult

dgenerate.pipelinewrapper.enforce_cache_constraints(collect=True)

Enforce dgenerate.pipelinewrapper.CACHE_MEMORY_CONSTRAINTS and clear caches accordingly

Parameters:

collect – Call gc.collect() after a cache clear ?

Returns:

Whether any caches were cleared due to constraint expressions.

dgenerate.pipelinewrapper.enforce_control_net_cache_constraints(new_control_net_size, collect=True)

Enforce dgenerate.pipelinewrapper.CONTROL_NET_CACHE_MEMORY_CONSTRAINTS and clear the ControlNet cache if needed.

Parameters:

• new_control_net_size – estimated size in bytes of any new control net that is about to enter memory

• collect – Call gc.collect() after a cache clear ?

Returns:

Whether the cache was cleared due to constraint expressions.

dgenerate.pipelinewrapper.enforce_pipeline_cache_constraints(new_pipeline_size, collect=True)

Enforce dgenerate.pipelinewrapper.PIPELINE_CACHE_MEMORY_CONSTRAINTS and clear the diffusers.DiffusionPipeline cache if needed.

Parameters:

• new_pipeline_size – estimated size in bytes of any new pipeline that is about to enter memory

• collect – Call gc.collect() after a cache clear ?

Returns:

Whether the cache was cleared due to constraint expressions.

dgenerate.pipelinewrapper.enforce_vae_cache_constraints(new_vae_size, collect=True)

Enforce dgenerate.pipelinewrapper.VAE_CACHE_MEMORY_CONSTRAINTS and clear the VAE cache if needed.

Parameters:

• new_vae_size – estimated size in bytes of any new vae that is about to enter memory

• collect – Call gc.collect() after a cache clear ?

Returns:

Whether the cache was cleared due to constraint expressions.

dgenerate.pipelinewrapper.estimate_pipeline_memory_use(pipeline_type: PipelineTypes, model_path: str, model_type: ModelTypes, revision='main', variant=None, subfolder=None, vae_uri=None, lora_uris=None, textual_inversion_uris=None, safety_checker=False, auth_token=None, extra_args=None, local_files_only=False)

Estimate the CPU side memory use of a model.

Parameters:

• pipeline_type – dgenerate.pipelinewrapper.PipelineTypes

• model_path – huggingface slug, blob link, path to folder on disk, path to model file.

• model_type – dgenerate.pipelinewrapper.ModelTypes

• revision – huggingface repo revision if using a huggingface slug

• variant – model file variant desired, for example “fp16”

• subfolder – huggingface repo subfolder if using a huggingface slug

• vae_uri – optional user specified --vae URI that will be loaded on to the pipeline

• lora_uris – optional user specified --loras URIs that will be loaded on to the pipeline

• textual_inversion_uris – optional user specified --textual-inversion URIs that will be loaded on to the pipeline

• safety_checker – consider the safety checker? dgenerate usually loads the safety checker and then retroactively disables it if needed, so it usually considers the size of the safety checker model.

• auth_token – optional huggingface auth token to access restricted repositories that your account has access to.

• extra_args – extra_args as to be passed to create_torch_diffusion_pipeline() or create_flax_diffusion_pipeline()

• local_files_only – Only ever attempt to look in the local huggingface cache? if False the huggingface API will be contacted when necessary.

Returns:

size estimate in bytes.

dgenerate.pipelinewrapper.get_data_type_enum(id_str: DataTypes | str | None) → DataTypes

Convert a --dtype string to its DataTypes enum value

Parameters:

id_str – --dtype string

Returns:

DataTypes

dgenerate.pipelinewrapper.get_data_type_string(data_type_enum: DataTypes) → str

Convert a DataTypes enum value to its --dtype string

Parameters:

data_type_enum – DataTypes value

Returns:

--dtype string

dgenerate.pipelinewrapper.get_flax_dtype(dtype: DataTypes | str | Any | None)

Return a jax.numpy datatype from a DataTypes value, or a string, or a jax.numpy datatype itself.

Passing None results in None being returned.

Passing ‘auto’ or DataTypes.AUTO results in None being returned.

Parameters:

dtype – DataTypes, string, jax.numpy dtype, None

Returns:

jax.numpy dtype

dgenerate.pipelinewrapper.get_model_type_enum(id_str: ModelTypes | str) → ModelTypes

Convert a --model-type string to its ModelTypes enum value

Parameters:

id_str – --model-type string

Returns:

ModelTypes

dgenerate.pipelinewrapper.get_model_type_string(model_type_enum: ModelTypes) → str

Convert a ModelTypes enum value to its --model-type string

Parameters:

model_type_enum – ModelTypes value

Returns:

--model-type string

dgenerate.pipelinewrapper.get_pipeline_type_enum(id_str: PipelineTypes | str | None) → PipelineTypes

Get a PipelineTypes enum value from a string.

Parameters:

id_str – one of: “txt2img”, “img2img”, or “inpaint”

Returns:

PipelineTypes

dgenerate.pipelinewrapper.get_pipeline_type_string(pipeline_type_enum: PipelineTypes)

Convert a PipelineTypes enum value to a string.

Parameters:

pipeline_type_enum – PipelineTypes value

Returns:

one of: “txt2img”, “img2img”, or “inpaint”

dgenerate.pipelinewrapper.get_torch_dtype(dtype: DataTypes | dtype | str | None) → dtype | None

Return a torch.dtype datatype from a DataTypes value, or a string, or a torch.dtype datatype itself.

Passing None results in None being returned.

Passing ‘auto’ or DataTypes.AUTO results in None being returned.

Parameters:

dtype – DataTypes, string, torch.dtype, None

Returns:

torch.dtype

dgenerate.pipelinewrapper.have_jax_flax()

Do we have jax/flax support?

Returns:

bool

dgenerate.pipelinewrapper.is_valid_device_string(device, raise_ordinal=True)

dgenerate.pipelinewrapper.load_scheduler(pipeline: DiffusionPipeline | FlaxDiffusionPipeline, scheduler_name=None, model_path: str | None = None)

Load a specific compatible scheduler class name onto a huggingface diffusers pipeline object.

Parameters:

• pipeline – pipeline object

• scheduler_name – compatible scheduler class name, pass “help” to receive a print out to STDOUT and raise SchedulerHelpException

• model_path – Optional model path to be used in the message to STDOUT produced by passing “help”

Returns:

dgenerate.pipelinewrapper.model_type_is_flax(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an Flax model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_floyd(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an floyd “if” of “ifs” type model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_floyd_if(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an floyd “if” type model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_floyd_ifs(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an floyd “ifs” type model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_pix2pix(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an pix2pix type model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_sdxl(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an SDXL model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_torch(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an Torch model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.model_type_is_upscaler(model_type: ModelTypes | str) → bool

Does a --model-type string or ModelTypes enum value represent an upscaler model?

Parameters:

model_type – --model-type string or ModelTypes enum value

Returns:

bool

dgenerate.pipelinewrapper.pipeline_cache_size() → int

Return the estimated memory usage in bytes of all diffusers pipelines currently cached in memory.

Returns:

memory usage in bytes.

dgenerate.pipelinewrapper.pipeline_create_update_cache_info(pipeline, estimated_size: int)

Add additional information about the size of a newly created diffusers.DiffusionPipeline to the cache.

Tag the object with an internal tag.

Parameters:

• pipeline – the diffusers.DiffusionPipeline object

• estimated_size – size bytes

dgenerate.pipelinewrapper.pipeline_off_cpu_update_cache_info(pipeline: DiffusionPipeline | FlaxDiffusionPipeline)

dgenerate.pipelinewrapper.pipeline_to_cpu_update_cache_info(pipeline: DiffusionPipeline | FlaxDiffusionPipeline)

dgenerate.pipelinewrapper.scheduler_is_help(name: str | None)

This scheduler name is simply a request for help?, IE: “help”?

Parameters:

name – string to test

Returns:

true of false

dgenerate.pipelinewrapper.set_vae_slicing_tiling(pipeline: DiffusionPipeline | FlaxDiffusionPipeline, vae_tiling: bool, vae_slicing: bool)

Set the vae_slicing and vae_tiling status on a created huggingface diffusers pipeline.

Parameters:

• pipeline – pipeline object

• vae_tiling – tiling status

• vae_slicing – slicing status

Returns:

dgenerate.pipelinewrapper.supported_data_type_enums() → List[DataTypes]

Return a list of supported DataTypes enum values

dgenerate.pipelinewrapper.supported_data_type_strings()

Return a list of supported --dtype strings

dgenerate.pipelinewrapper.supported_model_type_enums() → List[ModelTypes]

Return a list of supported ModelTypes enum values

dgenerate.pipelinewrapper.supported_model_type_strings()

Return a list of supported --model-type strings

dgenerate.pipelinewrapper.uri_hash_with_parser(parser)

Create a hash function from a particular URI parser function that hashes a URI string.

The URI is parsed and then the object that results from parsing is hashed with dgenerate.memoize.struct_hasher().

Parameters:

parser – The URI parser function

Returns:

a hash function compatible with dgenerate.memoize.memoize()

dgenerate.pipelinewrapper.uri_list_hash_with_parser(parser)

Create a hash function from a particular URI parser function that hashes a list of URIs.

Parameters:

parser – The URI parser function

Returns:

a hash function compatible with dgenerate.memoize.memoize()

dgenerate.pipelinewrapper.vae_cache_size() → int

Return the estimated memory usage in bytes of all user specified VAEs currently cached in memory.

Returns:

memory usage in bytes.

dgenerate.pipelinewrapper.vae_create_update_cache_info(vae, estimated_size: int)

Add additional information about the size of a newly created VAE model to the cache.

Tag the object with an internal tag.

Parameters:

• vae – the VAE object

• estimated_size – size bytes

dgenerate.plugin module

dgenerate.plugin.load_modules(paths: List[str] | None) → List[module]

Load python modules from a folder or directly from a .py file. Cache them so that repeat requests for loading return an already loaded module.

Parameters:

paths – list of folder/file paths

Returns:

list of types.ModuleType

dgenerate.plugin.LOADED_PLUGIN_MODULES: Dict[str, module] = {}

Plugin module in memory cache

dgenerate.preprocessors module

exception dgenerate.preprocessors.ImagePreprocessorArgumentError

Bases: Exception

Raised when an image preprocessor receives invalid arguments.

exception dgenerate.preprocessors.ImagePreprocessorNotFoundError

Bases: Exception

Raised when a reference to an unknown image preprocessor name is made.

exception dgenerate.preprocessors.PreprocessorHelpUsageError

Bases: Exception

Raised on argument parse errors in image_preprocessor_help()

class dgenerate.preprocessors.CannyEdgeDetectPreprocess(lower=50, upper=100, aperture_size=3, L2_gradient=False, blur=False, gray=False, threshold_algo=None, sigma=0.33, pre_resize=False, **kwargs)

Bases: ImagePreprocessor

Process the input image with the Canny edge detection algorithm for use with a ControlNet. The “lower” argument indicates the lower threshold value for the algorithm, and the “upper” argument indicates the upper threshold. “aperture-size” is the size of Sobel kernel used for find image gradients, it must be an odd integer from 3 to 7. “L2-gradient” specifies the equation for finding gradient magnitude, if True a more accurate equation is used. See: https://docs.opencv.org/3.4/da/d22/tutorial_py_canny.html.

If “blur” is true, apply a 3x3 gaussian blur before processing. If “gray” is true, convert the image to the cv2 “GRAY” format before processing, which does not happen automatically unless you are using a “threshold_algo” value, OpenCV is capable of edge detection on colored images, however you may find better results by converting to its internal grayscale format before processing, or you may not, it depends.

If “threshold_algo” is one of (“otsu”, “triangle”, “median”) try to calculate the lower and upper threshold automatically using cv2.threshold or cv2.median in the case of “median”. “sigma” scales the range of the automatic threshold calculation done when a value for “threshold_algo” is selected. “pre-resize” is a boolean value determining if the processing should take place before or after the image is resized by dgenerate.

__init__(lower=50, upper=100, aperture_size=3, L2_gradient=False, blur=False, gray=False, threshold_algo=None, sigma=0.33, pre_resize=False, **kwargs)

Parameters:

• lower – lower threshold for canny edge detection

• upper – upper threshold for canny edge detection

• aperture_size – aperture size, an odd integer from 3 to 7

• L2_gradient – Use L2_gradient? https://docs.opencv.org/3.4/da/d22/tutorial_py_canny.html

• blur – apply a 3x3 gaussian blur before processing?

• gray – convert to cv2.GRAY format before processing?

• threshold_algo – optional auto thresholding algorithm. One of “otsu”, “triangle”, or “median”. the lower, and upper threshold values are determined automagically from the image content if this argument is supplied a value.

• sigma – scales the range of the automatic threshold calculation

• pre_resize – process the image before it is resized, or after? default is after (False)

• kwargs – forwarded to base class

post_resize(image: Image)

Post resize, canny edge detection may or may not occur here depending on the boolean value of the preprocessor argument “pre-resize”

Parameters:

image – image to process

Returns:

possibly a canny edge detected image, or the input image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Pre resize, canny edge detection may or may not occur here depending on the boolean value of the preprocessor argument “pre-resize”

Parameters:

• image – image to process

• resize_resolution – purely informational, is unused by this preprocessor

Returns:

possibly a canny edge detected image, or the input image

NAMES = ['canny']

class dgenerate.preprocessors.ImagePreprocessor(called_by_name: str, device: str = 'cpu', output_file: str | None = None, output_overwrite: bool = False, **kwargs)

Bases: object

Abstract base class for image preprocessor implementations.

__init__(called_by_name: str, device: str = 'cpu', output_file: str | None = None, output_overwrite: bool = False, **kwargs)

argument_error(msg: str)

static call_post_resize(preprocessor, image: Image) → Image

Invoke a preprocessors ImagePreprocessor.post_resize() method.

Implements important behaviors depending on if the image was modified.

This is the only appropriate way to invoke a preprocessor manually.

Parameters:

• preprocessor – ImagePreprocessor implementation instance

• image – the image to pass

Returns:

processed image, may be the same image or a copy.

static call_pre_resize(preprocessor, image: Image, resize_resolution: Tuple[int, int] | None) → Image

Invoke a preprocessors ImagePreprocessor.pre_resize() method.

Implements important behaviors depending on if the image was modified.

This is the only appropriate way to invoke a preprocessor manually.

Parameters:

• preprocessor – ImagePreprocessor implementation instance

• image – the image to pass

• resize_resolution – the size that the image is going to be resized to after this step, or None if it is not being resized.

Returns:

processed image, may be the same image or a copy.

classmethod get_accepted_args(called_by_name: str) → List[str]

Get a list of accepted argument names for a preprocessor class.

Parameters:

called_by_name – The name used to invoke the preprocessor. Arguments may vary depending on what name was used to invoke the preprocessor.

Returns:

list of argument names

classmethod get_accepted_args_with_defaults(called_by_name) → List[Tuple[str, Any]]

Retrieve the argument signature of an image preprocessor implementation. As a list of tuples which are: (name,) or (name, default_value) depending on if a default value for the argument is present in the signature.

Parameters:

called_by_name – The name used to invoke the preprocessor. Argument signature may vary by name used to invoke.

Returns:

mixed list of tuples, of length 1 or 2, depending on if the argument has a default value. IE: (name,) or (name, default_value). Arguments with defaults are not guaranteed to be positioned at the end of this sequence.

static get_bool_arg(name: str, value: str | bool | Dict) → bool

Convert an argument value from a string to a boolean value. Throw ImagePreprocessorArgumentError if there is an error parsing the value.

Raises:

ImagePreprocessorArgumentError –

Parameters:

• name – the argument name for descriptive purposes, and/or for specifying the dictionary key when value is a dictionary.

• value – a boolean value as a string, or optionally a dictionary to get the value from using the argument name.

Returns:

bool

classmethod get_default_args(called_by_name: str) → List[Tuple[str, Any]]

Get the names and values of arguments for this preprocessor that possess default values.

Parameters:

called_by_name – The name used to invoke the preprocessor. Default arguments may vary by name used to invoke.

Returns:

list of arguments with default value: (name, value)

static get_float_arg(name: str, value: str | float | Dict) → float

Convert an argument value from a string to a float. Throw ImagePreprocessorArgumentError if there is an error parsing the value.

Raises:

ImagePreprocessorArgumentError –

Parameters:

• name – the argument name for descriptive purposes, and/or for specifying the dictionary key when value is a dictionary.

• value – a float value as a string, or optionally a dictionary to get the value from using the argument name.

Returns:

float

classmethod get_help(called_by_name: str) → str

Get formatted help information about the image preprocessor.

This includes any implemented help strings and an auto formatted description of the preprocessors accepted arguments.

Parameters:

called_by_name – The name used to invoke the preprocessor. Help may vary depending on how many names the preprocessor implementation handles and what invoking it by a certain name does.

Returns:

Formatted help string

static get_int_arg(name: str, value: str | int | Dict) → int

Convert an argument value from a string to an integer. Throw ImagePreprocessorArgumentError if there is an error parsing the value.

Raises:

ImagePreprocessorArgumentError –

Parameters:

• name – the argument name for descriptive purposes, and/or for specifying the dictionary key when value is a dictionary.

• value – an integer value as a string, or optionally a dictionary to get the value from using the argument name.

Returns:

int

classmethod get_names() → List[str]

Get the names that this class can be loaded / invoked by.

Returns:

classmethod get_required_args(called_by_name: str) → List[str]

Get a list of required arguments for this preprocessor class.

Parameters:

called_by_name – The name used to invoke the preprocessor. Required arguments may vary by name used to invoke.

Returns:

list of argument names

post_resize(image: Image)

Implementation of post_resize that does nothing. Inheritor must implement.

This method should not be invoked directly, use the class method ImagePreprocessor.call_post_resize() to invoke it.

Parameters:

image – image to process

Returns:

the processed image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Implementation of pre_resize that does nothing. Inheritor must implement.

This method should not be invoked directly, use the class method ImagePreprocessor.call_pre_resize() to invoke it.

Parameters:

• image – image to process

• resize_resolution – image will be resized to this resolution after this process is complete. If None is passed no resize is going to occur. It is not the duty of the inheritor to resize the image, in fact it should NEVER be resized.

Returns:

the processed image

property called_by_name: str

The name the preprocessor was invoked by.

Returns:

name

property device: str

The rendering device requested for this preprocessor.

Returns:

device string, for example “cuda”, “cuda:N”, or “cpu”

class dgenerate.preprocessors.ImagePreprocessorChain(preprocessors: Iterable[ImagePreprocessor] | None = None)

Bases: ImagePreprocessor

Implements chainable image preprocessors.

Chains preprocessing steps together in a sequence.

__init__(preprocessors: Iterable[ImagePreprocessor] | None = None)

Parameters:

preprocessors – optional initial preprocessors to fill the chain, accepts an iterable

add_processor(preprocessor: ImagePreprocessor)

Add a preprocessor implementation to the chain.

Parameters:

preprocessor – dgenerate.preprocessors.preprocessor.ImagePreprocessor

post_resize(image: Image)

Invoke post_resize on all preprocessors in this preprocessor chain in turn.

Every subsequent invocation receives the last preprocessed image as its argument.

This method should not be invoked directly, use the class method dgenerate.preprocessors.preprocessor.ImagePreprocessor.call_post_resize() to invoke it.

Parameters:

image – initial image to preprocess

Returns:

the processed image, possibly affected by every preprocessor in the chain

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Invoke pre_resize on all preprocessors in this preprocessor chain in turn.

Every subsequent invocation receives the last preprocessed image as its argument.

This method should not be invoked directly, use the class method dgenerate.preprocessors.preprocessor.ImagePreprocessor.call_pre_resize() to invoke it.

Parameters:

• image – initial image to preprocess

• resize_resolution – the size which the image will be resized to after this step, this is only information for the preprocessors and the image will not be resized by this method. Image preprocessors should never resize images as it is the responsibility of dgenerate to do that for the user.

Returns:

the processed image, possibly affected by every preprocessor in the chain

HIDDEN = True

class dgenerate.preprocessors.ImagePreprocessorMixin(preprocessor: ImagePreprocessor, *args, **kwargs)

Bases: object

Mixin functionality for objects that do image preprocessing such as implementors of dgenerate.mediainput.AnimationReader

__init__(preprocessor: ImagePreprocessor, *args, **kwargs)

Parameters:

• preprocessor – the preprocessor implementation that will be doing the image preprocessing.

• args – mixin forwarded args

• kwargs – mixin forwarded kwargs

preprocess_image(image: Image, resize_to: Tuple[int, int] | None, aspect_correct: bool = True)

Preform image preprocessing on an image, including the requested resizing step.

Invokes the assigned image preprocessor pre and post resizing with appropriate arguments and correct resource management.

Parameters:

• image – image to process

• resize_to – image will be resized to this dimension by this method.

• aspect_correct – Should the resize operation be aspect correct?

Returns:

the processed image, processed by the preprocessor assigned in the constructor.

preprocessor_enabled: bool

Enable or disable image preprocessing.

class dgenerate.preprocessors.Loader

Bases: object

__init__()

add_class(cls: Type[ImagePreprocessor])

Add an image preprocessor implementation by its class, this is unused by dgenerate and is provided for the convenience of library users.

Parameters:

cls – class the implements dgenerate.preprocessors.preprocessor.ImagePreprocessor, (not an instance, use the type itself)

get_all_names() → List[str]

Get all dgenerate.preprocessors.ImagePreprocessor.NAMES values visible to this loader.

Returns:

list of names (strings)

get_available_classes() → List[Type[ImagePreprocessor]]

Return a list of all dgenerate.preprocessors.ImagePreprocessor implementations visible to this loader.

Returns:

list of dgenerate.preprocessors.ImagePreprocessor

get_class_by_name(preprocessor_name) → Type[ImagePreprocessor]

Get a dgenerate.preprocessors.ImagePreprocessor implementation from the loader using one of the implementations defined dgenerate.preprocessors.ImagePreprocessor.NAMES

Raises:

• RuntimeError – if more than one class was found using the provided name.

• ImagePreprocessorNotFoundError – if the name could not be found.

Parameters:

preprocessor_name – the name to search for

Returns:

dgenerate.preprocessors.ImagePreprocessor

get_help(preprocessor_name: str) → str

Get the formatted help string for a specific preprocessor by name.

Raises:

• RuntimeError – if more than one class was found using the provided name.

• ImagePreprocessorNotFoundError – if the name could not be found.

Parameters:

preprocessor_name – the preprocessor name to search for

Returns:

formatted help string

load(uri: str | Iterable[str], device: str = 'cpu') → ImagePreprocessor | ImagePreprocessorChain | None

Load an image preprocessor or multiple image preprocessors. They are loaded by URI, which is their name and any module arguments, for example: canny;lower=50;upper=100

Specifying multiple preprocessors with a list will create an image preprocessor chain object.

Raises:

• RuntimeError – if more than one class was found using the provided name mentioned in the URI.

• ImagePreprocessorNotFoundError – if the name mentioned in the URI could not be found.

• ImagePreprocessorArgumentError – if the URI contained invalid arguments.

Parameters:

• uri – Preprocessor URI or list of URIs

• device – Request a specific rendering device, default is CPU

Returns:

dgenerate.preprocessors.ImagePreprocessor or dgenerate.preprocessors.ImagePreprocessorChain

load_plugin_modules(paths: List[str])

Add a list of python module directories or python files to Loader.search_modules

They will be newly imported or fetched if already previously imported.

Parameters:

paths – python module directories, python file paths

extra_classes: Set

Additional directly defined implementation classes. This is empty by default and is for allowing library users to quickly add a class implementing dgenerate.preprocessors.preprocessor.ImagePreprocessor if desired without creating a new file for it.

search_modules: Set

Additional module objects for this loader to search, aside from the preprocessors sub module.

class dgenerate.preprocessors.MirrorFlipPreprocess(**kwargs)

Bases: ImagePreprocessor

Implements the “mirror” and “flip” PIL.ImageOps operations as an image preprocessor

__init__(**kwargs)

Parameters:

kwargs – forwarded to base class

static help(called_by_name)

post_resize(image: Image)

Post resize, for this preprocessor nothing happens post-resize.

Parameters:

image – image to process

Returns:

the same image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Mirrors or flips the image depending on what name was used to invoke this preprocessor implementation.

Parameters:

• image – image to process

• resize_resolution – purely informational, is unused by this preprocessor

Returns:

the mirrored or flipped image.

NAMES = ['mirror', 'flip']

class dgenerate.preprocessors.OpenPosePreprocess(include_body=True, include_hand=False, include_face=False, pre_resize=False, **kwargs)

Bases: ImagePreprocessor

Generate an OpenPose rigging from the input image (of a human/humanoid) for use with a ControlNet. “include-body” is a boolean value indicating if a body rigging should be generated. “include-hand” is a boolean value indicating if a detailed hand/finger rigging should be generated. “include-face” is a boolean value indicating if a detailed face rigging should be generated. “pre-resize” is a boolean value determining if the processing should take place before or after the image is resized by dgenerate.

__init__(include_body=True, include_hand=False, include_face=False, pre_resize=False, **kwargs)

Parameters:

• include_body – generate a body rig?

• include_hand – include detailed hand rigging?

• include_face – include detailed face rigging?

• pre_resize – process the image before it is resized, or after? default is after (False)

• kwargs – forwarded to base class

post_resize(image: Image)

Post resize, OpenPose rig generation may or may not occur here depending on the boolean value of the preprocessor argument “pre-resize”

Parameters:

image – image to process

Returns:

possibly an OpenPose rig image, or the input image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Pre resize, OpenPose rig generation may or may not occur here depending on the boolean value of the preprocessor argument “pre-resize”

Parameters:

• image – image to process

• resize_resolution – purely informational, is unused by this preprocessor

Returns:

possibly an OpenPose rig image, or the input image

NAMES = ['openpose']

class dgenerate.preprocessors.PosterizePreprocess(bits, **kwargs)

Bases: ImagePreprocessor

Posterize the input image with PIL.ImageOps.posterize.

Accepts the argument ‘bits’, an integer value from 1 to 8.

__init__(bits, **kwargs)

Parameters:

• bits – required argument, integer value from 1 to 8

• kwargs – forwarded to base class

post_resize(image: Image)

Post resize, for this preprocessor nothing happens post-resize.

Parameters:

image – image to process

Returns:

the same image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Posterize operation is preformed by this method.

Parameters:

• image – image to process

• resize_resolution – purely informational, is unused by this preprocessor

Returns:

the posterized image

NAMES = ['posterize']

class dgenerate.preprocessors.SimpleColorPreprocess(**kwargs)

Bases: ImagePreprocessor

Implements the “grayscale” and “invert” PIL.ImageOps operations as an image preprocessor.

__init__(**kwargs)

Parameters:

kwargs – forwarded to base class

static help(called_by_name)

post_resize(image: Image)

Invert or grayscale the image depending on which name was used to invoke this preprocessor.

Parameters:

image – image to process

Returns:

the inverted or grayscale image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Pre resize, for this preprocessor nothing happens pre-resize.

Parameters:

• image – image to process

• resize_resolution – purely informational, is unused by this preprocessor

Returns:

the same image

NAMES = ['grayscale', 'invert']

class dgenerate.preprocessors.SolarizePreprocess(threshold=128, **kwargs)

Bases: ImagePreprocessor

Solarize the input image with PIL.ImageOps.solarize.

Accepts the argument “threshold” which is an integer value from 0 to 255.

__init__(threshold=128, **kwargs)

Parameters:

• threshold – integer value from 0 to 255, default is 128

• kwargs – forwarded to base class

post_resize(image: Image)

Post resize, for this preprocessor nothing happens post-resize.

Parameters:

image – image to process

Returns:

the same image

pre_resize(image: Image, resize_resolution: Tuple[int, int] | None)

Solarize operation is preformed by this method.

Parameters:

• image – image to process

• resize_resolution – purely informational, is unused by this preprocessor

Returns:

the solarized image

NAMES = ['solarize']

dgenerate.preprocessors.image_preprocessor_help(args: Sequence[str], throw: bool = False)

Implements --image-preprocessor-help command line option

Parameters:

• args – arguments (preprocessor names, or empty list)

• throw – Should we throw exceptions or handle them?

Raises:

• PreprocessorHelpUsageError –

• ImagePreprocessorNotFoundError –

Returns:

return-code, anything other than 0 is failure

dgenerate.prompt module

class dgenerate.prompt.Prompt(positive: str | None = None, negative: str | None = None, delimiter: str = ';')

Bases: object

Represents a combined positive and optional negative prompt split by a delimiter character.

__init__(positive: str | None = None, negative: str | None = None, delimiter: str = ';')

Parameters:

• positive – positive prompt component

• negative – negative prompt component

• delimiter – delimiter for stringification

__str__()

Return str(self).

static parse(value: str, delimiter=';') → Prompt

Parse the positive and negative prompt from a string and return a prompt object.

Parameters:

• value – the string

• delimiter – The prompt delimiter character

Raises:

ValueError – if value is None

Returns:

Prompt (returns self)

dgenerate.textprocessing module

exception dgenerate.textprocessing.ConceptPathParseError

Bases: Exception

class dgenerate.textprocessing.ConceptUri(concept: str, args: Dict[str, str])

Bases: object

Represents a parsed concept path.

__init__(concept: str, args: Dict[str, str])

args: Dict[str, str]

Provided keyword arguments with their (string) values.

concept: str

The primary concept mentioned in the URI.

class dgenerate.textprocessing.ConceptUriParser(concept_name: str, known_args: Sequence[str] = None)

Bases: object

Parser for dgenerate concept paths with arguments, IE: concept;arg1=”a”;arg2=”b”

Used for --vae, --loras etc. as well as image preprocessor plugin module arguments.

__init__(concept_name: str, known_args: Sequence[str] = None)

Raises:

ValueError – if duplicate argument names are specified.

Parameters:

• concept_name – Concept name, used in error messages

• known_args – valid arguments for the parser, must be unique

parse_concept_uri(uri: str)

Parse a string.

Parameters:

uri – the string

Raises:

ConceptPathParseError –

Returns:

ConceptPath

concept_name: str

Name / title string for this concept. Used in parse error exceptions.

known_args: Set[str]

Unique recognized keyword arguments

dgenerate.textprocessing.contains_space(string: str) → bool

Check if a string contains any whitespace characters including newlines

Parameters:

string – the string

Returns:

bool

dgenerate.textprocessing.dashdown(string: str) → str

Replace ‘-’ with ‘_’

Parameters:

string – the string

Returns:

modified string

dgenerate.textprocessing.dashup(string: str) → str

Replace ‘_’ with ‘-’

Parameters:

string – the string

Returns:

modified string

dgenerate.textprocessing.debug_format_args(args_dict: Dict[str, Any], value_transformer: Callable[[str, Any], str] | None = None, max_value_len: int = 256)

Format function arguments in a way that can be printed for debug messages.

Parameters:

• args_dict – argument dictionary

• value_transformer – transform values in the argument dictionary

• max_value_len – Max length of a formatted value before it is turned into a class and id string only

Returns:

formatted string

dgenerate.textprocessing.format_size(size: Iterable[int])

Join together an iterable of integers with the character x

Parameters:

size – the iterable

Returns:

formatted string

dgenerate.textprocessing.is_quoted(string: str) → bool

Return True if a string is quoted with an identical starting and end quote.

Parameters:

string – the string

Returns:

True or False

dgenerate.textprocessing.justify_left(string: str)

Justify text to the left.

Parameters:

string – string with text

Returns:

left justified text

dgenerate.textprocessing.long_text_wrap_width() → int

Return the current terminal width or the default value of 150 characters for text-wrapping purposes.

Returns:

int

dgenerate.textprocessing.oxford_comma(elements: Sequence[str], conjunction: str) → str

Join a sequence of strings with commas, end with an oxford comma and conjunction if needed.

Parameters:

• elements – strings

• conjunction – “and”, “or”

Returns:

a joined string

dgenerate.textprocessing.quote(string: str) → str

Wrap a string in double quotes.

This is not equivalent to shell quoting.

Parameters:

string – the string

Returns:

The quoted string

dgenerate.textprocessing.quote_spaces(value_or_struct: Any | Sequence[Any | list | tuple]) → list | tuple | Any

Quote any str(value) containing spaces, or str(value)s containing spaces within a list, or list of lists/tuples.

The entire content of the data structure is stringified by this process.

This is not equivalent to shell quoting.

Parameters:

value_or_struct – value or (list of values, and or lists/tuples containing values)

Returns:

input data structure with strings quoted if needed

dgenerate.textprocessing.underline(string: str, underline_char: str = '=') → str

Underline a string with the selected character.

Parameters:

• string – the string

• underline_char – the character to underline with

Returns:

the underlined string

dgenerate.textprocessing.unquote(string: str) → str

Remove quotes from a string, including single quotes.

Parameters:

string – the string

Returns:

The un-quoted string

dgenerate.textprocessing.wrap(text: str, width: int, initial_indent='', subsequent_indent='', break_long_words=False, break_on_hyphens=False, **fill_args)

Wrap text.

Parameters:

• text – The prompt text

• width – The wrap width

• initial_indent – initial indent string

• subsequent_indent – subsequent indent string

• break_long_words – Break on long words?

• break_on_hyphens – Break on hyphens?

• fill_args – extra keyword arguments to textwrap.fill() if desired

Returns:

text wrapped string

dgenerate.textprocessing.wrap_paragraphs(text: str, width: int, break_long_words=False, break_on_hyphens=False, **fill_args)

Wrap text that may contain paragraphs without removing separating whitespace.

Parameters:

• text – Text containing paragraphs

• width – Wrap with in characters

• break_long_words – break on long words? default False

• break_on_hyphens – break on hyphens? default False

• fill_args – extra keyword arguments to textwrap.fill() if desired

Returns:

text wrapped string

dgenerate.types module

class dgenerate.types.SetFromMixin

Bases: object

Allows an object ot have its attributes set from a dictionary or attributes taken from another object with an overlapping set of attribute names.

set_from(obj: Any | dict, missing_value_throws: bool = True)

Set the attributes in this configuration object from a dictionary or another object possessing keys / attributes of the same name.

Parameters:

• obj – The object, or dictionary

• missing_value_throws – whether to throw ValueError if obj is missing an attribute that exist in this object

Returns:

self

dgenerate.types.class_and_id_string(obj) → str

Return a string formatted with an objects class name next to its memory ID.

IE: <ClassName: id_integer>

Parameters:

obj – the object

Returns:

formatted string

dgenerate.types.default(value, default_value)

Return value if value is not None, otherwise default :param value: :param default_value: :return: bool

dgenerate.types.fullname(obj)

Get the fully qualified name of an object or function

Parameters:

obj – The object

Returns:

Fully qualified name

dgenerate.types.get_accepted_args_with_defaults(func) → Iterator[Tuple[str] | Tuple[str, Any]]

Get the argument signature of a simple function with any default values present.

Parameters:

func – the function

Returns:

an iterator over tuples of length 1 or 2, length 2 indicates a default argument value is present. (name,) or (name, value)

dgenerate.types.get_default_args(func) → Iterator[Tuple[str, Any]]

Get a list of default arguments from a simple function with their default values.

Parameters:

func – the function

Returns:

iterator over tuples of length 2 (name, value)

dgenerate.types.get_public_attributes(obj) → Dict[str, Any]

Get the public attributes (excluding functions) and their values from an object.

Parameters:

obj – the object

Returns:

dict of attribute names to values

dgenerate.types.get_public_members(obj) → Dict[str, Any]

Get the public members (including functions) and their values from an object.

Parameters:

obj – the object

Returns:

dict of attribute names to values

dgenerate.types.get_type(hinted_type)

Get the basic type of hinted type

Parameters:

hinted_type – the type hint

Returns:

bool

dgenerate.types.get_type_of_optional(hinted_type)

Get the first possible type for an optional type hint

Parameters:

hinted_type – The hinted type to extract from

Returns:

the type, or None

dgenerate.types.is_optional(hinted_type)

Check if a hinted type is optional

Parameters:

hinted_type – The hinted type

Returns:

bool

dgenerate.types.is_type(hinted_type, comparison_type)

Check if a hinted type is equal to a comparison type.

Parameters:

• hinted_type – The hinted type

• comparison_type – The type to check for

Returns:

bool

dgenerate.types.is_type_or_optional(hinted_type, comparison_type)

Check if a hinted type is equal to a comparison type, even if the type hint is optional (compare the inside if necessary).

Parameters:

• hinted_type – The hinted type

• comparison_type – The type to check for

Returns:

bool

dgenerate.types.module_all()

Return the name of all public non-module type global objects inside the current module.

Can be used for __all__

Returns:

list of names

dgenerate.types.parse_bool(string_or_bool: str | bool)

Parse a case insensitive boolean value from a string, for example “true” or “false”

Additionally, values that are already bool are passed through.

Raises:

ValueError – on parse failure.

Parameters:

string_or_bool – the string, or a bool value

Returns:

python boolean type equivalent