vision_unlearning.unlearner

Submodules

• vision_unlearning.unlearner.base
• vision_unlearning.unlearner.fade
• vision_unlearning.unlearner.lora
• vision_unlearning.unlearner.uce_sd_erase

Attributes

logger
logger
logger

Classes

Unlearner	performs the actual finetuning
Unlearner	performs the actual finetuning
MetricImageTextSimilarity	!!! abstract "Usage Documentation"
EvaluatorTextToImage	!!! abstract "Usage Documentation"
GradientWeightingMethod	Method used to conciliate/harmonize/combine/weight the gradients of the different tasks
GradientWeightingMethodSimple	Fixed weights for each component
UnlearnerLora	Fine-tuning script for Stable Diffusion for text2image with support for LoRA.
UnlearnerLoraDirect	Straight-forward finetuning
Unlearner	performs the actual finetuning
EvaluatorTextToImage	!!! abstract "Usage Documentation"
MetricImageTextSimilarity	!!! abstract "Usage Documentation"
ConceptType	Enum representing the type of concept to unlearn.
UCE	Unified Concept Editing for unlearning in Stable Diffusion models.

Functions

get_logger(→ logging.Logger)
plot_gradient_conflict_hist(→ PIL.Image.Image)
log_validation(→ Dict[str, PIL.Image.Image])	Adapted from The HuggingFace Inc. team. All rights reserved.
save_model_card(repo_id, base_model, ...[, images, ...])	The resulting file looks like this: https://github.com/huggingface/hub-docs/blob/main/modelcard.md
unwrap_model(model, accelerator)	Adapted from The HuggingFace Inc. team. All rights reserved.
preprocess_train(examples, tokenizer, caption_column, ...)	Adapted from The HuggingFace Inc. team. All rights reserved.
collate_fn(examples)	Adapted from The HuggingFace Inc. team. All rights reserved.
unlearn_lora(...)	id can be both a local dir or a huggingface model id
save_model_card(repo_id, base_model, ...[, images, ...])	The resulting file looks like this: https://github.com/huggingface/hub-docs/blob/main/modelcard.md

Package Contents

vision_unlearning.unlearner.get_logger(name: str, level=logging.INFO) → logging.Logger

vision_unlearning.unlearner.logger

class vision_unlearning.unlearner.Unlearner(/, **data: Any)

Bases: pydantic.BaseModel, abc.ABC

performs the actual finetuning

One unlearner may have variations/parametrizations that correspond to different unlearning algorithms/methods

abstract train() → List[huggingface_hub.repocard_data.EvalResult]

class vision_unlearning.unlearner.Unlearner(/, **data: Any)

Bases: pydantic.BaseModel, abc.ABC

performs the actual finetuning

One unlearner may have variations/parametrizations that correspond to different unlearning algorithms/methods

abstract train() → List[huggingface_hub.repocard_data.EvalResult]

vision_unlearning.unlearner.logger

class vision_unlearning.unlearner.MetricImageTextSimilarity(/, **data: Any)

Bases: vision_unlearning.metrics.base.Metric

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

metrics: List[Literal['clip']]

_clip_metric: torchmetrics.multimodal.clip_score.CLIPScore | None = None

model_post_init(__context: dict | None = None) → None

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

_load_image(image: PIL.Image.Image | numpy.ndarray | str) → torch.Tensor

score(image: PIL.Image.Image | numpy.ndarray | str, text: str) → Dict[str, float]

score_batch(images: List[PIL.Image.Image | numpy.ndarray | str], texts: List[str]) → List[Dict[str, float]]

Warning: this function don’t improve performance. The underlying libraries still work serially. Returns per-pair results in the same order.

score_batch_same_text(images: List[PIL.Image.Image | numpy.ndarray | str], text: str) → List[Dict[str, float]]

Batch CLIP scoring when all images share the same text prompt.

This is meaningfully faster than calling score() N times because the CLIP text encoder runs once for the shared text. Images are processed individually through the CLIP image processor (as in the serial path) but the text encoder forward pass is done only once.

Uses _clip_score_update from torchmetrics (private API, tested against torchmetrics 1.x) which returns per-pair scores as a 1-D tensor. The result is numerically equivalent to calling score() N times (max diff < 2e-5 on 512x512 SD1.4 images).

NOTE: _clip_score_update is a private torchmetrics symbol — if a future torchmetrics version removes it, fall back to the serial score() loop.

Parameters:

• images – List of N images (PIL Image, np.ndarray, or file path).

• text – Single text caption applied to all images.

Returns:

float}, one per image in input order.

Return type:

List of N dicts {‘clip’

class vision_unlearning.unlearner.EvaluatorTextToImage(/, **data: Any)

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

model_config

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pipeline_original: diffusers.StableDiffusionPipeline | None

pipeline_learned: diffusers.StableDiffusionPipeline | None

pipeline_unlearned: diffusers.StableDiffusionPipeline

prompts_forget: List[str]

prompts_retain: List[str]

metric_clip: vision_unlearning.metrics.MetricImageTextSimilarity

compute_runtimes: bool = True

plot_show: bool = True

evaluate() → Tuple[List[huggingface_hub.repocard_data.EvalResult], Dict[str, PIL.Image.Image]]

vision_unlearning.unlearner.plot_gradient_conflict_hist(similarities: List[float], title: str, color: str) → PIL.Image.Image

vision_unlearning.unlearner.log_validation(pipeline, accelerator, epoch, num_validation_images, validation_prompt, seed, is_final_validation=False) → Dict[str, PIL.Image.Image]

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

vision_unlearning.unlearner.save_model_card(repo_id: str, base_model: str, dataset_forget_name: str, dataset_retain_name: str, repo_folder: str, images: Dict[str, PIL.Image.Image] = {}, eval_results: List[huggingface_hub.repocard_data.EvalResult] = [], tags: List[str] = [], hyperparameters: dict = {}, similarities_gr: List[float] = [], similarities_gf: List[float] = [])

The resulting file looks like this: https://github.com/huggingface/hub-docs/blob/main/modelcard.md This looks hugginface-specific, so you may think it should be in integrations/huggingface.py, but it is actually a generic Readme saving

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

vision_unlearning.unlearner.unwrap_model(model, accelerator)

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

vision_unlearning.unlearner.preprocess_train(examples, tokenizer, caption_column, image_column, train_transforms, overwrite_column: str | None = None, concept_overwrite: str | None = None)

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

concept_overwrite: concept to be used for overwriting, described as an textual string (used to modify the prompt).

TODO: this handling of concept_overwrite is weird… I wish this were somewhat more structured/organized/clear. For example, the overwriting string may need a more complex prompt than just “an image of f{concept_overwrite}”, or with a different article

vision_unlearning.unlearner.collate_fn(examples)

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

class vision_unlearning.unlearner.GradientWeightingMethod(/, **data: Any)

Bases: pydantic.BaseModel, abc.ABC

Method used to conciliate/harmonize/combine/weight the gradients of the different tasks

Inspired by @article{navon2022multi,

title={Multi-Task Learning as a Bargaining Game}, author={Navon, Aviv and Shamsian, Aviv and Achituve, Idan and Maron, Haggai and Kawaguchi, Kenji and Chechik, Gal and Fetaya, Ethan}, journal={arXiv preprint arXiv:2202.01017}, year={2022}

} Source: https://github.com/AvivNavon/nash-mtl/blob/main/methods/weight_methods.py

abstract weight_grads(grads_forget: List[torch.Tensor], grads_retain: List[torch.Tensor], accelerator) → torch.Tensor

@return scaled_grad

class vision_unlearning.unlearner.GradientWeightingMethodSimple(/, **data: Any)

Bases: GradientWeightingMethod

Fixed weights for each component

forget_weight: float = 1.0

retain_weight: float = 1.0

weight_grads(grads_forget: List[torch.Tensor], grads_retain: List[torch.Tensor], accelerator) → torch.Tensor

@return scaled_grad

vision_unlearning.unlearner.unlearn_lora(model_original_id: str, model_lora_id: str, device: str, weight_name: str = 'pytorch_lora_weights.safetensors', requires_inversion: bool = True, return_original: bool = True, return_learned: bool = True) → Tuple[diffusers.StableDiffusionPipeline | None, diffusers.StableDiffusionPipeline | None, diffusers.StableDiffusionPipeline]

id can be both a local dir or a huggingface model id return pipeline_original, pipeline_learned, pipeline_unlearned

Inspired by @inproceedings{zhang2023composing,

title={Composing Parameter-Efficient Modules with Arithmetic Operations}, author={Zhang, Jinghan and Chen, Shiqi and Liu, Junteng and He, Junxian}, booktitle={Advances in Neural Information Processing Systems}, year={2023}

} Source: https://github.com/hkust-nlp/PEM_composition/tree/main/exps/composition_for_unlearning

class vision_unlearning.unlearner.UnlearnerLora(/, **data: Any)

Bases: vision_unlearning.unlearner.base.Unlearner

Fine-tuning script for Stable Diffusion for text2image with support for LoRA. Strongly based on the huggingface example (see credits in the end)

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

lora_r: int = None

lora_alpha: int = None

target_modules: List[str] = None

is_lora_negated: bool = None

seed: int = None

model_name_or_path: str = None

revision: str | None = None

variant: str | None = None

dataset_forget_name: str = None

dataset_retain_name: str = None

dataset_forget_config_name: str | None = None

dataset_retain_config_name: str | None = None

image_column: str = None

caption_column: str = None

validation_prompt: str | None = None

num_validation_images: int = None

validation_epochs: int = None

resolution: int = None

center_crop: bool = None

random_flip: bool = None

max_train_samples: int | None = None

dataloader_num_workers: int = None

final_eval_prompts_forget: str | List[str] = None

final_eval_prompts_retain: str | List[str] = None

prediction_type: str | None = None

per_device_train_batch_size: int = None

gradient_accumulation_steps: int = None

num_train_epochs: int = None

learning_rate: float = None

lr_scheduler_type: str = None

should_log: bool = None

local_rank: int = None

device: str = None

n_gpu: int = None

output_dir: str = None

cache_dir: str | None = None

hub_token: str | None = None

hub_model_id: str | None = None

logging_dir: str = None

logging_steps: int = None

save_strategy: str = None

save_total_limit: int = None

gradient_checkpointing: bool = None

enable_xformers_memory_efficient_attention: bool = None

mixed_precision: str | None = None

allow_tf32: bool = None

use_8bit_adam: bool = None

report_to: str = None

gradient_weighting_method: vision_unlearning.utils.gradient_weighting.GradientWeightingMethod = None

compute_gradient_conflict: bool = None

compute_runtimes: bool = None

compute_memory: bool = None

max_train_steps: int | None = None

lr_warmup_steps: int = None

adam_beta1: float = None

adam_beta2: float = None

adam_weight_decay: float = None

adam_epsilon: float = None

max_grad_norm: float = None

checkpointing_steps: int = None

checkpoints_total_limit: int | None = None

resume_from_checkpoint: str | None = None

noise_offset: float = None

_accelerator: accelerate.Accelerator | None = None

_output_dir_checkpoints: str | None = None

_output_dir_lora: str | None = None

_lora_weight_name: str = 'pytorch_lora_weights.safetensors'

_images: Dict[str, PIL.Image.Image]

_weight_dtype: Any = Ellipsis

_similarities_gr: List[float] = []

_similarities_gf: List[float] = []

_noise_scheduler: Any = None

_tokenizer: Any = None

_text_encoder: Any = None

_vae: Any = None

_unet: diffusers.models.unets.unet_2d_condition.UNet2DConditionModel | None = None

_train_forget_dataloader: torch.utils.data.DataLoader | None = None

_train_retain_dataloader: torch.utils.data.DataLoader | None = None

_optimizer: Any = None

_lr_scheduler: Any = None

_lora_layers: Any = None

_peak_mem: int = 0

model_post_init(__context: dict | None = None) → None

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

_pre_checks() → None

_get_lora_config() → peft.LoraConfig

_get_accelerator()

_hook_after_lora_init()

_hook_before_load_model()

_save_lora_layers()

Side-effects: modifies self._unet in-place (casts to float32), saves two directories self._output_dir_super and self._output_dir_sub

train()

abstract _prepare_dataloaders() → Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]

abstract _train_one_batch(batch_forget, batch_retain)

class vision_unlearning.unlearner.UnlearnerLoraDirect(/, **data: Any)

Bases: UnlearnerLora

Straight-forward finetuning

_prepare_dataloaders() → Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]

_train_one_batch(batch_forget, batch_retain)

class vision_unlearning.unlearner.Unlearner(/, **data: Any)

Bases: pydantic.BaseModel, abc.ABC

performs the actual finetuning

One unlearner may have variations/parametrizations that correspond to different unlearning algorithms/methods

abstract train() → List[huggingface_hub.repocard_data.EvalResult]

vision_unlearning.unlearner.logger

class vision_unlearning.unlearner.EvaluatorTextToImage(/, **data: Any)

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

model_config

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pipeline_original: diffusers.StableDiffusionPipeline | None

pipeline_learned: diffusers.StableDiffusionPipeline | None

pipeline_unlearned: diffusers.StableDiffusionPipeline

prompts_forget: List[str]

prompts_retain: List[str]

metric_clip: vision_unlearning.metrics.MetricImageTextSimilarity

compute_runtimes: bool = True

plot_show: bool = True

evaluate() → Tuple[List[huggingface_hub.repocard_data.EvalResult], Dict[str, PIL.Image.Image]]

class vision_unlearning.unlearner.MetricImageTextSimilarity(/, **data: Any)

Bases: vision_unlearning.metrics.base.Metric

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

metrics: List[Literal['clip']]

_clip_metric: torchmetrics.multimodal.clip_score.CLIPScore | None = None

model_post_init(__context: dict | None = None) → None

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

_load_image(image: PIL.Image.Image | numpy.ndarray | str) → torch.Tensor

score(image: PIL.Image.Image | numpy.ndarray | str, text: str) → Dict[str, float]

score_batch(images: List[PIL.Image.Image | numpy.ndarray | str], texts: List[str]) → List[Dict[str, float]]

Warning: this function don’t improve performance. The underlying libraries still work serially. Returns per-pair results in the same order.

score_batch_same_text(images: List[PIL.Image.Image | numpy.ndarray | str], text: str) → List[Dict[str, float]]

Batch CLIP scoring when all images share the same text prompt.

This is meaningfully faster than calling score() N times because the CLIP text encoder runs once for the shared text. Images are processed individually through the CLIP image processor (as in the serial path) but the text encoder forward pass is done only once.

Uses _clip_score_update from torchmetrics (private API, tested against torchmetrics 1.x) which returns per-pair scores as a 1-D tensor. The result is numerically equivalent to calling score() N times (max diff < 2e-5 on 512x512 SD1.4 images).

NOTE: _clip_score_update is a private torchmetrics symbol — if a future torchmetrics version removes it, fall back to the serial score() loop.

Parameters:

• images – List of N images (PIL Image, np.ndarray, or file path).

• text – Single text caption applied to all images.

Returns:

float}, one per image in input order.

Return type:

List of N dicts {‘clip’

vision_unlearning.unlearner.save_model_card(repo_id: str, base_model: str, dataset_forget_name: str, dataset_retain_name: str, repo_folder: str, images: Dict[str, PIL.Image.Image] = {}, eval_results: List[huggingface_hub.repocard_data.EvalResult] = [], tags: List[str] = [], hyperparameters: dict = {}, similarities_gr: List[float] = [], similarities_gf: List[float] = [])

The resulting file looks like this: https://github.com/huggingface/hub-docs/blob/main/modelcard.md This looks hugginface-specific, so you may think it should be in integrations/huggingface.py, but it is actually a generic Readme saving

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

class vision_unlearning.unlearner.ConceptType

Bases: str, enum.Enum

Enum representing the type of concept to unlearn.

Object = 'object'

Art = 'art'

class vision_unlearning.unlearner.UCE(**data: Any)

Bases: vision_unlearning.unlearner.base.Unlearner

Unified Concept Editing for unlearning in Stable Diffusion models. Adapted from:

GitHub: https://github.com/rohitgandikota/unified-concept-editing Arxiv: https://arxiv.org/pdf/2308.14761.pdf Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., & Bau, D. (2024). Unified concept editing in diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 5111-5120).

This unlearner do not use LoRA, and do not perform any fine-tuning (instead, it performs a closed-form weight update).

pretrained_model_name_or_path: str = None

erase_scale: float = None

preserve_scale: float = None

lamb: float = None

save_entire_model: bool = None

edit_concepts: str | None = None

guide_concepts: str | None = None

preserve_concepts: str | None = None

concept_type: ConceptType = None

expand_prompts: bool = True

final_eval_prompts_forget: str | List[str] = None

final_eval_prompts_retain: str | List[str] = None

output_dir: str = None

device: str = 'cuda:0'

compute_runtimes: bool = None

hub_model_id: str | None = None

_collect_text_embeddings(pipe: Any, concepts: list[str], device: str, torch_dtype: torch.dtype) → dict[str, torch.Tensor]

Return dict {concept: last_token_embedding}.

_collect_guide_outputs(concepts: list[str], embeds: dict[str, torch.Tensor], modules: list[torch.nn.Module]) → dict[str, list[torch.Tensor]]

Collect cross-attention outputs for guide/preserve concepts.

_update_weights(original_modules: list[torch.nn.Module], erase_embeds: dict[str, torch.Tensor], guide_outputs: dict[str, list[torch.Tensor]], edit_concepts: list[str], guide_concepts: list[str], preserve_concepts: list[str], erase_scale: float, preserve_scale: float, lamb: float, device: str, torch_dtype: torch.dtype) → list[torch.nn.Module]

Apply the UCE weight update to each module and return new modules.

_save_uce_weights(uce_modules: list[torch.nn.Module], uce_module_names: list[str]) → None

Save updated module weights to a safetensors file.

train() → List[huggingface_hub.repocard_data.EvalResult]

Main UCE training and concept erasure logic.

static get_pipeline_from_modified_weights(pretrained_model_name_or_path: str, device: str | torch.device, output_dir: str) → diffusers.DiffusionPipeline

evaluate() → Tuple[List[huggingface_hub.repocard_data.EvalResult], Dict[str, PIL.Image.Image]]