vision_unlearning.unlearner.lora

Classes

UnlearnerLora	Fine-tuning script for Stable Diffusion for text2image with support for LoRA.
UnlearnerLoraDirect	Straight-forward finetuning

Functions

unlearn_lora(...)

id can be both a local dir or a huggingface model id

Module Contents

vision_unlearning.unlearner.lora.unlearn_lora(model_original_id: str, model_lora_id: str, device: str, weight_name: str = 'pytorch_lora_weights.safetensors', requires_inversion: bool = True, return_original: bool = True, return_learned: bool = True) → Tuple[diffusers.StableDiffusionPipeline | None, diffusers.StableDiffusionPipeline | None, diffusers.StableDiffusionPipeline]

id can be both a local dir or a huggingface model id return pipeline_original, pipeline_learned, pipeline_unlearned

Inspired by @inproceedings{zhang2023composing,

title={Composing Parameter-Efficient Modules with Arithmetic Operations}, author={Zhang, Jinghan and Chen, Shiqi and Liu, Junteng and He, Junxian}, booktitle={Advances in Neural Information Processing Systems}, year={2023}

} Source: https://github.com/hkust-nlp/PEM_composition/tree/main/exps/composition_for_unlearning

class vision_unlearning.unlearner.lora.UnlearnerLora(/, **data: Any)

Bases: vision_unlearning.unlearner.base.Unlearner

Fine-tuning script for Stable Diffusion for text2image with support for LoRA. Strongly based on the huggingface example (see credits in the end)

Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py

lora_r: int = None

lora_alpha: int = None

target_modules: List[str] = None

is_lora_negated: bool = None

seed: int = None

model_name_or_path: str = None

revision: str | None = None

variant: str | None = None

dataset_forget_name: str = None

dataset_retain_name: str = None

dataset_forget_config_name: str | None = None

dataset_retain_config_name: str | None = None

image_column: str = None

caption_column: str = None

validation_prompt: str | None = None

num_validation_images: int = None

validation_epochs: int = None

resolution: int = None

center_crop: bool = None

random_flip: bool = None

max_train_samples: int | None = None

dataloader_num_workers: int = None

final_eval_prompts_forget: str | List[str] = None

final_eval_prompts_retain: str | List[str] = None

prediction_type: str | None = None

per_device_train_batch_size: int = None

gradient_accumulation_steps: int = None

num_train_epochs: int = None

learning_rate: float = None

lr_scheduler_type: str = None

should_log: bool = None

local_rank: int = None

device: str = None

n_gpu: int = None

output_dir: str = None

cache_dir: str | None = None

hub_token: str | None = None

hub_model_id: str | None = None

logging_dir: str = None

logging_steps: int = None

save_strategy: str = None

save_total_limit: int = None

gradient_checkpointing: bool = None

enable_xformers_memory_efficient_attention: bool = None

mixed_precision: str | None = None

allow_tf32: bool = None

use_8bit_adam: bool = None

report_to: str = None

gradient_weighting_method: vision_unlearning.utils.gradient_weighting.GradientWeightingMethod = None

compute_gradient_conflict: bool = None

compute_runtimes: bool = None

compute_memory: bool = None

max_train_steps: int | None = None

lr_warmup_steps: int = None

adam_beta1: float = None

adam_beta2: float = None

adam_weight_decay: float = None

adam_epsilon: float = None

max_grad_norm: float = None

checkpointing_steps: int = None

checkpoints_total_limit: int | None = None

resume_from_checkpoint: str | None = None

noise_offset: float = None

_accelerator: accelerate.Accelerator | None = None

_output_dir_checkpoints: str | None = None

_output_dir_lora: str | None = None

_lora_weight_name: str = 'pytorch_lora_weights.safetensors'

_images: Dict[str, PIL.Image.Image]

_weight_dtype: Any = Ellipsis

_similarities_gr: List[float] = []

_similarities_gf: List[float] = []

_noise_scheduler: Any = None

_tokenizer: Any = None

_text_encoder: Any = None

_vae: Any = None

_unet: diffusers.models.unets.unet_2d_condition.UNet2DConditionModel | None = None

_train_forget_dataloader: torch.utils.data.DataLoader | None = None

_train_retain_dataloader: torch.utils.data.DataLoader | None = None

_optimizer: Any = None

_lr_scheduler: Any = None

_lora_layers: Any = None

_peak_mem: int = 0

model_post_init(__context: dict | None = None) → None

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

_pre_checks() → None

_get_lora_config() → peft.LoraConfig

_get_accelerator()

_hook_after_lora_init()

_hook_before_load_model()

_save_lora_layers()

Side-effects: modifies self._unet in-place (casts to float32), saves two directories self._output_dir_super and self._output_dir_sub

train()

abstract _prepare_dataloaders() → Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]

abstract _train_one_batch(batch_forget, batch_retain)

class vision_unlearning.unlearner.lora.UnlearnerLoraDirect(/, **data: Any)

Bases: UnlearnerLora

Straight-forward finetuning

_prepare_dataloaders() → Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader]

_train_one_batch(batch_forget, batch_retain)