vision_unlearning.unlearner.lora ================================ .. py:module:: vision_unlearning.unlearner.lora Classes ------- .. autoapisummary:: vision_unlearning.unlearner.lora.UnlearnerLora vision_unlearning.unlearner.lora.UnlearnerLoraDirect Functions --------- .. autoapisummary:: vision_unlearning.unlearner.lora.unlearn_lora Module Contents --------------- .. py:function:: unlearn_lora(model_original_id: str, model_lora_id: str, device: str, weight_name: str = 'pytorch_lora_weights.safetensors', requires_inversion: bool = True, return_original: bool = True, return_learned: bool = True) -> Tuple[Optional[diffusers.StableDiffusionPipeline], Optional[diffusers.StableDiffusionPipeline], diffusers.StableDiffusionPipeline] id can be both a local dir or a huggingface model id return pipeline_original, pipeline_learned, pipeline_unlearned Inspired by @inproceedings{zhang2023composing, title={Composing Parameter-Efficient Modules with Arithmetic Operations}, author={Zhang, Jinghan and Chen, Shiqi and Liu, Junteng and He, Junxian}, booktitle={Advances in Neural Information Processing Systems}, year={2023} } Source: https://github.com/hkust-nlp/PEM_composition/tree/main/exps/composition_for_unlearning .. py:class:: UnlearnerLora(/, **data: Any) Bases: :py:obj:`vision_unlearning.unlearner.base.Unlearner` Fine-tuning script for Stable Diffusion for text2image with support for LoRA. Strongly based on the huggingface example (see credits in the end) Adapted from The HuggingFace Inc. team. All rights reserved. Licensed under the Apache License, Version 2.0. Source: https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py .. py:attribute:: lora_r :type: int :value: None .. py:attribute:: lora_alpha :type: int :value: None .. py:attribute:: target_modules :type: List[str] :value: None .. py:attribute:: is_lora_negated :type: bool :value: None .. py:attribute:: seed :type: int :value: None .. py:attribute:: model_name_or_path :type: str :value: None .. py:attribute:: revision :type: Optional[str] :value: None .. py:attribute:: variant :type: Optional[str] :value: None .. py:attribute:: dataset_forget_name :type: str :value: None .. py:attribute:: dataset_retain_name :type: str :value: None .. py:attribute:: dataset_forget_config_name :type: Optional[str] :value: None .. py:attribute:: dataset_retain_config_name :type: Optional[str] :value: None .. py:attribute:: image_column :type: str :value: None .. py:attribute:: caption_column :type: str :value: None .. py:attribute:: validation_prompt :type: Optional[str] :value: None .. py:attribute:: num_validation_images :type: int :value: None .. py:attribute:: validation_epochs :type: int :value: None .. py:attribute:: resolution :type: int :value: None .. py:attribute:: center_crop :type: bool :value: None .. py:attribute:: random_flip :type: bool :value: None .. py:attribute:: max_train_samples :type: Optional[int] :value: None .. py:attribute:: dataloader_num_workers :type: int :value: None .. py:attribute:: final_eval_prompts_forget :type: str | List[str] :value: None .. py:attribute:: final_eval_prompts_retain :type: str | List[str] :value: None .. py:attribute:: prediction_type :type: Optional[str] :value: None .. py:attribute:: per_device_train_batch_size :type: int :value: None .. py:attribute:: gradient_accumulation_steps :type: int :value: None .. py:attribute:: num_train_epochs :type: int :value: None .. py:attribute:: learning_rate :type: float :value: None .. py:attribute:: lr_scheduler_type :type: str :value: None .. py:attribute:: should_log :type: bool :value: None .. py:attribute:: local_rank :type: int :value: None .. py:attribute:: device :type: str :value: None .. py:attribute:: n_gpu :type: int :value: None .. py:attribute:: output_dir :type: str :value: None .. py:attribute:: cache_dir :type: Optional[str] :value: None .. py:attribute:: hub_token :type: Optional[str] :value: None .. py:attribute:: hub_model_id :type: Optional[str] :value: None .. py:attribute:: logging_dir :type: str :value: None .. py:attribute:: logging_steps :type: int :value: None .. py:attribute:: save_strategy :type: str :value: None .. py:attribute:: save_total_limit :type: int :value: None .. py:attribute:: gradient_checkpointing :type: bool :value: None .. py:attribute:: enable_xformers_memory_efficient_attention :type: bool :value: None .. py:attribute:: mixed_precision :type: Optional[str] :value: None .. py:attribute:: allow_tf32 :type: bool :value: None .. py:attribute:: use_8bit_adam :type: bool :value: None .. py:attribute:: report_to :type: str :value: None .. py:attribute:: gradient_weighting_method :type: vision_unlearning.utils.gradient_weighting.GradientWeightingMethod :value: None .. py:attribute:: compute_gradient_conflict :type: bool :value: None .. py:attribute:: compute_runtimes :type: bool :value: None .. py:attribute:: compute_memory :type: bool :value: None .. py:attribute:: max_train_steps :type: Optional[int] :value: None .. py:attribute:: lr_warmup_steps :type: int :value: None .. py:attribute:: adam_beta1 :type: float :value: None .. py:attribute:: adam_beta2 :type: float :value: None .. py:attribute:: adam_weight_decay :type: float :value: None .. py:attribute:: adam_epsilon :type: float :value: None .. py:attribute:: max_grad_norm :type: float :value: None .. py:attribute:: checkpointing_steps :type: int :value: None .. py:attribute:: checkpoints_total_limit :type: Optional[int] :value: None .. py:attribute:: resume_from_checkpoint :type: Optional[str] :value: None .. py:attribute:: noise_offset :type: float :value: None .. py:attribute:: _accelerator :type: Optional[accelerate.Accelerator] :value: None .. py:attribute:: _output_dir_checkpoints :type: Optional[str] :value: None .. py:attribute:: _output_dir_lora :type: Optional[str] :value: None .. py:attribute:: _lora_weight_name :type: str :value: 'pytorch_lora_weights.safetensors' .. py:attribute:: _images :type: Dict[str, PIL.Image.Image] .. py:attribute:: _weight_dtype :type: Any :value: Ellipsis .. py:attribute:: _similarities_gr :type: List[float] :value: [] .. py:attribute:: _similarities_gf :type: List[float] :value: [] .. py:attribute:: _noise_scheduler :type: Any :value: None .. py:attribute:: _tokenizer :type: Any :value: None .. py:attribute:: _text_encoder :type: Any :value: None .. py:attribute:: _vae :type: Any :value: None .. py:attribute:: _unet :type: Optional[diffusers.models.unets.unet_2d_condition.UNet2DConditionModel] :value: None .. py:attribute:: _train_forget_dataloader :type: Optional[torch.utils.data.DataLoader] :value: None .. py:attribute:: _train_retain_dataloader :type: Optional[torch.utils.data.DataLoader] :value: None .. py:attribute:: _optimizer :type: Any :value: None .. py:attribute:: _lr_scheduler :type: Any :value: None .. py:attribute:: _lora_layers :type: Any :value: None .. py:attribute:: _peak_mem :type: int :value: 0 .. py:method:: model_post_init(__context: Optional[dict] = None) -> None Override this method to perform additional initialization after `__init__` and `model_construct`. This is useful if you want to do some validation that requires the entire model to be initialized. .. py:method:: _pre_checks() -> None .. py:method:: _get_lora_config() -> peft.LoraConfig .. py:method:: _get_accelerator() .. py:method:: _hook_after_lora_init() .. py:method:: _hook_before_load_model() .. py:method:: _save_lora_layers() Side-effects: modifies self._unet in-place (casts to float32), saves two directories self._output_dir_super and self._output_dir_sub .. py:method:: train() .. py:method:: _prepare_dataloaders() -> Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader] :abstractmethod: .. py:method:: _train_one_batch(batch_forget, batch_retain) :abstractmethod: .. py:class:: UnlearnerLoraDirect(/, **data: Any) Bases: :py:obj:`UnlearnerLora` Straight-forward finetuning .. py:method:: _prepare_dataloaders() -> Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader] .. py:method:: _train_one_batch(batch_forget, batch_retain)