vision_unlearning.unlearner.fade ================================ .. py:module:: vision_unlearning.unlearner.fade .. autoapi-nested-parse:: Implementation of FADE (in its three variants: non-sparse, sparse-per-module, sparse-per-weight). Please cite the following paper if you use this code: @misc{kelsch2026fadeselectiveforgettingsparse, title={FADE: Selective Forgetting via Sparse LoRA and Self-Distillation}, author={Carolina R. Kelsch and Leonardo S. B. Pereira and Natnael Mola and Luis H. Arribas and Juan C. S. M. Avedillo}, year={2026}, eprint={2602.07058}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2602.07058}, } Classes ------- .. autoapisummary:: vision_unlearning.unlearner.fade.UnlearnerLoraDistillation vision_unlearning.unlearner.fade.UnlearnerLoraDistillationSparsePerModule vision_unlearning.unlearner.fade.SharedR vision_unlearning.unlearner.fade.ElasticLoRALinear vision_unlearning.unlearner.fade.UnlearnerLoraDistillationSparsePerWeight vision_unlearning.unlearner.fade.UnlearnerLoraDistillationSparse Functions --------- .. autoapisummary:: vision_unlearning.unlearner.fade.check_lora_sparsity vision_unlearning.unlearner.fade.is_valid_elastic_adapter_config vision_unlearning.unlearner.fade.is_valid_custom_adapter_config vision_unlearning.unlearner.fade.load_custom_adapter_config vision_unlearning.unlearner.fade.get_rank_value vision_unlearning.unlearner.fade.extract_sub_adapter vision_unlearning.unlearner.fade.make_lora_elastic vision_unlearning.unlearner.fade.find_module_name_by_weights vision_unlearning.unlearner.fade.calculate_sparsity_lora Module Contents --------------- .. py:class:: UnlearnerLoraDistillation(/, **data: Any) Bases: :py:obj:`vision_unlearning.unlearner.UnlearnerLora` SparsePEFT is not active in this trainer .. py:attribute:: overwritting_concept :type: Optional[str] :value: None .. py:attribute:: overwrite_column :type: str :value: 'overwrite' .. py:attribute:: json_metafile :type: Optional[str] :value: None .. py:attribute:: is_lora_negated :type: bool :value: None { "forget": [ { "file_name": "images/Architectures-Warm_Smear-19.jpg", "text": "An image of Architectures in Warm Smear style.", "overwrite": "An image of Architectures in Photo style." }, { "file_name": "images/Architectures-Warm_Smear-8.jpg", "text": "An image of Architectures in Warm Smear style.", "overwrite": "An image of Architectures in Photo style." }, { "file_name": "images/Architectures-Warm_Smear-12.jpg", "text": "An image of Architectures in Warm Smear style.", "overwrite": "An image of Architectures in Photo style." } ], "retain": [ { "file_name": "images/Architectures-Abstractionism-6.jpg", "text": "An image of Architectures in Abstractionism style.", "overwrite": "" }, { "file_name": "images/Architectures-Abstractionism-4.jpg", "text": "An image of Architectures in Abstractionism style.", "overwrite": "" }, { "file_name": "images/Architectures-Abstractionism-2.jpg", "text": "An image of Architectures in Abstractionism style.", "overwrite": "" } ] } .. py:method:: _pre_checks() -> None .. py:method:: _prepare_dataloaders() -> Tuple[torch.utils.data.DataLoader, torch.utils.data.DataLoader] Get the datasets: you can either provide your own training and evaluation files or specify a Dataset from the hub (the dataset will be downloaded automatically from the datasets Hub). In distributed training, the load_dataset function guarantees that only one local process can concurrently download the dataset. Downloading and loading a dataset from the hub. Characteristics of the returned dataloaders: * Batch size and number of workers are set according to the training arguments. * shuffled * collate behavior: Batches are created by stacking the per-example tensors (pixel_values stacked into contiguous FloatTensor) * Fields * pixel_values: preprocessed images, ready to be fed to the vae (i.e. resized, cropped, normalized...). Shape=[batch size, 3, resolution, resolution] * input_ids: tokenized captions, ready to be fed to the text encoder. Shape=[batch size, sequence length] * forget_ids: tokenized overwrite captions, ready to be fed to the text encoder; Returned just be forget dataloader. Shape=[batch size, sequence length] .. py:method:: _train_one_batch(batch_forget, batch_retain) .. py:class:: UnlearnerLoraDistillationSparsePerModule(/, **data: Any) Bases: :py:obj:`UnlearnerLoraDistillation` SparsePEFT is not active in this trainer .. py:attribute:: sparsity_inclusiveness :type: float :value: None .. py:attribute:: parameter_attribution_method :type: vision_unlearning.utils.parameter_attribution.ParameterAttributionMethod .. py:attribute:: attribution_overwrite_if_exists :type: bool :value: False .. py:attribute:: _attribution_path :type: Optional[str] :value: None .. py:method:: _get_lora_config() -> peft.LoraConfig .. py:function:: check_lora_sparsity(adapter_path: str, config_file_name: str = 'adapter_config.json', model_file_name: str = 'adapter_model.bin') -> float .. py:function:: is_valid_elastic_adapter_config(config) .. py:function:: is_valid_custom_adapter_config(config) .. py:function:: load_custom_adapter_config(custom_adapter_config_file) .. py:function:: get_rank_value(space, strategy) .. py:function:: extract_sub_adapter(adapter_model: str, elastic_adapter_config_file: str, output_dir: str, adapter_version: Literal['maximal', 'heuristic', 'minimal'] = 'heuristic') .. py:class:: SharedR(r_values) .. py:attribute:: r_values .. py:attribute:: current_r .. py:method:: __call__() .. py:method:: update_r() .. py:class:: ElasticLoRALinear(in_features, out_features, shared_r, is_lora_A, bias=False, is_group_head=False) Bases: :py:obj:`torch.nn.Linear` Applies an affine linear transformation to the incoming data: :math:`y = xA^T + b`. This module supports :ref:`TensorFloat32`. On certain ROCm devices, when using float16 inputs this module will use :ref:`different precision` for backward. :param in_features: size of each input sample :param out_features: size of each output sample :param bias: If set to ``False``, the layer will not learn an additive bias. Default: ``True`` Shape: - Input: :math:`(*, H_\text{in})` where :math:`*` means any number of dimensions including none and :math:`H_\text{in} = \text{in\_features}`. - Output: :math:`(*, H_\text{out})` where all but the last dimension are the same shape as the input and :math:`H_\text{out} = \text{out\_features}`. .. attribute:: weight the learnable weights of the module of shape :math:`(\text{out\_features}, \text{in\_features})`. The values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})`, where :math:`k = \frac{1}{\text{in\_features}}` .. attribute:: bias the learnable bias of the module of shape :math:`(\text{out\_features})`. If :attr:`bias` is ``True``, the values are initialized from :math:`\mathcal{U}(-\sqrt{k}, \sqrt{k})` where :math:`k = \frac{1}{\text{in\_features}}` Examples:: >>> m = nn.Linear(20, 30) >>> input = torch.randn(128, 20) >>> output = m(input) >>> print(output.size()) torch.Size([128, 30]) .. py:attribute:: shared_r .. py:attribute:: is_lora_A .. py:attribute:: is_group_head :value: False .. py:attribute:: _weight .. py:attribute:: _bias .. py:method:: init_weights(orig: torch.nn.Linear) .. py:method:: active_sub_adapter() .. py:property:: masked_weight .. py:property:: weight .. py:property:: bias .. py:method:: state_dict(*args, destination=None, prefix='', keep_vars=False) Return a dictionary containing references to the whole state of the module. Both parameters and persistent buffers (e.g. running averages) are included. Keys are corresponding parameter and buffer names. Parameters and buffers set to ``None`` are not included. .. note:: The returned object is a shallow copy. It contains references to the module's parameters and buffers. .. warning:: Currently ``state_dict()`` also accepts positional arguments for ``destination``, ``prefix`` and ``keep_vars`` in order. However, this is being deprecated and keyword arguments will be enforced in future releases. .. warning:: Please avoid the use of argument ``destination`` as it is not designed for end-users. :param destination: If provided, the state of module will be updated into the dict and the same object is returned. Otherwise, an ``OrderedDict`` will be created and returned. Default: ``None``. :type destination: dict, optional :param prefix: a prefix added to parameter and buffer names to compose the keys in state_dict. Default: ``''``. :type prefix: str, optional :param keep_vars: by default the :class:`~torch.Tensor` s returned in the state dict are detached from autograd. If it's set to ``True``, detaching will not be performed. Default: ``False``. :type keep_vars: bool, optional :returns: a dictionary containing a whole state of the module :rtype: dict Example:: >>> # xdoctest: +SKIP("undefined vars") >>> module.state_dict().keys() ['bias', 'weight'] .. py:method:: load_state_dict(state_dict, strict=True, assign=False) Copy parameters and buffers from :attr:`state_dict` into this module and its descendants. If :attr:`strict` is ``True``, then the keys of :attr:`state_dict` must exactly match the keys returned by this module's :meth:`~torch.nn.Module.state_dict` function. .. warning:: If :attr:`assign` is ``True`` the optimizer must be created after the call to :attr:`load_state_dict` unless :func:`~torch.__future__.get_swap_module_params_on_conversion` is ``True``. :param state_dict: a dict containing parameters and persistent buffers. :type state_dict: dict :param strict: whether to strictly enforce that the keys in :attr:`state_dict` match the keys returned by this module's :meth:`~torch.nn.Module.state_dict` function. Default: ``True`` :type strict: bool, optional :param assign: When set to ``False``, the properties of the tensors in the current module are preserved whereas setting it to ``True`` preserves properties of the Tensors in the state dict. The only exception is the ``requires_grad`` field of :class:`~torch.nn.Parameter` for which the value from the module is preserved. Default: ``False`` :type assign: bool, optional :returns: * ``missing_keys`` is a list of str containing any keys that are expected by this module but missing from the provided ``state_dict``. * ``unexpected_keys`` is a list of str containing the keys that are not expected by this module but present in the provided ``state_dict``. :rtype: ``NamedTuple`` with ``missing_keys`` and ``unexpected_keys`` fields .. note:: If a parameter or buffer is registered as ``None`` and its corresponding key exists in :attr:`state_dict`, :meth:`load_state_dict` will raise a ``RuntimeError``. .. py:function:: make_lora_elastic(model, r_values, target_modules, share_rank_within_layer=True, config_save_dir=None) .. py:function:: find_module_name_by_weights(model: torch.nn.Module, target_module: torch.nn.Module) -> str Find the module name in `model` that has the exact same parameters as `target_module` Same = match by memory address .. py:function:: calculate_sparsity_lora(lora_path: str, filename: str, device: str = 'cuda', threshold: float = 1e-06) Calculate what is the percentage of zero values in the product A*B, per target module. Load from HuggingFace (TODO: implemenet for local files). @param threshold: Tolerance for "effectively zero" .. py:class:: UnlearnerLoraDistillationSparsePerWeight(/, **data: Any) Bases: :py:obj:`UnlearnerLoraDistillation` SparsePEFT is not active in this trainer .. py:attribute:: nls :type: bool :value: None .. py:attribute:: nls_target_modules :type: List[str] :value: None .. py:attribute:: search_space :type: List[int] :value: None .. py:attribute:: share_rank_within_layer :type: bool :value: None .. py:attribute:: quantization_aware :type: bool :value: None .. py:attribute:: parameter_attribution_method :type: vision_unlearning.utils.parameter_attribution.ParameterAttributionMethod .. py:attribute:: attribution_overwrite_if_exists :type: bool :value: False .. py:attribute:: sparsity_inclusiveness :type: float :value: None .. py:attribute:: _output_dir_super :type: Optional[str] :value: None .. py:attribute:: _output_dir_sub :type: Optional[str] :value: None .. py:attribute:: _attribution_path :type: Optional[str] :value: None .. py:attribute:: _mask_dict :type: Dict[str, torch.Tensor] .. py:method:: model_post_init(__context: Optional[dict] = None) -> None Override this method to perform additional initialization after `__init__` and `model_construct`. This is useful if you want to do some validation that requires the entire model to be initialized. .. py:method:: _get_lora_config() -> peft.LoraConfig .. py:method:: _hook_after_lora_init() Side-effect: modifies self._unet in-place .. py:method:: _get_accelerator() .. py:method:: _save_lora_layers() Side-effects: modifies self._unet in-place (casts to float32), saves two directories self._output_dir_super and self._output_dir_sub .. py:class:: UnlearnerLoraDistillationSparse(/, **data: Any) Bases: :py:obj:`UnlearnerLoraDistillationSparsePerWeight` SparsePEFT is not active in this trainer