vision_unlearning.benchmarks.I_care.result_templates

Attributes

shap
logger
rt_name_to_class
rt_name_to_params

Classes

ResultTemplate	!!! abstract "Usage Documentation"
ResultTemplateMetricMetricAlignment	Measures how strongly two MetricInterferencePerEntity metrics are correlated.
ResultTemplateMetricSimilarityAlignment	To what degree similar entities interfere more with each other.
ResultTemplateMetricSimilarityAlignmentMulti	Multi-input Single-output Regression Generalization of ResultTemplateMetricSimilarityAlignment (see also Appendix E, adapted from the multi-output setting).
ResultTemplateSignificantRelationshipNumerical	Measures whether two numerical attributes are significantly correlated.
ResultTemplateSignificantRelationshipCategorical	Statistical significance of the average MetricInterferencePerEntity across all
ResultTemplateCountSignificantRelationship	Number of significant relationships across all combinations of attributes and
ResultTemplateImplicitAssociationTest	Measures how the strength of automatic associations B between two pairs of
ResultTemplateMinimumCutInterference	Interprets a task as a directed weighted graph and computes the minimum cut separating two entities
ResultTemplateUnlearningVisualSummary	!!! abstract "Usage Documentation"
ResultTemplateInterferenceVisualSummary	Compared generated images for 9 identities: target, 4 worst (excluding target), 4 best
ResultTemplateMatrix	!!! abstract "Usage Documentation"
ResultTemplateInterferenceMatrix	MetricInterferencePerEntityPair between each possible combination of two entities
ResultTemplateSimilarityMatrix	Similarities between each possible combination of two entities within a task.
ResultTemplateMethodComparisonByMetricEntity	Compares the distribution of one MetricInterferencePerEntity across multiple

Functions

jacc_metric_score(→ float)	Jaccard similarity between two entities, based on their attributes.
display_interesting_interferences(→ None)	Compared generated images for 9 identities: target, 4 worst (excluding target), 4 best
analyze_relationship_regression(→ bool)	Test linear relationship between two numerical variables with significance test
analyze_relationship_category(→ bool)
analyze_relationship_numerical(→ bool)	Analyzes the relationship between a numerical attribute and a numerical metric
analyze_relationship_categorical(→ bool)	Analyzes the relationship between a categorical attribute and a numerical metric
analyze_correlation_between_pairwise_metrics(→ bool)	df1 and df2 are square DataFrames; index and cols are the same within both and among both
check_eval_results(→ float)	Check if the metric satisfy the EXPECTED threshold

Module Contents

vision_unlearning.benchmarks.I_care.result_templates.shap = None

vision_unlearning.benchmarks.I_care.result_templates.logger

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplate(/, **data: Any)

Bases: pydantic.BaseModel

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

recompute_if_exists: bool = False

save_outputs: bool = True

base_folder: str = 'assets'

remote_repository_name: str = 'LeonardoBenitez/VisionUnlearningEvaluationTestbeds'

abstract _serialize_parameters() → str

_get_data_path_remote() → str

_get_data_path_local() → str

classmethod _fig_to_bytes(fig: matplotlib.figure.Figure) → bytes

abstract _compute_from_scratch() → dict | list

compute() → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateMetricMetricAlignment(/, **data: Any)

Bases: ResultTemplate

Measures how strongly two MetricInterferencePerEntity metrics are correlated.

Arguments: m, t, u, m_e1, m_e2. Result: Pearson p-value, Spearman p-value, Pearson correlation, scatter plot. Interpretation: quantitative; the higher the correlation, the lower the need to calculate both metrics for this specific choice of m, t, and u.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_entity_1: vision_unlearning.benchmarks.I_care.configuration.type_me

interference_entity_2: vision_unlearning.benchmarks.I_care.configuration.type_me

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateMetricSimilarityAlignment(/, **data: Any)

Bases: ResultTemplate

To what degree similar entities interfere more with each other.

Formalized in ap:prediction, which also proposes its natural expansion to a multivariable and non-linear predictive regression.

Arguments: m, t, u, m_p, s. Result: Pearson p-value, Spearman p-value, Pearson correlation, scatter plot. Interpretation: quantitative; if this value is high, interference between two entities can be approximated by similarity (which is cheaper to compute for any new entity). Equivalently, the amount of “transmission wires” can be summarized by this single similarity function.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_pair: vision_unlearning.benchmarks.I_care.configuration.type_mp

similarity_metric: vision_unlearning.benchmarks.I_care.configuration.type_s

significance_threshold: float = 0.05

_serialize_parameters() → str

classmethod plot(data: dict, figsize: Tuple[int, int] = (6, 5), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch(exclude_diagonal: bool = True) → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateMetricSimilarityAlignmentMulti(/, **data: Any)

Bases: ResultTemplate

Multi-input Single-output Regression Generalization of ResultTemplateMetricSimilarityAlignment (see also Appendix E, adapted from the multi-output setting). Also, the interpretability and feature engineering aspects are improved.

—

We consider a fixed model (m), task (t), and unlearning method (u), which are omitted for brevity.

The objective is to quantify whether interference between entities is aligned with their similarity, i.e., to what degree similar entities interfere more with each other.

For every ordered pair of distinct entities (e_i, e_j in t) with (i

eq j), we observe several SimilarityBetweenEntities measures, indexed by superscripts (ell = 1, 2, dots, |S|), and a single MetricInterferencePerEntityPair target (m_p(e_i,e_j)).

Each ordered pair ((e_i, e_j)) is therefore treated as one data point with feature vector

$$ mathbf{X}_{ij} = ig( s^{(1)}(e_i, e_j), dots, s^{(|S|)}(e_i, e_j) ig) $$

and scalar target

$$ Y_{ij} = m_p(e_i, e_j). $$

The resulting dataset is

$$ mathcal{D} = { (mathbf{X}_{ij}, Y_{ij}) mid e_i, e_j in t,i

eq j

}. $$

From this dataset, a regression model can be estimated using standard regression procedures with appropriate validation.

In the linear case,

$$ Y_{ij} = eta_0 + sum_{ell=1}^{|S|} eta_{ell} X^{(ell)}_{ij} +

arepsilon_{ij}.

$$

Given a specific entity (e_i) whose removal is considered, similarities

$$ X^{(ell)}_{ij} = s^{(ell)}(e_i, e_j) $$

can be computed for all remaining entities (e_j in t). The fitted model then yields predictions

$$ hat{Y}_{ij} = f(mathbf{X}_{ij}), $$

which approximate the expected interference on each receiver entity.

Furthermore, the concept of similarity may also encode several forms of practical data engineering. For example, one may define: - a distinct similarity function for each attribute, or - a similarity function based only on the attributes of the emitter entity.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_pair: vision_unlearning.benchmarks.I_care.configuration.type_mp

similarity_metric_list: List[vision_unlearning.benchmarks.I_care.configuration.type_s]

significance_threshold: float = 0.05

include_attribute_diff_similarity: bool = True

include_attribute_value_similarity: bool = True

regression_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_regression_algorithm = 'linear_regression'

random_state: int = 42

test_size: float = 0.3

_serialize_parameters() → str

_get_partial_path_local()

classmethod plot(data: dict, figsize: Tuple[int, int] = (6, 15), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch(exclude_diagonal: bool = True, entity_col: str = 'name') → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateSignificantRelationshipNumerical(/, **data: Any)

Bases: ResultTemplate

Measures whether two numerical attributes are significantly correlated.

Formalized in ap:rt_relationship.

Arguments: m, t, u, m_e, a. Result: Pearson p-value, Spearman p-value, Pearson correlation, scatter plot. Interpretation: qualitative; the researcher should decide if it is ethical or desirable that this attribute propagates interferences.

Pearson test

Use when you want to measure a linear relationship. Assumptions:

• Both variables are continuous

• Relationship is linear

• Bivariate normality (both jointly Gaussian)

• Homoscedasticity (constant variance)

• No strong outliers (very sensitive)

Detects: linear correlation only Fails when: relationship is monotonic but non-linear, or heavy outliers exist

Spearman test

Use when you want to measure a monotonic relationship (not necessarily linear) or data is non-Gaussian. Assumptions:

• Variables are at least ordinal

• Relationship is monotonic (increasing or decreasing)

• No distributional assumptions

• Robust to outliers

Detects: any monotonic trend (linear or curved) Fails when: relationship is non-monotonic (e.g., U-shaped)

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_entity: vision_unlearning.benchmarks.I_care.configuration.type_me

attribute: str

significance_threshold: float = 0.05

_get_data_path_remote() → str

classmethod plot(data: dict, figsize: Tuple[int, int] = (6, 5), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch() → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateSignificantRelationshipCategorical(/, **data: Any)

Bases: ResultTemplate

Statistical significance of the average MetricInterferencePerEntity across all entities, when grouped by each of its values.

Formalized in ap:rt_relationship.

Arguments: m, t, u, m_e, a, optional filterAttributeValue. Result: ANOVA p-value, Kruskal-Wallis p-value, average value of m_e grouped by each value of a, grouped boxplot. Interpretation: qualitative; similar to SignificantRelationshipNumerical. The optional argument filterAttributeValue restricts which emitter entities are included, allowing the analysis of interference flow distribution, such as whether politicians cause more interference to other politicians than artists cause to other artists.

ANOVA

Use when you want to test if group means differ across 3+ independent groups under parametric assumptions. Assumptions:

• Dependent variable is continuous

• Groups are independent

• Normality within each group

• Homoscedasticity (equal variances)

• No strong outliers

Hypothesis:

• H₀: all group means are equal

• H₁: at least one mean differs

Detects: differences in means Fails when: heavy skew, unequal variances, small n with non-Gaussian data

Kruskal-Wallis

Use when you want to test if group distributions differ without parametric assumptions. Assumptions:

• Dependent variable is ordinal or continuous

• Groups are independent

• Same shaped distributions (only medians should differ for clean interpretation)

• No normality or equal-variance requirement

Hypothesis:

• H₀: all group distributions are equal

• H₁: at least one group differs

Detects: differences in medians / distributions Fails when: distributions differ in shape (then result is ambiguous)

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_entity: vision_unlearning.benchmarks.I_care.configuration.type_me

attribute: str

attribute_value: str | int | None = None

min_samples_per_category: int = 5

significance_threshold: float = 0.05

_get_data_path_remote() → str

classmethod plot(data: dict, extra_title: str = '', figsize: Tuple[int, int] = (6, 5), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch() → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateCountSignificantRelationship(/, **data: Any)

Bases: ResultTemplate

Number of significant relationships across all combinations of attributes and MetricInterferencePerEntity.

Arguments: m, t, u, list of m_e, list of a. Result: integer, list of significances. Interpretation: quantitative; the lower the better. Since the attributes for which it is ethical to propagate interference are constant across all models and methods, a higher value directly implies a higher number of ethical violations, that is, a larger number of “transmission wires” in a given task effectively used by this method and model.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm_list: List[vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm]

interference_entity_list: List[vision_unlearning.benchmarks.I_care.configuration.type_me]

attribute_list: List[str]

top_n: int = 10

_serialize_parameters() → str

classmethod plot(data: dict, figsize: Tuple[int, int] = (6, 5), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch() → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateImplicitAssociationTest(/, **data: Any)

Bases: ResultTemplate

Measures how the strength of automatic associations B between two pairs of entities changes after unlearning.

Arguments: m, t, u, a_1, a_2, l. Result: |a| x |a| real-valued tensor ΔB. Interpretation: qualitative; a human should decide whether it is ethical or desirable for the unlearning process to cause this change in implicit association between the chosen attributes.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

attribute_1: str

attribute_2: str

latent_embedding: vision_unlearning.benchmarks.I_care.configuration.type_l

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateMinimumCutInterference(/, **data: Any)

Bases: ResultTemplate

Interprets a task as a directed weighted graph and computes the minimum cut separating two entities As a consequence of the max-flow min-cut theorem, it directly follows that the minimum cut is the smallest influence whose removal eliminates every directed influence path from $e_1$ to $e_2$. Based on this, we conjecture that if we need to unlearn $e_1$ while minimizing harm to $e_2$, then the ideal intervention in the unlearning process is to increase the preservation of the emitter-side nodes. More intuitively, we can think of this intervention as “blocking the interference path,” as performed in electrical circuits to protect sensitive components (such as ground partitioning, shielding traces, among others. Arguments: $m$, $t$, $u$, $e_1$, $e_2$, $m_p$. Result: list of entities (corresponding to the emitter-side nodes). Interpretation: qualitative; small set of nodes through which most of the interference from $e_1$ propagates to $e_2$.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_pair: vision_unlearning.benchmarks.I_care.configuration.type_mp

entity_1: str

entity_2: str

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateUnlearningVisualSummary(/, **data: Any)

Bases: ResultTemplate

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateInterferenceVisualSummary(/, **data: Any)

Bases: ResultTemplate

Compared generated images for 9 identities: target, 4 worst (excluding target), 4 best

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_pair: vision_unlearning.benchmarks.I_care.configuration.type_mp

entity: str | None = None

entity_index: int | None = None

seed: int = 42

images_max_dim: int = 124

_resolve_entity()

Ensures both entity andentity_index are filled. Modifies in place At the end, both are set and consistent with each other

_serialize_parameters() → str

classmethod plot(data: dict, figsize: Tuple[int, int] | None = (18, 4), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch()

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateMatrix(/, **data: Any)

Bases: ResultTemplate

!!! abstract “Usage Documentation”

[Models](../concepts/models.md)

A base class for creating Pydantic models.

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_fields__

A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.

__pydantic_computed_fields__

A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

metric_key_name: str

classmethod plot_make_title(data: dict) → str

Abstractmethod:

classmethod plot(data: dict, figsize: Tuple[int, int] | None = None, cmap: str = 'viridis', title: str = '', xlabel: str = 'Receiver entity', ylabel: str = 'Emitter entity', return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateInterferenceMatrix(/, **data: Any)

Bases: ResultTemplateMatrix

MetricInterferencePerEntityPair between each possible combination of two entities within a task.

Arguments: m, t, u, m_p. Result: |t| x |t| real-valued tensor. Interpretation: qualitative; visual patterns may be spotted, especially when rearranging indices in a meaningful manner (for example, grouping professions together). Further quantitative values may be derived, such as the average value or the ratio between the diagonal-average value and the non-diagonal-average value.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

unlearning_algorithm: vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm

interference_pair: vision_unlearning.benchmarks.I_care.configuration.type_mp

metric_key_name: str = 'interference_pair'

_serialize_parameters() → str

classmethod plot_make_title(data: dict) → str

_compute_from_scratch()

vision_unlearning.benchmarks.I_care.result_templates.jacc_metric_score(entity_1: str, entity_2: str, metadata_filtered: List[Dict[str, Any]], entity_col: str = 'name') → float

Jaccard similarity between two entities, based on their attributes. Each attribute (column) contributes between 0 and 1 to the similarity We do not know the types and ranges of the attributes beforehand. For each attribute, both values for the two entities must be non-NaN and of the same type, otherwise we ignore that attribute (contribution 0). The calculation for each attribute is as follows: * If the attribute is categorical (str or bool), the contribution is 1 if the two entities have the same value for that attribute, and 0 otherwise. * If the attribute is numerical, and both values are between 0 and 1, the contribution is 1 - abs(value_1 - value_2) * If the attribute is numerical, and both values are between 1 and 100, the contribution is 1 - abs(value_1 - value_2) / 100 * else, the contribution is 0 (we do not know how to handle it, so we ignore it)

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateSimilarityMatrix(/, **data: Any)

Bases: ResultTemplateMatrix

Similarities between each possible combination of two entities within a task. * Arguments: $m, t, s$ * Result: $|t| imes |t|$ real-valued tensor * Interpretation: qualitative; visual patterns may be spotted, similarly to InterferenceMatrix.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'scenes'

similarity_metric: vision_unlearning.benchmarks.I_care.configuration.type_s = 'clip'

metric_key_name: str = 'similarity_metric'

_serialize_parameters() → str

_get_partial_path_local()

classmethod plot_make_title(data: dict) → str

_compute_from_scratch() → dict

class vision_unlearning.benchmarks.I_care.result_templates.ResultTemplateMethodComparisonByMetricEntity(/, **data: Any)

Bases: ResultTemplate

Compares the distribution of one MetricInterferencePerEntity across multiple unlearning methods.

• Arguments: m, t, me, list of u

• Result: per-method mean, median, std, n, values; box plot

• Interpretation: lower or higher depending on me direction. Use to rank methods by a single interference-per-entity metric.

model: vision_unlearning.benchmarks.I_care.configuration.type_model = 'sd1.4'

task: vision_unlearning.benchmarks.I_care.configuration.type_task = 'people'

interference_entity: vision_unlearning.benchmarks.I_care.configuration.type_me

unlearning_algorithm_list: List[vision_unlearning.benchmarks.I_care.configuration.type_unlearning_algorithm]

_serialize_parameters() → str

classmethod plot(data: dict, figsize: Tuple[int, int] = (6, 5), return_fig: bool = False) → Tuple[matplotlib.figure.Figure, matplotlib.pyplot.Axes] | None

_compute_from_scratch() → dict

vision_unlearning.benchmarks.I_care.result_templates.rt_name_to_class

vision_unlearning.benchmarks.I_care.result_templates.rt_name_to_params

vision_unlearning.benchmarks.I_care.result_templates.display_interesting_interferences(metadata_filtered: List[Dict[str, Any]], interference_per_pair: Dict[str, Dict[str, float]], index: int, task: Literal['scenes', 'objects', 'breeds', 'people'], method: Literal['munba', 'uce', 'distil'], num_train_epochs: int, metric: str, is_worst_biggest: bool, seed: int = 42, save_path: str | None = None) → None

Compared generated images for 9 identities: target, 4 worst (excluding target), 4 best @param metadata_filtered: should be appropriate for this task (this is not verified inside the function) @param interference_per_pair: should be appropriate for this task+index+method+num_train_epochs (this is not verified inside the function) @param index: identities the target

The combination of task+index+method+num_train_epochs identifies a unique unlearned model

vision_unlearning.benchmarks.I_care.result_templates.analyze_relationship_regression(df: pandas.DataFrame, x: str, y: str, expected_positive: bool = True, plot: bool = True) → bool

Test linear relationship between two numerical variables with significance test and direction check.

Returns True only if:

1. the slope is statistically significant (p < 0.05)

2. the slope sign matches expectation.

vision_unlearning.benchmarks.I_care.result_templates.analyze_relationship_category(df, metric: str, category: str, plot: bool = True) → bool

vision_unlearning.benchmarks.I_care.result_templates.analyze_relationship_numerical(df: pandas.DataFrame, attribute: str, metric: str, plot: bool = False, plot_only_significant: bool = False) → bool

Analyzes the relationship between a numerical attribute and a numerical metric @param df: interference_per_entity; assumes df[attribute] and df[metric] are numerical @param plot: whether to plot the results @param plot_only_significant: whether to plot only significant relationships; Only applies if plot=True @return: whether any significant relationship was found

—

Pearson test

Use when you want to measure a linear relationship.

Assumptions: * Both variables are continuous * Relationship is linear * Bivariate normality (both jointly Gaussian) * Homoscedasticity (constant variance) * No strong outliers (very sensitive)

Detects: linear correlation only Fails when: relationship is monotonic but non-linear, or heavy outliers exist

---

Spearman test

Use when you want to measure a monotonic relationship (not necessarily linear) or data is non-Gaussian.

Assumptions: * Variables are at least ordinal * Relationship is monotonic (increasing or decreasing) * No distributional assumptions * Robust to outliers

Detects: any monotonic trend (linear or curved) Fails when: relationship is non-monotonic (e.g., U-shaped)

vision_unlearning.benchmarks.I_care.result_templates.analyze_relationship_categorical(df: pandas.DataFrame, attribute: str, metric: str, plot: bool = False, plot_only_significant: bool = False, show_axhline: float | None = None, min_samples_per_category: int = 5, extra_title: str = '') → bool

Analyzes the relationship between a categorical attribute and a numerical metric @param df: interference_per_entity; assumes df[attribute] is categorical and df[metric] is numerical @param plot: whether to plot the results @param plot_only_significant: whether to plot only significant relationships; Only applies if plot=True @param show_axhline: if provided, shows a horizontal line at this y-value; Only applies if plot=True @return: whether any significant relationship was found

---

ANOVA (f_oneway)

Use when you want to test if group means differ across 3+ independent groups under parametric assumptions.

Assumptions: * Dependent variable is continuous * Groups are independent * Normality within each group * Homoscedasticity (equal variances) * No strong outliers

Hypothesis: * H₀: all group means are equal * H₁: at least one mean differs

Detects: differences in means Fails when: heavy skew, unequal variances, small n with non-Gaussian data

---

Kruskal-Wallis (kruskal)

Use when you want to test if group distributions differ without parametric assumptions.

Assumptions: * Dependent variable is ordinal or continuous * Groups are independent * Same shaped distributions (only medians should differ for clean interpretation) * No normality or equal-variance requirement

Hypothesis: * H₀: all group distributions are equal * H₁: at least one group differs

Detects: differences in medians / distributions Fails when: distributions differ in shape (then result is ambiguous)

vision_unlearning.benchmarks.I_care.result_templates.analyze_correlation_between_pairwise_metrics(df1: pandas.DataFrame, df2: pandas.DataFrame, metric1_name: str, metric2_name: str, exclude_diagonal: bool = True, plot=True, plot_only_significant=True) → bool

df1 and df2 are square DataFrames; index and cols are the same within both and among both

vision_unlearning.benchmarks.I_care.result_templates.check_eval_results(eval_results, name, threshold: float, operator: Literal['gt', 'lt']) → float

Check if the metric satisfy the EXPECTED threshold