Research Focus

Explainability (XAI)

Verifiable interpretability of AI systems through formal and auditable methods

What is Explainability?

Explainability (XAI) in AI systems goes far beyond generating visualizations or superficial heuristics. We seek methods that provide verifiable, traceable, and evaluable explanations of model behavior.

Our work integrates formal rigor, algorithmic auditability, and methods that allow us not only to understand what the model decides, but why it decides and how to verify it.

Interpretability Approaches

Different methods to understand AI model decisions

Local

Local Explanations

Understand specific decisions of an individual instance, using techniques like LIME or SHAP that identify the most influential features in each prediction.

Global

Global Explanations

Allow understanding the overall behavior of the model, its implicit decision rules and patterns it follows in the complete dataset.

Sample

Sample Based

Explain predictions by comparing instances: Counterfactuals show minimal changes needed to alter an outcome; Prototypes identify the most representative examples for each class.

Concept

Concept Based

Explain model behavior through high-level human-understandable concepts (e.g., textures, shapes, medical patterns) rather than raw features, bridging the gap between latent representations and domain knowledge.

Formal

Formal Methods

Rigorous verification of model properties: reachability, safety, underlying logic. They use tools such as model checking and automatic synthesis.

Key Applications

Domains where explainability is critical

Medicine and Health

Diagnoses and treatments require explanations that doctors and patients can understand and verify.

Justice and Law

Automated legal decisions need to be auditable and justifiable under judicial review.

Finance and Credit

Credit decisions must be explainable to comply with regulatory requirements and protect rights.

Current Challenges

Open problems in verifiable explainability

Formal Scalability

Formal methods are rigorous but computationally expensive. We are investigating techniques that scale to deep neural networks.

Evaluation of Explanations

There is no consensus on metrics to evaluate the quality of an explanation. We are working on rigorous evaluation frameworks.

Accuracy-Interpretability Trade-off

More interpretable models often sacrifice performance. We seek architectures that balance both objectives.

Our Research

In the Explainability (XAI) line of the Trustworthy AI group at IAFER, we are developing methods to make AI system explainability a verifiable, auditable, and formally verifiable component.

🚀 Active Research Lines

Formal and Auditable Explainability: Methods that provide mathematically rigorous, evaluable, and traceable explanations
Evaluation of Explanations: Frameworks to measure and validate the quality, interpretability, and usefulness of generated explanations
Human-AI Collaboration: Systems that integrate explainability into the design of human-AI interaction for critical applications
Out-of-Distribution Detection Oriented to Explainability: Non-parametric statistical methods that detect out-of-distribution samples while maintaining transparency

📚 Publications

Loading publications...