Research Focus

Contrastive Learning

Learning representations by comparing pairs of similar and dissimilar examples

What is Contrastive Learning?

Contrastive Learning is a machine learning paradigm that trains models by learning to compare examples: bringing closer together in vector space those representations that should be similar (positive pairs) and pushing apart those that should be distinct (negative pairs). This approach allows the model to discover useful structures in the data without the need for manual labels, becoming one of the pillars of modern self-supervised learning.

The central idea can be summarized with an intuitive analogy: the model learns that two different augmentations of the same image should produce close representations to each other, while representations of different images should be far apart. The key component is the contrastive loss function, which guides this process of organizing the latent space.

What are contrastive pairs?

Contrastive learning is structured around the construction and comparison of two types of pairs

Similarity

Positive Pair

Two views of the same example generated through transformations or data augmentations. The model learns to produce close representations for these pairs, capturing invariance under irrelevant transformations.

Contrast

Negative Pair

Views from different examples in the dataset. The model learns to separate them in the vector space, forcing internal representations to capture semantically significant differences between instances.

Objective

Contrastive Loss

The most commonly used loss function is InfoNCE (Noise-Contrastive Estimation), which maximizes the similarity between positive pairs relative to negative ones. Variants such as NT-Xent loss and the frequency-aware loss for biometric signals extend this framework to specific domains.

Main Methods

Four families of algorithms that have defined the state of the art in contrastive learning

SimCLR

MoCo

BYOL & SimSiam

Frequency-Aware CL

Current Challenges

Despite its progress, Contrastive Learning faces fundamental challenges that define active research lines in the field

Selection of negative examples

Too-easy negatives don't provide learning; false negatives (similar instances treated as different) degrade representations.

Collapse of representations

Without negatives or with inappropriate architectures, the model can collapse and assign the same representation to all examples.

Domain-dependent augmentations

Transformations valid for images don't work on time series or EEG; defining which preserve relevant information requires domain knowledge.

Computational cost

Requires large batches or extensive queues of negatives, limiting direct use in resource-constrained environments.

Our Research

In the GPAIS group at IAFER chair, we apply Contrastive Learning to problems where labeled data is scarce and between-subject variability is high, with special focus on biomedical signals:

🚀 Active Research Lines

Frequency-Aware Contrastive Learning for EEG: Design of contrastive loss functions that incorporate knowledge of brain frequency bands for learning more transferable representations in seizure detection.
Cross-patient generalization: Contrastive methods that reduce domain shift between patients, key for deploying models in real clinical environments without per-patient retraining.
Federated contrastive learning: Adaptation of contrastive frameworks to federated learning environments where clinical data cannot be centralized due to privacy restrictions.

📚 Publications

Loading publications...

Practical Applications

Contrastive Learning has growing impact in multiple domains where unlabeled data is abundant but annotations are costly or scarce:

Computer Vision: Pre-training of visual representation models without annotations, with subsequent fine-tuning on classification, detection, or segmentation tasks with few labeled examples.
Natural Language Processing: Learning of sentence and document embeddings that capture semantic similarity, used in information retrieval and text classification.
Biomedical Signals and EEG: Automatic seizure detection, brain signal encoding for brain-computer interfaces, and analysis of physiological time series without labels.
Multimodal Learning: Alignment of representations from different modalities (image-text, audio-video), as in CLIP, enabling cross-modal search and understanding.
Recommendation Systems: Learning of user and item representations from implicit interactions, improving personalization without the need for explicit ratings.
Robotics and Control: Pre-training of control policies through unsupervised environment observations, facilitating transfer to new tasks with few demonstration examples.