Master Theses Topics – 2024/25

Supervisors: Tom Lenaerts, Yannick Molinghen

The Laser Learning Environment (LLE) is a cooperative Multi-Agent environment that has shown to be challenging due to its unique properties. At MLG, we’ve shown that agents were unable to accurately estimate the value of key states due to State Space Bottlenecks (SSBs) and identified this as the cause of poor performance in state-of-the-art mixing networks.

The objective of this thesis is to investigate Curriculum Learning (CL) methods and assess their ability to learn better policies. A particular focus will be on Unsupervised Environment Design (UED), where a meta-agent generates increasingly difficult yet feasible tasks. You’ll develop ways for a meta-agent to incrementally design an LLE environment and perform experiments to draw conclusions on CL regarding generalization capabilities of agents in LLE.

References:

• “Laser Learning Environment: a new cooperative environment for coordination-critical tasks”
• Laser Learning Environment Github repository: https://github.com/yamoling/lle
• “Unrewarded subgoals, a persisting problem in cooperative multi-agent Markov decision processes”
• “Value-Decomposition Networks For Cooperative Multi-Agent Learning”
• “QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning”
• “QPLEX: Duplex Dueling Multi-Agent Q-learning”
• “Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design”

Supervisors: Tom Lenaerts, Yannick Molinghen

For years, we’ve observed what looks like collaborative behavior in cooperative multi-agent reinforcement learning, but the reasons behind these behaviors remain unanswered. Previous work on single-agent RL distills policies in decision trees to provide clearer explanations on agent intentions. Other methods learn structural causal models of the environment or decompose reward signals into multiple signals corresponding to different game events.

This thesis aims to determine the best methods to analyze agent intentions when they exhibit collaborative behavior in multi-agent reinforcement learning. You’ll first evaluate these methods on single-agent scenarios before advancing to cooperative multi-agent ones. The suggested environment is the Laser Learning Environment (LLE).

References:

• “Distilling Deep Reinforcement Learning Policies in Soft Decision Trees”
• “Explainable Reinforcement Learning Through a Causal Lens”
• “Explainable Reinforcement Learning: A Survey”
• “Principle Component Analysis”
• “Visualizing data using t-SNE”
• “Explainable Reinforcement Learning via Reward Decomposition”

Supervisors: Pascal Tribel, Gianluca Bontempi

This thesis will focus on implementing a Digital Twin integrated with Internet of Things to monitor plant health, using various sensors as data collectors and a Raspberry Pi as a central device.

The student should be confident in Python/Linux, in working with hardware, and be proactive in discovering new libraries (including GPIO, MQTT, and FLASK). Interest in multidisciplinary applied research is essential.

References:

• https://www.researchgate.net/publication/378480725_Exploring_Digital_Twins_for_Plant_Growth_Monitoring
• https://ebjohnsen.org/publication/24-seams/24-seams.pdf

Supervisors: Yannick Molinghen, Tom Lenaerts

Reinforcement Learning comes in two flavors: model-based and model-free. Some algorithms learn a model of the environment to make predictions without requiring actual steps, which might be costly. Meanwhile, some single-agent exploration methods based on intrinsic curiosity build internal world models and check their accuracy to compute intrinsic rewards.

This thesis investigates how model-based multi-agent reinforcement learning can leverage internal environment models to improve exploration, comparing that to model-free MARL algorithms. The suggested environment is the Laser Learning Environment (LLE).

References:

• “Mastering Atari with Discrete World Models”
• “A Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers”
• “Curiosity-driven Exploration by Self-supervised Prediction”
• “Episodic Multi-agent Reinforcement Learning with Curiosity-driven Exploration”

Supervisor: Maarten Jansen

Selecting optimal models from broad non-nested model spectrums can be driven by criteria balancing good training set prediction and model complexity. Optimization over variable numbers is combinatorially complex and not feasible for high-dimensional data. This problem can be approximated by replacing the counting measure with a sum of magnitude estimators, changing a combinatorial problem into a convex, quadratic programming one.

This thesis applies variable selection in sparse inverse problems, or in deblurring and denoising images, using gradient projection or iterative thresholding.

Supervisors: Gianluca Bontempi and Gianmarco Paldino

This thesis focuses on designing and implementing machine learning methods for probability distribution classification to discover causal directionality from data.

The student should be an expert in R and Python programming, registered in the MA module on computational intelligence, proficient in Machine Learning, and passionate about interdisciplinary applied research.

References:

• https://link.springer.com/article/10.1007/s10115-021-01621-0
• CauseMe platform

Supervisors: P. Tribel, G. Bontempi

Earthquake monitoring includes analyzing seismic movements from measurements to determine tremors, foresee wave arrivals, retrieve initial conditions, and study seismic wave propagation. This thesis requires determining methods for optimal seismometer placement to minimize prediction error, relying on existing seismogram datasets and simulation libraries like PyAWD.

The student should be an expert in Python programming, registered in the MA computational intelligence module, proficient in Machine Learning, and passionate about interdisciplinary applied research.

References:

• PyAWD library

Supervisor: G. Bontempi

This thesis focuses on studying, designing, and implementing statistical learning techniques to calibrate traffic models based on counting data (from sensors or cameras). The student should be particularly expert in Python programming and learn to use the SUMO mobility simulator.

Registration in the MA computational intelligence module and passion for interdisciplinary research are required. A related internship is possible.

References:

• https://www.eclipse.org/sumo/
• https://link.springer.com/chapter/10.1007/978-981-16-9605-3_68

Supervisors: O. Caelen (SIRRIS) and G. Bontempi

Proposed by Dr. O. Caelen, MLG scientific collaborator and SIRRIS senior researcher. Details available separately.

The student should be registered in the MA computational intelligence module and have passion for interdisciplinary research. A related internship is possible.

Supervisor: Matthieu Defrance

Clustering analysis is routinely performed on omics data to explore or discover underlying cell identities. The high dimensionality and significant sparsity of these data (with false zero count observations) make clustering computationally challenging. This project studies state-of-the-art techniques for omics data clustering, emphasizing neural network approaches for initial data embedding.

Reference: https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-021-04210-8

Contact: Matthieu Defrance (matthieu.defrance@ulb.be)

Supervisor: Matthieu Defrance

High-throughput sequencing and genome-wide analyses have profoundly impacted genetic diagnosis of rare diseases. Beyond classical genetic variants calling, new methods based on epigenetic or transcriptomic alterations have emerged. This project develops and evaluates supervised classification methods for rare diseases classification.

Reference: Erfan Aref-Eshghi et al. Evaluation of DNA Methylation Episignatures for Diagnosis and Phenotype Correlations in 42 Mendelian Neurodevelopmental Disorders.

Contact: Matthieu Defrance (matthieu.defrance@ulb.be)

Supervisors: Tom Lenaerts, Axel Abels

Real-world decision-making requires balancing multiple, possibly conflicting objectives. Concerns like interpretability, fairness, and execution speed often conflict with primary performance metrics. This project evaluates algorithms for decision-making under uncertainty (multi-armed bandits) regarding these secondary objectives, potentially extending into procedural fairness and interpretability in contextual bandits.

References:

• Patil, Vishakha, et al. “Achieving fairness in the stochastic multi-armed bandit problem.”
• Turgay, Eralp, Doruk Oner, and Cem Tekin. “Multi-objective contextual bandit problem with similarity information.”
• Lattimore, Tor, and Csaba Szepesvári. Bandit algorithms.

Supervisor: Tom Lenaerts

This thesis examines how learning and evolution may find correlated equilibria, an extension of Nash equilibria in games. You’ll analyze the literature, formulate the state-of-the-art, and implement and test suggested approaches on learning problems to evaluate their usefulness.

References:

• Aumann, R.J. (1987). Correlated equilibrium as an expression of Bayesian rationality.
• Milgrom, P., and Roberts, J. (1991). Adaptive and sophisticated learning in normal form games.
• Foster, D.P., and Vohra, R.V. (1997). Calibrated learning and correlated equilibrium.
• Hart, S., and Mas‐Colell, A. (2000). A simple adaptive procedure leading to correlated equilibrium.
• Cripps, M. (1991). Correlated equilibria and evolutionary stability.
• Metzger, L.P. (2018). Evolution and correlated equilibrium.
• Arifovic, J., Boitnott, J.F., and Duffy, J. (2019). Learning correlated equilibria: An evolutionary approach.

Supervisors: Tom Lenaerts, Inas Bosch and Nassim Versbraegen

This thesis explores drug-disease association potential using knowledge graphs (KG) and KG embeddings. You’ll identify relevant contributions in this field, focus on one or two approaches to confirm published results, and examine their applicability for rare diseases and multi-mutant disease contexts.

References:

• Himmelstein, D. S., et al. (2017). Systematic integration of biomedical knowledge prioritizes drugs for repurposing.
• Roessler, H. I., et al. (2021). Drug repurposing for rare diseases.
• Bang, D., et al. (2023). Biomedical knowledge graph learning for drug repurposing.
• Johnson, R., et al. (2024). Graph Artificial Intelligence in Medicine.
• Perdomo-Quinteiro, P., & Belmonte-Hernández, A. (2024). Knowledge Graphs for drug repurposing.
• Wang, Q., et al. (2017). Knowledge graph embedding: A survey of approaches and applications.

Supervisors: Tom Lenaerts and Nassim Versbraegen

This thesis examines state-of-the-art machine learning methods for discovering and analyzing epistatic interactions between variants and genes. Epistasis occurs when different genetic loci contribute to a phenotype non-additively.

You’ll first document available methods for researching epistatic effects, creating a taxonomy based on technology or data usage. Then, you’ll select methods and datasets to implement and analyze, comparing their effectiveness in examining epistatic interactions.

References:

• Cordell, H. J. (2009). Detecting gene–gene interactions that underlie human diseases.
• Niel, C., et al. (2015). A survey about methods dedicated to epistasis detection.
• Chicco, D., & Faultless, T. (2021). Brief survey on machine learning in epistasis.
• Russ, D. (2023). Efficient strategies for epistasis detection in genome-wide data.
• Chang, Y. C., et al. (2020). GenEpi: gene-based epistasis discovery using machine learning.
• Abd El Hamid, M. M., et al. (2021). Machine learning for detecting epistasis interactions.

Supervisors: Tom Lenaerts and Nassim Versbraegen

This thesis investigates structural protein knowledge relevance for the OLIDA oligogenic diseases database. With Alphafold, we can now predict structures for disease cases involving multiple genes, potentially improving disease understanding and treatment.

You’ll collect protein sequence and structural knowledge for OLIDA instances, considering Alphafold’s confidence in generated structures for mutated regions. A comparison with monogenetic variants will be made, followed by developing a prototype for next-generation predictive methods.

References:

• Abramson, J., et al. (2024). Accurate structure prediction of biomolecular interactions with AlphaFold 3.
• Desai, D., et al. (2024). Review of AlphaFold 3: transformative advances in drug design and therapeutics.
• Lee, C. Y., et al. (2024). Systematic discovery of protein interaction interfaces using AlphaFold.
• Sebastiano, M. R., et al. (2022). AI-based protein structure databases for rare diseases research.
• Visibelli, A., et al. (2024). Molecular Origins of the Mendelian Rare Diseases.
• Scafuri, B., et al. (2022). Computational methods for pharmacological chaperones for rare diseases.
• Schmidt, A., et al. (2023). Predicting missense variant pathogenicity using AlphaFold2 features.

Supervisors: Tom Lenaerts and Nassim Versbraegen

With language model success and clear associations between natural language and protein/genetic language, methods are emerging to improve variant pathogenicity prediction. This thesis investigates the state-of-the-art of such methods and explores their potential to enhance our team’s approaches.

References:

• Lin, W., et al. (2024). Enhancing missense variant pathogenicity prediction with protein language models.
• Brandes, N., et al. (2023). Genome-wide prediction of disease variant effects with a deep protein language model.
• Molotkov, I., et al. (2024). Making sense of missense: challenges in variant pathogenicity prediction.
• Fan, X., et al. (2023). SHINE: protein language model-based pathogenicity prediction.
• Zhan, H., & Zhang, Z. (2024). DYNA: Disease-Specific Language Model for Variant Pathogenicity.
• Sayeed, M. A., et al. (2024). Gene Pathogenicity Prediction using Genomic Foundation Models.