In human interactions, it is extremely common to communicate information with each other to achieve a task. The nature of this information can be diverse, from advertising intents to communicating knowledge or information about how our environment works.
Nowadays, when accomplishing complex cooperative tasks, it is rare in the field of Multi-Agent Reinforcement Learning to include any kind of communication between agents.
The objective of this proposal is to investigate the existing communication methods (see references) and to extend value-based algorithms (e.g. Deep Q-learning, VDN, QMIX) with communication.
Your scientific contribution will be the comparison and discussion of the relevance of using certain combinations of algorithms and communication method. Your contribution can be further extended by proposing new communication methods and comparing them to existing ones.

References:
– Simon Vanneste, Astrid Vanneste, Stig Bosmans, Siegfried Mercelis and Peter Hellinckx, Learning to Communicate with Multi-agent Reinforcement Learning Using Value-Decomposition Network, *International Conference on P2P, Parallel, Grid, Cloud and Internet Computing*, https://link.springer.com/chapter/10.1007/978-3-030-33509-0_69
– Sunehag, Peter, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, et al. Value-Decomposition Networks for Cooperative Multi-Agent Learning Based on Team Reward. _Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS_ 3 (2018): 2085 87,  https://arxiv.org/abs/1706.05296
– Tampuu, Ardi, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, and Raul Vicente. Multiagent Cooperation and Competition with Deep Reinforcement Learning . _PLoS ONE_ 12, no. 4 (2017): 115. https://doi.org/10.1371/journal.pone.0172395
– Kajić, Ivana, Eser Aygün, and Doina Precup. Learning to Cooperate: Emergent Communication in Multi-Agent Navigation 1, no. 1969 (2020). http://arxiv.org/abs/2004.01097
– Peng, Zhaoqing, Libo Zhang, and Tiejian Luo. Learning to Communicate via Supervised Attentional Message Processing. _ACM International Conference Proceeding Series_, no. Nips (2018): 11 16. https://doi.org/10.1145/3205326.3205346

Contacts:

Yannick Molinghen, Tom Lenaerts

There exists two main families of reinforcement learning algorithms: value-based methods such as Q-learning that learns a value function for each action, and policy gradient methods such as Actor-Critic that directly optimises the action probability distribution.
While both methods are widely studied in single agent setups, policy gradient methods are less studied in multi agent ones.
During this master thesis, you will first have to become familiar with single agent policy gradient algorithms and be able to apply them on a specific environment.
From that point, your scientific contribution is manifold. First of all you will compare the performance of agents with existing value-based algorithms and discuss these results in light of the literature. Then you should have a discussion about the relevance of this comparison because of the different natures of these algorithms (value based vs policy gradient) and of the training process. Moreover, you will contribute to the development of an existing framework by adding new algorithms to it. Finally, you can propose extensions of existing algorithms and compare your results with existing ones.

References:
– Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms . _ArXiv_, 2017, 1 12. https://arxiv.org/abs/1707.06347
– Mnih, Volodymyr, Adri Puigdomnech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous Methods for Deep Reinforcement Learning . *ArXiv,* 16 June 2016. http://arxiv.org/abs/1602.01783
– Lowe, Ryan, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. _Advances in Neural Information Processing Systems_ 2017-Decem (2017): 6380 9 https://proceedings.neurips.cc/paper/2017/file/68a9750337a418a86fe06c1991a1d64c-Paper.pdf
– Sunehag, Peter, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, et al. Value-Decomposition Networks for Cooperative Multi-Agent Learning Based on Team Reward. _Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS_ 3 (2018): 2085 87, https://arxiv.org/abs/1706.05296

Contacts:

Yannick Molinghen, Tom Lenaerts

NO MORE AVAILABLE

This MA thesis will focus on study and design of eXplainable AI solutions for supporting marketing decision making in the context of a collaboration between MLG and Worldline Brussels (Dr. Olivier Caelen).

The student should be particularly expert in Python programming, be registered at the MA module on computational intelligence and have a passion for interdisciplinary applied research.

References:

• Worldline problem description
• Towards a rigorous science of interpretable ML
• General instructions about the MA thesis with Pr. Bontempi

PS. For interested students there is also the opportunity of an internship in the same company.

The selection of an optimal model from a broad spectrum of non-nested models can be driven by a criterium that balances a good prediction of the training set and complexity of the model, that is, the number of selected variables.  Optimization over a number of variables, or even comparison of models with a given number of variables is a problem of combinatorial complexity, and thus not feasible in the context of high-dimensional data. Part of the problem can be well approximated by changing the number of selected variables in the criterium by the sum of absolute values of the estimators of these variables within the selected model. The counting measure is replaced by a sum of magnitudes, thus changing a combinatorial problem into convex, quadratic programming problem. This problem can be solved by a wide range of algorithms, including direct methods, such as least angle regression, or iterative methods, such as iterative thresholding or gradient projection.  Moreover, for a fixed value of model complexity, the relaxed problem selects approximately the same model as the original combinatorial one. This is no longer the case when the model complexity is part of the optimization problem, but a correction for the divergence between the combinatorial and quadratic problem can be established.  The thesis is about the application of the variable selection in sparse inverse problems, or in deblurring and denoising images, using gradient projection or iterative thresholding.

The thesis will focus on the design and implementation of online time series forecasting techniques for massive streaming data.

NO MORE AVAILABLE

The student should be an expert in R and Python programming, be registered in the MA module on computational intelligence, be proficient in Machine Learning and have a passion for interdisciplinary applied research.

References:

• Forecasting with streams
• Forecasting Spotify streams with R
• General instructions about a MA thesis with Pr. Bontempi

The thesis will focus on the design and implementation of computational data-driven methods to infer causal directionality from multivariate time series.

The student should be an expert in R and Python programming, be registered in the MA module on computational intelligence, be proficient in Machine Learning and have a passion for interdisciplinary applied research.

References:

• https://link.springer.com/article/10.1007/s10115-021-01621-0
• CauseMe platform
• General instructions about a MA thesis with Pr. Bontempi

NO MORE AVAILABLE

This MA thesis will focus on

1. the simulation of traffic data by means of the SUMO simulator.
2. the use of the simulated data to design spatio-temporal techniques for interpolation, data imputation and forecasting

The student should be particularly expert in Python programming, be registered at the MA module on computational intelligence and have a passion for interdisciplinary research. Collaboration with a company (and internship)   is possible.

References:

• https://www.eclipse.org/sumo/
• https://link.springer.com/chapter/10.1007/978-981-16-9605-3_68
• MLG thesis on traffic forecasting

Single-cell RNA sequencing (scRNA-seq) has emerged has a main strategy to study transcriptional activity at the cellular level. Clustering analysis is routinely performed on scRNA-seq data to explore, recognize or discover underlying cell identities. The high dimensionality of scRNA-seq data and its significant sparsity accentuated by frequent dropout events, introducing false zero count observations, make the clustering analysis computationally challenging. The objective of this project is to study state of the art technique used to perform scRNA-seq data clustering with an emphasis on techniques involving neural networks to perform an initial embedding of the data.

Reference: https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-021-04210-8

High-throughput sequencing and genome-wide analyses have profoundly impacted the genetic diagnostic of rare diseases. Beside the classical genetic variants calling that target alterations of the DNA sequence itself, a new field of methods based on epigenetic (at the DNA level) or transcriptomic (at the RNA level) alterations has emerged. The objective of the project is to develop and evaluate supervised classification methods applied to rare diseases classification.

Reference: Erfan Aref-Eshghi et al. Evaluation of DNA Methylation Episignatures for Diagnosis and Phenotype Correlations in 42 Mendelian Neurodevelopmental Disorders. The American Journal of Human Genetics, Volume 106, Issue 3, 2020.

NO MORE AVAILABLE

This MA thesis will take place in the context of a collaboration between MLG and the Laboratory of Neurophysiology and Movement Biomechanics (LNMB).  An electroencephalogram (EEG) uses multiple electrodes to measure the electrical activity of post-synaptic potentials of cortical neurons located at specific parts of the brain. LNMB is composed of several researchers who developed a solid expertise in EEG signal acquisition and analysis. Over the years they acquired a large amount of EEG data from different domains (NASA astronauts in the ISS, hockey players from the national Belgian hockey team, tennis players from the Justine Henin Academy, children and adults with hyperactivity disorder…) and for various applications (brain-computer interface, increase human performance, diagnostic tool…).

The objective of the MA thesis is to work with cutting-edge technology and use state-of-the-art signal processing and Machine Learning techniques on EEG data.

The work will focus on i) exploring different EEG datasets ii) extracting relevant features from the brain state that may not be directly visible with standard EEG analysis iii) deploying different classification models to reach or improve state-of-the-art results.

The student should be expert in Python programming,  registered at the MA module on computational intelligence, have a passion for interdisciplinary research and be available to visit frequently the Erasme lab.

References:

– MNE : Gramfort, M. Luessi, E. Larson, D. Engemann, D. Strohmeier, C. Brodbeck, L. Parkkonen, M. Hämäläinen, MNE software for processing MEG and EEG data, NeuroImage, Volume 86, 1 February 2014, Pages 446-460, ISSN 1053-8119
https://martinos.org/mne/stable/index.html
– EEG Lab : A Delorme & S Makeig (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics (pdf, 0.7 MB) Journal of Neuroscience Methods 134:9-21
https://sccn.ucsd.edu/eeglab/index.php

PS. For interested students, LNMB also offers the possibility of an internship in the contexts of the MA2 TRAN-F-501 course.

Solving real world decision-making problems typically requires a careful trade-off between multiple, possibly conflicting, objectives. For example, essential concerns such as interpretability, fairness, and execution speed often conflict with the primary performance metric, such as classification accuracy. The objective of this project is to evaluate algorithms for decision-making under uncertainty (i.e., multi-armed bandits) in terms of these secondary objectives. If time permits, an extension into procedural fairness and interpretability in contextual bandits can be considered. As contextual bandits involve decisions made based on a set of features, it is crucial to ensure that these decisions are interpretable and made fairly with regards to a set of sensitive features (e.g., gender).
References:
Patil, Vishakha, et al. “Achieving fairness in the stochastic multi-armed bandit problem.” Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 04. 2020. https://ojs.aaai.org/index.php/AAAI/article/view/5986/5842
Turgay, Eralp, Doruk Oner, and Cem Tekin. “Multi-objective contextual bandit problem with similarity information.” International Conference on Artificial Intelligence and Statistics. PMLR, 2018. http://proceedings.mlr.press/v84/turgay18a/turgay18a.pdf
Lattimore, Tor, and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020. https://tor-lattimore.com/downloads/book/book.pdf

Studying genetic diseases with machine learning or AI techniques often requires a lot of data.  Especially in the realm of rare diseases this is a huge problem.  A solution may be to produce models that can generate synthetic data on which predictive systems can be trained.  Variational auto encoders (VAE) may be able to help in this context.

The goal of this thesis proposal is to first study the formalism and state-of-the-art of VAE and the applications they have been used. Once this context is defined, the ambition is then to explore how VAE can be used to capture variant information in genetic data collected for a panel of genes or a Mendeliome, and how this can be used to improve predictions.

References:

• Kingma, D. P., & Welling, M. (2019). An Introduction to Variational Autoencoders. Foundations and Trends® in Machine Learning, 12(4), 307-392.
• Doersch, C. (2016). Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908.
• Hawkins-Hooker, A., Depardieu, F., Baur, S., Couairon, G., Chen, A., & Bikard, D. (2021). Generating functional protein variants with variational autoencoders. PLoS computational biology, 17(2), e1008736.
• Battey, C. J., Coffing, G. C., & Kern, A. D. (2021). Visualizing population structure with variational autoencoders. G3, 11(1), 1-11.

Deep learning on knowledge graphs (Tom Lenaerts, Robin Petit, Alexandre Renaux)

This master thesis proposal aims to examine how deep learning can be used to learn things about the information in knowledge graphs.  As a first part you will get acquainted with both topics.  Once this is established you will define a learning problem (e.g link prediction, node classification) within the context of an knowledge graph that captures the state-of-the-art on digenic diseases and provide an algorithmic solution for it.

References:

Bacciu, D., Errica, F., Micheli, A., & Podda, M. (2020). A gentle introduction to deep learning for graphs. Neural Networks, 129, 203-221.

HOGAN, Aidan, et al. Knowledge graphs. Synthesis Lectures on Data, Semantics, and Knowledge, 2021, 12.2: 1-257.