This topic requires the student to speak both French and English !

In this master thesis proposal, the student is going to use Natural Language Processing (NLP) techniques to identify insulting or offensive sentences. The objective is to develop a software that can be plugged in to the “évaluation des enseignements” (evalens) system in order to suggest offensive or insulting comments that the “commission pédagogique” may want to hide.

A particularity of this topic is that comments on evalens can be written in multiple languages (mainly French and English). That specificity must be accounted for.

In the first year, the student is expected to explore the state of the art in the field of sentiment analysis.
In the second year, the student is expected to work on three different aspects.
1. Academic research: since this is a master thesis, some scientific contribution is expected.
2. Software development: the student has to develop a software that performs sentiment analysis that can be plugged in to the “évaluation des enseignements” system.
3. Technical: the student has to identify one or multiple technical solutions for this problem considering the system in place today. This also includes discussions with the team in charge of the “évaluation des enseignements” platform.

Useful links:
– VUB NLP course: https://ai.vub.ac.be/course/natural-language-processing-2/
– NTLK: https://www.nltk.org/
– Évaluation des enseignements: https://evalens.ulb.ac.be

For years, we have observed what looks like collaborative behaviour in cooperative multi-agent reinforcement learning, but the question of the reasons behind those behaviours remains unanswered.
Previous work on single-agent reinforcement learning distil policies in decision trees [1] to provide clearer explanations on the intents of the agents.
Other methods learn a structural causal model of the environment during the reinforcement learning phase [2] and encode causal relationships between variables of interest. A third approach to explainable single-agent RL is to decompose the reward signal into multiple signals corresponding to different events of the game [6].
[3] provides a good overview of the state of the art techniques used in explainable RL. Simultaneously, methods such as Principle Component Analysis [4] and t-SNE [5] from other areas of Machine Learning might turn out helpful to explain the reasons behind the behaviour of the value function.

The objective of this master thesis is to determine the best suited methods to analyse the intents of agents when they exhibit a collaborative behaviour in the scope of multi-agent reinforcement learning. A first part of the work would be to evaluate those methods on single-agent scenarios before going to cooperative multi-agent one. The suggested environment for this work is the Laser Learning Environment (LLE) https://github.com/yamoling/lle.

References:

1. “Distilling Deep Reinforcement Learning Policies in Soft Decision Trees”, Youri Coppens et al., 2019. https://researchportal.vub.be/en/publications/distilling-deep-reinforcement-learning-policies-in-soft-decision-
2. “Explainable Reinforcement Learning Through a Causal Lens”, Prashan Madumal et al., 2019.  https://arxiv.org/pdf/1905.10958.pdf
3. “Explainable Reinforcement Learning: A Survey”, Erika Puiutta and Eric MSP Veith, 2020. https://arxiv.org/pdf/2005.06247.pdf
4. “Principle Component Analysis”, Andrzej Maćkiewicz and Waldemar Ratajczak, 1993, https://www.sciencedirect.com/science/article/abs/pii/009830049390090R
5. “Visualizing data using t-SNE”, Laurens van der Maaten, 2008, https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
6. “Explainable Reinforcement Learning via Reward Decomposition”, Zoe Juozapaitis et al., 2020, https://web.engr.oregonstate.edu/~afern/papers/reward_decomposition__workshop_final.pdf

Reinforcement Learning often comes in two different flavours: model-based and model-free. Because the assumption of owning a perfect representation of the model is too strong in many cases, some reinforcement learning algorithms learn a model of the environment [1] and then use it to make predictions about their future without requiring to actually take steps in this environment, which might be costly.
Simultaneously, some single-agent exploration methods based on intrinsic curiosity [2] also build an internal model of the world and check how accurate it is [3] to compute the intrinsic reward added to the reward signal from the environment.
The suggested objective of this master thesis proposal is to investigate how model-based multi-agent reinforcement learning can leverage the internal model of the environment to improve exploration, and compare that to other model-free MARL algorithms [4]. The Laser Learning Environment (LLE) is the suggested environment for this topic https://github.com/yamoling/lle.

References:

1. “Mastering Atari with Discrete World Models”, Danijar Hafner and Timothy Lillicrap and Mohammad Norouzi and Jimmy Ba, 2022, https://arxiv.org/pdf/2010.02193.pdf
2. “A Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers”, Jurgen Schmidhuber. In From Animals to Animats, edited by Jean-Arcady Meyer, International Conference on Simulation Adaptive Behavior: From Animals to Animats., 222 27. The MIT Press, 1991. https://doi.org/10.7551/mitpress/3115.003.0030
3. “Curiosity-driven Exploration by Self-supervised Prediction”,  Deepak Pathak et al., 2017. https://arxiv.org/pdf/1705.05363.pdf
4. “Episodic Multi-agent Reinforcement Learning with Curiosity-driven Exploration”, Lulu Zheng and Jiarui Chen, et al.  https://arxiv.org/pdf/2111.11032.pdf

The generation of synthetic data is more and more important in machine learning. It can be used for data augmentation,  datasets anonymisation, oversampling, and classifier training. This MA thesis will assess existing techniques for synthetic tabular data generation in big data analytics (notably fraud and churn detection).   Conventional and generative AI techniques will be studied and evaluated. The student should be particularly expert in Python programming, be registered for the MA module on computational intelligence, and have a passion for interdisciplinary applied research.

References:

• Article on tabular data generation
• Generating synthetic data in finance
• Evaluation of synthetic data generators
• Time series data augmentation
• The synthetic data vault
• Instructions about MA thesis

The selection of an optimal model from a broad spectrum of non-nested models can be driven by a criterium that balances a good prediction of the training set and complexity of the model, that is, the number of selected variables.  Optimization over a number of variables, or even comparison of models with a given number of variables is a problem of combinatorial complexity, and thus not feasible in the context of high-dimensional data. Part of the problem can be well approximated by changing the number of selected variables in the criterium by the sum of absolute values of the estimators of these variables within the selected model. The counting measure is replaced by a sum of magnitudes, thus changing a combinatorial problem into convex, quadratic programming problem. This problem can be solved by a wide range of algorithms, including direct methods, such as least angle regression, or iterative methods, such as iterative thresholding or gradient projection.  Moreover, for a fixed value of model complexity, the relaxed problem selects approximately the same model as the original combinatorial one. This is no longer the case when the model complexity is part of the optimization problem, but a correction for the divergence between the combinatorial and quadratic problem can be established.  The thesis is about the application of the variable selection in sparse inverse problems, or in deblurring and denoising images, using gradient projection or iterative thresholding.

The thesis will focus on the design and implementation of computational data-driven methods to infer causal directionality from multivariate time series.

The student should be an expert in R and Python programming, be registered in the MA module on computational intelligence, be proficient in Machine Learning and have a passion for interdisciplinary applied research.

References:

• https://link.springer.com/article/10.1007/s10115-021-01621-0
• CauseMe platform
• General instructions about a MA thesis with Pr. Bontempi

Some MA thesis in collaboration with Macq Electronics are possible. More details here.

The student should be particularly expert in Python programming, be registered at the MA module on computational intelligence, and have a passion for interdisciplinary research. An internship on related topics is possible.

References:

• https://www.eclipse.org/sumo/
• https://link.springer.com/chapter/10.1007/978-981-16-9605-3_68
• MLG thesis on traffic forecasting

Stellar physics is entering an era of data-driven discoveries, mainly due to modern data mining and machine learning (ML) techniques enabling new powerful ways to extract information out of observational material. During the last decades, large ground-based sky surveys and all-sky space missions have harvested hundreds of terabytes of data corresponding to hundreds of millions of astronomical sources. In the Gaia era, we expect that the data volume will move towards the petabyte domain, requiring the handling of billions of sources. Astrophysics has emerged from a data-poor science to an intensive data-driven discipline. Current stellar survey datasets present novel algorithmic, computational, and statistical challenges. In this context, the study of stellar properties in single and binary stars can greatly benefit from the application of such ML techniques. We propose, in co-supervision with the [Machine Learning Group|Institut d’Astronomie et d’Astrophysique], to investigate such data mining and ML algorithms with an application to large and homogeneous sample of stellar spectra (e.g. from the Gaia-ESO survey, RAVE, GALAH, or Melchiors database). This project does not require specific background in theoretical Physics.

Contact: Matthieu Defrance (matthieu.defrance@ulb.be)

Clustering analysis is routinely performed on omics data (data procuced by DNA, RNA sequencing) to explore, recognize or discover underlying cell identities. The high dimensionality of omics data and its significant sparsity accentuated by frequent dropout events, introducing false zero count observations, make the clustering analysis computationally challenging. The objective of this project is to study state of the art technique used to perform omics data clustering with an emphasis on techniques involving neural networks to perform an initial embedding of the data.

Reference: https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-021-04210-8

Contact: Matthieu Defrance (matthieu.defrance@ulb.be)

High-throughput sequencing and genome-wide analyses have profoundly impacted the genetic diagnostic of rare diseases. Beside the classical genetic variants calling that target alterations of the DNA sequence itself, a new field of methods based on epigenetic (at the DNA level) or transcriptomic (at the RNA level) alterations has emerged. The objective of the project is to develop and evaluate supervised classification methods applied to rare diseases classification.

Reference: Erfan Aref-Eshghi et al. Evaluation of DNA Methylation Episignatures for Diagnosis and Phenotype Correlations in 42 Mendelian Neurodevelopmental Disorders. The American Journal of Human Genetics, Volume 106, Issue 3, 2020.

Contact: Matthieu Defrance (matthieu.defrance@ulb.be)

Solving real world decision-making problems typically requires a careful trade-off between multiple, possibly conflicting, objectives. For example, essential concerns such as interpretability, fairness, and execution speed often conflict with the primary performance metric, such as classification accuracy. The objective of this project is to evaluate algorithms for decision-making under uncertainty (i.e., multi-armed bandits) in terms of these secondary objectives. If time permits, an extension into procedural fairness and interpretability in contextual bandits can be considered. As contextual bandits involve decisions made based on a set of features, it is crucial to ensure that these decisions are interpretable and made fairly with regards to a set of sensitive features (e.g., gender).
References:
Patil, Vishakha, et al. “Achieving fairness in the stochastic multi-armed bandit problem.” Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34. No. 04. 2020. https://ojs.aaai.org/index.php/AAAI/article/view/5986/5842
Turgay, Eralp, Doruk Oner, and Cem Tekin. “Multi-objective contextual bandit problem with similarity information.” International Conference on Artificial Intelligence and Statistics. PMLR, 2018. http://proceedings.mlr.press/v84/turgay18a/turgay18a.pdf
Lattimore, Tor, and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020. https://tor-lattimore.com/downloads/book/book.pdf

In this thesis we will explore how social media platforms or recommender systems can be modelled as a game between multiple players, the users, the providers and the platform creators.  You will first examine the articles below making an overview of the theories that could be applied and the models that may be used to study the question.  For the thesis these models will be created and analysed to see how the interactions between the different stake-holders and their preferences influences the outcome and may make systems more or less polarised or more less biased.

1. Encarnação, S., Santos, F. P., Santos, F. C., Blass, V., Pacheco, J. M., & Portugali, J. (2016). Paradigm shifts and the interplay between state, business and civil sectors. Royal Society open science, 3(12), 160753.
2. Encarnação, S., Santos, F. P., Santos, F. C., Blass, V., Pacheco, J. M., & Portugali, J. (2018). Paths to the adoption of electric vehicles: An evolutionary game theoretical approach. Transportation Research Part B: Methodological, 113, 24-33.
3. Pitoura, E., Stefanidis, K., & Koutrika, G. (2022). Fairness in rankings and recommendations: an overview. The VLDB Journal, 1-28.
4. Abdollahpouri, H., Adomavicius, G., Burke, R., Guy, I., Jannach, D., Kamishima, T., … & Pizzato, L. (2020). Multistakeholder recommendation: Survey and research directions. User Modeling and User-Adapted Interaction, 30, 127-158.
5. Sanchez‐Cartas, J. M., & León, G. (2021). Multisided platforms and markets: A survey of the theoretical literature. Journal of Economic Surveys, 35(2), 452-487.
6. Santos, F. P., Encarnação, S., Santos, F. C., Portugali, J., & Pacheco, J. M. (2016). An evolutionary game theoretic approach to multi-sector coordination and self-organization. Entropy, 18(4), 152.