An increasing number of systems are now controlled by artificial intelligence (AI): Autonomous vehicles – such as drones or satellites – can be deployed in difficult to access places and used to gather information in real-time. Autonomous systems can also be used simultaneously and cooperate to reach better performances compared to using only one device – but not without challenges: it is no easy task to programme these entities to behave collectively as we want them to. We speak to computer scientist and researcher Florian Felten about his research in this area and the associated challenges.

From fighting wildfires, monitoring water quality, inspecting infrastructure, or real-time road traffic monitoring, using multiple drones is finding increasing applications.

Most of these applications consist of a single manually operated drone. Of course, when tens or hundreds of drones are deployed, manually controlling them is not feasible, hence pre-programmed systems, such as drone light shows have emerged, but there is room for improvement: this approach is time-consuming and does not handle unforeseen events well.

Finding the best algorithm to help the drones cooperate

“A new approach proposes to automatically design these collective behaviours by relying on reinforcement learning techniques,” explains Florian Felten, computer scientist and PhD researcher at the SnT of the University of Luxembourg.

Florian explains that in such an approach, a computer will run simulations of drones in an environment and associate good actions of every drone with positive feedback. Based on such feedback, the algorithm can determine the right actions to undertake in most situations.

“The main challenges consist in finding efficient ways – we call them algorithms when we want to look clever – to automatically derive these behaviours as well as making sure the results are what we expect.rnrnFor now, the biggest obstacle is the computational needs of the methods used to design such autonomous behaviours. We are obviously making a lot of progress in AI as you can read on the press e.g., chatGPT. Yet, these techniques are for now limited to companies or universities having large computational resources. ”

Florian Felten Computer scientist, PhD researcher, SnT (University of Luxembourg)

Designing behaviours: The question of making trade-offs between multiples objectives

Up to now, Florian’s research has been focussed on the aspect of finding the right behaviours.

“I decided to dedicate some time on the question of making trade-offs between multiple objectives when designing behaviours. Indeed, I was startled early by the fact that most of the current AI systems we see out there only try to optimize for one objective while in the reality we very often make compromises between various aspects.”

“For example, an AI charged to plan your next commuting trip would probably try to reduce the price of such trip, telling you to take your car, whereas in the real-world I expect somebody with more sensitivity to climate change to take the train or ride a bike.”

“Now the issue is that most of the AI making headlines in the press are not able to appreciate that there is no “universal good way to behave”. Obviously, we have the same problems with the drones: we must find a good balance between energy consumption, speed, risk etc., depending on the user’s preferences.rn ”

Florian Felten Computer scientist PhD researcher, SnT (University of Luxembourg)

Multi-objective reinforcement learning (MORL)

Florian explains that this is why he decided to rely on multi-objective reinforcement learning (MORL) to design the behaviours of the drones. This is quite a new field of research, opening the door to a many possibilities in automated design applications.

The project Florian works on is ongoing, but the research has already led to the publication of a standardised interface to train these algorithms.

“With researchers from other universities, we realised that we were lacking standardised tools to implement our ideas and compare them fairly. We published a library for conducting MORL research which is currently one of the most popular libraries in the field. ”

Florian Felten Computer scientist, PhD researcher, SnT (University of Luxembourg)

Florian Felten is a Computer Scientist and PhD researcher on the FNR funded CORE project ADARS at the SnT of the University of Luxembourg.

About Spotlight on Young Researchers

Spotlight on Young Researchers is an annual FNR campaign where we shine a Spotlight on early-career researchers across the world with a connection to Luxembourg. Over 100 features have been published since the first edition in 2016.