An increasing number of systems are now controlled by artificial intelligence (AI): Autonomous vehicles – such as drones or satellites – can be deployed in difficult to access places and used to gather information in real-time. Autonomous systems can also be used simultaneously and cooperate to reach better performances compared to using only one device – but not without challenges: it is no easy task to programme these entities to behave collectively as we want them to. We speak to computer scientist and researcher Florian Felten about his research in this area and the associated challenges.
From fighting wildfires, monitoring water quality, inspecting infrastructure, or real-time road traffic monitoring, using multiple drones is finding increasing applications.
Most of these applications consist of a single manually operated drone. Of course, when tens or hundreds of drones are deployed, manually controlling them is not feasible, hence pre-programmed systems, such as drone light shows have emerged, but there is room for improvement: this approach is time-consuming and does not handle unforeseen events well.
Finding the best algorithm to help the drones cooperate
“A new approach proposes to automatically design these collective behaviours by relying on reinforcement learning techniques,” explains Florian Felten, computer scientist and PhD researcher at the SnT of the University of Luxembourg.
Florian explains that in such an approach, a computer will run simulations of drones in an environment and associate good actions of every drone with positive feedback. Based on such feedback, the algorithm can determine the right actions to undertake in most situations.
“The main challenges consist in finding efficient ways – we call them algorithms when we want to look clever – to automatically derive these behaviours as well as making sure the results are what we expect. For now, the biggest obstacle is the computational needs of the methods used to design such autonomous behaviours. We are obviously making a lot of progress in AI as you can read on the press e.g., chatGPT. Yet, these techniques are for now limited to companies or universities having large computational resources. ”Florian Felten Computer scientist & PhD researcher, SnT (University of Luxembourg)
“I decided to dedicate some time on the question of making trade-offs between multiple objectives when designing behaviours. Indeed, I was startled early by the fact that most of the current AI systems we see out there only try to optimize for one objective while in the reality we very often make compromises between various aspects.” “For example, an AI charged to plan your next commuting trip would probably try to reduce the price of such trip, telling you to take your car, whereas in the real-world I expect somebody with more sensitivity to climate change to take the train or ride a bike.” – Florian Felten
Designing behaviours: The question of making trade-offs between multiples objectives
Up to now, Florian’s research has been focussed on the aspect of finding the right behaviours.
“I decided to dedicate some time on the question of making trade-offs between multiple objectives when designing behaviours. Indeed, I was startled early by the fact that most of the current AI systems we see out there only try to optimize for one objective while in the reality we very often make compromises between various aspects.”
“For example, an AI charged to plan your next commuting trip would probably try to reduce the price of such trip, telling you to take your car, whereas in the real-world I expect somebody with more sensitivity to climate change to take the train or ride a bike.”
“Now the issue is that most of the AI making headlines in the press are not able to appreciate that there is no “universal good way to behave”. Obviously, we have the same problems with the drones: we must find a good balance between energy consumption, speed, risk etc., depending on the user’s preferences. ”Florian Felten Computer scientist & PhD researcher, SnT (University of Luxembourg)
Multi-objective reinforcement learning (MORL)
Florian explains that this is why he decided to rely on multi-objective reinforcement learning (MORL) to design the behaviours of the drones. This is quite a new field of research, opening the door to a many possibilities in automated design applications.
The project Florian works on is ongoing, but the research has already led to the publication of a standardised interface to train these algorithms.
“With researchers from other universities, we realised that we were lacking standardised tools to implement our ideas and compare them fairly. We published a library for conducting MORL research which is currently one of the most popular libraries in the field. ”Florian Felten Computer scientist & PhD researcher, SnT (University of Luxembourg)
Florian Felten is a Computer Scientist and PhD researcher on the FNR funded CORE project ADARS at the SnT of the University of Luxembourg.
MORE ABOUT FLORIAN FELTEN
Describing his research in one sentence
“Automated design of autonomous robot swarm.”
More on his research, peer to peer
“Besides publishing a standard API and set of environments for MORL, we also provide a library containing various learning algorithms. Additionally, we published a few works with the idea of making a bridge between multi-objective optimization and reinforcement learning to ground MORL research on these older fields.”
What drives him as a researcher
“I believe AI has a huge potential to help in solving upcoming societal challenges. Yet, solving such problems require going beyond state-of-the-art methods, and I aspire to be part of such effort.
“Constantly being challenged is probably the most exciting part of science. There is always something more to do, something to discover or interesting things to read about.”
Where he sees himself in 5 years
“I think I would like to go back to the industry to be able to have more impact on the current challenges we face such as climate change, efficient transport or energy crisis.”
About Spotlight on Young Researchers
Spotlight on Young Researchers is an annual FNR campaign where we shine a Spotlight on early-career researchers across the world with a connection to Luxembourg. Over 100 features have been published since the first edition in 2016.Discover all FNR Highlights
Spotlight on Young Researchers: Bio-waste from beer brewing – upcycling for a more sustainable future
The growth in world population means an increase in food and beverage production and ultimately more bio-waste. Beer is the…
Spotlight on Young Researchers: Improving educational assessment with cognitive psychology
Assessing competency in some form or another has been around almost as long as civilisation – and it has come…
Spotlight on Young Researchers: Saving the coral reefs
Coral reefs are one of the most vital ecosystems on our planet: Home to a quarter of all marine life,…
Spotlight on Young Researchers – revisited 5 years later: A post-PhD life in finance
When Eric Finn Schaanning was featured in Spotlight on Young Researchers in 2017, he had just defended his AFR PhD…
Spotlight on Young Researchers: Improving how industrial plants are engineered
Current engineering systems have an issue: It is fragmented – process, Piping and structural design are carried out by separate…
Spotlight on Young Researchers: Understanding the impact climate change has on crop-threatening insects
A type of insect known as whitefly spreads plant viruses and has devastating effects on the vegetables that we depend…