Efficient UAV Swarm Coordination via Graph-Based Multi-Agent Reinforcement Learning

Abstract

Swarm robotics seeks to achieve coordinated collective behaviour among multiple autonomous agents through local interactions and distributed decision-making. This thesis investigates the use of Graph Neural Networks (GNNs) together with Multi-Agent Reinforcement Learning (MARL) for learning coordination policies in drone swarms. SpeciĄcally, it explores how graph-based representations can encode the spatial relationships within a swarm and how a centralised-training, decentralised-execution scheme enables scalable and fully decentralised control. A multi-agent simulation framework was developed in which a Ćock-to-goal mission Ů driving a team of quadrotors to a common goal while remaining cohesive, aligned, and collision-free Ů is formalised as a graph-structured decentralised partially observable Markov decision process (G-Dec-POMDP). A parameter-shared GNN actor paired with a centralised critic was trained end-to-end, enabling each drone to derive its control action from only its local neighbourhood. Two graph encoders (GCN and GATv2) were combined with two learning algorithms (MAPPO and MADDPG) in a controlled 2 × 2 study. Experimental evaluation across multiple swarm conĄgurations demonstrated that only the MAPPO+GAT model learns the task, reaching the goal efficiently while maintaining stable formation and collision-free motion, and reproducing this behaviour on unseen validation scenarios. Deployed without retraining, it scaled from eight to Ąfteen agents. The Ąndings highlight the potential of graph-based reinforcement learning to generalise swarm coordination policies across varying team sizes and conditions.