Optimizing Federated Learning with Deep Reinforcement Learning: Enhanced Strategies for Client Selection and Resource Allocation

Abstract

Federated Learning (FL) has emerged as a distributed machine learning paradigm that enables multiple decentralized clients to collaboratively train a global model without sharing raw data, preserving privacy and reducing the need for centralized data storage. However, this decentralized nature introduces new challenges, particularly in client selection and resource management, which significantly affect model performance, communication efficiency, and system fairness. Hierarchical Federated Learning (HFL) further extends FL by introducing intermediate aggregation levels (e.g., edge servers), which can improve scalability and reduce communication bottlenecks but also add layers of complexity in resource coordination and scheduling. Various approaches have been proposed to address the challenges of client selection and resource management in Federated Learning. Among these, Deep Reinforcement Learning (DRL) stands out for its ability to adaptively optimize decisions in dynamic, heterogeneous environments. By learning from system feedback, DRL has demonstrated promising potential in improving model performance, fairness, and resource efficiency. This thesis explores the landscape of client selection and model aggregation strategies in both FL and HFL settings, with a focus on DRL-based approaches. Through a comparative analysis of existing techniques,including clustering methods, graph-based models, and multi-objective selection policies,this study aims to shed light on their strengths, limitations, and applicability under various constraints such as non-i.i.d. data, limited computational resources, and variable network conditions. The goal is to contribute toward the development of more robust, efficient, and fair federated learning frameworks tailored for real-world deployment scenarios.