The wide adoption of large-scale Internet of Things (IoT) systems has led to an unprecedented increase in backhaul traffic congestion, making it critical to optimize traffic management at the network edge. In IoT systems, the backhaul network is supported by various backhauling technologies that have different characteristics. Also, the characteristics of the backhaul links can be sometimes time-varying and have an unknown state, due to external factors such as having the resources shared with other systems. It is the responsibility of the edge devices to be able to forward IoT traffic through the unknown-state backhaul network by selecting the suitable backhaul link for each collected data flow. To the best of our knowledge, this type of backhaul selection problem is not addressed in the literature. Therefore, there is a crucial need to develop intelligent approaches enabling edge devices to learn how to deal with unknown-state (partially observable) components of the backhaul network, which is the primary goal of this paper. We propose an edge-based backhaul selection technique for improving traffic delivery by exploiting multi-objective feedback on delivery performance. The proposed approach relies on the Advantage-Actor-Critic Deep Reinforcement Learning (DRL) methods. Moreover, to improve the DRL training performance in large-scale deployments of distributed IoT systems, Federated Learning (FL) is applied enabling multiple edge devices to collaborate in training a shared backhaul selection policy. The proposed Federated DRL (F-DRL) approach is able to solve the backhaul selection problem as verified and demonstrated through extensive simulations.
A. Jarwan and M. Ibnkahla, “Edge-Based Federated Deep Reinforcement Learning for IoT Traffic Management,” IEEE Internet of Things Journal, pp. 1–1, 2022, doi: 10.1109/JIOT.2022.3174469.
For more details: IEEE Explorer