Aprimoramento de um algoritmo de roteamento baseado em aprendizado por reforço: um estudo de caso usando VoIP

Abstract

The channel capacity, the routers processing capability, and the routing algorithms are some of the main factors that directly impact on network performance. Network parameters such as packet loss, throughput, and delay affect the users’ quality–of–experience in different multimedia services. Routing algorithms are responsible for choosing the best route between a source node to a destination. However, conventional routing algorithms do not consider the history of the network data when making about, for example, overhead or recurring equipment failures. Therefore, it is expected that routing algorithms based on machine learning that use the network history for decision making present some advantages. Nevertheless, in a routing algorithm based on reinforcement learning (RL) technique, additional control message headers could be required. In this context, this research presents an enhanced routing protocol based on RL, named e-RLRP, in which the control message overhead is reduced. Specifically, a dynamic adjustment in the Hello message interval is implemented to compensate for the overhead generated by the use of RL. Different ad-hoc network scenarios are implemented in which network performance parameters, such as packet loss, delay, throughput and overhead are obtained. In addition, a Voice over IP (VoIP) communication scenario is implemented, in which E-model algorithm is used to predict the communication quality. For performance comparison, the OLSR, BATMAN and RLRP protocols are used. Experimental results show that the e-RLRP reduces network overhead compared to RLRP, and overcomes in most cases all of these protocols, considering both network parameters and VoIP quality.