A Comparative Study of Static and Decentralized Q-Learning Communication Policies in Dense Wireless IoT Networks

Abstract

The unprecedented density of modern Internet of Things (IoT) deployments has rendered conventional static communication policies fundamentally obsolete. In ultra-dense environments, rigid transmission parameters inevitably lead to severe MAC-layer contention, catastrophic latency spikes, and the "tragedy of the commons," where blind retransmissions deplete limited node energy without yielding successful data delivery. To overcome these limitations, this paper presents a fully decentralized Q-learning communication framework that transforms IoT nodes into autonomous, cognitive agents. Unlike existing centralized approaches that suffer from scalability bottlenecks, our framework enables each node to independently optimize its transmission power and interval using only locally observed states Round-Trip Time (RTT), packet loss, throughput, and residual energy guided by a carefully engineered multi-objective reward function and an experience replay mechanism. Evaluated extensively using the ns-3 simulator across scalable topologies (50 to 150 nodes, with a 100-node dense scenario as the primary baseline), the results demonstrate that the proposed framework drastically stabilizes network throughput and suppresses delay variance. Most notably, the study introduces and validates the concept of "Energy Neutrality" in AI-driven IoT; the marginal computational cost of decentralized learning is entirely offset by the significant RF energy saved by intelligently avoiding collision domains. Furthermore, microscopic analysis reveals a fascinating emergent behavior: without any centralized coordination, nodes naturally learn implicit cooperative back off strategies to reach a network-wide Nash Equilibrium. While a pragmatic 60% node convergence rate highlights the physical limits of purely decentralized learning in severely shadowed environments, the framework unequivocally proves that decentralized reinforcement learning is a viable, scalable, and energy-efficient paradigm for orchestrating next-generation, intelligent IoT ecosystems.