Attention-Based Recurrency for Multi-Agent Reinforcement Learning under State Uncertainty
Thomy Phan, Fabian Ritz, Jonas Nüßlein, Michael Kölle, Thomas Gabor and Claudia Linnhoff-Popien
Abstract: State uncertainty poses a major challenge for decentralized coordination. However, state uncertainty is largely neglected in multi-agent reinforcement learning research due to a strong focus on state-based centralized training for decentralized execution (CTDE) and benchmarks that lack sufficient stochasticity like StarCraft Multi-Agent Challenge (SMAC). In this work, we propose Attention-based Embeddings of Recurrence In multi-Agent Learning (AERIAL) to approximate value functions under agent-wise state uncertainty. AERIAL uses a learned representation of multi-agent recurrence, considering more accurate information about decentralized agent decisions than state-based CTDE. We then introduce MessySMAC, a modified version of SMAC with stochastic observations and higher variance in initial states, to provide a more general and configurable benchmark. We evaluate AERIAL in a variety of MessySMAC maps, and compare the results with state-based CTDE.
Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pp. 2839-2841 (2023)
Cite This Work
Switch between citation styles and copy the format you need.
Citation
Thomy Phan, Fabian Ritz, Jonas Nüßlein, Michael Kölle, Thomas Gabor, and Claudia Linnhoff-Popien. “Attention-Based Recurrency for Multi-Agent Reinforcement Learning under State Uncertainty.” Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems , pp. 2839-2841 , 2023. https://doi.org/10.5555/3545946.3599096