Learning to Communicate in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence

Abstract

Multi Agent Reinforcement Learning (MARL) trains multiple Reinforcement Learning (RL) agents to either achieve a common goal or compete against each other. Popular methods in cooperative MARL with partially observable environments, only allow agents to act independently during execution which may limit the coordinated effect of the trained policies. However, by facilitating the sharing of critical information such as network topology, known or suspected threats, and event logs, effective communication can lead to a more informed decision-making in the cyber battle-space. While a game theoretic approach has shown success in real world applications, its applicability to cybersecurity is an active area of research. The aim of this thesis is to demonstrate the importance and effectiveness of communication between blue agents and to show that relaying key information will allow these agents to stop a malicious actor from compromising hosts across subnets. This thesis also hopes to contribute in the development of techniques that can enhance autonomous cyber defence on an enterprise network. The results demonstrate that through Differentiable Inter Agent Learning, the defender agents play sequential games in Cyber Operations Research Gym and learn to communicate to prevent imminent cyber threats. The tactical policies learned by the autonomous RL agents to achieve the coordination is akin to the human experts that communicate with each other during an incidence response to avert cyber threats.