A Reinforcement Learning-Based Multi-Agent System for Advanced Network Attack Prediction

Abstract

This paper addresses the challenges of traditional Network Intrusion Detection Systems (NIDS) in handling the increasing complexity and volume of modern cyberattacks. The authors suggest a novel multi-agent deep reinforcement learning (MADRL) approach, employing a deep Q-network (DQN) architecture with convolutional and fully connected layers. This architecture incorporates Target networks and Experience Replay to enhance learning and adaptation. A hierarchical reinforcement learning strategy decomposes complex intrusion detection tasks into manageable subtasks, enabling efficient exploration of high-dimensional state-action spaces. The proposed model, trained and evaluated on the CICIDS2017 dataset using a 70% training set and 30% test split and 10-fold cross-validation, achieves exceptional performance. It attains 97.71% accuracy, 98.34% recall, 97.29% precision, and 96.76% F1-score after 50 iterations, surpassing existing NIDS solutions in comparative analysis. The model's strength lies in its ability to effectively mimic environmental characteristics through multi-agent learning, leading to robust detection of intricate attack patterns. Furthermore, our approach demonstrates strong generalization capabilities on unseen data, indicating its potential for real-world deployment. This research contributes significantly to the evolution of intelligent network security systems by introducing an innovative MADRL framework. Future research directions include implementing the solution in real-time network environments, expanding the agent network, and extending the model's application to outlier detection and software-defined networking. This work lays the foundation for future advancements in cyber threat detection and mitigation, paving the way for more robust and adaptive network security solutions.