This paper addresses the challenges of traditional Network Intrusion Detection Systems (NIDS) in handling the increasing complexity and volume of modern cyberattacks. The authors suggest a novel multi-agent deep reinforcement learning (MADRL) approach, employing a deep Q-network (DQN) architecture with convolutional and fully connected layers. This architecture incorporates Target networks and Experience Replay to enhance learning and adaptation. A hierarchical reinforcement learning strategy decomposes complex intrusion detection tasks into manageable subtasks, enabling efficient exploration of high-dimensional state-action spaces. The proposed model, trained and evaluated on the CICIDS2017 dataset using a 70% training set and 30% test split and 10-fold cross-validation, achieves exceptional performance. It attains 97.71% accuracy, 98.34% recall, 97.29% precision, and 96.76% F1-score after 50 iterations, surpassing existing NIDS solutions in comparative analysis. The model's strength lies in its ability to effectively mimic environmental characteristics through multi-agent learning, leading to robust detection of intricate attack patterns. Furthermore, our approach demonstrates strong generalization capabilities on unseen data, indicating its potential for real-world deployment. This research contributes significantly to the evolution of intelligent network security systems by introducing an innovative MADRL framework. Future research directions include implementing the solution in real-time network environments, expanding the agent network, and extending the model's application to outlier detection and software-defined networking. This work lays the foundation for future advancements in cyber threat detection and mitigation, paving the way for more robust and adaptive network security solutions.