EXPLAINABLE AI IN CYBERSECURITY: A COMPREHENSIVE REVIEW OF XAI TECHNIQUES FOR MALWARE DETECTION AND THREAT ANALYSIS

Authors: Er. Kritika
Affiliation: Independent Researcher

Category:

Keywords: Explainable Artificial Intelligence (XAI), Machine Learning Interpretability, Threat Detection, Network security analytics, Adversarial Machine Learning
ABSTRACT. The growing use of artificial intelligence in cybersecurity has increased the necessity of transparency, interpretability, and accountability. The Explainable AI (XAI) provides a way to mitigate those issues, but it has not been fully developed or implemented in the security fields yet. The paper presents a systematic comparative synthesis and critical analysis of 50 articles on malware detection, intrusion detection systems, hybrid XAI models, and regulatory or ethical frameworks. These studies are assessed using a methodological framework on the aspects of technical rigor, human-centered design, and governance alignment. The findings indicate that there is a strong dependence on post-hoc methods like SHAP and LIME, which, despite their flexibility, do not necessarily have fidelity, scalability, and adversarial robustness. Intrinsic and hybrid models are more promising as far as interpretability can be embedded without reducing accuracy, but can only be validated on fixed benchmark datasets, which is limiting to real-world applicability. A significant gap in human-centered assessment is also indicated by the review: not many studies determine the effect of explanations on the trust calibration, cognitive load, or operational performance of analysts. Regulatory and ethical aspects are also largely under-researched, and the majority of implementations do not respond to the needs of compliance with standards like the GDPR and EU AI Act. Taken together, these results indicate the disjointed nature of XAI in the field of cybersecurity and the dire necessity of combined frameworks that can synchronize technical innovation with human usability and compliance with governance.

References:

Ozkan-Okay, Merve, Erdal Akin, Ömer Aslan, Selahattin Kosunalp, Teodor Iliev, Ivaylo Stoyanov, and Ivan Beloev. “A Comprehensive Survey: Evaluating the Efficiency of Artificial Intelligence and Machine Learning Techniques on Cyber Security Solutions.” IEEE Access 12 (2024): 12229–12256.
De Azambuja, André J. G., Christian Plesker, Klaus Schützer, Reiner Anderl, Benjamin Schleich, and Vinicius R. Almeida. “Artificial Intelligence-Based Cyber Security in the Context of Industry 4.0—A Survey.” Electronics 12, no. 8 (2023): 1920.
Patil, Dnyaneshwar. “Explainable Artificial Intelligence (XAI) for Industry Applications: Enhancing Transparency, Trust, and Informed Decision-Making in Business Operation.” (2024).
Zhang, Zhiyuan, Hamed Al Hamadi, Ernesto Damiani, Chan Yeob Yeun, and Fadi Taher. “Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research.” IEEE Access 10 (2022): 93104–93139.
Cesarini, Marco, Luca Malandri, Francesco Pallucchini, Alessandro Seveso, and Fang Xing. “Explainable AI for Text Classification: Lessons from a Comprehensive Evaluation of Post Hoc Methods.” Cognitive Computation 16, no. 6 (2024): 3077–3095.
Knap, Martin. “Model-Agnostic XAI Models: Benefits, Limitations and Research Directions.” (2024).
Roshan, Kamran, and Asif Zafar. “Utilizing XAI Technique to Improve Autoencoder Based Model for Computer Network Anomaly Detection with Shapley Additive Explanation (SHAP).” arXiv preprint arXiv:2112.08442 (2021).
Abhishek, K., and Dinesh Kamath. “Attribution-Based XAI Methods in Computer Vision: A Review.” arXiv preprint arXiv:2211.14736 (2022).
Gupta, Chaitanya, Ishita Johri, Kannan Srinivasan, Yu-Chen Hu, Saad M. Qaisar, and Kai-Yu Huang. “A Systematic Review on Machine Learning and Deep Learning Models for Electronic Information Security in Mobile Networks.” Sensors 22, no. 5 (2022): 2017.
Giri, A. L., and S. Annamalai. “Intrusion Detection System for Local Networks—A Review Study.” In Proceedings of the 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), 1388–1393. IEEE, 2022.
Zschech, Patrick, Simon Weinzierl, Nico Hambauer, Sebastian Zilker, and Matthias Kraus. “GAM(e) Changer or Not? An Evaluation of Interpretable Machine Learning Models Based on Additive Model Constraints.” arXiv preprint arXiv:2204.09123 (2022).
Kalutharage, Chamika S., Xin Liu, Christos Chrysoulas, and Olufemi Bamgboye. “Neurosymbolic Learning in the XAI Framework for Enhanced Cyberattack Detection with Expert Knowledge Integration.” In IFIP International Conference on ICT Systems Security and Privacy Protection, 236–249. Cham: Springer Nature Switzerland, 2024.
Abualhaj, Mohammad M., Ahmad S. Al-Shamayleh, Ahmad Munther, Saeed N. Alkhatib, Mohammad O. Hiari, and Muamer Anbar. “Enhancing Spyware Detection by Utilizing Decision Trees with Hyperparameter Optimization.” Bulletin of Electrical Engineering and Informatics 13, no. 5 (2024): 3653–3662
Manthena, Harsha, Soroush Shajarian, Jonathan Kimmell, Mohamed Abdelsalam, Saeid Khorsandroo, and Manish Gupta. “Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challenges.” IEEE Access (2025).
Saqib, Muhammad, Saeed Mahdavifar, Benjamin C. M. Fung, and Patrick Charland. “A Comprehensive Analysis of Explainable AI for Malware Hunting.” ACM Computing Surveys 56, no. 12 (2024): 1–40.
Manthena, Harsha, Soroush Shajarian, Jonathan Kimmell, Mohamed Abdelsalam, Saeid Khorsandroo, and Manish Gupta. “Explainable Malware Analysis: Concepts, Approaches and Challenges.” arXiv preprint (2024).
Ma, Kyoung P., Dong J. Ryu, and Seung J. Lee. “Reverse Analysis Method and Process for Improving Malware Detection Based on XAI Model.” Computers, Materials & Continua 81, no. 3 (2024).
Adhikari, Dipesh, and Suresh Thapaliya. “Explainable AI for Cyber Security: Interpretable Models for Malware Analysis and Network Intrusion Detection.” NPRC Journal of Multidisciplinary Research 1, no. 9 (2024): 170–179.
Alvi, Muhammad A., and Zubair Jalil. “XRGuard: A Model-Agnostic Approach to Ransomware Detection Using Dynamic Analysis and Explainable AI.” IEEE Access (2025).
Ch, Ramesh, J. Manoranjini, S. Pallavi, U. Naresh, S. Telang, and S. Kiran. “Advancing Malware Detection Using Memory Analysis and Explainable AI Approach.” In Proceedings of the 2024 Second International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI), 518–523. IEEE, 2024
Hafiz, Md. F. B., N. A. Khan, Z. Kamal, S. Hossain, and S. Barman. “A Robust Malware Classification Approach Leveraging Explainable AI.” In Proceedings of the 2024 International Conference on Intelligent Systems for Cybersecurity (ISCS), 1–6. IEEE, 2024.
Manthena, Harsha. Explainable Machine Learning Based Malware Analysis. Master’s thesis, North Carolina Agricultural and Technical State University, 2022.
Pradhan, Umesh, K. M. Navaneeth, and M. N. Aditya. “Malware Analysis Using Hashing and Explainable AI: A Comparative Study of LIME and SHAP Techniques.” In Proceedings of the 2025 1st International Conference on AIML—Applications for Engineering & Technology (ICAET), 1–6. IEEE, 2025.
Arreche, Omar, Tejas Guntur, and Mahmoud Abdallah. “XAI-IDS: Toward Proposing an Explainable Artificial Intelligence Framework for Enhancing Network Intrusion Detection Systems.” Applied Sciences 14, no. 10 (2024): 4170.
Mahbooba, Bibi, Milan Timilsina, Rania Sahal, and Manuel Serrano. “Explainable Artificial Intelligence (XAI) to Enhance Trust Management in Intrusion Detection Systems Using Decision Tree Model.” Complexity 2021 (2021): 6634811
Sivamohan, S., and S. S. Sridhar. “An Optimized Model for Network Intrusion Detection Systems in Industry 4.0 Using XAI-Based Bi-LSTM Framework.” Neural Computing and Applications 35, no. 15 (2023): 11459–11475.
Arreche, Omar, Tejas R. Guntur, Jason W. Roberts, and Mahmoud Abdallah. “E-XAI: Evaluating Black-Box Explainable AI Frameworks for Network Intrusion Detection.” IEEE Access 12 (2024): 23954–23988
Shand, Christopher, Ruth Fong, and Umar Butt. “How Explainable Artificial Intelligence (XAI) Models Can Be Used Within Intrusion Detection Systems (IDS) to Enhance an Analyst’s Trust and Understanding.” In International Conference on Global Security, Safety, and Sustainability, 321–342. Cham: Springer Nature Switzerland, 2023
Patil, S., V. Varadarajan, S. M. Mazhar, A. Sahibzada, N. Ahmed, O. Sinha, and K. Kotecha. “Explainable Artificial Intelligence for Intrusion Detection System.” Electronics 11, no. 19 (2022): 3079
Wali, Shayan, Y. A. Farrukh, and Imran Khan. “Explainable AI and Random Forest Based Reliable Intrusion Detection System.” Computers & Security (2025): 104542.
Barnard, Peter, Nicola Marchetti, and Luiz A. DaSilva. “Robust Network Intrusion Detection through Explainable Artificial Intelligence (XAI).” IEEE Networking Letters 4, no. 3 (2022): 167–171.
Upadhyay, Ujjwal, Ashish Kumar, S. Roy, U. Rawat, and S. Chaurasia. “Defending the Cloud: Understanding the Role of Explainable AI in Intrusion Detection Systems.” In Proceedings of the 2023 16th International Conference on Security of Information and Networks (SIN), 1–9. IEEE, 2023.
Mohale, Victor Z., and Isaac C. Obagbuwa. “Evaluating Machine Learning-Based Intrusion Detection Systems with Explainable AI: Enhancing Transparency and Interpretability.” Frontiers in Computer Science 7 (2025): 1520741
Mohammed, S. J., and B. M. Nema. “Threat Detection Based on Explainable AI (XAI) and Hybrid Learning.” Mesopotamian Journal of CyberSecurity 5, no. 2 (2025): 477–490
Kuppa, Anil, and Nhien-An Le-Khac. “Black Box Attacks on Explainable Artificial Intelligence (XAI) Methods in Cyber Security.” In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), 1–8. IEEE, 2020
Kuppa, Anil, and Nhien-An Le-Khac. “Adversarial XAI Methods in Cybersecurity.” IEEE Transactions on Information Forensics and Security 16 (2021): 4924–4938.
Garikapati, Harsha, K. Challapalli, S. V. Ramineni, V. M. S. Adusumilli, R. S. C. Kothamasu, and S. Anamalamudi. “An Explainable AI-Driven Hybrid Model for Enhanced Intrusion Detection in Network Security.” In Proceedings of the 2025 Fifth International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), 1–7. IEEE, 2025.
Taufik, Muhammad, M. S. Aziz, and A. Fitriana. “Hybrid Explainable AI (XAI) Framework for Detecting Adversarial Attacks in Cyber-Physical Systems.” Journal of Technology Informatics and Engineering 4, no. 1 (2025).
Kalutharage, Chamika S., Xin Liu, Christos Chrysoulas, and Olufemi Bamgboye. “Neurosymbolic Learning in the XAI Framework for Enhanced Cyberattack Detection with Expert Knowledge Integration.” In IFIP International Conference on ICT Systems Security and Privacy Protection, 236–249. Cham: Springer Nature Switzerland, 2024
Rahmati, Mahdi. “Towards Explainable and Lightweight AI for Real-Time Cyber Threat Hunting in Edge Networks.” arXiv preprint arXiv:2504.16118 (2025).
Gwassi, Oussama A. H., O. N. Uçan, and Esteban A. Navarro. “Cyber-XAI-Block: An End-to-End Cyber Threat Detection and FL-Based Risk Assessment Framework for IoT-Enabled Smart Organizations Using XAI and Blockchain Technologies.” Multimedia Tools and Applications (2024): 1–42.
Kumar, B. G., and E. Shanthini. “From Attacks to Insights: XAI and Defense in Network Security.” In Proceedings of the 2024 International Conference on Sustainable Communication Networks and Application (ICSCNA), 395–401. IEEE, 2024
Hemalatha, A., V. Kumar, F. T. Graf, P. Pavithra, and R. Suresh. “A Hybrid Intrusion Detection System Using Explainable AI for Enhanced Accuracy and Transparency.” In Proceedings of the 2025 International Conference on Electronics and Renewable Systems (ICEARS), 923–929. IEEE, 2025
Moorthy, U. M. K., A. M. J. Muthukumaran, V. Kaliyaperumal, S. Jayakumar, and K. A. Vijayaraghavan. “Explainability and Regulatory Compliance in Healthcare: Bridging the Gap for Ethical XAI Implementation.” In Explainable Artificial Intelligence in the Healthcare Industry, 521–561. 2025.
Ristovska, Tanja, Georgi Gospodinov, Lyubomir Gotsev, Stefka Syarova, and Velina Angelova. “A Review on AI in Cybersecurity: Ethical Challenges and Regulatory Frameworks.” In Environment. Technology. Resources., vol. 2, 285–291. 2025.
Alabdulatif, Abdullah. “A Novel Ensemble of Deep Learning Approach for Cybersecurity Intrusion Detection with Explainable Artificial Intelligence.” Applied Sciences 15, no. 14 (2025): 7984.
Khayat, Mohammad, El Houssaine Barka, Mohamed A. Serhani, Farid Sallabi, Khaled Shuaib, and Hanan M. Khater. “Empowering Security Operation Center with Artificial Intelligence and Machine Learning—A Systematic Literature Review.” IEEE Access (2025).
Senevirathna, Thilina, Van-Hieu La, S. Marcha, B. Siniarski, Madhusanka Liyanage, and Shuo Wang. “A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions.” IEEE Communications Surveys & Tutorials 27, no. 2 (2024): 941–973.
Herrera, Francisco. “Making Sense of the Unsensible: Reflection, Survey, and Challenges for XAI in Large Language Models Toward Human-Centered AI.” arXiv preprint arXiv:2505.20305 (2025).
Chinnaraju, Anand. “Explainable AI (XAI) for Trustworthy and Transparent Decision-Making: A Theoretical Framework for AI Interpretability.” World Journal of Advanced Engineering Technology and Sciences 14, no. 3 (2025): 170–207.
Powell, William. The CISO 3.0: A Guide to Next-Generation Cybersecurity Leadership. Boca Raton, FL: CRC Press, 2025
Dotan, Roy, Benjamin Blili-Hamelin, Raghuram Madhavan, Justin Matthews, and Joseph Scarpino. “Evolving AI Risk Management: A Maturity Model Based on the NIST AI Risk Management Framework.” arXiv preprint arXiv:2401.15229 (2024).
Xiong, Min, Hong Wang, Chao Che, and Rui Lin. “Toward Safer Aviation: Application of GA-XGBoost-SHAP for Incident Cognition and Model Explainability.” Journal of Risk and Reliability 238, no. 6 (2024): 1195–1208
Pissanidis, Dimitrios L., and Konstantinos Demertzis. “Integrating AI/ML in Cybersecurity: An Analysis of Open XDR Technology and Its Application in Intrusion Detection and System Log Management.” (2024).
Kritika, E. “A Neuroscience Perspective on AI and Cybersecurity.” ISACA Journal, no. 1 (2025).