ABSTRACT. This study focuses on the development and optimization of algorithms to enhance network security and traffic management in information and communication systems. Advanced methods were designed to adaptively modify rule actions based on traffic behavior, reducing risks in network packets and improving security responses. The research emphasizes minimizing vulnerabilities by implementing a dynamic rule-prioritization mechanism that ensures optimal decision-making when accepting or rejecting network packets. Key contributions include the creation of algorithms that effectively identify and mitigate harmful biases in network traffic, enhancing workstation security. Additionally, strategies for optimizing traffic filtering in Software-Defined Networks (SDN) were explored to address evolving network threats. Methods for detecting and managing anomalies in network packets, as well as tools for securing packet headers through the use of inter-network screens, were also developed. The outcomes align with global advancements, incorporating principles from research institutions worldwide, such as cognitive inter-network screens, Next-Generation Firewalls, and Honeypot systems. This work significantly contributes to the development of efficient packet filtering rules and protective measures, ultimately reducing the likelihood of phishing attacks and other network-based threats. These advancements hold promise for bolstering the resilience of modern information systems against increasingly sophisticated cyberattacks.