Adversarial challenges and defenses in ml-driven cybersecurity systems

mso-ansi-language:EN-US">Machine learning has rapidly become a key enabler of modern cybersecurity, offering scalable and adaptive solutions to detect and mitigate evolving threats. Yet, unlike traditional domains, security applications operate in adversarial environments, where attackers actively evade and exploit weaknesses in defensive systems. This raises fundamental challenges around robustness to adversarial manipulation, generalization to unseen threats, as well as trustworthiness of cybersecurity systems in practice.  mso-ansi-language:EN-US">This thesis addresses these challenges through contributions spanning different security domains, including phishing webpage detection and software supply-chain security. We examine the robustness of threat detection systems based on machine-learning to adversarial manipulations, demonstrating how realistic and efficient attacks can evade phishing and malicious package detectors. We then comprehensively evaluate defenses based on adversarial training, revealing their effectiveness and limitations in practical settings. Furthermore, we investigate the role of feature representations in enhancing generalization and robustness to concept drift in the underexplored context of macOS malware detection, showing how domain-specific features can significantly improve detection accuracy and resilience to novel threats. Finally, we broaden the scope to software supply-chain security by evaluating trust metrics for open-source ecosystems through a threat modelling inspired to adversarial machine learning, showing their limitations and vulnerabilities to adversarial manipulation. Overall, this thesis highlights both the opportunities and pitfalls of applying machine learning to cybersecurity. By exposing weaknesses and proposing improved adversary-aware methodologies, it contributes to building more robust and trustworthy defenses against evolving adversarial threats.