The Cyber Show Explores Measuring Cybersecurity Effectiveness and the Role of AI

This episode of The Cyber Show explores the challenges and innovations in measuring cybersecurity effectiveness, particularly through the lens of a company called Secor (pronounced 'Secor' in Norway). The discussion centers on whether security can be quantified, how automation and AI might assist in compliance and risk management, and the practical applications of such tools for organizations. The conversation also touches on the ethical use of AI in cybersecurity, the difficulties of comparing security products, and the importance of balancing automation with human oversight. One of the core topics is the concept of measuring security quantitatively. The guests, Dr. Basil and Ryan Maruga from Secor, argue that security can indeed be measured by evaluating two key aspects: controls (which increase confidence in security) and vulnerabilities (which decrease it). Controls refer to security measures like firewalls, encryption, or access policies that help protect systems, while vulnerabilities are weaknesses that attackers could exploit. The company’s approach involves assigning weights to controls and risks to vulnerabilities, creating a matrix that calculates a security assurance score between 0 and 10. This method allows organizations to compare different systems or products objectively, rather than relying on subjective assessments or vendor pitches. The idea is rooted in earlier software engineering metrics, where code complexity and test coverage were used to gauge quality. However, security metrics are far more complex because they must account for dynamic threats, evolving standards, and organizational context. The practical implication is that CISOs and security teams can use such tools to prioritize investments, identify gaps, and justify security spending to executives or regulators. Another major theme is the role of AI and automation in security compliance. The guests clarify that while AI is often overhyped, it can play a valuable role in processing large volumes of data, such as security policies, risk assessments, and compliance reports, to generate recommendations. For example, AI can analyze documents to suggest relevant security controls or help design test plans to verify whether a system meets specific standards. However, the guests emphasize that AI should not replace human judgment. Instead, it should act as an assistant, reducing the manual effort required for tasks like reading through lengthy compliance frameworks or identifying overlapping requirements across multiple standards. Secor’s platform, for instance, uses AI to automate parts of the security evaluation process, such as generating mitigation plans or calculating the impact of addressing specific vulnerabilities. This automation is particularly beneficial for small and medium-sized enterprises (SMEs), which may lack the resources to hire dedicated compliance teams. The episode also highlights the risk of over-reliance on AI, noting that some companies claim to offer fully AI-driven security solutions, which may not be feasible or safe given the current limitations of the technology. The discussion also delves into the challenges of comparing security products and the limitations of traditional sales methods. The guests point out that CISOs often rely on vendor relationships or subjective demonstrations when selecting security tools, which can lead to suboptimal choices. Secor’s platform aims to address this by providing a standardized way to evaluate and compare products based on measurable security metrics. For example, the tool can assess how well a product meets specific compliance requirements or how it performs against known vulnerabilities. This is particularly useful in industries with multiple overlapping standards, such as healthcare (which must comply with GDPR, ISO 27001, and national regulations) or critical infrastructure (which may have sector-specific requirements). The platform allows users to combine different standards into a single evaluation, identifying contradictions or redundancies. For instance, one standard might require an 8-character password, while another mandates 15 characters. The tool helps organizations navigate these conflicts by highlighting the most stringent requirements and suggesting cost-effective ways to improve their security posture. This approach not only saves time but also helps organizations allocate their security budgets more efficiently. A critical concern raised in the episode is the security and privacy of the data collected by such platforms. Since tools like Secor’s require organizations to input sensitive information about their systems, networks, and vulnerabilities, they become attractive targets for attackers. The guests acknowledge this risk and explain that Secor follows industry-standard security practices, such as secure storage, multi-factor authentication, and access controls, to protect customer data. However, they admit that the platform does not currently use advanced privacy-preserving techniques like zero-knowledge proofs or homomorphic encryption, which would allow data to be processed without exposing it to the service provider. The hosts and guests agree that data minimization—collecting and retaining only what is necessary—is a key principle for mitigating this risk. The episode also touches on the broader tension between software-as-a-service (SaaS) models and the desire for local processing, as some organizations may prefer to run security tools on their own infrastructure to maintain control over their data. This highlights a growing demand for transparency and flexibility in security tools, particularly among organizations with strict compliance or privacy requirements. Finally, the episode addresses the rapid pace of change in the cybersecurity landscape and how tools like Secor’s can adapt. The guests note that the threat environment, regulatory frameworks, and geopolitical risks are constantly evolving, making it difficult for organizations to stay compliant and secure. Secor’s platform is designed to be flexible, allowing users to add new standards or update their security profiles as needed. For example, the tool can incorporate emerging frameworks like NIST’s guidelines or sector-specific regulations, ensuring that evaluations remain relevant. The guests also emphasize the importance of user education, as even the most advanced tools are only effective if they are used correctly. Secor provides tutorials and consultations to help users understand how to tailor the platform to their needs, whether they are CISOs making strategic decisions or engineers implementing specific controls. The episode concludes by reinforcing the idea that while automation and AI can significantly improve security management, they are not a substitute for human expertise. Instead, they should be seen as tools that augment decision-making, reduce manual effort, and help organizations navigate the complexities of modern cybersecurity.