Smaller Models Can Be More Cost-Efficient for Vulnerability Research

For vulnerability research, smaller models run repeatedly can outperform larger frontier models on cost-to-recall. The post presents benchmark findings comparing the cost and recall rates of different models in identifying real, recent zero-day vulnerabilities. It notes that while a large model may have a higher probability of finding a zero-day (e.g., 90%), a smaller model with a lower success rate (e.g., 50%) but significantly lower cost (e.g., 10x cheaper) can be more efficient. The research was conducted by Hacktron, the company behind the study. The results aim to help the community make informed decisions about model selection.