On-Demand

Ultimate Web Hacking | Blackhat Briefing Series

The conventional metrics for measuring autonomous pentesting are exploitability (rewarding a system for constructing a working exploit) and coverage (finding as many bug types as possible). But this does not reflect real world resource-constrained product security where you need to look at the cost and human triage effort required to validate, prioritize, and remediate bugs.

Watch On Demand

You’re measuring autonomous pentesting agents incorrectly, which means they’ll fail in the real world

The conventional metrics for measuring autonomous pentesting are exploitability (rewarding a system for constructing a working exploit) and coverage (finding as many bug types as possible). But this does not reflect real world resource-constrained product security where you need to look at the cost and human triage effort required to validate, prioritize, and remediate bugs.

In this webinar, Xint.io researcher Quoc Tran breaks down how to measure operational efficiency through the decisioning inputs that matter to product security teams in practice:  recall per dollar and per hour, triage load, and the marginal value of ensembling. He then demonstrates how time and cost intensity varies using commercially available frontier LLMs with light harnessing as well as through more mature harnessing structures. This includes a deep dive into the incremental cost to find each marginal bug and its implication to how product security teams should weigh those costs against the capabilities of real attackers.

Speakers

Meet the experts behind the conversation.

Quoc Tran
Senior Software Engineer, Xint
NGINX2Root

nginx 치명적 취약점(Critical) 긴급 안내 영향받는 모든 서버를 지금 바로 패치하세요.

nginxの深刻な脆弱性に関する緊急警告 今すぐすべての対象サーバーにパッチを適用してください。

Critical nginx vulnerability alert. Patch every affected server now.

패치 가이드 対処方法を見る Get the fix