Finding Bugs Isn’t Enough: The Difference Between Pointing LLMs to Find Bugs and a Mature AI AppSec Platform

Bug discovery has become nearly commoditized by AI but that has actually increased the need for a platform to go from discovery to validation to remediation

Hector Leano

Sep 02, 2026

Finding Bugs Isn’t Enough: The Difference Between Pointing LLMs to Find Bugs and a Mature AI AppSec Platform

At the time our results last year at AIxCC and Wiz’s ZeroDay Cloud competitions were a big deal because it demonstrated for the first time that LLMs could be used to find complex, logic vulnerabilities with the same quality as an experienced human pentester.

Since the launch of Claude Code Security in late February and then Mythos in April, bug discovery has become nearly commoditized. OpenAI’s latest agents breaking out of their sand boxes and hacking external applications such as Hugging Face demonstrated AI agents can adapt attack behaviors like a human but at a speed and depth no human can match.

Our customers have shared they are drowning in true positives from what they get from Xint, their own bug bounty programs, and their early access to state of the art models like Mythos because AI has become so good at finding bugs.

So where does that leave pure play platforms like Xint?

We’ve found that this work by the frontier labs has actually increased the demand for our mature AppSec platform. Why? Because flagging possible bugs is not the hardest problem product security actually requires. For example:

Discovery and Bug Quality: We find the important, critical bugs that attackers care about, such as the most recent OS updates for Apple and Android that require no user interaction for an attacker to essentially get admin-level control of their device. Or Copy Fail, which went viral as the most severe Linux threat in years impacting all versions of Linux over the last ten years. In theory any experienced pentester could have found these bugs but the limiting factor has always been the time and cost to test an entire codebase or live application in depth. AI has changed it so now attackers can probe every corner of an application looking for these basic security failures.
Validation and False Positives: Because of our practical experience in real world security, we knew the biggest issue facing teams was using code scanning tools that usually generated more noise than signal - often with false positives and duplicates in the 80%+ range. From the public disclosures at frontier labs, we know they often bring in teams to sift through the findings in order to manually validate (though the latest models are a step change in generating exploits to help with validation). 100% of the critical bugs we have submitted to project maintainers for applications trusted by billions, including Apple, Android, Linux, PostgreSQL, MariaDB and more have been accepted. For all findings as a whole, we are below 25% false positives and even those are often valid findings that don’t apply at this moment but could in the future.
Safety: The Hugging Face hack by OpenAI agents underscores something we have been working on since last year which is how to safely deploy autonomous agents to test live applications without causing harm or crashing the applications. We still find the same vulnerabilities but we know what guardrails to put in place so our agents don’t overstep. This is one of the biggest gaps we are seeing with the whole “point an LLM with minimal harnessing at an app and see what happens” approach
Predictable cost: As frontier labs have moved to token-based pricing, this has caused the biggest shift in security team behavior. With Xint our customers know exactly what their costs will be even as their applications become larger and more complex. This includes not just bug discovery, but also the larger triage effort to validate and patch.
Remediation: Timely patching was time consuming even before ProdSec teams started drowning in true positives coming from AppSec tools like Xint, their own bug bounty programs, and even early access to frontier cyber models. We’ve designed our report outputs to accelerate safe remediation while still maintaining best practices for humans in the loop before publishing any changes live.
Data management: When you interact with an LLM API without a ZDR agreement, the provider's default behavior often includes logging your inputs and outputs — sometimes for days, sometimes indefinitely.
Under a ZDR policy, the provider contractually commits that your prompts and completions will not be stored, logged, or used beyond serving the immediate request. At Xint, we store your source code only while analyses are being run, with a fail-safe deletion mechanism after 2 days of storage, and we have Zero Data Retention agreements in place with our model API providers to keep our clients’ data safe.

FAQ Product

Finding Bugs Isn’t Enough: The Difference Between Pointing LLMs to Find Bugs and a Mature AI AppSec Platform

Bug discovery has become nearly commoditized by AI but that has actually increased the need for a platform to go from discovery to validation to remediation

Hector Leano

Sep 02, 2026

So where does that leave pure play platforms like Xint?

Discovery and Bug Quality: We find the important, critical bugs that attackers care about, such as the most recent OS updates for Apple and Android that require no user interaction for an attacker to essentially get admin-level control of their device. Or Copy Fail, which went viral as the most severe Linux threat in years impacting all versions of Linux over the last ten years. In theory any experienced pentester could have found these bugs but the limiting factor has always been the time and cost to test an entire codebase or live application in depth. AI has changed it so now attackers can probe every corner of an application looking for these basic security failures.
Validation and False Positives: Because of our practical experience in real world security, we knew the biggest issue facing teams was using code scanning tools that usually generated more noise than signal - often with false positives and duplicates in the 80%+ range. From the public disclosures at frontier labs, we know they often bring in teams to sift through the findings in order to manually validate (though the latest models are a step change in generating exploits to help with validation). 100% of the critical bugs we have submitted to project maintainers for applications trusted by billions, including Apple, Android, Linux, PostgreSQL, MariaDB and more have been accepted. For all findings as a whole, we are below 25% false positives and even those are often valid findings that don’t apply at this moment but could in the future.
Safety: The Hugging Face hack by OpenAI agents underscores something we have been working on since last year which is how to safely deploy autonomous agents to test live applications without causing harm or crashing the applications. We still find the same vulnerabilities but we know what guardrails to put in place so our agents don’t overstep. This is one of the biggest gaps we are seeing with the whole “point an LLM with minimal harnessing at an app and see what happens” approach
Predictable cost: As frontier labs have moved to token-based pricing, this has caused the biggest shift in security team behavior. With Xint our customers know exactly what their costs will be even as their applications become larger and more complex. This includes not just bug discovery, but also the larger triage effort to validate and patch.
Remediation: Timely patching was time consuming even before ProdSec teams started drowning in true positives coming from AppSec tools like Xint, their own bug bounty programs, and even early access to frontier cyber models. We’ve designed our report outputs to accelerate safe remediation while still maintaining best practices for humans in the loop before publishing any changes live.
Data management: When you interact with an LLM API without a ZDR agreement, the provider's default behavior often includes logging your inputs and outputs — sometimes for days, sometimes indefinitely.
Under a ZDR policy, the provider contractually commits that your prompts and completions will not be stored, logged, or used beyond serving the immediate request. At Xint, we store your source code only while analyses are being run, with a fail-safe deletion mechanism after 2 days of storage, and we have Zero Data Retention agreements in place with our model API providers to keep our clients’ data safe.