AI in Vulnerability Discovery: A Call for Human Oversight and Caution

Key Takeaways

• AI tools used in vulnerability discovery are generating a high volume of false positives and 'AI slop' bug reports.
• The influx of unverified AI-generated reports is overwhelming bug bounty programs, leading to shutdowns like the curl utility's program.
• Human oversight and manual validation are critical to ensure the accuracy of AI-generated vulnerability reports.
• Inexperienced individuals are using AI to spam bug bounty programs for financial gain, degrading trust in third-party research.

Affected Systems

• Bug Bounty Programs
• CVE Database
• Vulnerability Triage Workflows

Attack Chain

Inexperienced individuals or researchers use AI tools like Claude Code to automatically analyze codebases for vulnerabilities. The AI generates reports based on pattern matching, often misinterpreting benign code (e.g., integer inputs) as exploitable flaws like OS Command Injection. These unverified, false-positive reports are submitted en masse to bug bounty programs or CVE databases. This floods the system, overwhelming maintainers, wasting triage resources, and degrading the overall vulnerability management process.

Detection Availability

• YARA Rules: No
• Sigma Rules: No
• Snort/Suricata Rules: No
• KQL Queries: No
• Splunk SPL Queries: No
• EQL Queries: No
• Other Detection Logic: No

No detection rules are provided as this article discusses procedural and operational challenges with AI-generated vulnerability reports rather than a cyber attack.

Detection Engineering Assessment

EDR Visibility: None — This is a procedural issue regarding vulnerability reporting, not an endpoint attack. Network Visibility: None — No malicious network traffic is associated with this reporting issue. Detection Difficulty: N/A — This is not a detectable cyber attack, but rather an abuse of reporting mechanisms.

Required Log Sources

• Web Application Firewall (WAF)
• Ticketing System Logs

Hunting Hypotheses

Hypothesis	Telemetry	ATT&CK Stage	FP Risk
Monitor bug bounty submission portals for high-frequency submissions from single users or IP addresses, which may indicate automated AI-driven reporting.	Web Application Firewall (WAF) or Application Logs	Reconnaissance	High

Control Gaps

• Bug bounty submission filtering
• Automated vulnerability validation

Key Behavioral Indicators

• High volume of bug bounty submissions from a single source
• Submissions lacking functional proof-of-concept (PoC) exploits
• Reports containing common LLM phrasing or hallucinated code paths

False Positive Assessment

• High

Recommendations

Immediate Mitigation

• Implement rate limiting on bug bounty submission portals.
• Require functional Proof of Concept (PoC) exploits for all vulnerability submissions to filter out AI hallucinations.

Infrastructure Hardening

• N/A

User Protection

• N/A

Security Awareness

• Train triage teams to identify common hallmarks of AI-generated false positives.
• Establish strict guidelines for researchers regarding the responsible use of AI in vulnerability discovery.