TL;DR

• Finding vulns in OSS in 2026, “thanks” to LLMs: easy, YES
• Being the first to report them: possible, BUT

The finding part has been commoditized. The real challenge is being first, and surviving triage. This is what the rest of this article is about.

Why would you do that in the first place?

I’ve been interested in cybersec for like… forever. My second program as a child was a dumb VB 3.0 piece of malware that filled up the Startup folder of Windows 95/98 with thousands of auto-executable netsend commands (if you don’t know what that was, you should take a look!).

But I never made it my trade, even though the thought occurred on various occasions. For the last few months, I’ve worked closer than usual with my colleagues from the cybersecurity team, and I finally decided to try out what I could do with LLMs in the cybersecurity space.

Full disclaimer: I’m NOT in any way a cybersecurity specialist. If you don’t know me, I’m just a platform engineer who knows about Kubernetes, virtualisation and observability. I’m also an “okay” golang and python dev.

What are you trying to achieve?

For the last 2 months, I’ve been hunting (on my free time) vulnerabilities in popular open source software I’ve been using daily for work. Some are covered by a bug bounty, some accept GitHub Security Advisories. Some are not answering at all.

Why OSS? Well, obviously, it being open source makes it easy for me to check the code when the LLM thinks something is off, and form my own opinion. The honest downside is that LLMs are also the main reason why OSS has been drowning in security reports for about a year now.

But more importantly: I work with these tools daily, so I already know HOW they should work, and what is NOT normal. That’s also something we’ll cover.

One thing before we start: everything described below was done on my own infrastructure: throwaway clusters, local installs, test VMs. Every bug bounty program has rules of engagement: no scanning third-party instances, no touching other people’s data, no denial of service on shared infra. Read them. If a finding requires breaking those rules to prove it, it’s not worth the report.

Ok ok, enough introduction: can LLMs find vulns for you even with no cybersec experience?

Short answer: yes. The real question is whether it’s worth it, and that mostly depends on whether you want to be the first or not, and WHEN you start.

Here is my own experience with vulnerability hunting on OSS. “YMMV” as they say.

You’re late to the party

The first thing to know is that (if you start today) you are very late.

This is especially true for software covered by bug bounties, because hunters were already searching for vulns before LLMs. But the pressure on those projects increased a thousandfold with LLMs.

Concrete numbers, from the public program pages of two very large projects (90-day stats, September 2026), left unnamed on purpose, the difference is the point:

• One program, still functioning: 4,193 reports received, ~$454,000 paid out
• The other, effectively dead: 551 reports received, $1,000 paid out, last resolved report a month old

Read that again. The volume is mostly AI-generated noise. curl shut down its paid bug bounty entirely in early 2026 after the ratio of valid reports collapsed. Other projects (Nextcloud, Grafana) suspended payouts or went invite-only. Maintainers are drowning.

So if you think you are smarter than everyone and that you can make money fast with your Claude account, think again. Hundreds thought of this a year ago already, and their number grows daily. You are competing with them, with seasoned professionals, and with people who have far more AI resources than you.

Let’s talk about models (choose your weapons)

I tried the obvious options first, and it went poorly:

• Claude: anything even remotely close to cybersecurity gets flagged and switched to a heavily restricted mode. After a few hours, the session gets frozen outright. The only way around is starting fresh sessions and losing your context every time - and the blocks get more frequent over time, even for legitimate work. The official path is Anthropic’s Cyber Verification Program (OpenAI has an equivalent, the Cybersecurity Grant Program), but validation seems arbitrary: several security researchers I know personally got refused without explanation, and they’re not alone.

  

• Google Antigravity: worked well for 2 days, then the blocks started. Google has no program to certify legitimate hunters, so you can forget it completely. 1 month subscription for nothing.

The workable answer: drop the most famous US providers and switch to less picky alternatives. Either subscription-based access to open-weight/Chinese models, or agentic platforms with generous quotas.

What about pay-per-token APIs? Very bad idea for this kind of work. Bug hunting requires a ludicrous amount of read tokens. My own numbers: about 11 billion tokens in 2 months, the vast majority (over 90%) served from provider-side cache. Even with one of the cheapest options at the time of writing (GLM 5.3 on a famous broker), that’s roughly $1,800. A frontier model would cost several times that. Choosing a provider with cached tokens helps a lot, but you still don’t want that bill.

Subscriptions that give access to GLM, DeepSeek, Kimi, … are probably good options (haven’t tried yet). What I’ve actually used for 2 months is devin.ai (Cognition’s agentic tooling, formerly Windsurf): their in-house model is competent and, at the time of writing, unmetered in the base subscription. Promos like this come and go, so check before you subscribe.

Not sponsored, I pay for it 🥲.

What about local models?

The idea is really nice. Being self-sufficient with local hardware (or renting it) rather than paying per token for the expensive tasks at hand, while avoiding provider censorship, seems like a genius idea. It was the first thing I tried after the Claude/Antigravity debacle.

So… can you leverage local open-weight models to avoid burning your quota in 20 minutes? With a ~30B model (Qwen-class, or whatever the current best variant is), you can find some vulns on large, old codebases… but it’s like running a marathon with a backpack full of stones. You can do it, but expect it to be neither easy nor efficient.

If you go that route, the pipeline below matters even more than with bigger models.

My pipeline

Whatever the model, the first thing you need is a robust framework. Here’s what I did (pretty basic stuff, there may be more efficient ways, but it’s a good start):

1. Analyse one component of your target at a time. Ask the model to map how it works beforehand. Do it completely before moving on to another.
2. Once it’s mapped, split the job (code files mostly) between subagents, each focused on a single bounded part. Ask them to look for known anti-patterns in functions, badly checked variables and so on, and to write down everything even remotely weird/dangerous to a markdown file for later analysis.
3. Have the main agent review the subagent work. Every finding is routed either to a “non-security findings” folder (bugs too weak or too narrow to exploit - I use those for improvement PRs upstream) or a “security findings” folder for counter-expertise.
4. Once the initial scan is over, ask the main agent for a counter-expertise of all the newly validated security findings.

Counter-expertise

Most of the value of my LLM scans comes from this step. It took me two months to refine it, and I’m still working on it.

My current checklist:

• Search GitHub/GitLab for issues, PRs, improvement documents when available. Submitting a security issue that’s already known is obviously a bad idea.
• Also search official docs for known limitations, documented weaknesses/tradeoffs, security best practices.
• When you’re 100% sure there’s no public fix or previous report, ask the model to reproduce on real software. Depending on the target, you may want runbooks for various setups on various platforms. This step is especially important with local models, which will produce absolutely convincing vuln reports that never work.
• Once reproduced, rate it. Most bug bounties ask for a CVSS score, but you need to go deeper than that.

Concretely, a finding in my pipeline looks like this: a subagent flags a suspicious pattern in a file nobody reads -> the orchestrator dedups it against public issues and CVE feeds -> if it’s clean, it gets reproduced on a throwaway kind cluster or a local install of the real software -> then scored, written up, submitted… and then you wait. Triage usually takes weeks/months. Sometimes it’s days, and on rare occasions years (more on that later, that’s usually the sign of an overloaded or dead program).

Rate it fairly

Once you have real vulns, rating them with a CVSS is a necessary step. But the work is far from over. We are all aware that CVSS doesn’t tell the whole story. And LLMs will tend to over-rate their own findings.

Make the agent ask these questions during counter-expertise:

• Is the finding realistic?
• Are the prerequisites realistic?
• Would they exist / happen in a real deployment, not just theoretically?
• Is the configuration sane (not a footgun where the admin in our scenario deserves to be hacked)?
• Would an attacker actually try to use this?
• Does the finding grant more power than the attacker already has?

All those answers (or at least most of them) should be yes. If not, this is probably not a security issue (or at least, that’s what the security teams of those projects will tell you!), regardless of what the CVSS says.

There are obviously exceptions, but for the sake of this article I’ll only give the general case.

Own it

Once all of this is done, take some time to look at it (you, the human). Depending on the quality of the model, you may find the report ridiculous:

• Read the LLM report carefully, understand it, own it!
• If something feels weird, challenge the LLM
• If a claim is exaggerated, challenge the LLM

That’s where either a cybersecurity background or knowledge of the tool you audit is really useful. It’ll keep you from making a fool of yourself.

Don’t be part of the problem

Let’s address the elephant in the room: I just told you maintainers are drowning in AI-generated reports, and I sent 30+ of them. Fair point. Here is what I try to do differently:

• Nothing gets submitted without being reproduced on the real software
• Every report is read, understood and owned by me before it goes out. If I can’t explain it to the maintainers without the LLM, it doesn’t go out
• Low-severity or borderline findings don’t go through security channels: they end up as public issues or PRs upstream
• Ratings are challenged: no LLM-inflated CVSS
• Reports go out one at a time on small projects, waiting for triage before sending more

Duplicates still happen (9 of them so far), but a duplicate of a real, reproduced vuln costs the triager a few minutes. A hallucinated report costs them hours, and some of their patience for everyone else.

Keep track of everything

This one caught me off guard. After 30-ish findings and about 10 reports sent, I honestly couldn’t tell anymore where I was. Which finding was reproduced? Which one was already submitted, and to which program? Which report was waiting on me, and which one on the maintainers? A folder of markdown files doesn’t scale past that point, and giant markdown tables maintained by the LLM quickly became unmanageable.

So I ended up “Clauding” a tracking dashboard (yes, the irony: Claude happily codes a UI for bug hunting, it just won’t do the bug hunting itself). The source of truth is now a JSON database, which the LLM updates through scripts (never by hand-editing the file). On every commit, a static HTML kanban board is regenerated from it, following each finding through its whole lifecycle.

(Sorry, the UI is in French, and the titles are redacted for obvious reasons)

Each finding is a card that moves from left to right:

• Candidate: flagged by a subagent, not verified yet
• Confirmed: survived counter-expertise and reproduction
• Writing then Ready: report being written, then ready to send
• Sent and Triage: submitted, waiting for the program to look at it
• Accepted (embargo): accepted by the program, fix or disclosure pending
• Fixed: fix released
• Closed: rejected as duplicate, wontfix or informative

Every card carries a composite score (CVSS + wontfix risk + duplicate risk, more on that in the next section), the CVSS, how many times it was submitted, and whether it was already reported upstream. Sorting by composite score tells me what to write next. Filtering by program tells me how many slots I have left before hitting the submission cap.

The stats view is also a good reality check:

163 findings (after excluding 215 non-security ones), across 26 projects, and only 31 submitted.

The “Ready” column alone holds 118 findings. Finding is cheap. What limits me isn’t the LLM anymore: it’s my own time to reproduce, write and own each report (the human in the loop is the bottleneck), followed by the programs’ submission caps. And even then, being first and surviving triage is still up to luck.

Whatever tool you use (a spreadsheet works too), set this up before you start submitting. Retrofitting it after 10 reports, digging back through emails and platform inboxes, is painful.

You’re not alone

Even if the program gives no bounty, expect competition: through bug bounty platforms or GHSA (GitHub Security Advisory) reports. You can safely assume that every “easy to spot” issue was found months ago, multiple times. On projects with overloaded security teams, make that years.

I can’t give details yet, but I’ve stumbled on a vuln that had been reported years earlier, with a CVE reserved the year before, still neither fixed nor public. That’s the state of OSS security triage nowadays: the queue is longer than the throughput, and it’s not getting better.

So if you aim to be first (for the bounty or the fame), you need to be more strategic than average:

• Avoid bugs that can be found just by reading a single file, or that a SAST/DAST tool would flag. LLMs are good at this, and you can be sure someone found it first. Unfortunately that may be the only kind you’ll get out of local/small models (or you need a very specific harness).
• Target in priority code that isn’t the most audited. Don’t aim for pre-auth RCE unless you know what you’re doing - that code has been audited by people far better than you. Aim for recent beta/GA code, or on the contrary old legacy code that feels harmless.
• Target issues that require knowing how the tool works and what shouldn’t work. LLMs are bad at this, so if you have this knowledge, you have an edge.
• If you don’t pay per token, don’t be afraid to feed the whole file to the subagent. I’ve discovered by chance that an agent restricted to “sensitive functions” misses A LOT. This may be less true for frontier models, but it’s absolutely true for lesser ones.

With this in mind, I’ve come up with a score that rates my vulns not only by CVSS, but also by the risk of a “wontfix” from maintainers and the risk of a “duplicate” because it was too easy to find.

Conclusion

Will it work? Yes, absolutely. I’ve found dozens of real, unfixed vulnerabilities in production OSS software, ranging from medium to critical.

The harder truth: I’ve sent out 30+ reports so far. Of the verdicts that came back, most were duplicates - someone else got there first (my rookie mistake: I assumed easy findings would make it, when easy means someone already found them). One got me a co-credited CVE that just went public (CVE-2026-77524, a Critical namespace-admin-to-cluster-admin escalation in KEDA), one more advisory was accepted but is not public yet, and one fix got merged via a public issue. The rest are either still in triage or queued behind rate limits - some platforms cap submissions, e.g. 4 reports per program over a rolling 30-day window (more if you hunt on several programs), plus a global cap across the platform.

And when a project simply doesn’t answer? I don’t have a perfect rule yet: I nudge once after a few weeks. The common convention is a public write-up after a reasonable embargo (90 days), but I haven’t decided yet whether I’ll apply it. There are intermediate steps too, like escalating to a CNA or to GitHub Security Lab for projects on GitHub. A report that rots in a private queue helps no one, but neither does dropping an unfixed vuln on users.

So: leverage the constraint. Turn the weakness into a strength. Everyone focuses on the same high-profile targets (who doesn’t want a pre-auth RCE?) or the cheap, obvious vulns. If you want results (being first), ignore both. Focus on the dark edge cases spreading across 3 components. Yes, it requires more work, better LLMs, and a bit of luck.

If you start now, the easy wins are gone. But that’s precisely where your edge is: knowing how the tools you use every day should behave is something nobody can rent by the token.