Somewhere in the last year, every AI coding tool I use started checking its own work. Lovable added a security scan to its publish flow. Replit wired in a scanner that runs against your Python and JavaScript before you ship. Anthropic gave Claude Code a built-in security review command you can run against your own diff. On paper this is good news. The tools that write the code now flag some of what is wrong with it too, before a human ever looks.
I have been paying attention to what actually gets caught this way, and what does not. There is a pattern, and it is not flattering to the self-review model.
The bug the tool did not find in itself
The clearest example is still the Row Level Security issue in Lovable-generated backends. An independent researcher, Matt Palmer, found it in March 2025: Lovable was generating Supabase backends where Row Level Security was missing or set up backwards, so anyone who knew the right endpoint could read or write data that was supposed to be locked down. By the time it was publicly disclosed in late May, 170+ production apps were confirmed affected, across 303 vulnerable endpoints in a sample of 1,645 scanned projects. Lovable did not catch this in itself. An outsider did, working from the outside in, with no stake in the tool looking finished.
Lovable's own security scan shipped about a month after Palmer's report went in, folded into a version 2.0 release. It is a genuinely useful feature today. But look at the order of events. The flaw existed first. An external person found it. Only after that did the self-review get built to catch that specific class of problem going forward.
Self-review checks what it already knows to check
That order of events is not a coincidence. It is the whole shape of the problem. A tool that reviews its own output is running the same reasoning twice. If the model did not know Row Level Security needed to be on by default the first time, asking it to double-check its own work does not introduce a new fact into the room. It is still working from the same assumptions that produced the gap in the first place.
Independent testing backs this up at a larger scale. Veracode's 2025 GenAI Code Security Report ran more than a hundred models across 80 coding tasks and found that AI-generated code failed to defend against a common category of vulnerability in 45 percent of tests. That is not a story about careless developers. That is the baseline rate at which the code these tools produce has a gap in it before anyone reviews anything.
The Base44 story makes the same point from a different angle. The flaw there was not a missing default, it was a logic hole: unauthenticated endpoints that let anyone register into someone else's private app if they had the public app ID. Base44 reportedly hit 250,000 users around the same time Wix bought the company for 80 million dollars in June 2025. Its own systems never surfaced the bug. A security research team looking at the platform from the outside did, in July, and it was patched within about a day of the report landing.
What a second, independent pass gives you that self-review cannot
None of this means the built-in scanners are useless. They catch real things: hardcoded secrets, obvious missing validation, patterns the model has been trained to flag. They are worth running every time. They are free, and skipping them just leaves easy points on the table.
What they cannot do is step outside the assumptions that built the app. An independent reviewer is not reasoning from the same starting point as the tool that wrote the code. They did not decide Row Level Security could stay off for now. They did not assume an app ID was safe to expose because who would guess it. They come in with one job: find what is wrong, not confirm that what got built looks right. That is a structurally different exercise than asking the author to grade their own work, no matter how good the author is.
If you have shipped something with one of these tools recently, run the built-in check first. It is a decent filter and it takes no time. Then, before real users and their data touch the app, get someone who was not in the room when it got built to actually look at it. It does not need to be elaborate. It just needs to be a different set of eyes than the one that wrote the code.