Security in the Age of AI: What Product Leaders Need to Know Now

AI has become very good at finding and exploiting security flaws in existing software, and being small or obscure no longer protects you.

What changed in the security landscape

Modern AI can find and exploit latent vulnerabilities in software far faster than people can. All software sits on a big stack of layers written by humans, and AI is showing that those layers are full of mistakes, some of them years old. What's new is that those mistakes are now easy to find and easy for a bad actor to use.

A recent PHP vulnerability is one example. It would let an attacker take over any server running PHP, and it had gone unnoticed for years. This time the good guys found it, using AI, and a patch was released. But all it takes is someone with access to a powerful AI model and a request to look for security flaws in a piece of software.

The reason this is happening now is access. Anthropic and OpenAI lock down their models so they can't be used this way, which keeps that capability with defenders. Open-weight models, which anyone can download and run however they like, are now powerful enough to do the same work. These tools make the bad guys far more capable, and that trend will continue.

Why "too small to target" isn’t a thing anymore

It used to be a fair assumption, and most software quietly relied on it. Programs that scan the whole web for security holes have existed for years, much as search engines scan it for content. But outside the big, obvious holes, it took a lot of skilled work to turn a vulnerability into something useful, like a ransomware attack or stolen data. That work was expensive, so attackers went after the juicy targets.

AI changes the economics. Systems that are as capable as most human hackers, or more, now work for pennies and dollars instead of thousands of dollars. They run at superhuman speed, and you can have as many as you can pay for. Someone can put a thousand AI hackers on your software, probing it for weaknesses and working out how to exploit them. They can do the same for thousands of other sites at once.

The obscurity that protected you was a limit on human attention. There were only so many people doing this work, and it was slow. Now that work can be automated at scale. Software that has run well for years, and was never a tempting target, can be one.

What to do about it now

Two practices matter most, one old and one new.

Keep everything up to date, and faster than before. AI-driven analysis is surfacing more vulnerabilities than ever, in your own application and in everything it builds on. There are two clocks: the time between a flaw being found and a patch being released, and the time between that patch and attackers using the flaw. The second one is collapsing. Attackers can study a fix, work out the bug it closes, turn it into a repeatable attack and deploy it at scale. That means updating faster than teams have historically.

Use AI to audit your own software before someone else does. Humans have always built software with security flaws, and defenders can use the same AI to find them first. This takes two steps. First, run a full scan and make a concerted effort to find and fix flaws, to get to a more secure baseline. Some fixes will be simple. Others will mean a deeper change to make the system inherently more secure. Second, put a process in place so every change your team makes gets scanned too, and the software gets more secure over time instead of picking up new bugs.

Expect this to be a back-and-forth for a while. Each new, more capable model will find flaws that earlier models couldn't, so securing software will keep getting harder before it gets easier. Eventually things will probably stabilize, but many people in the industry think getting there means rebuilding much of the software stack, and possibly rethinking hardware too. Before long, teams will likely need AI to build software at all, because humans alone won't produce software that is secure enough.

Where people belong in the loop

People decide what the risks are. A product leader should ask what the downside is if data is stolen or deleted, whether the business would come to a screeching halt, and whether protected health information could be released to the world. That risk analysis tells you which steps to take and how far to take them.

AI doesn't do this automatically. Engineers still have to work out how to do the analysis and which risks are worth protecting against. They also have to judge when to plug the holes in the ship and when the ship is rotten and part of it needs rebuilding. From there, they design the tools, practices and automated security analysis to match. In some cases the right call is to throw a piece out and have AI rebuild it more securely, and the industry is seeing a lot of activity around exactly that.

How to vet your software team

A good team answers these questions with specifics. Be wary of vague answers, and of answers that would have sounded the same two years ago.

1. What would hurt our business most if someone broke into this software?‍

A strong team asks about your data and your operations before it talks about tools. The more a breach would cost you, the more security work the team should propose.

‍2. How do you check for security flaws as you build, and do you use AI to do it?

The team should run an automated check on every change, and a deeper review before launch. Engineers should confirm each finding and fix it.

3. If an attacker gets in anyway, how far can they go?

A careful team gives each person and system only the access it needs, so one break-in does not expose everything.

4. What will we need to do to keep this secure after you hand it over?

Software needs security updates for as long as it runs. The team should leave you a clear plan that says who applies updates, who receives alerts and what to do if something goes wrong.

5. How has your approach to security changed in the past year?

A team that follows this shift can name specific changes it has made. A team that says nothing has changed has not noticed it.

‍