What AI changed about phishing, and what it did not

For twenty years the cheapest phishing tell was the writing. Messages were translated badly, punctuated oddly and signed by a “Customer Service Department” that no real company has. Security training leaned on it, and it worked often enough to stick.

Language models removed that tell for anyone with a browser. They also removed the cost of the two things that used to separate cheap phishing from expensive spear-phishing: research and follow-up.

Two versions of the same credential-phishing lure. Both are fictional.

The second message has no errors, references a real-sounding meeting, a plausible supplier and a deadline, and gives a reason for the login prompt. Every word of it could be generated from a LinkedIn profile and a company “About” page. Nothing in its wording is evidence of anything.

Everything AI improved is about the text. The structure of the attack stayed the same.

In the example above, “Tom” would be writing from something like [email protected] when your company is harbourline.com, and the sign-in link would lead to a domain that is a few days old. Those facts are not changed by better prose.

Detection that judges how a message reads - grammar, tone, “phishy” wording - degrades as writing quality improves. Detection that weighs facts about the message holds up:

• Has this sender, or this domain, ever exchanged mail with this organization?
• Does the domain imitate one the organization already trusts?
• How old is the domain the links point to, and what is hosted there?
• Is the message asking for a credential, a payment or a change of payment details?

This is the approach Zoes takes: each Dangerous or Suspicious verdict lists the signals behind it, such as an unfamiliar sender or a link to a low-reputation site, rather than a judgement about writing style.

The same tools that write convincing email can imitate a voice from a few minutes of recorded speech - an interview, a podcast, a conference talk. That turns the classic “verify by phone” advice into something more specific: verify by calling a number you already have, not by trusting a call or voice message you receive.

• A voicemail from the owner asking for an urgent transfer is the same request as an email from the owner - and gets the same callback.
• For larger payments, some companies agree a simple check that is not written anywhere online, such as a question only the real person can answer.
• Video calls arranged at short notice through a link in an email deserve the same suspicion as the email itself.

Stop saying

• "Look out for spelling and grammar mistakes." It now teaches people to trust polished fraud.
• "If it knows your name, it's probably real." Personalisation is free.

Start saying

• Read the actual address, character by character, not the name shown.
• Any request for money, bank details or a password is checked on a second channel, however normal it reads.
• A message that is unusually well informed about you is a reason for more caution, not less.
• Reporting a false alarm is fine. Nobody is ever in trouble for checking.

A small team does not need a training platform to fix this. One session, about 45 minutes:

Repeat a shorter version when someone new joins. The checks do not change, which is the point: they work regardless of how good the writing gets.