The lazy lesson from this campaign is “don’t trust fake sites”. The better lesson is that a fake ChatGPT can live on OpenAI’s own domain, so trusting the address was never going to be enough. Huntress found at least 40 incidents that started this way.
Table of Contents
How the fake ChatGPT campaign worked
The attackers built a Custom GPT called “Plus 5.6”. Custom GPTs are chatbots anyone can publish inside ChatGPT, and each one gets a page on chatgpt.com. That is the whole point of the trick. The link looked genuine because, technically, it was.
Victims reached it through a sponsored Google result for “chatgpt”. The GPT then showed a notice saying the main service had limited availability and pointed people to a “backup domain”. Huntress reported that the fake ChatGPT backup site was hosted on Google Sites, so again the address looked respectable.
The paste-a-command trick
That backup page imitated a Cloudflare CAPTCHA. It asked visitors to prove they were human by copying a command and running it. This technique is known as ClickFix, and it works because the victim does the dangerous part themselves. No exploit is needed, and nothing is “downloaded” in a way a browser warning would catch.
The pasted command was a PowerShell one-liner. It fetched a script from a server addressed in an unusual decimal format, which can slip past simple filters. That script pulled down an installer named ISOSimple.msi, dressed up as a printer configuration tool.
Eight stages to a working RAT
Huntress counted eight stages in the chain. The interesting part is how much of it borrows trust from legitimate software:
- The installer drops a genuine Canon-signed application,
COTFileReadApp.exe. - That program loads a malicious DLL planted beside it, a technique called sideloading.
- The payload hides inside a file made to look like audio, and the final tool lives in a custom encrypted filesystem.
The end result is a full remote access trojan. According to Huntress, it offers remote desktop control, screen, camera and microphone capture, browser data theft and the ability to fetch more malware. Command traffic uses DNS-over-HTTPS, which blends into normal web traffic.
OpenAI took it down, then it came back
Huntress reported the first GPT to OpenAI on 25 September, and it was removed that day. A second, linked GPT appeared within days. Takedown works, but only after someone has spotted and reported the page. Until then the lure is live.
The campaign shows a shift in where attackers hide. Blocking a dodgy domain is easy. Blocking chatgpt.com is not something most businesses can or want to do. When the lure lives on a platform your staff use every day, reputation-based filtering has nothing to flag.
Why a fake ChatGPT is so convincing
Most phishing advice says to check the web address. Here that advice fails, because the first address was real. The second hop was a Google Sites page, and plenty of people trust anything on a Google domain. So the usual checks pass twice before the trap closes.
There is also the context. People are used to ChatGPT telling them it is busy, and a “try the backup site” message feels plausible. Few staff would think twice, especially if they were in a hurry to get a task done. That is how a fake ChatGPT page can beat a sensible, careful person.
Who is most at risk
Small firms often have the widest gap here. Staff use AI tools on work laptops, local admin rights are common, and nobody watches PowerShell activity. If one person pastes the command, the attacker gets a foothold with camera, microphone and browser data in reach. From there, stolen sessions and saved passwords can lead into email and cloud accounts.
What UK businesses should check this week
You do not need to ban AI tools. You do need to tighten a few habits and controls:
- Brief staff on one rule. No genuine website asks you to paste a command to prove you are human. Huntress puts it nearly word for word that way, and it is the single most useful sentence to share.
- Restrict who can run PowerShell. Most office staff never need it. Constrained language mode or application control can stop a pasted one-liner from doing damage.
- Watch for odd process chains. PowerShell launching
msiexec, or an unsigned DLL sitting next to a signed program, are both worth an alert. - Treat sponsored search results with suspicion. Encourage staff to use bookmarks for the tools they rely on.
It is also worth testing whether your people and your tooling would catch this. A phishing simulation can include a ClickFix-style lure, and wider security testing shows whether endpoint controls stop the follow-on steps if someone does paste the command.
Domain reputation is the wrong defence
Fake ChatGPT pages are only the current costume. The same pattern has already run through fake CAPTCHAs, fake browser updates and fake meeting software. What changes is the brand on the lure. What stays the same is a person being persuaded to run something themselves, so defences should focus on the person and the paste.
Subscribe to our newsletter
Honest updates, straight to your inbox. Unsubscribe any time.
