In the age of AI that we have entered, we are increasingly more exposed and vulnerable than ever before. Some of the threats in the future will seem predictable and familiar, while others will take us completely by surprise. As a PSA of sorts I am jotting down my experiences and thoughts about trust and security in the AI age; Hopefully, I can add to this over time. Perhaps it will reach someone and give them the moment of pause that could make all the difference.

There are two fundamental things that make securing our person and property in the AI age different. The first is perhaps more obvious — that AI will generate ever more creative and sophisticated scams and in a greater volume than was previously possible. The second fundamental factor is simply economic — That the unit or per attempt / per individual cost of attacks will fall to zero.

Today , there is a significant and restrictive labor cost to mount a robo-call fishing campaign against a list of potentially vulnerable individuals. If you have ever had a Coinbase account you have likely already received these calls after their entire user database was hacked and then leaked online. Just like in business where there is a unit cost associated with aquiring a client, there is likewise a unit cost to pursue a potential target or victim to scam. Even though an individual phone call might cost little, at scale, large campaigns add up and limit the feasibility of wide-spread, prolonged, complex, multi-step attacks.

The AI agents that are emerging today are not easily distinguished from humans by voice or even video. It will cost essentially nothing for an AI agent to spend all day on the phone or zoom with you and the cost will be de minimus to perpretate even the most laborious of scams at massive scale. Please consider that it will cost next to nothing to have an AI agent custom design, engineer, and execute a scam created just for you, based on public, leaked or hacked personal info. You can imagine AI agents might start with publicly available and leaked information, and use it to convince the person to compromise their email credentails. Once in the email account, the agent will methodically read your entire email history and search for points where you might be vulnerable to manipulation, exploitation, extortion and other compromise. This is in principle feasible and inexpensive today, but will be all the more so with the AI models a year from now.

It is worth noting that the game theory of scams changes with AI in the toolbox. Previously, if your email or other accounts are compromised by some bad actor, they are incentivized to use the advantage immediately as there is zero marginal gain to waiting. With inexpensive AI, a genius AI agent could skim your entire email archive formulating a detailed picture of all of your contacts and associations, online accounts (what online account has not emailed you at some point?) and so on. The AI agent could form a better picture of you than you have of yourself and you might never know it was there, unless it immediately chose to act upon scam with a calculated high likelihood of success. If not, your profile would be added or updated in some database, along with info about your contacts, until an AI agent in the future has a scam with a better calculated success rate.

The days of getting the laughable and transparent emails from a friend who is traveling and lost all of their money and needs your help … these are drawing to a close. Fortunately, these phishing emails from a hacked email account do not usually convince you to compromise yourself, much less send money. But consider why they fail? These email are by nature generic and often with poor spelling and grammar; or just not a communication or style, that your friend or colleague would ever use to you. Also, this friend or colleague who has been hacked, discovers this and emails their entire contact list that their email account has been compromised and to please disregard, etc. Now imagine what a genius AI with unlimited time might do instead with this same phishing scheme … it might choose only the small handful of a friends / colleagues who are well characterized and deemed better targets; and craft the perfect phishing email from your account, or even text from a telephone number it found in your inbox. The AI bot could even carry on an email or text (or phone, or even video) conversation perfectly well in your likeness and devise a perfectly good reason for your friend to wire you money in a pinch. You get the idea … the sky is the limit for scams here and this is inevitable and is likely to be worse than I might even be able to imagine.

If your phone caller ID says Amazon or Chase, etc, you cannot trust it (I have had a number of these ID spoofs call my phone). If the caller on the line has detailed information that they could not have unless they were so and so … you cannot trust it.

In the coming months you can expect to encounter an increasing multitude of sophisticated and novel scams engineered to compromise your credentials and resources. You should have multi-factor authentication on everything that secures your resources or personal information about you. And you should be cautious about answering a message or call from an unknown or untrusted number. If you have a millisecond of doubt about an interaction, you should pause what you are doing, slow things down and think how you can confirm and ensure safety.

[What follows here are some examples that represent the lengths and cleverness of some recent scams, that don’t even employ AI yet]

Sometimes, you can’t even trust numbers that you call — let me give an example … About a year ago I google searched ‘amazon customer service number’ so that I could call Amazon. I called the number that was the primary hit in google and is set apart from the other results. I was routed to an agent pretending to be with amazon, seeking my amazon credentials. I was incredulous, doubting that google or the google algorithm could be gamed and somehow allow this blatant fraud to happen. But I checked the number against the real amazon number obtained through other means and sure enough, I confirmed that the top hit in google search for amazon customer service was indeed a fraud scam. With silly bravado I called the fraud number back and when the agent came on the line, I told him he would be caught, and all of the sudden his tone changed and he replied with an icy calm certitude before hanging up “No … I won’t be”.

I felt duped and betrayed and deeply disturbed by this. In the distractions of the day I forgot about it until a month ago when I heard a similar, less benign example: A family friend who needed to change an airline ticket called the first hit for the airline on the google search page. It turned out to be a fraudulent phone number with a very real sounding agent who was able to cancel and then steal the value of the ticket with information and actions that my friend readily provided. When I step back and reflect, this particular scam is fairly clever as it has the victims calling the fraudster … And the victims call with trust already conferred from google, so the abuser has a soft ball when the victim gets on the line.

There are lot’s more recent examples … Anyone who has wired money in the last couple of years has likely experienced how difficult it is and the many associated warnings. This is because wire fraud has become rampant and ubiquitous. I have heard a number of disturbing stories from close contacts who have lost money or nearly lost money through wire transfers, via quite sophisticated scam operations. In short, try to avoid wiring money at all, and if forced, verify the details and destination with great caution and suspicion. You cannot trust wire information in an email as these can and do get compromised (and swapped with an alternate destination). If you do wire money, be on the lookout for subsequent follow-up scams. A year ago a close family member was replacing an HVAC system with a heat-pump and wire transferred money between different banks for this. A day later they had a browser pop-up which convinced them to call an emergency number and the story gets crazier from there. I will skip the details, but it was later evident that my family member was targeted because the wire transfer had been leaked such that the fraudster knew their victim had a chunk of money on hand in at a local bank account with a nearby ATM that could be withdraw to cash. The scammer on the phone even took the victim’s address and told her where to find the ATM. This story very nearly ended in disaster, humiliation and loss of the money.

You should be careful about trusting audio voice or even video identity unless you are physically in front of the person. Assume all of your past email and text communications are available to an AI agent. Assume that an AI agent can formulate a detailed profile of you and your history and the people in your life … even their schedules, along with a multi-contingency plan for how to convince you of whatever claim it makes. The AI agent does not have to know human psychology to know the social approaches that have worked in the past and it will be able to generate, test and customize endless variants.

The best defense is to make yourself a hard target — And not to answer (or even open) odd emails or phone calls from unknown sources … and assume scam when there is any doubt or question.