Deepfakes and Scam-states

A narco-state versus your grandmother.

That is the level of asymmetry at play in the war against cybercrime.

Scam organizations are making ungodly amounts of money. Some have become powerful enough to corrupt entire governments. By one measure, Cambodia’s online-scam industry makes over $12.5bn a year, equivalent to about half the country’s formal GDP. That means the broader economy in these places—construction, hospitality and so on—is also dependent on scams. The level of co-option of state actors exceeds anything seen in Latin American narco-states of the 1990s.

At the same time, deep fake technology - encompassing voice and face cloning—has dramatically increased the potency of scamming people. Once the domain of experts, these tools are now accessible to non-technical users through open-source platforms and commercial services. Scammers can now convincingly impersonate individuals via video calls, voice messages, or even live interactions, creating new forms of deception that can bypass traditional suspicion and verification methods.

Many scams target employees of large enterprises, particularly financial institutions. In a recent high-profile case, a Hong Kong bank employee was instructed by his CFO, over a video call, to wire $25 million on behalf of a client. It turned out the CFO was a deepfake impersonation manufactured by a cyber-criminal. They infiltrated the bank’s video calling system, sampled enough image and voice data from the employee’s superiors to create convincing replicas, and then conducted an internal call to instruct the wiring of money to their personal accounts. No one on the call was who they said they were.

Startups like Reality Defender, Mirage, Deep Trust are three of a growing number of companies offering enterprise’s tools to detect deepfakes and prevent fraud. For financial clients, these companies are attempting to stop the damage at the point of a traditional money transfer – when a customer attempts to hand over cash to an attacker.

Others, like Dune and Anzenna concentrate on bolstering the human element by providing security awareness training and user-adaptive risk management.

You can imagine a combination of these technologies successfully stopping the wire of money in the example above.

But the growing popularity of P2P (peer-to-peer) payment networks and cryptocurrency exchanges means most of the stolen money flows outside of these traditional financial intermediaries. The JPMorgan’s of the world have robust security teams, dedicated fraud departments and mechanisms for transaction reversals. They also spend hundreds of millions of dollars per year to placate regulators and protect against losses.

By contrast, while real time payments providers offer speed and convenience, they often lack the consumer protections historically provided by traditional banks. The ACI estimates that 63 per cent of these fraudulent scams were already conducted over the real-time payment networks in 2023 and that by 2028 this will increase to 80 per cent.

Many of the aforementioned startups are working on consumer solutions, software that could be downloaded through an app store. Anyone with a phone would be able to scan incoming messages for deepfakes. But right now, the majority of solutions exist exclusively for large enterprises.

That leaves every day consumers vulnerable to a rapidly advancing criminal threat.

1 in 10 Americans were victims of a cyber-attack with that number only expected to grow. Sue Lin Wang, an investigative reporter for The Economist, put it most bluntly: “The first time I realized the extent of the problem, my biggest fear is that no one in America is going to have any money left.” While such statements can be viewed as a bit of hyperbole, it is hard to deny that this is now a national security concern.

Existing identity monitoring solutions have shown they are not up to the task. If you take offense to that, let’s not forget the time that Equifax, one of the three major credit monitoring bureaus, suffered one of the most widespread data breaches of all time, exposing the personal information of approximately 147 million people, including social security numbers, credit card numbers and license information.

I hope this new wave of startups can find a viable business model for the average consumer. Because right now, we appear to be pretty f**cked.

Honorable mentions:

Later stage private companies like Aura have made progress, but have still not taken significant consumer mind-share.

Chainalysis, a cryptocurrency investigation and compliance software, has made strong inroads in helping victims track and recover funds as it is laundered through a vast network of digital wallets, with a landmark ruling that validated that their analytics are admissible evidence in court cases.

Appendix: The Cyber Defense Matrix

If we think about what an enterprise CISO (Chief Information Security Officer) cares about, the “user” attack vector is what we are focused on in this article, here is where the aforementioned companies play within the Cyber Defense Matrix: