The emergence of deepfakes and their disruptive potential: The case of Arup

When smartphones emerged, it took the world a while to grasp what they could do before they became truly disruptive. We are at a similar point with deepfakes. While many of us recognize that deepfakes are on the rise and have seen a few clips that leave us stunned by how real they seem, we may not fully understand how they work or recognize the impact they are already starting to have on various aspects of business and society.

To illustrate the devastating effects deepfake technology can have, let us look at the case of Arup, a UK engineering firm that recently fell victim to a GBP 20 million deepfake fraud. 

How did the scam unfold?

1. The employee in Hong Kong received a message about a confidential transaction from someone claiming to be the UK branch’s Chief Financial Officer

2. The initial message raised some suspicion, but the employee was also sent an invite to a video conference call.

3. During the call, the deepfake of the CFO instructed the employee to transfer a large amount of money, claiming it was for a critical business need. In addition to the CFO, there were many AI generated employees in the call

4. Since it was framed as a confidential and urgent business necessity, it pressured the employee to act immediately, without making further checks

5. The employee transferred the funds asked through 15 separate transactions. By the time the scam was uncovered, the money was already moved through multiple channels, making it impossible to trace 

Why was the scam successful? 

• Firstly, the scam was not a mere phishing attempt. It was sophisticated and involved text, video and audio elements

• The call also had many senior officials on it, and there were multiple deepfakes, making it look more legitimate 

• The fact that it happened on a real-time video call made it more convincing 

• The employee in Hong Kong was less likely to suspect anything unusual since they were not physically present in the same office as the UK-based executives. This made it easier for the scammers to mimic the identities of these senior officials without raising suspicion

• Human deepfake detection capabilities are limited, especially when the technology is used in such a sophisticated manner. The scammers replicated the voices and appearances of senior company officials, adding layers of authenticity that would have been difficult to question in the moment

The way forward

A recent report found that deepfake digital identity fraud has surged 10-fold in 2023 when compared to the previous year. Further, deepfake technology has grown to an extent where it is not just good for audio calls, but also for convincing video calls. With remote and hybrid work rising exponentially post-pandemic, companies could be more vulnerable to deepfake frauds. A recent study also found that humans cannot always reliably detect deepfake, which poses a bigger problem as these technologies get more sophisticated. 

The Arup case serves as a wake-up call for businesses around the world. As deepfake technology becomes more advanced and accessible, this will not be the only successful deepfake scam. Companies must proactively educate their employees and invest in technologies that will keep their meetings safe and secure.