Speculation over the potential for misuse of AI and machine learning technologies have been bandied about for longer than the practical applications of these technologies have been available. We’ve seen concerns manifest in popular culture as depictions of a dystopian future, and while reality is considerably less dramatic, there have been some developments that are nonetheless worthy of examination.
To be clear, there is still no semblance of real “intelligence” in AI tech in the way we understand human intelligence. What does exist today is a fairly good approximation of human decision-making that can be applied to perform specific tasks with great efficiency at scale. That said, some advancements in AI tech have raised some eyebrows.
One in particular is deepfakes, a rapidly advancing technique for generating very realistic media that has the potential to be very disruptive when misused.
Deepfakes in Brief
Deepfakes are a machine-aided synthetic media technology used for the generation or manipulation of text and video that can appear quite realistic to the untrained eye, and they have the potential for considerable implications across culture, geopolitics, and security.
For example, AI researchers at the Massachusetts Institute of Technology recently completed a seven month project to create a deepfake video that purports to be footage of the late President Nixon delivering a somber announcement that the much celebrated Apollo 11 mission to the moon had ended in tragedy.
The script for the video was based on an archived copy of text prepared in the case that there had been any complications with the historic event. AI algorithms were employed to help create the eerily convincing “live presidential address” from existing audio and video footage:
While advances in technologies that have the potential to be exploited by threat actors like those used to produce deepfakes are generally controlled through regulation, the cat may already be out of the bag where deepfakes are concerned. We are at the point where most legitimate uses of deepfake tech can be controlled to a degree by limiting legitimate access, but these controls likely will have little effect on the potential for malicious use.
It is important to delineate the three main types of deepfakes so we can understand their implications:
- Mimicking fakes: this is a technique where video of one person is superimposed on a target video using AI to enhance and mask the manipulation. Think of this as an advanced “green screen” process that effectively lets one person do all the talking while it appears to be someone else, right down to movements and gestures.
- Generative fakes: this technique also employs AI algorithms, but in this case, they are used to completely synthesize new audio and video from existing materials to produce ultra-realistic content as is seen in the video from MIT above.
- Generative text fakes: this more common technology uses AI applications such as the OpenAI GPT-3 to allow computers to generate text content on almost any subject that is incredibly close to actual human writing.
Each of these applications pose a separate and distinct threat on their own if misused, but in combination have the potential to produce “generative personas” that in the near future will be extremely difficult to distinguish from the real thing. This is where the security implications come into play.
Malicious Deepfake Applications
Generative text fakes using GPT-3 level tech have the potential to make phishing attacks far more convincing and much harder to detect in an automated fashion, as the newer iterations of these techniques can generate text in individualized speaking styles that closely resemble specific people. This means it can be used to create deep-phishing messages that will be impossible for the current generation of anti-phishing systems to identify, leaving targets at risk.
Right now, advanced generative text technology is largely confined to a defined group of people that OpenAI granted access to. They blocked the last generation of the tech from public release due to fear of abuse. So, for the moment, that channel isn’t a significant threat. That said, the massive knowledge base used in model training means the tech getting very close to being indistinguishable from actual human authorship, and it’s not clear that a strategy exists to better detect these fakes in the future.
Mimicking fakes with superimposed video - while still an evolving technology - are already good enough to fool an unwitting viewer to a reasonable degree. If this technology can be applied in real-time and coupled with advanced audio fakes, there is the potential that you might one day find yourself in a video conference call with someone you think is your CEO but is actually an attacker. The technology is not quite there yet, but before long it may result in a very effective attack vector.
Generative fakes with synthesized video have the most potential to have serious implications if used for nefarious purposes given the technology creates fake content that could convincingly have almost anyone saying and doing almost anything.
Think of the impact that the abuse of fake social media accounts has had on the success of malicious state-sponsored disinformation and influence campaigns in recent years, then imagine it’s not just mono-dimensional sock puppet personas inundating media channels, but dynamic manifestations of what appear to be real people saying and doing real things.
Then consider the implications a convincing deepfake video of a high-profile figure like the one above going uncontrollably viral on the Internet could have on public safety or geopolitical relations. The potential for real harm is significant.
Identifying Deepfakes
Generally speaking, the successful detection of fake generated media will be based on our ability to recognize repeated patterns generated by the algorithms used in their production. In the case of text fakes that leverage GTP-3, the machine learning relies on a massive collection of human generated text, and the same machine learning techniques used to create the fakes would also be used to identify “true” text vs. “fake” text. This is a case where the better machine learning application has the advantage.
GTP-3 tech is so advanced that it could be abused to create deep-phishing messages that speak in the voices of many and will be impossible for the current generation of anti-phishing systems to identify. Algorithms for both generation and detection of deepfakes rely on the same data pool for training the models, so it’s very hard to distinguish between text generated by the newer algorithms.
As for video deepfakes, at this point they generally have a perceivable “artificial quality” to them, even really well-executed fakes such as the video above. The best weapon we humans have for determining if a video is a deepfake for now is for us to learn to recognize and acknowledge that uneasy feeling that what we’re looking at doesn’t feel real and is somewhat alien. This “uncanny valley” is the key to unraveling fakes - they just feel wrong intuitively.
But as these technologies improve, it’s clear that more than human intuition will be required to make a solid determination on authenticity, and it may be the case that it is simply not going to be possible without the creators of the technologies or the fakes themselves intentionally leaving an indicator of some sort, like a virtual watermark.
Either way, deepfakes and our ability to ensure they cannot be utilized maliciously, are definitely a subject worthy of further discussion before the technology advances to the point where we can’t put the genie back in the bottle.
