Do you even know how to Deepfake ?

I’m sure you have heard of a type of deep learning algorithm that uses GAN to create fake images using celebrities. They are called deepfakes and are getting more and more popular as it is getting easier to create one, and of good quality.

This is one of the most popular exemple :

								Were you able to tell it was fake ?

How did they do it ?

Well, it depends of the algorithm actually. Let say you have person A from which you want to animate the face like person B does. You will need to train two pair of encoders-decoders - one for each face - to create a latent representation of the person and be able to decode it.

Then, you can use the decoder from person B to decode the latent representation of person A to create the deepfake. This explanation is clearer in the following image :

However, to be able to train those autoencoders, you will need A LOT of video footage of person A and B. And presumably, of good quality. It also requires a lot of computation power to treat all of this video data. But if you have all that, and the time and knowledge needed, you can create deepfake of excellent quality like the one you saw with Obama.

Do you we need to be afraid of this kind of algorithm ?

A bit. Because every powerful tool in the wrong hand can turn into a dangerous weapon, the same can happen with deepfake algorithm. At a time where fake news are being more and more propagated, and people trust less and less conventional medias, having the ability to make say anything to anyone could lead to a dangerous path.

Fortunatly, if we are aware of this danger, which many people are, there are ways to counter such malicious use of those algorithms. For example, using… AI ! Yes, battle fire with fire. Many papers already proposed algorithms specialised in detecting deepfake videos (D. Güera and E. J. Delp, “Deepfake Video Detection Using Recurrent Neural Networks,”). This may be however an endless war between detectors and deepfake algorithms, just like with computer virus and antivirus softwares, where each one get a step on the other, without ever having a real winner.

Doesn’t need to be used for harm

Do you it is the same kind of algorithm that are used behind some snapchat filters ? Or behing the now famous FaceApp ? When you do a face swap with someone, this is actually a deepfake ! These are very cool ways to use those algorithms, for pure entertainement. One a personal note, this is an use of deepfake that I just found amazing and fun :

This channel is amazing, they do a tone of deepfakes on movies, changing the casting like it was nothing ( spoiler : it is probably very hard).

Small experimentation

Thanks to the channel Two minutes paper, I discovered a new type of deepfake algorithms that just recently came out. Called First Order Motion Model for Image Animation, it is surprising how simple it is now to create a deepfake ! Let’s used again our example of person A and person B from earlier, where we want to animate person A just like person B does. Well, now, we only need a picture from person A and the video of reference from person B to make it work ! No need for hours of videos to train an autoencoders for each face.

So I decided to give it a try, and wanted to see if the algorithm coud animate my face following Macron’s facial expression during his speech where he announced global containment. Here is the result :

It took me only 45 minutes to pull out using the paper’s code, and the result is incredible, though there are still some imperfections. I made several more tests, this time animating Poutine’s face following mine. Such a shame however my russian accent is not on point !

Be sure to check the blog post of the paper in question !