Co-authors: Shivansh Mundra, Gonzalo Aniano Porcile, Smit Marvaniya, Hany Farid

A core part of what we do on the Trust Data Team at LinkedIn is create, deploy, and maintain models that detect and prevent many types of abuse. This spans the detection and prevention of fake accounts, account takeovers, and policy-violating content. We are constantly working to improve and increase the effectiveness of our anti-abuse defenses to protect the experiences of  our members and customers. And as part of our ongoing work, we’ve been partnering with academia to stay one step ahead of new types of abuse tied to fake accounts that are leveraging rapidly evolving technologies like generative AI.

With the rise of AI-generated synthetic media and text-to-image generated media, fake profiles have grown more sophisticated. And we’ve found that most members are generally unable to visually distinguish real from synthetically-generated faces, and future iterations of synthetic media are likely to contain fewer obvious artifacts, which might show up as slightly distorted facial features. To protect members from inauthentic interactions online, it is important that the forensic community develop reliable techniques to distinguish real from synthetic faces that can operate on large networks with hundreds of millions of daily users, like LinkedIn. 

In this blog, we will describe how LinkedIn partnered with academia to research advanced detection techniques, provide details on our research, and share our results on how our jointly developed detection technique technique is able to detect 99.6% of a common type of AI-generated profile photos, while rarely–only 1% of the time–misclassifying a real profile photo as synthetic.

Partnering with Academia

As AI-generated faces become increasingly indistinguishable from real faces, academia and industry must collaborate closely on detection solutions. Academia brings expertise on cutting-edge research on detection techniques, while industry possesses a wealth of data, experience with real-world challenges and limitations, and the ability to translate research into production at internet scale. By working together, academia and industry can develop robust solutions for AI-generated photo detection that have a positive real-world impact.

Advertisement

The LinkedIn Scholars program enables academics to contribute directly to LinkedIn’s vision of creating economic opportunity for every member of the global workforce. LinkedIn Scholars has enabled the LinkedIn Trust Data team and University of California, Berkeley Professor Hany Farid to collaborate on new techniques for AI-generated image detection. This unique partnership enabled the creation of a novel approach for detecting a common type of AI-generated profile photos that is lightweight and highly effective. Our approach is detailed in the full paper that was released at the Workshop on Media Forensics at the 2023 Conference on Computer Vision and Pattern Recognition (CVPR) in June.

Details on Our Research

In our paper, we describe two related approaches that learn to recognize structural differences between AI-generated faces and real faces. We then show how these approaches are highly effective at recognizing synthetically-generated profile photos.

A common technique used to create AI-generated photos is the generative adversarial network (GAN). The structural difference between GAN-generated and real profile photos can be seen in Figure 1 consisting of the average of 400 GAN-generated profile images (left) and 400 LinkedIn profile images (right). The highly regular GAN-generated facial structure is revealed in the clarity of the facial features in the averaged image. The real LinkedIn profile photos have no such regular structure, which results in a blurry averaged image. Our embedding-based detection technique exploits the highly regular structure in GAN-generated photos.

Figure 1: Average of 400 GAN-generated profile images (left) and 400 LinkedIn profile images (right). The averaged image on the left reveals a highly regular GAN-generated facial structure. Our approaches learn simple/compact embeddings to capture the regularities in GAN-generated photos.

We used six data sets consisting of 100,000 real LinkedIn profile photos, and 41,500 synthetically-generated faces spanning five different synthesis engines: StyleGAN1, StyleGAN2, StyleGAN3, Generated.photos, and Stable Diffusion. A representative sample of the generated synthetic faces is shown in Figure 2.

Figure 2: A representative set of synthetic faces from (a) StyleGAN1, (b) StyleGAN2, (c) StyleGAN3, (d) Generated.photos, and (e) Stable Diffusion. In order to respect member privacy, we don’t show examples of real profile photos.

Sharing Our Results

The paper discusses the results from our two embedding-based approaches for distinguishing between synthetic and real profile photos: a learned linear embedding based on a principal components analysis (PCA) and a learned embedding based on an autoencoder (AE). For a baseline comparison, we also create a fixed linear embedding based on a Fourier analysis. The goal of the Fourier-based embedding is to demonstrate that a generic embedding is not sufficient to distinguish synthesized faces from photographed faces, and that the learned embeddings are required to extract sufficiently descriptive representations.

Our low-dimensional embedding models are trained on StyleGAN, StyleGAN2, and StyleGAN3 faces. We test generalization of the models on photos created using Generated.photos and Stable Diffusion. Generated.photos are GAN-synthesized faces generated using a network trained on a proprietary dataset of tens of thousands of high-quality images recorded in a photographic studio. Our approach does somewhat generalize to the GAN-based Generated.photos faces, but it does not generalize to Stable Diffusion faces. The latter is not surprising, because the diffusion-based process does not rely on the same type of training from aligned faces as the GAN-based process. We also report results on the robustness of our technique to geometric transformation attacks. We show that our approach is somewhat resilient to these transformation attacks.

True positive rate (TPR) is the percentage of synthetic photos that are correctly classified as synthetic. False positive rate (FPR) is the percentage of real photos that are incorrectly classified as synthetic. Our approach is able to detect 99.6% (TPR) of synthetic StyleGAN, StyleGAN2, and StyleGAN3 faces, while only incorrectly classifying 1% (FPR) of real LinkedIn profile photos as synthetic. For the benchmark results in our research paper we chose a 1% FPR target, because–for real-world applications on a large professional network–it is important for AI-generated image detection models to catch most of the synthetic images, while only rarely classifying a real image as synthetic.

We compare our technique to a state of the art CNN-based image-forensic classifier from the academic literature; our approaches outperform the state-of-the-art CNN model. At least one reason for this may be that the CNN-based classifier was trained to detect a synthesized image from any category, whereas we focus exclusively on faces. We also see that while the CNN classifier is somewhat able to detect StyleGAN1 and StyleGAN2 images, it struggles significantly on the most recent StyleGAN3 images.

Please see the full paper for a complete quantitative discussion of our results.

Advertisement

Conclusions

In our research, we have developed a method to distinguish GAN-generated faces from real faces. We found that a simple model can effectively accomplish this task at a level of performance meeting or exceeding state-of-the-art CNN-based approaches in the academic literature. Our approach takes advantage of the fact that GAN-generated faces have consistent characteristics due to the way they are trained on cropped and aligned faces. We validated our technique on a large sample of synthesized faces spanning five different synthesis engines and real LinkedIn member profile photos to understand the real-world performance.

This cutting-edge research helps LinkedIn continue to improve and increase the effectiveness of our automated anti-abuse defenses to help detect and remove fake accounts before they have a chance to reach our members and customers.

Acknowledgements

This work is the product of a collaboration between Professor Hany Farid and the Trust Data team at LinkedIn. We thank the LinkedIn Scholars program for enabling this collaboration. We also thank Ya Xu, Daniel Olmedilla, Kim Capps-Tanaka, Jenelle Bray, Shaunak Chatterjee, Vidit Jain, Ting Chen, Vipin Gupta, Dinesh Palanivelu, and Milinda Lakkam for their support of this work. We appreciate Siddharth Dangi and Bharat Jain for their valuable technical input while reviewing the paper. We are grateful to NVIDIA for facilitating our work by making the StyleGAN generation software, trained models, and synthesized images publicly available, and for their valuable suggestions.