Facial recognition & political inference — Codex v1 | Budots Media
Ghost in the MachineBudots Media · Research series
Research file 001 / Facial recognitionPublished research + simulation
Political orientation from photos

Facial recognition& politicalinference

Published studies find a statistical signal across groups. The controlled-photo result explains about 5% of the variation in political orientation.

Explore what those effect sizes imply under a simple simulation — and how much uncertainty remains.

Examine the evidence Original edition ↗
01 / Study explorer

Controlled lab photos · 2024

59%
Simulated accuracy
59 correct / 41 wrong
per 100 balanced cases
50 chance baseline9 beyond chanceWrong
Illustration from summary statistics. Equal-size groups, equal-variance Gaussian scores. Read the assumptions ↓
591

Controlled photographsNeutral expression, hair back, no makeup.

r = .22

Reported correlationAlgorithm score vs political orientation.

~5%

Variance explainedFor the controlled-photo result, r² = .0484.

01 The published results

Three settings.
Different signals.

The studies report different metrics. The last column translates them using the project's simulation assumptions so they can be compared on one scale.

Published metrics and model-derived accuracy estimates
PhotographsStudy sampleReportedSimulated
Everyday photos2021 · Scientific Reports ↗1,085,795 photosDating sites + Facebook.72AUC · pairwise ranking66%34 wrong per 100
Controlled lab photos2024 · American Psychologist ↗591 participantsControlled presentation.22r · correlation59%41 wrong per 100
Politicians' photos2024 · American Psychologist ↗3,401 politiciansExternal photograph sample.13r · correlation55%45 wrong per 100

The percentages are rounded outputs of the project's Gaussian conversion, not measured classification accuracies from these experiments. AUC measures ranking across pairs; it is not the percentage of individual people labelled correctly.

4.84 percent of variance explainedA ring chart of the controlled-photo correlation squared: 4.84 percent explained and 95.16 percent unexplained by this result. 4.84%
Variance
explained

Controlled lab photos
r² = .22 × .22
4.84% explained
95.16% unexplained

02 Reading the numbers

A group effect
leaves uncertainty.

A correlation of .22 does not mean 22% accuracy. Squaring it gives about 5% of variance explained. The simulation then models how overlapping score distributions affect a binary decision.

Even in this balanced scenario, the lab-photo estimate produces about 41 incorrect calls per 100. It does not establish a person's politics from their photo.

Simulation method and assumptions

The project assumes two equally common groups with normally distributed scores and equal variance. It converts r to d using d = 2r / √(1 − r²), then accuracy = Φ(d / 2). For AUC, d = √2 × Φ⁻¹(AUC). Φ is the standard normal cumulative distribution function.

The 100 cells illustrate rounded expected counts; they are not observations, identities, or individual predictions. These model assumptions do not validate use on new populations or CCTV footage.

Inspect the project simulation ↗

03 Interpreting the signal

What a photo
contains

The project's review distinguishes facial structure from presentation, demographics and photographic conditions. A model can use all of them.

01 / Presentation

Head pose

The 2021 study reported differences in how directly people faced the camera.

02 / Presentation

Expression

Expression was associated with the signal in everyday photos. The lab study used neutral expressions.

03 / Presentation

Grooming

Hair, glasses, makeup and facial hair can carry information about self-presentation.

04 / Population

Demographics

Adding demographic variables raised the 2024 lab result from r = .22 to r = .31.

05 / Measurement

Photo conditions

Lighting, framing and platform style are part of an image. They are distinct from facial structure.

06 / Study design

Sample selection

Dating-site photos, controlled lab portraits and politicians' photos represent different settings.

See the political-orientation studies and critiques below for the underlying research.

04 The wider research

Identity, traits
& targeting

The complete report covers related trait studies, advertising, surveillance, critiques and the regulatory record.

Research context

Other trait claims

The review covers personality, sexual orientation and the effects of demographic stereotypes in vision-language models, with study limitations and critiques.

Browse the grouped research ↓
Deployment context

Digital profiling

The report examines digital-footprint targeting, Cambridge Analytica, face-search systems and companies marketing trait classifiers.

Read the complete report ↗
05 References

Inspect the sources

Research compiled on 24–25 October 2026. This iteration reuses the project's source list and preserves its verification labels.

Expand a topic for papers, official records and critical responses. The bibliography records which references were verified, partly verified or still need checking.

Annotated bibliography ↗
Political orientation from faces · 5
  1. Kosinski, M. (2021). Facial recognition technology can expose political orientation from naturalistic facial images. Scientific Reports 11, 100. doi — 1,085,795 dating-site and Facebook photos (US, UK, Canada); pairwise AUC .72, human raters .55.
  2. Kosinski, M., Khambatta, P., & Wang, Y. (2024). Facial recognition technology and human raters can predict political orientation from images of expressionless faces even when controlling for demographics and self-presentation. American Psychologist 79(7), 942–955. doi · arXiv — 591 lab photos: algorithm r = .22, humans r = .21, r = .31 with age, gender and ethnicity; 3,401 politicians r = .13.
  3. Rasmussen, S. H. R., Ludeke, S. G., & Klemmensen, R. (2023). Using deep learning to predict ideology from facial photographs. Scientific Reports 13, 5257. doi — 3,323 Danish candidates, 61% accuracy. Predictions tracked smiling and, for women, attractiveness; the model also used areas outside the face.
  4. Lindholm, A., Hjorth, F., & Schuessler, J. (2025). Facial finetuning: using pretrained image classification models to predict politicians' success. Political Science Research and Methods 13(4), 1031–1041. doi — 7,080 Danish politicians; model scores for attractiveness and trustworthiness correlate with votes.
  5. Rule, N. O., & Ambady, N. (2010). Democrats and Republicans can be differentiated from their faces. PLoS ONE 5(1), e8733. doi — human judges above chance on candidate photos.
Personality from faces · 10
  1. Kachur, A., Osin, E., Davydov, D., Shutilov, K., & Novokshonov, A. (2020). Assessing the Big Five personality traits using real-life static facial images. Scientific Reports 10, 8487. doi — 12,447 volunteers, 31,367 photos; mean r = .24 with self-reports. Three of the five authors worked for the company that supplied the data.
  2. Guenzel, M., Kogan, S., Niessner, M., & Shue, K. (2026). AI personality extraction from faces: Labor market implications. NBER Working Paper 34808. NBER — LinkedIn photos of about 96,000 MBA graduates; "Photo Big 5" scores (Kachur model) predict pay, seniority and job moves. The authors warn against use in hiring.
  3. Hu, S., Xiong, J., Fu, P., et al. (2017). Signatures of personality on dense 3D facial images. Scientific Reports 7, 73. doi — 3D scans of 834 Han Chinese adults (CAS-MPG Shanghai, Fudan); face shape explains 0.67–3.95% of personality variance.
  4. Xu, J., Tian, W., Lv, G., Liu, S., & Fan, Y. (2021). Prediction of the Big Five personality traits using static facial images of college students with different academic backgrounds. IEEE Access 9, 76822–76832. doi — Northwestern Polytechnical University, Xi'an; 13,347 student photo–questionnaire pairs; "above 70%" accuracy on binned classes, not comparable with correlations.
  5. Cai, L., & Liu, X. (2022). Identifying Big Five personality traits based on facial behavior analysis. Frontiers in Public Health 10, 1001828. doi — Institute of Psychology, CAS; facial movement in 30-second videos, N = 82, r = .39–.48, no independent test sample.
  6. Jaeger, B., Sleegers, W. W. A., Stern, J., Penke, L., & Jones, A. L. (2024). Testing perceivers' accuracy and accuracy awareness when forming personality impressions from faces. European Journal of Personality 38(6), 907–927. doi — preregistered: human face impressions of personality are not accurate, and people overestimate their accuracy.
  7. Foo, Y. Z., Sutherland, C. A. M., Burton, N. S., Nakagawa, S., & Rhodes, G. (2022). Accuracy in facial trustworthiness impressions: Kernel of truth or modern physiognomy? A meta-analysis. Personality and Social Psychology Bulletin 48(11), 1580–1596. doi — face-level r = .14, "unlikely to be of practical utility".
  8. Hinds, J., & Joinson, A. (2024). Digital data and personality: A systematic review and meta-analysis. Psychological Bulletin 150(6), 727–766. doi — benchmark across all digital footprints: computer prediction pooled ρ = .30 (42 studies).
  9. Kosinski, M. (2017). Facial width-to-height ratio does not predict self-reported behavioral tendencies. Psychological Science 28(11), 1675–1682. doi — a null result from Kosinski himself.
  10. 刘晨, 温芳芳, 佐斌 (2019). 以貌取人可行吗？——基于面孔的特质推理及准确性. 《心理科学》42(1), 150–156. jps.ecnu.edu.cn中文 — Central China Normal University review: face-based trait inference has "moderate accuracy" that depends on target, perceiver, culture and context.
Sexual orientation from faces · 7
  1. Wang, Y., & Kosinski, M. (2018). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology 114(2), 246–257. doi — 35,326 dating-site images; pairwise accuracy 81% (men) and 74% (women) from one image.
  2. Leuner, J. (2019). A replication study: Machine learning models are capable of predicting sexual orientation from facial images. arXiv 1902.10739 — 20,910 new dating-site photos: 68% / 77%. A model on blurred photos still reached 63% / 72%, so much of the signal is not facial structure.
  3. Agüera y Arcas, B., Todorov, A., & Mitchell, M. (2018). Do algorithms reveal sexual orientation or just expose our stereotypes? Medium. archive — glasses, makeup, facial hair and selfie angle differ between groups.
  4. Gelman, A., Mattson, G., & Simpson, D. (2018). Gaydar and the fallacy of decontextualized measurement. Sociological Science 5, 270–280. doi — the classifier measures self-presentation on a dating site, not orientation in the population.
  5. GLAAD & Human Rights Campaign (2017). Joint statement on the Stanford study.unverified — called it "dangerous and flawed"; the primary page was not reachable, quotes come from press reports.
  6. 积木 (2017). 看脸的世界？通过面部识别技术，AI 可以了解许多个人信息. 爱范儿. ifanr.com中文 · 方嘉文 (2017). 你以为人工智能看出了"直"或"弯"？它只是看出了你的偏见. 爱范儿. ifanr.com中文 — Chinese coverage of the study and of the Western critiques.
  7. 柯鸣 (2018). 性取向被识别后，AI伦理的边界又在哪里？ 36氪. 36kr.com中文 — commentary on AI ethics after the study, with a Sun Yat-sen University relationship-recognition model.

No peer-reviewed replication on a new population (2020–2026) and no test of vision-language models for sexual orientation was found.

AI models and stereotypes · 3
  1. Jeon, S., Lee, M. H. J., Montgomery, J. M., & Lai, C. K. (2026). From faces to politics: Vision-language models (sometimes) link visual demographic characteristics to ideological labels. Political Analysis 34(4), 613–621. doi — GPT-4o, GPT-4o-mini and LLaVA rated identical ads with women as more liberal (b = .41 against a survey gap of .15); Gemini refused to rate faces.
  2. Hausladen, C. I., Knott, M., Camerer, C. F., & Perona, P. (2025). Social perception of faces in a vision-language model. FAccT '25, 639–659. doi · arXiv — CLIP's judgments shift with expression more than with age, and with lighting about as much as with age.
  3. Peterson, J. C., Uddenberg, S., Griffiths, T. L., Todorov, A., & Suchow, J. W. (2022). Deep models of superficial face judgments. PNAS 119(17), e2115228119. doi · One Million Impressions — models how people judge synthetic faces.
Critiques and the history of physiognomy · 7
  1. Agüera y Arcas, B., Mitchell, M., & Todorov, A. (2017). Physiognomy's new clothes. Medium. summary
  2. Stark, L., & Hutson, J. (2022). Physiognomic artificial intelligence. Fordham Intellectual Property, Media & Entertainment Law Journal 32(4). Fordham — defines "physiognomic AI" and argues for legal bans.
  3. Andrews, M., Smart, A., & Birhane, A. (2024). The reanimation of pseudoscience in machine learning and its ethical repercussions. Patterns 5(9), 101027. doi — discusses Kosinski 2024, Wang & Kosinski 2018, Wu & Zhang 2016 and Kachur 2020.
  4. Todorov, A., Olivola, C. Y., Dotsch, R., & Mende-Siedlecki, P. (2015). Social attributions from faces. Annual Review of Psychology 66, 519–545. doi · Todorov, A. (2017). Face Value. Princeton University Press.
  5. Wu, X., & Zhang, X. (2016). Automated inference on criminality using face images. arXiv 1611.04135 — Shanghai Jiao Tong University; ID photos of convicted men against web photos of others, a sampling confound. Version 3 (2017) is the authors' reply to critics.
  6. 虞涵棋, 杨漾 (2017). 谷歌研究员质疑"看脸识罪犯"，交大教授撰文：我被扣帽子了. 澎湃新闻. thepaper.cn中文 — Wu Xiaolin's response; reports 1,856 photos and 89% claimed accuracy.
  7. Eckles, D., Gordon, B. R., & Johnson, G. A. (2018). Field studies of psychologically targeted ads face threats to internal validity. PNAS 115(23). doi — Facebook's ad delivery means the Matz 2017 arms were not randomised comparisons.
Cambridge Analytica and psychographic targeting · 14
  1. Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. PNAS 110(15), 5802–5805. doi — Facebook Likes predict personality, politics and sexual orientation.
  2. Youyou, W., Kosinski, M., & Stillwell, D. (2015). Computer-based personality judgments are more accurate than those made by humans. PNAS 112(4), 1036–1040. doi
  3. University of Cambridge Psychometrics Centre. myPersonality. psychometrics.cam.ac.uk — "no longer active and data is no longer being shared"; over 6 million volunteers until 2012.
  4. Information Commissioner's Office (2018). Investigation into the use of data analytics in political campaigns. ICO (PDF) — up to 87 million Facebook users; Stillwell refused Cambridge Analytica access to myPersonality, so Kogan built his own app; £500,000 penalty for Facebook.
  5. Denham, E. (2020). Letter to the DCMS Committee on the ICO investigation. ICO (PDF) — the models were "well recognised processes using commonly available technology"; real-world accuracy on new people "was likely much lower"; GSR data could not have been used in the Brexit referendum.
  6. House of Commons DCMS Committee (2019). Disinformation and 'fake news': Final Report, HC 1791. PDF · Oral evidence of C. Wylie, 27 Mar 2018, HC 363. parliament.ukpartly verified
  7. US Federal Trade Commission (2019). Final settlement with Cambridge Analytica's former CEO and the app developer. ftc.gov · $5 billion penalty for Facebook. ftc.gov — the harvested data and derived models had to be deleted.
  8. Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. J. (2017). Psychological targeting as an effective approach to digital mass persuasion. PNAS 114(48), 12714–12719. doi — 3.5 million people reached; up to 40% more clicks and 50% more purchases.
  9. Tappin, B. M., Wittenberg, C., Hewitt, L. B., Berinsky, A. J., & Rand, D. G. (2023). Quantifying the potential persuasive returns to political microtargeting. PNAS 120(25), e2216261120. doi — targeting on more than one attribute added nothing; gains may not need large amounts of personal data.
  10. Simchon, A., Edwards, M., & Lewandowsky, S. (2024). The persuasive effects of political microtargeting in the age of generative AI. PNAS Nexus 3(2), pgae035. doi · Matz, S. C., et al. (2024). The potential of generative AI for personalized persuasion at scale. Scientific Reports 14, 4692. doi
  11. Gibney, E. (2018). The scant science behind Cambridge Analytica's controversial marketing techniques. Nature. doi · Allen, J., & Abbruzzese, J. (2018). Cambridge Analytica's effectiveness called into question. NBC News. nbcnews.com
  12. Bakir, V. (2020). Psychological operations in digital political campaigns: Assessing Cambridge Analytica's psychographic profiling and targeting. Frontiers in Communication 5, 67. doi · Hersh, E. D. (2015). Hacking the Electorate. Cambridge University Press. doi
  13. Lomas, N. (2020). Latest Cambridge Analytica leaks ("Hindsight Files"). TechCrunch. techcrunch.com — internal documents released by Brittany Kaiser; no public index found.
  14. DeepTech深科技 (2018). "剑桥分析"究竟如何通过"点赞"来影响投票倾向？ 安全内参. secrss.com中文 — Chinese explainer; notes that evidence the method swung elections is lacking.

Data. Public: the ICO, FTC and DCMS documents above. Not public: the profiles harvested by Kogan's app (deletion ordered by the FTC; seized copies destroyed by the ICO) and the myPersonality dataset (sharing stopped in 2018). This project links to the official record and does not use leaked copies.

Who builds it: companies and labs · 9
  1. US Treasury (10 Dec 2021). Sanctions on SenseTime. treasury.gov — says its software "can determine a target's ethnicity, with a particular focus on identifying ethnic Uyghurs". SenseTime called the accusations unfounded.
  2. US Treasury (16 Dec 2021). Chinese military-industrial complex companies, incl. Megvii, Yitu, CloudWalk. treasury.gov
  3. IPVM (2020). Huawei / Megvii "Uyghur alert" test report. ipvm.com — Huawei: "simply a test"; Megvii: not designed to label ethnic groups.
  4. Rollet, C. (2020). Uniview Uyghur recognition. IPVM. ipvm.com — an SDK ethnicity code listing only Uyghurs; also cites Hikvision (2019) and Dahua (2020).
  5. Asher-Schapiro, A. (2021). Chinese tech patents tools that can detect, track Uighurs. Thomson Reuters Foundation. context.news — Huawei + CAS, Megvii and Alibaba patent filings; company replies included.
  6. Wang, C., Zhang, Q., Liu, W., Liu, Y., & Miao, L. (2019). Facial feature discovery for ethnicity recognition. WIREs Data Mining and Knowledge Discovery 9(1), e1278. doi · IPVM (2021) on its ethics. ipvm.com — Dalian Minzu University with Curtin University; 300 Uyghur, Tibetan and Korean students; Expression of Concern, no retraction registered.
  7. Mozur, P. (2019). One month, 500,000 face scans: How China is using A.I. to profile a minority. The New York Times. nytimes.compartly verified
  8. Human Rights Watch (2017). China: Voice biometric collection threatens privacy. hrw.org — iFlytek; voice, not face.
  9. Globes (2016). Israeli co Faception claims it can spot terrorists. globes.co.il — 15 "classifiers" at a claimed 80%, no published validation.
Law: EU, China, US · 5
  1. Regulation (EU) 2024/1689 (AI Act), Art. 5(1)(g). EUR-Lex — bans biometric categorisation that infers race, political opinions, religion, sex life or sexual orientation.
  2. 《中华人民共和国个人信息保护法》 (2021), Art. 28–29. cac.gov.cn中文 — biometrics and religious belief are sensitive personal information; separate consent required. No named ban on inferring politics.
  3. 最高人民法院关于审理使用人脸识别技术处理个人信息相关民事案件适用法律若干问题的规定, 法释〔2021〕15号. court.gov.cn中文 — face verification, identification or analysis in public places in breach of law infringes personality rights.
  4. 国家互联网信息办公室、公安部《人脸识别技术应用安全管理办法》 (2025), in force 1 June 2025. cac.gov.cn中文 — Art. 2(2) exempts R&D and algorithm training; filing required above 100,000 people.
  5. US: Illinois BIPA, Texas CUBI, California CPRA, FTC enforcement (Rite Aid, 2023).from memory
Bart Sakwerda, project owner
Principal investigator / Project owner

Bart Sakwerda

Budots Media Philippines. Original research workflow, literature survey and simulation: Claude Opus 5.5. Design and interactive study comparison for this iteration: ChatGPT Codex.

v1 / 27 Oct 2026