i let three ais guess my politics and they all flopped the same exact way

did the real political compass test first so i'd have an actual baseline instead of vibes. -1.5 economic, -2.5 social. mild left, mild libertarian, basically the political equivalent of ordering a medium coffee. got the certificate to prove it too, dated this week, red dot sitting near dead center of the left-libertarian quadrant while corbyn and aoc are way off in the distance doing their own thing. mamdani's on that chart too actually, also left-libertarian, and unlike most politicians the guy is genuinely just doing what he said he'd do (rent freeze got approved this week, the guy promised free buses and city grocery stores and is out here actually building them). meanwhile the actual president of the united states is somewhere in authoritarian-right doing the opposite of what an adult should do with power. anyway. not the point of this post but i'm not from nyc, i'm from moldova, and i'd still take that guy as my mayor over what we've got going on.

ok actual point: i gave gemini, claude, and chatgpt the exact same prompt, word for word, copy pasted. generate 10 statements, 5 economic 5 social, polarizing enough to isolate someone's leanings, wait for my answers, then give coordinates and a quadrant. zero leading, same instructions all three times.

every single one overshot. badly.

| | economic | social | quadrant | |---|---|---|---| | actual cert | -1.5 | -2.5 | left-libertarian | | gemini 3.5 flash | -7.5 | -6.0 | left-libertarian | | claude sonnet 4.6 | -3.0 | -7.0 | left-libertarian | | chatgpt (gpt-5.5) | -4.5 | -6.5 | left-libertarian |

all three nailed the quadrant. cool, except the quadrant is one of four boxes, so that's like bragging you guessed the right continent. the actual numbers are where it falls apart, especially on social, where every model was at least double my real score and gemini was almost triple.

exhibit a: the questions are doing the cheating

gemini's question 2, copy pasted directly: "essential services like healthcare, utilities, and public transport should be entirely state-owned and run for public good rather than profit." i said strongly agree because i think hospitals shouldn't be a profit center, not because i want the moldovan government nationalizing my internet provider. there's no answer option between "strongly agree" and "disagree" for "agree with the vibe, not the absolutism." so gemini got my strongest possible answer to its most extreme possible phrasing and then acted shocked, shocked, that the math came out extreme. that's not a result, that's a leading question doing exactly what leading questions do.

claude's set was the least broken of the three. it actually included a tension pair, q1 ("tax the wealthy") against q2 ("free markets with minimal interference, agree"), and i agreed with both, which is a perfectly normal thing a person can think. claude even flagged it itself: "items 1 and 2 pull in opposite directions, that's a real pattern, not noise." that's the one moment in this whole experiment where a model showed actual reasoning instead of just adding up agrees. it's also why claude landed closest to my real economic score, -3 against my actual -1.5, while gemini and chatgpt overshot to -7.5 and -4.5.

social axis though, all three fall apart the same way. chatgpt's question 7 was "people should be free to express even highly offensive or unpopular opinions, unless they directly incite violence," which is just. the actual aclu position. i agreed, normally, like a person who has heard of free speech before. chatgpt logged it as part of "a fairly clear pattern toward personal liberty" and helped drag my score to -6.5. agreeing with mainstream civil liberties shouldn't read as a maximalist position, but when every option on the scale is phrased like a manifesto, "agree" becomes "strongly agree" becomes "you must be an anarchist."

exhibit b: three competitors, one answer

gemini, anthropic, and openai are not the same company. different training runs, different alignment teams, presumably trying pretty hard to not be identical products. and yet all three independently guessed left-libertarian, all three overshot on social specifically, and two of three overshot economic by 2-3x too. that's not three companies confirming a result, that's three models that learned to talk from the same pile of internet text all inheriting the same idea of what "agreeing with personal freedom stuff" should be worth in points. there's actual research backing this exact pattern too, multiple independent studies running political compass tests across dozens of LLMs keep finding the same left-libertarian clustering regardless of who built the model, with gemini specifically noted in at least one analysis as one of the more centrist outliers compared to the pack. so even the "which model is most neutral" conversation has already been had, and the answer keeps not being any of these three.

exhibit c: read the actual reasoning, it's just vibes with extra steps

go back and read what claude wrote for my social score: "this one's unanimous and consistent, every answer points the same direction." that's it. that's the whole methodology. five agrees in the same direction equals -7 out of -10, near the absolute max of the scale, because consistency got treated as intensity. i wasn't extreme, i was just not contradicting myself, which is a low bar that apparently reads as ideological commitment to a language model.

there's no scoring rubric running underneath any of this. no weights, no calibration against a real dataset, nothing political compass dot org would recognize as math. it's autocomplete looking at a stack of "agree"s and going "well that's a lot of agreeing, better make the number bigger," then writing up the decision afterward to sound like analysis. (-7.5, -6.0) isn't a measurement. it's a vibe that learned to format itself like one.

meanwhile the actual test, the one with 62 real questions built by people who do this for a living, got me at -1.5/-2.5 on the first try, no drama, no overshoot, no model logic paragraph trying to justify itself. sometimes the boring methodology wins.