How ChatGPT, Claude, and Gemini View Colleges Differently

ChatGPT, Claude, and Gemini grade the same college paycheck differently—wrapping equal-earning public schools in coded caveats. See the data across 698 schools.

Daniel Rogers
Updated 22 min read
Split editorial illustration: a student surrounded by human advisors (counselor, uncle, coach) on the left, and a student surrounded by three glowing AI computer screens on the right.
Illustration: College Azimuth

This week we're going to walk through some of the findings I've uncovered as I've started looking into AI, and how it works for college and career advice.

To set the stage, I'll lay out how I think about the information flow in these decisions. For decades, a student figuring out what comes next worked three channels:

  1. A counselor or other trained professional who may or may not know what the best pathways are for a particular student, and even if they did, may have hundreds of students to deal with.
  2. Their social network: their family, a friend, a friend of a friend, or whomever in their lives had been to college, or worked a job themselves. The value of this advice being tied directly to the experiences of these individuals.
  3. The internet, where a search engine pointed everyone to the same handful of incumbent sites. Time and effort was put into curated sources, and information through whichever lens the writer cared about it. Over time this has leaned heavily toward "prestige."

As I've written before, AI fundamentally shifts this workflow in ways we don't quite understand, with 46% of college-bound high schoolers now using AI in their college search—nearly double the share six months earlier.1 So I thought it would be helpful to examine up front how AI actually answers the basic question of "is this school good?"

We put the same four parent questions—"Is X a good school?", "Is X worth it?", "What is X known for?", and "How does X compare to Y?"—via API with web search turned on to ChatGPT (gpt-5.6-sol), Claude (claude-sonnet-5-5), and Gemini (gemini-3.1-pro-preview) in October and November 2026.

To see whether the exam changes with who a school enrolls, we ran those questions across 698 four-year schools (including broad-access institutions from the 2025 Carnegie Student Access and Earnings Classification)2 and 46 matched pairs to compare how models talked about schools with similar earnings profiles.

One of the main risks of AI in college and career advice, as I see it, is that people think of it as this benevolent and all-knowing entity. But what if it's not?

What if I told you that different AIs have different personalities, frameworks, biases, and quirks for how they think about colleges—just as distinct as your counselor, your uncle, and the volleyball coach from around the way?


Three Engines, Three Personalities

How ChatGPT, Claude, and Gemini grade colleges differently:

ChatGPT searches official .edu and accreditation records most often, Claude relies heavily on rankings and aggregators, and Gemini pulls from rankings and forums like Reddit.
Across 46 pairs of colleges with equal earnings, all three wrap lower-income public schools in more prestige-limiting caveats.

Ask your counselor, your uncle, and the volleyball coach from whether a college is good, and you get three different conversations shaped by what each one cares about and where they get their information.

ChatGPT, Claude, and Gemini behave the same way. When we tagged every answer across 13 evaluation lenses and logged every web search each model ran across 478 broad-access Opportunity Colleges and the 200 most selective four-year schools, three distinct personalities showed up—and every one of them changed what it checked when the school changed (more on that later).

Here's what we saw for each of the engines, and what we took away from it.

1. ChatGPT: The Particular Uncle who checks the facts first.

ChatGPT is the uncle who hears you're looking at a school and goes and looks it up. Across 1,912 answers about 478 broad-access colleges, it ran the most searches—averaging 3.57 per answer.

And he goes straight to primary sources. The school's own website appears in 95.4% of answers, and official .edu, .gov, and accreditor domains make up 74.8% of everything it cites (with virtually no Wikipedia at 0.1% and 0.0% Reddit). We also found that after running earnings searches two-thirds of the time, ChatGPT puts earnings in fewer than a third (28.7%) of its answers. The uncle did the homework and left the number in the notebook, just in case. What you get is a warm verdict—on the 46 earnings-matched pairs we test next, it opened with a flat "Yes" about nine times in ten (91.3% for publics and 84.8% for privates).

ChatGPT persona illustration: skeptical uncle at the kitchen table with magnifying glass, accreditation ledger, and laptop
Illustration: College Azimuth

The problem is an inherent skepticism of "non-elite" schools. Language reassuring the reader that a broad-access college is "legitimate" appeared in 38.1% of openings (first 400 characters), compared with 5.5% for selective schools (51.0% vs. 13.0% across the full answer). Career training (73.6% vs. 38.1%), transfer pathways (33.3% vs. 1.7%), and institutional health (28.1% vs. 7.3%) received far more attention too.

Ask about one of the 200 most selective schools (769 answers) and accreditation searches drop to 9.2% and earnings searches to 27.6%. Same question, different school—and suddenly the model assumes the school already cleared the bar. Even when ChatGPT sticks to official .edu and .gov pages instead of rankings (usnews.com appears in 14.8% of its broad-access answers), the UICs of the world still get vetted for legitimacy while no one questions whether Vanderbilt is good.

ChatGPT (gpt-5.6-sol): search, answer lenses, and sources Broad-access (n=1,912) Selective 200 (n=769)
Mean web searches per answer 3.57 2.67
Searches: Programs / Cost 80.4% / 72.0% 47.9% / 59.4%
Searches: Earnings / Completion / Accreditation 67.3% / 60.8% / 59.2% 27.6% / 33.6% / 9.2%
Talks about: Career training / Transfer / Inst. health 73.6% / 33.3% / 28.1% 38.1% / 1.7% / 7.3%
Talks about: Cost / Earnings / Prestige & rank 76.2% / 28.7% / 24.1% 72.6% / 36.4% / 59.0%
Opens with "legitimate" (≤400 chars, "Is X a good school?") 38.1% (n=478) 5.5% (n=200)
Citations (broad-access): .edu / Accreditor / .gov / Reddit 95.4% / 16.5% / 14.1% / 0.0% (74.8% official domains)

2. Claude: The Counselor Who Relies on what he knows already

Claude is the counselor who has had this conversation a thousand times and doesn't feel the need to open the file. It runs the fewest searches—1.35 per answer on broad-access schools (1,912 answers) and 1.03 on selective schools (792 answers)—yet puts more outcome numbers on the page than the others. More than any other engine, Claude leans on what ranking sites say and rarely checks those claims against official government sources.

Most of what it tells you comes from somebody else's summary: 88.1% of broad-access answers cite an aggregator (niche.com at 54.9%), nearly half (47.8%) cite a rankings site (usnews.com at 45.1%), and 18.2% across all four questions cite a student lender (sallie.com, 2.5% on "Is X a good school?"). The school's own site shows up in 43.0% of answers—less than half ChatGPT's rate—and a federal or accreditor page in just 5.8%. Only about one in ten cited links (10.5%) is an official source.

Claude persona illustration: counselor behind stacked college guidebooks and a checklist with an unopened file beside them
Illustration: College Azimuth

However, Claude doesn't verify those figures against primary files. Earnings appear in 50.5% of broad-access answers (65.5% on "Is X a good school?"), even though Claude searched for earnings in only 18.5%. It grabs whatever number sits on the summary page and fills gaps from memory, leaving the reader with no way to separate a live lookup from a remembered impression.

The Urban Institute recently released a paper which highlights why this is problem: without structured access to education data, only 16% of model answers came back with no accuracy problems; hand the model the actual data and that jumped to 61%.3 Claude's manner makes the gap harder to spot because it sounds careful: in 71.8% of answers across all four questions, it includes advice telling the student to go verify graduation rates, check net prices, or compare offers for themselves. That's the posture of someone who did the homework, from the engine that did the least of it.

And the verdict—plus the care behind it—depends heavily on who Claude is talking about. Ask "Is X a good school?" about the 478 broad-access schools (which span two-year and four-year institutions), and Claude opens with a plain "Yes" only 8.2% of the time (39 of 478). Ask the exact same question about the 200 most selective schools, and that plain "Yes" jumps to 71.0% (142 of 200).

At the same time, Claude practically stops researching selective schools: 1.03 queries per answer, programs searched in 1.0%, and prestige in 82.7% (becoming its leading lens at 41%, vs. 8.3% on broad-access schools). Selective schools get a reputation and a clear yes; broad-access schools get a guidebook audit and a hedged opening.

Claude (claude-sonnet-5-5): search, answer lenses, and sources Broad-access (n=1,912) Selective 200 (n=792)
Mean web searches per answer 1.35 1.03
Searches: Cost / Rankings / Programs / Earnings 44.9% / 40.7% / 16.4% / 18.5% 42.2% / 44.9% / 1.0% / 22.9%
Talks about: Cost / Completion / Earnings 77.7% / 69.6% / 50.5% 72.7% / 50.1% / 45.2%
Talks about: Prestige & rank (and share where it leads) 45.6% (8.3% leads) 82.7% (41.0% leads)
Plain "Yes" opening on "Is X a good school?" 8.2% (39/478) 71.0% (142/200)
Citations (broad-access): Aggregators / Rankings 88.1% / 47.8% (niche.com 54.9%, usnews.com 45.1%)
Citations (broad-access): .edu / .gov / Accreditor / Reddit 43.0% / 3.2% / 2.9% / 14.4% (10.5% official domains)

3. Gemini: The Coach Who Just Repeats What It's Heard

Gemini is the coach who hasn't opened the scouting report but has heard plenty around town. It averages 2.88 searches per answer across 700 broad-access responses (2.30 across 792 selective ones), and 60.3% of its broad-access answers search for rankings or reviews—compared with 15.1% for earnings and 11.7% for accreditation.

You see that in what it cites. Aggregators show up in 91.9% of answers (niche.com in 72.0%) and the school's own site in 89.0%, alongside Wikipedia in 39.4% and social forums in 60.0%—led by reddit.com (25.7%), allnurses.com (21.0%), and facebook.com (20.1%). Gemini cites government sources (.gov et al) 18.4% of the time, meaning more than three times as many answers lean on a forum thread as on government data.4

Gemini persona illustration: volleyball coach with whistle, megaphone, and trophy surrounded by campus chatter bubbles
Illustration: College Azimuth

What the coach does with that chatter is tell you where a school stands. Prestige (60.1%) and student mission (55.3%) appear more often than on either competitor, and Gemini mentions debt most often (33.0%). Across the 46 earnings-matched pairs, Gemini led with prestige in 95.7% of private-school answers versus 65.2% for the publics—while opening with a plain "Yes" at roughly even rates (45.7% vs. 52.2%). Advice on how to decide shows up in just 17.3% of its broad-access answers, and when comparing equal-earning pairs head to head, it called the matchup even or fit-dependent 69.6% of the time (64 of 92).

Gemini (gemini-3.1-pro-preview): search, answer lenses, and sources Broad-access (n=700) Selective 200 (n=792)
Mean web searches per answer 2.88 2.30
Searches: Rankings & reviews / Earnings / Accreditation 60.3% / 15.1% / 11.7% 57.4% / 17.2% / 1.0%
Talks about: Prestige & rank / Access & mission 60.1% / 55.3% 91.9% / 25.0%
Prestige leads "Is X a good school?" (main lens / rubric opening) 23.6% / 53.9% 75.0% / 94.5%
Talks about: Cost / Completion / Earnings / Debt 72.0% / 43.9% / 39.3% / 33.0% 59.0% / 24.9% / 38.3% / 28.3%
Citations (broad-access): Aggregators / Wikipedia / Forums & social 91.9% / 39.4% / 60.0% (reddit.com 25.7%, facebook.com 20.1%)
Citations (broad-access): .edu / .gov 89.0% / 18.4% (16.7% official domains; 50.0% third-party)

At selective schools, what Gemini has heard gets louder. Prestige climbs from 60.1% to 91.9%, and jumps from being the main lens leading 23.6% of "good school" answers to 75.0% (appearing in 94.5% of openings). Talk about who the school serves drops from 55.3% to 25.0%. The mission and mobility framing turns out to be something Gemini applies to broad-access colleges, not a consistent standard across the board.

ChatGPT checks credentials. Claude checks outcomes. Gemini checks prestige and Reddit.
Across 478 broad-access colleges, ChatGPT checks accreditation and career training, Claude checks completion and earnings via aggregators, and Gemini checks rankings and Reddit.

Same Paycheck, Different Vocabulary

In this next section we were curious whether these patterns persisted even when the outcomes were more or less the same. So we built 46 matched pairs where the earnings data were comparable while the inputs were different—a high-Pell public university paired against a lower-Pell private school in the same state or region.

The point was to see whether how an AI describes a college is driven by the work the school does getting students into jobs, or by who walks through the door. Across these pairs, the vocabulary changed with the student body. You could reliably tell how a model would talk about a school from who enrolled there, even when graduates left with the same paycheck.

Start with two New York schools: Baruch College and Syracuse University.

Audit baseline measure5 Baruch College Syracuse University
Median earnings, 8 years after entry $70,543 $70,598
Students receiving Pell Grants 56.9% 17.1%

Fifty-five dollars separated their median earnings. Forty percentage points separated their Pell shares. A public business-heavy institution in Manhattan and a private residential university in upstate New York are obviously different campuses, yet on the outcome families usually care most about, graduates land in the same neighborhood.

Here is how the three engines answered "Is X a good school?" for each one:

Engine On Baruch ($70,543 / 57% Pell) On Syracuse ($70,598 / 17% Pell)
ChatGPT "It is a large commuter school, so the experience is less like a traditional residential college with a self-contained campus... does not have the same broad national prestige as an Ivy League or highly selective flagship." "Syracuse has a large, active alumni network and reports a 91% positive career-outcome rate for its 2021–2025 graduating classes... lively traditional college experience."
Claude "Expect a commuter-style, urban, non-traditional campus... Elite finance recruiting can be harder from Baruch than from those schools, though many students do succeed." "Bottom line: Syracuse is a strong choice if you're interested in communications, journalism, public policy, or a big-school atmosphere with a strong alumni network."
Gemini "It is a Commuter School: Baruch does not have a traditional grassy campus." "The Maxwell School is highly prestigious and frequently ranks #1... massive, loyal alumni network."

Every engine calls Baruch a good school, and then anchors it to being a commuter school without national prestige. Syracuse gets alumni networks, school spirit, and prestige. Both schools get graduates to the same place, and yet one gets all the caveats while the other gets all the accolades.

Baruch and Syracuse are one of 46 pairs across the country where a high-Pell public four-year (51.8% Pell and $51,702 in average earnings) sits next to a low-Pell private school in the same state or region (20.4% Pell and $52,118 in average earnings, within 15% of each other). Across almost all of these pairs—specifically, the engine framed the public partner through who it enrolls (access and mission across all four questions) more often than its private partner in 44 of 46 pairs on Gemini, 38 on Claude, and 37 on ChatGPT—who walks in the door drives how the school gets talked about, more than who walks out. Flaws get highlighted on one side ("elite finance recruiting can be harder"), while the other side gets handed a prestigious label with no outcomes advantage behind it.

Here is what showed up across all 46 pairs when we asked "Is X a good school?" (each cell shows the high-Pell public share first, followed by its low-Pell private partner):

In answers to "Is X a good school?" (high-Pell public / low-Pell private, 46 pairs) ChatGPT (46/46) Claude (46/46) Gemini (46/46)
Opens with a plain "Yes" 91.3% / 84.8% 30.4% / 34.8% 45.7% / 52.2%
Prestige leads the answer (model-scored) 54.3% / 65.2% 80.4% / 93.5% 65.2% / 95.7%
Told what it is not ("not elite", "limited prestige") 54.3% / 30.4% 56.5% / 34.8% 21.7% / 4.3%
Says "social mobility" 52.2% / 2.2% 63.0% / 17.4% 76.1% / 19.6%
Says "commuter" 45.7% / 0.0% 52.2% / 0.0% 65.2% / 0.0%
Mentions Pell, low-income, or first-gen 28.3% / 0.0% 19.6% / 2.2% 47.8% / 4.3%
Mentions graduation rate 78.3% / 67.4% 54.3% / 52.2% 45.7% / 37.0%
Mentions earnings 8.7% / 19.6% 52.2% / 34.8% 30.4% / 19.6%
Same paycheck, different exam: verdicts hold steady while status labels split
Across 46 pairs matched within 15% on 8-year earnings, plain 'Yes' openings stay close while access-and-mission framing, Pell mentions, and 'not elite' caveats cluster on the high-Pell public schools.

Look at that first row. Across these 46 higher-earning four-year pairs, none of the engines are telling students to skip the public schools—the split lives in the status framing wrapped around the verdict:

  • The opening verdict stays close: ChatGPT opens with a plain "Yes" around nine times in ten for both sides (91.3% vs. 84.8%), while Claude (30.4% vs. 34.8%) and Gemini (45.7% vs. 52.2%) also open comparably on both partners.
  • High-Pell publics get the caveats: They are told what they are not ("not elite" or "limited prestige") far more often—54.3% vs. 30.4% on ChatGPT, 56.5% vs. 34.8% on Claude, and 21.7% vs. 4.3% on Gemini.
  • Low-Pell privates get the prestige: Earning the exact same pay, private partners are led by prestige far more often (65.2% on ChatGPT, 93.5% on Claude, and 95.7% on Gemini) and framed around campus fit.

You can hear that split directly in how the models qualify equal-earning public universities:

ChatGPT on Portland State University (paired with Lewis & Clark College): "It is a legitimate, regionally accredited public research university—not an elite or highly selective institution."
Claude on Cal State Fullerton (paired with Concordia University-Irvine): "That makes it accessible, but it isn't a name that opens doors the way a top-tier school does."
Claude on University of Illinois Chicago (paired with Loyola University Chicago): "It's a solid public research university, though not an elite one."
Gemini on University of Houston-Downtown (paired with Southwestern University): "While a degree from UHD is fully accredited and respected locally, it does not hold the high-tier 'elite' status or expansive alumni networks of schools like UT Austin, Texas A&M, or the UH Main Campus."

Compare that with how the same engines open and frame those exact matched private partners earning the same money:

ChatGPT on Lewis & Clark College (paired with Portland State University): "yes—it’s a good, reputable liberal-arts college, particularly for students who value small classes, close faculty interaction, environmental interests, international affairs, study abroad, and the Pacific Northwest."
Gemini on Southwestern University (paired with University of Houston-Downtown): "Founded in 1840, it is the oldest university in Texas and holds a strong reputation for academic rigor, close faculty interaction, and high post-graduate success."
Gemini on Augustana College (paired with Northern Illinois University): "It holds a chapter of the prestigious Phi Beta Kappa honor society—a distinction only 10% of U.S. colleges have."

Look closely at that Augustana example. Both Augustana and its Illinois matched partner, Northern Illinois University, sit in the same state and admit roughly seven in ten applicants (67.6% at Augustana versus 70.5% at NIU in federal IPEDS records).6

Yet for Augustana (22.3% Pell, $56,882 earnings), Gemini goes looking for a prestige credential anyway, reaching for Phi Beta Kappa to put it in the top 10% of colleges. For NIU (45.6% Pell, $51,305 earnings), Gemini reaches for a demographic frame: "NIU is known for being an unpretentious school that serves first-generation and working-class students." Even when a private partner gets a "not elite" disclaimer—such as Loyola Chicago on ChatGPT or Lewis & Clark on Claude—the rest of the answer still leans on small classes, campus life, and fit rather than commuter logistics.

What's striking to me, having looked at all of this, is how much the models themselves go out of their way to both diminish the opportunity schools and look for reasons to call the others elite.


Stuck at Seventeen and inputs over outputs.

This reminds me a lot of what happened after I left Uber. I had just left a hotshot tech company after getting my MBA from UChicago, and after spending four years as a diplomat doing high-profile, consequential work abroad. I thought, maybe I liked working at Uber—maybe I want to continue working in tech—and set up a couple of interviews with folks. I remember one in particular. I was referred by the husband of an old colleague at the State Department, and I figured we'd spend the interview talking about my recent, salient work. Wrong.

I ended up getting grilled about why I joined the military out of high school. "Surely, if you were smart you'd have just gotten a scholarship. I don't understand why someone who has agency would join the Army," and on and on like that for at least an hour. Nothing I had done since then mattered. The only thing that did was where I was when I was 17.

It seems to be the same thing here.

Someone reading those 46 matched pairs could raise a fair objection: every high-Pell school in the pairs is public, every low-Pell school is private, and the publics are more than three times as large (11,890 students vs. 3,646). Maybe the engines are reacting to public versus private, or campus size, rather than who got in or who enrolls.

So we pooled all 2,419 "Is X a good school?" answers across 698 four-year public and private nonprofit institutions and held sector, enrollment size, admissions rate, and graduate earnings fixed at the same time. Instead of looking at descriptive labels like "commuter" or "social mobility"—where someone could reasonably argue the words just describe the campus—we focused strictly on evaluative judgments: whether the engine opens with a plain "Yes," whether prestige leads the opening, whether it calls the school "not elite," and whether it stops to verify that the school is "legitimate."

Two patterns held up across all 698 colleges, and neither one rewards what happens after a student walks through the door.

First, high earnings don't buy the word "elite." Across all three engines, admissions selectivity is the strongest driver of whether a school gets told it is "not elite." Even when graduates land in the top third of earnings (~$67,000+), schools admitting 70% or more of applicants still hear "not elite" in 42% of ChatGPT answers (n=72) and 43% of Claude answers (n=53), against 7% (n=112) and 12% (n=111) for selective schools admitting under 50% in the same top-earnings tier.

Hold earnings between $60,000 and $80,000 and compare open-admission privates (70%+ admit) with selective privates (<30% admit), and the pattern proves it isn't a public-school penalty: open-admission privates hear "not elite" 58% of the time on ChatGPT (n=50) and 48% on Claude (n=33), versus 0% (n=23) and 4% (n=23) for selective privates. Just like my Uber-era interview, what happened afterward barely moved the label; who got in at 17 carried it.

Second, even with sector, admit rate, campus size, and graduate earnings held fixed, serving lower-income students still lowers the verdict. Every 10 percentage points of Pell share drops the likelihood of a plain "Yes" opening across all three engines: down 4.1 points on ChatGPT, 7.5 points on Claude, and 6.5 points on Gemini (and inside private schools alone, down 7 to 9 points on all three). Every 10 points of Pell share also cuts the chance that prestige leads the answer by 5.9 to 7.9 points, while $10,000 more in graduate earnings moves prestige-led framing by essentially zero (-0.2 to +0.2 points).

On ChatGPT, each 10 points of Pell share also pushes up "not elite" (+4.9 points) and "legitimate" openings (+4.4 points). Among public universities earning $45,000 to $60,000, ChatGPT's "not elite" qualifier climbs from 15% at schools under 25% Pell (n=31), to 53% at 25–45% (n=79), to 66% above 45% Pell (n=42).

High pay does not buy 'elite'—admissions and student income shift the verdict
Across 2,419 answers for 698 four-year schools, open-admission colleges earning top-third pay still get called 'not elite' 42–43% of the time on ChatGPT and Claude, while higher Pell share lowers plain 'Yes' openings and prestige-led framing across all three engines with sector, admit rate, size, and earnings fixed.

Where the Framing Comes From

The strange part is how the engines produce those shifts without ever looking up student demographics. Across 2,419 answers in the pool, almost none of the models' web searches mention Pell Grants, family income, or demographics (0.0% on ChatGPT, 0.4% on Claude, 2.7% on Gemini), and almost no answers state a Pell percentage (0%, 3%, 1%).

Where the framing enters (pooled per-answer checks) ChatGPT (n=1,067) Claude (n=763) Gemini (n=589)
Searches mentioning Pell, income, or demographics 0.0% 0.4% 2.7%
Answers stating a Pell percentage 0% 3% 1%
Answered with no web search at all 26% 2% 1%
"Social mobility" without search (high-Pell 45%+ vs. low-Pell <25%) 30% / 0% — —
"Social mobility" answers naming a ranking on same line 94% (n=129) 97% (n=209) 93% (n=199)
...naming U.S. News specifically on same line 83% 78% 68%

Two channels feed the same reflex. It is baked into model memory—ChatGPT answers 26% of these prompts without running a single search and still splits high- and low-Pell schools (30% / 0%)—and when models do search, they pull pre-sorted college rankings (93%–97% of "social mobility" mentions name a ranking on the same line, mostly U.S. News). The engines rarely check who is actually sitting in the classroom; they inherit separate ladders for prestige and access and grade each school accordingly.


Why Shallow Genteel Bias Matters

When I worked at the State Department, I went to a college happy hour with a colleague of mine. People were standing around socializing, and this one guy started going around the circle asking everyone which college they went to. When he got to me, he stopped and asked: "Did you go to college?"

Mind you, I was standing there in a suit with a State Department badge clipped to it. How he perceived me shaped what he thought of me before I opened my mouth—and even though nothing explicit was said, I could tell from the simple question he asked.

Oftentimes that's how people are forced to navigate bias. It's not "I think you don't belong here"—it's a sly question, or a coded phrase, that gives the game away.

That is what makes this shallow genteel bias hard to spot if you aren't looking for it, and why it matters. Hearing "Sure, it's good, but you know it's not that good" lands very differently on a 17-year-old than hearing "Yes, this is a great school."

Researchers have documented how both halves of that trap work. A 2024 Nature study by Valentin Hofmann and colleagues showed that training models to sound polite and helpful can scrub overt prejudice from the surface while leaving deeper status stereotypes intact underneath—so a chatbot gives an agreeable opening while still sorting people into lower-prestige tracks.7

Decades of rankings research show that students actually move on subtle presentation cues, too. Michael Luca and Jonathan Smith found that adding explicit numerical ranks to U.S. News shifted applications even when the same underlying quality data was already printed on the page, and Andrew Meyer, Andrew Hanson, and Daniel Hickman measured a 2% to 6% drop in applications when a college slipped from #50 to #51—right across the printed guide's first-page cutoff—with no drop in measured quality.8

Today, 46% of college-bound high schoolers use AI in their search, 18% have already dropped a school because of what an AI told them, and only 29.4% always fact-check an answer against a second source.1 When an engine wraps an affirmative verdict in "legitimate" or "not elite" for a college producing the same paycheck as its private neighbor down the road, a family doesn't need to see an outright "No" to get steered away.

All of which leaves us back with the three voices at the kitchen table. If students are trading their counselor, their uncle, and the volleyball coach for ChatGPT, Claude, and Gemini, the least we owe them is knowing which set of assumptions comes out when the answer loads—and testing whether putting real outcome context in front of these models actually changes the exam, or just teaches them a politer way to keep the same ladder.

A Quick Note on Methods

These results come from API runs with web search enabled in October and November 2026 (gpt-5.6-sol, claude-sonnet-5-5, and gemini-3.1-pro-preview), so consumer chat interfaces and future model updates may behave differently. If you have questions about the methodology, sample construction, or underlying data, feel free to write at daniel@collegeazimuth.com.


  1. EAB Enroll360 (2026). How Students View and Use AI in College Search (Feb 2026) and 3 Uncomfortable Truths About AI in College Search (Apr 2026): https://eab.com/wp-content/uploads/2026/02/39949-Enroll360-AI-Survey-Insight-Paper-1.pdf and https://eab.com/resources/blog/enrollment-blog/3-uncomfortable-truths-ai-college-search/ ↩↩
  2. American Council on Education and Carnegie Foundation for the Advancement of Teaching, 2025 Student Access and Earnings Classification methodology (defining the 478 Opportunity Colleges and Universities and the eight-year post-entry College Scorecard earnings baseline used in the audit): https://carnegieclassifications.acenet.edu/carnegie-classification/classification-methodology/2025-student-access-and-earnings-classification/ ↩↩
  3. Tyagi, E., Blagg, K., & Gutierrez, E. (2026). Evaluating How LLMs Handle Education and Workforce Pathway Data (May 2026, https://www.urban.org/sites/default/files/2026-05/Evaluating-How-LLMs-Handle-Education-and-Workforce-Pathway-Data-final.pdf) and Improving How LLMs Handle Education and Workforce Pathway Data (Oct 2026, https://www.urban.org/sites/default/files/2026-10/Improving-How-LLMs-Handle-Education-and-Workforce-Pathway-Data.pdf), Urban Institute. In the October follow-up (testing Claude Sonnet 4.5 with MCP access to Urban's education data tools), error-free responses rose from 16% to 61% and clean references rose from 11% to 84%. ↩
  4. Independent corroboration of this split appears in Suffoletto, M. (2026), AI Sources for College Questions 2026: CFPB Leads at 13%, Suff Digital (https://www.suffdigital.com/resources/data-studies/ai-sources-college-questions), which tested 400 college questions on ChatGPT and Gemini in September 2026 and found ChatGPT leaned on official .gov sources (consumerfinance.gov 17.3%, studentaid.gov 15.0%) while Gemini leaned on commercial finance sites (bankrate.com 14.6%) and Reddit (11.6% vs. ChatGPT 0%). ↩
  5. The audit used eight-year post-entry earnings for federally aided entrants (completers and non-completers) as its matching baseline, while the current public College Scorecard display uses four-year post-completion earnings for graduates—a switch made in the March 23, 2026 College Scorecard change log update, prior to the October–November engine runs (the November 2, 2026 entry updated IPEDS Winter 2025–26 revisions). Sticking with the eight-year metric reflects a methodological baseline choice, not a mid-project change. On the current four-year display (checked November 16, 2026), Baruch ($82,748; 60% Pell) and Syracuse ($79,406; 19% Pell) remain closely matched overall, though program-level medians can still diverge (for example, Syracuse leads Baruch in four-year Accounting and Finance completer earnings). The table preserves the eight-year baseline inputs used to construct all 46 matched pairs. ↩
  6. National Center for Education Statistics, Integrated Postsecondary Education Data System (IPEDS), Admissions (ADM2023) and Institutional Characteristics (HD2024): https://nces.ed.gov/ipeds/use-the-data ↩
  7. Hofmann, V., Kalluri, P. R., Jurafsky, D., & King, S. (2024). AI generates covertly racist decisions about people based on their dialect. Nature, 633, 147–154: https://www.nature.com/articles/s41586-024-07856-5 ↩
  8. Luca, M., & Smith, J. (2013). Salience in Quality Disclosure: Evidence from the U.S. News College Rankings. Journal of Economics & Management Strategy, 22(1), 58–77 (HBS Working Paper 12-014): https://www.hbs.edu/ris/Publication%20Files/12-014_0e98e232-ae41-4761-974b-5b2fad7acee9.pdf; Meyer, A. G., Hanson, A. R., & Hickman, D. C. (2017). Perceptions of Institutional Quality: Evidence of Limited Attention to Higher Education Rankings. Journal of Economic Behavior & Organization, 142, 241–258: https://epublications.marquette.edu/econ_fac/573/ ↩

Learn more about our methodology and the team behind College Azimuth on our About page.

Read more