Everyone is raving about the speed of using AI to make our lives easier. But in my world of college admissions, AI needs a lot more work to be able to compete or surpass the quality of what a human can do. Currently, AI-generated admissions work lacks accuracy and soul.
My team and I took a look at three ways students and parents use AI in the college admissions process: putting together a college list, researching testing policies, and creating a college essay. We ran data through AI and then compared what AI generated to our own work. The difference between AI-generated research and writing versus what we can do were shockingly different.
Here's what we found:
Creating a College List
- We ran the exact data of a student who we created a college list for this past year through AI software.
- The acceptance rates for colleges that AI generated were not updated. In fact, they were inaccurate and were usually higher than what they actually are.
- Thus, the AI college list included many colleges that the student was simply not competitive for.
- AI was not able to discern the difference between a reach school and an "out-of-reach" school. It also identified reach, target, and likely colleges more generously than what we typically do. In other words, what AI thought was a target or likely college for a student is often a reach or out-of-reach college in reality.
- AI did not provide colleges that met the criteria that the student requested from us. For example, despite prompting AI to find colleges with generous merit scholarships, it included a number of colleges that offered zero merit scholarships.
Researching Testing Policies
- When researching which colleges did not superscore the SAT or ACT, AI provided inaccurate results. For example, AI indicated that the University of Pennsylvania did not superscore either test, yet Penn's undergraduate admissions website indicates the opposite.
- When researching which colleges will count the Science score of the ACT into the composite score if provided (even though it's optional), AI got 1/4 of the policies wrong and missed key nuances on others.
RELATED READING: 5 Questions Facing the Class of 2027 (and My Answers!)
Writing a College Essay
- When we took one of the essays we edited last year and plugged the same topic into an AI model, we got very different results. We provided the AI model with instructions on adding detail and emotion, while also providing specific names and experiences. We also instructed it to remove em dashes (a common indicator that AI was used in an essay).
- The AI detector reported that the essay created by the AI model was 91% AI-generated. The AI detector reported that the essay we edited was 0% AI-generated.
- The AI-generated essay had a very negative tone to it, while the edited essay was an all-time favorite for our team and resulted in better-than-expected admissions results for the student who wrote it.
Application Nation - Class of 2027 is now open and accepting new members! LEARN MORE
I wish AI had the complexity to do things that take experts like me hours to do. It would make my life so much easier. But the technology is simply not there yet.
For now, I urge students and parents to use AI with caution. An AI-generated college list misses key information to ensure the student is objectively competitive for the colleges on the list. If you are researching a college's policy, take the time to look at the primary source (the college's admissions website), rather than rely solely on the quick AI-generated answer. And, using AI to write a college essay is simply foolish. The essay sounds generic, at best. More concerning, though, is that if a college uses an AI-detector, you could have zero chance of being admitted if your essay is flagged for being AI-generated.
So, AI makes a lot of errors. The scary thing is that the errors are not obvious to an untrained eye like a student applying to college for the first time. Yes, humans make errors too. But no one expects us to be perfect; not even the colleges.
.jpg)
