The developers of GPT-4 claimed that the program can exhibit “human-level performance” on certain exams. While the chart of various academic test scores seems impressive at first, the software failed both AP English exams with a 2 out of 5. It scored extremely well on tests where all that a computer program needed was the right information, which isn’t very impressive to anyone who has used Google in the past few decades.
Again, I don’t think that we’re suddenly going to be bleeding lawyers. As someone who has actually performed reasonably well on the LSAT, I can also tell you that taking the test and practicing law are very different things. Law exams are about pattern recognition, which is exactly what AI is built to do. AP exams work similarly, and you can answer a large bulk of questions by word association.
I’m actually glad that GPT-4 can score well on the more rote tests that only measure how long you’ve spent drilling flashcards. I’ve never believed in exams that require students to memorize tons of information. Its failure to pass English exams is proof that critical thinking is the most important skill that you can get out of education.
