MIT Researchers Propose a New Benchmark for AI: Can It Write a Funny Quatrain?
CAMBRIDGE, Mass.—Researchers at MIT have posted a new preprint on arXiv proposing what may be the most unexpected large language model benchmark to date. Rather than measuring mathematical reasoning, coding accuracy, legal analysis, or scientific knowledge, the authors ask a different question:
Can the model write a genuinely amusing four-line poem?
The paper has not been peer reviewed, but it has already attracted considerable discussion among AI researchers, computational linguists, and, perhaps inevitably, English professors who until this week had successfully avoided machine learning.
The benchmark, provisionally titled the Light Verse Performance Index (LVPI), is deceptively simple. Models of AI are presented with short prompts requesting lightly humorous quatrains in a variety of styles: Victorian whimsy, newspaper doggerel, New England understatement, mock epic, and affectionate parody. A final category is described at length, but with admirable precision, as "the sort of verse one uncle insists on reading after Thanksgiving dinner."
Unlike many existing benchmarks, answers cannot be judged by factual correctness. Instead, each response receives independent ratings for meter, rhyme, originality, semantic coherence, and—most controversially—whether at least two of three human readers actually smile.
That final criterion has proven divisive. One reviewer reportedly objected that smiling is not reproducible across laboratories. The authors replied that neither is humor.
Several frontier models were evaluated, including Galileo, multiple versions of Claude (among them the recently released Claude Fabel), ChatGPT, Gemini, and other state-of-the-art systems. Unsurprisingly, all demonstrated near-perfect grammatical competence. The more interesting finding was how differently they approached comedy.
Some preferred clever wordplay. Some leaned toward literary allusion. Some produced technically impeccable verse that left evaluators with the tinny feeling that they had just attended... an efficient meeting about laughter.
Others displayed a curious willingness to risk an actual joke.
The MIT authors reported that a successful quatrain must satisfy numerous simultaneous constraints: rhyme, rhythm, narrative, brevity, surprise, and tone. It is, in miniature, a constrained optimization problem about wearing a funny hat.
The study also introduces a secondary metric called Delayed Appreciation Time, with the early consensus being that DAT's a good metric.
DAT measures how long readers continue thinking about a quatrain after finishing it.
The highest-scoring systems were not always those whose first draft generated the biggest laugh. Rather, they tended to produce verses that became funnier after thirty seconds, as readers saw an internal callback or an unexpectedly elegant rhyme.
Whether the Light Verse Performance Index becomes a standard benchmark or merely a memorable footnote in the history of artificial intelligence remains uncertain. Closing the press conference, one of the authors summarized:
They ranked the bots by jokes they'd spin,
Though no one quite knew how to score.
We measured wit—but learned therein
That laughter asks for something more.
