ChatGPT’s Potential for Quantitative Content Analysis: Categorizing Actors in German News Articles
by
Clarissa Hohenwalde,
Melanie Leidecker-Sandmann,
Nikolai Promies
and
Markus Lehmkuhl
We assessed ChatGPT's ability to identify and categorize actors in German news media articles into societal groups. Through three experiments, we evaluated various models and prompting strategies. In experiment 1, we found that providing ChatGPT with codebooks designed for manual content analysis was insufficient. However, combining Named Entity Recognition with an optimized prompt for actor Classification (NERC pipeline) yielded acceptable results. In experiment 2, we compared the performance of gpt-3.5-turbo, gpt-4o, and gpt-4-turbo, with the latter performing best, though challenges remained in classifying nuanced actor categories. In experiment 3, we demonstrated that repeating the classification with the same model produced highly reliable results, even across different release versions.
Volume 24 • Issue 2 • 2025 • Science Communication in the Age of Artificial Intelligence (Science Communication & AI)