We assessed ChatGPT's ability to identify and categorize actors in German news media articles into societal groups. Through three experiments, we evaluated various models and prompting strategies. In experiment 1, we found that providing ChatGPT with codebooks designed for manual content analysis was insufficient. However, combining Named Entity Recognition with an optimized prompt for actor Classification (NERC pipeline) yielded acceptable results. In experiment 2, we compared the performance of gpt-3.5-turbo, gpt-4o, and gpt-4-turbo, with the latter performing best, though challenges remained in classifying nuanced actor categories. In experiment 3, we demonstrated that repeating the classification with the same model produced highly reliable results, even across different release versions.
Date of publication
Apr 14, 2025
Category:
Article
ChatGPT’s potential for quantitative content analysis: categorizing actors in German news articles
by
Clarissa Hohenwalde
,
Melanie Leidecker-Sandmann
,
Nikolai Promies
,
Markus Lehmkuhl