• Zverev, Samuel R.
• Lindvall, Charlotta
• Tulsky, James A.
• Pollak, Kathryn I.
• Sciacca, Kate
• Kwok, Anne
• Tarbi, Elise C.

Abstract

<jats:title>Abstract</jats:title><jats:sec><jats:title>Objectives</jats:title><jats:p>As computational methods for detecting symptoms can help us better attend to patient suffering, the objectives of this study were to develop and evaluate the performance of a natural language processing keyword library for detecting symptom talk, and to describe symptom communication within our dataset to generate insights for future model building.</jats:p></jats:sec><jats:sec><jats:title>Materials and Methods</jats:title><jats:p>This was a secondary analysis of 121 transcribed outpatient oncology conversations from the Communication in Oncologist-Patient Encounters trial. Through an iterative process of identifying symptom expressions via inductive and deductive techniques, we generated a library of keywords relevant to the Patient-Reported Outcome version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework from 90 conversations, and tested the library on 31 additional transcripts. To contextualize symptom expressions and the nature of misclassifications, we qualitatively analyzed 450 mislabeled and properly labeled symptom-positive turns.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>The final library, comprising 1320 terms, identified symptom talk among conversation turns with an F1 of 0.82 against a PRO-CTCAE-focused gold standard, and an F1 of 0.61 against a broad gold standard. Qualitative observations suggest that physical symptoms are more easily detected than psychological symptoms (eg, anxiety), and ambiguity persists throughout symptom communication.</jats:p></jats:sec><jats:sec><jats:title>Discussion</jats:title><jats:p>This rudimentary keyword library captures most PRO-CTCAE-focused symptom talk, but the ambiguity of symptom speech limits the utility of rule-based methods alone, and limits to generalizability must be considered.</jats:p></jats:sec><jats:sec><jats:title>Conclusion</jats:title><jats:p>Our findings highlight opportunities for more advanced computational models to detect symptom expressions from transcribed clinical conversations. Future improvements in speech-to-text could enable real-time detection at scale.</jats:p></jats:sec>

Topics

• gold
• size-exclusion chromatography