Köhler, S.1, König, A. 2,3, Linz, N. 2, Altenstein, S. 4, Butryn, M.5, Glanz, W. 5, Jessen, F.6-8, Munk, M. H.9,15,16, Osterrath, A.10,17, Schott, B.H.11,12, Spottke, A.6,13 & Teipel, S.J.1,14
1) Deutsches Zentrum für Neurodegenerative Erkrankungen (DZNE), Rostock/Greifswald, Germany. 2) ki:elements GmbH, Saarbrücken, Germany. 3) Cobtek (Cognition-Behaviour-Technology), Université Côte d’Azur, Nice, France. 4) Charité – Universitätsmedizin Berlin, Berlin, Germany. 5) Medical Faculty University Hospital Magdeburg, Magdeburg, Germany. 6) Deutsches Zentrum für Neurodegenerative Erkrankungen (DZNE), Bonn, Germany. 7) University Hospital Cologne, Cologne, Germany. 8) Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany. 9) Deutsches Zentrum für Neurodegenerative Erkrankungen e.V. (DZNE), Tübingen, Germany. 10) Deutsches Zentrum für Neurodegenerative Erkrankungen e.V. (DZNE), Dresden, Germany. 11) University Medical Center Göttingen, Göttingen, Germany. 12) Deutsches Zentrum für Neurodegenerative Erkrankungen e.V. (DZNE), Göttingen, Germany. 13) University Hospital Bonn, Bonn, Germany. 14) University Medical Center Rostock, Rostock, Germany. 15) University Hospital Tübingen, Dept. Psychiatry, Germany. 16) Systems Neurophysiology, Technical University Darmstadt, Germany. 17) University Hospital Carl-Gustav-Carus Dresden, Dresden, Germany.
* Poster presented at the AD/PD™ Alzheimer’s disease and Parkinson’s disease Conference 2024, Portugal
Abstract
Objective
Automated speech recording via telephone using a chatbot technology and AI-driven speech analysis would enable resource-saving cognitive assessment for people with cognitive decline. To extract reliable digital speech biomarkers through a chatbot it needs to be adapted to the users’ needs. Here, we implemented a user centered design approach to evaluate usability of a phone-based chatbot system for automated speech assessments for people with prodromal or preclinical Alzheimer’s disease (AD).
Methods
Within the study PROSPECT-AD, participants of ongoing national cohort studies are automatically called for six times every three months by our chatbot “Mili”. To date, 189 participants have been recruited out of 300 planned cases through September 2023. Each call consists of three cognitive tasks (Wordlist, Semantic Verbal Fluency, Story Telling). We applied a six-stage usability check for Mili (see Figure 1). For deeper insights regarding the chatbot’s usability, we conducted semi-structured interviews (n=22).
Results
To date, Mili has completed 356 assessments. The SUS revealed a good usability for Mili (n=140, Age: Ø=72.6, SD=6.6, SUS: 69.3/100, SD=22.2). Interviewees perceived the Wordlist without visual input as challenging. However, some wished for variety in the tasks. Additionally, two participants reported frustration as they did not have a positive event to report (Story Telling). Some interviewees preferred a more human like chatbot, while others perceived Mili as a human being. All interviewees wished for feedback regarding their performance.
Conclusions
Our results revealed a high usability and feasibility of the automated phone calls by Mili. Socially intelligent chatbots may be able to address emotional strain during cognitive assessments. Furthermore, based on our usability data, we would aim for an adequate feedback system for the participants regarding their cognitive performance.
Figure 1. Stages of usability checks during the study (developed for the study).
