An Icon-Based Brain-Computer Interface that Leverages Semantic Frames
Lab Members: Karl Wiegand, Rupal Patel
Collaborators: Deniz Erdogmus (Northeastern University)
Supported by NSF Grant No. HCC-0914808.
Icon-based assistive communication devices typically present users with arrays of semantic concepts that are concatenated to formulate messages. For users with motor impairment, navigating through these multilayered hierarchical arrays is slow and fatiguing. RSVP-iconCHAT is a system that leverages Rapid Serial Visual Presentation (RSVP) and frame-based semantics toward the design of a small-footprint, icon-based communication system that can be controlled with a single input signal without sacrificing vocabulary size.
Initial comparisons of message construction speed and complexity with a traditional, mouse-controlled array system showed that message complexity was comparable in both systems and construction speed was only twice as slow using a one-key system. We are currently replacing the one-key input with EEG-BCI detection of P300 brainwaves to further reduce motor fatigue and increase communication speed.
Wiegand, K., Patel, R., & Erdogmus, D. (2010). Leveraging Semantic Frames and Serial Icon Presentation for Message Construction. ISAAC Conference for Augmentative and Alternative Communication, Barcelona, Spain, July 2010. PDF
Wiegand, K., & Patel, R. (2012). Non-Syntactic Word Prediction for AAC. In Proceedings of the Third Workshop on Speech and Language Processing for Assistive Technologies, (pp. 28-36). Montreal,Canada: Association for Computational Linguistics. PDF