Projects

SPIDERS: Wearable Emotion Sensing

Left: SPIDERS glasses platform (red: hardware sensors; blue: PCB boards; green optional attachments). Right: SPIDERS glasses on a mannequin.

SPIDERS (System  for  Processing  In-situ  Bio-signal  Data  for  Emotion  Recognition  and  Sensing) is low-cost, lightweight and compact wearable platform that can monitor human emotions will benefit a wide area of research and applications, such as continuous health monitoring, elderly care, depression treatment, entertainment, so on and so forth. Thus, we present SPIDERS which is a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user’s facial expressions and real emotions. SPIDERS costs less than $20 to assemble and can continuously run for up to 9 hours before recharging. (Read More)

Naturalistic speech perception in bilinguals

Under the direction of Nima Mesgarani and in collaboration with Giovanni Di Liberto , we are investigating how modulated semantic surprisal can be observed in the EEG signals of native and non-native speakers (native English: 22 subjects; native Chinese speakers with different proficiency in English: 50 subjects) listening to continuous, natural speech. The main methodology we use is the multivariate Temporal Response Function(mTRF) which allows us to compare an EEG signal to a continuous speech stream, instead of brief discretized events as in ERPs. Specifically, we apply mTRF to quantify the coupling between EEG signals from each participant and the corresponding speech stimulus properties at the level of acoustics, phonemes, and semantics. We are also looking at whether these and other measures can be used to decode proficiency from an EEG signal. (paper)

%d bloggers like this: