SpeechMatE: speech-driven mathematical editor

This project consists in the developmentof a software solution for dicrìtation and editing of mathematical formulas, based on vocal interaction features with visual and vocal feedback of the dictated content. It is useful to people with motor and visual impairment and people with specific learning disorders, for helping them in writing, elaborating and solving mathematical expressions. People with motor impairment concerning upper limbs, often encounter difficulties in writing mathematical content and numerical or simbolic simplifications of mathematical expressions are a complex task for people with SLDs. Currently there are no softwares with dictation and editing features available in Italian. There are only partial solutions in English.

S. Polin Laboratory developed a prototype starting from the SpeechMatE 0.1, a software with multimodal stable interaction, and the prototype has dictation features for advanced mathematical content, and navigation and editing features for formulas.

Video

Cristian Bernareggi, Mattia Ducci, Alessandro Mazzei, Dragan Ahmetovic, Tiziana Armano, Anna Capietto, Sandro Coriasco, Nadir Murru (2020) SpeechMatE: A Speech-driven Maths Editor for Motor-Impaired People, ICCHP open access compendium “Future Perspectives of AT, eAccessibility and eInclusion”, 2020.