NOVAGRAPHS Project – Laboratory S. Polin

The NoVAGraphS Project(Non-Visual Access to Graphical Structures) aims at coping with the issue of accessing scientific contents of graphical structures for visually impaired people. Modern technologies can help blind people and low vision people, but those technologies are valid enough only for textual informations; for different kind of contents the validity of those tools results being insufficient. Scientific books, for example, often contain “structured” graphical information, e.g. tables and diagrams. The idea behind NoVAGraphS project is to provide standards that make possible the comprehension of the graphical strucured informations, those standards being both best practices and software tools.
The key idea is to make those structures navigable for visually impaired people.

Principal Investigator: Alessandro Mazzei

Departments and Institutions:

Department of Computer Science
Laboratory “S.Polin”
Department of Mathematics
Department of Physics
Department of Philosofy and Science of The Education
Department of Economy and Statistics

Rationale

Visually Impaired Students encounter many difficulties in accessing university courses. Among all the tasks, the fruition of structured graphical representations, such as tables, diagrams, circuits etc. results being difficult for three main reasons. In first place, those images are meant to be transmitted by the visual channel. Secondly, non-visual representations of images are affected by numerous limitations: e.g. tactile representations cannot represent details with enough resolution; moreover, aptic devices are very expensive and not common enough. Thirdly, in order to create non-visual representations of structured images, a deep knowledge is required both in the scientific domain and in general in non-visual exploration. There is an urgent need of instruments to transform graphical structured representations in representations that are accessible to visually impaired people; this need is not only v.i. students’ need, but also a need of all those people who strive to adapt teaching resources to the v.i. students.

The Project on the Piedmontese territory

The Polin Laboratory since 2013 works on reaserch, development and dissemination of accessibility technologies for the impaired people who want to access academic studies. During the ordinary activities of the laboratory, the v.i. students, their tutors and their transcribers have identified some specific needs that paved the way to this project. Those kind of needs and opinions were then shared with the Italian v.i. association “UICI”, with schools and directly with v.i. students and with experts of assistive technologies. Some of the experts of accessibility were involved in the activities of the Laboratory as reasearchers, some of them being visually impaired themselves.

NoVAGraphs project and the state of the art

Numerous solutions were proposed to let v.i. people use some kind of images. We can describe 4 categories of techniques for the non-visual representation of images: textual representation, tactile or audio-tactile representation, aptic representation and sonification.

NoVAGraphS project aims at enhancing the textual representation by giving it an interactive structure, in such a way that the description itself will be navigable. Modern human-language elaboration techniques make it possible to create new textual and speech interactions between the human and the machine. Thanks to computational linguistics development, in a specific application domain we can make computers understand and producing human speech.
The innovative idea of the project consists in applying those modern techniques to analyze and generate interactively human language in such a way that the interaction textually describes the image/graph structure.

Concrete actions to be undertaken by the NoVAGraphs project

The NoVAGraphs project has three main goals:
1. building a transformer software, that would transform structured images(tables, diagrams, E-R diagrams, UML, trees, graphs and circuits) in an accessible textual representation;
2. building an explorer software that would navigate and in case modify those content via speech computation;
3. validation of the transformer and explorer softwares with a testing phase to be conducted in different teaching environments (Universities and high schools).

The transformer software assumes that the input is a semantic representation of the image structure(in LaTeX or PPT), that has to be converted in SVG (W3C Standard Vector Graphics format). The latter would be then enriched with metadata that are necessary to non-visual exploration. Since many times there is no semantic representation in LaTeX nor PPT (say we are dealing with a book’s BMP image) the project will identify guidelines and practices to help the transcriber do a manual conversion in SVG format.
The explorer software will be a browser Firefox or Chrome extension to let reading and navigation of the image via keystrokes, touchscreen input, speech or Braille input. The explorer software will also adapt screen visual parameters for users affected by low-vision.
Finally, all the developed tools will be evaluated with supervised studies and longitudinal studies in specific educational paths.