This panel session discusses new approaches to making mathematical expressions accessible to computer users with print-disabilities. The panel members focus on three software products: WinTriangle[6], ChattyInfty[9], and MathPlayer[4]. All three are accessible presently in audio, and all either have or will soon have Braille access. The panel will also present background information and related software such as MathML to Braille translators.

MathML[16] is the W3C recommendation for including mathematical expressions in XML documents. Because MathML is a vendor-neutral, information rich description of math, it is now a lingua-franca for most math-related applications. The W3C math site www.w3.org/Math/Software lists over 90 applications that use MathML.

In addition to software, MathML has been or is being incorporated into many standards including the DAISY (ANSI/NISO Z39.86-2005) standard for the Digital Talking Book (DTBook)[1]. This is important addition because in 2004, the United States Congress enacted legislation that endorsed a national file format (NIMAS)[3] based on the DTBook format. The legislation requires publishers to create an accessible version of their K-12 textbooks in this format and submit it to a national repository (NIMAC). The specification of MathML for math in NIMAS allows math and science textbooks to be easily converted to various formats.

WinTriangle, ChattyInfty, and Web pages (Internet Explorer + MathPlayer) are all potential conversion targets from NIMAS.

WinTriangle is a self-voicing Windows application. People who are blind or have other serious print disabilities may use it to read, write, and manipulate scientific information that is otherwise difficult or impossible for these users to access.

WinTriangle’s visual display is a standard RTF text screen. Text is displayed normally and equations are displayed in a linear form with normal subscripts and superscripts. A small set of markup indicators are used to define structures that would normally be displayed in a two-dimensional notation. For example, fractions are represented using a start fraction symbol (a heavy left square bracket with a small f inside), a heavy slash symbol to separate numerator from denominator and an end fraction symbol (a heavy right square bracket with a small f inside). A sighted professional scientist can usually read this notation if a “cheat sheet” card showing the markup symbols is available. A major advantage of this notation is the ease of cutting and pasting in a variety of clipboards when manipulating expressions as required for solving complex scientific problems.

A screen shot of WinTriangle is shown below. The screen shot shows text and math (from simple algebra to integral calculus) mixed together.
Screen shot of a WinTriangle screen with both text and math being displayed.

WinTriangle can import XHTML/MathML scientific documents such as those produced using the MathType editor with Microsoft Word (the most common authoring environment for scientists and academics), the Infty Reader, and the Infty Editor. The Tex2Tri utility[15], developed by David Thompson of Harvard University provides the ability to import documents written in a TeX language. Together, these two formats comprise the vast majority of electronic formats used today. WinTriangle can export as XHTML/MathML, so it is fully capable of reading and writing in mainstream formats. Using export, equations can be read by sighted people in standard two dimensional notation.

ChattyInfty is a self-voicing text and math-expression editor for visually disabled persons to read, write, or edit scientific documents. ChattyInfty extends InftyEditor by adding speech output for both the text and math in a document. Both ChattyInfty and InftyEditor are part of a multiple University and Institute research project called Infty Project.[13]. InftyEditor (and hence ChattyInfty) also includes an experimental hand writing recognition program for math[10].

Text in ChattyInfty is displayed on screen normally and math is displayed in standard two-dimensional notation. Input is loosely based on TeX commands. Using the arrow keys, users can navigate around the document to hear both text and math in the document. A screen shot of ChattyInfty is shown below. The displayed content is equivalent to what is shown in the WinTriangle example above.
Screen shot of a ChattyInfty screen with both text and math being displayed.

ChattyInfty can import LaTeX and InftyReader (see below) formats. It can export a variety of formats including LaTeX and XHTML+MathML.

A problem that many people with severe print disabilities face is that electronic versions of a book are often not available. OCR is often employed to scan the books into an electronic form. However, OCR for math is not currently part of any commercial OCR software. As part of the InftyProject, InftyReader was developed. InftyReader can recognize printed scientific documents including math expressions or other technical notations and convert them into LaTeX, XHTML+MathML, and the InftyEditor format. InftyReader’s recognition rate in 2003 was 99.4% for text and 95.2% for math, and work continues on improving those rates[14].

MathPlayer is a free plug-in for Internet Explorer (IE) that displays MathML within a web page. Unlike images, MathPlayer matches the properties of the surrounding text such as the font size and color when it displays the math. This means that people who need special color schemes or larger fonts can read the math in the document without additional work.

Mathematical typography uses smaller font sizes for subscripts and superscripts. These smaller characters are sometimes hard to read even though it is not hard to read the rest of the document. MathPlayer allows users to click on an expression and see a magnified view. A screen shot of IE+MathPlayer is shown below with the last expression magnified. The text and math are the same as what is shown in the WinTriangle and ChattyInfty examples above.
Screen shot of Internet Explorer with MathPlayer displaying math in a XHTML document.

Web pages containing MathML can be easily generated from a number of applications including Microsoft Word + MathType.

MathPlayer supports Microsoft’s standard accessibility interface (MSAA) so that it works with JAWS, Window-Eyes, and other screen readers to seamlessly integrate math on web pages into the screen readers. User’s can navigate around the expressions to aid their comprehension.

In addition to speaking mathematical expressions, MathPlayer can also synchronize highlighting of subexpressions as they are spoken. As with synchronized text highlighting, synchronized highlighting of math focuses readers’ attention on what they are reading along with providing audio reinforcement of what they read.

MathPlayer is being extended so that math in Word and PDF documents will be accessible. Work is also being done so that the speech rules can be customized or internationalized.
Related Efforts

WinTriangle, ChattyInfty, and MathPlayer rely on speech to communicate math to someone who is blind. Many larger expressions or expressions that are tabular in nature (such as long division) are probably best understood if they are encoded in Braille or other tactile or tactile/audio formats.

WinTriangle, ChattyInfty, and MathPlayer provide enhanced accessibility by creating tactile hard copy on embossers employing Tiger technology[7]. MathPlayer can emboss math equations in standard two-dimensional format with characters embossed in the DotsPlus[5] extension to Braille; DotsPlus expresses letters and numbers in Braille and most other characters graphically. Triangle also can emboss math equations using DotsPlus Braille, but the quasi-linear Triangle math format is used instead of two-dimensional notation. ChattyInfty can emboss an enlarged view of visual equations for blind users who need to correct OCR errors in math equations.

There are several projects working on converting MathML to/from Braille. Two projects focusing on Nemeth code are John Boyer’s work as part of xml2brl[2] and Eitan Gurari’s work[8] as part of an NSF grant. There are at least two international groups who have a broader focus of converting MathML to/from various braille math codes. IGroup UMA[11] is working on two-translation for the Nemeth, Marburg, and French math codes along with voice navigation. The LAMBDA project[12] is a European Union funded project focusing on an integrated system for writing and reading mathematical texts for the use of blind students. The braille math codes that are being targeted by the LAMBDA project include those used in the United Kingdom, Italy, France, Portugal, and Spain. One or more of these efforts may be used by WinTriangle, ChattyInfty, and MathPlayer.

A great deal of progress in the area of math accessibility has occurred over the last few years. Research continues in the area, but there are already solutions that can make electronic documents containing math partially or fully accessible with little intervention. All of the work involves MathML in one way or another. Perhaps the largest problem that remains is getting content providers to update their tools to deliver accessible documentation.

The work of John Gardner and Neil Soiffer was supported in part by grants from the National Science Foundation’s (NSF) Small Business Innovation Research (SBIR) program. John Gardner’s was also supported in part by the National Institutes of Health’s SBIR program. The work of Masakazu Suzuki was supported in part by the Grants-in-Aid for Scientific Research of the Japan Society for the Promotion of Science.
[1] ANSI/NISO Z39.86-2005. Specifications for the Digital Talking Book, NISO Press, 2005, http://www.niso.org/standards/resources/Z39-86-2005.html.
[2] Boyer, J.J. New xm2l2brl Release With Some Math, 2004, http://www.nfbnet.org.
[3] CAST. NIMAS Technical Specification, 2005, http://nimas.cast.org/about/technical/index.html.
[4] Design Science MathPlayer.
[5] Gardner, J., Dotsplus—better than Braille? in International Conference on Technology and Persons with Disabilities, (Los Angeles, CA, 1993), http://dots.physics.orst.edu/publications/csun93.html.
[6] Gardner, J. WinTriangle: A Scientific Word Processor for the Blind, 2005, http://dots.physics.orst.edu/wintriangle.html.
[7] Gardner, J., Stewart, R., Francioni, J. and Smith, A., TIGER, AGC, AND WIN-TRIANGLE, REMOVING THE BARRIER TO SEM EDUCATION. in CSUN International Conference on Technology and Persons with Disabilities, (Los Angeles, CA, 2002), http://www.csun.edu/cod/conf/2002/proceedings/299.htm.
[8] Gurari, E. Automatic Translation of Scientific Literature to Braille, National Science Foundation Research, ITR03-12487, http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0312487.
[9] InftyProject. About ChattyInfty, 2005, http://www.inftyproject.org/download/AboutChattyInftyE.txt.
[10] Kanahori, T., Fujimoto, M. and Suzuki, M., Authoring Tool for Mathematical Documents – Infty -. in 3rd International Conference MKM2004, (Bialowieja, Poland, 2004), http://www.inftyproject.org/articles/2004_MKM_Kanahori.pdf.
[11] Karshmer, A.I., Gupta, G., Pontelli, E., Miesenberger, K., Ammalai, N., Gopal, D., Batusic, M., Stöger, B., Palmer, B. and Guo, H.-F. UMA: a system for universal mathematics accessibility. in Proceedings of the ACM SIGACCESS conference on Computers and accessibility, ACM Press, Atlanta, GA, USA, 2004, 55-62.
[12] Nicotra, G. and others. Lambda Project, http://www.lambdaproject.org/.
[13] Suzuki, M. Infty Project, http://www.inftyproject.org/.
[14] Suzuki, M., Tamari, F., Fukuda, R., Uchida, S. and Kanahori, T., INFTY: an integrated OCR system for mathematical documents. in Proceedings of the 2003 ACM symposium on Document engineering, (Grenoble, France, 2003), ACM Press, 95-104, http://www.inftyproject.org/articles/2003_DocEng_Suzuki.zip.
[15] Thompson, D.M., LATEX2TRI: PHYSICS AND MATHEMATICS FOR THE BLIND OR VISUALLY IMPAIRED. in Technology And Persons With Disabilities Conference, (Los Angeles, CA, 2005), http://www.csun.edu/cod/conf/2005/proceedings/csun05.htm.
[16] W3C. MathML Software, http://www.w3.org/Math/implementations.html.

Leave a Reply

Your email address will not be published. Required fields are marked *