Pen-Based Interfaces for Mathematical Documents

In the reserch project named "InftyProject", we developed a handwriting interface to input mathematical expressions using on-line handwritten character recognition. It is included in an free software called InftyEditor released from the InftyProject web site: http://www.inftyproject.org. In our interface, as soon as a character is written, it is recognized and rewritten by neat strokes in an appropriate position and size automatically. This {\it Automatic Rewriting Method} improves the accuracy of the structure analysis of the written mathematical expressions. Recently, we made a tool to customize our on-line character recogniton engine to adapt it to individual users. In the talk, I will give an outline of our system and give demonstration of our handwriting interface updated by using the new customizing tool.

We explore a family of methods to increase the accuracy of mathematical handwriting analysis by using context information. Likely expression continuations are pre-computed from an analysis of a database of mathematical expressions. These continuations are used to rank the candidates of isolated symbol recognition. In this talk we describe how the predicted continuations are derived, how they are combined with the recognition results, and the effectiveness of the results. We first review the techniques we have used to build and represent a mathematical context database. We then describe different strategies to combine context information with results obtained from the recognition of individual characters. Finally we present a summary of a case study, using a fixed dataset of common mathematical expressions to test the accuracy of on-line analysis.

In this talk, I will describe our prototype pen-based system for entering and manipulating mathematical expressions, currently called MathBrush. MathBrush allows a user to enter mathematical expressions with an electronic stylus and to then perform mathematical computations using a computer algebra system such as Maple or Mathematica. The talk will first outline the challenges involved with recognizing mathematical expressions. Next, I will provide an overview of our system, and describe how we have overcome some of these challenges. I will also address what challenges remain, the status of our current research. Finally, I will demonstrate MathBrush in action. This talk is based on work done by myself and George Labahn, Scott MacLean, Mirette Marzouk and Ian Rutherford from the Symbolic Computation Group.

We address the question how pen-based interfaces to mathematical systems can be organized. We describe an an overall software architecture for the treatment of mathematical handwriting: We present our approach to mathematical handwriting analysis and expression manipulation, and the way in which various software components interact in our architecture. These components include modules for mathematical character recognition, two-dimensional expression grouping, embedding in mathematical computing software (such as Maple) and document processing software (such as word). We describe how we use a database of several million mathematical expressions extracted from a popular digital library in order to guide the expression recognition process. Finally, we describe how this pen-based architecture can use InkML for interactive mathematical collaboration.