Principal Investigator Larry Rudolph
The paper and pencil approach to recording information is powerful and convenient. It accepts virtually all types and formats of notes, bounded only by the fact that the paper is two-dimensional. It provides an easy and natural way to delete notes and correct mistakes (i.e. using the eraser). And it is incredibly reliable--you never have to worry about the “system” crashing and wiping out all of your work.
Despite these strengths, the paper and pencil approach leaves much to be desired in the areas of information exchange and replay. After a notetaking session, one often has to type up the notes on a computer to send to others. Scanners and digital cameras help by allowing digital capture of the raw information as images. However, a more desirable solution would provide a meaningful interpretation of the raw information--the actual text that was written rather than an image of it. This interpretation would be more useful and easier to store. It would also be helpful if one could replay the entire notetaking session.
We aim to augment the paper and pencil model by designing a notetaking system that is robust, real-time, and human-centric. Because of its basic strengths as an information capture medium, the paper and pencil model is at the core of our own approach.
In the future, we plan to generalize the notetaking system for use with whiteboards. This should only require minor modifications to the video capture module. This feature would be useful in meeting situations where the minutes of the meeting are to be recorded and made available to all participants.
Also, adding a speech recognition module to provide context to the handwriting recognition module may increase accuracy. Such an architecture would take advantage of multimodal input to provide better guess as to what is being written. In fact, the information exchange between the handwriting and speech recognition modules could be in either direction (or both), depending on which recognition task requires the most accuracy.