Authoring tool for SignSupport

An authoring tool for SignSupport is computer-based software that helps a domain expert, with little or no programming skills, to design and populate a limited domain conversation scenario between a Deaf person and a hearing person, e.g., when a Deaf patient collects medication at a hospital pharmacy or when a Deaf learner is taking a computer literacy course. So that SignSupport provides instructions to the Deaf person in signed language videos on a mobile device. The authoring tool enables the creation and population of such scenarios on a computer for subsequent “playback” on a mobile device.

The output of this authoring tool is an XML script, alongside a repository of media files that can be used to render the SignSupport mobile app on any platform.

Screenshot of the authoring tool
Screenshot of the authoring tool

 

The interface of the authoring tool is divided into four areas:

  • The button area contains four buttons, namely ‘Add template’, ‘Clear all’, ‘Add video’ and ‘Export to XML’. When the domain expert clicks on the ‘Add template’ button, a window pops up and presents all the default screen templates of the authoring tool. The user then chooses the template that s/he would like to add to the canvas by clicking the checkbox on the left hand side of the preferred template and then clicks the ‘OK’ button at the bottom of the window. The selected screen template then appears on the canvas as active. The ‘Clear all’ button clears all the screen templates from the canvas. The ‘Add video’ button helps the domain expert add videos from the computer as an asset within the authoring tool. The videos line up vertically as they are being added from the computer to the video area. The ‘Export to XML’ button invokes a method that reads all the screen templates, their assets and connections to each other; and then produces an XML file that includes all of this information. Since the XML file contains only text, it only stores an asset’s information such as file name and relative path, in text. An XML parser on a target device consumes the XML file in order to render a platform-specific mobile app.
  • The video area contains a series of signed language videos that are added by the domain expert from the computer. The videos can be recorded before or after the scenario based conversation flow is designed. Then each video is given an English tag/file name as a way to help the scenario domain expert include the correct video in the right place. Videos in this area are played (and paused) by clicking on them. They can also be dragged and dropped to other screens.
  • The icons tab area contains icons in various categories. The domain expert can also add new tabs of icons. The icons can be dragged and dropped to the screens, and linked to other screens and/or actions.
  • The Canvas area is where the selected screen templates appear when added. The domain expert populates a screen by typing in the intended message and dragging and dropping icons or a video to the indicated areas in the screen. If the domain expert drops an icon into the wrong area, s/he can right click on that area and select the ‘remove icon’ option and the icon will be removed. To link the screens to one another, the domain expert clicks on the icon that represents a button from one screen, confirms that s/he wants to connect that icon to another screen at a prompt dialogue pop-up, then selects the screen to link with. A blue link line will appear in the Canvas, between these two screens. This link gets updated and redrawn as the screens are moved from one position to another.