Web Video Captioning Technologies

Why Captioning is Important

Captioned video is important for a number of different reasons. There is a legal and moral imperative to ensure equal access to material, especially when access to that material is critical to a user completing a task or understanding a fundamental lesson. A number of current accessibility standards and guidelines mandate captioned video for this reason:

Equally important is that, when implemented in certain ways, SMIL text transcripts are searchable and index-able by various search engines and other data-management tools.

Definitions:

Captioning: Rendering of speech and other audible information in the written language of the audio.

Subtitling: Rendering a translation of dialogue and certain onscreen elements in visible words.

Open Captioned: the text remains on screen at all times.

Closed Captioned: the text can be made visible or not visible by user intervention

Transcript: the text file of the audio feed.

SMIL

The Synchronized Multimedia Integration Language (SMIL, pronounced "smile") enables simple authoring of interactive audiovisual presentations.

SMIL is typically used for "rich media"/multimedia presentations which integrate streaming audio and video with images, text or any other media type.

SMIL is an easy-to-learn HTML-like language (XML based), and many SMIL presentations are written using a simple text-editor.

Uses

[Screen capture - a captioned QuickTime Video]

SMIL can be used to:

Status / History

SMIL 1.0 - W3C Recommendation - June, 1998
[http://www.w3.org/TR/REC-smil]

SMIL 2.0 - W3C Recommendation - January, 2005
[http://www.w3.org/TR/2005/REC-SMIL2-20050107/]

SMIL 2.1 - W3C Recommendation - December, 2005
[http://www.w3.org/TR/2005/REC-SMIL2-20051213/]

Anatomy of a SMIL File:

Creating a SMIL File for RealMedia

First the text version of the script. Notice the start timings being assigned to the different speech fragments:


<window type="teleprompter" duration="0:35.0"
width=210 height=180 bgcolor="000000">

<font face="arial" color="white" size="+2">
<b>Ontario Hydro Television 
Commercial</b></font><br/>
<font face="arial" color="white"  size="-1">
© 2002 Ontario Hydro</font>

<time begin="0"/><p><font face="Arial" 
color="#FFFFFF">
(Mother)<br/><b>Ontario is a great place to 
live, work and raise a family.</b>
<br/><br/></font>

<time begin="3.9"/><font face="Arial" 
color="#00FF00">
(Job Site Foreman)<br/><b>That's why it 
keeps growing.</b><br/>
<br/></font>
<time begin="5.0"/><font face="Arial" 
color="#FFFFFF">
(Construction Worker)<br/><b>And as we grow,
we're going to need more electricity</b>
<br/><br/></font>

<time begin="7.9"/><font face="Arial" 
color="#00FF00">
(Business Executive)<br/><b>...a continuous 
supply for the future</b><br/><br/>
</font>


<time begin="9.8"/><font face="Arial" 
color="#FFFFFF">
(Office Worker)<br/><b>...and more companies 
generating electricity</b></font>


<time begin="12.3"/><p><font face="Arial" 
color="#00FF00">
(Announcer)<br/><b>On May First, Ontario 
opened it's electricity market to competition.
</b><br/><br/></font>

<time begin="16.5"/><font face="Arial" 
color="#00FF00"><b>This means you can 
either stay with your current supplier at 
market rates, or you can choose to buy 
your electricity at a fixed rate from a 
retailer licensed by the Ontario Energy Board. 
</b><br/><br/></font>

<time begin="25"/><font face="Arial" 
color="#00FF00"><b>The choice is up to you.  
<br/><br/> For more information, or a free 
brochure, call us, we're here to help.</b>
<br/><br/><br/><br/></font>

<time begin="30.2"/><p><font face="Arial" 
size="-1" 
color="#FFFFFF">© 2002 Ontario Hydro</font></p>
</window>

Anatomy of a SMIL File:

Creating a SMIL File for RealMedia

Then the actual SMIL File:



<smil>
 <head>
 
  <meta name="title" 
       content="Ontario Hydro Commercial" />
  <meta name="author" 
       content="Video supplied by 
       Global News" />
  <meta name="copyright" 
       content="©2002 Ontario Hydro" />
  
  <layout type="text/smil-basic-layout">
  
   <region id="VideoChannel" 
           title="VideoChannel" 
           left="5" 
           top="40" 
           height="200" 
           width="170" 
           background-color="#000000"/>
		
   <region id="TextChannel" 
           title="TextChannel1" 
           left="175" 
           top="0" height="240" 
           width="230" 
           background-color="#000000" 
           fit="hidden"/>
 </layout>
 </head>
 
 <body> 
 
  <par title="Ontario Hydro Commercial">
  
  <text src="hydro.rt" 
        id="Captioning" 
        region="TextChannel" 
        title="Captioning"/>

  <video src="hydro.rm" 
         id="Video" 
         region="VideoChannel" 
         title="Video"/>
		
  </par>
 </body>
</smil>

Options:

Captioned videos can be streamed in a number of different formats:

Each format has it's strengths and weaknesses, as we shall see.

QuickTime

[Screen capture - a captioned QuickTime Video]

The QuickTime player supports captioning (using SMIL) in two ways; "embedded" captioning and "linked" captioning.

Method 1 (Embedded)

This method involves creating what is called a QuickTime text track and making it a part of your QuickTime movie. You will end up with one QuickTime movie file that contains audio, video, and your captions. This method requires QuickTime Pro, which can be purchased and downloaded from Apple.

Method 2 (Linked)

The second method involves creating a text track movie as a separate file. The QuickTime audio/video movie and the text track are put together with SMIL. You will end up with three files:

  1. The QuickTime movie containing audio and video
  2. The text (caption) track
  3. The SMIL file that puts them all together

How-to/Tutorial: http://www.webaim.org/techniques/captions/QuickTime/

RealMedia

[Screen capture - a captioned RealMedia Video]

RealPlayer uses SMIL to combine media content with a RealText (.rt) file. The .rt file contains the captions themselves and information about how and when they should appear.

The SMIL file is really just a pointer file. It contains information about where and how your captions and media content should display.

RealText files are text files that have an .rt extension. They can be created using a text editor or a tool such as MAGpie or HiCaption.

How-to/Tutorial: http://www.webaim.org/techniques/captions/real/

SAMI

[Screen capture - a captioned WindowsMedia Video]

SAMI (Synchronized Accessible Media Interchange) is similar to SMIL but was developed by Microsoft and is supported solely by Microsoft products, including Microsoft Encarta® Encyclopedia, Windows Media™ Player, and some Microsoft multimedia titles.

Unlike SMIL, which points to an external caption file (.rt for RealPlayer and .mov or .txt for QuickTime - linked), SAMI files contain the captions themselves.

Windows Media Player uses an ASX (or sometimes WVX or WAX) to combine the SAMI captions with the media file. The captions can also be automatically added to a media file when they are both embedded within a web page.

Note

For captions to display in the stand-alone Windows Media Player, they must usually be turned on manually. This can be done by selecting View > Captions or View > Now Playing Tools > Captions in older versions of Windows Media Player. In Windows Media Player 9 or newer, select Play > Captions and Subtitles > On if Available (or English). In Windows Media Player 10, you may have to enable Tools > Options > Security > Show local captions when present to view local caption files.

Adding SAMI Closed Captions to Digital Media: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/wmp7_sami.asp

Flash

[Screen capture - Flash Player]

SMIL as XML can also be incorporated into Flash videos using third party extensions. However, without a fair bit of post-production, the index-ability of captioned Flash files is still poor.

Additional Resources

Tools for adding captioning to Flash videos include HiSoftware's Hi-Caption viewer Component, Camtasia and Captionate's captioning skins.

Issues & Problems

Media Players:

Currently the biggest issue for deploying SMIL is that while files may use the .smil file extension (Mime-type), more than one media player supports the mime-type, but each player supports it differently, and due to their construction files are not inter-operable. Thus, serving up a raw .smil file may potentially fail for a significant portion of your audience.

There are easy solutions for this however - you can use a real player meta file (.rpm) to contain a reference to the .smil file - RealPlayer will play the .rpm which points it to the .smil (or .smi). For QT, you can create a QT reference movie to accomplish the same task; Apple provides a free tool to do this (MakeRefMovie). The reference movie is a .mov file that just references the .smil (or .sml - another recognized smil file).

Embedded vs. Linked:

As mentioned earlier, embedded captioning is not searchable by the major search engines or file management tools. For this reason, linked transcripts are preferable, although they may also need to be processed via XSLT style sheets for re-purposing on the web.

Recommendations

Indexing & Searchability

  • QuickTime SMIL
  • RealMedia SMIL

QuickTime SMIL and RealMedia SMIL with separate text files are the better solution, as text files remain index-able and readable to users unable to access the media content, or devices that do not support the media stream.

Accessibility

  • Windows Media Player
  • RealOne Player

The standalone versions of the three media players are far more accessible than the embedded versions. Of the three standalone players, Windows Media Player and RealOne Player presented a high level of accessibility. In all cases, newer versions of media players were at least as accessible as their predecessors, if not exponentially so. When possible, it is probably a good idea to provide the user with multiple options for accessing your media content. [ http://www.webaim.org/techniques/captions/mediaplayers/]

User-acceptance

  • Flash
  • Windows Media

Captioned Flash media avoids media-player conflicts.

Tools

Tools (cont.)

Tools (cont.)

Services

    • Stanford Video
    • Streaming Media

    • Services (Cont.)

    SOAP

    Stanford Online Accessibility Program

    [Screen Capture - the Stanford Online Accessbility Program web site]