"Annotated Videos"

How to easily add meta-information to instructional on-line videos - a practical guide from production to implementation

by Christof Daetwyler, M.D.
Drexel University College of Medicine
2900 Queen Lane, Philadelphia, PA 19129, USA

Background

Video is a wonderful tool to capture how a specialist role-models best practice - and then to make it available for the learner to watch it when the curriculum or personal interest calls for it.

However, there are several major issues known that are in the way of making linear instructional videos a valuable tool for education.

They are linear
The video has a begin, a middle and an ending - this is a good format to tell a story, but not necessarily to transport learning content.
They are self contained
The video contains of what is cut into the video - and nothing more.
They show the same content to beginner and advanced students alike
The needs for explanations is very different if a student is a beginner or almost graduate.

We propose the "Annotated Video" technology as an answer to the question how new media can encounter the shortcomings of traditional instructional video.

"Annotated Video":

is non-linear
has a meta-layer of additional information available
might include links to external sources

So far, "Annotated Videos" are a substantial part of "doc.com - a set of 40 media-rich online-modules for the teaching and learning medical communication skills" (see http://webcampus.drexelmed.edu/doccom/) and "Headache Interactive - on-line" (see http://piripirei.net/headache/).

Some examples of "Annotated Videos"

You might check out some examples of "annotated videos" to get a clear picture:

Example: Mark Mumenthaler takes medical history of headache sufferer
Example: Tim Quill gives very bad news (you have cancer)

How to produce the video for "Annotated Videos"

When producing the 40 doc.com modules which all include "annotated videos" as a central part, we came up with a very streamlined production process that allows us to produce all "annotated videos" that we need per doc.com module withing 4 to 6 hours. And here is how we do this:

First we capture how the specialist(s) interact with one or more standardized patients in a role-play
Second we play back the best version to the specialist (see comment about "finding best takes" below)
The specialist tells us when a comment seems appropriate. We stop tape, log the timecode and record the specialist making the comments, then we role tape until the specialist stops again.. and so on..

Finding the best takes: when logging a shoot, we usually take a note of the timecode, scene and take. If a take goes well we circle it once, if a take goes extremly well, we circle it twice. Later, in post production, we look only at the "circled" takes.

If you are new to video production, please take a look at the Word-File of my script "How to Easily Make and Integrate Quality Videos for Medical Education"

Exercise 1: Build groups of 3-4: 2 actors and 1 to 2 camera-operators. Film a role-play about a topic you come up with. Log the timecode of your takes. Cercle the best take. Play back the best take and stop when you'd like to add a comment. Log the comments for easy post-production. Take also a log-note at what timeframe the comment is to be placed on the main movie. Logging means: take notes of reel-tape name, date, actors, event - and for each scene and takes the timecodes.

In the "materials" section you find an uncompressed movie-file, so you could go on working through the work-shop even if you have no camcorder.

How to compress movies to be used in "Annotated Videos"

In this section you'll learn about size and compression for on-line video.

First we have to save the movie files from to the harddrive of our computer. There are many ways to do this, depending on the media you are using and the software on your computer. The size and data rate of MiniDV is about 30 megabits/second - you need about 2 GB for 10 minutes of source video.

When the files are transferred to your computer, you can open them with QuickTime PRO. This allows you to export the movie file and at the same time to change its compression and size so it will become small enough to run "on-line".

The settings that I found work best for the movies are:

Compression Video	MPG4 H.264 or SORENSEN 3 @ 140kb/s
Compression Audio	MONO @ 44.100 Hz, MPG3 or AAC Codec @ 40kb/s
Size	Half of the original: for NTSC 320x240
Key Frames	Every 60th frame
De-Interlace	If you use half of the original size: top field
Video Frame Rate	Half of the original: 15 (NTSC) or 12.5 (PAL)

Remark: Sometimes I have the movies available in HiRes for a better viewing. However, I use very similar settings but display it at double size. To have most details as crisp as possible, I allow the video compression to use 200 kb/s - it still runs smooth over a ADSL broadband connection (256kb/s).

For the smaller "talking heads" comment-movies, I use even lower bandwith settings:

Compression Video	MPG4 H.264 or SORENSEN 3 @ 80 kb/s
Compression Audio	MONO @ 44.100 Hz, MPG3 or AAC Codec @ 40kb/s
Size	160x120, numeric crop Left 20,Top 20,Right 20,Bottom 0
Key Frames	Every 60th frame
De-Interlace	Eliminate top field
Video Frame Rate	Half of the original: 15 (NTSC) or 12.5 (PAL)

Exercise 2: use QuickTime Pro or another software to upload the movie files onto your computer. Export the role play movie and the comments using the settings as explained above.

In the "building-materials" for the work-shop you find already compressed movies in the appropriate format.

How to add a HREFTrack to a QuickTime movie?

In this section you'll learn how to create HREFTracks to have QuickTime trigger timed events.

Part 1: The text in a HREFTrack

QuickTime allows you to add a special track to a movie that can cause actions at defined movie-times. This track is a text track that must be named "HREFTrack" (please be cautious: the upper and lower cases of HREFTrack must be entered exactly as written here).

Here is an example of the text of a "HREFTrack":

{QTtext}{font:Geneva}{timeScale:30}{width:300}{height:20}{timeStamps:absolute}
[00:00:04.0]
A<javascript:HiliteElement(1,2)>T<MovieRightFrame>
[00:00:16.0]
A<javascript:DoSomething(myParameter)>T<MovieRightFrame>
[00:00:37.0]

The first line gives the specificatons. You won't have to change any of the values given in this line.

The second line defines the time at which the action as defined in the third line will happen. The time is entered as followed [hh:mm:ss:f] (hh=hours, mm=minutes, ss=seconds, f=frames). The frames per second are defined in the timeScale parameter in line 1.

The third line defines the Action and the Frame where it should happen.
A<javascript:DoSomething(myParameter)>T<MovieRightFrame>
A means Action, in this case the JavaScript Function DoSomething is called with the parameter (myParameter).
T indicates the frame where the JavaScript that handels the Function resides.

CAUTION: calling JavaScripts out of HREFTracks does only work if the JavaScript resides or is referenced by the frame where the Action is sent to!

Part 2: How to add a HREFTrack to a QuickTime Movie

Save your description of the HREFTtrak as a text file.
Open the HREFTrack-text file with QuickTime: it will show as a movie containing a text-track at the appropriate time.
Copy all of the HREFTrack-QuickTime movie
Open now the movie where you'd like to add the HREFTrack
Use "add to movie" - this adds the HREFTrack as a text-track to the movie
Check the "Movie Properties" in the Menu "Windows" - this allows you to change the name of the added text-track to HREFTrack and also to set its visibility to "invisible"
Save the movie (on the Macintosh as "self-contained")

Since the way how QuickTime handels HREFTracks changes with almost every major upgrade of QuickTime, you'll might have to Google "how to add HREFTrack to QuickTime" to find a current description of the process.

Example 3: Create a HREFTrack for your movie where the time-codes indicate where the comments should show up (use your log-file to figure out the time-codes, then watch the movie and fine-tune).

How to put the pieces together and make the "Annotated Videos" work?

In this section you'll learn how to set up a frame-set that combines the main-movie with the HREFTrack together with the annotation list into a functional unit.

Since the way I'm using "annotated videos" work is in a "frame-set"ting, I'd like to talk a bit about frames and their implications.

Here is a graphic showing the frames for our example of "annotated viedos":