
                            Illustrated Audio

                         Revision 0.3 (95-04-12)
                     (c) 1994,95 Government of Canada


Introduction:
------------

    Illustrated Audio is our first attempt at designing a system for
    sychronizing image displays with a sound track.  This will be useful
    for computer-based slide shows and for distributing video programs over
    low-bandwidth networks.  When working with video material, an author can
    pick key frames to illustrate and augment the sound track, and avoid
    the problems of random frames with slow-scan systems.

    Illustrated Audio data files consist of an audio track, various image
    and text files, and a script that provides instructions for display and
    synchronization.  The parts are assembled into a simple "tar" archive
    for distribution, and the "iaunpack" tool will unpack the archive and
    start the presentation.


Installation:
-------------

    An Illustrated Audio system is made up of several pieces:

    iaplay     - Plays an Illustrated Audio script.
    iaunpack   - Unpack a packed script, play it and then remove the unpacked
                 portion of the script.  Also optionally just unpacks a script.
    *.ia       - Packed Illustrated Audio scripts.
    *.iascript - Unpacked script files which reference audio and image files
    script     - Default script name used when building .ia files.

    For information on how to install the required files, consult the file
    INSTALLATION.



Playing a *.ia file (iaunpack)
------------------------------

    Iaunpack is designed to be used as a means of playing an Illustrated
    Audio script without retaining it on your system.  Ideally, this would be
    used as a MIME type for Mosaic.
    
    The following command line syntax applies to iaunpack:

    iaunpack [-k] [-v] [-p <directory>] <packed script name> [options to iaplay]
        
        -p <directory> : Set the default path to <directory>
               This option will set the default directory for whichever
               mode of operation is being used.  If -k is also specified
               then this is the destination directory for the untarred files.
               If -k is not specified then <directory> will be used as the
               location in which to create a temporary directory.
               
               If -p is not used the default is:
                  if -k then defaults to .
                  otherwise  defaults to /tmp
                  
        -k  :  Keep the unpacked form
               This flag, when set, will simply untar the .ia file to
               <directory> and exit without playing the script.
        
        -v  :  Verbose mode
               Will cause iaunpack to display each step in its processing.
               
  
    If -k is not specified the script will be played by calling iaplay.
    Any options for iaplay may be specified after the packed script file name
    and the first 9 options will be passed on intact.
    
    Examples:  iaunpack -k -p foo foo.ia
          or:  iaunpack -kp foo foo.ia
               will unpack the file foo.ia and store it in the directory foo.

               iaunpack foo.ia -geometry +100+100
               will unpack the file into /tmp/iatmp[.#], play `script`
               and then remove /tmp/iatmp[.#].
               
	       Where [.#] is replaced by a number if necessary to build
	       a unique temporary directory.
               
               iaunpack -vp /usr/tmp foo.ia foo.iascript
               will unpack the file into /usr/tmp/iatmp[.#], play foo.iascript
               and then remove /usr/tmp/iatmp[.#].
               


Playing a *.iascript script (iaplay)
------------------------------------

    A script file is a text file which tells iaplay the sequence and timing
    of the various associated pieces of data.
    
    To play a script file, the user must execute 'iaplay <filename>'.
    If no filename is specified the filename 'script' will be assumed.

    The following command line syntax applies to iaplay:

    iaplay [=<geometry>] [-geometry <geometry>] [-font <font>]
           [-display <display>] [-visual <visual>] [-Depth <depth>] [-debug]
           [-time] [-play <audio player>] [script file name] [-slow] [-TEXT]

               
        =<geometry>
            Specify an X11 geometry string which specifies the coordinates of
            the image window.  This geometry can be over-ridden by the
            following:
        
        -geometry <geometry>
            Specify an X11 geometry string.  This string will override any
            =<geometry> arguments.
            The two methods of specifying geometry allow a shell script to
            run iaplay setting a default geometry while still allowing the
            user to override those defaults.
        
        -font <font name>
            Specify a font to use in the text window.
        
        -display <display name>
            Use <display name> as the display instead of the default.
        
        -visual <visual class>
            Try to use the requested visual class for the image window.
            The class may be any of the following:
            
               -  TrueColor, DirectColor
               -  PseudoColor, StaticColor
               -  GrayScale, StaticGray
               
        -Depth <depth>
            Try to use the requested depth for the image window.  This is
            a hint which iaplay will try to honour but is not bound to do so.
       
        -debug
            Show debugging output for process.  This output is fairly large
            and isn't of general interest to the user.
         
        -time
            Display timing information.  This flag is useful for evaluating
            how a particular script performs.  It tells you when images or
            text are being loaded, when audio processes are queued, when
            items are displayed as well as when they should have been
            displayed (if not displayed at the correct time).  Finally, it
            also informs you when events are being skipped.
        
        -play <audio player>
            Use an external audio player.  Audio player must be the name
            of the external process to execute and must either be the full
            path name or it must reside in the user's path.
            The audio player should be able to play an audio track by
            simply passing it a filename as a command line argument.
        
        -slow
            Perform a slower two pass quantization of the jpeg images when
            they are loaded.  By default iaplay will do a single pass
            quantization.
        
        -TEXT
            Toggle the text window positioning mode.  The actual positioning
            modes aren't as important as the end result.  If the text window
            "slides" up or down on the screen as text items are displayed
            then you know that you should be using the -TEXT option to
            use the alternative mode.
            
            The catch is that the mode you need might change between different
            window managers.  If you are always using the same window manager
            and iaplay was compiled with the wrong default mode then you
            should probably alias iaplay and change your mailcap to pass
            the -TEXT option from Mosaic/Netscape to iaunpack to iaplay.       
            
    All arguments may be abbreviated to one or two characters.
    
    Example:  iaplay -display foo:0 -geometry +100+100 -font courr10 -time bar
        and:  iaplay -di foo:0 -g +100+100 -f courr10 -t bar
        and:  iaplay -di foo:0 =+100+100 -f courr10 -t bar
              will all use the display 'foo:0', having the image window appear
              with a top left hand corner at (100,100) using the font 'courr10'
              for the text window, displaying the timing information and
              using the script 'bar'.
              

    While the Illustrated Audio is being displayed, either window may be
    iconified, moved, resized (although the image window will be automatically
    resized when the next image is displayed) or have its ordering changed.
    In addition, the text window may be closed at any time.
    
    To halt the Illustrated Audio display you may hit <ESCAPE> or 'q' (or 'Q')
    in either window or close the image window.  As part of the process
    termination, the audio output will be killed.  On some (or most) systems,
    the audio will not actually stop immediately due to buffering.
    
    
    
Script File Format:
-------------------

    The Illustrated Audio script files are very straight-forward.  They
    consist of a series of events to occur at different times.
    
    The times may be specified as either an absolute time (an event to
    occur at a certain time) or as a relative time (an event to occur x
    seconds after the last input event of that type).
    
    A time is specified as either '@ hh:mm:ss' or '+ hh:mm:ss' where @
    means this time is absolute or + meaning relative to the last event
    * OF THIS TYPE *.  The time 'hh:mm:ss' is a notation of convenience.
    It can be specified as 'mm:ss' or merely as a number of seconds.
    That is '4:10' is the same as '00:04:10' and the same as '250'.

    Comments may be placed in script files.  Any line that starts with
    '#' is considered to be a comment and ignored.  All blank lines are
    ignored.
    
    Any lines in a script file which contain errors cause an error message
    to be displayed to stderr and then the offending line is ignored.
    
    Otherwise the events are as follows:
    
    Audio Event   audio   <filename>    <time>
    -----------
    
        An audio event specifies an audio file to be played and the
        time when it should be played.  There are no limits as to the
        number of audio files which may be queued in this manner.
        
        Note:  If foo.au is playing, bar.au has been requested to play
               and it is now the correct time to play bar.au the following
               happens:
               
               The process playing foo.au is killed.
               bar.au is played.
               
               This has the unhappy consequence of causing bar.au to be
               delayed on systems whose audio may not be cancelled immediately.
               The best solution to this problem is to ensure that you do not
               overlap audio by more than a half second.

    Image Event  image  [type]   <filename>    <time>
    -----------
        
        Any image event may or may not specify the image type.  Valid
        image types are "JPEG" or "GIF".  If no image type is specified then
        gifs and jpegs should always be correctly auto-detected.
        
        An image event specifies an image file to be display and the
        time when it should be displayed.  There are no limits to the number of
        image files which may be queued in this manner.
    
    Ending Time Event   ending_time     <time>
    -----------------
    
        An ending event specifies the time when the script should end.
        When the script ends the audio file will be killed and the image
        window is destroyed.
        

    Text File Event    text  <filename> <time>
    ---------------
    
        A text event will cause the text window to contain up to 128 lines of
        text (each line up to 254 characters long) in the text window at the
        appropriate time.  A text event may not occur until 2 seconds have
        passed in the script.
        
        If an event is specified to start before this 2 second period, a
        warning message will appear saying that the text event has been
        delayed.
        
        The text will be centered both horizontally and vertically in the
        window.
        


Sample Script:
--------------

    # This is a sample script file to demonstrate all the different
    # components of a typical script file.

    audio     audio_file     @ 0:00
    image     gif_image.1    @ 0:00
    text      copyright      @ 0:03    # Display any copyrights for this mat.
    image     gif_image.2    @ 0:05
    image     gif_image.3    +    5    # This is equivalent to @ 0:10
    image     jpg_image.1    + 0:05    # This is equivalent to @   15
    audio     second_audio   @ 0:20
    image     jpg_image.2    @ 0:20
    text      copyright2     @ 0:20
    image     gif_image.4    @ 0:30
    ending_time              @ 0:35



Script File Processing:
-----------------------

    A script file is processed by seperating all audio, image and text events.
    These events are then stored (without sorting them - consult the
    source file script.c for a brief explanation) in their separate
    categories.
    
    Whenever iaplay needs a new event (eg: the loaded image was just
    displayed and we want to start loading the next image) it searches
    all types of events to discover which event(s) is/are the next to occur.
        
    Any event which is due to occur at a time prior to the current time is
    ignored and the next possible event(s) are found.  Multiple events can
    occur at the same time if they are of different types.  The sample script
    file has two examples of simultaneous events.
    
    
    
Script Grammar:
---------------

    Following is the grammar associated to parsing script files:
    
    
    SCRIPT   ::= { COMMENT | AUDIO | IMAGE | TEXT | ENDING } { <character> }
                 <newline>.
    COMMENT  ::= '#'.
    AUDIO    ::= 'audio' FILE_NAME TIME.
    IMAGE    ::= 'image' FILE_TYPE FILE_NAME TIME.
    TEXT     ::= 'text'  FILE_NAME TIME.
    ENDING   ::= 'ending_time' TIME.
    FILE_TYPE::= 'GIF' | 'JPEG' | .
    FILE_NAME::= { <any character but a space -- defined by ctype> }.
    TIME     ::= AT_TIME | AFTER_TIME.
    AT_TIME  ::= '@' TIME_STRING.
    AFTER_TIME::='+' TIME_STRING.
    TIME_STRING ::= [{ '0'..'9' } ':'] [{ '0'..'9'} ':'] { '0'..'9' }.



Bugs / Suggestions / Etc.
-------------------------

    Please direct all enquires about this software to:
    
        ia@debra.dgbt.doc.ca
