- Forward


Synchronized Auditory and Visual Content
An Introduction with Examples in Java


Prof. David Bernstein
James Madison University

Computer Science Department
bernstdh@jmu.edu

Print

Formats
Back SMYC Forward
  • The Issue:
    • How the set of frames is organized
    • Not the format of individual frames
  • Separating Tracks/Channels:
    • Demultiplexing
  • Common File Formats:
    • Quicktime
    • Vivo
CODECs
Back SMYC Forward
  • Purpose:
    • Compress (i.e., encode)
    • Decompress (i.e., decode)
  • Approaches:
    • Spatial correlation
    • Temporal correlation
  • Common Examples:
    • H.261
    • Cinepak, H.263, Indeo
    • JPEG, MPEG-1
Performance Issues
Back SMYC Forward
  • Latency:
    • The amount of time required before the presentation can begin
    • Usually depends on format and bandwidth
  • Quality:
    • The aesthetic appeal of the rendered content
    • Usually depends on CODECs and bandwidth
Initiating the "Transfer"
Back SMYC Forward
  • Pull:
    • Client initiates and controls the flow
    • Protocols: HTTP, FILE
  • Push:
    • Server initiates and controls the flow
    • Protocols: RTP, MediaBase, Real
Synchronized AV Content in Java
Back SMYC Forward

The Player Class

javaexamples/video/PlayerPanel.java
 
javaexamples/video/PlayerPanelDriver.java
 
Processing Synchronized AV Content in Java
Back SMYC Forward
  • Demultiplexing
    • Extracting the tracks
  • Decoding:
    • Decompressing each track
  • Filtering:
    • Applying effects algorithms
  • Rendering:
    • Presentating to the user
Processor States in Java
Back SMYC Forward
  • Configuring:
    • Connected to data source
    • Demultiplexed
  • Configured:
    • Data format has been determined
    • Can now obtain TrackControl objects
  • Realized:
    • Fully constructed and ready to go
Processor States in Java (cont.)
Back SMYC Forward

javaexamples/video/ProcessorPanelDriver.java
 
javaexamples/video/ProcessorPanel.java
 
Frame-Based Processing
Back SMYC Forward

A Simple Example

javaexamples/video/RGBRenderer.java
 
javaexamples/video/RendererCanvas.java
 
javaexamples/video/RendererDriver.java
 
Frame-Based Processing
Back SMYC Forward

A More Complicated Example

javaexamples/video/PopUpRenderer.java
 
javaexamples/video/PopUpRendererDriver.java
 
There's Always More to Learn
Back -