- Forward


Sampled Auditory Content
An Introduction with Examples in Java


Prof. David Bernstein
James Madison University

Computer Science Department
bernstdh@jmu.edu

Print

Introduction
Back Forward
  • Sampling Auditory Content:
    • Involves two discretizations
    • Is often called analog-to-digital conversion
  • The Discretizations:
    • Temporal Sampling
    • Quantization
Introduction (cont.)
Back Forward

Temporal Sampling

images/soundwave-sampling.gif
Introduction (cont.)
Back Forward

Quantization

images/soundwave-quantization.gif
Introduction (cont.)
Back Forward
  • Common Sampling Rates:
    • CDs - 44.1kHz
    • DVD Audio - 96kHz
  • Common Sample Sizes:
    • 16 bits
Introduction (cont.)
Back Forward
  • Nyquist-Shannon Sampling Theorem:
    • A digital signal cannot unambiguously represent signal components with frequencies above half the sampling rate
  • Implication:
    • The sampling rate must be greater than twice the highest frequency of the input signal if you want to be able to reconstruct the original perfectly
An Overview of the Java Sound API
Back Forward
  • The Package:
    • sampled javax.sound.sampled
  • Encapsulating the Sampling Process:
    • A AudioFormat javax.sound.sampled.AudioFormat object contains the number of channels (e.g., mono, stereo), the sampling rate, the quantiziation (i.e., the number of bits per sample), and the encoding technique (e.g., linear pulse code modulation, nonlinear mu-law).
    • Note that some encoding techniques (like ULAW and ALAW) compress the amplitude values while others (like PCM) do not.
An Overview (cont.)
Back Forward

Conceptual Model of the Presentation of Sampled Audio

images/audiomixer.gif
An Overview (cont.)
Back Forward
  • The Clip javax.sound.sampled.Clip Interface:
    • A type of line that contains data that can be loaded prior to presentation.
  • The Process:
    1. Create an AudioInputStream from a File or an InputStream
    2. Create an appropriate Line.Info object
    3. Create a Clip object based on the Line.Info object.
    4. Open the Clip.
    5. Start the Clip.
An Overview (cont.)
Back Forward

A Simple Application

javaexamples/auditory/sampled/ClipPlayer.java
 
Encapsulating Sampled Auditory Content
Back Forward
  • What's Needed:
    • An AudioFormat
    • The samples
  • To Simplify the Discussion:
    • We will use sampling processes that vary only in their sampling rates
    • We will standardize all other aspects of the process
Encapsulating Sampled Audio (cont.)
Back Forward

A BufferedSound Class

javaexamples/auditory/sampled/BufferedSound.java
 
Encapsulating Sampled Audio (cont.)
Back Forward
  • Constructing BufferedSound Objects:
    • Use a factory
  • Approaches:
    • Use our understanding of the physics of sound
    • Read from a file
Encapsulating Sampled Audio (cont.)
Back Forward

A 100Hz Sine Wave

images/sine-100.gif
Encapsulating Sampled Audio (cont.)
Back Forward

Sampling from a Wave

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: 0)
 
Encapsulating Sampled Audio (cont.)
Back Forward

Reading from a File: Use a Common Encoding

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: format)
 

Reading from a File: Create a Buffer and Read

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: buffer)
 

Reading from a File: Convert the Raw Bytes

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: rawbytes0)
 
javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: rawbytes1)
 

Reading from a File: Process the Channels

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: channels)
 
Operating on Sampled Auditory Content
Back Forward
  • The Process:
    • Construct a compatible destination if necessary
    • Operate on the source signal(s)
  • Number of Operands (i.e., Sources):
    • One (i.e., unary)
    • Two (i.e., binary)
Requirements of a Generic Operation
Back Forward

Unary Operators

javaexamples/auditory/sampled/BufferedSoundUnaryOp.java
 

Binary Operators

javaexamples/auditory/sampled/BufferedSoundBinaryOp.java
 
An Abstract Implementation
Back Forward

Creating the Compatible Destination Object(s)

javaexamples/auditory/sampled/AbstractBufferedSoundOp.java
 
An Abstract Implementation (cont.)
Back Forward

Channel Handling

javaexamples/auditory/sampled/AbstractBufferedSoundUnaryOp.java
 
javaexamples/auditory/sampled/AbstractBufferedSoundBinaryOp.java
 
Types of Operations
Back Forward
  • Filters:
    • Causal (only previous samples) vs. Non-causal
    • Finite (only use the input signal) vs. Infinite
    • Linear vs. Nonlinear
    • Time-invariant vs. Adaptive
  • Other Kinds of Operations:
    • Addition
    • Mixing Channels
    • Reversal
    • ...
Some Common Names of Operations
Back Forward
  • Chorusing/Flanging/Phasing (i.e., slight variations in frequency)
  • Echo
  • Fading (in and out)
  • Reverberation (i.e., echo off of multiple surfaces)
  • Wah Wah
An Addition Operation
Back Forward
javaexamples/auditory/sampled/AddOp.java
 
An Addition Operation (cont.)
Back Forward

Harmonics (100Hz + 200Hz)

images/sine-100plus200.gif
An Addition Operation (cont.)
Back Forward

Beating/Phasing (100Hz + 105Hz)

images/sine-100plus105.gif
A Reverse Operation
Back Forward
javaexamples/auditory/sampled/ReverseOp.java
 
Filters
Back Forward

An Infinite, Linear, Causal Filter

\( d_{i} = \sum_{k=0}^{n} s_{i-k} w_{k} + \sum_{j=0}^{m} d_{i-j} v_{j} \)


A Finite, Linear, Causal Filter (Finite Impulse Response Filter)

\( d_{i} = \sum_{k=0}^{n} s_{i-k} w_{k} \)

Finite Impulse Response (FIR) Filters
Back Forward

An Illustration

images/fir-filter.gif
FIR Filters (cont.)
Back Forward
javaexamples/auditory/sampled/FIRFilter.java
 
javaexamples/auditory/sampled/FIRFilterOp.java
 
FIR Filters (cont.)
Back Forward

An Example

javaexamples/auditory/sampled/FIRFilterDriver.java (Fragment: 1)
 
Presenting Sampled Auditory Content
Back Forward

Using the Composite Pattern

images/soundapi.gif
Presenting Sampled Audio (cont.)
Back Forward
javaexamples/auditory/sampled/BufferedSound.java
 
Presenting Sampled Audio (cont.)
Back Forward

The BoomBox

javaexamples/auditory/sampled/BoomBox.java
 
Presenting Sampled Audio (cont.)
Back Forward

Getting Information on Capabilities

javaexamples/auditory/sampled/ShowCapabilities.java
 
There's Always More to Learn
Back -