- Forward


Sampled Auditory Content
An Introduction with Examples in Java


Prof. David Bernstein
James Madison University

Computer Science Department
bernstdh@jmu.edu

Print

Introduction
Back SMYC Forward
  • Sampling Auditory Content:
    • Involves two discretizations
    • Is often called analog-to-digital conversion
  • The Discretizations:
    • Temporal Sampling
    • Quantization
Introduction (cont.)
Back SMYC Forward

Temporal Sampling

soundwave-sampling
Introduction (cont.)
Back SMYC Forward

Quantization

soundwave-quantization
Introduction (cont.)
Back SMYC Forward
  • Common Sampling Rates:
    • CDs - 44.1kHz
    • DVD Audio - 96kHz
  • Common Sample Sizes:
    • 16 bits
Introduction (cont.)
Back SMYC Forward
  • Nyquist-Shannon Sampling Theorem:
    • A digital signal cannot unambiguously represent signal components with frequencies above half the sampling rate
  • Implication:
    • The sampling rate must be greater than twice the highest frequency of the input signal if you want to be able to reconstruct the original perfectly
An Overview of the Java Sound API
Back SMYC Forward
  • The Package:
    • sampled
  • Encapsulating the Sampling Process:
    • A AudioFormat object contains the number of channels (e.g., mono, stereo), the sampling rate, the quantiziation (i.e., the number of bits per sample), and the encoding technique (e.g., linear pulse code modulation, nonlinear mu-law).
    • Note that some encoding techniques (like ULAW and ALAW) compress the amplitude values while others (like PCM) do not.
An Overview (cont.)
Back SMYC Forward

Conceptual Model of the Presentation of Sampled Audio

audiomixer
An Overview (cont.)
Back SMYC Forward
  • The Clip Interface:
    • A type of line that contains data that can be loaded prior to presentation.
  • The Process:
    1. Create an AudioInputStream from a File or an InputStream
    2. Create an appropriate Line.Info object
    3. Create a Clip object based on the Line.Info object.
    4. Open the Clip.
    5. Start the Clip.
An Overview (cont.)
Back SMYC Forward

A Simple Application

javaexamples/auditory/sampled/ClipPlayer.java
 
Encapsulating Sampled Auditory Content
Back SMYC Forward
  • What's Needed:
    • An AudioFormat
    • The samples
  • To Simplify the Discussion:
    • We will use sampling processes that vary only in their sampling rates
    • We will standardize all other aspects of the process
Encapsulating Sampled Audio (cont.)
Back SMYC Forward

A BufferedSound Class

javaexamples/auditory/sampled/BufferedSound.java
 
Encapsulating Sampled Audio (cont.)
Back SMYC Forward
  • Constructing BufferedSound Objects:
    • Use a factory
  • Approaches:
    • Use our understanding of the physics of sound
    • Read from a file
Encapsulating Sampled Audio (cont.)
Back SMYC Forward

A 100Hz Sine Wave

sine-100
Encapsulating Sampled Audio (cont.)
Back SMYC Forward

Sampling from a Wave

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: 0)
 
Encapsulating Sampled Audio (cont.)
Back SMYC Forward

Reading from a File: Use a Common Encoding

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: format)
 

Reading from a File: Create a Buffer and Read

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: buffer)
 

Reading from a File: Convert the Raw Bytes

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: rawbytes0)
 
javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: rawbytes1)
 

Reading from a File: Process the Channels

javaexamples/auditory/sampled/BufferedSoundFactory.java (Fragment: channels)
 
Operating on Sampled Auditory Content
Back SMYC Forward
  • The Process:
    • Construct a compatible destination if necessary
    • Operate on the source signal(s)
  • Number of Operands (i.e., Sources):
    • One (i.e., unary)
    • Two (i.e., binary)
Requirements of a Generic Operation
Back SMYC Forward

Unary Operators

javaexamples/auditory/sampled/BufferedSoundUnaryOp.java
 

Binary Operators

javaexamples/auditory/sampled/BufferedSoundBinaryOp.java
 
An Abstract Implementation
Back SMYC Forward

Creating the Compatible Destination Object(s)

javaexamples/auditory/sampled/AbstractBufferedSoundOp.java
 
An Abstract Implementation (cont.)
Back SMYC Forward

Channel Handling

javaexamples/auditory/sampled/AbstractBufferedSoundUnaryOp.java
 
javaexamples/auditory/sampled/AbstractBufferedSoundBinaryOp.java
 
Types of Operations
Back SMYC Forward
  • Filters:
    • Causal (only previous samples) vs. Non-causal
    • Finite (only use the input signal) vs. Infinite
    • Linear vs. Nonlinear
    • Time-invariant vs. Adaptive
  • Other Kinds of Operations:
    • Addition
    • Mixing Channels
    • Reversal
    • ...
Some Common Names of Operations
Back SMYC Forward
  • Chorusing/Flanging/Phasing (i.e., slight variations in frequency)
  • Echo
  • Fading (in and out)
  • Reverberation (i.e., echo off of multiple surfaces)
  • Wah Wah
An Addition Operation
Back SMYC Forward
javaexamples/auditory/sampled/AddOp.java
 
An Addition Operation (cont.)
Back SMYC Forward

Harmonics (100Hz + 200Hz)

sine-100plus200
An Addition Operation (cont.)
Back SMYC Forward

Beating/Phasing (100Hz + 105Hz)

sine-100plus105
A Reverse Operation
Back SMYC Forward
javaexamples/auditory/sampled/ReverseOp.java
 
Filters
Back SMYC Forward

An Infinite, Linear, Causal Filter

\( d_{i} = \sum_{k=0}^{n} s_{i-k} w_{k} + \sum_{j=0}^{m} d_{i-j} v_{j} \)


A Finite, Linear, Causal Filter (Finite Impulse Response Filter)

\( d_{i} = \sum_{k=0}^{n} s_{i-k} w_{k} \)

Finite Impulse Response (FIR) Filters
Back SMYC Forward

An Illustration

fir-filter
FIR Filters (cont.)
Back SMYC Forward
javaexamples/auditory/sampled/FIRFilter.java
 
javaexamples/auditory/sampled/FIRFilterOp.java
 
FIR Filters (cont.)
Back SMYC Forward

An Example

javaexamples/auditory/sampled/FIRFilterDriver.java (Fragment: 1)
 
Presenting Sampled Auditory Content
Back SMYC Forward

Using the Composite Pattern

soundapi
Presenting Sampled Audio (cont.)
Back SMYC Forward
javaexamples/auditory/sampled/BufferedSound.java
 
Presenting Sampled Audio (cont.)
Back SMYC Forward

The BoomBox

javaexamples/auditory/sampled/BoomBox.java
 
Presenting Sampled Audio (cont.)
Back SMYC Forward

Getting Information on Capabilities

javaexamples/auditory/sampled/ShowCapabilities.java
 
There's Always More to Learn
Back -