- Forward


Sampled Static Visual Content
An Introduction with Examples in Java


Prof. David Bernstein
James Madison University

Computer Science Department
bernstdh@jmu.edu

Print

Sampling Static Visual Content
Back SMYC Forward
  • Color Sampling:
    • Converting from a continuous "rainbow" of colors to a discrete finite palette of colors
    • Sometimes called quantization
  • Spatial Sampling:
    • Another name for spatial discretization
    • The most common scheme involves the use of a finite grid with equal size cells
Sampling Static Visual Content (cont.)
Back SMYC Forward
  • The Result:
    • A grid (or matrix) of picture elements (or pixels), each of which contains a single color in the palette
    • Sometimes called a raster representation
  • An Illustration:
    • raster
Examples of Sampled Static Visual Content
Back SMYC Forward
  • Images
  • Bitmapped Fonts
Important Java Classes
Back SMYC Forward
  • The Abstract Parent:
    • Image
  • BufferedImage
    • Has a ColorModel
    • Has a Raster
"Quick Start"
Back SMYC Forward
Reading Sampled Visual Content
javaexamples/visual/statik/sampled/ImageCanvasApp.java (Fragment: part2)
 
"Quick Start" (cont.)
Back SMYC Forward
Rendering Sampled Visual Content
javaexamples/visual/statik/sampled/ImageCanvas.java
 
"Quick Start" (cont.)
Back SMYC Forward
The Complete App
javaexamples/visual/statik/sampled/ImageCanvasApp.java
 
Manipulating BufferedImage Objects
Back SMYC Forward
javaexamples/visual/statik/sampled/IdentityOp.java (Fragment: copy)
 
Constructing BufferedImage Objects
Back SMYC Forward
javaexamples/visual/statik/sampled/ImageFactory.java (Fragment: step)
 
Constructing BufferedImage Objects (cont.)
Back SMYC Forward
From a File/Resource
javaexamples/visual/statik/sampled/ImageFactory.java (Fragment: method2)
 
Operating on Sampled Static Visual Content
Back SMYC Forward
  • Single-Input/Single-Output Operations:
    • Performed by objects that implement the BufferedImageOp interface
    • Some can be performed "in place" and some require a destination image
  • The Identity Operation:
    • The simplest such operation
    • Just creates a copy of the source
Operating on Sampled Content (cont.)
Back SMYC Forward
An IdentityOp
javaexamples/visual/statik/sampled/IdentityOp.java (Fragment: 1)
 
javaexamples/visual/statik/sampled/IdentityOp.java (Fragment: 2)
 
javaexamples/visual/statik/sampled/IdentityOp.java (Fragment: 3)
 
Operating on Sampled Content (cont.)
Back SMYC Forward
A More Interesting Example: GrayExceptOp
javaexamples/visual/statik/sampled/GrayExceptOp.java (Fragment: skeleton)
 
Operating on Sampled Content (cont.)
Back SMYC Forward
GrayExceptOp (cont.)
  • An Observation:
    • We probably want to keep colors that are "similar to" to the given color
  • Two Metrics:
    • \(d_E = \sqrt{(R_1 - R_2)^2 + (G_1 - G_2)^2 + (B_1 - B_2)^2} \)
    • \(d_M = |R_1 - R_2| + |G_1 - G_2| + |B_1 - B_2| \)
Operating on Sampled Content (cont.)
Back SMYC Forward
GrayExceptOp.areSimilar()
  • Design Alternatives:
    • Implement the metric in the method
    • Use the strategy pattern
  • Choosing Between the Two:
    • Even though we are only going to use one metric, the flexibility afforded by the strategy pattern makes it worthwhile
Operating on Sampled Content (cont.)
Back SMYC Forward
Metrics
javaexamples/math/Metric.java
 
javaexamples/math/RectilinearMetric.java
 
Operating on Sampled Content (cont.)
Back SMYC Forward
GrayExceptOp.areSimilar() (cont.)
javaexamples/visual/statik/sampled/GrayExceptOp.java (Fragment: areSimilar)
 
Operating on Sampled Content (cont.)
Back SMYC Forward
GrayExceptOp (cont.)
  • Another Observation:
    • We want the "gray scale" to be the same brightness/luminance as the color image
  • What We Know:
    • Perceived luminance is a function of the total signal transmitted to the brain. So, \(R_g + G_g + B_g = R_c + G_c + B_c\)
    • Shades of gray are mixtures of black and white. So, \(R_g = G_g = B_g\)
  • The Implication:
    • \(R_g = G_g = B_g = (R_c + G_c + B_c) / 3 \)
Operating on Sampled Content (cont.)
Back SMYC Forward
GrayExceptOp (cont.)
javaexamples/visual/statik/sampled/GrayExceptOp.java (Fragment: filter)
 
Convolutions
Back SMYC Forward
  • Defined:
    • In its most general usage, a convolution is a product of two functions
    • As we are using it here, a convolution is a way of calculating the color of a pixel in the destination image from that same pixel and its neighbors in the source image (the product of the source and a kernel)
  • Kernels:
    • A kernel contains weights that are applied to the source pixels to obtain the destination pixels
    • Kernel
Convolutions (cont.)
Back SMYC Forward

The Kernel as a Grid/Matrix

source-destination-images

kernel

spatial_convolution

Convolutions (cont.)
Back SMYC Forward

Applying the Convolution to One Pixel

\( d_{i,j} = \sum_{r=-1}^{1} \sum_{c=-1}^{1} s_{i+r, j+c} k_{r,c} \)

Convolutions (cont.)
Back SMYC Forward

Identity Kernel

0 0 0
0 1 0
0 0 0

Convolutions (cont.)
Back SMYC Forward

Blurring Kernel

1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
A System for Constructing Convolutions
Back SMYC Forward

Requirements

  1. Create different BufferedImageOp objects
  2. Conserve memory used for kernels
A System for Constructing Convolutions (cont.)
Back SMYC Forward

An Inflexible Factory

PresizedImageOpFactory
A System for Constructing Convolutions (cont.)
Back SMYC Forward

An Wasteful Factory

WastefulImageOpFactory
A System for Constructing Convolutions (cont.)
Back SMYC Forward

A Good Design

BufferedImageOpFactory
A System for Constructing Convolutions (cont.)
Back SMYC Forward

A Low-Level Design that Leads to a Repetitive Implementation

javaexamples/visual/statik/sampled/RepetitiveBufferedImageOpFactory.java
 
A System for Constructing Convolutions (cont.)
Back SMYC Forward

A Good Low-Level Design

javaexamples/visual/statik/sampled/BufferedImageOpFactory.java (Fragment: createOp)
 
A System for Constructing Convolutions (cont.)
Back SMYC Forward

An Unmaintainable Imeplementation of the Enumeration

javaexamples/visual/statik/sampled/UnmaintainableConvolutions.java
 
A System for Constructing Convolutions (cont.)
Back SMYC Forward

A Good Imeplementation of the Enumeration

javaexamples/visual/statik/sampled/Convolutions.java
 
A System for Constructing Convolutions (cont.)
Back SMYC Forward

A Good Factory

javaexamples/visual/statik/sampled/BufferedImageOpFactory.java
 
Affine Transformations
Back SMYC Forward
  • Examples:
    • Scaling
    • Rotation
    • Translation
  • In Java:
    • AffineTransform
    • AffineTransformOp
Affine Transformations (cont.)
Back SMYC Forward

Scaling

javaexamples/visual/statik/sampled/BufferedImageOpFactory.java (Fragment: AffineTransformOp)
 
Affine Transformations (cont.)
Back SMYC Forward

Rotation

javaexamples/visual/statik/sampled/RotationApp.java (Fragment: rotate)
 
Look-Ups
Back SMYC Forward
  • Defined:
    • Use a look-up table to transform the source into the destination
  • In Java:
    • LookupOp
    • LookupTable
Look-Ups (cont.)
Back SMYC Forward

Creating a Photo Negative

javaexamples/visual/statik/sampled/BufferedImageOpFactory.java (Fragment: LookupOp1)
 
Look-Ups (cont.)
Back SMYC Forward

"Night Vision"

javaexamples/visual/statik/sampled/BufferedImageOpFactory.java (Fragment: LookupOp2)
 
Color Space Conversion
Back SMYC Forward
javaexamples/visual/statik/sampled/BufferedImageOpFactory.java (Fragment: ColorConvertOp)
 
Cropping/Cutting
Back SMYC Forward
  • Defined:
    • Extraction of a rectangular "sub-image" from an image
  • In Java:
    • Use methods in the Image class
Cropping/Cutting (cont.)
Back SMYC Forward

An Example

javaexamples/visual/statik/sampled/SurveillanceApp.java (Fragment: crop)
 
Design of a Sampled Static Visual Content System
Back SMYC Forward
Requirements
  1. Support the addition of sampled static visual content
  2. Support the removal of sampled static visual content
  3. Support \(z\)-ordering of sampled static visual content
  4. Support the transformation of sampled static visual content
  5. Support the rendering of sampled static visual content
Design of a Sampled Static Visual Content System (cont.)
Back SMYC Forward
Bad Alternatives (to add the functionality required for rendering)
  • Sepcialize the BufferedImage class -- will lead to code duplication when we consider described content
  • Use the decorator pattern -- confusing because of the methods in the Image interface
Design of a Sampled Static Visual Content System (cont.)
Back SMYC Forward
A Good Design using Delegation
TransformableContent_sampled
Implementing this Design
Back SMYC Forward
The Overall Structure
javaexamples/visual/statik/sampled/Content.java (Fragment: skeleton)
 
Implementing this Design (cont.)
Back SMYC Forward
The "Setters"
javaexamples/visual/statik/sampled/Content.java (Fragment: set)
 
Implementing this Design (cont.)
Back SMYC Forward
Transformations
javaexamples/visual/statik/sampled/Content.java (Fragment: transform)
 
Implementing this Design (cont.)
Back SMYC Forward
Rendering
javaexamples/visual/statik/sampled/Content.java (Fragment: render)
 
Implementing this Design (cont.)
Back SMYC Forward
A Factory
javaexamples/visual/statik/sampled/ContentFactory.java
 
There's Always More to Learn
Back -