PROJECT: PERCEPTMEDIA

SCALABLE STREAM CODING FOR ADAPTIVE FOVEATION ENHANCED PERCEPT MEDIA COMMUNICATION
Javed I. Khan
ACTIVENET PI Meeting , June 2001, Jackson Hole
Networking and Media Communications Research Lab
Dept of Math & Computer Science
Kent State University
Sponsor: DARPA/ITO

Network Adaptation: A case study
with scalable video across network asymmetry

Stream Adaptation
Link Capacity Adaptation
Splices wide-band links with low capacity links.
Application aware rate adaptation.
Greater down-scalability range (4/5th vs. 1/100th)
Perceptual encoding
Interactive data compression with  perceptual control over where and how quality is compromised.
Node Capacity Adaptation
Allows network junction nodes with unequal CPUs to perform rate adaptation.
Auto-configuration by motion vector bypass
Reuses motion vector and saves computation for  quality
Cycle sweeping (network spread)
If one node is not enough it spreads the computation upstream and sweeps additional cycles from active nodes.

3 Way Tradeoff

APPROACH

Rate Adaptive Channels

Full Cascade Adaptation

MPEG-2  Rate Control

Motion Vector Bypass

Transcoder’s Self-Organizing States

3-way Tradeoff

Flow: “Single Stream Multiple Form”

PERCEPTUAL
(DOWN) SCALABILITY

MPEG-2  Spatial Rate Control

Slide 15

Sensitivity: Spatial SNR Control

DYNAMIC OBJECT TRACKING

After December demo we  tried a content based object detection and temporal scalability..

Object Tracking in Stream by Inverse Motion Vector Projection

TEMPORAL SCALABILITY MODEL

Motion Vector Transformation for Temporal Scalability

DEMO

object tracking in video
Spatial scalability
Temporal scalability

CONTROL SENSITIVITY

Reflection on Scalable Transcoding
Scalability=deeper knowledge+system symbiosis
Rate adaptation at network level? Only 80%.
We are trying to gain deeper interaction:
Content protocol aware adaptation (requantization, DCT filtering, etc.)
Content aware adaptation (object detection based).
Interactive viewer aware adaptation.
Active network insight:
Look at the source of info that forms the base of adaptation?
video protocol designer
content producer
viewer
above all-- the network resource awareness
Each knowledge component have their distinct role.  How to connecting all the  parties involved?

Few Additional Works

Reflection
Active Programmable Harness
We needed two things:
Collect link and node capacities to find out optimum transcoding points.
Now set the configuration parameters.
We consequently designed a network local state collection and propagation tool
User application sends the propagation and synthesis math, the harness takes care of the propagation by 2.5 phase communication.
A building block for transcoder deployment in a multicast network.
A generalization of SNMP
Instead of P2P it is now graph wide.
Local composition on local MIB2 elements at active nodes.

Transcoding Tree Determination

Active Harness Harness

Generalized Harness: State Model

Beyond Video: Active Hypermedia Accelerator
Sits quite at the other end of a  narrow link and monitors surfer’s behavior. Accordingly pre-fetches, caches, transcodes, and performs customized stream scheduling.

Lessons Learned:A Top Down View

What an adaptive stream might need from active network?

Reflection: Resource Adaptation
In a large network diversity is ingrained
Whether it is an adaptive service or not, but the ingredient components should not make very restrictive assumption about network resources.
This requires certain behaviors (potential “Active Network Layer Services)
Network resource awareness
State awareness about the service
Self-organization help (state maintenance signaling?)
Plug-in in the applications themselves (footprint).
Seamless reorganization

Reflections: Performance
Performance and Onion Peeling:
Encapsulation does not stop at  TCP layer it continues upward…
[IP[TCP[TS[SEQ[PIC[SLICE[MB]]]]]]]
Active network have to worry about performance of access.
To get access to one field-- generally we have to access. somewhere up to 5-10 other fields! Major performance drainage.
Current protocols = logic + field placement
Proposal = Separate logical definition from  locution in packet.
Optimum locution based on access dependency of the capsules.

Top Down Reflection
Active network feels like very crude bare-bone architectures.
Powerful yet simple construction formalism is critically required to encourage usability of the architecture and reusability of the efforts spent to develop middleware on them.
Some of our organized thoughts hinting solutions:
Language to express ‘deployment constraints’.
Language to express ‘resource requirements’.
Language to express ‘bounds of adaptation’.
“Virtual instrument”
Definitely I am out of time by now! Pl. see our publications.
Papers are available at http://medianet.kent.edu/