Notes
Outline
PROJECT: PERCEPTMEDIA

SCALABLE STREAM CODING FOR ADAPTIVE FOVEATION ENHANCED PERCEPT MEDIA COMMUNICATION
Javed I. Khan
ACTIVENET PI Meeting , June 2001, Jackson Hole
Networking and Media Communications Research Lab
Dept of Math & Computer Science
Kent State University
Sponsor: DARPA/ITO
Network Adaptation: A case study
with scalable video across network asymmetry
Stream Adaptation
Link Capacity Adaptation
Splices wide-band links with low capacity links.
Application aware rate adaptation.
Greater down-scalability range (4/5th vs. 1/100th)
Perceptual encoding
Interactive data compression with  perceptual control over where and how quality is compromised.
Node Capacity Adaptation
Allows network junction nodes with unequal CPUs to perform rate adaptation.
Auto-configuration by motion vector bypass
Reuses motion vector and saves computation for  quality
Cycle sweeping (network spread)
If one node is not enough it spreads the computation upstream and sweeps additional cycles from active nodes.
3 Way Tradeoff
APPROACH
Rate Adaptive Channels
Full Cascade Adaptation
MPEG-2  Rate Control
Motion Vector Bypass
Transcoder’s Self-Organizing States
3-way Tradeoff
Flow: “Single Stream Multiple Form”
PERCEPTUAL
(DOWN) SCALABILITY
MPEG-2  Spatial Rate Control
Slide 15
Sensitivity: Spatial SNR Control
DYNAMIC OBJECT TRACKING

After December demo we  tried a content based object detection and temporal scalability..
Object Tracking in Stream by Inverse Motion Vector Projection
TEMPORAL SCALABILITY MODEL
Motion Vector Transformation for Temporal Scalability
DEMO

object tracking in video
Spatial scalability
Temporal scalability
CONTROL SENSITIVITY
Reflection on Scalable Transcoding
Scalability=deeper knowledge+system symbiosis
Rate adaptation at network level? Only 80%.
We are trying to gain deeper interaction:
Content protocol aware adaptation (requantization, DCT filtering, etc.)
Content aware adaptation (object detection based).
Interactive viewer aware adaptation.
Active network insight:
Look at the source of info that forms the base of adaptation?
video protocol designer
content producer
viewer
above all-- the network resource awareness
Each knowledge component have their distinct role.  How to connecting all the  parties involved?
Few Additional Works
Reflection
Active Programmable Harness
We needed two things:
Collect link and node capacities to find out optimum transcoding points.
Now set the configuration parameters.
We consequently designed a network local state collection and propagation tool
User application sends the propagation and synthesis math, the harness takes care of the propagation by 2.5 phase communication.
A building block for transcoder deployment in a multicast network.
A generalization of SNMP
Instead of P2P it is now graph wide.
Local composition on local MIB2 elements at active nodes.
Transcoding Tree Determination
Active Harness Harness
Generalized Harness: State Model
Beyond Video: Active Hypermedia Accelerator
Sits quite at the other end of a  narrow link and monitors surfer’s behavior. Accordingly pre-fetches, caches, transcodes, and performs customized stream scheduling.
Lessons Learned:A Top Down View

What an adaptive stream might need from active network?
Reflection: Resource Adaptation
In a large network diversity is ingrained
Whether it is an adaptive service or not, but the ingredient components should not make very restrictive assumption about network resources.
This requires certain behaviors (potential “Active Network Layer Services)
Network resource awareness
State awareness about the service
Self-organization help (state maintenance signaling?)
Plug-in in the applications themselves (footprint).
Seamless reorganization
Reflections: Performance
Performance and Onion Peeling:
Encapsulation does not stop at  TCP layer it continues upward…
[IP[TCP[TS[SEQ[PIC[SLICE[MB]]]]]]]
Active network have to worry about performance of access.
To get access to one field-- generally we have to access. somewhere up to 5-10 other fields! Major performance drainage.
Current protocols = logic + field placement
Proposal = Separate logical definition from  locution in packet.
Optimum locution based on access dependency of the capsules.
Top Down Reflection
Active network feels like very crude bare-bone architectures.
Powerful yet simple construction formalism is critically required to encourage usability of the architecture and reusability of the efforts spent to develop middleware on them.
Some of our organized thoughts hinting solutions:
Language to express ‘deployment constraints’.
Language to express ‘resource requirements’.
Language to express ‘bounds of adaptation’.
“Virtual instrument”
Definitely I am out of time by now! Pl. see our publications.
Papers are available at http://medianet.kent.edu/