Google developers logo

populated from slide_config.json


Two trends
The rise of mobile
The rise of media

Offline media

53% of adults media multi-task while watching TV

Online media

Video will be 80 to 90 percent of global internet consumer traffic by 2017

Chrome media technologies


WebRTC across platforms

  • Chrome
  • Firefox
  • Opera
  • Java
  • Objective-C: API, app
  • Qt
Firefox/Chrome interoperability Firefox/Chrome interoperability
WebRTC endpoints
Sky TV interview done via WebRTC
Webcam and Yeti mic used for Sky interview
RTC via a server
Peer to peer
RTC peer to peer

What do we need for RTC?

  • Acquire audio and video
  • Establish a connection between peers
  • Communicate audio and video
  • Communicate arbitrary data

Three JavaScript APIs

  • MediaStreams (aka getUserMedia)
  • RTCPeerConnection
  • RTCDataChannel

Communicate Media Streams

WebRTC video chat: caller

WebRTC video chat: callee

RTCPeerConnection does a lot

  • Signal processing
  • Codec handling
  • Bandwidth management
  • Peer to peer communication
  • Security


Communicate arbitrary data

Game: caller
onreceivemessage = handle(data); ... var myData = [ { id: "ship1"; x: 24, y: 11, velocity: 7 }, .... ] send(myData);

onreceivemessage = handle(data); ... var myData = [ { id: "ship7"; x: 19, y: 4, velocity: 18 }, .... ] send(myData);
Game: callee


  • Same API as WebSocket
  • Ultra-low latency
  • Optionally unreliable or reliable:
    — Firefox and Chrome 31, Chrome 30 behind a flag
  • Secure
Peer to peer file distribution with RTCDataChannel

Access local media


It's pretty simple.

var constraints = {video: true};

function successCallback(stream) {
  var video = document.querySelector("video");
  video.src = window.URL.createObjectURL(stream);

function errorCallback(error) {
  console.log("navigator.getUserMedia error: ", error);

navigator.getUserMedia(constraints, successCallback, errorCallback);

gUM screencapture!

Media Stream Recording API

Media Stream Image Capture API

  • Demo
  • Spec
  • getFrame() creates an ImageData object available in onframegrab
  • takePhoto() creates a Blob available in onphoto

Web Audio

Why Web Audio when we have <audio>?

  • Precise timing of multiple sounds
  • Audio pipeline for routing and processing
  • Analysis of audio data
  • <audio> can be used as input to Web Audio!

The Web Audio pipeline

  • Oscillators
  • 3D positioning of sounds
  • Fade-ins/fade-outs/sweeps
  • Time-based event scheduling
  • Doppler shift for moving sources
  • Frequency and waveform analysis
  • Acoustic environments: reverb, etc.
  • Waveshaping (non-linear distortion)
  • Dynamics processing (compression)
  • Filtering effects: radio, telephone, etc.
  • Low latency: input→speaker as low as 5ms
  • Integrated with WebRTC through MediaStream

Web Audio is cross-browser!

  • Chrome desktop and Android — including gUM input
  • Firefox 25 desktop and Android!
  • Safari 6.0+ and iOS6+
More information? Make the Web Rock!

getUserMedia ☞ Web Audio

// Success callback when requesting audio input stream
function gotStream(stream) {
    var audioContext = new webkitAudioContext();

    // Create an AudioNode from the stream
    var mediaStreamSource = audioContext.createMediaStreamSource(stream);

    // Connect it to the destination or any other node for processing!

navigator.webkitGetUserMedia( {audio:true}, gotStream);

gUM ☞ Web Audio ☞ RTCPeerConnection

Microphone ☞ processing ☞ stream!

navigator.getUserMedia('audio', gotAudio);
function gotAudio(stream) {
  var microphone = context.createMediaStreamSource(stream);
  var filter = context.createBiquadFilter();
  var peer = context.createMediaStreamDestination();
More Media Stream integration examples


  • New standard under development
  • Connect controllers, synthesizers and more
  • Implemented in Mac Chrome 30 behind a flag, others coming

Synths and drum machines

Codecs for the modern Web

VP8 and VP9

Cross-platform tools


Media Source Extensions

JavaScript can generate streams for playback


Dynamic Adaptive Streaming over HTTP

Encrypted Media Extensions

Synchronised metadata

<video poster="images/poster.jpg"
  autoplay preload="metadata">
  <source src="chrome.webm" type="video/webm" />
  <source src="chrome.mp4" type="video/mp4" />
  <track src="track.vtt" />
  <p>Video element not supported.</p>
In-band WebVTT: track data

And finally...

Chrome media technologies

Find out more

What works where


More about media