Adding Filter Effects to Video via Canvas
Problem
You’re interested in not only playing video in your web page but also playing modified versions of the video, such as one that has been grayscaled, or manipulated in some way.
Solution
Use HTML5 video with the Canvas element, playing the video to a scratch Canvas
element:
function drawVideo() {
var videoObj = document.getElementById("videoobj");
// if not playing, quit
if (videoObj.paused || videoObj.ended) return false;
// draw video on canvas
var canvasObj = document.getElementById("canvasobj");
var ctx = canvasObj.getContext("2d");
ctx.drawImage(videoObj,0,0,480,270);
...
setTimeout(drawVideo,20);
}
You can then add the ability to capture the image data using the Canvas element’s getImageData(), modify the image data with whatever filter you want, and then play the image data to a second, visible Canvas element:
var pData = ctx.getImageData(0,0,480,270); // grayscale it and set to display canvas pData = grayScale(pData); ctx2.putImageData(pData,0,0);
EXPLAIN
The best thing about all the new HTML5 media elements is how you can use them
together. Using video and Canvas, you can not only provide custom controls, you can
also provide custom video filters, too.
To play a video in a Canvas element, we’ll need to add both elements to the web page:
<video id="videoobj" controls width="480" height="270"> <source src="videofile.mp4" type="video/mp4" /> <source src="videofile.webm" type="video/webm" /> </video> <canvas id="canvasobj" width="480" height="270"></canvas>
Both the canvas and video elements are the same width and height. To draw the video onto the canvas, we’re going to use the Canvas drawImage(). There are several variations of parameters we could use with this method, but the signature we’re interested in is the following:
void drawImage( in nsIDOMElement image, in float dx, in float dy, in float dw, in float dh );
These parameters are:
• image: A reference to a Canvas element, an img element, or a Video element
• dx: x coordinate of the top-left corner of the source image
• dy: y coordinate of the top-left corner of the source image
• dw: Width of the source image (can be scaled)
• dh: Height of the source image (can be scaled)
Demonstrates a first pass at an application to modify the video playing the Canvas element. It just takes what’s showing in the video and plays it, as is, in the Canvas.
The application uses setTimeout() to test whether the video is still playing and grabs the video every 20 milliseconds, which is fast enough to provide smooth playback for human perceptions.
There is a timeupdate event handler for the Video element, but it’s only invoked every 200 milliseconds (per the W3C specification on the media elements), which is way too slow for our purposes.
A first cut at drawing video data to a canvas element
<!DOCTYPE html>
<head>
<title>Play video in canvas</title>
<meta charset="utf-8" />
<script>
window.onload=function() {
document.getElementById("videoobj").
addEventListener("timeupdate", drawVideo, false);
}
function drawVideo() {
var videoObj = document.getElementById("videoobj");
var canvasObj = document.getElementById("canvasobj");
var ctx = canvasObj.getContext("2d");
ctx.drawImage(videoObj,0,0);
}
</script>
</head>
<body>
<video id="videoobj" controls width="480" height="270">
<source src="videofile.mp4" type="video/mp4" />
<source src="videofile.webm" type="video/webm" />
</video>
<canvas id="canvasobj" width="480" height="270"></canvas>
</body>
In the code, during each time out event, the video is tested to see if it’s still playing before it’s grabbed and displayed in the Canvas element. The application works in all modern browsers.
The next step is to modify the video data before it’s streamed to the Canvas element. For the example, I’m going to do a crude modification of the original video to simulate how a person could perceive the video if they suffered from a form of color blindness known as protanopia.
This type of color blindness is one of the most common, and those who have it can’t perceive red light. I say “crude” because a much more accurate representation is so computationally expensive that the playback visibly stutters.
To modify the video playback, we need two things: the function to modify the data, and a scratch canvas object used to capture the video data, as it is, and then serve as our intermediate in the transformation.
We need a scratch Canvas element because we’re using the Canvas element’s getImageData() to access the actual video data, and putI mageData() to play the video data after it has been manipulated.
The getImageData() function returns an object consisting of three values: the width, the height, and the image data as a Uint8ClampedArray typed array.
Video with applied color blind filter, playing side by side with original
<!DOCTYPE html>
<head>
<title>Protanopia</title>
<meta charset="utf-8" />
<script>
// Protanopia filter
function protanopia(pixels) {
var d = pixels.data;
for (var i=0; i<d.length; i+=4) {
var r = d[i];
var g = d[i+1];
var b = d[i+2];
//convert to an approximate protanopia value
d[i] = 0.567*r + 0.433*g;
d[i+1] = 0.558*r + 0.442*g;
d[i+2] = 0.242*g + .758*b;
}
return pixels;
}
// event listeners
window.onload=function() {
document.getElementById("videoobj").
addEventListener("play", drawVideo, false);
}
// draw the video
function drawVideo() {
var videoObj = document.getElementById("videoobj");
// if not playing, quit
if (videoObj.paused || videoObj.ended) return false;
// create scratch canvas
var canvasObj = document.getElementById("canvasobj");
var bc = document.createElement("canvas");
bc.width=480;
bc.height=270;
// get contexts for scratch and display canvases
var ctx = canvasObj.getContext("2d");
var ctx2 = bc.getContext("2d");
// draw video on scratch and get its data
ctx2.drawImage(videoObj, 0, 0, 480, 270);
var pData = ctx2.getImageData(0,0,480,270);
// grayscale it and set to display canvas
pData = protanopia(pData);
ctx.putImageData(pData,0,0);
setTimeout(drawVideo,20);
}
</script>
</head>
<body>
<video id="videoobj" controls width="480" height="270">
<source src="videofile.mp4" type="video/mp4" />
<source src="videofile.webm" type="video/webm" />
</video>
<canvas id="canvasobj" width="480" height="270"></canvas>
</body>
Join Our Telegram Channel to Stay Updated
![Javascript DOM Tutorial Part 1 [ Selectors ] How to Select HTML Elements Using Javascript](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGFE3NttPjKRUzbBqnGQNT8mtxG71mrDGLb-k8f-rw_I9Ov3Tu9uIFQtUgoxOozsx2TNf449IJ7SPgrEFFqrDdN98mWUWXsp8zFUteAmA7UaXDDjyQHE5b0oVkCzXT_aIgCVFD_ksqt18/s72-c/jspart10.jpg)
No comments:
Post a Comment