GLSL - Extrusion and Noise Example - Kinect




I was inspired by the very cool plugin from franz of _1024 that was posted yesterday, which turns the kinect tools (http://kineme.net) output into points, and then makes a noise flurry.... sort of a "snow flurry", so in carrying on with the winter theme, I figured I would give a GLSL shader version rundown today.

I decided to program the same basic functions in OpenGL/GL Shader Language, using the QC Shader patch to run the code, over my morning coffee. This is because I wanted to be able to customize in a few ways; mainly, I wanted to be able to get Depth Testing with the look so that near pixels draw front to back, but I also wanted to be able to use other polygon modes besides points, and I wanted possibility of texturing differently down the road, adding lattice type effects (making something render in square wireframe, or vertical/horizontal striped looks), or manipulating normals.

It didn't take too much time to get an alpha, and thankfully, one can just hit "Run" QC, and instantly check the shader results without compiling *and* have the boilerplate setup stuff already done and optimized (and ready to take advantage of GPU in many scenarios).


This is available in the "Box" widget/link.

This is taking in a depth image, and extruding the GLSL grid based off of luminosity. In this case, the Kinect Tools Depth Image is being sent to the shader. There is an additional input to texture the mesh off of.

Then, there is a vertex noise routine that can be added in with the vertices made by the extrusion routine. In addition to this, there is a setup that creates a clip of far geometry by coloring those pixels alpha.... and that's the nuts and bolts of the effect!

I'm using GL Tools to make the calls for points, lines or triangle, and adding in a v002 Dilate Option for fattening up the depth image in a pleasing way (the non-stock options I'm throwing into the mix; http://kineme.net and http://v002.info for downloads.)

(I'm using the 10.6 3D transform for scaling, so if you have 10.5 installed, you may wish to use GL Scale (GL Tools, Kineme).)

Vertex Program

//george toledo, 2010. www.georgetoledo.com, gtoledo3@gmail.com
uniform sampler2D depthMap, texture;
uniform float amount,clip;
varying float displacement;
uniform vec3 offset;
uniform float scaleIn;
uniform float scaleOut;

void main()
{
gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
displacement=float(texture2D(depthMap, gl_TexCoord[0].xy).x)-clip;
vec3 vertex = gl_Vertex.xyz + noise3(offset + gl_Vertex.xyz * scaleIn) * scaleOut;
vec3 normal = gl_Normal;
normal = normalize(gl_NormalMatrix * normal);
//vec3 position = vec3(gl_ModelViewMatrix * vec4(vertex,1.0));
gl_Position = gl_ModelViewProjectionMatrix * (gl_Vertex+vec4(0,0,displacement*amount,0)+vec4(vertex,1.0));
gl_FrontColor = gl_Color;
}

////

Fragment Program

uniform sampler2D texture;
varying float displacement;

void main()
{
gl_FragColor = gl_Color*texture2D(texture,gl_TexCoord[0].xy)*vec4(1.,1.,1.,float(displacement>0.));
}

To run the composition en totale, please install Kinect Tools, and GL Tools from Kineme, and for using the dilate processing option, install v002 Dilate.