GLSL Vignette and Alpha

The GLSL Vignette example in the dropbox here is a fairly commonly downloaded item. The code itself is something I probably picked up from the Orange Book or Clockwork Coders GL examples (?). The code is something I see pop up again and again, in all kinds of GLSL examples all over the internet; it's simply the way to make a Vignette, unquestioned.

One thing that irks me about the method, is that when you're in any system that actually respects alpha as being clear, instead of black, you can wind up with unexpected results:

The top is the result of the vignette method that everyone and their brother uses. Now, for example, in QC, when you're working in the QC Editor app, it *will look like the bottom image*, but if you take a snapshot or process as a movie that respects alpha channel, you'll wind up with something that looks like the top image.

The bottom is the result of *not performing the vignette function on the alpha channel*. Instead, I've dropped that from that part of the function, and just made black be a 1.0 value.

Seeing downloads for this (and a few today), made me finally take the minute to do this, and upload and example. With the proliferation of work with GLSL fragment shaders in environments that don't necessarily "show" you that Alpha "Black" is not the same as Black (particularly WebGL stuff), it's good to keep in mind what's really going on there.


uniform sampler2D Image; // 0
uniform vec2 Vignette;
void main()
{
  vec3 rgb = texture2D(Image, gl_TexCoord[0].xy).xyz;
  float d = distance(gl_TexCoord[0].xy, vec2(0.5,0.5));
  rgb *= smoothstep(Vignette.x, Vignette.y, d);
  gl_FragColor = vec4(vec3(rgb),1.0);
}


Download: https://www.box.net/shared/yrr67e7y748behddd9a4

Clod Cousin

"You Massive Clod", by Fearmoths, is pretty famous in the demoscene, and possible reason for that is that not only does it look cool, but the source is also available. Tigrou posted a version for Windows that contained the important shader code here: clod_win32 .

I was looking at it today, and decided to replace the portion that's setting up the distance equations with what started as a torus. Yesterday, when manipulating a torus in another program, I noticed how when repeating the torus with a sin or cos, and tweaking other parameters, the general shape was very "clod-like". This made me interested to see how the same equation would look with the clod coloring and lighting scheme.

I wound up really liking it.

I included the optimization by iq, which removes the break from the loop that does the ray marching. I decided to let the mouse/trackpad influence the camera viewpoint as well as the distance equation. I also ended up adding a vignette function.

The file is here, and requires Quartz Composer: Clod Cousin

Frag-gle Rock

The theme of this blog entry is something like "fragment shaders rock", mixed with some kind of weird conflict about it. :-)

I read Iñigo Quilez's "Rendering Worlds with Two Triangles with raytracing on the GPU in 4096 bytes" not long after it came out, and was really fascinated with the principles that were outlined, but some of it was extremely far away from the type of programming I was doing at the time (mainly image processing based Java,  Processing, and some Q.C.). I also read the infamous "Zeno.pdf" from 2000, where much of this originated. (Here's a treat; I've noticed links for this that used to be the "go-to" places to download it from seem to be broken, so here's a link to my copy of the  Zeno.pdf .)

At that time it sounded really great, but my computer couldn't actually run many of the shaders referenced. The intel gpu just took a poop while running shaders that in "modern day"(a few years later!), run exceedingly quickly. So, at that time it became a bit of a curiosity, more than something that I considered that practical, at least for me and the hardware I had.

Over the years, I've noticed that the hardware I've encountered (my current computer, and others), run these kind of fragment shaders pretty well, to blazingly fast. With the release of Lion OS X, it's unveiled a bit of a new world where Mac users not only have access to "modern GL" through Core Profile, but the "classic flavor" fixed function pipeline GL stuff, can be accessed in web browser in many cases, which is tremendous. It seems to be causing there to be more innovation in what can be done in GLSL 1.2, entirely in the fragment shader, or at least, fueling people to share past innovation.

So, I have these moments, where I'm programming something in OpenGL in a more traditional way, generating a bunch of vertices, texturing them, etc. In the back of my mind, I can't help but thing "George... the fragment shader will give you better lighting and almost freebie shadows. George, the fragment shader renders countless replicas of an object with basically no measurable hit. George, the fragment shader will let you mix between objects quickly." Then I think, "oh yeah, none of that exactly matters for this instance."

There's the real world, knocking at the door. QC doesn't have a Core Profile context. So, you can't do "modern GL" stuff, like treat pixels and structures the same, use geometry shader(well, this isn't core profile, but still), declare multiple outs, etc. If you want to pass something like a structure, what can you do? Create a texture, and read the pixel values as if they're xyzw type values, do whatever math needs to be done to get it to "work" for you, and deal with it (the only workaround I've been able to think of). Then, you're fighting limited steps of resolution, texture limits, potentially stuff like Mac OS X color correction, color curves not working right, etc., etc. There's a whole toolbox that needs to be worked up, which is both exciting, and a big pain in the butt.

Working strictly with Core Profile is a possibility (theoretically), but I feel like it's a little too bleeding edge to go full bore with it on Mac right now. Maybe it's because I've been burned by lack of consistent support for OpenCL, and even fixed function OpenGL stuff across Macs. I'm even a little leery of using good old GLSL 1.20 shaders too much, lest I be caught with my pants down because of some obscure GPU bug, and that should be totally solid by now. I also wonder... do all of the new Macs, that have Intel GPU's, run this stuff ok? I don't really know, but past experiences with Intel GPU's make me really wary (like, the reason I originally *didn't* look into heavy frag shader use some years back, with my piss-poor X3100).

Still, I feel like there's going to be increasing interest in doing stuff "all in the fragment shader", and probably in other technologies that allow one to "talk to" the gpu in similar ways.

That's it for now :-)

Here are some pics of some "all fragment shader" stuff. This stuff owes vast credit to observations made looking at Iñigo Quilez, Paulo Falcao, Knighty, Las, Tigrou, psonice, code from ShaderToy and GLSL Sandbox, and others, I'm sure. There are pics derived from heavy mashups of the Menger Sponge, 704, Knighty's palindrome/kaleidoscope theory, etc. They're are all "real time", and I've added blurring and some heavy sepia, and some other  color curving to the stuff.

Leak Testing Qtz's / Instruments.app Redux

One thing I've found, through trial, error, gnashing of the teeth, etc., is that the QuartzComposer.app, leaks like a sieve. 

There are *many* times (almost every?) that patch instances on the editor actually leak the contents, and few that they don't. 

So, essentially, the editor is a tool for making the graphs/qtz's, but it's just not a great idea to use it for deployment. Furthermore, when leak testing, there are so many leaks that the Editor inherently has, that it's necessary to shift to running your qtz in the context of an Xcode project to run the composition, and monitor that for leaks. Then you have a baseline where if something is truly problematic, it's easier to see if it's your programming, any 3rd party plugins, or an unavoidable "qc" leak. Again, this means, running stuff in the QuartzComposer.app (the Editor), just for deployment in general, is not a great idea. 

All of these comments are informed by experiences with 10.6. I haven't tested every bit of this in 10.7, and while the Editor seems to have less leaks, this is probably still a very good idea.

You may also notice large performance boosts as well; I immediately got a 20~30 fps boost with the composition in question just from not running in the Editor.

QC Evaluation

https://developer.apple.com/library/mac/#documentation/GraphicsImaging/Conceptual/QuartzComposerUserGuide/qc_concepts/qc_concepts.html#//apple_ref/doc/uid/TP40005381-CH212-SW9

Figure 1-5, the "Evaluation Path", does a great job of illustrating evaluation and how making things into "macros" can change evaluation.

This past week, I'd been finding a scenario where movie playback on something was doing an annoying halt at an unpredictable time, inside of QC. With the qtz running in an actual Xcode application (not the Quartz Composer editor), I don't think I would ever notice the little "tick". It was very minimized in comparison to how it exhibited itself in the Quartz Composer (editor).app.

After looking at it more, I decided, "why not see what happens if I undo the macro?". The patch itself had multiple input splitters running to it, and "exploding macro" revealed that it had more inside.

So, in order for the value to get propagated to the port of a patch that could do something to it, it was already going through two superficial patches (that unfortunately do execute code, and rob cycles), and an additional macro layer, adding another port. Exploding macro made it more obvious that there were two input splitters going on, when may not have been as immediately obvious. Before removing those, I simply ran the composition. The little tick in movie playback disappeared in the actual QC Editor, and was just as unnoticeable in the app context. Movie playback wasn't flying (because of other issues with codec, patch, etc)., but the uneven seeming evaluation had vanished.

With QC, keeping things with a minimal amount of nesting of macros seems to be an important principle in keeping performance consistent.

It also seems that, in general, external providers and related processors should be placed at top level of the patch whenever possible; there is almost never any scenario where a composition can't be reorganized to achieve this.

QC wraps up many things in a way that makes them less obvious to the end user. For example, why wrap up a video provider, or processor patches inside of a 3D transform? Do you need to be performing a GL Transform on those patches? Not really. (I'm not inferring that a GL Transform takes place on patches that won't respond to it). Keeping the provider and processor patches at the root of your compositions tends to make all of the values that dictate the "logic" of your patch, available to all consumer macros that may need it. It also tends to lead to smoother and faster evaluation of your graph.