A depth-data-driven brand experience

Front-End

There were a few prerequisites going into the build for this application that I was able to establish at the very onset. I knew that the depth input was going to require some heavy lifting to return meaningful data. And in order for the system to run at a decent framerate, the gpu was going to likely need to play a role. So I set off to build a test front-end that could perform well above 60fps. Cinder seemed to be a perfect candidate to assist me through this.

The visual architecture is broken up into two levels which can simply be described as anchor points and control points. The anchor points act as a skewed particle grid. And the control points interpolate between their respective anchors.

In order to produce a continuous curve, I employed a cubic interpolation method found here: http://www.paulinternet.nl/

and so to get the interpolated location of a control point, say C2, it might look like this:

you can add as many segments as will suit your needs here. In this sense, they act much more like the middle segments more than control points… only a difference in nomenclature. For N points between two anchors, their offset is an interval of 1.0f/(N+1).

The idea here was to limit the number of points required to update their velocities after receiving user input.

In my initial test, I decided to test-drive Path2d to test out my math and to see how it would perform with a simple mouse input.

The red specs are bezier control points along the curve which are being interpolated by their four nearest anchors.

In this test, the particle grid was a std::vector of 50(per shape side)*2 *160(number of shapes)=16,000 objects that would resolve their position and velocity independently based on their mass and the mouse position and velocity. The framerate was too far below my predetermined ideal. It was probably around 10. This was expected though. So, it was a breakthrough as far as I was concerned.

The next step was to shift my thinking to build a polygon mesh set with a controlled number of curve segments. So I played around a little with some VBO samples provided with the cinder framework to understand how to work with and initialize vertex positions, indices and eventually texture coordinates.

Initially I was constructing a single VBO mesh and instancing it in a draw loop and iterating through the particle vector. But eventually I ended up with the code below. This is the initialization code for the VBO portion of the application. In fact, this is the only code for VBO anything other than the draw calls.

This loop will construct a cinder VBOmesh for each of the diagonal shapes in the application. Notice that I am looping through a const “VERTICES_Y” and not “ANCHORS_Y”. That is because these meshes are going to include every single point with no differentiation of whether they’re an anchor or not. Also notice that the vertex positions are not being set. These things will happen later in a shader in a very awesome way. The trickiest part about this chunk was setting up the texture coordinates to make sense for all VBOs to belong to a singular 2d texture.

Setting up the textures is the fun part because this is when you start to see all of it make sense. To grok the concept of a hardware accelerated particle system, I constantly have referred back to http://www.2ld.de/gdc2004/ and always try to keep an eye on what is going on in this thread. So a big thank you to Num3ric for his activity on this post.

This was the most revealing point of the paper for me. With 32 bit floating point precision on GPUs now, pixels of a texture can represent what I initially had built as 16,000+ objects. With a little creativity and some elbow grease to set up, this is a virtually limitless opportunity for code-artists to elevate visions.

Setup steps are pretty sensible:
1. Initialize a Surface for each particle texture and populate it.
2. Initialize Textures and populate their data with their Surfaces.
3. Initialize a 2 buffer FBO with X color buffers based on how many Textures there are and bind them.

Surfaces I need: Initial position, Initial Velocity/Mass.
Textures I need: Initial position, Position (data from initial position is used here as well), Velocity/Mass.
FBO color buffers I need: Initial position (GL_COLOR_ATTACHMENT0_EXT), Position (GL_COLOR_ATTACHMENT1_EXT), Velocity/Mass(GL_COLOR_ATTACHMENT2_EXT).
All are formatted as 32f RGBA and match the number of anchors (not the same as VBO since it includes the vertices between them as well). During surface creation, I zero out the channels I don’t need.

The 2 biggest hangups I got stuck on through this process were making sure that the formatting of Surfaces, Textures, FBO color buffers were compatible and properly set AND setting up the initial position Surface. A color channel is represented as a normalized 32f. So given a particle origin at (-500, 200) in a 1920×1080 window, the correct origin would need to be provided two new properties, particle bounds: dim, and an offset from window’s coordinates: zero_offset. the result: ((-500.0f+zero_offset.x)/dim.x, (200.0f+zero_offset.y)/dim.y).

However, there is one reason why initializing this way would be disadvantageous. Precision is unnecessarily reduced. Besides, who cares about what you can’t see?
Instead, allow the values to be capped by the nature of the texture format, GL_REPEAT. i.e. (-500.0f/window.x, 200.0f/window.y).

Setting the origin positions is really the only component that is overly-specific to this application within the cpp code-base.

The last bits of that need written — other than the shaders — are the update and draw procedures:
During update, one frame buffer’s textures are bound while the other is written to. The writing is a frag shader’s job.

You’ll notice some other crap in here, like the user textures. These relate to the next chapter, input.

During draw, a separate vertex shader takes the latest frame buffer and applies the position color buffer to the VBOmeshes.

coordInc is probably more commonly known as a step in the Y direction of the texture coordinates.

If you want to jump straight to how Part III – Shaders correlate, you can. But you might want to know how Part II – Input applies first.

Part II – Input