Andy Li's Blog

Here above is my haXe implementation of head-tracking 3D, creating motion-parallax effect using Away3D 4. You can play with the live demo or grab the source.

This type of head tracking VR has been around for quite a long time. The first popular one should be Johnny Lee’s Wiimote hack, back in 2007,  built in C# running as a desktop app. Two years later it appeared in the browser, a Flash version made by Mr. doob. But it was merely a hack and far from accurate comparing to Johnny Lee’s. Boffswana also created one in the same year (2009), and then an improved one with better head tracking algorithm. Sadly Boffswana hasn’t release the source.

Head tracking and camera movement is relatively simple, since there are so many libs for these. The tricky part is the perspective projection matrix. The projection point (the head), unlike most of the implementation assumption, isn’t always perpendicular to the projection surface (screen).

Simplest implementation would ignore the problem. It places a camera at the position of viewer’s head and render it to the screen, like what Mr. doob did. The problem is illustrated below.

When viewer’s head is perpendicular to the screen, everything is perfectly aligned:

However, when the viewer moves, for example to the left, there is misalignment of the rendering on screen and the “actual” position of the virtual object:

Knowing little about C#, it is hard for me to dig out Johnny Lee’s matrix code. I found the suitable code form the paper “Generalized Perspective Projection” from Robert Kooima. It is written in C++, but translating it to haXe isn’t hard, just remember matrix in OpenGL is column-major order but Flash’s is row-major order.

Head-detection is done using my fork of hxmarilena. My fork is simply some API changes and switching the XML parsing from flash.xml.XML to Xml so it may be used in C++ target in the future. In order to reduce jittering, I’ve included a simple optical flow tracking on the head. It is a block-matching process applied to four points on previous head detection result. The optical flow tracking result is merged to head-detection result by a ratio. It improved a bit, but I guess to address the problem it is better to port some better algorithms, like FaceTracker, in the future.

Finally, remember you can play with the live demo and it is open source.

The generative arts are going to be shown in Playful Media, which is the showcase of BScCM, the Bachelor degree program I graduated last year. This is going to be my 4th year joining the exhibition(2010, 2009, 2008)! If you are in Hong Kong and have time, come to have a look!

It’s almost half a year, coding a piece of generative art everyday using haXe and openFrameworks. It was certainly fun and educational to me, as I had to keep reading, trying and mixing all those generating methods, brainstorming new algorithm, looking for best programming approach etc. Doing all those things everyday sharpen my skill a lot, and I’m really excited when people tweet about them, leave comment here or over Flickr. Thanks guys!

However, I decided to stop continuing the journey (at least for a while). Occurring to me often in the last few weeks, I either spent too much time on one piece, affecting my school works, or because I have to finish some urgent things, I tweaked only a little on the previous code, resulted in producing an unsatisfactory piece. I was sick and had a fever last week, which became a good time to close the project.

I now have more time to play with other interesting ideas, and finishing the works that queued up. hxOpenFrameworks will continue be developed, of course. Let’s expect some OpenCV, Kinect (OpenNI) and other goodness

Finally here are the selected pieces I made in June:

Kept the fractal circles idea going.

Some more saturated color with complementary color scheme.

Reduced opacity, the circles merged nicely to each other.

Simplified but surprisely eye-catching vector-style flower.

I ported Eugene Zatepyakin(in-spirit)’s AS3 FluidSolver to haXe and started to play with it.

Here I put several rotating color sources in the center and push out dark red colors. Pretty horror looking.

More sources, larger radius. It looks like an eclipse, does it?

An eye-catching fire ring. The key is to add color and force periodically other than continuously.

Simulating the diffusion of ink on paper.

A cup of generative latte. Think I can build an app for making coffee art.

Another color, another pattern.

The fifth month of doing a piece of generative art everyday using haXe and openframeworks. As a reminder, all the pieces can be viewed from my Flickr set. Also the haXe source code for generating those images can be found in the description in the corresponding Flickr page. Here below are the selected pieces made in this month:

Drawing only part of the lines generated by midpoint displacement algorithm can give out some interesting unknown creature contours, looking into each other’s eyes.

Another arrangement. It now looks like an island.

Used a algorithm that I read from somewhere, that start with a point with some color, for each iteration the color spread to the neighbors with a little variation. I really love its oil-painting texture.

Increased the spread’s random factor, allowing it to spread a little bit backward.

Keeped changing the spread implementation. This one keep increasing its hue from the center with a probability of reseting itself (to red).

Made the spread much more unstable and let there be more than one starting point. Resetting the lightness to 0 periodically gave out interesting organic layers.

Well, a beautiful rainbow color cup water mark on tablecloth…?

Played with fractal patterns. Full of circles in different sizes and positions in ratio.

Used only small circles. Pretty cyber-looking.

Used solid-filled circles. Colorful snowflakes.

Simpler smaller patterns.