FINALLY!
I finally got this working the way I wanted it tonight. It was such a simple solution I have no idea why it took me so long to get it…
You can now move in and out and the cube will scale appropriately. You can also go out of frame and the cube will no longer jump back to the default settings…it will ease back into frame once you move back.

The way I calculate Z depth is by taking the two IR locations on the X-axis and getting the difference. Basically triangulating them…so when the glasses are closer to the WiiMote the distance is physically greater apart, giving me a larger number, and when I was far away it was a smaller one. This all made sense except I couldn’t get a formula that worked well to actually make the camera move the way I wanted it to.
No matter what I did larger numbers always ment closer, but adding lager numbers onto the camera distance only made it get further away. And I couldnt just subtract it because that would cause the camera to constantly be moving away…it was a chore.
In the end I found that simply by choosing an absolute value for the camera’s distance I could simply move it closer and further. It turns out I was over thinking the whole process…

Check out the formula for movement in the Y direction:

camera.y -= (camera.y-(rY*0.5-w1Y)/(rY*0.5)*1200)*0.2;

And the Z:

camera.z = -2000 + (w1Z*2);

Much simpler…no? The -2000 will end up being dynamic based off some screen resolution tests. I’m running 1440×900, so maybe it will just be ResX+500 or something…I’ll figure that out later.

Anyway, since like always you can’t really run any of the examples I post up, I figured I make a video clip of it. Take this, and add 1 million more awesome points to it and that is how it looks in real life.