Andy Li's Blog

Impressed by the video here, I tried to simulate the atomic model of metal in Flash using Box2DFlash.

What is done in the program is just put a lot of spheres in the area randomly, and then let the physics engine to solve all the collisions. When all the atoms stop moving, a single still of the metal structure will be formed.

The atoms are only drawn on the screen when it is sleeping (ie. no movement). It can save some time from not rendering them in each frame.

caution: The browser will freeze for a few seconds after “start” is pressed. And it takes several minutes to an hour to finish (when sleep count == atom count).

[flickr-gallery mode="photoset" photoset="72157621394341446"]Click images above to enlarge. Impressed by the video here, I tried to simulate the atomic model of metal in Flash using Box2DFlash. What is done in the program is just put a lot of spheres in the area randomly, and then let the physics engine to solve all the collisions. When all the atoms stop moving, a single still of the metal structure will be formed. The atoms are only drawn on the screen when it is sleeping (ie. no movement). It can save some time from not rendering them in each frame. caution: The browser will freeze for a few seconds after "start" is pressed. And it takes several minutes to an hour to finish (when sleep count == atom count). [SWF]http://blog.onthewings.net/wp-content/uploads/2009/07/atomicModelSim.swf, 640, 480[/SWF] source

Recently I need to write a 3D tool to help making a installation artwork. I know I should use Maya, Processing or anything like that, but in order to get the least risk, I used my most experienced language, AS3.

Since I need to render A LOT of stuff (~400,000 objects), instantly I faced the performance issue. As you can imagine, even if I could generate all the objects in 15 seconds, the rendering process must excess the Flash Player’s  execution time limit.

So I try to make my own async render engine with the help of async-threading. I extended BasicRenderEngine to AsyncBasicRenderEngine. The class even has a showBusyCursor property that let you optionally show the busy cursor on rendering.

But note that this is only a proof-of-concept and actually has little help of preventing freeze-screen. It is because I only distribute the individual 3D object’s rendering to frames but leaving the sorting, filtering, post-process code unchanged. Also, the RenderStatistics returned by renderScene/renderLayers will contain only wrong data until the RENDER_DONE event is dispatched.

One last thing, Flash’s drawing algorithm seems affecting the async rendering performance too. So I do not add the viewport to stage. Instead, once RENDER_DONE, I draw the viewport on a Bitmap on stage.

source

Recently I need to write a 3D tool to help making a installation artwork. I know I should use Maya, Processing or anything like that, but in order to get the least risk, I used my most experienced language, AS3. Since I need to render A LOT of stuff (~400,000 objects), instantly I faced the performance issue. As you can imagine, even if I could generate all the objects in 15 seconds, the rendering process must excess the Flash Player's  execution time limit. So I try to make my own async render engine with the help of async-threading. I extended BasicRenderEngine to AsyncBasicRenderEngine. The class even has a showBusyCursor property that let you optionally show the busy cursor on rendering. But note that this is only a proof-of-concept and actually has little help of preventing freeze-screen. It is because I only distribute the individual 3D object's rendering to frames but leaving the sorting, filtering, post-process code unchanged. Also, the RenderStatistics returned by renderScene/renderLayers will contain only wrong data until the RENDER_DONE event is dispatched. One last thing, Flash's drawing algorithm seems affecting the async rendering performance too. So I do not add the viewport to stage. Instead, once RENDER_DONE, I draw the viewport on a Bitmap on stage. [SWF]http://blog.onthewings.net/wp-content/uploads/2009/06/asyncrendererdemo.swf,820,620[/SWF] source

Reasons why should we use colored marker instead of standard black and white only marker

Using a black and white only marker is boring

There are more and more AR applications in the world, but seems everybody use only the black square marker. May be the public is not yet bored by the black square yet, but if can use other color for the square, we can integrate the marker into real-life objects more beautifully and more flexible.

Increase marker detection performance

The internal algorithm of AR marker detection has a characteristic that, the less pixels’ color is the same of the marker, faster the detection can be done. When we use black square marker, the detector will waste a lot of time to process non-marker pixels as you can imagine that it is common to have black objects in real-life (especially we the Chinese people have black hair). So if we target less common color, the detector will be able to ignore most of the non-marker area.

How to achieve this in FLARtoolkit

We will create a custom detector class since we need to work at lower level of the FLARtoolkit. If you have not look into the internal classes of FLARtoolkit (because the FLARmanager is so much easier to work with ), there are two detectors: FLARSingleMarkerDetector and FLARMultiMarkerDetector. As their names suggest, one is for single marker detection and the other one can detect multiple markers. For simplicity, I will go through only the single detection for a red marker (if you want me to cover the other ones too, leave comment).

First we just copy all the codes inside FLARSingleMarkerDetector to a new class, I named it FLARSingleRedMarkerDetector.

To detect markers, we use the detectMarkerLite method, which is:

public function detectMarkerLite(i_raster:IFLARRgbRaster, i_threshold:int):Boolean

IFLARRgbRaster wraps a BitmapData, we can get it by:

var srcImg:BitmapData = (i_raster as FLARRgbRaster_BitmapData).bitmapData;

Originally, the method uses a threshold filter to extract black areas and give the result to other classes to process. As you might think of, we can simply swap the filter code. Originally the black areas will be turned to white pixels and others will be turned black and be ignored. We now want to detect red markers, so just try to turn everything black except the red regions to white. You may use ColorTransform if you want, but I am now obsessed with PixelBender. My version of kernel is:

kernel extractRedARMarker
<   namespace : "net.onthewings.filters";
    vendor : "Andy Li";
    version : 1;
    description : "used for FLARtoolkit, pre-proccess for red marker.";
>
{
    input image4 src;
    output pixel4 dst;
 
    parameter float threshold<
        minValue: 0.0;
        maxValue: 1.0;
        defaultValue: 0.4;
        description:"decrease to increase likelihood of marker detection.";
    >;
 
    void
    evaluatePixel()
    {
        float4 p = sampleNearest(src,outCoord());
        float sum = p.r+p.g+p.b;
        float val = p.r - (p.g + p.b)*0.5;
 
        if (val+(1.0-(sum/3.0))*0.1 <= threshold) {
            val = 0.0;
        } else {
            val = 1.0;
        }
        dst = float4 (val,val,val,1);
    }
}

Note that I let the dark red pass the threshold easier since I find it is common the web cam image is too dark.

To use the finished detector class:

tempFLARRgbRaster_BitmapData.bitmapData.draw(videoDisplay);
if (detector.detectMarkerLite(tempFLARRgbRaster_BitmapData,170)){
	detector.getTransformMatrix(lastDetectionResult);
	//FLARPVGeomUtils is from FLARmanager
	overlay3dObj.transform = FLARPVGeomUtils.convertFLARMatrixToPVMatrix(lastDetectionResult);
	overlay3dObj.visible = true;
} else {
	overlay3dObj.visible = false;
}
render();

Now you get all the concepts. So let’s skip all the boring stuff and see it in action. Or get the finished codes.

Happy AR!

[youtube]http://www.youtube.com/watch?v=DY7TPBRPpFI[/youtube] Reasons why should we use colored marker instead of standard black and white only marker Using a black and white only marker is boring There are more and more AR applications in the world, but seems everybody use only the black square marker. May be the public is not yet bored by the black square yet, but if can use other color for the square, we can integrate the marker into real-life objects more beautifully and more flexible. Increase marker detection performance The internal algorithm of AR marker detection has a characteristic that, the less pixels' color is the same of the marker, faster the detection can be done. When we use black square marker, the detector will waste a lot of time to process non-marker pixels as you can imagine that it is common to have black objects in real-life (especially we the Chinese people have black hair). So if we target less common color, the detector will be able to ignore most of the non-marker area. How to achieve this in FLARtoolkit We will create a custom detector class since we need to work at lower level of the FLARtoolkit. If you have not look into the internal classes of FLARtoolkit (because the FLARmanager is so much easier to work with ;-) ), there are two detectors: FLARSingleMarkerDetector and FLARMultiMarkerDetector. As their names suggest, one is for single marker detection and the other one can detect multiple markers. For simplicity, I will go through only the single detection for a red marker (if you want me to cover the other ones too, leave comment). First we just copy all the codes inside FLARSingleMarkerDetector to a new class, I named it FLARSingleRedMarkerDetector. To detect markers, we use the detectMarkerLite method, which is: public function detectMarkerLite(i_raster:IFLARRgbRaster, i_threshold:int):Boolean IFLARRgbRaster wraps a BitmapData, we can get it by: var srcImg:BitmapData = (i_raster as FLARRgbRaster_BitmapData).bitmapData; Originally, the method uses a threshold filter to extract black areas and give the result to other classes to process. As you might think of, we can simply swap the filter code. Originally the black areas will be turned to white pixels and others will be turned black and be ignored. We now want to detect red markers, so just try to turn everything black except the red regions to white. You may use ColorTransform if you want, but I am now obsessed with PixelBender. My version of kernel is: kernel extractRedARMarker < namespace : "net.onthewings.filters"; vendor : "Andy Li"; version : 1; description : "used for FLARtoolkit, pre-proccess for red marker."; > { input image4 src; output pixel4 dst; parameter float threshold< minValue: 0.0; maxValue: 1.0; defaultValue: 0.4; description:"decrease to increase likelihood of marker detection."; >; void evaluatePixel() { float4 p = sampleNearest(src,outCoord()); float sum = p.r+p.g+p.b; float val = p.r - (p.g + p.b)*0.5; if (val+(1.0-(sum/3.0))*0.1 <= threshold) { val = 0.0; } else { val = 1.0; } dst = float4 (val,val,val,1); } } Note that I let the dark red pass the threshold easier since I find it is common the web cam image is too dark. To use the finished detector class: tempFLARRgbRaster_BitmapData.bitmapData.draw(videoDisplay); if (detector.detectMarkerLite(tempFLARRgbRaster_BitmapData,170)){ detector.getTransformMatrix(lastDetectionResult); //FLARPVGeomUtils is from FLARmanager overlay3dObj.transform = FLARPVGeomUtils.convertFLARMatrixToPVMatrix(lastDetectionResult); overlay3dObj.visible = true; } else { overlay3dObj.visible = false; } render(); Now you get all the concepts. So let's skip all the boring stuff and see it in action. Or get the finished codes. Happy AR! Update: Here I have a PB kernel that works with any color. Try it!