Reasons why should we use colored marker instead of standard black and white only marker
Using a black and white only marker is boring
There are more and more AR applications in the world, but seems everybody use only the black square marker. May be the public is not yet bored by the black square yet, but if can use other color for the square, we can integrate the marker into real-life objects more beautifully and more flexible.
Increase marker detection performance
The internal algorithm of AR marker detection has a characteristic that, the less pixels’ color is the same of the marker, faster the detection can be done. When we use black square marker, the detector will waste a lot of time to process non-marker pixels as you can imagine that it is common to have black objects in real-life (especially we the Chinese people have black hair). So if we target less common color, the detector will be able to ignore most of the non-marker area.
How to achieve this in FLARtoolkit
We will create a custom detector class since we need to work at lower level of the FLARtoolkit. If you have not look into the internal classes of FLARtoolkit (because the FLARmanager is so much easier to work with 
), there are two detectors: FLARSingleMarkerDetector and FLARMultiMarkerDetector. As their names suggest, one is for single marker detection and the other one can detect multiple markers. For simplicity, I will go through only the single detection for a red marker (if you want me to cover the other ones too, leave comment).
First we just copy all the codes inside FLARSingleMarkerDetector to a new class, I named it FLARSingleRedMarkerDetector.
To detect markers, we use the detectMarkerLite method, which is:
public function detectMarkerLite(i_raster:IFLARRgbRaster, i_threshold:int):Boolean
IFLARRgbRaster wraps a BitmapData, we can get it by:
var srcImg:BitmapData = (i_raster as FLARRgbRaster_BitmapData).bitmapData;
Originally, the method uses a threshold filter to extract black areas and give the result to other classes to process. As you might think of, we can simply swap the filter code. Originally the black areas will be turned to white pixels and others will be turned black and be ignored. We now want to detect red markers, so just try to turn everything black except the red regions to white. You may use ColorTransform if you want, but I am now obsessed with PixelBender. My version of kernel is:
kernel extractRedARMarker < namespace : "net.onthewings.filters"; vendor : "Andy Li"; version : 1; description : "used for FLARtoolkit, pre-proccess for red marker."; > { input image4 src; output pixel4 dst; parameter float threshold< minValue: 0.0; maxValue: 1.0; defaultValue: 0.4; description:"decrease to increase likelihood of marker detection."; >; void evaluatePixel() { float4 p = sampleNearest(src,outCoord()); float sum = p.r+p.g+p.b; float val = p.r - (p.g + p.b)*0.5; if (val+(1.0-(sum/3.0))*0.1 <= threshold) { val = 0.0; } else { val = 1.0; } dst = float4 (val,val,val,1); } }
Note that I let the dark red pass the threshold easier since I find it is common the web cam image is too dark.
To use the finished detector class:
tempFLARRgbRaster_BitmapData.bitmapData.draw(videoDisplay); if (detector.detectMarkerLite(tempFLARRgbRaster_BitmapData,170)){ detector.getTransformMatrix(lastDetectionResult); //FLARPVGeomUtils is from FLARmanager overlay3dObj.transform = FLARPVGeomUtils.convertFLARMatrixToPVMatrix(lastDetectionResult); overlay3dObj.visible = true; } else { overlay3dObj.visible = false; } render();
Now you get all the concepts. So let’s skip all the boring stuff and see it in action. Or get the finished codes.
Happy AR!
Update: Here I have a PB kernel that works with any color. Try it!
hi andy. one thing worth mentioning is that a red marker does not show up as one color in a camera — it’s highly dependent on the lighting in the area the camera is viewing. depending on the lighting (which cannot be controlled for web applications), red can appear as anywhere from red to shades of green or blue or black.
the most consistent thing for a computer vision application operating under variable lighting conditions to detect is brightness. that’s why FLARToolkit uses black.
you do have a good point though, that since FLARToolkit first looks for black areas in the image, images with a lot of dark areas will cause slowdown in the marker detection. (this is especially true for someone with as much black hair as you!
and the PixelBender implementation is nice! have you benchmarked it to see how it compares with a ColorMatrixFilter?
Comment by ericsoco — 06/07/2009 @ 10:04 am
I see your point. That’s why we need to adjust the threshold carefully.
Maybe we even need a better threshold mechanism or some auto adjustment like color balance before marker detection. I think that will be a interesting research topic
I don’t have any benchmark yet since I think I will make a plug-in library for color marker detection some time in the future (may be, can’t promise). Don’t expect a dramatic difference between PB and ColorMatrixFilter for this case though.
Comment by Andy — 06/07/2009 @ 4:23 pm
Hi Andy!
I’m not such a great developper… but seeing your code it seems you try to detect red from and RGB data.
Wouldn’t it be better to try to detect brighntess only in the R channel?
Red in the R channel is white… that would mean processing only 8bit pixels instead of 24 bit pixels.
Or maybe I’m just wrong and you’re already doing that…
Anyway… good work!
Comment by Pier — 23/07/2009 @ 8:10 am
White in red channel surely can be red(#FF0000), but it can be yellow(#FFFF00), magenta(#FF00FF) or even white(#FFFFFF)!
In that case, the filtered area will be much larger, meaning the save in processing 8bit instead of 24bit wouldn’t result in net performance gain.
Comment by Andy — 23/07/2009 @ 3:58 pm
You’re so right…
Comment by Pier — 24/07/2009 @ 8:11 am
just three points:
1 usually when people think of “color support” in flart, they mean marker made of different colors, or markers that differ from each other by color combinations. this is not totally covered by swapping the filter.
2 using svn version you can swap filters without changing flart code.
3 your flickr humanizer displays unavailable images
Comment by makc — 02/10/2009 @ 12:58 am
@makc
Yup, you’re right. I just pushed a little bit on the flexibility and it’s still pretty limited at this point. I think more complicated and colorful markers can be done but the FLARtoolkit is still in rapid development, like what you’ve told me, that now we can set custom filter which is not possible at the time I write the post.
Anyway, I will update the codes soon. Thanks for the info
I’ve just swapped the captcha plug-in and reported the bug to the author. Hope that will be fixed soon
Comment by Andy — 02/10/2009 @ 4:44 pm
Hi Andy,
I readed you text about the red market.
I dont know nothing about PixelBender, e a few things about Flartoolkit.
Could you help me in a addaption of your code?
In your text, is not clear to me where I import your class “FLARSingleRedMarkerDetector.as”, in the Main.as?
And how can I create a Yellow Marker?
So much thanks for your time!
@Vamoss
—————————————————————
I runned the FLARSingleRedMarkerDetector.as with my Black Marker normally.
How can I run my Yellow Marker version, I have to generate it before? Where?
And how can I addapt your Red class to detect Yellow colors?
Thanks Andy!!
@Vamoss
Comment by Vamoss — 13/11/2009 @ 2:47 am
@Vamoss
For the file location, just unzip the whole thing and put it under the source folder.
It’s a bit hard to detect yellow marker since it is not one of the color channel. The get it done, you only need to change the pbk, recompile it into pbj. Sure, you can rename everything red to yellow.
The change in pbk, it can be from “float val = p.r – (p.g + p.b)*0.5;” to “float val = (p.r + p.g)*0.5 – p.b;”. What this line of code do is letting val be 1 if the color is matched, less if it is not. FYI, red is #FF0000 and yellow is #FFFF00.
I have not tested the yellow code, and it is not that accurate I can say. Feel free to make some change to improve it.
Actually I have tried to write a class for detecting marker with any color, but I have been busying these week and it has been put aside. I think I can finish it in a week but I don’t know can you wait that long
Comment by Andy — 14/11/2009 @ 1:57 am
Hi Andy,
thanx for this great work,
I’ve to do a school work based on Flartoolkit and I’m facing some difficulties.
Can Flartoolkit can recognize a circle or a specific picture in red color?
Thanx in advance
kaka
Comment by kaka — 21/11/2009 @ 2:50 am
@kaka No, FLARtoolkit only works with square markers. You may put the picture inside the marker, but the picture cannot be too complex. Try the Marker Generator at http://flash.tarotaro.org/blog/2008/12/14/artoolkit-marker-generator-online-released/
Comment by Andy — 21/11/2009 @ 10:24 pm
thanx Andy
Comment by kaka — 24/11/2009 @ 4:22 am
[...] While traveling with my gf in Europe, I’m planning to develop a better technique to use color marker in FLARtoolkit (which I’ve tried before). [...]
Pingback by Playing with chroma key and thresholding in Flash (with Pixel Bender) | Andy Li's Blog — 10/12/2009 @ 6:18 pm