Andy Li’s Blog - trick

hxml tricks every haxe user should know

2013-03-04T17:00:00+08:00

Haxe projects usually are compiled with a hxml file, which basically contains compiler arguments. So most of the tricks I am going to show are actually compiler arguments tricks, which may also applicable to nmml file or IDE that does not use hxml (though they should).

1. Run project right after compilation

Being able to do so is critical to rapid development. The simple trick to use -cmd, which run the a command after successful compilation. For different target, you have to use different command:

Neko

-neko Test.n  
-main Test  
-cmd neko Test.n

Or there is actually a shortcut that performs exactly the same as above:

-x Test

C++

-cpp bin
-main Test
-cmd ./bin/Test

Notice the resulting file will be Test-debug if you compile with -debug.

JavaScript

-js Test.js  
-main Test  
-cmd phantomjs Test.js

Here we use phantomjs to run the JS file, trace() will output to the terminal. In case it is something graphical, you have to launch a browser with a HTML file that loads the script.

PHP

-php bin  
-main Test  
-cmd php bin/index.php

I guess you knew you can execute a php script this way?

Flash

-swf Test.swf  
-main Test  
-cmd path/to/FlashDebugger Test.swf

Or if you’re on Mac, you can actually use -cmd open Test.swf, it will do the same as you double clicked on a swf file (or any other file!).

Java

-java bin  
-main Test  
-cmd java -jar bin/Test.jar

Notice the resulting file will be Test-Debug.jar if you compile with -debug.

-cs bin  
-main Test  
-cmd mono bin/bin/Test.exe

Mac/Linux has to use Mono to execute it. Windows should able to run it with -cmd bin\bin\Test.exe.
Notice the resulting file will be Test-Debug.exe if you compile with -debug.

2. Commenting a hxml file

Use a hash (i.e. #) to comment out the rest of the line. Despite of being useful to put documentation, it can let us switch between different compilation configurations:

-js Main.js  
-main Main  
### comment out one of the following logging levels  
#-D my-log-error  
#-D my-log-warning  
-D my-log-info

And in the source file:

function attack(target:Monster, power:Int):Void {
    if (target == null) {
        #if (my_log_error || my_log_warning || my_log_info)
        trace("target is null");
        #end
        return;
    }
    if (power <= 0) {
        #if (my_log_warning || my_log_info)
        trace("attack power should be positive");
        #end
        return;
    }
    mp -= power;
    target.hp -= power;
    #if my_log_info
    trace("attacked " + target + " by " + power);
    #end
}

3. Append extra compiler argument when using the command line

Another useful trick to switch between compilation configurations, is to simple supply more arguments after the hxml file. For example temporarily switch to debug mode:

haxe project.hxml -debug

4. Multiple compilations at once

The majority knows the existence of --next, used for separating different builds:

all.hxml

-js script/MainPage.js  
-main MainPage  
-lib jQueryExtern  
--next  
-js script/ContactPage.js  
-main ContactPage  
-lib jQueryExtern  
--next  
-js script/AlbumPage.js  
-main AlbumPage  
-lib jQueryExtern

There is an —each option, which reduce the repeating params. It is used in the haxe compiler unit test hxml. Rewriting the above, we will get:

all.hxml

-lib jQueryExtern  
--each  
-js script/MainPage.js  
-main MainPage  
--next  
-js script/ContactPage.js  
-main ContactPage  
--next  
-js script/AlbumPage.js  
-main AlbumPage

But really, separating each of them in different hxml is easier to maintain and has better compatibility with code completion (because many editors don’t read hxml with --next very well). So we may use the following instead:

MainPage.hxml

-js
script/MainPage.js  
-main MainPage  
-lib jQueryExtern

ContactPage.hxml

-js
script/ContactPage.js  
-main ContactPage  
-lib jQueryExtern

AlbumPage.hxml

-js
script/AlbumPage.js  
-main AlbumPage  
-lib jQueryExtern

all.hxml

MainPage.hxml  
--next  
ContactPage.hxml  
--next  
AlbumPage.hxml

Switch the hxml for code completion when working in different pages, and use the all.hxml to compile at once.

Haxe tips: everything is an expression

2012-10-14T05:02:00+08:00

In Haxe, nearly everything is an expression. (Things that aren’t: import statement, class declaration etc, which are at module level). And every expression can be evaluated to a value.

A block is an expression that is evaluated to the last expression inside the block:

var v = {
    //some code
    123;
}
trace(v);//123

It can be used for list comprehension:

var oneToTen = {
    var a = [];
    for (i in 0...10) a.push(i+1);
    a;
}   
trace(oneToTen); //[1,2,3,4,5,6,7,8,9,10]

In reverse, we can notice many things actually use any expression instead of only a block.

Function declaration:

function hello() return "world";

For loop:

for (i in 0...5) trace(i); //0 1 2 3 4

And of course if-else:

if (a > 100)
    trace("a is more than 100");
else if (a > 50)
    trace("a is between 50 and 100");
else
    trace("a less than 50");

If-else itself is an expression too. So we can simplify the above to:

trace(if (a > 100) "a is more than 100" else if (a > 50) "a is between 50 and 100" else "a less than 50");

Of course we can use the equivalent ternary operator, but in my opinion it is a bit less readable:

trace(a > 100 ? "a is more than 100" : a > 50 ? "a is between 50 and 100" : "a less than 50");

Switch is useful to be used as an expression when working with enum:

enum Color {
    Gray(v:Int);
    Rgb(r:Int, g:Int, b:Int);
}

class Main {
    static function main():Void {
        var redColor = Rgb(255,0,0);
        var red = switch(redColor) {
            case Rgb(r,g,b): r;
            default: throw "not rgb color";
        };
        trace(red);//255
    }
}

And actually try-catch also returns a value:

var noException = try {
    //some code
    true;
} catch (exception:Dynamic) {
    false;
}

Do you find any other good use of anything as an expression?

Haxe tips: Macro-Proxied Class = with macros for compile-time and implementation for run-time

2012-07-17T19:42:00+08:00

As mentioned in the previous post on overloading, I found a way to provide a reflection friendly, run-time implementation of a @:macro method or an extern class. The trick is simple: build the implementation with a @:native metadata.

Here is an implementation of using such I called macro-proxied class in the method overloading example:

//R.hx
#if macro 
import haxe.macro.Expr;
#end

#if !macro @:include("R") extern #end class R {
    /**
     * Helper to get the real run-time implementation of this class.
     * Only use at compile-time.
     */
    inline static public var _impl = R_impl;

    /**
     * Helper to mark an instance as the run-time implementation of this class.
     * Only use at compile-time.
     */
    @:macro function _to_impl(r:ExprOf):ExprOf {
        return {
            expr: ECheckType(macro untyped $r, TPath({
                pack:[], 
                name:"R", 
                params: [], 
                sub: "R_impl"
            })),
            pos: r.pos
        }
    }

    /**
     * Get a resource by its url.
     * Returns a Xml if url ends with ".xml", a String otherwise.
     */
    @:macro static public function get(url:ExprOf):Expr {
        switch(url.expr) {
            case EConst(c): switch (c) {
                //url is a constant String, we can use optimized versions
                case CString(str):
                    var dotPos = str.lastIndexOf(".");
                    if (dotPos != -1) switch (str.substr(dotPos).toLowerCase()) {
                        case ".xml": return macro R._impl.getXml($url);
                        default:
                    }

                    return return macro R._impl.getText($url);
                default:
            }
            default:
        }

        //url is not known at compile-time, let the method check it at run-time.
        return macro R._impl.get($url);
    }

    /**
     * Construct a resource loader.
     */
    public function new():Void {}

    /**
     * Function same as R.get(url). Just to demo the macro proxy on instance method.
     */
    @:macro public function load(ethis:ExprOf, url:ExprOf):Expr {
        ethis = macro $ethis._to_impl();
        switch(url.expr) {
            case EConst(c): switch (c) {
                //url is a constant String, we can use optimized versions
                case CString(str):
                    var dotPos = str.lastIndexOf(".");
                    if (dotPos != -1) switch (str.substr(dotPos).toLowerCase()) {
                        case ".xml": return macro $ethis.loadXml($url);
                        default:
                    }

                    return return macro $ethis.loadText($url);
                default:
            }
            default:
        }

        //url is not known at compile-time, let the method check it at run-time.
        return macro $ethis.load($url);
    }
}

/**
 * Real implementation of R.
 */
@:native("R") // <-- here is the trick ;)
class R_impl {

    static public function get(url:String):Dynamic {        
        var dotPos = url.lastIndexOf(".");
        if (dotPos != -1) switch(url.substr(dotPos).toLowerCase()) {
            case ".xml": return getXml(url);
            default:
        }

        return getText(url);
    }

    public function new():Void { }

    public function load(url:String):Dynamic {
        return get(url);
    }

    //Below are the optimized overload cases. Skipped file extension check.

    static public function getText(url:String):String {
        return haxe.Http.requestUrl(url);
    }

    static public function getXml(url:String):Xml {
        return Xml.parse(getText(url));
    }

    public function loadText(url:String):String {
        return getText(url);
    }

    public function loadXml(url:String):Xml {
        return getXml(url);
    }
}

A simple test:

//Test.hx
class Test {
    static function main():Void {

        trace(Type.getClass(new R().load("http://blog.onthewings.net/sitemap.xml"))); //Test.hx:4: Xml
        trace(Type.getClass(R.get("http://blog.onthewings.net/sitemap.xml"))); //Test.hx:5: Xml

        trace(Type.getClass(new R().load("http://www.google.com/"))); //Test.hx:7: String
        trace(Type.getClass(R.get("http://www.google.com/"))); //Test.hx:8: String

        //call using variable, so that the compiler does not know it is a xml
        //just to show the overloading logic is working even at run-time
        var a = "http://blog.onthewings.net/sitemap.xml";
        trace(Type.getClass(new R().load(a))); //Test.hx:13: Xml
        trace(Type.getClass(R.get(a))); //Test.hx:14: Xml

    }
}

Abilities of Macro-Proxied Class

Macro-proxied class provides a compile-time pre-processing stage for the methods. We can remap method call to another optimized method base on compile-time checking on the method arguments. e.g.. method overloading as shown above.

There is an implementation at run-time, with class and method names remained the same. Thus reflection friendly.

Some ideas that may use macro-proxied class includes:

A FastMath class that optimize math operation when arguments are known at compile-time.
cpp wrapper(ndll) that can use method overloading like the native API.
jQuery plug-in that is used by “using”, for better variable argument length support.
SPOD, to merge the macro version to the original version.
hxLINQ, like SPOD, to perform compile-time optimization on queries when possible.

Limitations of Macro-Proxied Class

Firstly, you cannot proxy the constructor, since the constructor(new) cannot be a @:macro method.

Secondly, care should be taken on handling inheritance. For the R example, if we want to extend R, the subclass should also be a macro-proxied class. Something like:

//SubR.hx
import R;

#if macro 
import haxe.macro.Expr;
#end

#if !macro @:include("SubR") extern #end class SubR extends R {
    inline static public var _impl = SubR_impl;

    @:macro override function _to_impl(r:ExprOf):ExprOf {
        return {
            expr: ECheckType(macro untyped $r, TPath({
                pack:[], 
                name:"SubR", 
                params: [], 
                sub: "SubR_impl"
            })),
            pos: r.pos
        }
    }

    @:macro override public function load(ethis:ExprOf, url:ExprOf):Expr {
        var ethis = macro $ethis._to_impl();
        return macro $ethis.load($url);
    }
}

@:native("SubR")
class SubR_impl extends R_impl {
    override public function load(url:String):Dynamic {
        return "overrided!";
    }

    //Remember to override the optimized overload cases too!!

    override public function loadText(url:String):String {
        return load(url);
    }

    override public function loadXml(url:String):Xml {
        return load(url);
    }
}

Both the sub classes of a macro-proxied class have to be a macro-proxied class too since the signature of some methods at compile-time are changed to Array<Expr>->Expr. Inherited methods that are not macro-proxied cannot be macro-proxie. Also we have to think about the case when a SubR instance is stored in a R variable, would the macros or the run-time version of the methods work correctly?

Moreover, the structure of a macro-proxied class is not preserved. i.e.. R is not a structure type of { function load(url:String):Dynamic; }, which R_impl is, since load is not an instance method, but a macro method at compile-time. You can workaround this by casting (actually is an untyped expression) to R_impl by using _to_impl().

Lastly, again this trick should be refactored into some lib as it is a bit too complicated to write every time. I guess at least we should use a @:build macro to copy all the properties and methods(for the ones we do not want to override).

Haxe tips: advanced method overloading with macros

2012-07-13T20:05:00+08:00

The @:overload metadata

Introduced in Haxe 2.08(released in 2011), @:overload metadata can be used to annotate a method to have several type signatures. For example, in JQuery of jQueryExtern, the method “html” has the following declaration:

@:overload(function(valueOrFunction:Dynamic):JQuery{})
public function html():String;

It declares the html setter (@:overload) on top of the getter, so they share the same method name with different argument/return types.

Limitation of @:overload

The @:overload metadata is mainly designed for writing externs, i.e.. to describe the native overloaded methods but not creating one. The Haxe compiler wouldn’t check with the function body if it is really performing in the way which @:overload metadata suggested. And definitely no runtime support if you do not implement it yourself.

It is also unable to describe some slightly complex, or unusual overloading. Such us jQuery.get, which has the following signature:

jQuery.get( url [, data] [, success(data, textStatus, jqXHR)] [, dataType] )

If dataType is the string "xml", the data going to be passed to success will be an XML root element. If dataType is "json", a JavaScript object will be passed instead. There are also cases for "script" and "html". Normally in a Haxe extern we can only declare it as:

static public function get(url:String, ?data:Dynamic, ?callBack:Dynamic->String->JqXHR->Void, ?dataType:String):JqXHR;

But instead it should be something like:

//Note: invalid!!!
@:overload(function get(url:String, ?data:Dynamic, ?callBack:Dom->String->JqXHR->Void, ?dataType:"xml"):JqXHR {})
@:overload(function get(url:String, ?data:Dynamic, ?callBack:Dynamic->String->JqXHR->Void, ?dataType:"json"):JqXHR {})
@:overload(function get(url:String, ?data:Dynamic, ?callBack:String->String->JqXHR->Void, ?dataType:"script"):JqXHR {})
@:overload(function get(url:String, ?data:Dynamic, ?callBack:String->String->JqXHR->Void, ?dataType:"html"):JqXHR {})
static public function get(url:String, ?data:Dynamic, ?callBack:Dynamic->String->JqXHR->Void, ?dataType:String):JqXHR;

I know, I know, this kind of API is designed for JS which has a dynamic type system that does not fit perfectly to a static type system like Haxe’s. And the above case does not hurt much if we simply declare callback as Dynamic->String. But, when writing extern, still we prefer declaring as close to as the original JS signature and use as much compiler checking as we can.

Macros to the rescue!

Yes, macros can improve that!

Here is a demo extern class R that I make up:

//R.hx
import haxe.macro.Expr;

#if !macro extern #end class R {
    /**
     * Get a resource by its url.
     * Returns a Xml if url ends with ".xml", a String otherwise.
     */
    @:macro static public function get(url: ExprOf):Expr {
        //the original calling expression, R is untyped to avoid endless loop in this macro
        var expr = macro (untyped R).get($url);

        switch(url.expr) {
            case EConst(c): switch (c) {
                case CString(str): //url is a constant String
                    var dotPos = str.lastIndexOf(".");
                    if (dotPos != -1) switch (str.substr(dotPos).toLowerCase()) {
                        case ".xml": return { //mark return type as Xml
                            expr: ECheckType(expr, TPath({pack:[], name:"Xml", params: []})),
                            pos: expr.pos
                        }
                        default:
                    }

                    return { //mark return type as String
                        expr: ECheckType(expr, TPath({pack:[], name:"String", params: []})),
                        pos: expr.pos
                    }
                default:
            }
            default:
        }

        //return type is not known at compile-time
        return expr;
    }
}

Here is a simple test, that will output the type info when compile:

//Test.hx
class Test {
    static function main():Void {
        var a = "123";
        $type(R.get("123.xml"));  //src/R.hx:10: characters 19-40 : Warning : Xml
        $type(R.get("123"));      //src/R.hx:10: characters 19-40 : Warning : String
        $type(R.get(a));          //src/R.hx:10: characters 19-40 : Warning : Unknown<0>
    }
}

Note that the above implementation used some macro features from Haxe 2.10, which ~~will be released very soon~~ is released!
The macro part is a bit too long to write for every method but there is nothing to stop you refactoring this trick into an lib ;)

Still, the above R is merely an extern, the actual function has to be already implemented in the native lib. I will let you know one more trick to actually provide an implementation of the overloaded method, which can be called at run-time, with reflection support.

Haxe tips: better untyped expression

2012-07-08T04:20:00+08:00

tl;dr

Remember to put the brackets

    (untyped new JQuery("#images").imageLoader)({});
//  ^                                         ^
//  |________these____________________________|

Abstract

Static type system of Haxe helps you a lot by performing type-checking in the compilation phase. You will be notified errors before the application is run.

Since some Haxe targets (e.g.. JS and PHP) are dynamic in nature, many of the native libraries/frameworks are also very dynamic. You have to use the untyped keyword in Haxe very often to workaround the compiler type-checking.

As a best-practice, to avoid unnecessary suppression of type-checking, brackets should always be placed in order to constrain affecting slope when using the untyped keyword in an expression.

Example: using jQuery plug-in

For illustration, I will explain a frequently faced problem that is how to use jQuery plug-in in Haxe (following will use jQueryExtern). In case there is a plug-in that will add a method, .imageLoader(config), to jQuery. But since there is no such method declared in the JQuery extern class, if you simply compile new JQuery("#images").imageLoader({});, it will certainly fail with error “jQuery.JQuery has no field imageLoader“.

You have two options:

Create a plugin class and then “using” it.
Use a quick and dirty untyped keyword.

For 1, you can refer to some experimental plug-in externs from jQueryExtern. Read my blog post on how to use it with ”using”. This method takes some more amount of coding and does not work well with optional arguments.

For 2, adding a untyped keyword will shut the complier’s mouth up: untyped new JQuery("#images").imageLoader({});.

The above of course works, but you will lose many type-checking in the expression where are unnecessarily suppressed. e.g.. if you forgot to define config in

untyped new JQuery("#images").imageLoader(config);

…it will compile and fails at runtime with “Uncaught ReferenceError: config is not defined“.

Instead, constrain the untyped scope by placing a pair of brackets:

(untyped new JQuery("#images").imageLoader)(config);

…it now (correctly) fails to be compiled with error “Unknown identifier : config“.

It is useful because sometimes you may code like this:

(untyped new JQuery("#images").imageLoader)({
    images: images,
    async: 10,
    complete: function(e, ui) {
        progressbar.progressbar("value" , (++loaded).map(0, numOfFrames, 0, 100));
    },
    allcomplete: function(e, ui:Array) {
        new JQuery(function(){
            var playback = new JQuery("#playback").html("");
            for (item in ui) {
                new JQuery(item.img)
                .attr("id", "playback-" + Std.string(item.i))
                .appendTo(playback)
                .hide();
            }

            if (displayMode.enumEq(PlaybackMode))
                imageLoop(12);

            new JQuery("#toggle-play-btn").removeAttr("disabled");
            new JQuery("#toggle-slow-btn").removeAttr("disabled");

            new JQuery("#main").removeClass("loading");
        });
    }
});

If you do not place brackets around untyped (at line 1 as shown above), you will unnecessarily lose type-checking for over 90% of codes (begin with line 2 of above), which is pretty bad.

How to know objects are really GC’ed in Flash(AS3)

2010-10-18T01:48:00+08:00

Memory leak is a bug that pretty hard to deal with. Usually people use some profilers to observer the memory usage while stressing the program, if there is no increase of memory usage after repeating calls of a function, we can conclude there is no memory leak in that function. It is of course fine to detect memory leak in this way, but it is always good to have a way that asserts your objects are really GC‘ed at some point.

The trick is to use a Dictionary with weak-reference keys and check its element count. It is already quite popular in the Java world but look like there aren’t many Flash devs talking about it.

A simple demo on how to write such test, source is available in wonderfl:

FLARtoolkit trick: use a colored marker

2009-05-23T10:40:00+08:00

Reasons why should we use colored marker instead of standard black and white only marker

Using a black and white only marker is boring

There are more and more AR applications in the world, but seems everybody use only the black square marker. May be the public is not yet bored by the black square yet, but if can use other color for the square, we can integrate the marker into real-life objects more beautifully and more flexible.

Increase marker detection performance

The internal algorithm of AR marker detection has a characteristic that, the less pixels’ color is the same of the marker, faster the detection can be done. When we use black square marker, the detector will waste a lot of time to process non-marker pixels as you can imagine that it is common to have black objects in real-life (especially we the Chinese people have black hair). So if we target less common color, the detector will be able to ignore most of the non-marker area.

How to achieve this in FLARtoolkit

We will create a custom detector class since we need to work at lower level of the FLARtoolkit. If you have not look into the internal classes of FLARtoolkit (because the FLARmanager is so much easier to work with ;-) ), there are two detectors: FLARSingleMarkerDetector and FLARMultiMarkerDetector. As their names suggest, one is for single marker detection and the other one can detect multiple markers. For simplicity, I will go through only the single detection for a red marker (if you want me to cover the other ones too, leave comment).

First we just copy all the codes inside FLARSingleMarkerDetector to a new class, I named it FLARSingleRedMarkerDetector.

To detect markers, we use the detectMarkerLite method, which is:

public function detectMarkerLite(i_raster:IFLARRgbRaster, i_threshold:int):Boolean

IFLARRgbRaster wraps a BitmapData, we can get it by:

var srcImg:BitmapData = (i_raster as FLARRgbRaster_BitmapData).bitmapData;

Originally, the method uses a threshold filter to extract black areas and give the result to other classes to process. As you might think of, we can simply swap the filter code. Originally the black areas will be turned to white pixels and others will be turned black and be ignored. We now want to detect red markers, so just try to turn everything black except the red regions to white. You may use ColorTransform if you want, but I am now obsessed with PixelBender. My version of kernel is:

kernel extractRedARMarker
<   namespace : "net.onthewings.filters";
    vendor : "Andy Li";
    version : 1;
    description : "used for FLARtoolkit, pre-proccess for red marker.";
>
{
    input image4 src;
    output pixel4 dst;

    parameter float threshold<
        minValue: 0.0;
        maxValue: 1.0;
        defaultValue: 0.4;
        description:"decrease to increase likelihood of marker detection.";
    >;

    void
    evaluatePixel()
    {
        float4 p = sampleNearest(src,outCoord());
        float sum = p.r+p.g+p.b;
        float val = p.r - (p.g + p.b)*0.5;

        if (val+(1.0-(sum/3.0))*0.1 <= threshold) {
            val = 0.0;
        } else {
            val = 1.0;
        }
        dst = float4 (val,val,val,1);
    }
}

Note that I let the dark red pass the threshold easier since I find it is common the web cam image is too dark.

To use the finished detector class:

tempFLARRgbRaster_BitmapData.bitmapData.draw(videoDisplay);
if (detector.detectMarkerLite(tempFLARRgbRaster_BitmapData,170)){
    detector.getTransformMatrix(lastDetectionResult);
    //FLARPVGeomUtils is from FLARmanager
    overlay3dObj.transform = FLARPVGeomUtils.convertFLARMatrixToPVMatrix(lastDetectionResult);
    overlay3dObj.visible = true;
} else {
    overlay3dObj.visible = false;
}
render();

Now you get all the concepts. So let’s skip all the boring stuff and see it in action. Or get the finished codes.

Happy AR!

Update: Here I have a PB kernel that works with any color. Try it!

How to hold a key without your hands on a standard keyboard.

2008-07-26T12:59:00+08:00