Friday, June 18, 2010

Symmetric getters and setters

Using getters and setters is a very common pattern for classes in Java projects. In general, I think they should be avoided, especially since setters eliminates the possibility of having final fields. Also, blindly adding getters to your classes is a bad idea since you expose implementation details to the outside world. Of course, some times you do no need them, and the rest of this post assumes that you do.

Sometimes you want to do some preprocessing on the inputs to the setters, to support cases where the input is out of range or other things of that nature. For instance, your domain objects may have a requirement that the values of the field should be in a specific range. You can then either:
  • Throw an illegal argument exception - if the ranges are clearly defined and the caller should know them, this is a valid approach.
  • Silently clamp the input value to be inside the range. This can sometimes be the desired approach, but it ultimately depends on the system.
  • Accept the value as it is, and just hope that the callers behave nicely. This is usually not a good idea, and should only be done if you control all the callers and its in a cpu critical section of the code.
There is a symmetry law which I think should hold for pairs of getters and setters.
Note that is a completely personal opinion. I don't expect everyone to agree with it. It's a good thing this blog is also completely personal!

Anyway, here is the law:
Values provided from a getter should be legal values when calling the setter, and if you run the following code, then x and y should be equal:

x = obj.getFoo();
obj.setFoo(x);
y = obj.getFoo();
assert x.equals(y);


For simple getters and setters that just uses a single field, this obviously always holds, even if the setters throws exceptions on bad values, or clamps the values out of range. This probably one of the reasons that many people assume that this symmetry usually holds.

However, sometimes the getters and setters don't really correspond to a specific field, but instead triggers other code to be modified. In this case, it's not clear that the symmetry holds.

for instance, you may have something like this:

public class Foo {
private final Bar bar;
public void setValue(int value) {
bar.setValue(value);
}

public int getValue() {
return offset + bar.getValue();
}
}

This is clearly asymmetric, so how do you fix it?
You could try something like this, but it's very prone to error, depending on how bar.setValue is implemented:

public void setBar(int value) {
bar.setValue(value - offset);
}


A better option is to remove the setter entirely and expose the bar object directly. I don't think anyone would take for granted that foo.getBar().setValue(...) needs to be symmetric with foo.getValue()

Yet another variant is simply renaming setValue to setBarValue (or setBaseValue or setDelegateValue, or something else, depending on the context) - this simple transformation also makes it clear that it's not symmetric.

Kahlua on Android

Yes, that's right, Kahlua supports Android (or Android supports Kahlua, depending on how you see it). And it worked surprisingly easy. All in all, I spent two hours installing the Android SDK, reading the Android documentation and creating an example program that used Kahlua. I even managed to create a very simple interpreter. And it didn't just load the Kahlua core, it also supports the J2SE extensions. It was almost out of the box, and I'll expand on that later.

This was a great success, and I attribute it to the following reasons:
  • The Android SDK is really polished. It was easy to install, easy to get started, and the documentation was pretty good!
  • Kahlua (core and J2SE) is compatible with the same subset of Java 5 that Android supports.
As far as I can tell, the classes in Android are fully compatible with Suns Java classes; at least I haven't found any bugs in Kahlua on Android yet.

The only problem I had was that I got a NoSuchMethodError when invoking Arrays.copyOfRange. It turns out that this method was introduced in Java 6, which explains why Android didn't support it. I simply converted the code into using the old and trusted System.arrayCopy and the code worked!

The Android interpreter even supported the reflection- and annotation-based exposing of Java methods, which I demonstrated by exposing a method that called Thread.sleep. (This was also useful for testing that the UI components in Android worked correctly while executing a script.)

The Android Kahlua interpreter is checked in to the git repository for Kahlua2, if anyone is interested in seeing it. It is basically a single Java file in 148 lines of code.

Sunday, June 6, 2010

Automated tests for content

Games are generally considered to consist of code and content.
  • The code is the game engine that defines all the rules of the game, input handling, network, graphics rendering, and so on.
  • Content in a game includes a lot of things: textures, sounds, level designs, ai, animations, meshes, dialogues and more.
Another (less formal) way of looking at it is that code is the part of the game produced by the programmers, and content is the part of the game produced by artists, game designers or level designers.

Programmers are used to writing tests for their code (if they're using TDD, they're even writing some, if not all, tests before they write the code), but content creators generally are not. They instead rely on manual testing, both by themselves and by gameplay testers.

Why is code tested and not content?
I think the code is what makes the game robust, coherent and logical. It's easy to detect broken code - broken code either crashes or behaves incorrectly. This makes it feasable to write tests for the code.

Content on the other hand is harder to verify automatically. How can a machine decide that a texture is broken? We could certainly verify that the texture has the correct dimensions and correct color depth, and other properties needed for the game engine to support it. We should have tests to verify the integrity of the content to as large extent as possible, and I think we do.

The things we can't verify automatically is the fun and the user experience.

The corner case
This was just the setup. Now for one of the corner cases, the parts that can be considered both code and content: AI!

AI is definitely a part of the content. Meeting unique non-player characters (NPC) with custom AI adds a new user experience that is something unknown, and therefore possibly exciting. The AI however is also expressed in some sort of code, either as a script in some embedded language, as a behaviour tree or behaviour stack.

The problem with AI as code, is that it tends to mutate at a much faster pace than game engine code. AI is driven by the content around it, not technical demands, and so it must change often.

Writing tests for each individual AI is expensive, since fully testing all the possible code paths can require a lot of manual setup, and verifying correctness is also tricky for a suffiently complex AI. What's worse, the test is bound to be very fragile, since content makers like to tweak and redo AIs if it's too easy, too difficult or simply not fun enough.

Should content like this be covered by automated tests? To some extent, definitely! I am not convinced that content should be tested individually, but testing the content by type is quite useful to detect errors.

For AIs this could mean:
  • Running automated smoke tests that run the AI characters for a long time (game time, not real time) to increase the chance of running many code paths (possibly with the AI opponent doing random moves) and simply checking that it doesn't crash. This type of test could apply to most if not all AI. It would be cheap to implement, and probably be fairly stable.
  • Smoke testing the AI with measurements of key values, such as damage output, movement speed, actions per minute, et.c. and check that the values are in a sane range. If some of the values are zero, this would indicate that the AI is blocked somewhere and one of the code paths is never run, and extremely high values would indicate that the AI is stuck in one of the code path loops.
Conclusion
You should always consider the cost versus the gain of writing tests. For game engine code, the gain is almost always greater than the cost, but for content it's not as clear.
You should also consider how to test something. A small unit test is suitable for many parts of code development, but it's probably overkill for content (which tends to change a lot).
If you can write one or a few tests that can verify something about a specific content type, and apply the test to all the content, the gain will quickly become large, since you can scale up the size of the content without scaling up the number of tests.

Friday, June 4, 2010

More details of the exposer

As I said before, there is more to the exposer than I wrote before.

Lua is, as you already know, a language that supports multiple return values. Java however does not, and that makes integration slightly less straightforward.

It is not obvious how to do it, but it is possible to expose Java methods and have them return multiple values.

Exposing multiple return values
The workaround is to create your java method like this:
@LuaMethod
public void getUnitLocation(ReturnValues rv, String unitName) {
Unit unit = service.getUnit(unitName);
rv.push(unit.getX());
rv.push(unit.getY());
}
You can then use the function like this:
local x, y = getUnitLocation("theUnit")
This looks a lot like how it's done using a raw JavaFunction, you might exclaim! This is not by coincidence. In fact, ReturnValues is just a thin layer on top of the original LuaCallFrame.

In order for this to work, the exposer mechanism performs special handling if the first parameter of the method is of the type ReturnValues.

Varargs
If the java method is a vararg method, i.e. defined like this:
public String concat(String... strings) {
return Arrays.toString(strings);
}
then Lua scripts can call it with concat("a", "b", "c") and get "[a, b, c]" back. This may seem obvious and it should be, but I'll mention it anyway.

Error handling
What if you do something wrong, such as calling getUnitLocation(), thus not passing in the name? The exposer will detect an incorrect number of arguments and throw an error message at you.

This however, is ok: getUnitLocation("theName", "something else"). The exposer will simply disregard the extra parameter. You may expect it to similarly throw an error, but this is in fact in line with how Lua generally handles too many arguments.

If you supply the incorrect type, you'll also get an error (unless the type can be converted to the Java type of course). If you supply nil, it'll go through into the method - so you have to add the null checks yourself.

Method overloading
I mentioned method overloading earlier, and that the exposer can handle it to some extent. I'll clarify that here.

Assume you have the following class:
public class Example {
public void myMethod() {
}
public void myMethod(Object a, Object b, Object c) {
}
public void myMethod(String a, String b) {
}
public void myMethod(int a, int b) {
}
}
If all these methods are exposed, the exposer will detect that it's already been exposed, and will convert the generated JavaFunction to a dispatching JavaFunction.
This dispatcher will inspect the arguments passed in the call to try to determine which method to call. The actual algorithm for selecting is very naive, but most likely good enough for almost all cases. In any case, you shouldn't use overloading too much - that's bound to confuse your users.

The dispatcher first sorts the list of matching methods by decreasing parameter count. Varargs count as zero. In our case, this means the following ordering:
public void myMethod(Object a, Object b, Object c);
public void myMethod(String a, String b);
public void myMethod(int a, int b);
public void myMethod();
The middle two have the same number of parameters, and thus can occur in any order.
When getting a call, it simply goes through the list in order and tries to call it. If it can't be called, it simply tries the next.
A call like myMethod("a", "b") will fail the first method, since it doesn't have enough arguments, but it will match the second.
myMethod("a", 123) won't match 1, 2 or 3, but it could be reduced to the 4th.
myMethod(nil, nil, nil) will match the first (since nil matches all types).

Methods and functions
What is a method? What is a function?

I'd like to define method as a function that's associated with an object. In Java, all instance methods falls under that definition of method. Static methods in Java can be considered functions, since they are not associated with specific objects.

Real methods don't really exist in Lua - all functions are plain functions. However, you can simulate methods easily. There is a simple syntactic sugar in Lua for this:

Sugar for a method definition:
function obj:method(args...)
end
is equivalent to:
function obj.method(self, args...)
end
That means that in the first case, it simply injects a new local variabel "self" assigned to the first argument.

Similarly, there is sugar for a method call:
obj:method(args...)
is equivalent to:
obj.method(obj, args...)
which of course is a regular function call.

This also how the exposer differentiates between methods and functions. If it's annoted with @LuaMethod(global = true) or is static, it's a function, and arguments are passed straight through from Lua to Java.

If global = false (which is the default), the first argument is consumed by the exposer and used to find the actual object to invoke the method on, and it's not passed along to the method.

Thursday, June 3, 2010

Differences between Kahlua2 and Lua 5.1

There are some noticable differences between Lua 5.1 and Kahlua2, and it may be good to know what they are.

Tables
Tables in Kahlua2 are implemented using a backing Hashtable or ConcurrentHashMap and as a result, implementing "next" properly would be inefficient and instead of implementing "next", it was removed. The function "pairs" still works, so use that instead. One common idiom using next in Lua 5.1 was checking if a table is empty:
function isempty(t) return next(t) == nil end


An alternative implementation of that in Kahlua2 would be:
function isempty(t) return pairs(t)() == nil end

but it would be inefficient, since it would create an iterator for each call to isempty.

Instead, you can use the method table.isempty(t) in Kahlua2.

Another side effect of using Java maps to implement tables, is that key equality is slightly different between Lua 5.1 and Kahlua2. In Lua 5.1, keys are equal in a table if they refer to the same object while in Kahlua2 they are equal if their hashcodes are equal and their equals()-methods return true. For strings, it works just as in Lua 5.1, but for numbers it's slightly different. Java treats -0 and +0 as different keys while Lua treats them as the same.

tostring
When converting things to strings in Kahlua2 (for example when calling tostring(obj)), it will not just return "userdata" if it doesn't have a better guess, it will instead use the toString() method for the object.

Weak tables
Kahlua2 doesn't support the __mode = "[k][v]" metatable magic to create weak tables. If you need weak tables, consider exposing a new method for creating a KahluaTable backed by a WeakHashMap or something similar.

Kahlua2 doesn't support the __gc feature for metatables, since there is no good way in Java to get notified when an object is about to get garbage collected. If you need this functionality, consider putting the object in a SoftReference to manually check when it's collected.

Libraries
Kahlua2 does not include any io or debug library. Some parts of the debug library may be added in the future, but the io library is better left added as a custom extension if necessary. The same thing goes for the os library, which is only partially implemented.

xpcall
Kahlua2 doesn't implement xpcall yet, but it might come in the future.

random and randomseed
random and randomseed in Lua uses a mutable global state, which is undesirable in Kahlua2.
Instead, use the following:
local myrandom = newrandom()
myrandom:random() -- real number in the range 0 to 1
myrandom:random(max) -- integer number in the range 1 to max
myrandom:random(min, max) -- integer number in the range min to max
myrandom:seed(obj) -- seeds the random instance with the hashcode of the object.

Wednesday, June 2, 2010

Kahlua - the J2SE goodies

In this post, Lua will generally refer to the language specification, whereas Kahlua will refer to this actual implementation so don't be confused when I talk about Lua scripts running under Kahlua.

The goodies
Kahlua has a couple of useful extensions that makes your life as a developer easier when integrating Kahlua into your applications. Some of them depend on each other, so I'll go through them in dependency order to make the reading more digestible.

Converting objects
Since Lua only has a few built in types (String, Double, Boolean, KahluaTable) and Java has many many more types, it's sometimes useful to automatically convert types when going back and forth between Lua and Java.

For instance, if Java were to send an Integer object to Kahlua, that object would just be an opaque reference, without any operations. If we instead convert it to a Double, Kahlua can work with it without any problems. This is the rationale for introducing the first J2SE goodie - the KahluaConverterManager.

This is a class that can help you convert types back and forth between Lua and Kahlua. By itself, it can't convert anything at all, but you can register actual converters with it. For convenience, a couple of generic converters are bundled with Kahlua and you can also write custom converters if you need to.

The converter manager has two conversion methods: fromLuaToJava and fromJavaToLua.
Converting from Lua to Java takes an object and a Java class.
The converter manager then uses the type of the object and the Java class to determine which converter best matches the conversion. If nothing matches it returns null instead.

Converting from Java to Lua is different - it doesn't have a target type, since Lua is dynamically typed. Instead the converter just looks at the Java type to choose converter. If no suitable converter is found, it returns the object as it is.

Here are the already existing converters:

The number converter
Registering the number converter allows you to automatically convert Lua numbers (which are implemented as Double) to the numeric types in Java (int, long, char, byte, short, float, double).

It also works the other way around, converting the numeric types in Java to Double.

The table converter
This one also converts both ways. KahluaTable instances can be converted both to List and Map, and Lists and Maps can be converted back to KahluaTable. It also converts maps recursively, so be careful to avoid converting cyclic structures.

The enum converter
You can also convert enums to their string representations (enum.name()) and back, which means that you can
call Java methods that takes an enum just by passing in a string that matches one of the enums.

Calling Lua from Java
LuaCaller.protectedCall is a utility method that simplifies the usage of thread.pcall. Instead of returning a raw array where the first return value always is true or false, it returns an abstraction that hides the details: the LuaReturn class.

LuaReturn has the methods isSuccess(), getErrorMessage(), getStacktrace(), getJavaException() which makes it easier to use than extracting result[0] (and 1, 2, 3, respectively).

LuaCaller.protectedCall additionally applies the installed converters to the inputs before the call. It doesn't apply it to the return values, because it doesn't know which types it wants.

Calling Java from Lua
Be prepared - here comes the most advanced, the most useful and simply best feature of the J2SE goodies.

The LuaJavaClassExposer is a class that eliminates the need to manually define JavaFunction implementations for things that you want to expose to Lua.

Imaging writing a simple game, which is scriptable with Kahlua, and you want to be able to query a game units health from the Lua script.
Without the exposer, you'd have to write something like this:

public class GetUnitHealth implements JavaFunction {
public int call(LuaCallFrame callFrame, int nArguments) {
if (nArguments < 1) {
throw new IllegalArgumentException("Expected at least one argument");
}
Object arg = callFrame.get(0);
if (arg == null || !(arg instanceof String)) {
throw new IllegalArgumentException("Expected a string argument but got " + arg);
}
String s = (String) arg;
Unit unit = unitService.getUnit(s);
callFrame.push(unit.getHealth());
return 1;
}
}

Using the exposer instead, you can write something like this:

public class UnitFunctions {
@LuaMethod
public int getUnitHealth(String unitName) {
Unit unit = unitService.getUnit(unitName);
return unit.getHealth();
}
}

It removes a lot of the boilerplate right? The exposer takes care of all the conversions and argument matchings for us!

Instead, you can write an ordinary method in any class, and let the exposer expose that class. One JavaFunction per method will be created, and the method will be available in Lua. The generated JavaFunction will automatically use the installed converters which makes it really easy to integrate the Java and Lua code.

There are basically two different styles available when exposing:

Exposing like java
The first style is exposing things to work almost the same as in Java.
You simply call exposer.exposeLikeJava(yourClass, staticBase).
After that, a Lua script may call obj:yourMethod(arg1, arg2, ...) if obj is of type yourClass and yourMethod is a method in yourClass.

You may be wondering what staticBase means (and if not, you're not paying enough attention). A method are only available for the object if it's a regular java method.
Static methods and constructors can't be used this way. Instead, Kahlua uses the staticBase argument, which is a KahluaTable, to fill up with methods that don't belong to specific objects. If you expose the class java.util.Date, then the constructors will be available as:
staticBase.java.util.Date.new
and static methods such as parse are available as:
staticBase.java.util.Date.parse

If the class name is unique enough, it is also available directly from the staticBase root, such as staticBase.Date

These static methods and functions are what I call functions instead of methods.
Instead of invoking them with obj:method, do it like this:

local unixTimeStamp = 123
local dateObject = staticBase.java.util.Date.new(unixTimeStamp)
local dateObject = staticBase.Date.new(unixTimeStamp)
local NewDate = staticBase.Date.new
local dateObject = NewDate(unixTimeStamp)

Any of those would work.

You can also use exposer.exposeLikeJavaRecursively(yourClass) which works the sam but additionally it also exposes all other classes that yourClass interacts with in any way. This can be useful if you want to avoid manually setting up a lot of exposing, but don't use it when dealing with untrusted code.

Annotation based exposing
For untrusted code, or simply when you need more custom configuration, consider using the annotation based exposing instead.

Use one of these methods to expose a class or objects by using annotations:
exposer.exposeGlobalFunctions(object) and exposer.exposeClass(class),

Exposing global function takes an actual Java objects and scans it for method annotations. All methods which are marked as global will be exported to Lua as functions, implicitly bound to the object. This means that you should not use the obj:method() notation of invoking them, just use method().

Exposing a class instead creates Lua methods for objects of the class, and also exposes static methods as function, if they're marked as global.

The available annotations are: @LuaMethod and @LuaConstructor.

@LuaMethod has two optional parameters. "name" gives you the option to expose
the method with a different name. If this is left out, it uses the method name. The second parameter is "global" which can be true or false. True means that the method will be exposed as a function, and false means that it will be an object method. False is the default.

@LuaConstructor similarly takes a "name" parameter.
Note that it doesn't take a "global" , since constructors are never bound to an object; they are always implicitly global.

Overloading
Java supports method overloading based on parameter types, but since functions are first class objects in Lua, it can't support overloading. However, the Kahlua exposer can detect overloaded Java methods and create a limited form av overloading by creating a function that can call any of the overloaded Java methods.
Kahlua then uses the parameter types to try to invoke the method that best matches the supplied arguments.

Here are some concrete usage examples:

public class Vector {
private final double x, y, z;
@LuaConstructor(name = "newvector")
public Vector(double x, double y, double z) {
this.x = x;
this.y = y;
this.z = z;
}

@LuaMethod(name = "length")
public double length() {
return Math.sqrt(x*x + y*y + z*z);
}
}


If this class were exposed by exposer.exposeClass(Vector.class), then Lua scripts could use it like this:

local v = newvector(3, 4, 0)
local len = v:length()
assert(len == 5)

The other method is mostly useful for service-type features:

public class NetworkService {
private final Pinger pinger;

@LuaMethod(name = "ping", global = true)
public int ping(String host) {
return pinger.ping(host);
}
}
exposer.exposeGlobalFunctions(new NetworkService());

Used as:

local pingTime = ping("localhost")
That's it for this time! I could probably go on more in depth with the advanced features in the exposer, but this should already be enough for you to digest.

Time for you get started with embedding Kahlua in your Java application, and let me know how it works out! Feedback is always appreciated.