April 24, 2012
Last weekend I decided to take part in Ludumdare 23 Jam. Here is a short postmortem.
The theme was “Tiny World”. After a little brainstorming and inspiration I decided to have a go at making an RTS.
After coming up with a basic list of things to do, I started to work on the engine. Since I wanted the minimum of fuss when it came to actually deploying the game, I decided to try out NME, a flash-like API written in haXe which describes itself as a “a free, open-source framework that enables development for iOS, Android, webOS, BlackBerry, Windows, Mac, Linux and Flash Player from a single codebase” - perfect!
While the “game” ended up being a complete unsubmittable disaster, I still found the experience to be quite insightful.
What went right
After a few hours I managed to get a basic prototype working with a unit and walls. At this stage, the unit merely moved towards the mouse cursor and collided against the walls. Next came unit-unit collision, which turned out to be a bit more complicated since I decided to use arbitrarily sized bounding boxes.
Path finding ended up being the simplest thing to implement. For this I used AStar. While initially I stumbled across a rather insightful article with illustrations, I eventually turned to a more helpful article on wikipedia which filled in the gaps.
At the end of the second day, after asking myself “am I really having fun making this?”, I decided not to continue. In the end all I really got implemented was a simple map with units that moved around walls.
What went wrong
Collision detection
It’s easy to write code to determine if a collision has occured. It’s another thing to handle what happens after a collision has occured. While I managed to eventually ironed out most of the issues with the tile collision, later on I found there were still problems if the unit entered a tile incorrectly where it could get stuck.
Simulations are hard to debug, especially when you add in movement and collision. Knowing how annoying buggy collision detection is, I really wanted to get this solid and ended up spending far too much time resolving collision and movement issues.
NME wasn’t that great
One of my biggest problems was with NME. Not only did I find drawing to be CPU intensive, I stumbled across a rather odd bug with rotations in the HTML5 target. As soon as an element was rotated, the transform for both the sprite and its children became incredibly screwed up. This did not occur in any of the other targets tested.
In addition later on while it worked fine in Chrome and Firefox, Safari was a different matter: I encountered tons of scripting errors and even made it crash!
Finally of note is that text drawing didn’t work properly in the neko target. While I ended up not needing to draw text, it still put a dampner on things.
Loss of motivation
To start off with, while I hyped myself up beforehand I found that when it came to it, I wasn’t really “in the zone”. This made progress much slower than expected. Half-way through, I started to REALLY loose motivation and ended up procrastinating a lot. Coupled with bumping into silly implementation issues, this really killed development. Realy, I underestimated the effort required to implement something like an RTS from scratch.
In the end, While my game made for a nice tech demo with prototype vector graphics, it was distinctly lacking in gameplay.
To Conclude
Using something you don’t know well in a development contest is a recipe for disaster. Having not used NME before, I wasn’t quite sure how certain things worked so I was constantly looking back and forward through the documentation.
In comparison had I used something like Unity, 99% of the underlying functionality would already have worked leaving me to work more on the gameplay and art elements which are far more important for a game.
When developing a game, getting gameplay in ASAP is a must. Without it, all you are really staring at is a tech demo, not a game.
For something like Ludumdare, simpler is better. While an RTS is conceptually simple, the underlying framework isn’t. Looking at some of the entries, a lot of the better ones are based on simpler gameplay concepts or backed by powerful game engines.
Finally, it’s worth remembering that most of the time, a good game isn’t made in a day. In fact, a good game can take years to create. So don’t treat Ludumdare as a way of making a new cool hit game. It probably isn’t going to happen.
For reference, I have put up the code to the “game” up on github.
April 05, 2012
Recently I have been tackling problems in OpenAL code for an iOS app. The trouble with OpenAL is that despite there being a spec, the underlying implementation for this audio API is mostly undefined. What can work for one system can fail miserably in another.
In the case of iOS, no sourcecode is provided so the underlying implementation is partly a mystery, though we can infer from the documentation it uses the AudioUnit API.
Here are a some best practices i’ve developed based on experimentation:
Do you really need to use OpenAL?
In some cases OpenAL may be overkill. For instance if you are just playing a one-off sound effect when pressing a button, it’s probably a better idea to use AVAudioPlayer.
If however you are making a fully immersive 3D AAA shooter, you’re probably a better off using OpenAL. If you are really crazy, for ultimate control you could even try writing your own audio mixer.
Never re-use the same source twice
The implementation of OpenAL on iOS 5.x acts rather oddly when it comes to streaming sources. Lets say you make a music manager and decide to allocate a source. You allocate some buffers then queue them for the source. You then re-use this source to play various music tracks.
However a problems arise as soon as the source is re-used. If you simply stop it and de-allocate buffers, when you queue up a new set of buffers for a new music track the OpenAL implementation seems to get confused and only plays the first buffer you allocate.
Attempting to recover the source at this point is impossible. You can stop it, rewind it, throw buffers away… nothing seems to get it to work properly. Based on this I can only assume you are not meant to use the same source more than once, at least for streaming sources.
Don’t allocate sources you don’t need
Each source you allocate and play in OpenAL will add a mixing unit to the mixer, which will be mixed to produce the final stream. In addition every source you allocate uses memory.
Instead of allocating a bunch of sources, only allocate sources as you need them. From what i’ve been able to determine, sources aren’t that expensive to allocate. It might make sense to limit your allocations however, as there seems to be no fixed limit to the amount of sources you can allocate on iOS.
At one point after receiving a general speedup by changing my source allocation, I theorized that perhaps every source you allocate in OpenAL is mixed regardless of whether or not its playing. However basic experimentation seems to indicate there is no major performance issue with merely allocating lots of sources.
Cache your buffers to disk
If for example you choose to encode your sound effects in something like OGG format to save space, you might notice it takes a noticeable amount of time to decode the audio each time you buffer it, even when using Tremor. This is no fun to an end-user.
Using an easier to decode codec is one solution, but if you have long sound effects it’s still going to take time to decode all the samples.
So it makes sense to keep buffers around for sound effects. However if you have too many buffers you will likely bump into low memory alerts, forcing you to purge non-playing buffers. When you need to play your sound effects again, you need to decode them all over again.
One way you can solve this is by storing your decoded buffers temporarily to disk in the temporary folder (NSTemporaryDirectory). That way when you get a memory alert you can dump your buffers, then whenever you need them again you can quickly re-load them straight from disk.
To conclude
Despite implementation-specific bugs, OpenAL is actually quite a nice and simple library for playing both 2d and 3d audio. There are no licensing fees, the specification is open. It’s available for practically every relevant modern development platform, what more could you want?
Have fun using OpenAL!
March 09, 2012
A while ago i came across SynchTube, a rather neat service which allows you to synchronise the playback of a Youtube videos (as well as other services).
This creates a rather interesting viewing experience: one can discover and collaboratively critique videos they otherwise would never have seen before.
After playing around with Synchtube, i was curious as to how it worked. It turns out the embeddable youtube plugin allows you to query and set the time of a playing video. As long as you have some sort of central authority and a realtime messaging service, you can synchronise playback between multiple users.
In the case of Synchtube, it synchronises the video by sending commands over Websockets. For instance, if you play around with the javascript code, you’ll find simple JSON messages being processed from the server:
Handling add_user with data ["11b85c64",null,null,null,false,false,0].
Handling num_votes with data {"votes":0}.
Handling remove_user with data "0282e6d7".
Sending < with data "=D".
Handling < with data ["66716fea","=d"].
Handling < with data ["55c2f7fa","cats and boots and cats and boots and boots and cats and boots and cats"].
Handling add_user with data ["bbabb3dc",null,null,null,false,false,0].
Handling num_votes with data {"votes":0}.
Handling remove_user with data "bbabb3dc".
Onto the Project
Making youtube videos sync over the web sounded like a cool idea for a project, so i decided to make a simple simple synchronised playback system: play a video in one window, and it’s replicated in another via a simple Websocket protocol.
Putting together Sinatra, ActiveRecord, and thin-websocket i made an initial prototype using a simple JSON messaging protocol with two key commands: “video” to set the video, and “video_time” to set the video time:
[client] {'t': 'subscribe', 'channel_id': 1}
[server] {'t': 'userjoined', 'user': {id: => 'anon_123', name: 'Anonymous', anon: true }}
[server] {'t': 'skip', 'count': 0}
[server] {'t': 'video', 'time': 0, 'force': true, 'url': 'EJ_wXOFQV3M', 'provider': 'youtube', 'title': 'STALLMANQUEST', 'duration' => 152.953, 'playlist' => false, 'position': 0, 'added_by': 'Anonymous'}
A designated leader simply polled the youtube control and sent updates to all the other clients.
Nice enough, but i decided to continue on by adding more functionality: the playlist, video skipping, chat and moderation. All of this functionality ended up being passed around as simple JSON messages through the Websocket connection.
All was going well until i bumped into a design crisis. Originally i wanted users to sign up in order to create and moderate channels, but this solution was becoming more and more undesirable. I wanted to try something different.
So i borrowed an idea from a certain anonymous image board: make everyone anonymous and use Tripcodes to identify users who want to be identified.
Why? well after using Synchtube for a while, i found the only thing i was interested in was the sharing and discovery of videos, not the excessive point scoring by the community. I also noticed a general hostility to users without user accounts, which to me detracted from the experience of watching cool videos. By making everyone anonymous by default i hoped to emphasise the viewership aspect.
While this required rewriting half the authentication mechanism, it was well worth it. Except for the administration interface i didn’t have to worry about implementing user account logic.
Another change i made was to use Backbone.js to help structure the front-end, as well as the administration interface. While this significantly slowed down development, i felt it really helped to keep the back-end service simple and lightweight.
Problems with WebSockets
Websockets are undeniably great as they solve a fundamental problem of how to push messages to clients. Unfortunately though they suffer from poor support in web servers. For instance, during development i was able to serve both the front-end and the Websocket communication through a single port, but i found replicating such a configuration in a production environment to be practically impossible.
Using nginx, i was not able to open a web socket through its HTTP 1.1 backend proxy. This led to the rather undesirable solution of having to serve Websockets directly from the app server on a separate port.
Another problem with Websockets is they don’t seem to work reliably with cookies. So if for example you want to tie a Websocket connection to a logged in user, you need to use another mechanism to authenticate the user such as generating a user token.
Usage of backbone
I have to admit, i hated Backbone.js. It always seemed like a rather arbitrary solution for synchronising models between a client and server. A lot of the examples i saw seemed needlessly complicated and abstracted.
Half-way through development i was getting a bit annoyed at using so many views for something as simple as a list of items, so i decided to refactor and use a different design pattern: use a single view and take advantage of element manipulation and event bubbling in JQuery. This greatly simplified my code in many places, for example:
// Instead of this:
var BanListRow = {
tagName: 'div',
className: 'ban_row',
events: {
"click a.edit": "edit"
}
}
BanListRow.initialize = function() {
this.model.bind("change", this.render, this);
}
...
var BanListPanel = {
tagName: 'div',
id: 'banedit',
events: {
"click a.add_ban": 'createBan'
}
}
BanListPanel.initialize = function() {
BanList.bind('add', this.addBan, this);
BanList.bind('remove', this.removeBan, this);
BanList.bind('reset', this.addBans, this);
BanList.fetch();
}
...
// Consolidate everything together like this :
var BanListPanel = {
tagName: 'div',
id: 'banedit',
events: {
"click a.add_ban": 'createBan'
}
}
BanListPanel.initialize = function() {
BanList.bind('add', this.addBan, this);
BanList.bind('remove', this.removeBan, this);
BanList.bind('reset', this.addBans, this);
BanList.bind('change', this.updateBan, this);
}
Generally speaking i found it best to keep objects to a minimum and take advantage of event bubbling. After realising this i felt a bit more comfortable with using Backbone.js.
To conclude
In the end TestTube turned into an anonymous synchronised youtube playlist. For those interested, i put up an instance so you can check it out:
http://testtube.cuppadev.co.uk/r/1
In all i felt this was a really cool project. It goes to show if you have an idea, even if someone has already implemented it there is nothing stopping you from having a go at implementing it yourself.
February 16, 2012
One of my first rails projects was Railscollab. It started off as nothing more than a proof-of-concept port of ActiveCollab to ruby, and ended up being something more feature complete and in some ways better than the original.
Unfortunately after nearly 5 years of sporadic development, i’m permanently calling it quits.
Over the years, i’ve had various people help out with RailsCollab and enquire about it. Even though it took a while, i kept updating it. I’ve had everything from hate mail to genuine appreciation. It was even instrumental in landing me a job at a cool startup. But really i have never felt a longing attachment to it, and so the project has never really taken off into its own entity.
Lets not forget that even in its minimal state, an open source project is costly to maintain. Rails has a major update pretty much every year, causing problems every time the code is updated. And lets face it: you really do need to keep rails up to date in such a project, since thats where peoples interest lies. It’s a never-ending cycle.
One also has to look at the state of project management apps to realise that RailsCollab really doesn’t offer anything revolutionary or compelling to the market space. Basecamp-like “killer” systems have been cloned to death, so much that i don’t even find it funny anymore.
Do i really want to maintain a boring project management apps nobody really needs? The answer is of course no.
November 29, 2011
Recently i had a project which had some of the worst memory leaks in C++ i’ve ever had to deal with. It had just about every memory leak problem you could think of, all of which could have been solved with a little bit of planning.
Using tools such as Valgrind or Instruments surely helps, but they can only help you so much.
So if you have a nightmarish C++ project with memory leaks, heres a few ways in which you can solve them.
Stage 1: Forgetfulness
We start off with a simple case: when you make an object but never delete it. e.g.:
Object *foo = new Object(); // foo never deleted
Which can be solved by:
delete foo; // <<< delete the object
Stage 2: Garbage Collection
Sometimes you have a pointer to an object which is re-assigned at one point, but the old object is never deleted.
Object *foo;
foo = new Object();
// ... later on ...
foo = new Object();
Which can be solved by deleting the object before re-assigning:
Object *foo;
foo = new Object();
// ... later on ...
delete foo; // <<< delete the old object
foo = new Object();
Stage 3: Destructors
Some people assume if you make a couple of classes like this:
class Foo
{
Foo();
~Foo();
};
class Woo : public Foo
{
Woo();
~Woo();
};
If you destroy an instance of Woo both ~Woo and ~Foo will be called. Only it wont: only ~Woo will be called. Anything you free in ~Foo will never be freed.
So if you want ~Foo to be called too, the destructor for Foo needs to be virtual, i.e.:
class Foo
{
Foo();
virtual ~Foo(); // <<<
};
Stage 4: Spaghetti
Things start getting complicated when you have objects which can be referenced by multiple objects. For example:
Object *foo, *child1, *child2;
foo = new Object();
child1 = new Object();
child1->parent = foo;
child2 = new Object(foo);
child1->parent = foo;
Now when do we delete foo? If we make child1 or child2 delete it, we’ll probably get a crash when we delete foo twice. If we delete it elsewhere, how do we know child1 or child2 aren’t still using it?
One possible solution is to use a reference counting system like in Objective C, so when we reach 0 we delete the object:
class Object
{
Object* retain()
{
retainCount++; // object is being used
return this;
}
void release()
{
--retainCount; // object is no longer being used
if (retainCount <= 0)
delete this;
}
virtual ~Object()
{
if (parent) parent->release();
}
Object *parent;
};
// ...
Object *foo, *child1, *child2;
foo = new Object();
child1 = new Object();
child1->parent = foo->retain(); // object is being used by child1
child2 = new Object(foo);
child1->parent = foo->retain(); // object is being used by child2
If you want to be more fancy you can make a smart pointer class, e.g.
// Modified Object
class Object
{
Object* retain()
{
retainCount++;
return this;
}
void release()
{
--retainCount;
if (retainCount <= 0)
delete this;
}
virtual ~Object()
{
parent = NULL;
}
ObjectReference parent;
};
// The smart pointer
class ObjectReference
{
public:
// Constructor
ObjectReference()
{
object = NULL;
}
// Assignment initializer
ObjectReference(const ObjectReference &ref)
{
object = ref.object ? ref.object->retain() : NULL;
}
// Assignment operator
ObjectReference& operator=(const ObjectReference &ref)
{
if (object) object->release();
object = ref.object ? ref.object->retain() : NULL;
return *this;
}
// Pointer operator
operator Object*() { return object; }
Object *object; // reference to Object
};
// ...
Object *foo, *child1, *child2;
foo = new Object();
child1 = new Object();
child1->parent = foo; // automagically retains foo
child2 = new Object();
child1->parent = foo; // automagically retains foo
Beware however that when you get a circular reference your objects may never be released using this method.
Stage 5: Runaway Spaghetti
Even if you have a reference counting system, you might encounter situations where you release or retain objects too much. Typically memory leak tools only tell you where objects were allocated, not who the retain/release culprit is.
One way of solving this is to keep track of where you retain and release objects
class Object
{
Object* retain(char *file=NULL, int line=0, char *owner=NULL, int addr=0) {
retainCount++;
if (owner)
printf("%x: retain (%i) [%s @ %i] OWNER %s[%x]", this, retainCount, file ? file : "", line, owner, addr);
else
printf("%x: retain (%i) [%s @ %i]", this, retainCount, file ? file : "", line);
return this;
}
void release(char *file=NULL, int line=0, char *owner = NULL, int addr=0) {
--retainCount;
if (owner)
printf("%x: release (%i) [%s @ %i] OWNER %s[%x]", this, retainCount,file ? file : "", line, owner, addr);
else
printf("%x: release (%i) [%s @ %i]", this, retainCount,file ? file : "", line);
if (retainCount <= 0)
delete this;
}
// ...
};
// ...
Object *foo, *child1, *child2;
foo = new Object();
child1 = new Object();
child1->parent = foo->retain(__FILE__, __LINE__, "Object", child1);
child2 = new Object(foo);
child1->parent = foo->retain(__FILE__, __LINE__, "Object", child2);
Then you can simply examine your logs and spot the problematic line of code for that extra release or retain.
Final boss
Of course once you have solved all of your leaks, you might find you bump into the arch nemesis: Memory Corruption. Specifically, this:
class Entity
{
public:
float mNextThink;
Entity();
void think();
};
Entity::Entity()
{
}
What is wrong with this? Well say we have some code like this….
for (int i=0; i<mEntities.size(); i++)
{
if (smCurrentTime >= mEntities[i]->mNextThink)
mEntities[i]->think();
}
Then think may never be called, since mNextThink is never initialized, so its value will be undefined. It could be 0, it could be -10000. Who knows. The solution is simple:
Entity::Entity() :
mNextThink(0) // set a default value
{
}
With all of your memory leaks solved, you should now be able to sleep better.
» View more in the Archives
