Cuppadev

Postmortem of a Ludumdare 23 Game

James Urquhart — Tue, 24 Apr 2012 00:00:00 BST

Last weekend I decided to take part in Ludumdare 23 Jam. Here is a short postmortem.

The theme was “Tiny World”. After a little brainstorming and inspiration I decided to have a go at making an RTS.

After coming up with a basic list of things to do, I started to work on the engine. Since I wanted the minimum of fuss when it came to actually deploying the game, I decided to try out NME, a flash-like API written in haXe which describes itself as a “a free, open-source framework that enables development for iOS, Android, webOS, BlackBerry, Windows, Mac, Linux and Flash Player from a single codebase” - perfect!

While the “game” ended up being a complete unsubmittable disaster, I still found the experience to be quite insightful.

What went right

After a few hours I managed to get a basic prototype working with a unit and walls. At this stage, the unit merely moved towards the mouse cursor and collided against the walls. Next came unit-unit collision, which turned out to be a bit more complicated since I decided to use arbitrarily sized bounding boxes.

Path finding ended up being the simplest thing to implement. For this I used AStar. While initially I stumbled across a rather insightful article with illustrations, I eventually turned to a more helpful article on wikipedia which filled in the gaps.

At the end of the second day, after asking myself “am I really having fun making this?”, I decided not to continue. In the end all I really got implemented was a simple map with units that moved around walls.

What went wrong

Collision detection

It’s easy to write code to determine if a collision has occured. It’s another thing to handle what happens after a collision has occured. While I managed to eventually ironed out most of the issues with the tile collision, later on I found there were still problems if the unit entered a tile incorrectly where it could get stuck.

Simulations are hard to debug, especially when you add in movement and collision. Knowing how annoying buggy collision detection is, I really wanted to get this solid and ended up spending far too much time resolving collision and movement issues.

NME wasn’t that great

One of my biggest problems was with NME. Not only did I find drawing to be CPU intensive, I stumbled across a rather odd bug with rotations in the HTML5 target. As soon as an element was rotated, the transform for both the sprite and its children became incredibly screwed up. This did not occur in any of the other targets tested.

In addition later on while it worked fine in Chrome and Firefox, Safari was a different matter: I encountered tons of scripting errors and even made it crash!

Finally of note is that text drawing didn’t work properly in the neko target. While I ended up not needing to draw text, it still put a dampner on things.

Loss of motivation

To start off with, while I hyped myself up beforehand I found that when it came to it, I wasn’t really “in the zone”. This made progress much slower than expected. Half-way through, I started to REALLY loose motivation and ended up procrastinating a lot. Coupled with bumping into silly implementation issues, this really killed development. Realy, I underestimated the effort required to implement something like an RTS from scratch.

In the end, While my game made for a nice tech demo with prototype vector graphics, it was distinctly lacking in gameplay.

To Conclude

Using something you don’t know well in a development contest is a recipe for disaster. Having not used NME before, I wasn’t quite sure how certain things worked so I was constantly looking back and forward through the documentation.

In comparison had I used something like Unity, 99% of the underlying functionality would already have worked leaving me to work more on the gameplay and art elements which are far more important for a game.

When developing a game, getting gameplay in ASAP is a must. Without it, all you are really staring at is a tech demo, not a game.

For something like Ludumdare, simpler is better. While an RTS is conceptually simple, the underlying framework isn’t. Looking at some of the entries, a lot of the better ones are based on simpler gameplay concepts or backed by powerful game engines.

Finally, it’s worth remembering that most of the time, a good game isn’t made in a day. In fact, a good game can take years to create. So don’t treat Ludumdare as a way of making a new cool hit game. It probably isn’t going to happen.

For reference, I have put up the code to the “game” up on github.

Using OpenAL on iOS

James Urquhart — Thu, 05 Apr 2012 00:00:00 BST

Recently I have been tackling problems in OpenAL code for an iOS app. The trouble with OpenAL is that despite there being a spec, the underlying implementation for this audio API is mostly undefined. What can work for one system can fail miserably in another.

In the case of iOS, no sourcecode is provided so the underlying implementation is partly a mystery, though we can infer from the documentation it uses the AudioUnit API.

Here are a some best practices i’ve developed based on experimentation:

Do you really need to use OpenAL?

In some cases OpenAL may be overkill. For instance if you are just playing a one-off sound effect when pressing a button, it’s probably a better idea to use AVAudioPlayer.

If however you are making a fully immersive 3D AAA shooter, you’re probably a better off using OpenAL. If you are really crazy, for ultimate control you could even try writing your own audio mixer.

Never re-use the same source twice

The implementation of OpenAL on iOS 5.x acts rather oddly when it comes to streaming sources. Lets say you make a music manager and decide to allocate a source. You allocate some buffers then queue them for the source. You then re-use this source to play various music tracks.

However a problems arise as soon as the source is re-used. If you simply stop it and de-allocate buffers, when you queue up a new set of buffers for a new music track the OpenAL implementation seems to get confused and only plays the first buffer you allocate.

Attempting to recover the source at this point is impossible. You can stop it, rewind it, throw buffers away… nothing seems to get it to work properly. Based on this I can only assume you are not meant to use the same source more than once, at least for streaming sources.

Don’t allocate sources you don’t need

Each source you allocate and play in OpenAL will add a mixing unit to the mixer, which will be mixed to produce the final stream. In addition every source you allocate uses memory.

Instead of allocating a bunch of sources, only allocate sources as you need them. From what i’ve been able to determine, sources aren’t that expensive to allocate. It might make sense to limit your allocations however, as there seems to be no fixed limit to the amount of sources you can allocate on iOS.

At one point after receiving a general speedup by changing my source allocation, I theorized that perhaps every source you allocate in OpenAL is mixed regardless of whether or not its playing. However basic experimentation seems to indicate there is no major performance issue with merely allocating lots of sources.

Cache your buffers to disk

If for example you choose to encode your sound effects in something like OGG format to save space, you might notice it takes a noticeable amount of time to decode the audio each time you buffer it, even when using Tremor. This is no fun to an end-user.

Using an easier to decode codec is one solution, but if you have long sound effects it’s still going to take time to decode all the samples.

So it makes sense to keep buffers around for sound effects. However if you have too many buffers you will likely bump into low memory alerts, forcing you to purge non-playing buffers. When you need to play your sound effects again, you need to decode them all over again.

One way you can solve this is by storing your decoded buffers temporarily to disk in the temporary folder (NSTemporaryDirectory). That way when you get a memory alert you can dump your buffers, then whenever you need them again you can quickly re-load them straight from disk.

To conclude

Despite implementation-specific bugs, OpenAL is actually quite a nice and simple library for playing both 2d and 3d audio. There are no licensing fees, the specification is open. It’s available for practically every relevant modern development platform, what more could you want?

Have fun using OpenAL!

Syncing Youtube with Websockets

James Urquhart — Fri, 09 Mar 2012 00:00:00 GMT

A while ago i came across SynchTube, a rather neat service which allows you to synchronise the playback of a Youtube videos (as well as other services).

This creates a rather interesting viewing experience: one can discover and collaboratively critique videos they otherwise would never have seen before.

After playing around with Synchtube, i was curious as to how it worked. It turns out the embeddable youtube plugin allows you to query and set the time of a playing video. As long as you have some sort of central authority and a realtime messaging service, you can synchronise playback between multiple users.

In the case of Synchtube, it synchronises the video by sending commands over Websockets. For instance, if you play around with the javascript code, you’ll find simple JSON messages being processed from the server:

Handling add_user with data ["11b85c64",null,null,null,false,false,0].
Handling num_votes with data {"votes":0}.
Handling remove_user with data "0282e6d7".
Sending < with data "=D".
Handling < with data ["66716fea","=d"].
Handling < with data ["55c2f7fa","cats and boots and cats and boots and boots and cats and boots and cats"].
Handling add_user with data ["bbabb3dc",null,null,null,false,false,0].
Handling num_votes with data {"votes":0}.
Handling remove_user with data "bbabb3dc".

Onto the Project

Making youtube videos sync over the web sounded like a cool idea for a project, so i decided to make a simple simple synchronised playback system: play a video in one window, and it’s replicated in another via a simple Websocket protocol.

Putting together Sinatra, ActiveRecord, and thin-websocket i made an initial prototype using a simple JSON messaging protocol with two key commands: “video” to set the video, and “video_time” to set the video time:

[client] {'t': 'subscribe', 'channel_id': 1}
[server] {'t': 'userjoined', 'user': {id: => 'anon_123', name: 'Anonymous', anon: true }}
[server] {'t': 'skip', 'count': 0}
[server] {'t': 'video', 'time': 0, 'force': true, 'url': 'EJ_wXOFQV3M', 'provider': 'youtube', 'title': 'STALLMANQUEST', 'duration' => 152.953, 'playlist' => false, 'position': 0, 'added_by': 'Anonymous'}

A designated leader simply polled the youtube control and sent updates to all the other clients.

Nice enough, but i decided to continue on by adding more functionality: the playlist, video skipping, chat and moderation. All of this functionality ended up being passed around as simple JSON messages through the Websocket connection.

All was going well until i bumped into a design crisis. Originally i wanted users to sign up in order to create and moderate channels, but this solution was becoming more and more undesirable. I wanted to try something different.

So i borrowed an idea from a certain anonymous image board: make everyone anonymous and use Tripcodes to identify users who want to be identified.

Why? well after using Synchtube for a while, i found the only thing i was interested in was the sharing and discovery of videos, not the excessive point scoring by the community. I also noticed a general hostility to users without user accounts, which to me detracted from the experience of watching cool videos. By making everyone anonymous by default i hoped to emphasise the viewership aspect.

While this required rewriting half the authentication mechanism, it was well worth it. Except for the administration interface i didn’t have to worry about implementing user account logic.

Another change i made was to use Backbone.js to help structure the front-end, as well as the administration interface. While this significantly slowed down development, i felt it really helped to keep the back-end service simple and lightweight.

Problems with WebSockets

Websockets are undeniably great as they solve a fundamental problem of how to push messages to clients. Unfortunately though they suffer from poor support in web servers. For instance, during development i was able to serve both the front-end and the Websocket communication through a single port, but i found replicating such a configuration in a production environment to be practically impossible.

Using nginx, i was not able to open a web socket through its HTTP 1.1 backend proxy. This led to the rather undesirable solution of having to serve Websockets directly from the app server on a separate port.

Another problem with Websockets is they don’t seem to work reliably with cookies. So if for example you want to tie a Websocket connection to a logged in user, you need to use another mechanism to authenticate the user such as generating a user token.

Usage of backbone

I have to admit, i hated Backbone.js. It always seemed like a rather arbitrary solution for synchronising models between a client and server. A lot of the examples i saw seemed needlessly complicated and abstracted.

Half-way through development i was getting a bit annoyed at using so many views for something as simple as a list of items, so i decided to refactor and use a different design pattern: use a single view and take advantage of element manipulation and event bubbling in JQuery. This greatly simplified my code in many places, for example:

// Instead of this:

var BanListRow = {
  tagName: 'div',
  className: 'ban_row',
  events: {
    "click a.edit": "edit"
  }
}
BanListRow.initialize = function() {
  this.model.bind("change", this.render, this);
}

...

var BanListPanel = {
  tagName: 'div',
  id: 'banedit',
  events: {
    "click a.add_ban": 'createBan'
  }
}
BanListPanel.initialize = function() {
  BanList.bind('add', this.addBan, this);
  BanList.bind('remove', this.removeBan, this);
  BanList.bind('reset', this.addBans, this);

  BanList.fetch();
}
...

// Consolidate everything together like this :

var BanListPanel = {
  tagName: 'div',
  id: 'banedit',
  events: {
    "click a.add_ban": 'createBan'
  }
}
BanListPanel.initialize = function() {
  BanList.bind('add', this.addBan, this);
  BanList.bind('remove', this.removeBan, this);
  BanList.bind('reset', this.addBans, this);
  BanList.bind('change', this.updateBan, this);
}

Generally speaking i found it best to keep objects to a minimum and take advantage of event bubbling. After realising this i felt a bit more comfortable with using Backbone.js.

To conclude

In the end TestTube turned into an anonymous synchronised youtube playlist. For those interested, i put up an instance so you can check it out:

http://testtube.cuppadev.co.uk/r/1

In all i felt this was a really cool project. It goes to show if you have an idea, even if someone has already implemented it there is nothing stopping you from having a go at implementing it yourself.

Calling It Quits on RailsCollab

James Urquhart — Thu, 16 Feb 2012 00:00:00 GMT

One of my first rails projects was Railscollab. It started off as nothing more than a proof-of-concept port of ActiveCollab to ruby, and ended up being something more feature complete and in some ways better than the original.

Unfortunately after nearly 5 years of sporadic development, i’m permanently calling it quits.

Over the years, i’ve had various people help out with RailsCollab and enquire about it. Even though it took a while, i kept updating it. I’ve had everything from hate mail to genuine appreciation. It was even instrumental in landing me a job at a cool startup. But really i have never felt a longing attachment to it, and so the project has never really taken off into its own entity.

Lets not forget that even in its minimal state, an open source project is costly to maintain. Rails has a major update pretty much every year, causing problems every time the code is updated. And lets face it: you really do need to keep rails up to date in such a project, since thats where peoples interest lies. It’s a never-ending cycle.

One also has to look at the state of project management apps to realise that RailsCollab really doesn’t offer anything revolutionary or compelling to the market space. Basecamp-like “killer” systems have been cloned to death, so much that i don’t even find it funny anymore.

Do i really want to maintain a boring project management apps nobody really needs? The answer is of course no.

Help! I Have a Memory Leak

James Urquhart — Tue, 29 Nov 2011 00:00:00 GMT

Recently i had a project which had some of the worst memory leaks in C++ i’ve ever had to deal with. It had just about every memory leak problem you could think of, all of which could have been solved with a little bit of planning.

Using tools such as Valgrind or Instruments surely helps, but they can only help you so much.

So if you have a nightmarish C++ project with memory leaks, heres a few ways in which you can solve them.

Stage 1: Forgetfulness

We start off with a simple case: when you make an object but never delete it. e.g.:

  Object *foo = new Object(); // foo never deleted

Which can be solved by:

  delete foo; // <<< delete the object

Stage 2: Garbage Collection

Sometimes you have a pointer to an object which is re-assigned at one point, but the old object is never deleted.

  Object *foo;

  foo = new Object();
  // ... later on ...
  foo = new Object();

Which can be solved by deleting the object before re-assigning:

  Object *foo;

  foo = new Object();
  // ... later on ...
  delete foo; // <<< delete the old object
  foo = new Object();

Stage 3: Destructors

Some people assume if you make a couple of classes like this:

  class Foo
  {
    Foo();
    ~Foo();
  };

  class Woo : public Foo
  {
    Woo();
    ~Woo();
  };

If you destroy an instance of Woo both ~Woo and ~Foo will be called. Only it wont: only ~Woo will be called. Anything you free in ~Foo will never be freed.

So if you want ~Foo to be called too, the destructor for Foo needs to be virtual, i.e.:

  class Foo
  {
    Foo();
    virtual ~Foo(); // <<<
  };

Stage 4: Spaghetti

Things start getting complicated when you have objects which can be referenced by multiple objects. For example:

  Object *foo, *child1, *child2;

  foo = new Object();
  child1 = new Object();
  child1->parent = foo;
  child2 = new Object(foo);
  child1->parent = foo;

Now when do we delete foo? If we make child1 or child2 delete it, we’ll probably get a crash when we delete foo twice. If we delete it elsewhere, how do we know child1 or child2 aren’t still using it?

One possible solution is to use a reference counting system like in Objective C, so when we reach 0 we delete the object:

  class Object
  {
    Object* retain()
    {
      retainCount++; // object is being used
      return this;
    }
    void release()
    {
      --retainCount; // object is no longer being used
      if (retainCount <= 0)
        delete this;
    }
    
    virtual ~Object()
    {
      if (parent) parent->release();
    }
    
    Object *parent;
  };

  // ...

  Object *foo, *child1, *child2;

  foo = new Object();
  child1 = new Object();
  child1->parent = foo->retain(); // object is being used by child1
  child2 = new Object(foo);
  child1->parent = foo->retain(); // object is being used by child2

If you want to be more fancy you can make a smart pointer class, e.g.

  // Modified Object
  
  class Object
  {
    Object* retain()
    {
      retainCount++;
      return this;
    }
  
    void release()
    {
      --retainCount;
      if (retainCount <= 0)
        delete this;
    }
  
    virtual ~Object()
    {
      parent = NULL;
    }
  
    ObjectReference parent;
  };

  // The smart pointer

  class ObjectReference
  {
  public:
    // Constructor
    ObjectReference()
    {
      object = NULL;
    }
    
    // Assignment initializer
    ObjectReference(const ObjectReference &ref)
    {
      object = ref.object ? ref.object->retain() : NULL;
    }
    
    // Assignment operator
    ObjectReference& operator=(const ObjectReference &ref)
    {
      if (object) object->release();
      object = ref.object ? ref.object->retain() : NULL;
      return *this;
    }
    
    // Pointer operator
    operator Object*()        { return object; }
    
    Object *object; // reference to Object
  };
  
  // ...
  
  Object *foo, *child1, *child2;

  foo = new Object();
  child1 = new Object();
  child1->parent = foo; // automagically retains foo
  child2 = new Object();
  child1->parent = foo; // automagically retains foo

Beware however that when you get a circular reference your objects may never be released using this method.

Stage 5: Runaway Spaghetti

Even if you have a reference counting system, you might encounter situations where you release or retain objects too much. Typically memory leak tools only tell you where objects were allocated, not who the retain/release culprit is.

One way of solving this is to keep track of where you retain and release objects

  class Object
  {
    Object* retain(char *file=NULL, int line=0, char *owner=NULL, int addr=0) {
       retainCount++; 
       if (owner)
         printf("%x: retain (%i) [%s @ %i] OWNER %s[%x]", this, retainCount, file ? file : "", line, owner, addr);
       else
         printf("%x: retain (%i) [%s @ %i]", this, retainCount, file ? file : "", line);  
       return this;
    }

    void release(char *file=NULL, int line=0, char *owner = NULL, int addr=0) {
       --retainCount;
       if (owner)
         printf("%x: release (%i) [%s @ %i] OWNER %s[%x]", this, retainCount,file ? file : "", line, owner, addr);
       else
         printf("%x: release (%i) [%s @ %i]", this, retainCount,file ? file : "", line);
   
       if (retainCount <= 0)
         delete this;
    }
    
    // ...
  };

  // ...

  Object *foo, *child1, *child2;

  foo = new Object();
  child1 = new Object();
  child1->parent = foo->retain(__FILE__, __LINE__, "Object", child1);
  child2 = new Object(foo);
  child1->parent = foo->retain(__FILE__, __LINE__, "Object", child2);

Then you can simply examine your logs and spot the problematic line of code for that extra release or retain.

Final boss

Of course once you have solved all of your leaks, you might find you bump into the arch nemesis: Memory Corruption. Specifically, this:

  class Entity
  {
  public:
    float mNextThink;
  
    Entity();
    void think();
  };

  Entity::Entity()
  {
  
  }

What is wrong with this? Well say we have some code like this….

  for (int i=0; i<mEntities.size(); i++)
  {
    if (smCurrentTime >= mEntities[i]->mNextThink)
      mEntities[i]->think();
  }

Then think may never be called, since mNextThink is never initialized, so its value will be undefined. It could be 0, it could be -10000. Who knows. The solution is simple:

  Entity::Entity() :
  mNextThink(0) // set a default value
  {
  }

With all of your memory leaks solved, you should now be able to sleep better.

The Melancholy of Startups

James Urquhart — Fri, 25 Nov 2011 00:00:00 GMT

Once upon a time i started working for a startup which in hindsight turned into one of the worst moments of my development career. While it certainly wasn’t all bad, it didn’t live up to expectations.

Before all of this, i thought working in a startup was quite a cool concept. Developers at the start have the power to be instrumental in developing the business and its products, so in a way it sounded like the ultimate creative effort. That combined with the potential of “The big buyout” was very attractive proposition.

Before i joined, i debated the options. Being cautious i didn’t want to go full-blown into the business, so instead i telecommuted as a freelancer with my other existing commitments.
In hindsight this was the best decision i ever made.

At first the working environment was great. I was tasked with implementing some rather cool features into the main product, working with fairly interesting people, some of whom were instrumental in improving the product.
Although i technically didn’t have any obligation to work flat out at times, i did so under the guise of making a difference.

As time went on, i ended up visiting their offices. But it was nothing like i had expected: instead of a “warm buzz” of a startup i instead found more of an awkward cold silence. Perhaps this was the first warning something wasn’t quite right.

Around this time i noticed operations were commonly micro-managed and interrupt-driven, which was ok unless you wanted to get anything done. There always seemed to be a crisis that needed resolving, so i was always on edge.

Besides that, all seemed fine until the next major iteration of the product when development focus changed.

The new focus? Crunch on getting the new release out. Ignore everything else you are doing, no excuses. I developed the dreaded “Tunnel Vision” and relentlessly worked. After all, if i wasn’t doing something, what was i there for?

As the focus on features dwindled, my main role gradually turned into the bug fixer, which from what i can tell was another way of saying “we don’t appreciate you anymore.”

The biggest demotivation came as the team decided to completely rewrite the next version using the next big thing. This effectively meant everything i had worked on and worried about for the past year had amounted to nothing.

As the startup grew, weekly meetings were introduced as a means to improve communication and determine who was working on what, which besides wasting time constantly reminded me how boring and mundane my work had become.

By this time I was suffering increasingly from stress and burnout from dealing with the startup and other commitments to make things work out.
You could probably have measured my productivity by the amount of lolcats and memes i posted into the company chatroom as a means of coping.

Eventually, things got so worse i had to take a break.

Several months later, i had a hard think: did i really want to continue working in this fashion?
The answer was no. I decided to leave for good.

Safe to say in the space of 2 years my illusion of startups being cool, fun companies to work with had been shattered.

I will probably never work for a startup ever again.

You can't pirate a web app

James Urquhart — Sat, 16 Jul 2011 00:00:00 BST

…or can you?

Since the dawn of consumer computing, people have pirated applications. Why should web apps be any different?

Ripping off website designs or application concepts is nothing new. Though usually maintaining such a rip-off is costly and requires a significant effort to maintain. Not to mention one is always playing cat and mouse.

I’ve noticed recently the trend towards fat clients. This is where the application is written using an MVC javascript framework (Backbone, Sproutcore, Cappuccino, etc…) and the application simply uses a JSON API on the server and renders all the views on the client.

A great deal of logic and assets are stored on the client. The server is merely used as a lightweight data storage and control system.

So if you can write a replacement server, it’s trivial to “pirate” a web service.

An example

Let’s take QuietWrite for example. A rather nice web-based text editor developed by James Yu.

If we simply save the page to disk, most of the core functionality already works!

Of course, editing text is not very useful - we need to save it too.

To sum it up, within an hour i was able to replicate the backend enough so that i could save documents.

After finishing the working implementation, i was surprised to find out that James supplies the source code to an example app CloudEdit which has a similar API to QuietWrite.

I needn’t have written all that code!

Given how easy it is to deploy an app nowadays you could potentially rip off an application, stick your own ads on it, and get it running all within a day. You can even get free updates for the front-end straight from the original developer.

Provided this fat client trend continues, I wonder how long it will be till we start seeing “CD Keys”, SecuROM, and other elaborate anti-piracy solutions in web applications.

How to make a great conference

James Urquhart — Wed, 06 Jul 2011 00:00:00 BST

I received a bit of critique for my Review of Euruko 2011. Granted I was a bit more critical of a few things compared to my twitter feed, but well…

But what do people look for in a conference? No idea. So what do I look for in a conference?

The marketing

There’s nothing worse than hearing about a conference just when its finished. Maybe I forgot about it, or maybe I just don’t follow the right people on Twitter…

Whatever the case I really wanted to go, but since the marketing failed I missed out!

Lanyrd solves this problem partially, but not every conference is listed there. I also don’t check it regularly.

Even when I hear about an event, there are still more problems that can crop up. Quite a few times i’ve noticed events listed with poor directions. Please, tell me where your event is, otherwise I can’t get to it!

The talks

A great talk is made up of several factors, notably…

The speaker
The content
Relevance to the event

So I would suggest a great talk has the best speaker, with the best content which is completely related to the event.

Whereas the worst talk is narrated by the worst speaker, with the most boring content, which is completely unrelated to the event.

Personally I think this is where Barcamp-style conferences shine: there are no rules. A talk could literally be about anything. Everything is relevant.

Then again themed conferences are better in that you know what to expect when you get there. If you go to a ruby conference you get Ruby, not PHP. Not to mention the schedule is usually predetermined.

The venue

A venue can be the breaking point of the event. It needs to be large enough to accommodate the attendees. It also needs a good layout otherwise congestion will form, making everything look too crowded.

Crowds are great, until everyone starts talking and we bump into the nightclub problem: I CAN’T HEAR WHAT ANYONE IS SAYING.

Besides the specifics of the building, the venue also needs to be in an accessible location. I don’t want to trek 10 miles to the venue, and if there is no food provided people need somewhere near to go to eat.

After-events

I think after-events are also a key part of a conference. Without them your post-conference networking opportunities are limited to the people you just happened to bump into at the conference.

Its also a good opportunity for forming a lasting impression of the event in the heads of attendees. If the party last night was awesome, i’ll probably be thinking of it months down the line when the next conference is on.

Thoughts on Euruko 2011

James Urquhart — Sun, 12 Jun 2011 00:00:00 BST

2 weeks ago I attended the Euruko 2011 event in Berlin. In all I would say it was well worth going to, despite a few flaws.

The event was hosted in the Kino International, a cinema close to the centre of Berlin. So it was quite easy to find. Connectivity was not a problem either, as the best free wifi i’ve seen in ages was provided.

The quality of the talks was extremely varied. For instance, David Calavera’s “JRuby hacking guide” talk drew me into a state of REM sleep. In fact the next time I have insomnia, I shall be sure to seek his assistance.

In contrast to this, Paolo Perrotta’s “The Revenge of method_missing()” was a stroke of genius. Great pacing, casual tone. I found the pictures accompanying the code to be quite complimentary and hilarious.

Really, I could divide the main talks in Euruko into one of several categories.

“Here’s my code”

A talk in which the speaker meticulously describes code, a design pattern, or something equally as boring to the outside observer. Quite often the code does not have any relation to what you are doing.

“The other day…”

A talk in which the speaker tells a story with illustrations or code to accompany their points in a way which is easy to remember.

“Use my product”

The speaker describes how they solved a problem, creating a product or project in the process.

EDIT: i would also like to point out that despite my critique of the talks, overall i thought the event was great. Things such as the keynotes and the sing-along at the end were fantastic.

Enough of the talks…

Of course, a conference is more than just watching people talk in front of an impressive cinema screen. During breaks there was an opportunity to cram into the lobby area for a free drink, or to talk to one of the many developers squished against the walls.

Not to say the venue was small, but given the natural tendency of people to congregate in areas near the seating, doorways, and conference room the feeling of being in a small space was inevitable.

Predictably the free drinks and food ran out quite quickly on both days. Rather deviously someone decided to serve sparkling water next to the still water. My taste-buds still haven’t recovered.

The after-events

On both days there were events afterwards to go to. I have to say, the party at “Tante Käthe” was the most difficult thing to find, even with some helpful pointers provided by the organisers and the modern wonders of GPS.

In fact, if I had not bumped into some other attendees I would never have guessed I needed to go down a dark pathway, over a fence, round a corner and down another dark pathway.

Sadly the planned BBQ was cancelled so there was nothing to eat, though free drinks were available. To be honest though, the event itself did not live up to expectations considering you needed the skills of Indiana Jones to find it.

The second day after-event was the Github drinkup. Kudos to Github for the free drinks. However I somewhat suspect if the Github guys weren’t around, this would never have happened.

Still, drinking in a nice easy to locate venue was a great experience.

So really, was it really worth attending Euruko? Yes. It’s a great magnet for ruby developers and business people, so networking opportunities were plentiful. And there were several inspiring gems in the talks to make it worth it.

Bring on Euruko 2012!

Maintaining sanity with Automated Software Testing

James Urquhart — Sat, 15 Jan 2011 00:00:00 GMT

Back when i started on the long and windy road of Software Development, nobody ever seemed to talk about Automated Software testing. None of the articles or books i read about mentioned anything about “Test driven development” or even “Unit Testing”.

In fact, the only testing i heard about was getting a load of people to test your software and send feedback.

Before testing...

Quite often i would encounter stupid bugs during development which made no sense, which could in the worst case scenario take weeks to track down and resolve.

Why did it take so long? Because typically i was following this methodology:

Step through code in debugger, find problematic line.
Assume everything not related to the said line(s) worked perfectly.
Spend an extortionate amount of time tweaking lines wondering why "impossible" things were happening based on incorrect assumptions.

And when i got problems such as heap corruption… that was not a pretty sight!

Really, the first time i heard about things such as “Unit Testing” was back when i started working with “Ruby on Rails”. A lot of rails developers seemed to be bragging about using “Unit Testing”, “rspec”, or even “Behaviour Driven Development”.

However my C++ game developer driven background kicked in and i failed to see the usefulness of such testing. I reasoned:

"Automated Software testing doesn't work well for games, they need REAL people to test them"
"I'm too busy writing code to make tests for it!"
"If i write code once and it works, i shouldn't need to check it works again"

The turning point

Once i started to work with larger teams however, things like this started to happen:

Checked in code, works perfectly
Other guy checks in code, unrelated code breaks
Other guy fails to notice everything else broke

When working with a large team, you simply cannot count on everyone knowing as much about the code as you. If something they are not working on breaks, nobody notices. That is where Automated Software Testing comes in handy!

It was about this time that i noticed i was growing tired of dealing with countless numbers of regressions and bizarre bugs with my own projects. Sometimes the stupidest mistake when developing would go unnoticed for months.

Then of course after fixing the stupid mistakes, others would crop up - an endless cycle of stamping out fires!

I simply could no longer reasonably dedicate time to haphazardly fixing all these problems, so i caved in and started to take automated software testing more seriously.

Now i can safely say i feel more saner and confident that my code works as it should. I no longer fret about people changing code, since when it is covered by a test they will have to make sure it still works as intended.

So if you want to keep your sanity developing software, be sure to use automated software testing too!

OpenAL Sucks, Write your own Audio Mixer

James Urquhart — Sat, 11 Dec 2010 00:00:00 GMT

For one of my recent projects, as part of a video recording feature i needed to save mixed audio from OpenAL.

The only problem was, you simply cannot do that using any OpenAL implementation i’ve seen. The only thing which comes remotely close it is the WAV-out device in OpenAL Soft, which for all intensive purposes is useless.

So after lots of research, i ended up writing my own audio mixer. But “how do you write an audio mixer?” you may ask. It is in fact quite simple, assuming you keep it simple.

All an audio mixer does is take input samples, multiplies them according to the volume then adds (mixes) them together into an output buffer.

The output buffer typically uses a higher bit-depth than the buffers which are mixed, otherwise there’s a fair chance for clipping. As a final step the output buffer is converted to the correct sampling rate for output, or in my case recording.

My Implementation

For my mixer, i decided upon the following abstraction which handles managing audio channels and buffers

SoundMixer - where all channels are mixed
SoundChannel - a channel of audio, which links to one or more buffers of audio samples
SoundBuffer - where the actual audio samples to be mixed are stored

The actual mixing takes place in a scratch buffer, which is repeatedly filled then converted and dumped to the target output. The mixing function is as simple as this!


// For each channel....
short* ptr = (short*)in;
float lVol = (channel->mLeftVolume*mMasterVolume);  // calculate volume
float rVol = (channel->mRightVolume*mMasterVolume);
for (int i=0; i


In the end the whole thing works like this:

short out[44100*2*2]; // output samples (stereo, 16bit pcm, 2 seconds)

SoundMixer *mixer = new SoundMixer(32, 44100); // #channels, sampling rate

SoundChannel *channel = mixer->allocChannel();
SoundBuffer *buffer = mixer->allocBuffer();
SoundBuffer *buffer2 = mixer->allocBuffer();

// Make some buffer data

unsigned short random[11025]; // 1s random data
for (int i=0; i<11025; i++) {
   random[i] = randi(0,65536);
}

unsigned short random2[11025]; // 1s random data
for (int i=0; i<11025; i++) {
   random2[i] = sin((float)i) * (65536/4);
}

// Store data in buffers
// (samples, data, source sampling rate, dest sampling rate, format)
buffer->buffer(11025, (unsigned char*)random, 11025, 44100, SOFT_MONO16); 
buffer2->buffer(11025, (unsigned char*)random2, 11025, 44100, SOFT_MONO16);

// Setup channel
channel->mBuffer = NULL;
channel->mLeftVolume = 1.0;  // volume is 0->1.0
channel->mRightVolume = 1.0;
channel->mLeftVolume = 100;
channel->play();             // state is now "playing"

channel->mBufferQueue.pushBuffer(buffer);  // this will play first
channel->mBufferQueue.pushBuffer(buffer2); // followed by this

// Actually mix the audio
mixer->mix(out, 44100/2);       // 0.5 seconds
mixer->mMasterVolume = 100;
mixer->mix(out+44100, 44100/2); // +0.5 seconds
mixer->mix(out+44100, 44100/2); // +0.5 seconds

// Dump
FILE *fp = fopen("dump.raw", "w");
fwrite(out, 1, sizeof(out), fp);
fclose(fp);

// Cleanup
mixer->freeChannel(channel);
mixer->freeBuffer(buffer);
delete mixer;


On top of this fancy audio mixer, i basically made my own wrapper for OpenAL. It basically dispatches OpenAL instructions to both the systems OpenAL and my audio mixer.

ALvoid     talBufferData( ALuint   buffer,
                          ALenum   format,
                          ALvoid*  data,
                          ALsizei  size,
                          ALsizei  freq )
{
   sMainBackend->talBufferData(buffer, format, data, size, freq);
   sSecondBackend->talBufferData((ALuint)mBufferMap.retreive(buffer), format, data, size, freq);
}


So there you have it, a simple audio mixer which works great when you want to do cool things such as record audio from a game.

In case you dont fancy writing your own audio mixer however, there are plenty of good examples of complete audio mixers scattered about. E.g. OpenAL Soft, Allegro (addons).



Encoding Video in Games
James Urquhart — Wed, 24 Nov 2010 00:00:00 GMT
A while ago i had a problem in a project i was working on. I needed to record a video of the game and upload it to YouTube, which at first sounds simple but can is in fact quite complex once you factor in everything.
Decisions, Decisions...
One of the most important things to decide upon was how the encoder should function. My criteria was:

It should encode in realtime.
It needs to record audio from the gameplay.
The resulting video needs to be uploaded to YouTube.

Since I could not afford to block the main thread, so i decided upon using a processing thread to encode the frames. For the audio, i ended up writing my own audio mixer as i could not find any suitable audio libraries that could mix to a raw audio buffer. As for YouTube, i decided to use the patent-free Ogg Theora codec.
The specifics
To handle the actual capture, i made a VideoRecorder object which would periodically extract frames and audio samples, and dispatch them to the encoder thread which was managed by a VideoEncoder. This went through several re-designs until i ended up with something like this:

i.e. the VideoRecorder sends the frames to the VideoEncoder which in turn adds them to the Frame Queue which in turn is processed by the actual Encoder, finally inserting them into the encoded video stream (in this case using Ogg Theora).

I ended up using two queues for video and audio frames, as i needed control over when i could insert video/audio into the encoded stream. In fact, this turned out to be one of the major problems.

In Ogg Theora you need to make sure the video and audio pages you generate are inserted in the correct order in the output stream, otherwise you will get odd playback corruption where frames are dropped, since the decoder will get confused with the timestamps.

Beyond that, uploading a video to YouTube is remarkably simple. Though to save time, i ended up using the code from Claus Hofele’s excellent Share Your Experience: YouTube Integration In Games on Gamasutra.


Palettes in ScummC
James Urquhart — Sat, 14 Aug 2010 00:00:00 BST
One thing which i have been meaning to incorporate into ScummC for quite a while now is better palette manipulation tools.

It seems that sometime after the turn of the century, the art of making an image editor with good palette manipulation seems to have been lost. This is very unfortunate, as it makes it VERY hard to make graphics for scummc.

So i improved scummc in two ways…
The palcat tool is now more useful
Have tons of non-indexed graphics? Well now you can tell palcat to combine and quantise all the colours in your images, so you have a single common palette.
palcat -o out -col 32 costume1_*.bmp costume2_*.bmp
Quite commonly in scumm games, you’ll notice the room palettes will follow the same general schema:

OFFS SIZE DESC
0 16 System Colors
16 X Room Colors
16+X Y Costume Colors

To reproduce with palcat, simply do:
palcat -o out sys.bmp room.bmp costumes.bmp
(where costumes.bmp contains your combined costume palette(s))
The costume compiler helps you
In scumm, costumes have a palette which indexes into the current room palette. Before you had to specify this manually, like so:
palette([0-32]);
Now you can pass in your combined room palette instead, like so:
palette("out_blank_room.bmp");
Which will match up colors in out_blank_room.bmp with your costume bitmap.

If you have costumes which will be re-used across rooms, i suggest you make a blank room bitmap so it won’t accidentally match colors in your room when you don’t want it to.
palcat -o out sys.bmp blank_room.bmp actors.bmp
I’ve incorporated the costume palette trick in the examples on my scummc fork. Feel free to check it out on github!


Where is ScummC?
James Urquhart — Sat, 17 Jul 2010 00:00:00 BST
With the recent remakes of certain Lucasarts adventure games, i thought to myself “i wonder how the scummc compiler project is getting on?”

To my shock however, it seems the scummc website suddenly died sometime last year, so nobody was able to check out and learn from the only functioning third party SCUMM compiler.

Thankfully i was able to obtain a fairly recent copy of the files in the SVN repository, and coupled with the SVN commit logs on the mailing list i have been able to re-create the repository using GIT as far back as revision 359.

So what are you waiting for? Check it out on github!


Don't write that framework
James Urquhart — Wed, 03 Mar 2010 00:00:00 GMT
Writing frameworks is a waste of time.

That is the conclusion after spending most of my time on my last project writing a framework, in order to make the project “easier” to write.

The trouble is every minute you divert to the framework, half of your project dies. And since there is an inherent division between project and framework, by the time you have everything working you have lost your original train of thought.

Then you realize: the project is way behind schedule; there goes a cool feature that could have landed you a multitude of sales. Or even worse, everyone has lost interest.

Instead, I suggest the following mode of thought:


Find an existing framework that does most of what you require.

If the framework falls short, rethink your approach.

If you find your code has become a mess, re-factor it into a small library.


Don’t be tempted to spend time writing a monolith of a framework. And beware the trap of thinking you can turn it into a product in itself; after all, there is a reason why a large proportion of frameworks are free.


Twackr: Twitter-style time tracking
James Urquhart — Sun, 21 Feb 2010 00:00:00 GMT
It’s been a while since my last rails post, so i decided to try out something different.

Specifically, i’ve always wanted to track my time, but i never seem to find a solution that works. The solution is either too simple and doesn’t give me any feedback, or it’s extremely complicated and hard to use.

So i set about to make a time tracking solution that was both ridiculously simple to use, and could give me some useful reports back. Thus, Twackr was born!

I modelled the entry system after Twitter: at its simplest, all you need to do is type in what you are doing, and Twackr will start tracking your time instantly. When you are done, simply click on “Finish” and it will be done!

Already spent some time doing something but couldn’t get to Twackr? Then you can backdate entries by adding ”-1H”. Still working on something? Then try ”-1H+”. Blocking out some time? Then use ”+1H”. Want to see if you’ll make the deadline? try typing ”1H”.

Time can also be categorised into “Services” (#service) and “Projects” (@project).

You can check out a live demo of twackr on heroku. Just sign up, and check it out - it couldn’t be any simpler!

Like most of my other rails projects, Twackr is Open Source. So feel free to fork it on github.


More CSS Animations
James Urquhart — Sat, 06 Feb 2010 00:00:00 GMT
A few days ago i popped across Anthony Calzadillas ”Pure CSS3 AT-AT Walker”, which is a great application of using CSS3 Transforms and Animations to create a scene of an AT-AT Walker walking along in a simulated iPad.

Anthony explains in great detail how he built the AT-AT, but i couldn’t help but think ”I’ve got a better way of doing this!”.

Last year i made a CSS Transform Exporter for Blender, which can take any scene and generate a corresponding version in a nice and simple HTML page.

So it can take this:

And turn it into this:

Admittedly simple, but it does show off the potential power in adopting CSS3 and HTML5. After all, who needs to use Flash when you can make everything work in actual web browser instead?


The Ovi Store
James Urquhart — Fri, 15 Jan 2010 00:00:00 GMT
As any mobile app developer knows, there comes a time when you need to release your creations out to the general public. On the iPhone, one has the App Store. On Android, the Android Market. On Nokia devices, the Ovi Store.

Whenever i hear someone talk about the Ovi Store, they never have anything good to say about it. Having not experienced the store myself, i remained skeptical.

Recently the Ovi Store on the Nokia N900 opened, so naturally i decided to check it out and see if it really was as bad as people suggested. What follows are my experiences, illustrated with accompanying screenshots.
To Start
First i tap on the “Ovi Store” icon on my device. After a short while, a web browser opens in fullscreen mode. So far, not bad.

Not sure what i want to do, i tap around the tabs (each loading a new page). Now i realise: i just saw something interesting on the last page, and want to see it again. But how do i go back? Well first i need to exit fullscreen mode. Next, i need to tap the back button. Then from the history list which appears: 

I must then select the previous slide. THEN if i am really unlucky, i get to see the page load again.

Now i decide i want to download a game, so i tap on the “Games” tab. The “Games” page loads, showing:

So now i think ”I’ll try and download Angry Birds.” Herein lies a puzzling problem. Nothing happens. A few more taps, and the browser zooms in. Finally after a bit more tapping, i manage to get it to recognise i selected the app. The “Angry Birds” page loads.

Ok, so “Download”. But first i need to sign up for an Ovi Account!

After filling out all the forms, i get a “Thanks for signing up” page. By this time, i’ve all but forgotten what app i was going to purchase.

What would come in handy here would be… i don’t know, some sort of “Click here to continue your purchase” button perhaps? Instead, i have to use… the back button.

Finally, i tap the download button again. This time it loads the “Application Manager”, which after a while asks me if i want to continue installing “Angry Birds.”

At last, the app is installed. Eager to run it, i looked in my applications list, but couldn’t find it. Until i looked closer…

Yes, Angry Birds is represented by… a blue box.

Thankfully though, the app ran fine.

All in all, i found the experience of downloading my free app to be unnecessarily frustrating. I can’t help but feel that nobody was really thinking when they designed the Ovi Store. From the constant page reloading, to the confusing appearance of the app on the device itself. I’d dread to think what would happen if i tried to buy an app.

I can safely say that currently the criticisms i have heard of the Ovi Store are well founded.


CSS Transform Exporter for Blender
James Urquhart — Wed, 16 Dec 2009 00:00:00 GMT
Following on from Creating CSS Animations, i have decided to release the exporter script used to create the animated orange test guy.

As the title implies, the exporter script runs in Blender. Simply construct a 2D scene using planes, add in an animation track, run the script and click Export. It’s as simple as that!

Each plane is exported as a HTML DIV. If present, any Ipos will be exported as CSS keyframes.

The exporter even supports parenting, so you can build up complex hierarchies of elements which can easily be transformed in an animation, or even javascript.

Note that currently the exporter only supports Webkit-based browsers (i.e. Chrome, Safari, MobileSafari). It also has limited support for Firefox (sans animations).

For the curious, there are 3 examples:

Walking Orange Guy
Transform Test (not animated)
Animated Image

As for the actual exporter script, it’s available on github.

Happy animating!
EDIT


Thumbnails and Palettes in glgif
James Urquhart — Mon, 14 Dec 2009 00:00:00 GMT
Recently i incorporated support for creating thumbnails in glgif, the backend gif animation library used in Anim8gif.

Rather than taking the easy route out and attempting to load the gif using UIImage, i decided to take advantage of glgif’s decode routines to create an accurate representation of the current state of the gif playback.

Put simply, i took the palette texture which is generated for display, and decoded it into a new UIImage.

The actual process is triggered by assigning a delegate (thumbDelegate), which will be invoked when the next frame in the gif is decoded. The delegate is then cleared.

Simple, right?
More Technical Details
The format of a palette texture in GLES is as follows:

Palette (16 or 256 entries) Indexed Pixels (4bit or 8bit)

The palette can either be 16 or 256 entries long, depending on the format you use. Likewise the pixel data will either use 4 bits or 8 bits. To keep things simple, glgif uses an 8bit palette with 3 8bit entries for the color (GL_PALETTE8_RGB8_OES).

(Note that an 8bit palette can be insufficient as a gif can actually display more than 256 colours across multiple frames, but this is a necessary trade-off for speed)

So in order to decode it: 

Read and decode the palette
Decode each pixel as an index into the palette
Using the decoded pixel data, create a new CoreGraphics bitmap context (using CGBitmapContextCreate)
Convert the bitmap context into a UIImage (using CGBitmapContextCreateImage and [UIImage imageWithCGImage:])


The resultant UIImage can then easily be used as a base to generate a sized down thumbnail of the gif, as illustrated below.

As always, the code for glgif is hosted on github, so feel free to check it out!


Creating CoreAnimation Animations
James Urquhart — Wed, 25 Nov 2009 00:00:00 GMT
Moving on from Creating CSS Animations, i thought "How can i make this exporter more useful?".

The answer is of course to completely remove Webkit from the equation, which leaves us with CoreAnimation. The end result being that the animation can be directly compiled into an iPhone or Mac OS X application.

Not going into much detail, CoreAnimation is essentially the library used by Webkit to display and animate any elements which have been transformed by a CSS Transform.

The actual scene is simply constructed using transformed CALayers. As for the animation, a CAKeyframeAnimation is created and attached to each CALayer. In the case of multiple properties being animated at the same time, a CAAnimationGroup is used to group these animations together.

The end result is a handy animation class which creates both the scene and the related animation tracks. i.e.:

CALayer *root = [self.view layer];

// Make anim
MyAnim *anim = [[MyAnim alloc] init];
CALayer *animRoot = anim.root;
[root addSublayer:animRoot];

[anim play]; // easy!

As an example, the MyAnim class for the previously featured walking orange figure is as follows.

Which leads onto a repeat of the question, is there a real-world demand for such a tool yet?


Creating CSS Animations
James Urquhart — Sun, 22 Nov 2009 00:00:00 GMT
CSS Transforms combined with animations are undoubtedly useful. Unfortunately though, the only way to create them is to manually input values and pretty much guess the result.

This methodology pretty much kills any creative aspirations anyone might have in making these animations. So while there are quite a few cool looking examples out there they lack the glamour of say, flash animations.

It seems to me there is a need for a tool which allows anyone to create these css animations in a visual way. As far as i know, no such tool currently exists. So naturally, i set about making something myself.

Rather than make an entire tool from scratch though, i decided to simply write an exporter script for another tool - in this case, Blender.

Why Blender? Well, it exposes pretty much all of the information i require to construct both a scene and animation tracks. Also, since Blender handles 3D scenes, there is a potential to work with 3D CSS Transforms in the future.

As an example, here’s a simple animation of an orange stick man walking across the screen. The stick man was both constructed and animated entirely within Blender, and exported using my export script.

And the scene as seen in blender:

Which is simply a set of planes linked together and animated. Really, it couldn’t be any simpler.

Currently the only browsers which seem to support both CSS transforms and animations are based off Webkit. i.e. Safari and Chrome (including the iPhone and Android browsers).

Which leads onto the question, is there a real-world demand for such a tool yet?


Making Money in the App Store: I Give Up
James Urquhart — Sat, 14 Nov 2009 00:00:00 GMT
After almost a year of developing a and releasing a handful of iPhone apps to the App Store, i should be rolling in money, right?

Wrong.

In fact, i can officially reveal that with the exception of development contracts, my iPhone apps have netted me a grand total of:
£0
Yes, £0. It takes a lot of face up to facts, but it can’t be much more blunt than that. Granted i did sell a few units, but these were nowhere near the amount required to release any real payment into my bank account.
So what went wrong?
Well to be quite honest, i didn’t really market my apps at all well. I was hoping that simply being in the App Store combined with a word-of-mouth viral networking effect combined with some online ads would be sufficient. From there i could perhaps get some feedback, and introduce improvements in line with demand.

Did it work? Far from it.

Sadly since the iPhone market is pretty much saturated with thousands of apps, this is pretty much impossible to accomplish. You’d probably have more chance winning the lottery than going with that approach.

I even tried making one of my apps free and promoting one of my paid apps in the description. The actual response in sales was negligible.

It seems to me the only viable ways of attracting people to a new app are:

Getting it featured on the App Store
Pushing it out to as many review sites as you can ahead of time
Making it free

With regards to #3, not getting into in-depth analysis i will say this: if you have an app with a remotely useful purpose, people will consistently download it.

For example one of my apps, Anim8gif, gets at least 20 downloads a week - mind you, that figure isn’t really useful since none of those 20 people bother to write a review or send any useful feedback.

Right now though, i’m a bit tired of spending my budget coming up with the next big idea with elaborate marketing that’s likely to bring in tons of revenue. So here’s what i’m going to do.

I’m just giving up on making money off my own iPhone apps altogether. All my current apps: Anim8gif, VidSplit, and Overthecam are now free.

As for making future free apps, i am still undecided. Am i really getting anything out of the experience? Current signs point to no.


The Trouble with CSS Transitions
James Urquhart — Fri, 23 Oct 2009 00:00:00 BST
Recently in a little experiment, i made a rather cool demo which makes use of CSS Transforms, combined with CSS Transitions.

Note: Examples only work in Safari 4+, or other recent webkit-based browsers.

”Cool?” you say. Well admittedly i could have gone for something a bit more elaborate, however as it stands it pretty much serves its purpose - that is to demonstrate that using the magic of CSS Transitions you can create fast, useful web effects which run at acceptable framerates on an iPhone, and even on the desktop (since the tweening of properties is done by the browser).

However to get this admittedly simple demo working, i had to battle through a few odd issues with how these transitions operate.
The problems
To start off, resetting transitions is a bit tricky. Since properties are not processed immediately, you need to wait for the browser to detect you have reset transition values before you transition to any new values.

Thankfully a nice event called webkitTransitionEnd is provided which is fired as soon as a transition is complete, so you could conceivably use it to handle resetting the transition.

The only problem? webkitTransitionEnd seems to only fire when it feels like it.

The above example demonstrates the peculiarity of webkitTransitionEnd. The green/pink circle is meant to change colour every time the event is fired. Since there is an interval running every 100ms which changes the rotation of the circles, what typically happens 99.9% of the time is the transition never really ends, thus no event.

In general, the transition event seems to behave as follows:

Setting... Result
Initial transition (element just created) FAIL
Transition continuously (during transition) MOSTLY FAIL
Transition after a sufficient undefined delay PASS

So safe to say, unless you’ve got a transition you aren’t going to mess about with then don’t rely on webkitTransitionEnd.
Onto quirky platforms...
Finally, with regard to the iPhone platform there are even more quirks. Compared to desktop Safari, the tweening acts quite differently.

One notable quirk is if you make a new element with a transition on Desktop safari, it performs that transition. If however you do the same thing on Mobile Safari, the transition starts at the end. Whether or not this is some bizarre timing issue, i have no idea.

So safe to say, while CSS Transitions are a powerful tool to use, beware of the quirks.


Displaying Image Hints on an iPhone
James Urquhart — Sun, 11 Oct 2009 00:00:00 BST
Recently it has come to my attention that there is no way to view hints on images on an iPhone.

Normally on any modern desktop web browser when you hover your mouse cursor over an image, a hint if provided will be displayed. Certain websites use these hints to comedic effect, and it’s a shame you can’t see them.

Fortunately i have come up with a nice and simple bookmarklet to solve this issue. Simply add it to your bookmarks in Safari, sync over to your iPhone and run the bookmarklet.

To view a hint, simply hold down two fingers over an image. The associated hint will be displayed in a pop-up alert.
REXML could not parse this XML/HTML: 
Drag this link
Or you can check out the full code in this gist.


A Browser within a Browser: HtmlCanvas
James Urquhart — Thu, 08 Oct 2009 00:00:00 BST
It’s been a while since my last truly silly project, so i thought i’d come up with something else.

There has been a lot of talk recently about how the web is essentially replacing desktop applications, which to me sounds great.

But there is something missing from this new age of web applications. In order to run web apps you need a web browser, which is still a desktop app. It seems to me odd that a platform which replaces another still requires that platform to operate.

Which is where htmlcanvas comes in. It’s a HTML renderer implemented in JavaScript, which runs nicely in any capable web browser which supports the Canvas element.

In essence, it’s a web browser implemented within a web browser. The above example was rendered from this HTML:



  


  
Rendering HTML...
  In Canvas!
  0_0




This sort of recursive abstraction is not uncommon in the computing world. For example, C compilers are written in C, Pascal compilers are written in Pascal. There’s even a version of Python implemented within Python. So a web platform implemented within a web platform is quite natural.

Still, it will be a long while before htmlcanvas is remotely capable of running within itself, so a truly recursive web browsing experience is still a long way off.

As with most of my silly projects, the code to htmlcanvas is hosted on github. Fork it here!

Note:: Currently only known to work in Safari 4, FireFox 3.5, or Google Chrome.


The Mystery of Jekyll
James Urquhart — Thu, 01 Oct 2009 00:00:00 BST
I recently stumbled across the much-loved Jekyll, a “blog aware” website generator.

I say “blog aware” since i have yet to be convinced it is. Jekyll seems to work fine if you have a flat layout of _posts with topics, but as soon as you stray away from that, it begins to fall apart.
An Example
I’ll give you a real world example. This blog. Simply enough, i’ve got posts organised into categories. I’ve also got a few tags on the posts, though they are mainly intended for the website feed. After reading the Jekyll documentation, it says:
Categories are derived from the directory structure above the _posts directory. For example, a post at /work/code/_posts/2008-12-24-closures.textile would have this field set to ['work', 'code']."

So i think ”right, i'll make folders for my existing categories then stick their posts into each respective _posts folder”.

This actually works rather well: if i list {{ site.posts }} from /index.html, each post gets listed with their .categories attribute set.

However if i list a posts .categories in the actual post, the list is mysteriously empty. It’s like Jekyll decided to make the categories act as some weird reverse breadcrumb system. Why this would be remotely useful, i have no idea.

The problem with how posts are listed gets further compounded when you start to use topics. All of my real blog posts are tagged “blog”, so in order to list them i actually use site.topics.blog. This works fine, until i try and list posts from within a “category”.

For some bizarre reason, the list of posts varies depending on which category you list from. For example, i have a /feed folder which contains an rss feed template. The only posts listed in the feed are those in my “dev” topic.

My only guess is that since alphabetically “d” comes before “f”, Jekyll is going through each category one-by-one and making posts on-the-fly. Pre-processing? We don’t need pre-processing!

The only real solution seems to be to store all of your posts in the top-level _posts folder, but that sort of defeats the purpose of using a folder-based category system.
Onto pagination and beyond!
Now assuming you have got your posts sorted out, you’ll probably want to paginate them, right? Well good luck with that; while Jekyll will allow you to split posts up into pages, the paginator is a global option. So if you want pages, you’ve got to have pages on everything.

Got multiple lists of posts on a single page? Well watch out, you could be in for a nasty surprise!

Which brings me onto the rather limited Liquid markup Jekyll uses. Now i don’t have much of an issue with the Liquid markup itself, after all i thought it was used rather well in Mephisto.

But in Jekyll, i’m constantly trying to find a workaround for its rather limited set of liquid tags. Want to list posts by date? How about multiple tags? Good luck with that!

So safe to say, unless you have a pretty simple blog with a flat layout then out-of-the-box Jekyll quite simply doesn’t cut it yet - sad but true.


Frozen Synapse... on a Mac!
James Urquhart — Wed, 26 Aug 2009 00:00:00 BST

Yes indeed, it’s Frozen Synapse… Running on a mac.

“What is Frozen Synapse?”, you may ask. Well it’s nothing to do with web development. Nor ruby or rails. Nor even JavaScript for that matter!

Put simply, it’s a great new tactical strategy game from Mode 7 Games!

Unfortunately it’s still in development, so it’ll be a while till the general public can get its hands on the awesomeness of this game. Still, when it’s available you’ll be sure to find a nice mac version.

So until then, keep your eyes peeled for Frozen Synapse!


The Horror of Android Development
James Urquhart — Sat, 15 Aug 2009 00:00:00 BST
Back when the Android SDK was first released, i thought i’d check it out. Turns out there was a google competition where you could win cash prizes for making android apps. "Simple!", i thought. That was until i looked further at the SDK and found it to be far too incomplete and with little documentation.

Safe to say, i didn’t make an Android app.

Recently, the has been another Google competition so i thought i’d check out the SDK. "It's bound to be far more mature and usable!", i thought (not to mention it’s supposed to be the greatest platform in the world to develop for). Unfortunately, it was just as bad as it was the last time i looked at it. 
The Problems
 Firstly, it’s based on Java, which isn't really Java. Even so, it inherits all of Java’s problems. A weird and nonsensical class hierarchy for one. The overuse of abstraction for fairly simple API’s is another. Not to mention, the overload of XML files, which seem to permeate everything you do for no good reason.

In general i’d probably call it a rewrite of J2ME, only 10 times worse. Why worse? well at least with any decent J2ME SDK, you usually get some sort of usable GUI builder. Instead with Android, you get this mess… 


Seriously, it’s the 21st century and i’m reduced to building a GUI using a spreadsheet and dragging a few nondescript labels around the screen in the vain attempt to build something which actually looks good. It’s so bad, even the tutorials recommend editing XML files rather than using it.

Earth to Android: people were making far better gui builders as early as 1989.

The distributed emulator application doesn’t exactly instil me with confidence either. Not only do i have to wait 5 minutes for it to boot up, it’s also horrifically slow for anything other than a simple “Hello World” application. 


Seriously, if Android is supposed to be OS agnostic, why not make a simulated environment rather than a whole emulator? Would be a whole lot faster and usable if you ask me!

And of course, the real killer. The preferred IDE is…
Eclipse
If you asked a bunch of random strangers to build a bridge, you’d get a hideous construction which leads nowhere. In a similar vein, Eclipse is built by a complete bunch of strangers and ends up being an unusable mess. What’s worse is they’ve had 3 whole versions to sort it out, and rather than getting better it seems to be turning into an increasingly bloated mess.

I’ll give you an example. The code window is by default cramped in the middle of the window, it’s like looking through a keyhole to get at your code. Want to find the configuration settings? Good luck. You’ll have to dig through a whole myriad of menus just to change the font size.

Lets not forget the quirks. Want to scroll through a list using a mouse wheel? Well you’d better watch out for those pesky hint boxes which pop up and completely block focus.

But i digress; continuing with the Android SDK, another issue i have is with the actual documentation. Most of the time it follows a policy of listing what i can do, but not giving me any good examples of how to do it.

I’ll give you an example, looking at the ”3D with OpenGL” section: 
"Writing a summary of how to actually write 3D applications using OpenGL is beyond the scope of this text and is left as an exercise for the reader."
 Want to write OpenGL applications? We’ll just vaguely describe how you could do it and just leave you to guess it all out on your own. Not that your app will be usable on all devices of course.

It also likes to mention technologies with little to no explanation of what they do. Taking the ”Audio and Video” section as an example, a reference is made to “Playing JET content”. What is “JET content”? I seriously have no idea.

Not to mention, there are a lack of comprehensive tutorials.

Take the Notepad tutorial for example, for every exercise you have to start from an archive of what the project should look like. It is impossible to follow the tutorial through without starting all over again with these archives. There is even an ”Extra Credit” section. I’m not even going to comment on that one.

Really, it feels just like i’m copying and pasting code. i.e. not learning anything. 
To Conclude
 So safe to say, i’m not going to make an Android app anytime in the near future.

Now i realise why Google is offering cash prices for making android apps: The SDK is so horrible, you’ll need all the money to pay for the therapy you’ll need after you finish your app.


The Wonders of iPhone Camera Apps
James Urquhart — Fri, 14 Aug 2009 00:00:00 BST

Those of you who happen to follow me on Twitter might have heard a while back that i tried releasing a camera app for the iPhone, which simply allowed you to overlay arbitrary images as guides over the camera.

Unfortunately it was rejected last month because of the following criteria:

"3.3.1 Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs."  

Which to me was extremely annoying. Until you get rejected for something as arbitrary as this, i don’t think one truly understands how frustrating it can be to have a weeks worth of work flushed down the toilet.

My objections to this were as follows. All the API’s i used were documented. I simply made a UIImagePickerController, a UIView and overlaid it on top of the camera view. The only potentially problematic thing i did was to check for the presence of the TPCameraPushButton in the view hierarchy (again, a public api) to see when the overlay should vanish.

This check was written in such a way that if the class name of the camera button changed in a future release, the overlay would just stay visible all the time.

But no, apparently i didn’t use the API correctly "in the manner prescribed by Apple". Thus no camera app was released.

Now i would have just left it there and waited until Apple made a nice API for me to properly overlay images over the camera (if ever), but what do i find posted on tuaw today?
"Camera Genius for iPhone updates and improves"
Yes, a camera app which overlays images on the camera. But not only that, it completely modifies the camera, and what’s more, it was both updated and approved well before and after my application.

(Note that i did not know about this particular camera app before i made mine. In addition it doesn’t appear to allow custom overlay images, one of the features of my app)

I guess it’s ok to make a camera application which completely modifies the camera UI. But if you just overlay images and check to see if the camera button exists then no, that doesn’t utilise the API in the manner prescribed by Apple.

So i have a challenge for you iPhone Developers out there. I have released the sourcecode to my ill-fated camera app, OverTheCam, under the GPL. Your mission, should you choose to accept it is to get it approved by Apple.

I’m going to award points based on :

	How much of the app you need to change to get it approved. 
	How many "Camera Genius" UI enhancements you can get away with
	How long it takes to get approved
        The alternate name you pick

Additionally entries designed for any version of the OS later than 3.0.1 will not be counted. I’d also like to point out that you’ll also need to change the name of the application when submitting, for obvious reasons.

So what are you waiting for? Grab the code here on github and get submitting!


Moved to gandi.net
James Urquhart — Wed, 12 Aug 2009 00:00:00 BST
For a few years now, i’ve been hosting this site on Dreamhost. Despite the subpar performance, 90% of the time the site was up churning out requests, which for me worked out great.

Still, there’s something about shared hosting that makes me cringe, so i’ve been meaning to move to a VPS (Virtual Private Server) for quite some time now. After much consideration, i’ve decided to go with gandi.net. The reason? I’ve tried it before and they offer a fair price for what they offer: a single 256mb “slice” for $16/month, scalable up to 16 slices.

It may be a little bit more expensive than Dreamhost, but in the long run, i reckon it will pay back in terms of uptime and expandability.

So goodbye Dreamhost, hello Gandi!


RailsCollab isn't Dead
James Urquhart — Sat, 18 Jul 2009 00:00:00 BST
For those of you interested in web-based project management tools, you might have noticed i’ve not really been maintaining RailsCollab for quite a few months now. Why? well because i really wanted to concentrate on other things, such as making cool iPhone apps and the usual contracts.

(Plus to be honest i wasn’t really using it myself)

Still, since its open source other people were free to use and modify it for their own needs. I was pleased to see that there has been a healthy level of forking according to github's network graph.

Feature-wise, one fork made by scambra has even implemented a fully fledged wiki.

I’ve also seen partial localisation’s implemented for German and Simplified Chinese. So safe to say, i’m very impressed that despite my lack of updates, RailsCollab is still going strong - all thanks to its growing community! 
Now for my input...
 Projects like RailsCollab are hardly one-way streets. One has to give back as well as receive contributions.

So i’m pleased to announce that as well as merging the wiki and language updates into the main RailsCollab tree, i’ve also merged in my own updates which convert the Message, Comment, and File controllers into RESTful resource controllers.

Simply put, this means that you can now manipulate messages, comments, and files as XML. e.g.:
/project/1/messages/1.xml

  
  0
  1
  2
  true
  1
  2009-07-16T19:35:00Z
  1
  false
  false
  0
  1
  It's a *test*!
  Test message!
  1
  2009-07-16T19:35:16Z

/project/1/messages/1/comments.xml

  
    0
    
    1
    2009-07-17T20:34:12Z
    5
    false
    true
    Test REST comment - edited
  

/project/1/files.xml

  
    0
    true
    1
    2009-07-18T13:22:20Z
    kitten-logo.png
    9
    false
    false
    false
    2009-07-18T13:34:50Z
  


So what are you waiting for? fork it, use it, enjoy!


New iPhone Puzzle Game: VidSplit
James Urquhart — Tue, 26 May 2009 00:00:00 BST
Just thought i’d let all you devoted readers know, i just released my second iPhone app on iTunes, vidsplit.

vidsplit is a simple picture puzzle game with an added twist: instead of a picture, you have to piece together a full-motion video. With a grand total of two difficulty settings and 3 puzzles, it should keep pretty much any puzzle enthusiast entertained for hours.

On the technical side of things, vidsplit is basically a really souped up fork of anim8gif. Instead of animated gifs however, i’m using far higher quality PVR compressed videos.

Why PVR? Well, it really is the only thing you can use to get smooth video playback, apart from the hardware mpeg decoding chips which sadly aren’t accessible using the SDK. All other methods and texture formats are far too slow to get any acceptable playback speed.

For the curious, you might also be interested to know the video frames are basically 256x256 pixels in size, stretched to the dimensions of the screen. Any bigger and the video files would balloon in size. Any smaller and the video would look like complete mush, so it’s pretty much an acceptable sweet spot in terms of quality.

The decoder is pretty simple. Every frame, a PVR texture is read from the video file (basically concatenated PVR textures) and uploaded as an OpenGLES texture. Ad-infimum. It really couldn’t be any simpler than that!

As for the actual graphical interface, i’m using a custom solution based on my kitten game engine.
To conclude
So what are you waiting for, why not buy it from itunes? :)


Playing Animated GIFs on the iPhone
James Urquhart — Tue, 31 Mar 2009 00:00:00 BST
Following on from my Animated images on the iPhone post, i thought i’d take things a step further.

To re-cap, i pretty much summarised the available options for playing back simple animations on the iphone in this nice table:


Method Problem


Use UIImageView It doesn't scale
Re-draw a UIView every frame Far too slow
Use GLES Beyond the scope of the last article, but not this one!
Transform a clipped UIView each frame We did that last time


To take things further, OpenGLES needs to be used. Specifically, you need to upload and draw a texture for every frame of your animation.

Now while this sounds like a great idea, there is a slight problem: uploading textures on the iPhone is hideously slow. Excluding the requirements of a video decoder, you only have enough time per frame to be able to playback a small stop motion video in RGB format.

All is not lost though. The iPhone supports two rather interesting texture formats: GL_COMPRESSED_RGB_PVRTC_* which is the native format, and GL_PALETTE* which is, as the name implies is a texture with a palette.

The fastest format to upload is PVR, but unfortunately there aren’t many native video codecs or animation formats about that decode to PVR format. Which leaves us with the palette format, which is just about fast enough to playback something more substantial…

Now which animation format uses a palette and is widely supported? Animated gif!

So to cut a long story short, i ended up writing a fairly elaborate library to decode and playback animated gifs on the iPhone, all in realtime.

Provided the gif is not gigantic (since the bigger the gif, the slower the decode + upload), it’s actually quite useable.

In fact, as a proof of concept i ended up writing an app using this library to playback animated gifs from any website in fullscreen (similar to the youtube app). Feel free to check it out - anim8gif.
The code
The code for this gif animation library, glgif, is located on github. As always, feel free to fork!


Double Entry Accounting with Invoicing in Rails
James Urquhart — Mon, 09 Mar 2009 00:00:00 GMT
You may remember a little post i made ages ago about Double Entry Accounting in Rails, in which i went over what one needed in order to make a simple double entry accounting system within the confines of the Ruby on Rails web development framework.

After a few requests by readers, i have decided to write the sequel. Double Entry Accounting with Invoicing in Rails.
To recap...
Last time i identified the following core tables you needed in your database:


Account (has many Postings)

Asset Type (e.g. £, $, monkeys)

Batch (has many Journals, links them into a group of transactions - though not really needed)

Journal (has many Postings, links them into a transaction)

Posting (associates with Account, Journal, and Asset Type)


Though i failed to elaborate on what you really needed to store in these tables. In truth, it can be as complicated or as simple as you’d like. No need for Batches? Then don’t include them.

As long as you at least store amounts in Posting’s, and practice the art of double entry accounting, you should be fine.

The next major point of note was how does one calculate the balances in the accounts? Well it turned out the solution was rather simple:

credit_amount = postings.sum(:value, :conditions => "amount < 0")
debit_amount = postings.sum(:value, :conditions => "amount > 0")

Coupled in with the balances were the amounts in the Postings, which had to be flipped depending on the type of account they resided in:

def real_amount
  return [:asset, :expense].include?(self.account.account_type) ? self.value : -self.value
end

def real_amount=(val)
  self.value = [:asset, :expense].include?(self.account.account_type) ? val : -val
end

In simpler terms: if an account is an asset or expense account, a positive value = a positive amount. Otherwise, a positive value = a negative amount. Vice versa.
So how do we factor invoicing into this?
There are two ways in which invoicing can be linked into a double entry accounting system: 

  Link the payment of each invoice to transactions in the system
  Link the amount owed by each client for each invoice to transactions in the system


Realistically though, #1 is a fairly stupid idea as in the real world, clients don’t always pay on time. Or with the correct amount or order. Not only that, they can also pay off multiple invoices at once!

Which brings us to #2 - linking the amount owed by each client for each invoice into the system.

How do we do this? Well to summarise: 

  Have a revenue account which tracks how much you make.
  Have a checking account which tracks what is in the bank.
  Make an asset account for each client.
  When you make out an invoice, record a transaction which transfers the total amount of the invoice from your revenue account into the client account.
  Link any transaction(s) generated by the invoice to the invoice (so you can revert them later if required)
   When the client pays, record a transaction which transfers the paid amount from the client account into your checking account.


Which might look like this in the system:


Account
Debit
Credit


Checking Accounts





Bank
100
0


Invoice Accounts





Microsoft
50
0



Apple
50
0


Revenue Accounts





Revenue
0
200


So…

For tax purposes, your revenue account will at least sum up to the total amount in all of your invoices.
If the client's account has a positive balance, they owe you money for any one of your invoices.
If the client's account has a negative balance, they've obviously overpaid you. So go party!!

… which more or less concludes this article. Happy accounting!


Animated images on the iPhone - sans memory leaks!
James Urquhart — Thu, 26 Feb 2009 00:00:00 GMT
For those of you who have developed on the iPhone, you may have cringed at the prospect of having an animated image in your app, due to the flaky operation of the UIImageView which is the de-facto method of making a simple image animation.

For small images and animations, UIImageView works well. However, when you get to bigger and longer animations, it simply doesn't scale. You might be fooled by the simulator, but on the real device your images will load very slowly before succumbing to the limited memory available.

Fortunately, there are other methods in which to playback and animated image. The first and perhaps the best method would be to implement everything in GLES. The second, which will be described here might be easier if you don’t want to setup GLES, considering it uses standard UIKit.
To start with
Here’s the theory: UIImageView sucks, so we need another method in which to animate images.


Method Problem


Use UIImageView It doesn't scale
Re-draw a UIView every frame Far too slow
Use GLES Beyond the scope of this article
Transform a clipped UIView each frame None, perfect!


As you can see, there are only two realistic methods in which to animate our image: either use GLES, or transform a view around the screen.

Since i don’t want to be messing around with GLES right now, i’ll tackle the UIView, or flip-book approach.

A flip-book can be made either by putting an image in a UIScrollView and scrolling through it, or by making a custom clipped UIView with the image inserted as a subview.

(The latter approach which i prefer as it is a bit cleaner)
The code put simply
To start off with, your master view needs to be set to clip its subviews on init. i.e.:

self.clipsToBounds = YES;

And the subview, which should be a UIView which basically needs to draw a big image containing all the frames, needs to have user interaction disabled. i.e.:

// MYAnimationViewSlide code

- (void)setImage:(UIImage*)aValue {
  [image release];
  image = [aValue retain];

  // Resize to fit
  CGSize sz = image.size;
  self.frame = CGRectMake(0, 0, sz.width, sz.height);

  [self setNeedsDisplay];
}

- (void)drawRect:(CGRect)rect {
  // ...
  // Just draw the image!
  [image drawAtPoint:CGPointMake(0, 0)];

  // No more need for the image
  [image release];
  image = nil;
  // ...
}

// in init...
  self.userInteractionEnabled = NO;

Then all you need to do is move your slide around in your view by setting the .transform property, i.e.:

// MYAnimationView code

// slide size == child view size
CGSize slideSz = slide.bounds.size;
// frame size == root view size
CGSize viewSz = self.bounds.size;

// Calculate x,y position for the frame
// (assuming your frames are packed in a block rather than a strip)
int framesPerRow = floor(slideSz.width / viewSz.width);
int x = currentFrame % framesPerRow;
int y = currentFrame / framesPerRow;

slide.transform = CGAffineTransformMakeTranslation(-viewSz.width*x, 
                                                   -viewSz.height*y);

While tied to an NSTimer:

// MYAnimationView code

advanceTime = (1.0 / 25.0);
animTimer = [NSTimer scheduledTimerWithTimeInterval:advanceTime
                     target:self 
                     selector:@selector(animateTimer:) 
                     userInfo:nil repeats:YES];

// ...

- (void)animateTimer:(NSTimer*)timer
{
  NSTimeInterval timeInterval=timer.timeInterval;
  NSTimeInterval currentTime=[NSDate timeIntervalSinceReferenceDate];

  if (lastTime != 0) {
    double delta = (currentTime - lastTime) / timeInterval;

    // advance by at least 1 frame, at most frames in delta
    if (delta < 1)
      self.currentFrame += 1;
    else
      self.currentFrame += (int)delta;
  }

  lastTime = currentTime;
}

It’s as simple as that!

(unless your slides exceed 1024x1024, in which case you need to use more than one slide. i’ll let you figure that one out)
The Code
For those of you who are interested in the code for this flip-book, i have pasted a gist on github. As always, feel free to fork!

And in case you want an example image, here’s a set of frames for a 50 frame, 96x66 animation.



Making a Real Calendar in Rails
James Urquhart — Wed, 28 Jan 2009 00:00:00 GMT
For a while now, i have been looking at the various calendar helpers on Github. One thing stands out over all of them: they all suck. 


Why? well, they all implement the same basic concept - that is, they make a table containing rows for each week, and cells for each day, yielding in case the user wants to display anything other than a day number in the cell.

This is fine if all you want to do is display single-day events, but if you want to go to the next level and display multiple “all day” events it is practically impossible as the events are constrained in their table cell. Instead, another approach is needed.

The approach? Well, put simply the events are placed in a row below each set of day cells. Events which span more than 1 row are repeated, and a layout algorithm handles vertical placement.

Now rather than writing a whole post describing in agonising detail how to implement this approach, i thought i’d post a gist on github describing how to do it. Read it here. Any improvements? Feel free to fork. Comments? feel free to post below! 
 


Update: The great guy’s over at elevation have turned this code into a plugin. So if you don’t fancy implementing this all by yourself, just use the easy to install plugin. :)


Surviving a Game Development Job Interview
James Urquhart — Tue, 30 Dec 2008 00:00:00 GMT
The recent demise downfall of Free Radical brought back some fond memories of when i was trying to get a real programmer job in the Game Development industry. Sad to say, i didn’t make it - but then again after second thoughts, i kind of figured it wasn’t the best career path for me anyway.

Still, i learnt quite a lot about what is expected from a game development job interview. And i thought i’d list my top 3 points here, which are general enough to apply to any game development profession: 
1) Bring a laptop to show off how good you are
 This is perhaps the most important point. You cannot rely on thumb drives, cd’s, or the internet.

There is a high probability that your demos will not load properly on your interviewers systems. So be sure to show them on one that you know does work - yours.

In addition, the very fact that your demos are on your own hardware gives that extra inclination that you made them yourself. 
2) Brush up on common knowledge pertaining to your profession.
 Chances are that you will be given some form of test, either on paper or through meticulous quizzing. The aim of this is to weed out clueless candidates. Make sure you are not clueless. 3) Have something to say, questions to ask.
 If you find that the interviewer is talking more than you are, then chances are they are more interested in themselves rather than who they should be interested in - you.

Have something well-formed to talk about. Don’t let the interviewer get carried away talking about some obscure technical concept.

The focus should be on you.

So after waffling on with the usual interview talk, your interviewer will usually ask ”Any more questions?”. At this point, if you answer ”No” then chances are you will fail the interview.

Why? Well, remember that there are two parties being interviewed. You and your interviewer. They are giving the reins over to you, in a sense.

So now is your last chance to stand out from the crowd (if you haven’t done so already). If you do not take it, then there will be little to distinguish yourself from the rest of the candidates. 
To conclude
 Anyone else have any more suggestions? Feel free to comment below!


C to ActionScript via Adobe Alchemy
James Urquhart — Sun, 23 Nov 2008 00:00:00 GMT
Recently Adobe's Alchemy was released to the public for testing. It claims to be a tool which can “compile C and C++ code that is targeted to run on the open source ActionScript Virtual Machine (AVM2)”.

In English, that means you can take C and C++ applications and run them in Flash 10.

Since i am always interested in the next big thing, i though i’d check it out by porting something interesting over. Previously i managed to rewrite a SCUMM interpreter from C to HaXe for Flash 9. So naturally, to form a good comparison i decided to compile this interpreter using Adobe’s Alchemy instead. 
What went right
 


Well, i managed to get it ported, using a front-end written in AS3, and the back-end compiled to a swc via Alchemy. Above you can see a screen shot of the version written in C (the haXe version for comparison here). Compared to the haXe implementation, the C version is much more feature complete.

It should also be noted that it took me only 2 days to get the Adobe Alchemy version to work properly, vs several weeks to get the HaXe version barely functional. 
What went wrong
 While Adobe Alchemy is cool, it was a nightmare to work with. For starters, compiling in C++ was broken (as the new operator didn’t work), so i could only compile programs written in C. This obviously meant no ScummVM port, which would have been far more featureful.

It also meant that i had to put up with the full quirkiness of the horrible Alchemy API. For example, this grabs the current time:

static unsigned sdl_scvm_get_time(scvm_backend_sdl_t* be) {
  AS3_Val ns;
  AS3_Val func;
  AS3_Val ares;
  AS3_Val und = AS3_Undefined();
  AS3_Val params = AS3_Array("");
  int res;

  ns = AS3_String("flash.utils");
  func = AS3_NSGetS(ns, "getTimer");
  ares = AS3_Call(func, und, params);
  res = AS3_IntValue(ares);

  AS3_Release(ares);
  AS3_Release(ns);
  AS3_Release(func);
  AS3_Release(und);
  AS3_Release(params);

  return res;
}

Ok, maybe i am exaggerating a bit here, but it seems that almost every function requires an AS3_Val, which you need to release. And it gets really annoying when you have to keep remembering to explicitly release everything you use. Hasn’t anyone at Adobe heard of autorelease pools?

Another thing is i could not find a proper way of debugging the C code. I had to resort to printing out values and trying to decipher the cryptic error messages which don’t really help me figure out the real problem.

Error: Error #1502: A script has executed for longer than the default timeout period of 15 seconds.
        at ()[45593.achacks.as:3340]
        at ()[45593.achacks.as:3902]
        at ScummTest/initEngine()
        at ScummTest$/process_loadQueue()
        at ScummTest$/swfLoaded()
        at flash.events::EventDispatcher/dispatchEventFunction()
        at flash.events::EventDispatcher/dispatchEvent()
        at flash.net::URLLoader/onComplete()

It would also have helped if there was proper documentation. True, there were a few examples, and you can figure out a few things from looking at the headers, but source code is not documentation.

And finally, even if you ignore the failings of the API and documentation, you still have to contend with the slow performance. In my example, the SCUMM interpreter was notably slower than its natively compiled counterpart, especially when it printed all of its debugging statements to the trace log. And even when it wasn’t printing anything out, it still took a long time doing seemingly nothing. 
To conclude
 To be fair, Adobe Alchemy is still “pre-release” software, so i have no doubt it will improve over the coming months.

In addition Adobe specifically mention that “Alchemy is primarily intended to be used with C/C++ libraries that have few operating system dependencies” and “is not intended for general development of SWF applications using C/C++”.

So i don’t think i will be making a habit of porting C/C++ applications to Flash using Alchemy.

Ok, libraries perhaps. But applications?

I really think it is better to write as much as you can in ActionScript or HaXe. It isn’t like in .NET where you can access objects and classes in the runtime via a transparent interface. You have to explicitly use the AS3 interface in order to manipulate native ActionScript objects.

Still, i look forward to see what becomes of Alchemy.


What's New in Rucksack
James Urquhart — Mon, 27 Oct 2008 00:00:00 GMT
A while ago, i posted a little piece about my Backpack-inspired organisation tool, Rucksack. At the time, i only just got the basics working, and it was nowhere near “production ready”.

Now, it still isn’t really “production ready”, but it has drastically improved these past few months. In August, it started out looking like:

And now it looks like this: 


Which to say the least, is a big improvement.

But ”What's new?”, you may ask. Well apart from the new grey spaced out look…

	You can now see a general overview of recent activity
	You can now upload files
	You can send emails to to pages
	Basic reminders have been implemented with email functionality
	Journals and Status (think Twitter) have been implemented
	Fed-up of the default page width? You can now resize Pages!
	Pages can now be shared to the public
	Client-side interface re-written in jQuery
	Bug fixes and stability improvements

So safe to say, a lot of improvements. But still, a long way to go before it is truly useful for every day organisation.

As always, Rucksack is open source and can be checked out on github.


iPhone Developer NDA Lifted.
James Urquhart — Wed, 01 Oct 2008 00:00:00 BST
After trawling the interwebs today, i came across this little gem on the Apple web site:

( http://developer.apple.com/iphone/program/?nda )

Well, it’s about time.

Being very interested in iPhone development myself, i naturally have posted quite a bit about the iPhone on this blog. However, due to the NDA i couldn’t really post a lot - the ”Running your own iPhone applications without paying the developer fee” was pushing it.

In any case, look forward to more iPhone-related development posts.


Disqus
James Urquhart — Thu, 18 Sep 2008 00:00:00 BST
As a little experiment, i decided to move this blog’s comments to Disqus, a third party service which well… hosts comments.

Setup was extremely simple. It went something like this:

	Make a disqus account
	Add cuppadev.co.uk as a site
	Download wordpress plugin
	Install wordpress plugin on server
	Click "Import" button
	Comments now go through disqus

Wow… that was quite simple. And i like simple.

Hopefully this will help weed out the tons of spam i have been getting recently. I think also that it’s simpler to use, so who knows, i might get more comments.

Now, where did i leave those blog posts?


Software Development Meme
James Urquhart — Wed, 17 Sep 2008 00:00:00 BST
Looks like i’ve been tagged with a meme. This time, it’s about Software Development, courtesy of Gary Vaughan. Since that’s my field, i thought i might as well respond to it.
How old were you when you first started programming?
About 10.
How did you get started in programming?
I picked up a copy of Borland Delphi from PC Plus (a home computing magazine here in the UK). Ran my first program, and thought it was kind-of cool. Little did i know what i would be getting myself into.
What was your first language?
Programming language? Delphi, otherwise known as Object Pascal. Language? English.
What was the first real program you wrote?
The first ever would of course be a “Hello World”. The first one which i would consider “real” would be a neat web-based file storage app, which i didn’t make until many months later.
What languages have you used since you started programming?
C, C++, C#, Pascal, Java, Python, Ruby, JavaScript, SQL, Visual Basic. 6502 assembly.
What was your first professional programming gig?
The first professional programming gig would be freelancing for Mode7 Games.
If you knew then what you know now, would you have started programming?
No. Instead, i would have jumped on the dot com boom by hiring a set of competent programmers and making some really cool website with substance, pulling in crazed investors by the hordes.

Maybe after the bubble burst i would think about starting programming.
If there is one thing you learned along the way that you would tell new developers, what would it be?
Problem solving is the most crucial aspect of programming. Make sure you learn how to do it.
What's the most fun you've ever hadÖ programming?
After “Hello World” ran… it all went downhill from there. :)
Who's next?
Nihilogic


Google Chrome
James Urquhart — Tue, 02 Sep 2008 00:00:00 BST
Well it looks like Google have finally jumped on the Web Browser bandwagon. Google Chrome was released today, which looks like a very interesting browser indeed.

Trying it out myself, i couldn’t figure what the fuss was about until i read the associated comic, which goes into details about the JavaScript engine, tabs in separate processes for security, and of course Gears integration.

So all in all, i think it is a nice and refreshing take on the web browser. For far too long i have had to put up with Firefox the memory hog being slow and chuggy, Safari randomly crashing, Opera not being good enough, and Internet Explorer which is like driving a stake through my ears. Google Chrome on the other hand… not bad.

Still, it does have a few problems. Rather than moan about them though, i’ll just be concise and list them all:

No getImageData support in the Canvas
Runs on Windows only (for now)
Scrolling seems chunky
The Flash plugin slows everything down just as usual

Apart from that, great. Will be interesting to see how Google Chrome advances in the future.


A good use for ":group" in Rails
James Urquhart — Sun, 24 Aug 2008 00:00:00 BST
Recently i needed to make a custom view for activity logs in my Ruby on Rails app. I wanted to create a summary of the activity grouped by the day, without any duplicates. e.g. if i performed an action an object twice, i didn’t want it to be listed twice.

So how does one do this in Rails? Simple - use the ”:group” (i.e. GROUP BY) parameter when doing a find() to combine results into single results.
The nitty gritty
For reference i used the following schema on my logs:

  create_table "application_logs" do |t|
    t.integer  "rel_object_id"
    t.text     "object_name"
    t.string   "rel_object_type"
    t.datetime "created_on",
    t.integer  "created_by_id",
    t.boolean  "is_private",
    t.boolean  "is_silent",
    t.integer  "action_id",
    t.integer  "page_id",
  end

In my case, most of my objects were linked to pages. And for those, i only wanted page activity to be listed once. e.g. “Modified page X” instead of “Modified object 1 on page X”, “Modified object 2 on page X”.

So i needed to group by page_id, created_by_id, created_on (as a date), and both rel_object_id and rel_object_id.

First i came up with the following :group :

"created_by_id, 
 date(created_on), 
 page_id, 
 rel_object_type || rel_object_id"
Note: the "||" operator in SQLite and PostgreSQL concatenates strings.
Unfortunately that doesn’t work properly since i have other objects which aren’t linked to a page, and they would only get listed once (since for all of them page_id would be NULL). So i needed to use the “CASE” statement to differentiate between the two:

"created_by_id, 
 date(created_on), 
 CASE page_id ISNULL 
   WHEN 1 THEN rel_object_type || rel_object_id 
   ELSE page_id 
 END"

So now both the page objects and the regular objects were listed once per day. But there was another problem.

After the query i used the group_by method to group everything into blocks based on the date. But i also use the time zone support in Rails 2.1, and since the database stores its dates in UTC, i got this rather odd issue where in certain circumstances objects were listed twice.

In reality they were the same date, in UTC. But not in the current timezone i was using. So after a bit of investigation i came up with a solution. I needed to offset the date in the query by the UTC offset for the current timezone.

It turns out that there are a ton of different ways to do it, depending on which database you are using. In my case i was testing with SQLite, so the following sufficed:

date(created_on, '+#{Time.zone.utc_offset} seconds')

And for MYSQL (and perhaps others), using INTERVAL works:

date(created_on + INTERVAL #{Time.zone.utc_offset} SECOND)

Of course, this still has its problems. Like what about daylight savings time?

Unfortunately, since there doesn’t seem to be any set standard for specifying what timezone to evaluate times in, you are either going to have to write a specific case for it, or just put up with the dates potentially being off for an hour or two.
The end result
Well, it looks something like this:



Cloning Backpack
James Urquhart — Sun, 17 Aug 2008 00:00:00 BST
Just over a month ago now, i got sick of my chief open source project, RailsCollab. Nobody seemed interested in it, and the code was growing tired and deprecated. So i decided to try making another open Ruby on Rails project.

Like my previous chief open source project, i decided my new one was also going to be a clone. Mainly because i couldn’t really come up with anything else more interesting, bu i also wanted to see what it would be like cloning something from scratch.

After much deliberation, i decided the focus of my attention would be to clone another of 37Signals’ products. Not Basecamp this time - that has already been done to death. Instead, i decided to try and clone… Backpack!

Interestingly while i looked far and wide for a comparable product, i couldn’t seem to find anything. The only one that came remotely close was TiddlyWiki. But while that was free and open, i felt it completely missed the niche that Backpack filled. Collaboratively edited pages, which are actually easy to edit.
The nitty gritty
First of all - before i even writ one line of code - i decided to check out Backpack’s underlying data models. They told me a lot about how Backpack worked, and of course they gave me a few hints about what data i should store.

Backpack

Models

(generally created and modified dates exist for all of these) 

Pages
    - scope
    - title
    - email_address

    Belongings (basically reference from page -> list item + position.
                     referred to by id)
    Lists
        - name
        List Items
            - completed
            - content
            - order
    Notes
        - title
        - content
    Seperators
        - name
        - page id
    Tags
        - name
    WriteboardLink
Writeboard
    - title

    WriteboardPages
        - content
Reminders
    - at time
    - content
    - creator
    - remindees [Users]
Emails
    - title
    - content
Statuses (one per user)
    - message (text)
    - last update
    - owner
Journals
    - body (length=255)
    - owner
    - last update
Users
    - name
Calendar
    - color
    - subscription url (pulling from other calendar?)
    - name
    - token (used in url for sharing)

    Events
        - title
        - occurs_at
        - occurs_until
        - reminded_at
        - remind (bool, note fixed time before)
        - all day (bool)

Next i decided to sketch out some of the models. has_many, belongs_to, etc. I also decided to give some models such as the “Belongings” different names, as i thought the originals sucked.

(Note that by this time i decided to concentrate my efforts on making a Page editor, as i felt that was the key defining feature)

After i got that out the way, i built a simple page RESTful scaffold for editing pages. All it did was render page slots and the widgets contained therein. What it lacked was the rather crucial AJAX editing that was prevalent in Backpack.

So then i decided to take a closer look at how Backpack’s AJAX page editing worked. It turned out that there were a few crucial components:


The widgets - the real content of the page. e.g. Notes, Lists, Dividers

The insertion bar - this pops up when you hover your mouse over the top or bottom of the widgets.

The action bar - this pops up when you click “Insert Here” on the insertion bar, and is inserted between the relevant widgets.

The actions on the action bar - when you click on them, either a widget is inserted (e.g. list) or a form is shown in the slate.

The slate - this either exists in a container before the widgets, or in the action bar. It contains most of the forms.


These components are almost exclusively powered by client-side Javascript. Only when you get to the parts where content is added, edited, or removed from a page do you get an AJAX request sent to the server. e.g. Inserting edit forms, adding widgets and belongings, etc.

What you get back from the server is usually Javascript code which alters the content in the page according to the request.

So once i figured all that out, i eventually managed to implement a rather nifty AJAX-powered page editor which somewhat resembled Backpack’s. Mission accomplished!
The result

Well, here it is. It’s called Rucksack, and it acts similarly to Backpack. You can make pages, and share them with others to edit collaboratively. Strange but true.

”Now wait a minute!” you might be thinking, ”it looks like crap, and it doesn't implement everything”. True, but remember that i was only interested in implementing a proof-of-concept page editor. I wasn’t planning on implementing everything and making it look nice. That is for later.

Now while i did start off thinking of Rucksack as a Backpack clone, i don’t think it is the best frame of mind to be in. So for future development, i will be concentrating less on copying backpack, and more on tailoring it to what i think will be useful in organising.

Meanwhile, i have put the code up on github for anyone to check out.


JavaScript 2 dead? Have no fear, haXe is here!
James Urquhart — Wed, 13 Aug 2008 00:00:00 BST
Earlier on, i noticed this little gem on reddit:

Which led me to this rather interesting mailing list post by Douglas Crockford:

I apologize for my ultrablunt comment. Let me be clear about what I 
meant. It was decided at the Oslo meeting that the project formerly 
known as ES4 is no more. Instead, there will be a new project, 
Harmony, which will be the work of a unified working group. We have 
not yet agreed on a set of goals for Harmony.

I recommend that we close the ES4 list and open a harmony list to 
support this new effort.

Looks like nobody can agree on any set goals for JavaScript 2 (i.e. ES4). Which leads me to think that as with a lot of web standards, JavaScript 2 is just a pipe dream.

Fortunately though, the real world has a solution which already works. Its called haXe, and it implements a lot of what JavaScript 2 should be.

So instead of waiting about for some standards crackpots trying to cobble together a standard, you can write some real code.

Have fun with haXe.


Fuzzy Times
James Urquhart — Thu, 07 Aug 2008 00:00:00 BST
Recently i have been writing a simple web-based reminder app which requires one to input dates and times. To input the date and time, normally one would add some sort of calendar widget which pops up.

But personally i think this tends to be really awkward, especially if you don’t want to be specific about dates or times. e.g. Speculative Opportunities. It also requires a lot of mouse clicking to find and enter the correct date and time.

Fortunately, there are libraries about which aim to solve this issue by allowing you to specify the time in English. Typically they are referred to as “Natural language” date/time parsers. In my case, i found one called Chronic which is distributed as a gem for Ruby. It has even got a nice screencast.
sudo gem install chronic
Consisting of only one public function, Chronic is really easy to use:

require 'chronic'
Chronic.parse("tomorrow at 5")
 #=> Fri Aug 08 17:00:00 +0100 2008

See? Nice and simple. We can also use the options to help chronic get the right time. e.g. If i really meant to say “Tomorrow at 5 AM”, i could fix it like this:

require 'chronic'
Chronic.parse("tomorrow at 5", :ambiguous_time_range => :none)
 #=> Fri Aug 08 5:00:00 +0100 2008

The options you can choose from are as follows:

:context - the context in which the time is assessed (:past or :future).
:now - current time.
:guess - if false, this returns a time range instead of guessing at a specific time.
:ambiguous_time_range - range in hours in which an ambiguous time will be resolved. Best to think of it as hours to skip in the day when picking a time. e.g. setting this to 18 and inputting 5 will result in 5am the next day being chosen.

Sadly i couldn’t seem to find any way of setting which time zone to evaluate the time in. This would have been useful when working with Ruby on Rails 2.1's new TimeZone support. Fortunately though, i figured out a workaround which is as follows:

# grab time using current time zone as reference
ctime = Chronic.parse(value, :now => Time.zone.now)
# re-interpret time in current timezone
ctime = Time.zone.local(ctime.year, ctime.mon, ctime.day, ctime.hour, ctime.min, ctime.sec)

Basically this gives Chronic the time in the current time zone, which deals with relative times (e.g. “tomorrow”). It also re-interprets the calculated time in case you are a bit more specific (e.g. “5AM”).

For something a bit less hackish, one might want to check out technoweenie's fork on github. This appears to allow you to tell Chronic to use a different Time class for calculating times, which should solve the problem.

So to conclude, i think Chronic is a nice and simple solution that works rather nicely for common cases.
Other libraries
If you aren’t using Ruby, have no fear. There are similar “Natural language” parsers available for other programming environments which work in a similar fashion to Chronic.

Datejs for JavaScript
parsedatetime for Python
DateTime::Format::Natural for Perl
DateTimeEnglishParser for .net



Brain-Dead Canvas Quirk
James Urquhart — Sat, 02 Aug 2008 00:00:00 BST
The HTML5 Canvas. Great, isn’t it? You can draw pretty much anything you want in a pre-defined area. In more recent browsers, you can even get & set pixel data too, so you pretty much have full control over how things look.
The problem
Unfortunately, the Canvas suffers from a pretty brain-dead design flaw. That is while you can get and set pixels, you cannot control when the pixel data you grab with getPixelData() will be freed from memory. This is exacerbated by the fact that the whole interface runs on top of JavaScript.

”Now wait a minute”, you might think ”doesn't JavaScript have garbage collection for this sort of stuff?”. Well, yes. The trouble is, JavaScript has no standardized garbage collection system, so depending on which browser you happen to be using it’s pot luck whether or not your pixel data will get freed on time.
An example
To start off with, here's a really simple example test case, using ”putImageData(getImageData())”.

Now lets use a real example here. A while back, you might recall i was writing a SCUMM interpreter in haXe that just so happened to compile to JavaScript as well. For the SCUMM runtime, i am required to display room graphics overlaid with various objects. The graphics are 8bit and use a palette, and that palette can be changed at runtime.

For performance reasons, i decode the graphics and store them in Canvas elements using only the red channel. In fact, throughout the graphics processing pipeline, the red channel is the only part of the pixel data i use. I also make heavy use of the native blitting functions (e.g. fillRect).

It’s not until i come to display the graphics that i re-map the pixels from the red channel into the final colours using the current palette. This requires me to getImageData() the current pixels in the canvas, iterate through and set the data[], and then putImageData() back to the Canvas for the final result.

The trouble is since i am re-mapping the pixels every frame, memory usage on certain browsers (Opera, Firefox) skyrockets up to a certain threshold - anything up to 2gb depending on what is happening.

For the test case running on OS X, Opera> tops out at ~600mb. Webkit and Firefox both top out at around 1.25gb (honestly though, i just stopped looking as it was getting stupid). It’s not until i either suspend re-draw and wait several seconds, or simply close the browser window that memory usage returns to normal.

Oddly enough when re-mapping the pixels in my SCUMM interpreter (not in the test case), Webkit’s memory usage stays constant. This flies in the face of what one would expect to happen based on the test case. As for why, it could be anything. The smaller canvas size, assignment of variables - who knows.
Workarounds?
Realistic workarounds include:


Perform all graphics operations manually on a single set of pixel data (too slow!)

Use drawImage() as opposed to getImageData() & putImageData()

Limit the amount of screen updates

When working with palletised image data that needs re-mapping, store it as RGB, and re-calculate it only when the palette changes

Don’t use getImageData() at all, especially considering it doesn’t seem to work in Safari at the moment.


Either that, or simply implement everything which needs pixel-level graphics manipulation in Flash or a Java Applet, foregoing the pure Javascript ethic.


Running your own iPhone applications without paying the developer fee
James Urquhart — Tue, 22 Jul 2008 00:00:00 BST
Note: these instructions are outdated and should not be taken seriously.
For those of you wanting to build off-line, native applications for the iPhone / iPod Touch (with the 2.0 Software Update), you might have been disappointed to hear you need to shelve out $99 yearly for the privilege of being able to run your applications on a real iPhone.

Ok, $99 might not be a lot, but one also has to consider when you come to deploy your app for real, it might not pass through the rather stringent screening process to get it to the masses through the App Store.

Wouldn’t it be great if you could deploy any application you wanted for free?

Well thanks to the recently released pwnage tool, it’s now possible to run non-Apple-approved applications on your iPhone though “Jailbreaking”. I’ve spent quite a while trying to figure out how to get this working. Here’s what i have come up with so far.
First of all...
If you are not using a Mac, or you don't have the iPhone SDK installed, or alternatively the "open toolkit", then these instructions will be of no use to you.
If you haven’t done so yet, Jailbreak your iPhone / iPod touch via the “pwnage tool”. For reference, this is what i did:

Download the pwnage tool
Create and transfer Jailbroken firmware using the pwnage tool

After you’ve done all that, you’ll probably notice a new application called “Cydia” appears on your iPhone screen. This is basically the way end-users can access a wide range of free “homebrew” applications on their Jailbroken iPhone’s. If you want to deploy your application to end-users, you’ll have to get it into Cydia.

But enough about Cydia for now…
I've built applications, how on earth do i get them over to my iPhone?
If you are using XCode, make sure you set the SDK of your application to “Device - iPhone OS 2.0” (or the equivalent if it exists). Then all you need to do is click “Build and go”, and your application should appear on your iPhone.

Now you’ll probably get this rather “Unexpected error” popping up if you try and run your application with XCode.

Unfortunately while the pwnage tool allows you to run any application, that application still needs to be signed. So how do you sign it? Well, the best tutorial for that i have found is the ”Code Signing Guide” on Apple’s website.

You’ll need to make a self-signed certificate using the Keychain Access utility. Then you should run the following commands in a terminal on your built application:

export CODESIGN_ALLOCATE=/Developer/Platforms/iPhoneOS.platform//Developer/usr/bin/codesign_allocate
codesign -fs "Name of your certificate here" /Path/to/your/built/application.app

Unfortunately even when you do this, you still cannot run your app using XCode, so you can't use any of those fancy debug tools with it. In order to run your application, you'll have to copy it over yourself. You can either use OpenSSH (installed through Cydia), or iphonedisk. I used OpenSSH, as i had problems with the permissions not being preserved with iphonedisk:

scp -r /Path/to/your/built/application.app root@address_of_your_iphone_or_ipod:/Applications/

Re-load the springboard either by installing something with Cydia, or restarting your iPhone. And hey presto, your app appears!
Update: There is now a way to get your apps working directly from XCode. Simply follow the instructions here, and you should be able to debug your application straight from XCode. Neat!
Note: there are of course other ways to get your app working, but personally i found signing it to be the most obvious.Right, now my application is ready for public consumption
Great! So how do we get it into Cydia? Well there are two options:

Get the Cydia guy's to host it in their repository
Make your own repository

Well i couldn’t figure out how you are meant to contact the Cydia guy’s to get your application in their repository. Realistically speaking, this is the only way people are going to find your application, considering there is currently no user friendly way to add new repositories to Cydia.

After all, would you really want to type all of this into the terminal to add a single repository?

ssh root@address_of_your_iphone_or_ipod
echo "deb http://location.of/apt_repository/ ./" > /etc/apt/sources.list.d/apt_repository.list

To conclude
If you are really serious about making money off developing iPhone apps, then i’d cough up and pay the $99 developer fee. That way you’ll be able to target every iPhone / iPod Touch user, while being able to charge a fee for downloading.

If however you have an application Apple are never going to distribute (e.g. emulators), or you are just playing about and don’t mind the lack of XCode integration for on-device debugging, then developing for Jailbroken devices is for you. Personally though i’d recommend you compile your own “open toolkit”, as developing your own unofficial apps using the official SDK is legally questionable.


RailsCollab Alpha 3
James Urquhart — Wed, 09 Jul 2008 00:00:00 BST
Hot off the presses, i’ve just pushed alpha 3 of RailsCollab to both Rubyforge and Github. 
Railscollab? What's that?
 Railscollab is a re-write of the Project Management solution ActiveCollab (otherwise known as ProjectPier), which instead is written in Ruby and runs on the Ruby on Rails web development framework. Why on earth would you want to do that?
 Good question. Well, it all started when the developer of ActiveCollab announced they were ditching the open source version and going commercial and closed source. Of course, there were people like me that didn’t like that.

So i decided to make my own fork - however i added a twist. I decided to re-write it using Ruby on Rails, since the original PHP code was giving me a headache. Plus it seemed like a great way to get to know the Ruby on Rails framework.
Note that around the same time, another fork called ProjectPier arose, which opted to develop from the original PHP code.So what's new?
Quite a lot, actually. Pretty much everything that was present in ActiveCollab 0.7.x is now implemented, sans the the fancy web-based installer. I’ve also focussed a lot on fixing bugs and improving stability, as well as enhancing the time tracking components. 
 … and of course there is the configuration editor which takes advantage of Phusion Passenger. 
 In addition, i have updated the RailsCollab demo, so if you don’t have the time to set everything up, you can still try out Alpha 3.

I’d also like to thank everyone (you know who you are) who has reported issues with RailsCollab over the past few months. Without your help, i doubt i would have had the willpower to make another release.

So what are you waiting for, try it out already. :)
Edit:
Simply select the “Use OpenID” checkbox and type in your OpenID to login. An account should automatically be created for you if one does not already exist, provided that your OpenID provider provides a username, email, and name.

You can get an OpenID from many places - one of my favourites being myOpenID.


SCUMM in JavaScript
James Urquhart — Wed, 28 May 2008 00:00:00 BST
Soon after bumping into haXe and Doomed Online, they got me thinking. “Would it be possible to run SCUMM games in flash?”. The answer to that of course is yes, as was demonstrated by my SCUMM Interpreter written in haXe. Though as with anything, you really need to spend the time to implement everything, and justify it somewhat.
(Not to mention that Flash 10 is supposedly going to be able to compile C code, thus making my complete port more or less redundant)
Now i would have stopped there, if i hadn’t noticed that haXe targets multiple platforms, including every web developer’s best friend, JavaScript. Thus i thought, “Would it be possible to run SCUMM games in Javascript?”. For the answer… well, take a look!
Yes, that is an image of OpenQuest running in a web browser, specifically Firefox. It can run in other browsers too, though to date i have only tested Firefox and Safari, the latter of which mysteriously crashes.
And if you want, you can try it yourself by clicking here. A word of warning though, it is highly recommended that you use Firefox to run it. Otherwise you might get a nasty crash, or a big disappointment.
As previously, just about enough is implemented to get the first room of OpenQuest loaded. Actors, verbs, objects, and sounds are not implemented, though it would certainly be possible. However the engine runs much slower than in Flash, so i doubt anything that could be considered to be “playable” could be implemented with it.
Still, i feel I’ve now accomplished what i set out to do when writing this interpreter – that is to push haXe, Flash, and JavaScript to the limit and see what i can get out of them. And of course, learn a bit more about SCUMM too.
Again, i would like to congratulate the ScummVM team for their SCUMM interpreter and Alban Bedel for his scummc compiler. Without their efforts, i wouldn’t have had any code or references from which to base my haXe-based SCUMM interpreter.
Edit: Source code available from github.
Edit: No longer crashes in Safari provided you are using Safari 3.1.2 or later.


Yet another RailsCollab demo
James Urquhart — Fri, 23 May 2008 00:00:00 BST
Recently i have had another go at tweaking RailsCollab, the pet project of mine (which i have mentioned previously).
After moving the source repository from Subversion to Git, i decided to add in a long standing missing feature – the web-based configuration editor.
Originally i pretty much stayed away from implementing this as Ruby on Rails was a real pain in the ass when it came to deployment, so i couldn’t implement a configuration loader that i knew would work in 90% of deployment cases.
That was, until recently when Phusion Passenger was released.
For those of you that don’t know, Passenger is an Apache module which allows you to simply drop in your Rails application and run it with practically zero configuration. It pretty much falls under the “it just works” category. I have no doubt that anyone using RailsCollab will prefer using Passenger to deploy. Consequently, i have been able to implement a rather nice configuration loader based on this assumption.
Which brings me to the demo. A short while back, my web host, DreamHost added Phusion Passenger support to their web servers. Factor everything together, and you have… the RailsCollab demo!
Link to the demo
“But wait!” you ask, “how do i login?”. Well, using OpenID of course. As with my previous demo,  you should be able to select the “Use OpenID” checkbox and type in your OpenID to login.
This will automagically register you with the system, adding you to the “OpenID” company and the “RailsCollab” project. Afterwards if you want, you can reset your password so you can login using a regular username & password, though personally i think typing in your OpenID is a better idea.
Note that your OpenID provider needs to support handing over identity fields in order for RailsCollab to correctly register you. MyOpenID is an example of a provider that supports this. Also note that if there is a user with a duplicate username or email in the system, registration will fail.
One more thing
The RailsCollab demo also supports the Basecamp API. This means that you can take any existing widget, program, or service that integrates with Basecamp and use it with RailsCollab. e.g. Avalanche, Freshbooks, or Cashboard
So what are you waiting for? Try the demo already!: :)


An Analogue Clock using Safari Transforms
James Urquhart — Thu, 20 Mar 2008 00:00:00 GMT
Recently Safari 3.1 has been released for Mac and Windows, boasting a whole load of useful new features. One of these is support for CSS transforms – i.e. you can translate, scale, and rotate HTML elements in a web page.
CSS transforms can come in quite handy for making complex dynamic objects, without the need for 3rd party plugins or applets. For example i was able to make the following BBC-esque analogue clock, using only DIV’s and JavaScript :
(Note that currently this only works in Safari 3.1)
EDIT: Now works in FireFox too!
Yes, DIV’s and JavaScript. No Canvas, no Flash, no SilverLight, no Java Applets, no Plugins.
It makes me wonder, considering that browsers seem to be getting so powerful nowadays, should one really bother using Flash & co anymore?


What not to write in haXe, Part II
James Urquhart — Fri, 07 Mar 2008 00:00:00 GMT
I decided to have another go at writing some awe-inspiring haXe code again. This time, i picked up from where i left off – that was trying to get my infamous SCUMM interpreter working on additional platforms which haXe supports.
For reference, i concentrated on getting it to work on the neko platform.
To start off with, i took notice of a suggestion to simplify the import statements which littered the top of the source code. So instead of this:
#if flash9
import flash.display.Bitmap;
...
#else neko
import noflash.Bitmap;
...
#end
I consolidated everything into a single file which looked like this:
#if flash9
typedef Bitmap = flash.display.Bitmap;
...
#else neko
typedef ByteArray = noflash.ByteArray;
...
#end
So when i used “import hiscumm.Common”, i could now access all of the classes listed with little fuss!
Another change i made was to re-factor the way resources were loaded. Before, i just used flash 9’s ByteArray. However as there is no direct equivalent of this on the neko platform, i had to implement my own.
Thinking more, i determined that only a relatively small portion of my code actually needed to use all of the features of ByteArray, so instead i changed all of the IO code to use the neko api’s Input & Output classes.
Finally i made all 32bit integers Int32’s. Consequently, i had to re-implement the Int32 class for the flash platform, but all things considered it was the best solution.
Problems
As before, i ran into a fair share of problems. Most were minor, and others were solved when i upgraded from haXe 1.17 to 1.18. Here are some of the more notable ones i came across.
“No arrays larger than 115 elements”
I ended up finding a rather obvious solution to this problem which worked transparently with the other platforms. All one has to do is split up the troublesome array into several smaller arrays and then concatenate them together. e.g.
var longarray = [1,2,3].concat([4,5,6]).concat(7,8,9); // works
Problem solved.
“Beware of classes extending Int”
In an earlier implementation of my Int32 class, i had it extending Int. This turned out to be a fatal mistake, as when i tested out the code absolutely nothing worked.
The solution of course was just to not extend from Int.
“haxe.Timer does not have a constructor”
I kept getting a rather odd compile error. It seemed that no constructor was defined for the Timer class, which i used to run the game loop. This was very odd, as the API documentation didn’t state anything unusual about the Timer class on the neko platform.
However after checking out the code to the Timer, i soon came to realise the horrifying truth: the Timer class wasn’t implemented for the neko platform. Arrrgh! I got this far and now i’m stopped by something as simple as this?
Thankfully i figured out a rather obvious workaround. Instead of using the timer, i merely made a typical run-of-the-mill game loop. Problem solved.
while (true)
{
  onTime();
  neko.Sys.sleep(0.01);
}
The result
As the output below shows, the interpreter now actually runs on neko. Unfortunately there appears to be a rather nasty bug when decoding the room image. But then again i’ve not implemented any video code for the neko platform yet, so the usefulness of this is questionable to say the least.
$ neko test.n
NekoTest.hx:42: Engine init
SCUMM.hx:486: boot state
SPUTMResource.hx:308: Loading resource 1 from file 1, room 2 (SCRIPT)
SCUMM.hx:488: Started
SCUMM6.hx:1249: ARRAY = "" 
SCUMM6.hx:1249: ARRAY = "ScummC Paused !" 
SCUMM6.hx:1249: ARRAY = "Are you sure you want to quit ? (Y/N)Y" 
SCUMM6.hx:1249: ARRAY = "Are you sure you want to restart ? (Y/N)Y" 
SCUMM6.hx:1249: ARRAY = "Save it" 
SCUMM6.hx:1249: ARRAY = "Load it" 
SCUMM6.hx:1249: ARRAY = "Continue" 
SCUMM6.hx:1249: ARRAY = "Cancel" 
SCUMM6.hx:1249: ARRAY = "Quit" 
SCUMM6.hx:1249: ARRAY = "Ok" 
SCUMM6.hx:1249: ARRAY = "Saveing '%s'" 
SCUMM6.hx:1249: ARRAY = "Loading '%s'" 
SCUMM6.hx:1249: ARRAY = "ScummC test Menu" 
SCUMM6.hx:1249: ARRAY = "Save game" 
SCUMM6.hx:1249: ARRAY = "Load game" 
SCUMM6.hx:1249: ARRAY = "Game NOT saved" 
SCUMM6.hx:1249: ARRAY = "Game NOT loaded" 
SCUMM6.hx:1249: ARRAY = "Insert disk %c" 
SCUMM6.hx:1249: ARRAY = "You must enter a name" 
SCUMM6.hx:1249: ARRAY = "Insert your save disk" 
SCUMM6.hx:1249: ARRAY = "Failed to open %s (%c%d)" 
SCUMM6.hx:1249: ARRAY = "Read error on disk %c (%c%d)" 
SCUMM6.hx:788: 103, 645
SPUTMResource.hx:308: Loading resource 2 from file 1, room 2 (ROOM)
SPUTMRoom.hx:173: RMIM == RMIM
SPUTMImage.hx:109: smap size == 8
SPUTM.hx:1096: Internal exception, aborting! (state=SPUTM_RUNNING)
SPUTM.hx:1097: >>
As before, the hiscumm code is available for reference. Unlike last time however, the code is now hosted in a git repository so you can now see all the nitty gritty changes i have made to it. Great!
Click here to check it out!


What not to write in haXe
James Urquhart — Sun, 27 Jan 2008 00:00:00 GMT
You might have notice that during the past few months i’ve written a few articles about haXe, a language and compiler which pits itself as a “toolbox for the web developer”.
Recently i decided to test out haXe’s platform support. I did this by taking my SCUMM interpreter code – which i had previously written for flash 9 – and tried to get it to work on the other two major platforms haXe supports, javascript and neko.
To start off with, i had to remove its dependency on flash. This was achieved by making replacement classes and switching between the two using the conditional pre-processor. i.e.:
#if flash9
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.utils.ByteArray;
import flash.geom.Point;
import flash.geom.Rectangle;
import flash.utils.Timer;
import flash.events.TimerEvent;
#else neko
import noflash.ByteArray;
import noflash.Bitmap;
import noflash.BitmapData;
import noflash.Point;
import noflash.Rectangle;
import noflash.Timer;
import noflash.TimerEvent;
#end
Most of the modifications to the code looked like this, with liberal sprinklings of ”#if flash9 … #end” round the flash specific code.
Eventually i got to the point where things started to look like they were going to compile, or so i thought...
Problems
“Array too big”
The first problem i encountered was that it wouldn’t compile.
hiscumm/SCUMM6.hx:1870: lines 1870-2127 : This array declaration is too big, try to split it
Array declaration too big? What on earth was it going on about?
| TArrayDecl el ->
if List.length el > 115 then error "This array declaration is too big, try to split it" e.epos;
call p (field p (ident p "Array") "new1") [array p (List.map (gen_expr ctx) el); int p (List.length el)]
Oh, right. Seems i cannot have arrays declared with more than 115 elements. A seemingly arbitrary limitation that is not referenced anywhere in the documentation – great!
Not needing the opcode table for my little experiment, i commented most of it out. Problem solved.
“32bit? no, 31bit!”
Now the first time i got the interpreter to compile for neko, i thought “Wow, great! Looks like this is going to work.” Unfortunately though, it didn’t. Instead, i got a rather odd exception when reading out of my replacement ByteArray:

Uncaught exception - Overflow

Turns out the problem was with this code here, in neko.Input:

	public function readInt32() {
	  var ch1 = readChar();
	  var ch2 = readChar();
	  var ch3 = readChar();
	  var ch4 = readChar();
	  if( (ch4 & 128) != 0 ) {
	    if( ch4 & 64 == 0 ) throw Error.Overflow;
	    return ch1 | (ch2 << 8) | (ch3 << 16) | ((ch4 & 127) << 24);
	  } else {
	    if( ch4 & 64 != 0 ) throw Error.Overflow;
	    return ch1 | (ch2 << 8) | (ch3 << 16) | (ch4 << 24);
	  }
	}

So basically the reader was trying to stuff the whole 32 bit value into a neko Integer, which is actually 31 bits. Oddly enough though, the value being read was identical to a value i specified in the code.
But haXe never bothered to complain about an overflow!
This can be demonstrated by the following code, which prints “-766623411” instead of the expected “1380860237”.
trace(1380860237);
Thankfully, a solution exists. There is an “Int32” type which properly handles 32bit integers. However there is a really big snag with it – it’s a class.
That means you cannot use any fancy inbuilt operators with it – you have to use a set of functions instead.
import neko.Int32;

var result: Int = 1 + 2; // simple
var result32: Int32 = Int32.add(Int32.ofInt(1), Int32.ofInt(2)); // aaaarrrghh!!!

result += 1; // simple
result32 += 1; // don't even think about it, it won't work
And to top it all off, this “Int32” class only exists for the neko platform, you cannot use it with flash or javascript. Unless you write it yourself, but that is besides the point. It should at least have been there to fill in the gap.
Conclusion
If you are looking at writing code in haXe and want it to work across all 3 of its supported platforms (flash*, neko and javascript), then beware if you are reading, writing, or otherwise processing 32bit integers.
In addition watch out for mysterious arbitrary limitations such as the 115 item limit for declared arrays.


Accounting with Luca
James Urquhart — Thu, 03 Jan 2008 00:00:00 GMT
A while ago now, i stumbled across a web based double entry accounting solution called Luca (not to be confused with the Mac Application).
Unfortunately though, the documentation was a bit sparse. However with a bit of persistence, i have just about managed to figure out how to use it.
Starting out
(note that for this little walkthrough i am using Luca 1.02, so don’t be surprised if you are using a later version and things look that bit different)
Assuming you have managed to download and install Luca, as well as log in, you’ll be presented with this rather interesting interface:
Whoah! Lots of options to choose from, but which of those do we need to start off with?
First things first, you’ll need to create a chart of accounts, so select the “Charts of accounts” option. You’ll be presented with a list of charts, which should be empty. Click on “Add new chart”, and your screen will look something like this:
The only 2 values which are of interest are “Description” and “Format mask”, the former of which can be anything you like. The latter is a bit more tricky, and it took me a while to figure out. It seems that the “Format mask” is basically a sequence of 9’s using dots as separators. The more dots you use, the more you can nest your accounts.
For reference, i put the following values in this form:

Description: LucaCorp
Format Mask: 9999.99 (i.e. 2 levels one using four digits, the other using two)
You should now have a chart you can use. But to be of any use, it needs to be assigned to an Organization. To do this, go back to the main menu, and select “Organizations”. Then select “Add new Organization”. Your screen should now look something like this:
Give your Organization a name, a remark if you like, and of course a chart of accounts. The values i used are as follows:

Name: LucaCorp
Remarks: blank
Chart of Accounts: LucaCorp
Note that you will also need to give yourself (and anyone else you trust with a user account) permission to use the Organization when entering transactions. This can be done in the “Users X Organizations” screen.
Creating accounts
Now would be a good idea to actually make some accounts. To do this, go back to the main menu and select “Accounts of the chart”. Then for the “Parent chart being edited:” field, select your chart of accounts. Your screen should now look something like this:
For each account you want to create, click on “Add Account” and fill in the form, which should look something like this:
The “Class” is the type of account to be created. You can make Accounts for Assets, Liabilities, Equity, Revenue, and Expenses (note that this list can be altered by playing about with the “Account classes”).
The “Std. journal” corresponds to an id of a “Standard Journal Description” which you can make in “Standard Journal Descriptions”. From what i can tell this allows you to save on typing repetitive descriptions when entering transactions, though i have not really found a use for it yet.
“Code” is the identifier of this account. It needs to correspond to the “Format Mask” you created earlier, e.g. in my case, “1234” and “1234.01” would both be acceptable.
“Parent account” is the code of the account you wish to be the parent. You can use this to create complex hierarchies of accounts, something which i assume Accountants love to do.
The rest of the fields are pretty obvious, apart from “Mobile clients” which i can only assume allows people using Luca’s bluetooth functionality to enter transactions on the account (note though that i have not tried this out).
So now you can create accounts at will. As for which accounts to create, it’s really up to you. However if you are a bit stuck, below is a table of values you can enter to reproduce a set of accounts from of one of my favourite tutorials on Double Entry Accounting written by Joseph Mack.
RocketScience LLC Accounts


Class
Code
Parent
Description
Small description


Asset
1234 
cash
cash


Asset
1234.01 
1234
Joseph Mack, withdrawing
jm_withdrawing


Asset
1234.02 
1234
check account
check_account


Asset
1234.03 
1234
petty cash
petty_cash


Liabilities
1235 
liabilities
liabilities


Liabilities
1235.01 
1235
Joseph Mac, capital
jm_capital


Equity
1236 
equity
equity


Equity
1236.01 
1236
accumulated profit and loss
acc_profit_loss


Equity
1236.02 
1236
[clo]profit and loss
profit_loss


Revenue
1237 
revenue
revenue


Revenue
1237.01 
1237
fees earned
fees_earned


Expenses
1238 
expenses
expenses


Expenses
1238.01 
1238
advertising
advertising


Expenses
1238.02 
1238
food,entertainment
food_entertainment


Expenses
1238.03 
1238
office supplies
office_supplies

Entering transactions
To enter transactions in Luca, you’ll need to first create a Batch. To do this, from the main menu select “Batches”. Then on the next screen, select your Organization and click on “Add new batch”. Your screen should look something like this:
As you might have noticed, there is not much to a batch. The only thing to look out for is the dates you enter, which need to be in the format “yyyy/mm/dd”, and need to lie within the organization’s active period (the default being 1970/01/01 -> 1970/01/01).
If you ever want to change the date range of the active period, all you need to do is select “Active Period” from the main menu, then click on “Edit” next to the relevant accounting period for your organization. Your screen should look something like this:
Then all you need to do is enter an initial and final date. It’s as simple as that!
For my first batch, i used the following:

Description: First batch!
Initial date: 1970/01/01
Final date: 1970/01/01
Checksum: 0
Status: Open
Type: Normal batch
Now you can start entering your transactions, which in Luca are referred to as Journals. To do this, from the main menu select “Journals”. Then on the next screen, select your Organization and Batch, then click on “Add new journal”. Your screen should look something like this:
So to sum this up…
“DB account” and “CR account” mean “Account to debit” and “Account to credit” respectively. I’m going to assume you know enough about double entry accounting to figure out which accounts you credit and which you debit.
“Std. journal” i already mentioned when creating accounts.
“Cost center” i have yet to figure out, best to leave this as 1 unless you really need it set.
“Date”, “Description”, “Value”, and “Status” are all pretty obvious.
You can enter as many journals as you like in a batch. It is also interesting to note that you don’t have to enter both a credit and debit for an entry – you can quite easily add multiple debit or credits in a row.
Profit and Loss
At the end of the accounting year, you’ll probably want to determine profit and loss, which usually involves folding revenue and expense accounts into a single profit and loss account. This is otherwise known as “closing out the accounts”.
So how does one do this in Luca? Quite simply in fact. Select “Balance closing” from the main menu, and your screen should look something like this:
So basically all you need to do is select your organization, enter in start and end dates, then click “Prepare to close balance”.
However you’ll probably bump into a few issues first time, as three conditions need to be satisfied:

You need an account which contains “[clo]” somewhere in its description to store the closing values.
The period you specify needs to be within the current active period.
All batches in the specified period need to be set to “Closed”
The first condition is rather easy to satisfy. All you need to do is make an equity account named something like “[clo]Profit and Loss”.
The last two conditions we more or less went over when talking about batches.
Assuming you got the balance to close, your “Profit and Loss” equity account should now look something like this:
To sum it all up
Luca is a pretty promising accounting solution. It does accounting, reports, and also includes a bit of multiuser management too. In contrast to other solutions, it doesn’t try to do everything – which i believe is one of its strengths. It’s a relief not having to wade through a swamp of CRM to do something as simple as sorting out the accounts.
Sadly though Luca doesn’t appear to have a very active community, and from what i can tell Elvis Pfützenreuter puts most of the work into it.
Still from what i have been able to gather Luca is still under development so with any luck it’ll gain more useful features in the future such as multiple currency support.


RailsCollab on Rails 2.0
James Urquhart — Sat, 08 Dec 2007 00:00:00 GMT
Soon after hearing that version 2.0 of the Ruby on Rails web development framework had been released, i decided to try out my pet project RailsCollab to see if it would work properly in the new release.
Sadly it did not, though it didn’t take too long to fix everything. To summarise:

I still used a few deprecated functions and parameters, such as link_to_url, @request, start_form_tag, and so on. Changing references to their new counterparts solved this issue.
Two plugins i use, “enum-column” and “paginating_find” broke. I had to modify the code for these as illustrated below.
A new script, “script/performance/request” was added to i naturally had to add it to my svn repository
The default session container changed to cookies, so i had to fill in “config.action_controller.session” in config/environment.rb
“config.breakpoint_server = true” had to be removed from my development environment
Mongrel decided to through a wobbler and stop working, so i had to do a reinstall of it
How to update
Initially i was a bit confused regarding the correct update procedure, however i believe the following commands should suffice:
sudo gem update rails -y

# In your app's directory
rake rails:update
enum-column fix
Since database plugins for Rails have been seperated from the core, enum-column runs into a bit of trouble as it assumes the plugins for MySQL and PostgreSQL are loaded when in most cases they are not.
In addition i couldn’t even get the PostgreSQL plugin working, though this wasn’t so much an issue as i only use the plugin to support the initial database schema from ActiveCollab.
So in “vendor/plugins/enum-column/plugins/enum-column/init.rb”, change the following:
require 'enum/mysql_adapter'
require 'enum/postgresql_adapter'
To this:
require 'enum/mysql_adapter' if (ActiveRecord::ConnectionAdapters.const_defined? :MysqlAdapter)
# require 'enum/postgresql_adapter'
paginating_find fix
This one is rather simple. In “vendor/plugins/paginating_find/lib/paginating_find.rb”, look for this line:
options = extract_options_from_args!(args)
And change it to this:
options = args.extract_options!
Initial impressions
I was very impressed to find that RailsCollab was notably faster in development mode. I suspect this is due to a combination of that fancy query caching system and of course the change to cookie-based storage as the default session storage system.
Apart from the minor hiccups i had when updating RailsCollab, i’ve not had any more issues with Rails 2.0. For my future projects, i will definitely be targeting Rails 2.0.


Double Entry Accounting in Rails
James Urquhart — Thu, 06 Dec 2007 00:00:00 GMT
The first helpful article i found, Double Entry Accounting in a Relational Database, suggests i need at least the following models in my system:

Account (has many Postings)
Asset Type (e.g. £, $, monkeys)
Batch (has many Journals, though not really needed)
Journal (has many Postings)
Posting (associates with Account, Journal, and Asset Type)
However one problem i noticed with the article was that instead of separate "credit" and "debit" fields, everything is consolidated into an "amount" field. This meant that the abstraction between debiting and crediting needed to be handled in the model rather than the actual database.
Puzzled by this, i decided to look at the database schema used in other Accounting products: LedgerSMB and jGnash.
LedgerSMB uses a rather complex database schema. I couldn't make much sense of it, apart from like the aforementioned article it consolidated credits and debits into an "amount" field rather than have them as separate fields.
jGnash was a bit different as instead of a proper database it uses an XML file to store its data. Still, there were a few distinctive elements which are the equivalent of models:

Account (linked to Transaction and CurrencyNode)
SingleEntryTransaction (linked to Account, used to insert money out of thin air)
DoubleEntryTransaction (linked to 2 Accounts 'credit' and 'debit'. Also links with CurrencyNode)
SplitTransaction (used to link multiple SplitEntryTransaction's together to indicate a split transaction)
SplitEntryTransaction (linked to SplitTransaction, otherwise the same as DoubleEntryTransaction)
CurrencyNode (e.g. £, $, giraffes)
Personally i found jGnash's XML format needlessly complicated, though much more comprehensible than LedgerSMB's database schema.
Though really, the approach i liked the best was the one mentioned in the article. It made the most sense, and wasn't needlessly complicated. So i decided to implement it.
While implementing it however, i became a bit stuck on how to abstract the credit and debit in the Posting entries.
Whenever i have looked at Double Entry Accounting in the past, i always assumed that when you take money out of one account, you need to put the same amount back into another account. Thus i made this rather mistaken test data:
# Begin asset types

gbp = AssetType.new(:name => 'GBP', :symbol => '£')
gbp.save!

# Begin accounts

assets = Account.new(:parent_account => nil,
:asset_type => gbp,
:name => 'Assets',
:account_type => :asset)
assets.save!
equipment = Account.new(:parent_account => assets,
:asset_type => gbp,
:name => 'Equipment',
:account_type => :asset)
equipment.save!

liabilities = Account.new(:parent_account => nil,
:asset_type => gbp,
:name => 'Liabilities',
:account_type => :liability)
liabilities.save!
fred = Account.new(:parent_account => liabilities,
:asset_type => gbp,
:name => 'Fred',
:account_type => :liability)
fred.save!
george = Account.new(:parent_account => liabilities,
:asset_type => gbp,
:name => 'George',
:account_type => :liability)
george.save!

# Begin transactions

first_transaction = Journal.new(:transaction_type => :transfer, :start_date => Time.now)
first_transaction.save!
Posting.new(:account => george,
:asset_type => gbp,
:journal => first_transaction,
:account_period => 0,
:description => 'Funding',
:amount => -250.00).save!
Posting.new(:account => equipment,
:asset_type => gbp,
:journal => first_transaction,
:account_period => 0,
:description => 'Funding',
:amount => +250.00).save!
first_transaction.end_date = Time.now
first_transaction.save!

second_transaction = Journal.new(:transaction_type => :transfer, :start_date => Time.now)
second_transaction.save!
Posting.new(:account => equipment,
:asset_type => gbp,
:journal => second_transaction,
:account_period => 0,
:description => 'Payback',
:amount => -100.00).save!
Posting.new(:account => george,
:asset_type => gbp,
:journal => second_transaction,
:account_period => 0,
:description => 'Payback',
:amount => +100.00).save!
second_transaction.end_date = Time.now
second_transaction.save!

third_transaction = Journal.new(:transaction_type => :transfer, :start_date => Time.now)
third_transaction.save!
Posting.new(:account => fred,
:asset_type => gbp,
:journal => third_transaction,
:account_period => 0,
:description => 'Investment',
:amount => -300.00).save!
Posting.new(:account => equipment,
:asset_type => gbp,
:journal => third_transaction,
:account_period => 0,
:description => 'Investment',
:amount => +300.00).save!
third_transaction.end_date = Time.now
third_transaction.save!
Which actually appeared to work fine (as in everything sum'd to 0, and if i ignored the minus then Assets = Liabilities) before i started to delve into what debiting and crediting means in relation to whether or not an account is classed as an Asset or a Liability.
(Note that to simplify things when i say Asset i could alternatively be meaning Expenses, and when i say Liability i could alternatively be meaning Shareholder Equity or Revenue)


Account
Debit
Credit


Assets
+
-


Expenses
+
(-)


Liabilities
-
+


Shareholder Equity
-
+


Revenue
(-)
+

After applying the logic from the above table (courtesy of Wikipedia), i noticed something rather odd. The credit and debit columns on my printout didn't match. For reference, i was using the following queries to grab the credit and debit amounts for each account.:
credit_amount = Posting.sum(:amount, :conditions => "amount #{account_type == :asset ? '<' : '>'} 0")
debit_amount = Posting.sum(:amount, :conditions => "amount #{account_type == :asset ? '>' : '<'} 0")
However if i modified the code slightly:
credit_amount = Posting.sum(:amount, :conditions => "amount < 0")
debit_amount = Posting.sum(:amount, :conditions => "amount > 0")
The credit and debit columns mysteriously matched. So what happened?
Well i puzzled over this for a while, until i read Wikipedia's article on Double Entry Accounting again. This rather simple summary was very insightful:
Purchase of a Computer

Debit Computer A/c (Fixed Asset A/c)
Credit Creditors A/c (Liability A/c)
(A/c being an abbreviation of "account current")
Paying supplier for the computer

Debit Creditors A/c (Liability A/c) You are reducing a Liability A/c
Credit Bank A/c (Asset A/c) Money going Out, an asset account is being reduced
So in fact all the transactions in my test data should have been increasing the amount in the liability accounts rather than decreasing it (and vice versa when paying back).
It makes sense now i think about it. The second approach worked because in effect i flipped the meaning of debit and credit by decreasing the amount in the liability accounts and calling it credit rather than increasing the amount in the liability accounts and calling it credit (and vice versa).
Now i could just go ahead and fix my test data. But then my amounts would cease to sum up to zero, and i would have to run multiple queries (each for debit and credit) to ensure integrity of the data.
Thus what i really need to do is make an attribute in the Posting model which stores and retrieves the correct "amount" depending on what type of account it is. Something like this should suffice:
def real_amount
  return self.account.account_type == :asset ? self.amount : -self.amount
end

def real_amount=(val)
  self.amount = self.account.account_type == :asset ? val : -val
end
So to sum it up, the only thing i really have to do with liability accounts to satisfy accounting rules is to subtract when i really mean add, and then flip the balance when calculating it. Confusion eliminated!
Now lets hope i don't get confused again when i start to think "Did i get this right?".
Currently i am looking into how i can effectively incorporate business concepts such as Invoices, Taxation, and so on into my little Double Entry Accounting system. So expect to see a "Double Entry Accounting with Invoicing in Rails" post in the near future.


Flash Plays SCUMM, Take Two
James Urquhart — Thu, 22 Nov 2007 00:00:00 GMT
Last month after bumping into haXe and Doomed Online, i somehow got the strange idea that i could get flash to play SCUMM games, like ScummVM.
While i did get it working as a proof of concept, there was a really off-putting bug in the image decode routines which meant that any displayed room graphics were a jumbled mess.
Recently though i decided to do a bit of intensive debugging to try and solve the problem. To sum it up, after:

Turning on the trace output in the Flash debug player
Telling scvm and hiscumm to dump crucial info their trace logs
Comparing the output of scvm and hiscumm using diff to see where the differences were
Modifying the trace output to pinpoint the problem while dealing with script timeout errors.
I managed to fix the decode routines. Now instead of a jumbled mess like this:
You get something more like this:
Which safe to say is much nicer to look at. It’s also as far as i am willing to go with this SCUMM implementation.
With a lot of work put into it, i’m sure it could run Day of the Tentacle or Sam N Max Hit the Road. But there would not be much sense in doing that, except for that 5 seconds of awe after seeing it running in a web browser.
As previously, you can download the hiscumm code for reference here.


RailsCollab Prodding
James Urquhart — Sun, 18 Nov 2007 00:00:00 GMT
In my quest to learn the Ruby on Rails (a web development framework), i re-writ a PHP-based project management app (ActiveCollab) in it. In fact, i wrote about all of this a few months ago.
Now i’ve had my ups and downs with RailsCollab. The biggest down being that there is little interest in it, especially compared to alternate solutions such as ProjectPier. Which is unfortunate.
In any case, a little problem i noticed recently was that RailsCollab still wasn’t really finished. There were still numerous bugs and missing features which made it feel more of a hack than a real product.
So i set aside some of my spare time to fix it. After a few days, i managed to get it stable enough so that it worked properly in the latest version of Rails. I also got the mail notification working, which i felt was a very important feature of ActiveCollab.
But there was still a lingering problem: Apart from the OpenID support, RailsCollab didn’t really offer anything more than ActiveCollab or ProjectPier offered.
So i thought for a while: What feature could i implement that wasn’t present in either solutions?
And then it hit me: an API! Though not just any API, no. Rather, the  BaseCamp API .
For those of you that don’t know, BaseCamp was the inspiration behind ActiveCollab (and thus, RailsCollab). From an API standpoint, it pretty much has the same features, which was great for me as it mapped quite nicely to the BaseCamp API. There were only a few features missing, such as the Message Categories, but those didn’t take too long to implement.
So i pretty much ended up with a nice and shiny implementation of the Basecamp API (minus two minor functions which i have yet to implement). Which meant that i could use all of those fancy third party Basecamp tools widgets with RailsCollab as well.
However there was a snag. While RailsCollab worked fine in development mode, it failed to work properly in production mode due to some problem with controller helpers – which as you can imagine was highly frustrating.
Ruby on Rails might be a great web development framework for getting things up and running quickly, but when you start having such simple problems that cannot easily be debugged, and to top it all off it’s inherently slow, then it starts to look more like the web development framework from hell.
Rants aside, i somewhat hope the lure of the Basecamp API implementation will generate more interest in RailsCollab. Though even if it doesn’t, i will still have the satisfaction of knowing i put some effort into polishing it off and turning it into a real product.


Linux on the desktop? Was i crazy?!
James Urquhart — Mon, 29 Oct 2007 00:00:00 GMT
For those of you have read my blog previously, you may have noticed i bumped in a few linux related posts here and there. This was mainly due to the fact that i was experimenting with using linux as a desktop operating system (something which i have tried in the past), as i wasn’t too confident of where Mac OS X was going. This ultimately included installing it via BootCamp on my Macbook.
Unfortunately i have now reached the end of my teather with regards to using Linux as a Desktop operating system. Quite simply i kept bumping into too many issues which required manual workarounds, if there was even a solution at all.
-more->
I started off by trying one of the release candidates for the latest release of Ubuntu, version 7.10. This initially impressed me as when i booted off the installation CD, almost everything appeared to work out-of-the-box. However upon installing it on my Macbook, the novelty wore off as i realised that crucial features of the Macbook – the camera, the IR, even the trackpad either didn’t work properly, or didn’t work at all.
For example:

The default trackpad configuration was abysmal, and i couldn’t find any configuration tool which gave me the same amount of customizability as the one in Mac OS X in order to fix it.
I never got the iSight working – though then again this wasn’t much of a problem as Skype for Linux doesn’t even support video cameras.
Whilst the sound worked, configuring it was a pain in the ass as there were too many confusing options to choose from.
I tried to get the DVI out to work. I ended up breaking the X server and spent ages trying to fix it. Safe to say i never got it to work.
OpenGL support on the Macbook was sub-par.
Intergration between applications varied from being good to abysmal.
( It should be stressed that all of these issues persisted through to the stable release of Ubuntu 7.10 )
In fact, you can see a nice list of all the problems you have to fix out of the box here.
A week or two ago, Ubuntu 7.10 became officially stable, so i decided to try out one of its variants to see if it fared any better on my Macbook. I chose Kubuntu as i had previous experience with using KDE from using SuSE Linux and figured it might turn out to be a better desktop experience.
On the contrary, Kubuntu was anything but a better desktop experience when compared to Ubuntu. A lot of the killer features present in Ubuntu – Compiz Fusion, simple administration tools, even mappings for the brightness controls were not present in Kubuntu. It was also more difficult to configure as there were simply too many configuration options to choose from, as is illustrated by the comparison of the “Keyboard Shortcuts” dialog in Kubuntu (which uses KDE) and Ubuntu (which uses Gnome) bellow:
I even tried the beta KDE4 packages, in the hope that maybe KDE4 had improved on KDE3. I was sorely dissapointed, as instead of this i got this.
Safe to say i have come to the conclusion that currently neither Ubuntu or Kubuntu are quite up to scratch when compared to the standards set by Mac OS X. In fact if suddenly all copies of Mac OS X were obliterated, i’d probably take a risk and use Haiku.
As for the future, who knows? But as for now, i think i’ll stick to what i like best – Mac OS X.


Flash Plays SCUMM (sort of)
James Urquhart — Sat, 06 Oct 2007 00:00:00 BST
Soon after bumping into haXe and Doomed Online, i somehow got the strange idea that i could get flash to play SCUMM games, like ScummVM.
Safe to say, i ended up with this rather odd Flash-based app written in haXe which when provided with the ”.000” and ”.001” files of any unencrypted V6 SCUMM game will be able to load and play it. Or rather, attempt to.
Sadly i bumped into a problem in the image decode routines, so what is displayed isn’t pretty. But it is quite capable of executing enough opcode’s to get to the input event loop in Alban Bedel’s road example for his scummc compiler, which is great.
As for other games, i would doubt that they would get very far. A lot still needs to be implemented in the runtime.
In any case, if you dare you can try out my Flash-based SCUMM interpreter by downloading and extracting this archive. Simply load “test.html” into your web browser. Note though that you will need Flash 9 in order to run the interpreter.
In addition i also decided to open up the code for everyone to look at. It’s a mess, but it works. You can grab it from here.
Finally i would like to congratulate the ScummVM team for their SCUMM interpreter and Alban Bedel for his scummc compiler. Without their efforts, i wouldn’t have had any code or references from which to base my haXe-based SCUMM interpreter.


HaXe - Web oriented universal language!
James Urquhart — Thu, 04 Oct 2007 00:00:00 BST
A while after i bumped into Doomed Online, i decided to investigate the pipeline for making Flash 9 games.
Compared to a few years ago, it seems like an increasingly lucrative game development platform. Pretty much everything you would need to implement for a game is possible with Flash 9, including such things as 3D Rendering. In a way it’s comparable to the late-90’s PC gaming scene. I have no doubt that you could probably make something as sophisticated as, say, Tomb Raider with Flash 9.
However my big problem with Flash is that it is still a closed platform. I have yet to find any open source re-implementation of the Flash runtime that fully runs Flash 7 files, let alone Flash 9. And whilst the Flash 9 compiler is supposedly going to be open source’d by Adobe, i have yet to see any indication that this is really going to happen.

Thankfully, i found a solution : HaXe. It is essentially a fully featured programming language that can output to one of three formats: 

Adobe Flash (7, 8, or 9)
Neko VM (runs as a standalone app or a web app or perhaps embedded elsewhere)
JavaScript (though sadly doesn’t have a nice RIA toolkit which writes all the HTML like, say, OpenLaszlo)

So in essence, with HaXe i can bypass my “closed platform” nag point and instead make a version of my game or app that works in Neko or perhaps just in a bog standard Web Browser. In addition i can consolidate my web development into one cool programming language. Great!


Doomed Online
James Urquhart — Tue, 18 Sep 2007 00:00:00 BST
Just as i thought i had found some of the most freakiest flash apps, i found this rather playable re-implementation of doom called Doomed Online.
What is amazing about it is that it uses the original data files – something which i never considered would have been possible with flash. But then again i’ve not quite been keeping up with all of the technical advances in flash over the years, so it is really no surprise.
The demo is available here. For those of you who might have checked out the pre-release versions of Doom, the demo plays very much like them – as in you can’t shoot anything and there is no AI. Hopefully in the future these issues will be resolved and it’ll be that much more interesting to play.


Can you solve this river crossing puzzle?
James Urquhart — Sat, 08 Sep 2007 00:00:00 BST
After stumbling across the Entrepreneur Geek blog, i spotted this rather puzzling river crossing puzzle. Sadly, i haven’t managed to solve it yet, but i think i’m close.
Instructions:
	Help these people to cross the river, in the next screen.
	Only two people at a time, at least one adult.
	Criminal (in striped dress) cannot be left with others, without the Police.
	Mother cannot be with sons, while father is not around.
	Father cannot be with daughters, while mother is not around.
	Click on the BIG BLUE BUTTON to start!
	Click on people to get them on or off the boat.
	Click on the Red buttons to move the boat to opposite side
 


ScummC - Make your own SCUMM Adventure Game!
James Urquhart — Sat, 25 Aug 2007 00:00:00 BST
Back in the good ol’ days of DOS, i used to (and still do) like playing adventure games. Specifically, i thought those made by Lucasarts were the best, as they often had simple yet funny plots, and you could pretty much get around them easily. They all had (more or less) consistent interfaces – from Maniac Mansion to The Secret of Money Island, you could tell it was a Lucasart’s adventure game and that it was going to essentially work the same as the last Lucasart’s adventure game you played.
 
-more->
The consistency in part came from the game engine which they all shared in common: SCUMM (Script Creation Utility for Maniac Mansion).  Each game implemented its own set of scripts and resources which could be run in any version of SCUMM, provided that they were targeted for the version(s) of SCUMM that particular runtime of the engine supported. In a way, it was like the equivalent of Java for adventure games.
Anyhow, part of the reason i got interested in software development in the first place was that i always wondered how these games were made. I thought it was great how it all fitted together – the characters, the animation, the scripting, the lot. Eventually i found various projects on the ‘net which were aiming to create development system’s for other adventure game engines (such as those made by Sierra), but nobody seemed interested in SCUMM.
Thankfully in 2004 Alban Bedel made his first release of ScummC – a tool to compile SCUMM scripts (using a C-like language) and assets, and link them together so you could play them in any SCUMM runtime that supported version 6 scripts. Although it wasn’t until late last year that i was confident to download it and try it out. Still, it brought great a great new insight into how these SCUMM games were made.
By the time i tried ScummC out, Alban had already made a standalone demo SCUMM game which consisted of a single room and a player character. Surprisingly, it was all presented in a top-down view, which wasn’t quite what i expected from a SCUMM adventure. Still, i played around with it and was greatly impressed.
When i digged down into the demo code though, i was amazed at how much of the interface was powered by script. The action’s, the inventory, the hover text – you name it, it was powered by script. Considering almost every Lucasart’s adventure game i played had that same look and feel, i was really thinking a lot of it was going to be explicitly supported in the engine.
In any case, soon after i showed interest in the ScummC project, Alban updated his standalone demo with a second room. It was interesting to note that whilst it was incredibly easy to add room’s to script, it was difficult to create the graphics for the rooms as you had to leave space in the room palette for any actor’s which might be present in the scene. Nowadays you rarely see modern image editors with support for tailoring palettes in this way, so i was compelled to modify the palcat tool included with ScummC to support pre-appending palettes to images (which for some reason i forget, i needed).
Another thing which i tasked myself with was how to make a conversation. For this, i had to grab a copy of ScummVM and use the debugger to see which SCUMM functions were being called during a typical conversation in Day of the Tentacle. In the end, i concluded that once a conversation is entered, the following things happen:

saveVerbs() is called to hide all the interface elements (normally known as “verbs”)
All dialogue items are initialised
setVerbOff() is called on all dialogue items so they don’t appear
Exit conditions for the dialogue are checked
Each option of dialogue is loaded to a corresponding dialog item (setVerbName()) and turned on (setVerbOn())
The conversation script waits till a dialogue option is selected. If it is, the relevant action corresponding to the dialogue gets executed. If the conversation hasn’t ended, it loops back to 3. Otherwise it goes on to 7.
All dialogue items are hidden / deleted
restoreVerbs() is called to restore all the interface elements
Which safe to say wasn’t quite what i expected, which was some specific support in the engine for conversations. Which goes to show that SCUMM can be quite flexible.
Sadly there doesn’t seem to be a lot of people using ScummC. Though then again, its much more programmer oriented as there is no fancy graphical IDE to build games as there is in other adventure game engine’s such as Adventure Game Studio. Its more of a DIY toolset, which doesn’t quite work well in this age of easy to use graphical interfaces. I can’t quite see the typical user of one of those interfaces having to blindly type in the following (taken from the ScummC example) in order to define a room and some resources.
#include <scummVars6.s>
#include "common.sch" 

bit welcomed,lost,santaHello,santaPlace,santaReal;
actor santa;
#define SANTA_COLOR 14

// our room
room Road {
// first we define the room parameters
// the background picture
image = "road.bmp";
// the zplanes
zplane = { "road_mask1.bmp", "road_mask2.bmp" };

boxd = "road.box";
trans = 0;

//cycle testCycl = { 60, 0, 103, 111 };
cost santaCost = "santa.cost";

voice letsgothere = { "letsgothere.voc", 500 };
voice welcome = { "welcome.voc" };
voice tv = { "file.voc" };

object axe;
...
Still, i think it would be interesting to see where ScummC goes, as there is still a lot to be done with it. I would encourage anyone with some free time and who feels comfortable using lots of commandline tools to check it out.


Number Systems & Bases
James Urquhart — Thu, 16 Aug 2007 00:00:00 BST
A while back, i writ a program in which one could convert any number from any base to and from base 10. So for example, if i had the number “10” in base 3, it would churn out “3” in base 10, and vice versa.
Initially all i got it converting was base 62. However i was not just satisfied with base 62, i wanted it to convert any possible base, defined by any set of symbols. To top it all off, i also made it possible to use unicode symbols via UTF-8.
So for example if i could conjure up the following hideous system:


Symbol
A
B
C


Value
0
1
2

...and start saying thing’s like “C + C = AB”.
For the final touch, i added the ability to specify numbers with numerals. Although i didn’t quite get the conversion algorithm from base10 to numerals right.
Safe to say, i ended up with this rather hideous looking app programmed in C. But on the plus side i could:

Feed in a file containing a sequential list of UTF-8 formatted symbols to make my own number base system.
Feed in a file containing a list of “SYMBOL=VALUE” pair’s ordered by value to make my own numeral system.
Convert the output of either the previous examples back to base10.
In any case, one can grab the code here. It’s not pretty, nor is it quite the next killer app – but i had some fun writing it.


RailsCollab Demo
James Urquhart — Sun, 12 Aug 2007 00:00:00 BST
A while back Gary Vaughan suggested i host a demo of RailsCollab on my server. Sadly though, i didn’t have any quick & easy way of adding new demo users to the system, that is until  i cooked up OpenID support.
Consequently, there is now an OpenID enabled service running on http://brew.cuppadev.co.uk. Provided you have an OpenID, you should be able to select the “Use OpenID” checkbox and type in your OpenID to login. This will automagically register you with the system, adding you to the “OpenID” company and the “RailsCollab” project. Afterwards if you want, you can set your password so you can login using a regular username & password, though personally i think typing in your OpenID is a better idea.
Note that your OpenID provider needs to support handing over identity fields in order for RailsCollab to correctly register you. MyOpenID is an example of a provider that supports this. Also note that if there is a user with a duplicate username or email in the system, registration will fail.
I invite anyone to try RailsCollab out.
UPDATE: The demo has since been taken down. If you wish to try out RailsCollab, you will need to download and install it yourself.


OpenID in RailsCollab
James Urquhart — Sat, 11 Aug 2007 00:00:00 BST
As mentioned in my Cloning ActiveCollab post, i managed to make a reasonably good clone of ActiveCollab using the Ruby on Rails framework.
Recently after watching the rather interesting Google TechTalk on The implications of OpenID, i decided to see if i could add OpenID support to RailsCollab.
OpenID is a pretty nifty decentralised single sign-on system. The process of verifying a user’s identity is essentially outsourced to an OpenID identity provider, instead of being left to web developer’s to implement it themselves (and consequently doing a very poor job of it). As a user, i host my identity on any of the numerous OpenID identity providers, preferably one which i can trust. Or if i’m really paranoid, i can just make my own OpenID identity provider.
In relation to RailsCollab, it makes sense to have support for OpenID as a key feature is the ability for client’s to log-on to review progress on a project.
Without OpenID, you’d have to give each of your client’s a username and password, the latter of which you can almost guarantee that someone will forget. And then you need to muck about with setting up an email server so you can send out “Password reset” email’s, unless you want to do it all yourself each time you need to reset it.
With OpenID, you’d just have to give each of your client’s a username and an associated OpenID. There’s no need to much about with password’s, as you’ve outsourced that burden to the OpenID provider. All they need to do to login is type in their OpenID, and their OpenID provider will handle the sign-on process.
For anyone crazy enough to actually try RailsCollab with OpenID, you can grab a copy from the Subversion repository on RailsForge. Details are available on the project page.


OpenLaszlo XML Workaround
James Urquhart — Thu, 09 Aug 2007 00:00:00 BST
As mentioned in my previous OpenLaszlo post, i had a problem whereby in SOLO mode my app would send the XML data to the server in urlencode format. This meant that my poor REST-based web service barfed up as this was in no way valid XML. Safe to say, i wasn’t best pleased.
Thankfully though, considering i am interfacing with my own web service, i can change how the server interacts with the request data all i want. So it’s pretty easy to work around this issue.
In the case of the Ruby on Rails framework i am using, the easiest way is to make a new mimetype handler which handles our standards hostile custom mimetype (which i promptly called “application/xml-urlencode”). Like so:
Mime::Type.register "application/xml-urlencode", :xml

ActionController::Base.param_parsers[Mime::Type.lookup(
'application/xml-urlencode')] = Proc.new do
|data|
decoded_data =  CGI::unescape(data) # Decode data
XmlSimple.xml_in(data, {'keeproot' => true,
'forcearray' => false}) # Parse decoded XML
end
And then all i have to do in my OpenLaszlo app is something along the lines of:
datasetForSending.setHeader("Content-Type", "application/xml-urlencode");
And hey presto, the data is now recognised!


OpenLaszlo Version of the Previous Flex Example
James Urquhart — Wed, 08 Aug 2007 00:00:00 BST
As hinted in my Running into Flex Limitations post, i finally managed to make a version of the rather nifty REST-based project object editor.
After being told about the magical “lzpostbody” parameter by Henry Minsky, i was able to send my XML data in the content of my POST request (which causes the Rails app to create Project objects).
Sadly i needed to make a little concession on the “PUT” method (which would cause the Rails app to modify the specified object) as i couldn’t manage to find a way of passing in the “_method=POST” parameter in from the request as i could with Flex, like so:

	private function sendProjectUpdate():void
	{
	  var projectUpdate:Object = new Object();

	  projectUpdate['name'] = pname.text;
	  projectUpdate['description'] = description.text;

	  updateProject.url = "http://localhost:3000/projects/" +
	    projs_dg.selectedItem.id + ".xml?_method=PUT";
	  updateProject.send({project:projectUpdate});
	}

Doing a similar thing in OpenLaszlo didn’t work (my Rails app doesn’t seem to pick up the overriding _method). I can only assume that Flex is using some extra hocus pocus behind the scenes to make my app happy enough to go ahead. Maybe i’ll have to dissect the HTTP requests to get to the bottom of it.
To replace the requirement of “PUT”, i just added in another route in my Rails app that goes straight to the update method on the projects controller, like so:
map.connect 'test/update/:id', :controller => 'projects', :action => 'update'
map.connect 'test/update/:id.:format', :controller => 'projects', :action => 'update'
In any case, i was happy enough that i could get an equivalent version of my Flex app working in OpenLaszlo. Whilst it doesn’t have all the glitz and glamour of Flex’s components, it mostly still works the same, with the exception of the extra “Refresh” button which i had to add as i couldn’t seem to reliably determine when my requests to the server ended, which meant that the projects list doesn’t always automagically update to reflect the result on the server.
For reference, the code is located here.
UPDATE: Whilst this code works fine in the default PROXY mode, it seems that in SOLO mode the XML is escaped (e.g. ”New Project (TEST)!!!!<”) when sending data, which more or less sucks.


Running into OpenLaszlo Limitations
James Urquhart — Sun, 05 Aug 2007 00:00:00 BST
One of the things that got me interested in OpenLaszlo was its pretty cool databinding features, which meant i could take in data from any standard XML document (be it a web page, xmlrpc, or even SOAP) and prototype a little interface around it very quickly, as i previously mentioned in my previous posts.
However, today i decided to take it a step further. I wanted to provide an interface to edit the data i obtain from a request and send back the results to the server. I also wanted to make use of the Ruby on Rails framework’s support for REST.
Being the naive person i am, i thought it was going to be a walk in the park. In OpenLaszlo, all you need to do to grab and display your data from the server is:

Make a dataset and point it to your data (e.g. “http://localhost:3000/projects/1.xml”)
Make a control with “datapath” set to an appropriate location in your dataset in XPath format (e.g. “dset:/project”)
Call “doRequest” on your dataset

The call to “doRequest” can be done like so:

dset.setQueryType('GET');
dset.doRequest();

So naturally, i thought in order to POST the data from the dataset back to the server, i could do: 

dset.setQueryType('POST');
dset.doRequest();


Which of course was completely wrong. I assumed that the API would work both ways (i.e. sending the data when the querytype is ‘POST’), which of course it didn’t. The only slightly helpful thing i could do was set the query string. However, this wasn’t quite compatible with REST, so i couldn’t use it as a solution.
After searching about a bit though, i came up with another solution:
var dp = new LzDatapointer();
dp.setPointer(parent.datapath);

var xmlHttpRequest = new XMLHttpRequest();
xmlHttpRequest.open('POST', url, true);
xmlHttpRequest.send(dp.serialize());
“Great!” i thought, “This should work!”. Sadly though, it didn’t. It seems that OpenLaszlo’s XMLHttpRequest doesn’t send any data in a POST. Great.
A short while later, i realised i should be using a ‘PUT’ request for updating my REST-ful object. The final nail hit the coffin when i got a nice error message from XHttpRequest – “method ‘PUT’ not supported, use GET or POST”. Aaaagh!!
Safe to say, OpenLaszlo’s data binding API is pretty one-sided when it comes to data. Grabbing data? fine. Sending data? welcome to hell. And not even XMLHttpRequest is of any use, unless you want to send a long long query string, or just stuff everything in the headers, both options which contradict REST.
In the future i think i’m going to take a look at Adobe’s Flex. I suspect it might handle REST requests a bit better.


Testing a little Haiku
James Urquhart — Mon, 23 Jul 2007 00:00:00 BST
For a while now, i have been following the progress of the open source Haiku project, which aims to re-create BeOS 5. BeOs, for those of you that might not know, was a rather cool operating system developed during the late 90’s.
 
What amazed me about BeOS is that unlike many operating systems around at the time (and arguably, those around today) it: 
Booted up and shut down in mere seconds (no suspend or hibernate trickery needed)
Wasn’t bloated to hell (sadly a trend nowadays)
Did multi-tasking right (e.g. didn’t start choking when one little app decided to do something a bit intensive)
Supported live file searches (before the advent of spotlight & co)


Although of course, it wasn’t all perfect. It did lack a few things, like: 
Multiple user support (nowadays the norm)
Good hardware support (e.g. for stuff like 3d cards and network adapters)
Great third party support (apart from what you can find on BeBits)

Regardless, in 2002 Be Inc ceased operations and thus BeOS was more or less dead. That was until the controversial Zeta developed by Magnussoft came along. Controversial as it was often claimed that Zeta was developed without the permission of whoever owned the rights to the source code of Be OS. Safe to say, the nail hit the coffin earlier this year.
Luckily, Haiku is actually coming along quite nicely nowadays. After trying out the latest test image in qemu, i found it to be almost usable, with the exception of: 
The networking which i couldn’t seem to get working properly (although the DNS did work, so its probably a qemu issue)
The system started to hang a bit when running one of the demo apps (although on the plus side i could still move the windows around freely)
The sound, which i guess doesn’t work due to lack of appropriate drivers.

Still, considering it was running in qemu with no kernel acceleration module, i thought it was quite responsive. In the future, i hope that the Haiku team will improve stability and hardware support so that developers will seriously consider targeting the Haiku / BeOS platform. Hell, maybe even i’ll try using it more.
Ultimately, i hope that software developers in general stop following the trend of more bloat, slower functionality. Can’t we all just follow examples such as Haiku’s and make something that is simpler and faster without all of the bloat?


SSH Tunnel Manager
James Urquhart — Wed, 18 Jul 2007 00:00:00 BST
After reading Gary Vaughan’s recent blog post on MacBook Installation Applications 101, i decided to check out one of the tools he mentioned called SSH Tunnel Manager.
SSH Tunnel manager is a rather nifty tool which allows you to manage a list of SSH tunnels, which basically securely forwards ports on your own machine to a remote one.
So for example, if i had an administration control panel running on my web server, i might want to make it so that i could only connect to it locally (so there would be practically no chance of anyone on the internet accessing it).
But i would still want to access it remotely, and the only way for me to do that is make an SSH tunnel which acts as if i am connecting locally on the machine, when in reality i am not.
Any would-be hacker would need to figure out the login details on my SSH server in order to have a chance of accessing the control panel.
Now whilst this app was great, i noticed something a bit disturbing about it. It turns out that its been quite a while since it was last maintained, and thus it it still a PPC binary, meaning if i run it on my brand new shiny Intel mac, i have to let Rosetta run, which is memory hungry and gobbles up precious CPU time!
Luckily, the developer also provided the source code to the application. So the solution was obvious: i needed to recompile this application as a Universal Binary, which means it will run natively on both PPC and Intel mac’s.
This didn’t end up being very hard, i.e.: 
Download and unpack the source code
Open the project file in XCode (converting it to the new format)
Replace the ssh executable in the resources with the version from /usr/bin
Set the configuration to “Deployment” and build
Run it and hope it works

Thankfully for me, it built properly (with the exception of a few warnings, though they didn’t look too serious). So now i have a nice and shiny native version of the SSH Tunnel Manager on my mac.
In case anyone doesn’t want to go through the 5 step solution, or perhaps you don’t have XCode installed on your machine, here is a copy for you to download.


Experiments with OpenLaszlo
James Urquhart — Wed, 11 Jul 2007 00:00:00 BST
Recently i decided to try out OpenLaszlo, the RIA platform from Laszlo Systems. Previously i have tried, but failed to get to grips with it. But the iPhone demonstration on the OpenLaszlo blog spurred my interest again.
OpenLaszlo is a bit like Adobe’s Flex, except that instead of just Flash you can output content for multiple runtimes, such as DHTML or J2ME. This is very interesting as it opens the doors for creating functional web-based applications which can be run on a multitude of devices.
In a way, OpenLaszlo is the perfect solution for creating self-contained Flash or DHTML internet widgets, as a lot of the framework emphasizes the use of AJAX style requests to dynamically update information on the page rather than taking the aging “dumb terminal” approach where you reload the whole page.
Enough of the theory, i decided to try and make myself a little self-contained OpenLaszlo application. Its task was to get the latest updates off of Twitter, a popular web-based chat system.
Starting off
OpenLaszlo comes with a nicely designed presentation server which run’s off Tomcat. On this presentation server, you can develop and even deploy your applications. It is supposed to act as an intermediary between your application and any data it requests, which comes in handy in the case of Flash as it transcodes data such as images into a native format Flash can easily understand.
Thankfully, if you don’t fancy running a bloated Java application server just to deploy your OpenLaszlo application, you can choose to deploy it “SOLO” which means it run’s self-contained. Although of course you loose all the neat caching and transcoding that the presentation server can provide. But looking at the bigger picture, its not that much of a loss.
Being one who didn’t want to do things the easy way, i developed much of my Twitter application in the demonstration explorer (not the best environment it has to be said, although later on when i got to the deployment stage i moved the “lzx” code onto the actual presentation server).
Data binding & Replication
The first thing i did was to determine how i could get the recent updates feed from Twitter loaded into OpenLaszlo. Thankfully, OpenLaszlo provides a pretty powerful data binding system. You can take in XML from any source and create a rather feature full front end in minutes – provided of course you have read the documentation and know how to get from A to B.
Quite simply, in order to just view the data from Twitter, all i had to do was the following:



As you can see, i have a “dataset” which points to the xml feed for the recent activity on Twitter. This is referenced via the datapath in the view. There are also a few elements in the view which also reference XML elements in the dataset, relative to the datapath of the view.
Whenever the dataset is loaded, OpenLaszlo automatically replicates the view to show every “status” element in the dataset. It should also be noted that OpenLaszlo also has the common sense to delete the replicated view’s whenever you decide to reload the dataset (via calling “dset.doRequest();”).
You might also notice the “simplelayout” object which i have placed in both the root “canvas” and the replicated view. The purpose of this object is to automatically position elements in the element it is located – which is nice as it is one less thing for me to think about!
Polishing everything up
Now i could see the data from Twitter, i needed to polish everything up. Importantly, i needed a button to invoke the request on the dataset, and i needed a scrollbar to scroll the list of statuses which were replicated.
The button was easy. I decided to add a header at the top which contains the button and also some text which tells the user if something is happening. As for the loading text, it uses OpenLaszlo’s useful constraints feature so it always appears in the right place relative to the width of the page (in this case, right aligned).
The scrollbar was the hard part. I ended up having to nest the replicated views in another view in order to get the scrollbar to appear correctly (as otherwise the simplelayout would position the scrollbar below the list of statuses). I also had to play about with the width’s of the views and turn on the clipping so that when everything scrolled, the text didn’t draw over everything else.
The end result
After much frustration, here is the end result!
I also decided to add in a little fade-in animation when the images load, which makes everything look that bit cooler.
For reference, here is the code i ended up with. Feel free to copy and experiment with it!






The Web on a PocketPC
James Urquhart — Tue, 10 Jul 2007 00:00:00 BST
Unless you have been hiding in the dark these past few weeks, you’ll have heard about Apple’s new product, the iPhone. The which i think is most most striking about this new mobile phone is not the cool multi touch interface, or the springy interface, or even its minimalistic look. Rather, it has a web browser that actually works properly!
Sadly, whilst Apple may have seen the light and put the time and effort into making a good web browsing experience, the same cannot be said for a lot of other mobile device manufacturers.
Just to show you how bad things can get, i decided to try out a few web browsers on my PocketPC device, a Dell Axim X50V. For those of you who don’t know, it is one of Dell’s last PocketPC devices which uses Microsoft’s “Windows Mobile” (otherwise known as Windows CE). It has also got a nice VGA screen, giving 640×480 pixels of real screen estate.

The browsers i decided to try out are as follows: 
Pocket Internet Explorer
Opera Mobile
MiniMo


In each instance, i tried 3 of my favorite websites, i.e.: 
YouTube
BBC News
TechCrunch

In addition, to make things a bit more interesting i only used the default portrait mode on my device, meaning the resolution was really 480×640 pixels.
Pocket Internet Explorer
For reference, i used the version bundled with Windows Mobile 5.
This browser comes bundled with Windows Mobile. Like its desktop counterpart, it struggles to correctly render a lot of the pages i visited. It also didn’t help when all the images and whatnot were drawn twice the size.
Although on the plus side, it felt much more responsive than the other browsers i tried!
There were a total of three display modes available, i.e.:
Desktop
Default
One column
Another interesting feature of Pocket Internet Explorer is that if you download a plugin from Adobe, you can enable support for Flash movies in the browser. Unfortunately it only supports up to version 7, and even then you will be lucky if you can get it to work properly on your typical run-of-the-mill Flash game.
One major show stopper issue i did find with Pocket Internet explorer is that the JavaScript support appeared to be substandard at best. For example when i tried YouTube, instead of the videos all i got was a JavaScript related error message.
Opera Mobile
For reference, i used the 8.65 beta.
I first tried Opera Mobile last year when they released a public beta. In all, i didn’t have any major issues with responsiveness. It also rendered a lot of the pages i visited correctly, and even had a “Fit to Screen” option where i could view the page scaled down width-wise so it fitted onto my screen – great!
Sadly this was the only browser that made a good attempt at displaying each web site correctly, although in Portrait mode everything tends to get a bit squished up.
With regard to display modes, there were three: 
Desktop
Fit to screen
Single

Like Pocket Internet Explorer, it can utilize the Adobe Flash plugin for view Flash 7 content. Combined with the superior JavaScript support, you can actually use YouTube. However, the video’s slow everything down a lot, for instance you can tell in the “fit to screen” desktop that it spent so much time playing the embedded flash video’s that it didn’t want to load the images.
Unlike Pocket Internet Explorer, it supported tabbed browsing, which pretty much made sense on a PocketPC.
MiniMo
For reference, i used the 0.2 beta.
Essentially, this is Firefox for your Pocket PC. Which considering the hardware constraints, does not bode well. To put a long story short, this seemed the least responsive of the browsers i tried. I also encountered a lot of bugs, such as when i tried typing into text fields using the on-screen keyboard, i sometimes could not get rid of it. There was also the occasional crash, right when i least expected it.
Oddly enough i could not find any way of explicitly changing the display mode, so i was stuck with the default.
Like Opera Mobile, Minimo supported tabbed browsing. Although i couldn’t help thinking that the size of the tabs was a bit on the small side (compared to Opera Mobile).
Unlike Pocket Internet Explorer and Opera Mobile, Minimo could not make use of the Adobe Flash plugin. Considering the relative disappointment with compatibility with Flash content though, its not much of a loss. Still, it would have been nice.
To conclude
In general, browsing the web on a PocketPC device is a major disappointment. The only browser which stood out was Opera Mobile, which is great if you are willing to shelve out the money to pay for it. Arguably though, a decent web browser should have already been included in Windows Mobile.
Perhaps in the future web browsers on your typical PDA’s and Mobile Smartphone’s will get better. Until then, one can only hope.


Cloning ActiveCollab
James Urquhart — Fri, 22 Jun 2007 00:00:00 BST
Just over 3 months ago, the developer of a a nifty little open source project management tool called ActiveCollab decided to go commercial and more or less drop the open source version. This didn’t sound too bad at the time, but as i got thinking i realised that there was a definite lack of simple project management tools which were open source. Most of those i found had clunky interfaces or a questionable development status. And of course, more or less all of them are written in PHP, not my most favourite programming language to say the least.



In the end, i was left with uncertainty of the future of ActiveCollab, so i decided to hit two bird’s with one stone and fork my own version of ActiveCollab, but this time rewrite it in Ruby using the Ruby on Rails framework with a goal of making it open source. This seemed like a good idea at the time, as:

I didn't like PHP
I was a bit of a novice at programming in Ruby
My experience of dealing with the Ruby on Rails framework was sketchy to say the least
ActiveCollab's PHP framework was similar to Ruby on Rails
Open source is great, lots of people will be interested in working on it with me

The development process started as thus:

I started off with a run-of-the-mill started Ruby on Rails framework project
I ported ActiveCollab's DB Schema over into the schema.rb
Using the PHP code and the schema as reference, i'd make basic Model's for each type of object used.
I'd decide to implement a controller
Using the original php code as a reference, i'd replicate the controller and its action's in the Ruby on Rails project
I'd also import the associated templates from ActiveCollab and convert them into .rhtml files, which are essentially the same except they use Ruby as the scripting language instead of PHP
Whilst trying to replicate the functionality of the controller, i'd also tweak up the related Model to fill in the gaps of missing functionality
In addition, when i encountered anything missing from the Ruby on Rails framework that was required, i'd find a relevant 'plugin' and import it into the project, or just implement it all by myself
I repeated steps 4-8 until i the majority of the functionality was implemented

At first, it started out as a little pet project as mine. Though after a while, i grew a bit tired of developing it (developing in this way became very tedious and boring), and decided to push on with the open source aspect of the project and ask the ActiveCollab community if they were interested in my code. The answer i got back was yes. And thus i decided to open it up fully to the wonderful world of open source development and placed it on RubyForge. I also decided to give it a cool name, RailsCollab.

Initially when i placed it on RubyForge i was contacted by someone who was interested in assisting. So naturally, i added them to the project. I also added another person, purported to be a friend of theirs that was also interested. Sadly though a month down the line, i received no signs of life from either of them. Its as if they thought “wow great, an open source project. lets join in!” and then promptly died from the excitement. Or maybe they just couldn’t be bothered, i don’t know.

So in the end being the kind person i am, i essentially did all the work myself out of the sheer hope that someone else might be interested in genuinely assisting with development. Or failing that, maybe someone would try it out and provide some much needed feedback. Sadly though, neither happened even though it was apparent at the start that people were interested.



Right now, RailsCollab is usable as an alternative to ActiveCollab 0.7.1, with the exception of a few missing features which i have simply not got round to implementing yet. And of course it suffers from a few bugs and performance issues here and there. Still, for what it is, it works.

Will i ever do such a stupid thing like this again? I can conclusively say…. hell, no!


QR-Code and the quest for the elusive decoder
James Urquhart — Sun, 10 Jun 2007 00:00:00 BST
After reading Mechanic #031 on the Three Hundred game ideas blog, i got thinking. Specifically, i asked myself “How easy would it be to write a web-based tool for encoding and decoding data to 2d images as pixels?”.

Luckily, this mechanism has been implemented before. The simplest example would be a barcode. However, i wanted something a bit more advanced in order to pack as much data as possible into the image, as i would imagine quite a bit of information would have to be present in each “game image”. After much scouring the web, i found two promising solutions: QRCode and Semacode. Sadly though, Semacode appeared to have pretty dodgy licensing, so i decided to steer clear of it and see what i could make of QRcode. 
 QRCode was interesting, as in it was very easy to find an encoder, such as libqrencode. Unfortunately, it was very difficult finding a decoder that didn’t just run on a mobile phone (which seems to be a popular way of decoding QRCode’s). The only standalone library that could decode was written in java, oddly enough also called qrcode.
To be honest, i’m a bit put off of proceeding any further trying to get this question answered, mainly because everything seems to be written in Java, – which is fair enough, but considering my skills in programming Java are minimal, i might end up with a rather dodgy botched together solution.
Regardless, i think the concept of deciphering data from 2d images is neat. I’ll just have to find the elusive c-based decoder for QCode, or perhaps even better, find a better supported alternative system.


Javascript Draw
James Urquhart — Sat, 02 Jun 2007 00:00:00 BST
Quite a while ago when i embarked on the Javascript Bandwagon, one of the first things i implemented was a rather simple draw test which plotted a few rectangles and other primitives on a WhatWG canvas object.
One of the little problems i encountered with the WhatWG canvas was that you really needed to play around with the values for the control points on such things as arc’s and bezier curves in order to get the result you wanted. Either that, or draw everything in a vector based drawing tool and convert it to the appropriate set of drawing commands to feed into the canvas object.
Safe to say, i wasn’t so easily defeated by this little problem, and so i had a crazy idea: would it be possible to create a drawing tool written entirely in javascript which used the canvas object?
The first implementation of the JavaScript drawing tool (which i will now refer to as JSDT) resembled frankenstein on steroids. I didn’t completely understand how JavaScript’s “classes” worked, so it was more or less a bunch of hap-hazardly designed functions all placed in the global namespace. Whilst it worked, it was hell to maintain, plus it lacked crucial features such as the ability to rotate and scale objects – as well as group them hierarchically.
For reference, here is a copy of the first version for you to laugh at. Note though that everything (including the manipulators) is handled in the little WhatWG canvas – no DOM in sight!
For the second version i decided to use classes from the ground-up, and make use of one of the various javascript toolkit’s available. In the end i chose Mootools, on account of it being lightweight and seemingly faster than the other framework’s i tried. I also planned out the design from the beginning, which helped a lot.
The second version sadly ended up being the third, as half-way through i realized that i wasn’t using the mootools “Class” object correctly. In error, i assumed attaching function’s to Classes was a bit of a memory intensive operation, and thus i decided to eliminate functions from the JSDT’s canvas objects, and instead place them in an “ObjectHelper” class.
Soon after, i realized how stupid i was, considering the functions are actually assigned to the object’s prototype – not its instance – and thus would be shared between each instance of the class.
And thus, “ObjectHelper” was obliterated. A good thing too, as i noticed a big speed boost! I guess it just goes to show how much slower it is to use “this.object_helper.do_that(object);” rather than the more concise “object.do_that();”.
For reference, here is a copy of the third version. Its much improved over the first, and you can even change the selected object’s style properties via the nice toolbox. Sadly, i haven’t yet got round to implementing the “Save” and “Load” functions, which would be the next major step in implementing the JSDT – shouldn’t be too hard to write out the canvas object list as JSON and send it to / from the server.
Safe to say, i learned a lot about the peculiarities of JavaScript when making the various iterations of the JSDT. I also experienced first hand how utterly annoying it is to have code work perfectly in one browser, yet fail miserably in the other. One interesting issue i did find with regard to the WhatWG canvas was that Apple’s Safari browser couldn’t quite handle this code:

...
canvas.moveTo(10,10);
canvas.beginPath();
canvas.lineTo(20, 10);
canvas.lineTo(20, 20);
canvas.lineTo(10, 20);
canvas.closePath();
canvas.fill();
canvas.stroke();
...

In Firefox, Mozilla, and Opera this would draw a little filled in square with a line border. In Safari however, it draws a filled in square without a line border! It seems that Safari completely forgets about the path after you either fill or stroke it, so the only workaround is to just plot the path again, i.e.:

...
canvas.moveTo(10,10);
canvas.beginPath();
canvas.lineTo(20, 10);
canvas.lineTo(20, 20);
canvas.lineTo(10, 20);
canvas.closePath();
canvas.fill();
// +++
canvas.beginPath();
canvas.lineTo(20, 10);
canvas.lineTo(20, 20);
canvas.lineTo(10, 20);
canvas.closePath();
// +++
canvas.stroke();
...

…which as you can imagine is much slower, especially if you are drawing lots of primitives!
For my next rewrite of the JSDT, i think i might look at implementing it in an abstraction layer such as OpenLazlo or the Google Web Toolkit. Hopefully then i will be able to eliminate much of the manual workarounds i have had to hack in, instead leaving it to a nice automated tool.


rm -rf
James Urquhart — Sat, 26 May 2007 00:00:00 BST
A short while ago, i was having problems uploading files in wordpress. It seems that it was trying to write to /tmp, even though in my php configuration the clearly specifies the upload directory to be something completely different.
During the process of basically trying to solve this issue, i decided to have a little spring clean and remove some unneccesary redundant folders from my web server directory, which is when i typed in the following:
rm -rf htdocs
Safe to say, i wasn’t very pleased with the results. Not to mention, i haven’t done any recent backup’s, so my lovely customized version of wordpress more or less vanished in an instant.
The good news however is that i managed to more or less get everything working again. I even took the time to upgrade to the latest version of wordpress (which even included the sidebar widget functionality, which left me with one less thing to install).
So i guess the motto of this story is: BACKUP! BACKUP! BACKUP!
Until next time…


The JavaScript Bandwagon
James Urquhart — Sat, 19 May 2007 00:00:00 BST
A few month's ago, i decided to hop on the Javascript bandwagon. I wanted to find out what it could do, how well it could do it, and what limitations i would bump into.
What i discovered was not what i expected.
Incompatibility
First and foremost, one of the first problems one bump's into with javascript is that all the major web browsers implement it differently. Code can work perfectly in one browser and fail miserably in another. For example:

var myList = {
tea: 1,
coffee: 2,
};
Seems perfectly fine to me. In fact, it worked fine in Firefox. However, it didn't work in Safari, Apple's flagship browser. The only clue in the JavaScript console i got was "SyntaxError - parse error", which eventually led me to this solution:

var myList = {
tea: 1,
coffee: 2
};
Yes that's right... all that fuss was over a single little comma.
Simple? Not!
After figuring that Javascript was a pretty simple c-style programming language, i thought i had it all figured out.
REXML could not parse this XML/HTML: 
However, after checking out Dogulas Crockford's presentation "The JavaScript Programming Language", i was left a bit dumbfounded. What i thought was a simple boring run-of-the-mill scripting language was in fact a pretty extensible and interesting programming language. Neat!
Frameworks
The first time i tried writing something in JavaScript, i decided to go "all-pro" and not use any of the numerous development frameworks that have popped up over the years. This was a big mistake.
After a long period of writing everything from scratch (whilst only half knowing what on earth i was doing i might add), i stumbled across the mootools framework.
Selecting elements? simple. Handling events? simple. Making complex object-oriented front-end's? simple. Having it all work in every major web browser? priceless. In fact, pretty much everything was ten times easier to work with, and solutions took less than half the time to be implemented.
Slow, yet fast, yet slow
Sadly, since JavaScript does not have a single implementation, performance varies a lot between browsers. For simple things, the difference was hardly noticeable. For more complex things though (e.g. interfacing with the WhatWG canvas), differences definitely became much more noticeable, although i would not like to speculate which browser has the fastest implementation of JavaScript as i have not done any conclusive benchmarks yet.
Single Threaded
Sadly for the most part, there is no support for threading in JavaScript. You can only do one thing at once. Whilst this is not a bad idea, it does become a little bit of a problem if you are running multiple time-out's and intervals. Everything has to wait for its turn in line so to speak, which is pretty much bad news if you really want something to happen in X amount of time, irrespective of whatever else happens in-between...
...oh well, i guess i can live with it.
Safe to say, i'll be experimenting a bit more with JavaScript in the future.

Setting...	Result
Initial transition (element just created)	FAIL
Transition continuously (during transition)	MOSTLY FAIL
Transition after a sufficient undefined delay	PASS

Method	Problem
Use UIImageView	It doesn't scale
Re-draw a UIView every frame	Far too slow
Use GLES	Beyond the scope of the last article, but not this one!
Transform a clipped UIView each frame	We did that last time

Account		Debit	Credit
Checking Accounts
	Bank	100	0
Invoice Accounts
	Microsoft	50	0
	Apple	50	0
Revenue Accounts
	Revenue	0	200

Class	Code	Parent	Description	Small description
Asset	1234	cash	cash
Asset	1234.01	1234	Joseph Mack, withdrawing	jm_withdrawing
Asset	1234.02	1234	check account	check_account
Asset	1234.03	1234	petty cash	petty_cash
Liabilities	1235	liabilities	liabilities
Liabilities	1235.01	1235	Joseph Mac, capital	jm_capital
Equity	1236	equity	equity
Equity	1236.01	1236	accumulated profit and loss	acc_profit_loss
Equity	1236.02	1236	[clo]profit and loss	profit_loss
Revenue	1237	revenue	revenue
Revenue	1237.01	1237	fees earned	fees_earned
Expenses	1238	expenses	expenses
Expenses	1238.01	1238	advertising	advertising
Expenses	1238.02	1238	food,entertainment	food_entertainment
Expenses	1238.03	1238	office supplies	office_supplies

Account	Debit	Credit
Assets	+	-
Expenses	+	(-)
Liabilities	-	+
Shareholder Equity	-	+
Revenue	(-)	+

Symbol	A	B	C
Value	0	1	2