Like my previous chief open source project, i decided my new one was also going to be a clone. Mainly because i couldn’t really come up with anything else more interesting, bu i also wanted to see what it would be like cloning something from scratch.

After much deliberation, i decided the focus of my attention would be to clone another of 37Signals’ products. Not Basecamp this time - that has already been done to death. Instead, i decided to try and clone… Backpack!

Interestingly while i looked far and wide for a comparable product, i couldn’t seem to find anything. The only one that came remotely close was TiddlyWiki. But while that was free and open, i felt it completely missed the niche that Backpack filled. Collaboratively edited pages, which are actually easy to edit.

The nitty gritty

First of all - before i even writ one line of code - i decided to check out Backpack’s underlying data models. They told me a lot about how Backpack worked, and of course they gave me a few hints about what data i should store.

Backpack

Models

(generally created and modified dates exist for all of these) 

Pages
    - scope
    - title
    - email_address

    Belongings (basically reference from page -> list item + position.
                     referred to by id)
    Lists
        - name
        List Items
            - completed
            - content
            - order
    Notes
        - title
        - content
    Seperators
        - name
        - page id
    Tags
        - name
    WriteboardLink
Writeboard
    - title

    WriteboardPages
        - content
Reminders
    - at time
    - content
    - creator
    - remindees [Users]
Emails
    - title
    - content
Statuses (one per user)
    - message (text)
    - last update
    - owner
Journals
    - body (length=255)
    - owner
    - last update
Users
    - name
Calendar
    - color
    - subscription url (pulling from other calendar?)
    - name
    - token (used in url for sharing)

    Events
        - title
        - occurs_at
        - occurs_until
        - reminded_at
        - remind (bool, note fixed time before)
        - all day (bool)

Next i decided to sketch out some of the models. has_many, belongs_to, etc. I also decided to give some models such as the “Belongings” different names, as i thought the originals sucked.

(Note that by this time i decided to concentrate my efforts on making a Page editor, as i felt that was the key defining feature)

After i got that out the way, i built a simple page RESTful scaffold for editing pages. All it did was render page slots and the widgets contained therein. What it lacked was the rather crucial AJAX editing that was prevalent in Backpack.

So then i decided to take a closer look at how Backpack’s AJAX page editing worked. It turned out that there were a few crucial components:

• The widgets - the real content of the page. e.g. Notes, Lists, Dividers
• The insertion bar - this pops up when you hover your mouse over the top or bottom of the widgets.
• The action bar - this pops up when you click “Insert Here” on the insertion bar, and is inserted between the relevant widgets.
• The actions on the action bar - when you click on them, either a widget is inserted (e.g. list) or a form is shown in the slate.
• The slate - this either exists in a container before the widgets, or in the action bar. It contains most of the forms.

These components are almost exclusively powered by client-side Javascript. Only when you get to the parts where content is added, edited, or removed from a page do you get an AJAX request sent to the server. e.g. Inserting edit forms, adding widgets and belongings, etc.

What you get back from the server is usually Javascript code which alters the content in the page according to the request.

So once i figured all that out, i eventually managed to implement a rather nifty AJAX-powered page editor which somewhat resembled Backpack’s. Mission accomplished!

The result

Well, here it is. It’s called Rucksack, and it acts similarly to Backpack. You can make pages, and share them with others to edit collaboratively. Strange but true.

“Now wait a minute!” you might be thinking, “it looks like crap, and it doesn’t implement everything“. True, but remember that i was only interested in implementing a proof-of-concept page editor. I wasn’t planning on implementing everything and making it look nice. That is for later.

Now while i did start off thinking of Rucksack as a Backpack clone, i don’t think it is the best frame of mind to be in. So for future development, i will be concentrating less on copying backpack, and more on tailoring it to what i think will be useful in organising.

Meanwhile, i have put the code up on github for anyone to check out.

ShareThis

Which led me to this rather interesting mailing list post by Douglas Crockford:

I apologize for my ultrablunt comment. Let me be clear about what I
meant. It was decided at the Oslo meeting that the project formerly
known as ES4 is no more. Instead, there will be a new project,
Harmony, which will be the work of a unified working group. We have
not yet agreed on a set of goals for Harmony.

I recommend that we close the ES4 list and open a harmony list to
support this new effort.

Looks like nobody can agree on any set goals for JavaScript 2 (i.e. ES4). Which leads me to think that as with a lot of web standards, JavaScript 2 is just a pipe dream.

Fortunately though, the real world has a solution which already works. Its called haXe, and it implements a lot of what JavaScript 2 should be.

So instead of waiting about for some standards crackpots trying to cobble together a standard, you can write some real code.

Have fun with haXe.

ShareThis

Fortunately there is another way to use web apps offline on the iPhone. It turned out to be so simple i wondered why i didn’t think of it before. Well, here is how you do it:

Hey presto! While you do get a warning, you can still use your lovely web app offline. Unless of course, you restart your iPhone or close the window. If that happens, your data will still be safe if your web app uses the database api to store it. Just hope you get your signal back so you can start working again. :)

ShareThis

But personally i think this tends to be really awkward, especially if you don’t want to be specific about dates or times. e.g. Speculative Opportunities. It also requires a lot of mouse clicking to find and enter the correct date and time.

Fortunately, there are libraries about which aim to solve this issue by allowing you to specify the time in English. Typically they are referred to as “Natural language” date/time parsers. In my case, i found one called Chronic which is distributed as a gem for Ruby. It has even got a nice screencast.

sudo gem install chronic

Consisting of only one public function, Chronic is really easy to use:

require 'chronic'
Chronic.parse("tomorrow at 5")
 #=> Fri Aug 08 17:00:00 +0100 2008

See? Nice and simple. We can also use the options to help chronic get the right time. e.g. If i really meant to say “Tomorrow at 5 AM”, i could fix it like this:

require 'chronic'
Chronic.parse("tomorrow at 5", :ambiguous_time_range => :none)
 #=> Fri Aug 08 5:00:00 +0100 2008

The options you can choose from are as follows:

• :context - the context in which the time is assessed (:past or :future).
• :now - current time.
• :guess - if false, this returns a time range instead of guessing at a specific time.
• :ambiguous_time_range - range in hours in which an ambiguous time will be resolved. Best to think of it as hours to skip in the day when picking a time. e.g. setting this to 18 and inputting 5 will result in 5am the next day being chosen.

Sadly i couldn’t seem to find any way of setting which time zone to evaluate the time in. This would have been useful when working with Ruby on Rails 2.1’s new TimeZone support. Fortunately though, i figured out a workaround which is as follows:

# grab time using current time zone as reference
ctime = Chronic.parse(value, :now => Time.zone.now)
# re-interpret time in current timezone
ctime = Time.zone.local(ctime.year, ctime.mon, ctime.day, ctime.hour, ctime.min, ctime.sec)

Basically this gives Chronic the time in the current time zone, which deals with relative times (e.g. “tomorrow”). It also re-interprets the calculated time in case you are a bit more specific (e.g. “5AM”).

For something a bit less hackish, one might want to check out technoweenie’s fork on github. This appears to allow you to tell Chronic to use a different Time class for calculating times, which should solve the problem.

So to conclude, i think Chronic is a nice and simple solution that works rather nicely for common cases.

Other libraries

If you aren’t using Ruby, have no fear. There are similar “Natural language” parsers available for other programming environments which work in a similar fashion to Chronic.

• Datejs for JavaScript
• parsedatetime for Python
• DateTime::Format::Natural for Perl
• DateTimeEnglishParser for .net

ShareThis

The problem

Unfortunately, the Canvas suffers from a pretty brain-dead design flaw. That is while you can get and set pixels, you cannot control when the pixel data you grab with getPixelData() will be freed from memory. This is exacerbated by the fact that the whole interface runs on top of JavaScript.

“Now wait a minute“, you might think “doesn’t JavaScript have garbage collection for this sort of stuff?”. Well, yes. The trouble is, JavaScript has no standardized garbage collection system, so depending on which browser you happen to be using it’s pot luck whether or not your pixel data will get freed on time.

An example

To start off with, here’s a really simple example test case, using “putImageData(getImageData())“.

Now lets use a real example here. A while back, you might recall i was writing a SCUMM interpreter in haXe that just so happened to compile to JavaScript as well. For the SCUMM runtime, i am required to display room graphics overlaid with various objects. The graphics are 8bit and use a palette, and that palette can be changed at runtime.

For performance reasons, i decode the graphics and store them in Canvas elements using only the red channel. In fact, throughout the graphics processing pipeline, the red channel is the only part of the pixel data i use. I also make heavy use of the native blitting functions (e.g. fillRect).

It’s not until i come to display the graphics that i re-map the pixels from the red channel into the final colours using the current palette. This requires me to getImageData() the current pixels in the canvas, iterate through and set the data[], and then putImageData() back to the Canvas for the final result.

The trouble is since i am re-mapping the pixels every frame, memory usage on certain browsers (Opera, Firefox) skyrockets up to a certain threshold - anything up to 2gb depending on what is happening.

For the test case running on OS X, Opera> tops out at ~600mb. Webkit and Firefox both top out at around 1.25gb (honestly though, i just stopped looking as it was getting stupid). It’s not until i either suspend re-draw and wait several seconds, or simply close the browser window that memory usage returns to normal.

Oddly enough when re-mapping the pixels in my SCUMM interpreter (not in the test case), Webkit’s memory usage stays constant. This flies in the face of what one would expect to happen based on the test case. As for why, it could be anything. The smaller canvas size, assignment of variables - who knows.

Workarounds?

Realistic workarounds include:

• Perform all graphics operations manually on a single set of pixel data (too slow!)
• Use drawImage() as opposed to getImageData() & putImageData()
• Limit the amount of screen updates
• When working with palletised image data that needs re-mapping, store it as RGB, and re-calculate it only when the palette changes
• Don’t use getImageData() at all, especially considering it doesn’t seem to work in Safari at the moment.

Either that, or simply implement everything which needs pixel-level graphics manipulation in Flash or a Java Applet, foregoing the pure Javascript ethic.

ShareThis

Ok, $99 might not be a lot, but one also has to consider when you come to deploy your app for real, it might not pass through the rather stringent screening process to get it to the masses through the App Store.

Wouldn’t it be great if you could deploy any application you wanted for free?

Well thanks to the recently released pwnage tool, it’s now possible to run non-Apple-approved applications on your iPhone though “Jailbreaking”. I’ve spent quite a while trying to figure out how to get this working. Here’s what i have come up with so far.

First of all…

If you are not using a Mac, or you don’t have the iPhone SDK installed, or alternatively the “open toolkit”, then these instructions will be of no use to you.

If you haven’t done so yet, Jailbreak your iPhone / iPod touch via the “pwnage tool”. For reference, this is what i did:

1. Download the pwnage tool
2. Create and transfer Jailbroken firmware using the pwnage tool

After you’ve done all that, you’ll probably notice a new application called “Cydia” appears on your iPhone screen. This is basically the way end-users can access a wide range of free “homebrew” applications on their Jailbroken iPhone’s. If you want to deploy your application to end-users, you’ll have to get it into Cydia.

But enough about Cydia for now…

I’ve built applications, how on earth do i get them over to my iPhone?

If you are using XCode, make sure you set the SDK of your application to “Device - iPhone OS 2.0″ (or the equivalent if it exists). Then all you need to do is click “Build and go”, and your application should appear on your iPhone.

Now you’ll probably get this rather “Unexpected error” popping up if you try and run your application with XCode.

Unfortunately while the pwnage tool allows you to run any application, that application still needs to be signed. So how do you sign it? Well, the best tutorial for that i have found is the “Code Signing Guide” on Apple’s website.

You’ll need to make a self-signed certificate using the Keychain Access utility. Then you should run the following commands in a terminal on your built application:

export CODESIGN_ALLOCATE=/Developer/Platforms/iPhoneOS.platform//Developer/usr/bin/codesign_allocate
codesign -fs "Name of your certificate here" /Path/to/your/built/application.app

Unfortunately even when you do this, you still cannot run your app using XCode, so you can’t use any of those fancy debug tools with it.

In order to run your application, you’ll have to copy it over yourself. You can either use OpenSSH (installed through Cydia), or iphonedisk. I used OpenSSH, as i had problems with the permissions not being preserved with iphonedisk:

scp -r /Path/to/your/built/application.app root@address_of_your_iphone_or_ipod:/Applications/

Re-load the springboard either by installing something with Cydia, or restarting your iPhone. And hey presto, your app appears!

Update: There is now a way to get your apps working directly from XCode. Simply follow the instructions here, and you should be able to debug your application straight from XCode. Neat!

Note: there are of course other ways to get your app working, but personally i found signing it to be the most obvious.

Right, now my application is ready for public consumption

Great! So how do we get it into Cydia? Well there are two options:

1. Get the Cydia guy’s to host it in their repository
2. Make your own repository

Well i couldn’t figure out how you are meant to contact the Cydia guy’s to get your application in their repository. Realistically speaking, this is the only way people are going to find your application, considering there is currently no user friendly way to add new repositories to Cydia.

After all, would you really want to type all of this into the terminal to add a single repository?

ssh root@address_of_your_iphone_or_ipod
echo "deb http://location.of/apt_repository/ ./" > /etc/apt/sources.list.d/apt_repository.list

To conclude

If you are really serious about making money off developing iPhone apps, then i’d cough up and pay the $99 developer fee. That way you’ll be able to target every iPhone / iPod Touch user, while being able to charge a fee for downloading.

If however you have an application Apple are never going to distribute (e.g. emulators), or you are just playing about and don’t mind the lack of XCode integration for on-device debugging, then developing for Jailbroken devices is for you. Personally though i’d recommend you compile your own “open toolkit”, as developing your own unofficial apps using the official SDK is legally questionable.

ShareThis

The new features i am interested in however are all tied into the web browser. Specifically, the database support, touch event handling, CSS transforms, and to tie everything together to make more native-looking apps, full-screen support.

So is it worth it?

Well i am pleased to report that most of these features have made their way to the update. My database demo which i posted about previously works fine, though initially i was a bit concerned that i didn’t see a “databases” section in the configuration options in Safari.

Turned out that this only appears when at least 1 database is created. A word of warning though: when i deleted my test database, the section vanished and i could only get it back by restarting the system. So watch out if you are playing about with it.

Sadly it appears that one of the binding features, the full-screen support, hasn’t made it into the update. Supposedly, you are meant to insert the a “apple-touch-fullscreen” meta tag into your HTML document, but i didn’t notice any difference whatsoever when i tried it. A huge disappointment, IMO.

Regardless, i think this is a positive step towards making the iPhone/iPod’s web application development platform actually useful - in a stark contrast to just over a year ago when Apple were promoting a rather limited web development platform with none of these cool features as the only way of getting third party applications on the iPhone.

Let’s not forget that for those not content with working within the limitations of the iPhone’s web development platform, there is now also the native iPhone SDK. So all in all, there’s a bit for everyone in this update.

So is it worth installing? Hell, yes!

ShareThis

Railscollab? What’s that?

Railscollab is a re-write of the Project Management solution ActiveCollab (otherwise known as ProjectPier), which instead is written in Ruby and runs on the Ruby on Rails web development framework.

Why on earth would you want to do that?

Good question. Well, it all started when the developer of ActiveCollab announced they were ditching the open source version and going commercial and closed source. Of course, there were people like me that didn’t like that.

So i decided to make my own fork - however i added a twist. I decided to re-write it using Ruby on Rails, since the original PHP code was giving me a headache. Plus it seemed like a great way to get to know the Ruby on Rails framework.

Note that around the same time, another fork called ProjectPier arose, which opted to develop from the original PHP code.

So what’s new?

Quite a lot, actually. Pretty much everything that was present in ActiveCollab 0.7.x is now implemented, sans the the fancy web-based installer. I’ve also focussed a lot on fixing bugs and improving stability, as well as enhancing the time tracking components.

… and of course there is the configuration editor which takes advantage of Phusion Passenger.

In addition, i have updated the RailsCollab demo, so if you don’t have the time to set everything up, you can still try out Alpha 3.

I’d also like to thank everyone (you know who you are) who has reported issues with RailsCollab over the past few months. Without your help, i doubt i would have had the willpower to make another release.

So what are you waiting for, try it out already. :)

Edit: If you are stuck wondering “How do i login?”, have no fear, the solution is here. You will need to use OpenID to login.

Simply select the “Use OpenID” checkbox and type in your OpenID to login. An account should automatically be created for you if one does not already exist, provided that your OpenID provider provides a username, email, and name.

You can get an OpenID from many places - one of my favourites being myOpenID.

ShareThis

When i first started this blog, i decided the focus was going to be “my work and anything related”. This turned out to be a bad focus point, as my real work at the time was quite honestly boring, and writing about “anything” just simply wasn’t focussed enough.

More than anything, i’ve written about web development. It is a frontier which i find quite promising, and which i feel has leaped bounds these past few years. With the appearance of feature-rich development frameworks such as Google’s GWT and SproutCore has come the realisation that the web - or more specifically the web browser - truly is a viable application development platform.

(Not to mention that modern web development in general does away with archaic and slow tools, instead emphasising simple and fast toolsets and the adoption of agile methodologies for quick turnaround)

So my new focus for this blog will be to write informative posts about the latest and greatest in the wonderful world of web development.

In addition, i’m going to be putting “anything else” on my new blog, jamesu.net.

So, let the experiment commence!

ShareThis

Recently i’ve been playing about with the iPhone SDK. I’m actually quite impressed at the great tools included to make development easier, whether you choose to develop native applications or web applications.

Since i wasn’t quite up to scratch with my Objective C, i thought i’d see how much better the in-built browser was. Using Dashcode, i was easily able to prototype some pretty neat touch-based interfaces.

Still, one of the major criticism’s i seem to find about developing web apps for the iPhone is a lack of an off-line option. If you aren’t connected to a network, you can’t make use of any web applications.

Well, that’s not strictly true. You can use the “data:/” URI hack, where you bookmark a URL containing the HTML of the application you want to use off-life. But this is not a very useful solution if you want to store persistent data your application can use after re-loading.

Which is why i was pleased to find out that the iPhone simulator in the SDK includes support for WebKit’s new Database storage API. In essence this means that you can persistently store pretty much anything you want on the client.

How is this useful on the iPhone? Well, it means we can now make very useful offline web applications with persistent data storage.

Here is the “proof of concept” i have cooked up.

http://www.cuppadev.co.uk/iphone/database-images/

If you visit the above URL in the iPhone simulator, you should see three items which you can click on. (Note that these are all data:/ URI’s)

• The first creates a database (if it does not already exist) and stores the data for an image, which is then loaded back and shown on the page.
• The second inserts a second image into the database. If you click on the first item again, there will now be two images shown on the page.
• The third replaces the data of the first and the second image. If you click on the first item again, you might get a little shock.

There are of course limitations to this Database API. Access to databases is restricted on a per-domain basis (although in Webkit “data:/” URL’s are treated as a single domain), and on the iPhone simulator you only get up to 5mb of storage.

Still, it works and i can see it being a very useful feature for future web applications on the iPhone.

This actually works in Safari 3.1 too, so you don’t need to download the huge iPhone SDK to try this out. It technically should work in Firefox 3 too, though as of yet i haven’t managed to get it to allow access to the database.

ShareThis