Code On Fire Passionate About Cool Code

Comet support on GAE

What is Comet?

Comet is a set of web methodologies (like AJAX) that allows client browsers to have persistent http connections to their servers. It is also known as long polling. The benefits of making use of this type of connection is that any state change on the server can be immediately pushed to all connected clients. The physical result of this can be seen in applications like Google wave, where one user is typing a message into a wave, and another subscriber to that wave can see the letters appearing as they are being typed. [see wikipedia]

Browser Limitations

To maintain the security model, browsers are endpoints that don't allow connections to originate from a remote location. This means that whatever network connectivity is implemented in the browser environment has to be initiated in the browser and cannot come from the server.

This means when a state changes on the server, there is no way for the server to let the client know about the change. It has to wait until the client requests an update before it can be notified about the state change.

How Comet Works

There are several implementations of the comet meme, but they all rely on the same concept: opening a page on the server that never quite finishes loading. If, for instance, you load a JavaScript file and the server keeps the connection open, to the browser and to the rest of the world it just looks like a file that takes forever to download. The server, however, can now send updates to the client by pushing a JavaScript statement through the slow-loading file.

On the browser side, JavaScript includes are executed line by line as they arrive through the network. So when it gets the server's update code, it will immediately execute it, and your client can respond to the message.

Comet and GAE

As soon as you make use of Google AppEngine, you run into some limitations. Two of those limitations directly affect the comet concept:

• GAE does not allow streaming. This means that the entire resource has to be rendered on the server and the script on the server must exit before the response is sent off to the client. This makes using a long-loading file impossible to implement.
• GAE has a 30second execution restriction on scripts. So even if it were possible to stream files from GAE, it could only happen for 30 seconds before GAE automatically shuts it down.

This means that comet is not a viable option for achieving persistent HTTP connections between the client and the server. [see cometdaily.com]

Other Alternatives

There are other alternatives to accomplish a similar effect:

Java Sockets

Java plugins on an html page can create sockets that listen to incoming messages. This could be used by the server as an endpoint to which it can make announcements. Due to the fact that communications in this model is initiated on the server, this strategy comes with a heap of network baggage. Dealing with proxies, firewalls and non-standard endpoint network configurations becomes the responsibility of the Java plugin and the server announcer.

Very Frequent Short Polling

This is technically not in the same league, but attempts to recreate close to real-time connectivity with the server. In this model, the server is short-polled on a frequent basis (1 to 2 times a second, or whenever a previous poll completes).

As one can imagine, this method is very resource intensive and bandwidth hungry. Even if polling the server returns no data, just the TCP/HTTP overhead will consume a lot of bandwidth.

Also, when dealing with software residing on GAE, there are limitations imposed (and for premium users, costs applied) with regards to the number of connections made to the server. Even if these numbers were very large, it becomes a very real problem when very frequent short polling is used.

HTML5

With the wide adoption of this standard, browsers will have access to web sockets. This will essentially open up a two way connection to the server where the same connection can be re-used. It will be ideal for a RPC pipe. It will also be ideal for comet-type connectivity where the server can push information onto the pipe and the client can then respond to it. [see w3.org]