Last week I attended DjangoCon 2009 in Portland. Due to scheduling conflicts, I wasn’t able to attend DjangoCon last year, and I was disappointed that I missed that inaugural event. I’ve seen some Django stuff at PyCon, and I’ve written some Django code, being at a conference like DjangoCon helps me to understand the community and technology in a way that just reading the documentation doesn’t.

Talk Highlights

Here are some of the talks that I found notable:

Shawn Rider and Nowell Strite of PBS gave a talk titled: Pluggable, Reusable Django Apps: A Use Case and Proposed Solution. I think that very few people in the Django community had any idea that Django was being used extensively in PBS. That definitely falls into the category of pleasant surprises.    One of the strengths of Django is the focus on building small single purpose applications which can be used to build up larger applications. Doing this is harder than it sounds, and Shawn and Nowell described some of the problems that they ran into, as well as showing some ways of dealing with those issues. There was some crosstalk between the PBS guys and the Pinax developers, who are also doing a lot with reusable apps. I hope that these folks will work to share and combine their knowledge and then disseminate that to the broader Django community.

There were a number of talks which followed the theme of “how not to use parts of Django”. It’s interesting because people like Django, and even if they don’t like some parts, they want to use the rest, and are willing to work to make that possible. You would expect people to just walk away from the framework in cases like these.

Eric Ocean, one of the developers of SproutCore gave what I considered to be a pretty interesting talk. Unfortunately, his talk didn’t have much of a connection to Django, other than to suggest some things that Django could do to support SproutCore better. I know from watching the IRC and Twitter back talk, that people were put off by the style of presentation (a little too like a commercial), and the weak connection to Django. SproutCore is interesting to me because it’s at a different level than most of the Javascript frameworks. It’s at a higher level, which I think will be necessary as browser based applications become more sophisticated. I know that I am going to be taking a closer look at SproutCore, and I hope that a useful Django/Sproutcore collaboration will emerge from the sprints.

Simon Willison gave a keynote about Cowboy programming. The big piece for me was his description of how the Guardian built an application to help the public scrutinize the expenses of British MP’s. There’s something about these situations that appeals to me, against my knowledge and better judgment of “sound software engineering practices”. I guess it’s a guilty pleasure of sorts.

Ian Bicking gave a keynote which might be described as “a free software programmer’s midlife crisis”. Ian was very philosophical and reminded us that free software (as opposed to open source software) was rooted in a set of moral (not economic or process) imperatives. It was a very thoughtful speech, and I think that its worth several reads of his text (something which is hard to do on a train ride with an iPhone) and some additional ponderings.

Avi Bryant’s keynote took its root in his experiences building Trendly. As one might expect, Avi started building Trendly using Seaside. But by the time he finished, he noticed that very little of Seaside was actually being used. He attributed this to the fact that Trendly’s architecture involves loading a single HTML, with a ton of Javascript. That Javascript then manages all of the interaction with the server, which consists of snippets of JSON data. This range true to me because we used a similar architecture for Chandler Hub, the web based version of Chandler (our interaction with the server was based on atom and atompub, not JSON), and it’s the kind of architecture that GMail is based on. Avi also treated us to a demonstration of Clamato, his Smalltalk dialect that compiles to Javascript. Again, another attempt to deal with the challenges of engineering large Javascript applications in a web browser.

There were plenty of other good talks, and many of the slides are already available.

My keynote

I’m afraid that I am not equal to the task of writing out my presentation text in full as Ian and Jacob have done, so you will have to settle for the highlights and wait until the video appears.

My keynote was organized around two major sections.

The first section was a look at what I see in the Django community at present. This includes a look at some pseudo statistics around job postings and a poll of web frameworks being used by startups in an effort to get some view into whether and how much adoption of Django is happening. The short answer is that things look promising, but there is still plenty of room to grow. On the technology side, I pointed out the emphasis on combining applications and the work of the Pinax and PBS folks. The other major technology thing that I called out was GeoDjango, which is undoubtedly the most sophisticated GIS functionality in any web framework in any language or platform. This is going to be very attractive to people building location aware mobile apps, and I showed two examples of augmented reality applications as illustrations. This section ends with some observations about the Django community, using the PyCon sprints as an example. Ok, there are also some lighthearted slides about Django’s mascot, the djangopony.

The remainder of the talk was about the ways that web applications are changing and how Django might adapt to them. There are (at least) three groups of people that will be impacted by these changes. From the view point of users, the two big things are richer, more interactive applications, and access from location enable devices. Developers are going to need help in dealing with these new requirements, and the people who operate web applications need much more support than they currently have.   

I see several technologies that will be important in facilitating these changes. The first of these is some Rich Internet Application technology. The second is API’s to web applications. A digression on this point. When the iPhone was introduced, the only way to develop applications was using web technologies. This made a lot of people very angry, and Apple followed up with the ability to build native platform applications. It should be possible to build rich web interfaces on the iPhone. My observation is that given the choice beween a rich web interface and a native iPhone application, users pick the native application. Look no further than the furor over the native Google Voice application. The native applications are talking to the servers using API’s. Those API’s are not just cool Web 2.0 frosting. The last technology is cloud computing, which started out as a deployment/operations technology and is now moving up to impact application development at many levels.

In light of this, what are framework developers to do? I did a quick survey of several web frameworks which have interesting ideas or approaches in them, so that the Django folks could see what their “competitors” are up to. The frameworks that I included are Rails, Lift (Scala), Webmachine (Erlang), Nitrogen (Erlang), CouchApps (CouchDB + JavaScript – this isn’t quite a framework in the traditional sense, but it met the spirit of my criteria), and Javascript. In the case of Javascript, the observation is that the rapid increase in Javascript performance coupled with a good Javascript framework leads to something which is economically attractive (same technology in the server and client).

The talk finishes with a set of proposals for “science projects” that might be attempted in the context of Django. Some of what I outlined is emerging, and in some cases speculative. Django doesn’t need to blow itself up and start over. Instead, what’s needed is for people with Django sensibilities to look some of these problems and see if a Django flavored solution can be found. Here’s the list of projects:

1. Asynchronous Messaging – if there’s any use of messaging, it’s typically to do jobs in the background. What would happen if we made the use of messaging pervasive throughout the framework?
2. Comet – I think that the Django+orbited approach to Comet is limited in comparison to what you see in Lift or Nitrogen. Can Django do Comet support at the same level (or better) than these frameworks? What would happen if the Comet stuff were hooked up directly to the messaging stuff I just described? Imagine the equivalent of urls.py that routed Comet requests to messaging.
3. REST – There are several good packages for dealing with REST in Django. It would be nice to have this all packaged up neatly and made available for people.
4. Deployment – This is really a mess. Are there changes that could be made to Django to make it easier to deploy, or to work better with tools like Puppet, Chef, Fabric, etc?
5. Monitoring – Typically frameworks provide very little monitoring information. It seems like there is a lot that could be done here.
6. Analytics – Once you have raw monitoring information, the next step is to do some analytics on it. Django is famous for creating admin UI’s with a very small amount of effort. What if we applied that same thinking to analytics?
7. Cloud – If you add up the first 6 items, you are well on your way to what might be a cloud friendly framework. There is still some work to do in terms of making applications on the framework adapt to elastic deployment scenarios, but it would be a good step.
8. Stacks – A very basic step towards cloud stuff would be to build a preconfigured stack of software to run/develop a Django app. This is a controversial idea, because everyone has their own idea of what software should be in such a stack, and how all the configuration switches should be set. I still that having one (or more) such stack would help more than it would hurt. In my ideal world this stack would be delivered as a virtual machine image that could also be uploaded to cloud providers.

All of the talks were videotaped so those of you who were unable to attend will be able to catch up soon.
I had a great time hanging out with the Djangonauts. My thanks to Leah Culver and Robert Lofthouse for inviting me to speak.