Thoughts on Web Development: Automation, Code Generation, and Scheme
Every so often I become frustrated with my tools. I mostly spend my time building web applications and it's painfully obvious how much boilerplate code I write in Python to put live apps on the web. Python isn't alone in this respect. Any language will have some amount of boilerplate. Lately I've been spending some time on automating deployment of web applications. Any web developer worth his or her salt will have some process in place for automated deployments. Another thing worth automating is your initial project generation. The more popular web frameworks have something like this in place. Pylons, Django, Rails, etc are all capable of generating a starting point for your web app.
My own web framework, blueberry, initially had an automated starting point for web apps. It also had some code for managing database migrations. Over time, these features became somewhat constraining so I ripped them out completely. The last few web applications I've worked on used Tornado, which has zero code generation for a starting point. Not having a web app completely generated for me is a big bonus in my opinion. I'd rather build tools for automating web app generation to fit my style, rather than fitting my style to the web framework.
Another topic worth exploring is web form generation. There are tradeoffs to be made in this area and many ways of doing the same thing: building web forms, validating them, and storing the result in a database. Python has some really cool form generation libraries (ToscaWidgets, for example). There's also formencode for validating and inserting defaults/errors into your forms. Using ToscaWidgets and formencode together is pretty cool if you can get it working perfectly. ToscaWidgets is especially complex. At the time I was using it the docs were a little light, so I spent most of my time inside of the code learning how it worked.
Then there's the other side of the coin: Using straight HTML to build your forms. You can 100% use formencode for validating your hand-rolled forms, and it works really well. By the time you get into displaying errors, inserting defaults, etc, you'll start wishing you went with a form generator. Hand-rolled HTML for anything complex is A Bad Idea. I mean, unless you like feeling unproductive when you write your thousandth signup form.
There's also this new-ish trend of using jQuery to generate all of your markup. I'm not sure how I feel about this. Using jQuery is awesome and it's a great piece of technology, but it's possible to make your jQuery markup generation code really complex and it can even take up more of your editor space than your normal markup. That tradeoff may or may not be worth it to you.
Focusing back on server-side technologies, I think there is a lot to be learned from Arc and Scheme in general. I keep coming back to using scheme for my web projects, but a big problem I have is that there's no clear way to do it. With LISP there's Hunchentoot, but even then you have to install all kinds of libraries and it may not even install properly. I used to use PLT Scheme (now Racket) and even wrote a few articles on using it for web development. PLT is in a world of its own. It's an advanced Scheme that has tons of libraries. It even comes with its own web server.
Using languages like Scheme, if done correctly, your web app becomes a set of high-level, short functions that were built from the ground up to limit the amount of boilerplate you have to write. When I'm writing Python I try to incorporate this way of thinking into my own software. To do that really well, you almost have to create an entirely new programming language on top of the language you're using. That's what Arc does. Python doesn't make it easy to do that, but it can be done.
Lately I've been spending some time looking at other Schemes and toying around with Chicken Scheme. I'm hoping to get some web apps running with it one of these days. Just last night I wrote some C code and used SWIG to interface Chicken and libevent. It wasn't a complete interface to libevent though. I made a super simple timing event and called it from Chicken. I'll probably end up writing some simple servers in libevent and using them in Chicken. Maybe even a new web server or tying Chicken Scheme into Mongrel2.
