Python Web Frameworks: Are Template Languages Worth using, or is Python enough?
June 3rd, 2007 by
gloriajw
Last year I was tasked with the job of evaluating web frameworks and template languages, picking a set of tools, and using them to rewrite chunks of our web framework, one chunk at a time. It was a fun project (thanks be to Bill). I had months to try out various frameworks, content servers and template languages before we committed to anything and moved forward.
With respect to the web framework itself, certain things swayed my decision. We had an existing database and a whole suite of home-grown Psycopg objects to access this data the way we wanted. I was very reluctant to rewrite or replace all of that, because it contained much of the business logic in a way that the entire group understood and was used to seeing/changing.
I was tempted by Django, but realized that we would not need or use it’s very nice database administrative interface (even though I had no trouble whatsoever retrofitting our existing schema into Django. That was quite impressive). At the time, I wasn’t sure if we would use Django’s template language either. Then I discovered that Django did not have multiple database support. These factors made me shy from Django and back toward a thinner Python web framework. I wanted next to no explicit database support, and the freedom to choose whatever template language I wanted.
My best choices at the time were PythonPaste and CherryPy, which were running neck-and-neck in terms of rolling out new ease-of-use features, and WSGI support. I thought PythonPaste had a slight advantage over CherryPy at the time. I used it in earnest, only to find out that the bugs I posted to their user’s list were ignored. There were also times where the new release egg files were broken, and brought my prototype down to a grinding halt. I also found that the configuration file syntax was a bit obscure, and documentation was sparse. I was foced to ask the one guru, Ian Bicking, for help, and hope that he’d answer my post. Sometimes he did, and sometimes he did not. This was not comforting, to say the least. “Did he respond to your bug post yet?” My boss would ask. “Nope, I have to wait another day and see..” I responded time and again, until I finally dumped PythonPaste and gave up.
CherryPy had it’s own quirks. I experienced some header bugs, some oddities in cookie formatting between it and PHP (only to discover that CherryPy did it according to the official spec, and PHP did not). Its configuration syntax was also a bit obscure. But then CherryPy 3.0 was released, and this changed everything. WSGI support was easier to implement than ever in PythonPaste (although I think PythonPaste put the pressue on the CherryPy group to add this functionality, so PythonPaste did serve a good purpose in that respect). The configuration syntax, the root instantiation syntax, object and URL nesting all became much easier in 3.0. The support list is usually excellent, where several developers post responses in a day or two. There was no doubt that this was the direction to take, and it is still my favorite thin web framework.
CherryPy imposes no requirement to use it’s own template language, called CherryTemplate. This is a great! It gave me the opportunity to evaluate template languages as a separate entity. Here is what I found.
One of the goals of the new web framework was to migrate ourselves away from Zope. Much of our administrative interface was, and still is written in Zope using DTML. As part of the evaluation of web frameworks, I tried learning ZPT, which is a tag-based template language. Its syntax was a bit obscure, and it’s python interface was not as robust as I had hoped.
We were not generating dynamic XML content, so a tag based template laguage was not for us. I could see this as being a handy tool for XML content, but then again, there are other Python XML content generators with less obscure syntax. DTML is similarly obscure (although it has try/except syntax, which comes in handy when testing variable values). DTML is not pretty. It’s SQL interface is handy from within Zope, but not handy outside of that scope. Despite the fact that we used it for most of our web interface within Zope, none of us enjoyed using it. I found the magic DTML document online, and became decent at DTML, but I still did not like its obscure syntax. Also, all of us at various times were left wondering “Where did it get THAT value from?” I blame Zope for the variable ‘magic’ that had to constantly be deciphered when passing data to DTML, but it still made the whole experience one that none of us wanted to repeat.
In the end, it was difficult to justify either DTML or ZPT outside of the Zope scope, so both were out of the question in our new web framework.
I tried Cheetah, which seemed simple at the time, but too procedural for me. I wanted a template language which was more object-oriented in design, and as close to the Python syntax as possible. My logic was “Why should I learn/impose a new language on web developers if they already know and use Python?”, because in our small shop, the developers also handled web content.
Kid was the next template language I played with. It seemed interesting as an XML based template language. It has separate template and template generation components, nice for XML generation of fixed templates. But in our non-XML environment, many of our page templates are dynamic. They change based on whether the user is logged in as an administrative user, their group rights and access, etc. Kid turned out to be overkill, since for our application, it is much easier to dynamically generate both the template and the content based on the user’s login mode and group settings. Kid is nice, but not for this non-XML, dynamic template application.
MochiKit was next. I was thoroughly impressed with it’s ability to make javascript usable by a Python developer. Plus, the signal library sounds nifty. Contact the server when certain events happen, instead of pinging the server every N seconds, and letting the server figure out what event happened? Wow, how neat. Unfortunately this too was overkill for our particular application, and I could not justify it in the scope of the project. “Hey, who wants to learn javascript?” did not fly well in our group, so I skipped it.
Lumped along with CherryTemplate were some others, whose names escape me now, with the same philosophy: Provide Python-like syntax and variable expansion. This doesn’t offer much to native Python developers, where variable expansion is already so easy.
I ended up using no template language. Python seemed to serve our needs very well. I defined our own /Templates subdirectory for each module/URL path in CherryPy. Each template file contained syntax like this:
<table border="0" cellpadding="0" cellspacing="0">
<tr><td>
<p>Client ID: %(client_id)s</p>
<p>Account Expiry Information (%(VEDIT_OPTION)s)</p>
<a href="%(STATIC_URL)s/%(inventory_link)s/ViewInventory?client_id=%(client_id)s">Unfulfilled Inventory Orders</a>
<a href="%(STATIC_URL)s/%(history_link)s/ShowHistory?client_id=%(client_id)s">View Inventory History</a>
</td>
<td>
etc...
The %(variable)s Python template syntax expands out to the variable name by the following syntax:
class MyCherryPyPageGenerator:
def index(self):
STATIC_URL = "https://my_site.whatever"
TemplatePath=os.path.join(os.path.dirname(__file__)) + '/Templates' # set TemplatePath to current dir/Templates
client_id=505
inventory_link= 'manage/inventory'
history_link = 'manage/history'
if user.admin_mode: # Optionally set certain links based on login mode.
VEDIT_OPTION = '''
a href="%(STATIC_URL)s/admin/EditAccount?client_id=%(client_id)s">Change Your Account Settings
”’ % locals() # Expand STATIC_URL variable in the VEDIT_OPTION variable now.
else:
VEDIT_OPTION = “Access Denied”
fd = open(TemplatePath + ‘/template_file’)
this_page = fd.read() % locals()
fd.close()
return this_page
index.exposed=True
This syntax opens the template file shown above, conditionally sets the required local variables, expands them using the locals() syntax, and returns the CherryPy page to the browser. Ignoring the CherryPy syntax, you’ll see that the template logic involves setting local variables, reading in a template file, and expanding those variables.
This syntax, combined with CSS macros for standard page formatting, proves to be very powerful and simple. What is not showing up in my template, above, are CSS id tags, such as <td id=”leftcolumn”>, which are CSS macros, defined in a separate CSS template file.
For our needs, this seemed perfect. It is entirely in Python, not too bad to read when compared to reading another template language, and easy for everyone to modify and maintain. The advantage to this approach is that the templates themselves can be as dynamic as you wish. Entire section of a page can be displayed or not, based on your explicitly defined variables and logic. The formatting is entirely controlled by CSS, and can be overridden in Python when appropriate.
The downfall to this approach is that complex pages generate very long pieces of code, where the template and the business logic are intertwined. Is this a downfall? I am not sure, because I have yet to find any single template language which can cleanly and easily separate business logic from template code, and leave you with something legible.
My Conclusion
Each template language exists for a reason or a season. I think ZPT and DTML have had their days in the sun, and those days are now over. As AJAX grows in popularity, MochiKit seems like something very worthwhile, but requires an entirely separate development effort. It is also up against a very powerful Javascript library called Dojo, which offers a browser based editor and graphics package in addition to standard Javascript functional libraries. This is tough competition.
Many template languages are simple enough to provide the template abstraction you need, but are not robust enough to justify not using Python directly. So many of them promise to separate business logic from template design. But in reality I have yet to see this happen. Very often, the person making the template change has to know about the method returning it’s data, or even about the db schema itself. With the exception of CSS designers, I have not seen a template designer who did not need some level of programming experience and access to the interface code.
I see many template languages tout their speed and similarity to Python. My immediate question is: “Well, then why should I not simply use Python instead of your template language?” My wish for the future of template languages is that they do/don’t do the following:
1. Either make more use of CSS templates, or use a similar CSS ‘#’ macro syntax which can inherit from a CSS template passed into the template language. That would be so nice. CSS is not going anywhere anytime soon. And let’s face it, it rocks if you know how to use it.
2. Don’t just brag about how similar it is to Python, and how fast it is. Show me how it can help me truly separate business logic from content, without adding a level of obscurity.
3. Make it support DOM2, please! This one is personal for me, since I am doing DOM2 server-side manipulation in my current project. A template language with a good DOM2 structure, possibly one which can be passed to other Javascript libraries such as Dojo, would be so very nice.
Comments are welcome. Examples will be incorporated into this article with author credits, so please post or email some useful examples.
Did you work with a template language not mentioned here? Please let us know about your experience.
June 3rd, 2007 at 1:37 pm
For your home-grown templating, you may want to try the string.Template syntax, if you’re using at least Python 2.4:
http://docs.python.org/lib/node40.html
I find Django templates quite usable outside of Django, and striking a good compromise between usability and flexibility.
Your penchant for object-orientation, simple tools and Python templates steers you toward Quixote’s PTL templating: pure Python, with the convenience of built-in escaping.
Quixote itself, and its newer incarnation, look like a good fit:
http://www.mems-exchange.org/software/
About using CSS syntax and DOM2 on the server, I’m not so sure.
Nice article on the whole, thanks.
June 3rd, 2007 at 4:07 pm
Hi Nicola,
String Templates are useful in the respect that they do not pollute the local name space with a bunch of local variables. Thanks for the suggestion.
Quixote seems like a web framework, not just a template language. My favorite is still CherryPy, so I can’t switch web frameworks right now :) I wonder if the Quixote PTL can be cleanly separated and used independently.
Thanks again for the useful links.
~G~
June 3rd, 2007 at 10:51 pm
You might also want to have a look at web.py; it’s got a fairly minimal default template system, and can optionally use Cheetah as well if you ever want to try it again.
Of course, if you don’t really need the overhead of a full framework stack, don’t use a full framework — you should be using software that solves your problems ;)
Couple quick notes on Django, though (disclaimer: I’m the release manager for Django and work at the company that originally developed it):
If you want, you can use Django pretty easily without any database components at all, using just URL routing, view functions and your choice of templating. There currently aren’t any docs on using other components with Django, but it’s not terribly hard — it’s all just Python, so you just import and use the stuff you want instead of importing and using Django’s components.
And there is a branch in our SVN repo that’s somewhat stagnated, but was developed by someone who wanted support for specifying multiple databases (in the sense of “Model A lives in Database A, Models B and C live in Database C”, not in the sense of replication or pooling, which should really be handled by a clustered DB or a pooling connection manager and not by your application), though someone on IRC tonight said he’s got it running on trunk again.
June 3rd, 2007 at 10:57 pm
I had a similar annoyance with the state of templating. I find python coded embedded into my template files to be rather ugly and so I instead wanted something where a designer could hand me a nicely formatted xhtml file and I could never touch the file. So what I did was implement a templating system myself that converts the page to a set of HTMLElement and DataElement objects and connects them together in a tree. Each HTMLElement provides a set of query methods that let you find an element that is a child (or a child of a child) of that element. The idea was sort of based on PyMeld (which treats pages as a single depth tree where any Meld element’s __getattr__ method will return an element anywhere in the page by id), but I ended up being a bit more strict about it all. I ‘ve tested Kiln as I called it against the genshi/mako benchmark suite and have been happily surprised beating out most other templating languages.
The code is here: http://git.geekfire.com/?p=kiln.git;a=summary
though i haven’t had a chance to write as many examples and documentation as I would like.
June 4th, 2007 at 12:39 am
The template part of Quixote has been split out into Qpy http://www.mems-exchange.org/software/qpy/.
June 4th, 2007 at 1:33 am
Yes, you can use the Quixote templating on its own.
http://quixote.ca/learn/1.html#templating
June 4th, 2007 at 2:48 am
I agree that having another language makes no sense. But how are you going to generate content. Make table rows for data fetched from db, for example?
June 4th, 2007 at 3:25 am
Quixote is a web framework, but Qpy (http://www.mems-exchange.org/software/qpy/) is a standalone template engine.
The most Pythonic template language I found is web.py templator http://webpy.infogami.com/templetor.
Check also this related page: http://moinmo.in/TemplateSystems
June 4th, 2007 at 6:08 am
For that very reason I use web.py’s template system I’ve not found a simple enough template system to produce css+javascript so I write it all by hand. I love mootools & prototype (little bloated).
June 4th, 2007 at 6:11 am
Ivan, to answer your question, yes. The table formatting is turned into ‘#’ CSS macros, the data is retrieved from the DB, and a Python for loop is used. That’s how I do it.
June 4th, 2007 at 6:17 am
Can someone experienced in QPY show me some examples of why it’s so great? I read the readme file and cringed, quite honestly, because it’s only addressing problems with quoting. H8 is great, but what else can this template language do? Python has some beautiful built-in quoting and encoding functions, which I currently use. I already know how to quote from my UNIX shell experience ;) So I need to see some sharp examples for why I should use it. It’s easy to mention, but please demonstrate.
June 4th, 2007 at 6:27 am
To my knowledge, ZPT is DOM2 compliant. Since it doesn’t use non-XML syntax for the template scripting.
Or do I misinterprete your point?
Personally ZPT is my favourite templating method, but indeed it does not feel very intuitive at first.
DTML has been declared dead a few years ago :)
June 4th, 2007 at 6:35 am
Alexander, Kiln sounds very interesting, if the model could be made to be DOM2 compliant. It sounds like a nice fit for anyone doing back end web content manipulation and re-issuing to a front end (like myself). I wonder how it handles one’s simpler non-DOM template needs. Please post examples.
June 4th, 2007 at 7:55 am
RRN: Does ZPT support range() syntax? Even if it does, I have no desire to use it outside of Zope. Genshi or Kiln look like nicer alternatives anyway. I’m evaluating these now.
June 4th, 2007 at 8:22 am
I realize you’re don’t seem to have XML needs, but XSLT makes for a great template language. It is not like Python of course, but I found that difference in approaches is actually very helpful.
Of course I cannot suggest it without naming 4Suite-XML and Amara (both are easy_install-able). Amara makes working with XML simple and efficient with 4Suite picking up where Amara leaves off regarding more robust features.
I realize there is ElementTree and its friends such as lxml, but Amara is aware of subtleties that ElementTree and other seem to miss.
This comment is probably a bit off topic, but if you have any need or desire to work with XML and Python, 4Suite and Amara are the way to go.
June 4th, 2007 at 8:54 am
> I already know how to quote from my UNIX shell experience ;)
You know how to do it, but sooner or later you will forget to do it somewhere, and your web apps will be exposed to XSS attacks.
One beauty of PTL and QPy is that they always quote, unless you tell them otherwise. That way, you will not forget. :-)
The other beauty is the syntax. I like having HTML strings seamlessly embedded in Python scripts:
def header [html] (title, stylesheet):
”
‘ ‘
‘ %s’ % title
if stylesheet:
‘ ‘ % stylesheet
I was once fond of XML-based templates, but in the end dropped them. These days I’m coming back to seeing their utility, but in fat Ajax clients.
June 4th, 2007 at 8:56 am
HTML mangled in PTL example, sorry.
June 4th, 2007 at 9:40 am
James wrote:
> You might also want to have a look at web.py
The trouble with web.py is that it’s pretty much not documented.
June 4th, 2007 at 10:49 am
Nicola, I agree, it’s elegant directly in Python. That is why I just can’t bring myself to decide on a template language.
Eric: ElementTree has no DOM/DOM2 compliance at all. When I wrote to the author and asked him why, he responded that he didn’t care to add it. “It works fine” was the jist of the response. It only works fine if all you’re doing is reading in XML and rendering. It’s profoundly difficult to use for manipulation, because of it’s odd nesting of element nodes, such as:
<p>text</p>more text
“more text” is stored in something called a ‘tail’ of the p node, instead of nesting “more text” and p as peers. Very odd indeed.
June 4th, 2007 at 11:49 am
I’ve used CherryPy on two real projects and I actually like the HyperText package:
http://dustman.net/andy/python/HyperText/
I’ve had to make a few changes for Unicode support and to add some attributes that were missing but it’s easy to work with. Instead of making separate templates and then interpolating variables into them I can write more natural Python code. Each of my functions generates and returns a small chunk of HTML. There’s no gathering up a dict of variables and then feeding that into a template processor. Writing code like this:
return DIV(FORM(INPUT(type=”text”, …), …), id=”theform”)
where I can use variables or functions right in the HTML-generating function calls starts to feel natural real fast, and with nice indenting it even looks like HTML without all the chaff.
June 4th, 2007 at 1:01 pm
Hey, I like this!
June 4th, 2007 at 5:48 pm
oh, I implemented HyperText style support in kiln once (it looks a lot like how MochiKit does DOM objects). I pulled it because I was having issues generating the various partial functions dynamically in a way that looked right when imported etc, but the HTMLElement class is such that you pretty much just have to do partials for the various tag names to get that behavior.
June 5th, 2007 at 7:00 am
If you prefer to generate html from python, pyhtmloo also did the same thing as HyperText though I don’t know which one did it better. I would try HyperText soo. Jinja also provided a template language that’s quite similar to the django template.
June 5th, 2007 at 11:21 am
If you like the
“DIV(FORM(INPUT(type=â€Âtextâ€Â, …), …), id=â€Âtheformâ€Â)”
style, have a look at Breve. It has a similar style and supports template inheritance and such.
June 5th, 2007 at 12:40 pm
These comments have been excellent, thank you.
June 6th, 2007 at 1:36 am
gloriajw, Um. With “”" string literals it will not look so ugly as it is with php heredoc’s unindented trailing mark. Cool.
June 15th, 2007 at 12:25 pm
Gloria, it’s seems like the ‘Zope’ you’re migrating from is Zope2. Did you consider Zope3 as a migration strategy? Zope3 is componentized, so you wouldn’t have to use ZPT or DTML if you didn’t want to.
June 15th, 2007 at 3:44 pm
Michael, yes it was Zope2. We unanimously found Zope2 to be useful for it’s time, but we thought that time had expired. I started to poke at Zope3, but it did not pique my interest as much as CherryPy. It seemed easier to use a thin framework rather than strip down a thick framework and retrofit other pieces. Plus, the lack of good Zope3 docs at the time, our painful Zope2 experience, and rumours about the Zope3 development community not really enjoying the experience made us shy away.
Before this job, I did use the ZODB standalone for a touch screen POS system I wrote in WxPython. I had client restaurant machines synching to my server, and it was a beautiful thing. I was truly impressed. Without a ‘flat’ ORM db, I faced some challenges, but my data set was small, and easily manipulated in nested dictionaries. I could see this flopping for very large data sets. But the ZODB rocked. It was very nice, I think, the first in it’s class of small, free OODBs. Easy to administer and use, it’s code _was_ python, it didn’t just look like Python. I enjoyed that project. I wish I could have sold it, but, oh well.
If you’re reading this Frank, don’t fret. We will have other opportunities.
January 28th, 2008 at 11:44 am
Hi Gloria,
This comment is so late in the game, it’s a crime.
If you’re not using any of the gui tools such as wxDesigner, etc.. for generating gui code, is there a particular way in which you write your code that makes it easy to keep track of the where you in your layout.
It’s easy sometimes to lose track of where you are in nested boxes, etc..
I wrote an app for personal usage in wxPython, but associate some pain to tackling another one.
May 20th, 2008 at 6:24 pm
This works great :)