PyArkansas: Small town, big tech!


Last weekend I flew out to to Little Rock, Arkansas, took a two hour crawl through a snarl of traffic, and arrived just in time for my Friday night pre-PyArkansas tutorial in Conway. Held on the stunning campus of Hendrix College, I wended my way around buildings, a massive fountain, inspiring structures, until I found the building where my tutorial was about to happen. Standing in the foyer with the beautiful Foucault pendulum, I could not help but to stop for a moment, exclaiming “Oooo!!!!” aloud, wishing I had gotten there thirty minutes earlier.

My tutorial was intended to encourage women in computer science by serving two purposes: discussing the source code and functionality of a particular project, and openly discussing some of the issues they faced in their current programs and surroundings. I was pleasantly surprised to discover that these women needed no technical or social encouragement. They are already enthused, technically and socially well prepared, and on their way to a very bright future in engineering or computer science. I was pleasantly surprised that they were very comfortable in the bash shell, and as comfortable in their current college curriculum. Like children at play, they picked up the moderate-levelled tutorial code quickly, made great strides in such a short time, and had a lot of fun doing it.

It was profoundly encouraging to see such a small computer science program achieve diversity as well as such a high level of skill. It made me wonder why larger colleges and universities cannot accomplish the same on bigger budgets, with larger staff, and a more diverse mix of students. It touched me to hear and see the enthusiasm, eagerness, and skill of the students in this program. Their learning experience under the Department Chair, Dr. Burch, comes as close to perfect as I have ever seen.

The next day’s events at PyArkansas were held at the also-very-nice campus of University of Central Arkansas, where an entire day’s worth of tutorials took place. Two Python 101 tracks were held: one for programmers and one for non programmers (a great concept). An all-day Django Track was given, where the advanced course was taught by Jacob himself. I held an afternoon tutorial addressing advanced Python concepts, with downloadable example code, where we compared and contrasted build and deployment tools, played with regex, and showed examples of some internal Python oddities involving static variables. I unfortunately missed the Python Blender tutorial, held at the same time as mine, and I heard it went quite well.

The campus facilities were very accomodating. Everything was well organized,and up and running for us when we arrived. This is a very welcome surprise to anyone who has travelled a bit to do tutorials. I was specifically told by Dr. Chenyi Hu, the Department Chair of UCA, that he really does care about diversity, and it is something they strive to achieve. This was truly touching, quite impressive, and a pleasant surprise from such a small town.

Kudos to Greg Lindstrom, Dr. Carl Burch of Hendrix College, Dr. Chenyi Hu of UCA, and everyone else involved. You induced a big technical “tremor” through your small town, which echoed far and wide. It is yet another example of the great people drawn to the Python community, and the amount of quality effort they are willing to give back. I feel honoured to have been part of this event, and I hope to be involved in many more to come.


Python Web Frameworks: Are Template Languages Worth using, or is Python enough?


Last year I was tasked with the job of evaluating web frameworks and template languages, picking a set of tools, and using them to rewrite chunks of our web framework, one chunk at a time. It was a fun project (thanks be to Bill). I had months to try out various frameworks, content servers and template languages before we committed to anything and moved forward.

With respect to the web framework itself, certain things swayed my decision. We had an existing database and a whole suite of home-grown Psycopg objects to access this data the way we wanted. I was very reluctant to rewrite or replace all of that, because it contained much of the business logic in a way that the entire group understood and was used to seeing/changing.

I was tempted by Django, but realized that we would not need or use it’s very nice database administrative interface (even though I had no trouble whatsoever retrofitting our existing schema into Django. That was quite impressive). At the time, I wasn’t sure if we would use Django’s template language either. Then I discovered that Django did not have multiple database support. These factors made me shy from Django and back toward a thinner Python web framework. I wanted next to no explicit database support, and the freedom to choose whatever template language I wanted.

My best choices at the time were PythonPaste and CherryPy, which were running neck-and-neck in terms of rolling out new ease-of-use features, and WSGI support. I thought PythonPaste had a slight advantage over CherryPy at the time. I used it in earnest, only to find out that the bugs I posted to their user’s list were ignored. There were also times where the new release egg files were broken, and brought my prototype down to a grinding halt. I also found that the configuration file syntax was a bit obscure, and documentation was sparse. I was foced to ask the one guru, Ian Bicking, for help, and hope that he’d answer my post. Sometimes he did, and sometimes he did not. This was not comforting, to say the least. “Did he respond to your bug post yet?” My boss would ask. “Nope, I have to wait another day and see..” I responded time and again, until I finally dumped PythonPaste and gave up.

CherryPy had it’s own quirks. I experienced some header bugs, some oddities in cookie formatting between it and PHP (only to discover that CherryPy did it according to the official spec, and PHP did not). Its configuration syntax was also a bit obscure. But then CherryPy 3.0 was released, and this changed everything. WSGI support was easier to implement than ever in PythonPaste (although I think PythonPaste put the pressue on the CherryPy group to add this functionality, so PythonPaste did serve a good purpose in that respect). The configuration syntax, the root instantiation syntax, object and URL nesting all became much easier in 3.0. The support list is usually excellent, where several developers post responses in a day or two. There was no doubt that this was the direction to take, and it is still my favorite thin web framework.

CherryPy imposes no requirement to use it’s own template language, called CherryTemplate. This is a great! It gave me the opportunity to evaluate template languages as a separate entity. Here is what I found.

One of the goals of the new web framework was to migrate ourselves away from Zope. Much of our administrative interface was, and still is written in Zope using DTML. As part of the evaluation of web frameworks, I tried learning ZPT, which is a tag-based template language. Its syntax was a bit obscure, and it’s python interface was not as robust as I had hoped.

We were not generating dynamic XML content, so a tag based template laguage was not for us. I could see this as being a handy tool for XML content, but then again, there are other Python XML content generators with less obscure syntax. DTML is similarly obscure (although it has try/except syntax, which comes in handy when testing variable values). DTML is not pretty. It’s SQL interface is handy from within Zope, but not handy outside of that scope. Despite the fact that we used it for most of our web interface within Zope, none of us enjoyed using it. I found the magic DTML document online, and became decent at DTML, but I still did not like its obscure syntax. Also, all of us at various times were left wondering “Where did it get THAT value from?” I blame Zope for the variable ‘magic’ that had to constantly be deciphered when passing data to DTML, but it still made the whole experience one that none of us wanted to repeat.

In the end, it was difficult to justify either DTML or ZPT outside of the Zope scope, so both were out of the question in our new web framework.

I tried Cheetah, which seemed simple at the time, but too procedural for me. I wanted a template language which was more object-oriented in design, and as close to the Python syntax as possible. My logic was “Why should I learn/impose a new language on web developers if they already know and use Python?”, because in our small shop, the developers also handled web content.

Kid was the next template language I played with. It seemed interesting as an XML based template language. It has separate template and template generation components, nice for XML generation of fixed templates. But in our non-XML environment, many of our page templates are dynamic. They change based on whether the user is logged in as an administrative user, their group rights and access, etc. Kid turned out to be overkill, since for our application, it is much easier to dynamically generate both the template and the content based on the user’s login mode and group settings. Kid is nice, but not for this non-XML, dynamic template application.

MochiKit was next. I was thoroughly impressed with it’s ability to make javascript usable by a Python developer. Plus, the signal library sounds nifty. Contact the server when certain events happen, instead of pinging the server every N seconds, and letting the server figure out what event happened? Wow, how neat. Unfortunately this too was overkill for our particular application, and I could not justify it in the scope of the project. “Hey, who wants to learn javascript?” did not fly well in our group, so I skipped it.

Lumped along with CherryTemplate were some others, whose names escape me now, with the same philosophy: Provide Python-like syntax and variable expansion. This doesn’t offer much to native Python developers, where variable expansion is already so easy.

I ended up using no template language. Python seemed to serve our needs very well. I defined our own /Templates subdirectory for each module/URL path in CherryPy. Each template file contained syntax like this:

<table border="0" cellpadding="0" cellspacing="0">

        <p>Client ID: %(client_id)s</p>
        <p>Account Expiry Information (%(VEDIT_OPTION)s)</p>

        <a href="%(STATIC_URL)s/%(inventory_link)s/ViewInventory?client_id=%(client_id)s">Unfulfilled Inventory Orders</a>
        <a href="%(STATIC_URL)s/%(history_link)s/ShowHistory?client_id=%(client_id)s">View Inventory History</a>



The %(variable)s Python template syntax expands out to the variable name by the following syntax:

class MyCherryPyPageGenerator:

        def index(self):
                STATIC_URL = "https://my_site.whatever"
                TemplatePath=os.path.join(os.path.dirname(__file__)) + '/Templates' # set TemplatePath to current dir/Templates
                inventory_link= 'manage/inventory'
                history_link = 'manage/history'

                if user.admin_mode:   # Optionally set certain links based on login mode.
                   VEDIT_OPTION = '''
                      a href="%(STATIC_URL)s/admin/EditAccount?client_id=%(client_id)s"&gt;Change Your Account Settings</a>
                   ''' % locals()    # Expand STATIC_URL variable in the VEDIT_OPTION variable now.
                    VEDIT_OPTION = "Access Denied"

                fd = open(TemplatePath + '/template_file')
                this_page = % locals()
                return this_page

This syntax opens the template file shown above, conditionally sets the required local variables, expands them using the locals() syntax, and returns the CherryPy page to the browser. Ignoring the CherryPy syntax, you’ll see that the template logic involves setting local variables, reading in a template file, and expanding those variables.

This syntax, combined with CSS macros for standard page formatting, proves to be very powerful and simple. What is not showing up in my template, above, are CSS id tags, such as <td id=”leftcolumn”>, which are CSS macros, defined in a separate CSS template file.

For our needs, this seemed perfect. It is entirely in Python, not too bad to read when compared to reading another template language, and easy for everyone to modify and maintain. The advantage to this approach is that the templates themselves can be as dynamic as you wish. Entire section of a page can be displayed or not, based on your explicitly defined variables and logic. The formatting is entirely controlled by CSS, and can be overridden in Python when appropriate.

The downfall to this approach is that complex pages generate very long pieces of code, where the template and the business logic are intertwined. Is this a downfall? I am not sure, because I have yet to find any single template language which can cleanly and easily separate business logic from template code, and leave you with something legible.

My Conclusion

Each template language exists for a reason or a season. I think ZPT and DTML have had their days in the sun, and those days are now over. As AJAX grows in popularity, MochiKit seems like something very worthwhile, but requires an entirely separate development effort. It is also up against a very powerful Javascript library called Dojo, which offers a browser based editor and graphics package in addition to standard Javascript functional libraries. This is tough competition.

Many template languages are simple enough to provide the template abstraction you need, but are not robust enough to justify not using Python directly. So many of them promise to separate business logic from template design. But in reality I have yet to see this happen. Very often, the person making the template change has to know about the method returning it’s data, or even about the db schema itself. With the exception of CSS designers, I have not seen a template designer who did not need some level of programming experience and access to the interface code.

I see many template languages tout their speed and similarity to Python. My immediate question is: “Well, then why should I not simply use Python instead of your template language?” My wish for the future of template languages is that they do/don’t do the following:

1. Either make more use of CSS templates, or use a similar CSS ‘#’ macro syntax which can inherit from a CSS template passed into the template language. That would be so nice. CSS is not going anywhere anytime soon. And let’s face it, it rocks if you know how to use it.

2. Don’t just brag about how similar it is to Python, and how fast it is. Show me how it can help me truly separate business logic from content, without adding a level of obscurity.

3. Make it support DOM2, please! This one is personal for me, since I am doing DOM2 server-side manipulation in my current project. A template language with a good DOM2 structure, possibly one which can be passed to other Javascript libraries such as Dojo, would be so very nice.

Comments are welcome. Examples will be incorporated into this article with author credits, so please post or email some useful examples.

Did you work with a template language not mentioned here? Please let us know about your experience.