apk zipalign

I’m getting ready to work on a large Zope 2 system that we’ve developed in house over the last few years. This system is a bit of a framework in its own right, with some inspirations coming from the “Component Architecture” influencing bits of the CMF and also from Zope 3 itself (while Zope 3 was being developed). This internal framework has long since moved from code managed through the web and into Python based products. It’s fairly modular, and we’ve even been able to bridge a couple of very different systems together using some relatively smart architectures.

I was looking at some code I wrote for this system. This code is comparatively recent and lightweight compared to what is in the core. I noticed something as I was looking at it though – I felt really claustrophobic. It’s not that it’s bad Python code or anything, really. But it was just claustrophobic. Methods were too long. Lines were too long! There were too many strings, strangely enough…. There was just something off. And again, I must say that it’s not bad code. It’s not being too complex, too tricky, too verbose, nor is it riddled with shortcuts. There was just a gut feel that didn’t sit quite right with me.

I decided to do a quick comparison against some of the recent code I’ve written for Zope 3 based applications. I decided to check against some of the more hairy “written under deadline” code at that, and I tried to find comparable situations (doing custom adds / edits). The verdict? The Zope 3 targeted code did not feel so claustrophobic! At all.

Now, maybe it has something to do with the older code feeling more alien and the the new code feeling more fresh.

But I think it has to do with something else that I’ve noticed about Zope 3.

I think it’s easier to chunk code up into smaller units of work in Zope 3. I think it’s easier to refactor by doing common refactorings like “Extract Method” and not have to pass as much information around. It’s easier to look up adapters and utilities that provide certain functionality instead of having all of that functionality heaped onto an object through a large inheritance tree.

And it seems like it’s really easy to abstract out common concepts into helpful tools and utilities, which grow into ones own helpful module, toolkit, or framework. At least, it’s easy in comparison to Zope 2.

I certainly know that I’ve been able to easily extract out helpful tools from common actions and update my Zope 3 based code to use those abstractions. And every time I do it, my code gets tighter and more powerful and then taken for granted since we no longer have to worry about it.

For example, a few months ago I worked hard on making a nice “undo” interface. Many ZODB storages can undo actions at the database layer. As I saw applications like GMail provide a nice “Message discarded (Undo Discard)” message for certain actions, I thought “I’d like to provide that, and hey! the ZODB can already undo things for me so I don’t have to track them myself!” So I went to work on providing a nice undo message. I ran into some snags along the way (related to things like how I was using sessions to send page messages to the user), but soon got it all buttoned down nice and neat. I was able to wrap it up in a nice API and handle deletes like this:

        name = zapi.getName(item)
        message="Deleted %s" % name

        del self.context[name]
        undomsg = cmsapi.undomessage(self, message)
        cmsapi.flash(self, undomsg)

After I checked this work in, I also added it to our oldest Zope 3 code, a content management system, in just a couple of lines, and got rid of an annoying Javascript alert box in the process. And then I promptly forgot about it until I was watching a co-worker using the CMS yesterday and saw the “undo delete” message. And watched it work flawlessly, with even the undo being undoable.

The point of this? Zope 3 eases refactoring and restructuring. I’m still amazed at how quickly our content management system came together, and how easily it was to separate out the stack into increasingly vertical levels. We almost didn’t even have to think about it. During post-mortems after major releases, consolidating code that had been copied and pasted into three different customer libraries into a single unit, without breaking our customer’s sites, was almost too easy. And on more than one occasion, just after such a restructuring had happened, a feature request would come along that was easy to apply due to that recent restructuring. “We need to make all of these things cacheable.” “Wow, a week ago that would have been four times the effort had we not boiled all of this stuff down into a core package and updated everyone already.”

And I think that’s the difference I notice between the code I was looking at today from our well-factored Zope 2 efforts compared to our more recent Zope 3 ones: we’ve been able to get rid of a lot of the line noise in our Zope 3 code, and much of the good Zope 3 code (like ‘formlib’) does a good job of clearing out line noise already. Fascinating.

Today I added to my iPod collection with the iPod Shuffle. I actually wanted to get it last night, but a late day at work caused me to miss the store being open. A couple of other important errands got bumped back too. The plan was – get iPod Shuffle, go skiing. Since here is something nice and simple that can (theoretically) be operated with gloves. Or even, perhaps, by just hitting the middle of your chest. At some point soon I’ll have to test this theory for real. But in any case – I’m liking the iPod Shuffle conceptually right now. I have a playlist that I’m listening to quite heavily these days that just barely fits in the shuffle’s space. It has better battery life than my aging 3rd Gen iPod. It probably responds to cold environments better too. And it’s so tiny!

One problem. The Shuffle is geared towards USB 2.0. USB2 is only a feature on very recent Macs (within the last year or two). My main iTunes library is at the office, which is an older machine (dual 867 G4 tower). It’s not that old of a machine. But, no USB2. So I can fill it via USB1.1 at 12MBPS, and holy hell – slow.

I’ve been considering replacing my aging 3rd Gen main iPod with a new video model if I see a good price (refurbished, etc). It’s only been lightweight consideration so far. Today, however, I found out that the video (aka 5th generation) iPods and iPod Nano will not work with Firewire at all. They can charge via firewire, but need USB to be recognized by a computer.

It looks like it might take close to an hour for this 1GB shuffle to fill. Some of the slowness may be related to an extra option I have set, “Encode higher bit rate songs to 128KBPS for this iPod” to squeeze space out of it. But still – this is ridiculously slow.

I can’t imagine trying to fill a 30 or 60 GB iPod over USB 1.1. Yet if I were to get one and still expect to use it with the music library I have on my office machine, that’s the only option I have (short of buying a USB 2 card). This is a strange direction for Apple to take. I mean – I guess it is obvious that USB 2 has equalled or beaten Firewire in many markets and is used heavily in the ‘Wintel’ PC crowd. But still – to not offer support for Firewire at all? If I were still using my old home iMac G3, I’d be furious. There’s no easy upgrade option for it or for many fairly recent (2-5) year old Macs. I find that very strange.

This weekend I had the luxury of free time for what feels like the first time in a good while. After some discussions on the Zope 3 mailing lists this weekend I decided that I wanted to try some new development options in the application I’m working on. Normally I do my development on a server at the office and sometimes work from home via SSH, but I wanted to set up a good local copy on my desktop so I could try working more in the TextMate editor, and keep the number of connections to work to a minimum.

This led to me thinking about how to get all of the supporting libraries for my app checked out from our own repository and a couple of third party subversion repositories. A goal of mine is to start getting our release management under control, so this line of thought led to me starting work on a kind of packaging tool. I doubt it will ever be public – it’s not meant to compete with eggs or zpkg. I needed something simple so I could check out these five packages.

I first thought about how I was going to define the map – what needs to be checked out, from where, to where, what revision / tag, etc. Should it be YAML? XML? ZConfig? My own syntax? I looked at YAML first, but it looks like to use it best it needs to go through C bindings. I thought about XML, but I’ve never gotten so insanely comfortable with XML parsing that I could come up with something quickly. As I looked at the other options (including YAML again), the translation layer between some text file in some language and native (and potentially rich) Python types seemed too big, at least for me to want to deal with at the time.

Something that’s been in my head a lot lately is the content of the Snakes and Rubies presentations about Django and Rails. One thing that David Heinemeier Hansson mentions a lot about the design of Ruby on Rails is that Ruby makes it easy to do “domain specific languages” within ruby, and Rails takes advantage of that by using a fairly natural syntax in elements like Active Record models that creates a lot of methods on the fly. I believe that in Ruby you can fairly easily interact with the class while inside of the class. In Python, this isn’t that easy. Well, it’s not easily exposed. But it can be done, albeit with tricks. I’ve seen this in Zope 3 – there’s been a move away from this style of programming:

  class IFooContained(IContained):
      __parent__ = zope.schema.Field(
          constraint=ContainerTypesConstraint(IFooLib))

  class Foo(object):
      __implements__ = (IFoo, IFooContained)
      __used_for__ = IBar

To this style:

  class IFooContained(IContained):
      containers(IFooLib)

  class Foo(object):
      implements(IFoo, IFooContained)
      adapts(IBar)

And with metaclasses, decorators, descriptors, and so on, it’s certainly possible to do more domain specific language style things within Python while not straying far from being “Pythonic” (for all of the mystical meanings of that word).

So I decided to give it a try: using the Zope 3 component architecture (but none of the Zope 3 app server or web specific parts), could I write something that let me define a release map in Python that was largely declarative but showed structure? Could I still have it validate? Could I have it turn certain keyword arguments from a regular string into a templated string without the map being filled with things like template("${src}/bar"). Could I have those templates allow for namespaces so I could easily access environment variables like ${env:CVSROOT} ? It turns out, the answer is yes. It took me a while to get it all together, including a couple of start-overs to simplify things. I wrote a Python file with a couple of classes / declarations that was my model for what I wanted and worked from there. It changed slightly throughout the day and night, but at the end of it, I have something like this, and it’s working:

from mypkg.api import *

class DemoApp(Registrations):
    zope = Subversion('svn://svn.zope.org/repos/main')[
        source('zc.catalog',
            repository='Sandbox/zc/catalog',
            target='zc/catalog',
            revision='37377',
            #post=command(tools.touch, 'zc/__init__.py'),
            ),
        ]

    codespeak = Subversion('http://codespeak.net/svn/z3')[
        source('hurry',
            repository='hurry/trunk/src/hurry',
            target='hurry',
            revision='18433',
            ),
        ]

    personal = CVS('${env:CVSROOT}')[
        source('demolib', repository='Support/demolib', target='demolib'),
        source('demoapp', repository='Applications/demoapp', target='demo'),
        ]

if __name__ == '__main__':
    main(DemoApp)

There are a few things going on behind the scenes to support this so that even a lot of the front end code in main(registrations) is really quite straightforward. A factory method style design pattern combined with use of zope.interface and zope.schema to validate and/or convert values seen above into what’s wanted inside the system (like turning ’${env:CVSROOT}’ into a templated string) takes care of the parsing problem while helping keep the map code clean. A little bit of metaclass-ish intelligence turns Registrations based classes into easily accessible data structures. A couple of simple adapters help preserve an easy interface on how some items find each other or get formatted, and keep the registration information separate from the objects that are ultimately created from the registrations, and when the system is finished a few more will provide the full command line interface options and the relationship between those commands and what’s produced by the map above.

My thoughts after this? I don’t know. The project is half done and may still change. In general, I like it. Even without falling back on meta-classes, I’m trying to apply this pattern in many places, to use intelligent data structures, contracts, and collaborating components to do more things declaratively where applicable. The Zope Component Architecture makes this very nice – define a contract/interface like IRepository, and then start providing adapters for it to provide things like command line formatting, log querying, etc; or to parse or format schema fields for a plain file representation.

I’ve had my eye, idly, on the TextMate editor for Mac OS X for some time. I don’t do much editing on my desktop – most development is done on a development server and all the glory of VIM (I admit it – I converted from EMacs a couple of years ago and have been generally happy every since). But I’ve been tinkering a bit more with things at both home and office and I’m always looking for things to make my day better, if possible.

For what little desktop text editing I’ve done in recent months, BBEdit’s free little brother TextWrangler has been decent for me: code coloring, decent editing support, but none of the automation or contextual intelligence that BBEdit might have. Having used BBEdit since “back in the day” (1996, perhaps earlier), I’m still fairly comfortable with it but am not comfortable using it for long periods. The classic Mac OS feel still bleeds through, and I’m still not sure how I feel about how TextWrangler implements “tabs”. I like the single window interface option, but switching between open files is unpredictable and seems based on recent access rather than the order they appear. There may be a setting somewhere to change this behavior, but I haven’t really cared enough. For how little I need to use it, it’s worth the free price.

TextMate, on the other hand, is a fully native Mac OS X editor built on the Cocoa framework that descended from NeXTStep. Cocoa has long had extremely nice text editing options that all Cocoa based applications pick up automatically. I forget that people still whine about lack of spell checking in web browser text areas. Why? Because Safari’s textareas use Cocoa’s text editing, and all Cocoa text areas pick up on the spell check options automatically. All Cocoa text editing also supports some basic EMacs navigation commands – control-A and control-E for beginning / end of line, and even the Emacs style cut and yank (control-k and control-Y) which operates independent of the normal system clipboard. Good for quickly moving lines around. So while all applications can stylize things differently, there is a lot of very powerful and flexible text editing / viewing / manipulating features built in that work the same whether you’re editing an element in OmniOutliner or setting notes on an iCal event or using TextEdit – the free plain and rich text editor that seems to serve as a showcase for all that the text system in Cocoa can do. For example, when I upgraded to Mac OS X 10.4 (or maybe it was back with 10.3?) that there were named style options in TextEdit, which came in handy right away for some small documents I was working on. When I opened up Aquaminds NoteTaker after doing the upgrade, I saw that the same named styles system was available in there as well. And it appears that in the Mac OS X 10.4.4 update, released recently, that the text system has learned a new trick: rectangular selection. I can clip some code from the Python interpreter running in Terminal and paste it into just about anything – Safari, TextEdit, TextMate, Stickies even!, and by holding down the ‘option’ key (the mouse takes on a crosshair cursor) I can select and cut the leading characters of the interpeter (the >>> and the ...).

So – Cocoa’s text editing system is cool. Applications can all do their own thing, but almost all of them offer the same great editing features that are far and beyond what you see in other major operating environments. What does this have to do with TextMate?

I found out today that there’s a command in TextMate to install a Edit in TextMate… command. It installs an Input Manager which is a way for other tools to augment the text input system. On my desktop, I have a very old “platform completed input” manager, which I never really used. It was released in the early Mac OS X days, I don’t know by whom. Besides, auto-completion is now a built in feature of Mac OS X text input. I always thought it was a cool example of how the system could be extended. TextMate plugs in using this system, adding a new command and shortcut to the Edit menu. So right now, I fired up Blogger (which doesn’t support ‘rich text’ editing in Safari) and hit control-command-E and boom! TextMate!

TextMate is also a fascinating example of Mac / Unix integration. It’s a very good looking application. Text, of course, renders beautifully. And it’s incredibly extensible. It’s not a super-programmatic environment unto itself like Emacs. Instead it uses environment variables, shell commands, and launches into different programming languages (Python, Ruby, Perl, Shell Scripting) to do operations on text. So when I fired this entry up out of Safari, I can write away here in Textile. There’s a command to convert it to textile which just runs through Textile.rb. And then, just for fun, I made a command that uses html2text.py (which is in the Markdown bundle for TextMate), and I can turn the Textile source into Markdown, which is done through Python. I can convert it back to HTML from the Markdown bundle which uses Markdown.pl, which is (obviously) Perl. I’m impressed with how nicely this all flows together.

Why is it so exciting though? Good question. I’m about to do some hard work over the next couple of weeks on a Zope 2 based product, which has a lot of its major cough “logic” moved into disk based Python components but has some pretty hairy templates and a thin layer of Python Scripts in the ZODB. Most ZODB development is done through the web, although there are FTP and WebDAV options (TextWrangler supports direct FTP editing). There are advantages and disadvantages to Zope’s through-the-web capabilities and this isn’t a debate over those. I’m just looking forward to being able to use Zope’s admin interface (the ZMI) to go to the template I need to edit, hit command-control-E, and be able to edit it in a full featured native 100% Mac OS X editor.

For the last couple of weeks (last few days of the old year, first few days of the new), I've been working on a small application written in Zope 3. This time it's a Knowledge Base application. A simple setup, in general - add articles to a container, organized by tags (keywords). Some of it has been easy, some of it has been a bit trying. It's taken a while because I've been using this as an opportunity to explore some of the new features of Zope 3.2, which was released in final form recently, as well as some available components that I haven't been able to take advantage of until now.

Good: ZODB. The ZODB is still the best option for persistence. With Zope 3, separating out the persistent core model objects is a cleaner and more obvious design solution than it was in Zope 2. Unlike an object-relational mapping, the ZODB is pure python persistence and there's no translation needed from database values and Python values. Where does this really stand out? It's most obvious in container / contained relationships, which is the most common type of one-to-one mapping. Hierarchies are easily represented, and most containers are used just like any Python mapping (dictionary-style) object. But where else does the ZODB show its strength? When using Python values like tuples. 'keywords' can just be a tuple of strings. There's no need to create a lookup table to map an item in one row to one or more keywords. foo.tags = ('silly', 'example') saves as Python, loads as Python.

Mixed: SetIndexes. The only way to get these great plug-in indexes for Zope 3's "catalog" system, which provides indexing support to Zope 3 applications for rapid querying of content data, is to check them out from Subversion. The indexes that come with Zope 3 are barely-there, it seems. But they do their job competently. The zc.catalog code provides new indexes that are easier to query and provides some new features. The best one - SetIndex. With 'tags' set up as a SetIndex in the catalog, searching for AnyOf tags for an iterable value, like ('silly', 'example'), quickly returns all items whose tags (which can be adapted to) have either of those words in their set. The SetIndex works great. I hope these indexes can find their way into the core or into a more available release.

Mostly Good: Viewlets. Generally, I like viewlets for designing applications like this. Existing documentation is very developer heavy. But at least that documentation exists and I can (generally) understand it. Viewlets make it possible, at the very least, to componentize web pages. Knowing the best practices for how to use them or their managers is a little vague right now. In general, my experience with Viewlets was really good. They were handy for handling menus, little boxes on the side (like 'search' or 'related tags'). They can be applied to only certain content items, or even certain views, or even more constrained within that. They make it easier to pull a lot of "display logic" out of master templates and into Python classes with specific responsibilities. They make it easier to pull a web page (or different pages) apart into a specification that says "this is the header viewlet manager, this is the sidebar. Register for the ISidebarManager to put a viewlet in there". Then a new viewlet for the sidebar, such as a search box, can be written and registered for the ISidebarManager months later and should slide in easily, using the component architecture instead of brute-force templating or vague conventions.

Good: Annotations. "Annotations" are how other components and systems can add specialized data to a content object without interfering with that object's data. One of the nice things about relational databases is the way that tables can easily be made to "relate" to other tables. In a RDBMS content management system, a lot of the Dublin Core metadata spec would probably be kept in a dublincore table with columns for the different elements, and all content items might have a "dublincore_id" column to facilitate joining. In Zope 3, Annotations provide similar one-to-one mappings. This isn't contained data, per se. It's extra info for an object. Annotations are kept in their own namespaces, and everything that uses annotations maintains its own namespace. All that a content object has to do is declare support for an annotatable interface, the most common being "IAttributeAnnotatable" which stores the root annotations for that content object on a special attribute (__annotatable__ or something like that). The nice thing is that all of these internals are kept hidden. The component architecture is used to access the annotations. A ZODB based object can easily support this extra attribute, but an object from an object-relational mapper or some other data source might not. A different annotator could be provided for those. Client code shouldn't have to worry about how the annotations are stored (or even if annotations are how the code is being stored).

Good: Formlib. zope.formlib is a new feature of Zope 3.2. It's a more powerful and generalized form system than the one that's in zope.app.form. Note that neither of these form libraries are responsible for field specifications nor widgets. It does provide a lot of useful functions, classes, and decorators for getting and validating input from widgets against a set of field specifications, for rendering widgets, for handling errors (including invariants), and handling actions (different "submit" buttons, how to handle input errors if 'Preview' is clicked instead of 'Save'). Nicely, it puts a lot of this power into Python classes and not configuration. It allows for, I suppose, beautiful programming. My limited experiences with formlib have been more rewarding than my experiences with zope.app.form.

Frustrating, but good I guess: Custom Traversal. Zope, going back to Bobo, has always been a strong proponent of "nice URLs". In Zope, you traverse to objects to publish them. This has always been the core concept of the system. And you could always provide some custom traversal, but the APIs for doing it in the past were vague and increasingly forgotten. There were a couple of options available to me for doing these tag URLs, where the tags in the URL were just to be used to construct a catalog query. One option involved just getting all of the names past a certain point and chopping them off of the traversal stack, ending traversal at the root of the 'tags' search. This was easiest, but not really fitting with the object publishing style of Zope, since the 'tags/foo/bar' wasn't publishing an object at 'bar', but at 'tags'. I replaced this first implementation with one that actually published objects for each tag in the URL. This wasn't necessarily easy, at least not when compared to things like Django's URL Dispatch. But now that it's in place, I like it better. Perhaps I'm just too used to Bobo's concept of object publishing. The biggest frustration was that there are a few traversing APIs and adapters to choose from, but each does different things, and finding the one to use for URLs specifically was not obvious. Even more frustrating - I think I just realized a more Pythonic way to write my tag traverser (basically to allow 'getitem' use, so that ...tags/foo/bar/baz really turns into Python (or could be done in Python with) ..tags['foo']['bar']['baz'].

Bad: This Post. It's past midnight on Sunday. I'm posting this primarily because I've been promising to write something about my recent experiences. They've been mostly good. I'm having a lot of fun with Zope 3. There are some frustrations from time to time, and it sure could use a good site housing tips and tricks and recipes and articles like "New in Zope 3.2 - formlib. What it means to you as a form developer". I haven't given very much back to the community, but if time permits (which it seldom does), I'd like to provide more things like that in a better atmosphere than the one provided by this blog. I make no promises. Time is a rare commodity between personal, professional, and artistic obligations. And NFL playoffs and ski days.

After ensuring some existing code would work under Zope 3.2 (still in beta testing at this point), I've been allowed this week to start work on a small internal (for now) project. I decided to take this opportunity to start looking at some new Zope 3.2 features. And while my experience with any of these is severely limited at this point, I want to mention them because they're all very interesting.

Content Providers, Viewlets, and Viewlet Managers

Two new top level packages in the Zope 3.2 distribution are zope.contentprovider and zope.viewlet. They're related, with the components in zope.viewlet building on zope.contentprovider.

What are these, and why are they important? In short, content providers are a simple construct that can be used to modularize web pages. Zope Page Templates have provided a decent way to modularize pages, using slots and macros to allow writing into multiple page parts more easily than a basic "insert header / body body body / insert footer" style template system. But it doesn't benefit from the flexibility that the component architecture can provide. As I've covered recently, A view in Zope is a multi-adapter, which discriminates against context (the object being viewed) and the request (HTTP request with skin information, FTP requests, and so on). A content provider adds to that, discriminating against the view as well. It's a smaller view used for that bigger view. More common terms for this are things like 'portlet'. There are some pages that you have, I'm sure, that you look at and you can see clear boxes that you are rendering in the common template, in included files, or being forced together a bit clumsily. You might be thinking to yourself "There's a lot of logic involved in rendering those navigation menus. I've got this great component architecture behind me, and I'm pulling that into a page with a 'var' and hoping it's there? Agh. And I don't want that navigation to show up like that in all of my shopping cart views... I need a different set of information over there during checkout..."

It appears to me that this is what content providers (and the viewlets built on top of them) are meant to solve. Based on JSR 168 - Portlet Specification, a content provider must implement only two methods: update() and render(). update() is the chance that the provider has to respond to the request and change state. It's always called first. Then render() is called to, um, render. On its own, it doesn't seem impressive. But its simplicity belies its power. zope.contentprovider doesn't provide an implementation, only some Interface specifications and a TALES namespace for using content provider components in Zope Page Template documents using the registered name. The component architecture will deliver the right component for the page being rendered. One has to look at zope.viewlet for base implementations.

zope.viewlet provides a common content provider in the form of a ViewletManager. The ViewletManager is probably the most common content provider one might call directly in a template. Viewlets are then registered with a ViewletManager, which will filter, sort, update, and render each Viewlet. A viewlet adds a forth discrimination - the viewlet manager. It all still builds on the component architecture - it's still multi-adapters underneath it all. So how might one use it? One example is the good old "side column" part of a page. With viewlets, one might register this as a viewlet manager with an interface name like ISideColumn or IAdditionalInformation. Then, one might provide a small viewlet for ISideColumn for all pages that shows the title, created date, modified date. Let's then say that you install a system that can find related content based on the tags or text of the current object. You could write a new viewlet that lists those pages in the ISideColumn - but only on pages that implement 'ITaggable'. The original template or viewlet manager don't have to be modified to render this, or to have any logic in them to discriminate against 'ITaggable'. The component architecture does the bulk of the work to find viewlets that can be drawn in a particular manager, and a particular viewlet manager implementation may do its own sorting and filtering of those viewlets - the base manager class provided in Zope 3.2, for example, filters out viewlets that the current user can't see.

What does this mean? What are the benefits of content providers / viewlets? I'm only speculating, based on very little personal experience so far, but I'm impressed with the simplicity of the interface. So here's a bulleted list of where I expect to benefit from this, and where I think others could too:

• Easier to specify page parts for current and future development. Build the page out of ViewletManagers, which are backed by Zope Interfaces, and then provide page parts against those. Navigation sections, promotional image displayers, menu items, footer links, sidebars, main areas. You can find these parts and formally specify them, and then provide the viewlets to flow into them.


• More intelligence in small parts. Viewlets may be simple or complex, but basically they can now have the same great (or small) intelligence that full view components have had. You know how it is - the middle left column of the page is expected to provide a search form, a navigation menu listing local pages, and a couple of small banner ads. Now all of those pieces can be more easily coded as separate components, but all in the same manner. This is possible in Zope 3.1, but content providers make this a lot easier.


• From a single code base, provide unique viewlets for the home page, section fronts, and common templates. This is another common little thing - lets say you have to have to add a hidden field to the little search form box used on every page, and the one on the front page needs a pull-down menu to allow discriminating results against different parts of the site. Normally, this little form is a small bit of code that's just managed in the templates. No big deal, there's only three - but they're all slightly different. You can't just copy and paste without paying attention to the unique requirements for each scenario. Putting that search form into a viewlet, you'd have access to the code used in each. Maybe you're using the same in each one. You can now make a specialized viewlet for the home page that adds the drop-down menu, while keeping it all tied together on the search page. You can also ensure that the 'search' sidebar widget does NOT draw on the search results page, or provide a more advanced search sidebar to filter results.


• Make a viewlet manager to handle HEAD links and easily add extra javascript library or CSS loading to pages that need them. I've commonly made page template 'slots' to allow me to load or add page-specific javascript or style sheets to individual templates. Using a viewlet manager for these, you can ensure that 'foolib.js' is there for every IFooListing based page.


• Use Page Templates, use another template language, render HTML directly - it doesn't matter. The interface is so simple, you no longer have to wonder about whether to use __call__ or __str__ or render as the best way to render a little component. 'update()' and 'render()' are all you need to worry about. Yet they can all plug into a larger page with ease, regardless of how that page is rendered. There are niceties for ZPT to make using content providers easy, but one could easily make a big "standard template" without a template language at all - just call and render a few principal ViewletManagers.


• Caching. There hasn't been a formal way of being able to cache fragments of pages. There's decent caching support (with more pluggable) in Zope 3, but no easy way of being able to apply them to parts of a page. With viewlets, it should be easier to put in caching. With more intelligence put into a viewlet, it could manage its own cached data or output. More effectively, one could put this in at the Viewlet Manager level without having to effect individual viewlets. The spec for content providers is simple enough that this could be plugged in a lot more easily and as needed.


• Based on simple standard. It's impressive to see that this is based on JSR 168. When I first heard this, some time ago, I flinched. JSR specifications are from a world that I just don't understand anymore, it seems. Yet this one is nice and simple and effective. It also should quiet some not-invented-here criticisms of Zope 3. At the same time, it has been specified and implemented in a way that fits with Zope 3. When I first looked at content providers and viewlets, I flinched then too. But as I started to use them, I realized that they didn't differ much from what I had already learned about views, widgets, and other multi-adapters. I've made 'views' with the express purpose of being rendered as part of a page, but they haven't followed a common interface. There are many other elements I see now that can fit into this system and ease use.

zope.formlib

The other addition I wanted to cover, briefly, is zope.formlib. Formlib is a general toolkit and API for doing intelligent web form development in Python. It's not a widget library - most of those are provided by the widgets in zope.app.form.browser, but could come from anywhere. It's not a validation framework - that's provided by zope.schema based fields (which work with Zope Interfaces). It does use both of those, and provides a lot of extra features to collect all of the input, apply the validation or collect errors, do invariant based validation ('end date must be later than start date'), dispatch to different responsive actions, and more.

It's a toolkit capable of building powerful base classes and individual uses for forms that are simple (just a few fields) or very complex. There were some form views provided in Zope 3.0 and 3.1 that do similar, but they weren't so easy to use and understand and didn't handle complex forms. Custom field and widget combinations required a lot of custom work, and so did dispatching to multiple actions based on success, failure, and so on. Lastly, the older (non-formlib) techniques seemed to prefer being built via the Zope 3 component configuration language when it really required Python to get the kind of flexibility that form development requires.

In wrap-up, formlib is promising because it shows a preference for using Python and some of the recent features such as decorators over giving everything over to configuration based automation. Content Providers are a simple specification for modularizing web pages into manageable components, with viewlets and viewlet managers providing a reasonable base implementation to work with. The dual layers provide a boundary between a basic spec and small set of tools (the content provider interface, a TALES expression for use in zope page templates) that have broad usage and more specialized providers, built from real-world usage and previous art and examples like JSR 168.

My previous post showed how Interfaces and Adapters provide the heart of the Zope Component Architecture, which beats at the center of the Zope 3 framework and application server, and is also pumping new life into Zope 2. The other chief piece of the Zope Component Architecture are Utilities. Interfaces allow coding to a specification, which can be as detailed or as general as one chooses. Adapters allow for other objects to fit that specification. Utilities are more general purpose tools that match a specification. The rest of the core 'zope' packages enhance or build on these facets, and the 'zope.app' package provides the Zope application server which is the most akin to the Zope that people know and love or hate.

Yesterday I showed how a very simple web publishing system could be made using just the basic pieces of the component architecture. Today I want to augment yesterday's example by showing how the component architecture provides "the one way to do it".

Kevin Dangoor, the primary developer behind the Turbogears project, wrote a great post today about What Turbogears Is Not. In it he discusses how choices are made about what goes into Turbogears, which prides itself on not being new technology but a combination of already offered Python toolkits. Kevin goes into coverage "one way to do it" by mentioning how the Turbogears widgets system uses Kid, the primary template system used for Turbogears. It's the default and chosen template system, so why not take advantage and say "this is how we do widgets. period," right?

Anyways, it got me thinking about what I said yesterday about the Zope Component Architecture and how everything flows through it at some point. 'Multi-adapters', especially, have become powerful. Most commonly, they're used to provide views for objects. Yesterday showed a couple of custom views built to display simple and detailed information about racing greyhounds. Today I want to show how widgets work.

First off - I did a little bit of code cleanup of the module and use zope.publisher.interfaces.IRequest and zope.publisher.browser.TestRequest instead of defining my own dummy request.

import zope.component
import zope.interface
import zope.publisher.browser
import zope.publisher.interfaces
import zope.schema

I'm not going to show the entire module again, but I want to bring back IGreyhoundData. This interface specifies what we expect any greyhound data providing object to give us:

class IGreyhoundData(zope.interface.Interface):
    """
    zope.schema provides a way of defining the properties and their bounderies
    as they make up a component. It integrates with the interface system and
    can be used for documentation, validation, data retrieval and setting, and
    more. The fields define 'get' and 'set' methods, among others, useful for
    querying and setting data on an object without having to have intimate
    knowledge of the object's structure. Useful for web forms, but also for
    database storage and retrieval.
    """
    name = zope.schema.TextLine(title=u"The Greyhound's Name")
    raceName = zope.schema.TextLine(title=u"The name on the race card.")
    fastestTime = zope.schema.Float(title=u"Fastest race time (in seconds).")
    trackLength = zope.schema.Int(title=u"The length of the track, in yards, of the fastest time.")
    retired = zope.schema.Bool(title=u"Is the greyhound still racing?")
    adopted = zope.schema.Bool(title=u"Has the greyhound been adopted?")

Lastly, I created a 'simpleConfigure' function in the top level of the module that registers the main views and utilities with the component architecture. This configure option can be run without any requirements aside from the 'zope' Python packages being in the Python path.

def basicConfigure():
    """ 
    Register the views and utilities with the component architecture. This
    could have been done in the top-level of the module, but deferring until
    now is a little bit cleaner and allows other configuration options to be
    made.
    """
    greyhounds = GreyhoundFinder()
    zope.component.provideUtility(greyhounds, name='greyhound')
    zope.component.provideAdapter(RacerView, name='view')
    zope.component.provideAdapter(DetailsView, name='detail')

All of the provided components in this setup also provide their own information about what interfaces they in turn provide and adapt. As a refresher, 'greyhounds' is an IDataFinder utility, used by the publishing system to look up the "model" object. The two adapters are multi adapters for IGreyhoundData and zope.publisher.interfaces.IRequest. They provide 'ISimpleView', which has a 'render()' method to render the view to the response stream. I want to reiterate that this has no dependencies on anything that the world thinks of as Zope. This is all in a single module that could very easily be published via CGI without ZODB, transactions, long-running-anything, security restrictions, through the web programming... None of it.. This just uses the heart.

However, showcasing the widgets used by Zope 3 does require a full Zope setup. Technically, I could configure the needed widgets myself and show how that's done, but that's beyond the point of this exercise. I chiefly want to show how the Widgets flow through the same heart. There's just one way to do it. To make a rudimentary edit form for a Greyhound in the little system I have set up, we create another class to provide the edit form view. I try to document everything that's going on. Take note of the use of zope.component.getMultiAdapter() to get the widget.

class EditForm(object):
    """ Requires zope.app.form widgets to be configured. """
    zope.interface.implements(ISimpleView)
    zope.component.adapts(IGreyhoundData, zope.publisher.interfaces.IRequest)

    def __init__(self, context, request):
        self.context = context   # aka, the racer
        self.request = request

    def render(self):
        """ 
        Goes through the schema fields and looks up the IInputWidget adapter
        configured for the field and request type. 'zope.app.form' is imported
        here to avoid placing burden on the rest of this module, since this
        code requires the widget configuration to have occurred. Just importing
        'zope' does not provide this, but running a Zope application server
        does.
        """
        from zope.component import getMultiAdapter
        from zope.app.form.interfaces import IInputWidget
        racer = self.context
        request = self.request
        out = []
        for name, field in zope.schema.getFieldsInOrder(IGreyhoundData):
            value = field.get(racer)

            # Just like getting a full 'view' to render, widgets are also
            # provided as multi adapters, adapting an object (in this case, a
            # field) and the request to an IInputWidget implementation. A
            # display-only widget would use IDisplayWidget.
            widget = getMultiAdapter((field, request), IInputWidget)
            widget.setRenderedValue(value)

            # A widget has a label, which it often gets from the field's title.
            # To render the widget, just call it.
            out.append('%s: %s' % (widget.label, widget()))

        return '\n'.join(out)

The first few lines of the render() method just help establish some shorter names, and also defer loading zope.app.form so it's not required by this whole module - although in theory it wouldn't harm anything if it were imported at the top. The zope.schema introspection machinery is used again to walk through the fields of an Interface object and get the fields. We get the object's current value by going through the field. Then we get the widget. The Zope Component Architecture comes into play once again as we try to adapt both the field and request object to an IInputWidget. The value is set on the widget based on the object's value, and we add it to the output.

How is that widget made? Does it use templates? Just write out strings in Python directly? It doesn't matter. The IInputWidget interface is all we care about here. Form submission and validation goes through the widgets and fields too, but that's not important for right now. What is important and interesting is how adapters are used here to deal with widgets, and they're completely independent from the schema. Using the 'schema' objects, one could write completely different adapters to do object-relational mapping and provide converters to read/write SQL data based on the object specification, while still using that same specification to render web forms or write out dynamic XML-RPC or JSON request responses. sqlformatter = getMultiAdapter((field, databaseConnection), ISQLFormatter) could easily be used. As a technical aside - with how multi-adapters and interfaces work, there could be generic formatters for all data types but a specific datetime formatter could be provided for a SQLite or Oracle connection. Multi-adapter lookup will match the most specific adapter based on the specifications of the objects passed in. See what I mean about everything flowing through the same heart?

It's time to put our new view into play. I added an extra configuration method that would set up the basic components and add in the edit form:

def configureWithWidgets():
    """ 
    Establishes basicConfigure and also enables the edit form which uses the
    zope.app.form system for widgets. Requires a configured Zope app server
    setup - at least with the zope.app.form widgets configured to provide
    adapters.
    """
    basicConfigure()
    zope.component.provideAdapter(EditForm, name='edit')

And with how our 'simplePublish' system is set up (the only change since yesterdays code is the use of zope.publisher.browser.TestRequest instead of the self-defined simple request), we now have edit actions:


>>> simple.configureWithWidgets()

>>> print simple.simplePublish('greyhound/edit/1')

The Greyhound's Name: <input class="textType" id="field.name" name="field.name" size="20" type="text" value="Betty Joan" />

The name on the race card.: <input class="textType" id="field.raceName" name="field.raceName" size="20" type="text" value="Beauty" />

Fastest race time (in seconds).: <input class="textType" id="field.fastestTime" name="field.fastestTime" size="10" type="text" value="31.22" />

The length of the track, in yards, of the fastest time.: <input class="textType" id="field.trackLength" name="field.trackLength" size="10" type="text" value="503" />

Is the greyhound still racing?: <input class="hiddenType" id="field.retired.used" name="field.retired.used" type="hidden" value="" /> <input class="checkboxType" checked="checked" id="field.retired" name="field.retired" type="checkbox" value="on" />

Has the greyhound been adopted?: <input class="hiddenType" id="field.adopted.used" name="field.adopted.used" type="hidden" value="" /> <input class="checkboxType" checked="checked" id="field.adopted" name="field.adopted" type="checkbox" value="on" />