apk zipalign

I’ll admit – I’m very tempted by Ruby on Rails. I like its “Ruby Everywhere” philosophy. I like its “beauty leads to happiness” philosophy. I like its “damn straight it’s opinionated” attitude. I like how easily it gives you an application starting point out of the box (oh, you don’t know how I’ve been struggling with this this week). At least, I like all of this in theory. I haven’t tried making any real application in it yet. I did a quick version of the Knowledge Base application, just to see some results and see how using Tags felt compared to what I’ve done in Zope 3, but it's nothing close to complete (or even to matching the Zope 3 implementation).

I’ve been working on another customer’s application in recent days and have been thinking “this would be a fun test to develop and compare!” The thought became more prominent as I’ve been working on a batch of customer requests – seemingly simple little things that are turning out to be much harder (in most cases) than I had estimated. If not harder, then just long. Now, some of this struggle is in Javascript and all of the joys of form/widget/etc design. Some of it is just data migration – extracting objects, etc. Replacing a single text attribute with a single sub-object with multiple fields. Combining a couple of field pairs on an object into two instances of a sub-object. And so on. That’s where some of the arduous part has come in – all of the ZCML and Schema required just to get a couple of simple classes registered and secured. And I had to roll a new ObjectWidget base class (I didn’t like the one that’s in Zope 3 at this time). With all of that out of the way, however, now I’m moving on… Mostly.

Anyways, I’ve started exploring how this application might work in Rails. And it didn’t take me long to find fresh love for the ZODB.

The ZODB is a full Python persistence system. It’s native. A tuple is a tuple, string is a string, a CrewContact object is a CrewContact object, and so on. When developing in Zope 3 in the (general) fashion that I’ve been developing – straight Persistent model objects, all of the views, adapters, etc, coming in off disk (looked up and bound by the component architecture, not by Zope 2 techniques like acquisition) – using the ZODB for a new application is straightforward and quite easy.

I think an object-relational mapper is good when developing against an established database. And usually there needs to be some strong mapping for that to work and match database policies against expected object behavior. Rails’ ActiveRecord is a really nice system in many respects, and it provides a lot of nice features for fairly natural expressions for establishing relationships, doing validation, etc. But it also seems to expect a fairly rigid structure in place to work. Not that this is necessarily bad. But it begs the question: if you’re going through such terrific overhead and making certain restrictions (Single Table Inheritance, which I’ll get to in a minute), why use an RDBMS at all?

Well, I know some of the answer to this, and I agree with it in part: the DBMS vendors (including open source ones) are better at the data storage, retrieval, distribution, and connectivity side of things. “Let the database management system handle that,” some say. “Let SQLObject and ActiveRecord make it feel like natural objects with a minimum of fuss.” OK. That’s admirable.

But I realized today that it’s been a long time since I’ve had to think relationally just to store my objects – objects that have meaningful relations. Specifically, I ran into this problem.

• I have a class for Itinerary objects. An Itinerary is a container, which can contain many Legs. There are three types of legs, sharing a common base class/interface.
  
  
  
  • ScheduledMeeting – Just a meeting that’s taking place in this itinerary.
  
  
  • GroundTransportation – A car service, etc. Who’s providing it, any other notes (where to meet for pickup, etc).
  
  
  • Flight – many fields (and in my latest implementation, sub-objects) with information concerning a flight, its crew, its passengers (crew and passengers linked lists to their respective targets), crew contact information, flight details, other notes.
  
  

I’ve been wondering how to do this in Rails. In Zope 3, I have a base interface, IItineraryLeg and specify that an IItinerary implementation contains them:

  class IItinerary(IItinerarySchema, IContainer):
      """ 
      Base itinerary information.  ItineraryLegs are associated
      with Itineraries to give trip details.
      """ 
      contains(IItineraryLeg)

And in the implementation:

    class Itinerary(BTreeContainer):
        implements(IItinerary, IItineraryContained)

        title = DublinCoreProperty('title')

        start_date = None
        end_date = None
        # Use ``Business`` as default, as is also specified in the interface
        trip_type = u"Business" 
        notes = u"" 

        @property
        def legs(self):
            return sorted(self.values(), key=startDate)

    class ItineraryLeg(Persistent, Contained):
        implements(IItineraryLeg, IItineraryLegContained)

        start_date = None
        end_date = None
        notes = u"" 

    class ScheduledMeetingLeg(ItineraryLeg):
        """ 
        'Scheduled Meetings' add nothing to the IItinerayLeg schema, really
        """ 
        implements(IScheduledMeetingLeg)

    class GroundItineraryLeg(ItineraryLeg):
        implements(IGroundItineraryLeg)

        transport_type = u"" 
        service_name = u"" 

See, these two legs are very simple. In ActiveRecord or any object-relational mapper using Single Table Inheritance, this would have been easy. But it’s that Flight leg that gets messy. Very messy. By my understanding of Single Table Inheritance, the flight leg, ground leg, and scheduled meeting legs data would all be mashed up in one table. If I were designing tables in an RDBMS, I would never design that way – and I’m no genius Relational designer.

The point is, there are obviously situations where my expectations of objects AND relational storage, as mapped by ActiveRecord, don’t quite mesh up. I’m not belittling the system – I just realized that (a) I have no idea how I’d want to make these tables (especially since I have some equivalent of has_one to tiny little objects, a transition made this week. Not just to tiny little objects, but a couple of them. Has two?), and (b) after I got them built, I have no idea how to map them into ActiveRecord. I may, however, be distracted things that I may or may not need, like new features Polymorphic Associations and Through Associations.

I guess, after thinking about it, I would write the three legs as separate tables/objects and write one legs action to combine them (in Rails)?

  def legs
    (ground_itinerary_legs + scheduled_meetings).sort do |a,b| 
      a.start_date <=> b.start_date
    end
  end

Well, it let me do something like that in the console, but not in my active record class.

Anyways, the point is that for many (not all, but many) tasks where a new “web application” is being created, a system like the ZODB is great. No translation. No having to bend your brain around RDBMS table creation and styles, and then back to objects, and to what can be done in between. Python data, through and through. The object database doesn’t work for every situation. But damn, it can be nice.

I’ve spent way way way way way too much of the day trying to code in a couple of simple objects into a Zope 3 application. They’re ridiculously simple, for the most part, with a couple of specialized business requirements beyond them.

Some of the slowdown has come from me back-pedaling on a couple of design decisions earlier in the day. But as I slough through just the basic ZCML to get these stupid little persistent objects registered into Zope (after the amount of work it took just to get here), I feel another headache coming on. And I’m not even to the actual hard part of the work yet. And this task is still a small part of all that I had hoped to get done today. It’s something that really should have been done a couple of hours ago, and I bet I have an hours worth of work ahead of me on this one stupid task alone.

Mar 17, I'll be playing at No Fun Fest 2006, in the basement for Hospital Productions night. No Fun Fest runs Mar 17-19 at The Hook (18 Commerce Street) in Brooklyn.

Smegma, Ikue Mori, Zbigniew Karkowski, Sickness, and me, the always yodeling aodl.

Show starts around 7:30 each night, and I'm one of the first to go on. Come on down and watch me rock the trumpet tuning mic like you know I do it right.

... Update, 27 Mar 06: I realize now that this didn't get published until way too late. Blogspot was having connection issues when I rapidly tried to post this before I left town, and I forgot about it until today's update. oops.

A few months ago I defended ZCML pretty staunchly. These days, it's becoming feature #1 that makes me NOT want to use Zope.

For the uninitiated, ZCML is an XML configuration language to load up Zope components. I think that having registration separate from implementation, even in another 'language', is a good thing. But ZCML has also taken on heaps of automation capabilities. And those suck. Instead of respecting the Python programmer, it holds your hand and provides a lot of automatic features. Sometimes that's nice. A lot of times, that's wrong. You can't subclass, you can't override, you can't provide slightly different behavior. And don't even think that you can read the code behind the ZCML directives. Most of the code is full of internal handshakes in the configuration loading system. And if all you're trying to do is figure out whether you can provide javascript to a ZCML 'menuItem' directive: good luck trying to figure that one out.

So I stand firmly behind this: ZCML Needs To Do Less (although I don't agree with all of the examples shown).

A web application framework that truely fascinates me is Seaside. Seaside, written in Smalltalk, has great vision and ideals for web applications by using continuations to suspend and resume code. It breaks free of the typical request/response cycle, and manages state to a fault.

This code, from an example store application for Seaside, is the go method of the WAStoreTask object. This method runs the store. Stay with me now, this is important. This runs the store. This is the flow. It’s native Smalltalk. It’s not XML, YAML, An Array or list. It doesn’t get translated into any structure like that. Upon first visit to the application, this code starts executing.

go
    | shipping billing creditCard |
    cart := WAStoreCart new.
    self isolate: [
      [self fillCart. self confirmContentsOfCart] whileFalse
    ].

    self isolate: [
        shipping := self getShippingAddress.
        billing := (self useAsBillingAddress: shipping)
                    ifFalse: [self getBillingAddress]
                    ifTrue: [shipping].
        creditCard := self getPaymentInfo.
        self shipTo: shipping billTo: billing payWith: creditCard
    ].

    self displayConfirmation.

The isolate method calls wrap a block in a WATransaction, making that transaction run over multiple requests. I’m not sure if this WATransaction maps to how I’m used to them as a Zope developer, but I do gather that it cleans extraneous objects up that were racked up in that ‘isolate’ call.

The use of continuations and a sort of invisible ‘session’ management that runs behind it makes for very brief code. Looking at my own Zope 2 e-commerce application, I see many lines spent having to look things back up: getting data out of the Session, getting data out of the Request, getting data out of the Database(s), and so on and so on and so on. With each request/response cycle, especially late into the Checkout process (which depends on all of the accumulated data of the shopping experience), the list of dependencies gets bigger. Guards have to be in place to ensure that the session and account are still active late in the checkout process, and that going to one of the checkout pages with an empty cart is disallowed… So it’s interesting to look above and see just a small handful of expressions boil down the whole experience.

An aspect of Seaside that is especially interesting is action calling, via hyperlinks or form submission. I also like how it does HTML generation, preferring to use objects/methods instead of templates. This is from a WAStoreItemView. renderContentOn: html is a common method that all the views implement. Unlike the so-called ‘MVC’ that is being popularized in Ruby on Rails, Turbogears, etc, this is actually closer to Zope 3’s views wherein it’s an object that wraps (decorates) another object.

renderContentOn: html
    html heading: item title.
    html heading: item subtitle level: 3.
    html paragraph: item description.
    html italic: item price printStringAsCents.
    html form: [
        html submitButtonWithAction: [self addToCart] text: 'Add To Cart'.
        html space.
        html submitButtonWithAction: [self answer] text: 'Done'.
    ]

Still straightforward and readable. Especially interesting are the submitButtonWithAction calls. The first argument is a block that gets executed when that button is clicked. addToCart is straightforward enough:

addToCart
    cart add: item

The page gets rendered as normal, staying on the WAStoreItemView, but each time you click ‘addToCart’ you’ll watch the cart display component fill up and fill up. [html anchorWithAction: [cart remove: anItem] text: '-'] is in that cart display to, of course, remove the item.

Back to the WAStoreItemView renderContentOn buttons, the other interesting one is the button for Done. It calls self answer. What’s it answering? A call! Specifically, this call from WAStoreFillCart. WAStoreFillCart is the component that’s responsible for browsing, searching, and displaying items. Any link to displayItem: anItem calls this:

displayItem: anItem
    main call: (WAStoreItemView new item: anItem; cart: cart)

What calls this display item, and why does it matter? Think of most web applications – there are more than a few ways to get at the data inside the application. There’s searching, there’s browsing by ‘tag’, there are dashboards (big overview pages), there are personal pages, there are direct listings. When you go from one of those listings to something more detailed, how do you go back?

In Seaside, call and answer provide that functionality, among other things. Clicking the ‘Done’ button that calls ‘self answer’ returns control back up to whatever called it. So if you found the store item on page 3 out of 5 in a batched list, you’re sent back to page 3 out of 5. If you got there through a list of links from a search result, you’re taken back to those search results. It feels natural. The developer doesn’t have to maintain information about the calling page to return back to it.

Again, looking at my own big and old Zope 2 based e-commerce application, I have a few ‘handlers’ which are thin scripts over the business logic implemented in core components. Many handlers have paths passed in about which page to report errors to, and which page to go to on success. This allows them to be used from multiple pages, within the structure of the good old scripts’n’templates system (but the scripts are really thin layers to underlying code that does validation, processing, business logic, etc). Man, wouldn’t it be nice to have that be more automatic instead of some hidden fields like ‘errorto’, ‘returnto’ with paths?

All in all, what I’ve really liked about playing around briefly with Seaside is the feeling of liberation from not only the request/response cycle, but also from the Template model. I’m suddenly very tired of templates as string insertion languages being the ‘view’ (coff!). I don’t mind using them for the broadest of layouts. But whether it’s template attribute style languages or string insertion, I’m just suddenly tired of seeing and maintaining things like this:

  <ul>
    <li tal:repeat="item view/listItems">
      <a tal:attributes="href string:${context/absolute_url}/${item/name}" 
        tal:content="item/description">Description</a>
    </li>
  </ul>

  <ul>
    <% for item in view.listItems %>
      <li><a href="./<%= item.name %>"><%= item.description %></a></li>
    <% end %>
  </ul>

My problem with them is not so much the language itself, but the casual callous way important bits like URLs and more are constructed. I’ve generally been a big fan of Zope’s TAL (Template Attribute Language, used in the first example), but sometimes it’s too hard to do simple things. Even more frustrating, lately, has been the lack of good helper functions for use in Python expressions in those templates. Zope 2 has “ZTUtils” which has a small set of useful utility and objects. But if those are in Zope 3, I haven’t found them.

I’ve generally been against the PHP style of ‘templates’ where it all becomes super muddy (how many PHP ‘apps’ do you see with print/echo statements in the code that then have HTML inside of a string?). Most of the Python PHP/ASP/JSP style systems I’ve seen have been especially bad, primarily because of Python’s block structure and lack of ‘end’ statements or closing braces inside the language, so doing the ‘for’ loop like above has varied between different implementations. It’s just not what Python is built for, at all.

My experiences with Ruby on Rails has been better. Ruby has the ‘end’ statement. And it really is a nice concise language that doesn’t look all that bad in Erb templates. But what makes Ruby on Rails especially nice is the large collection of helpers. The helpers are a big collection of functions and objects (much bigger than ZTUtils) that makes it pretty nice to work in the template. So instead of the tag-within-a-tag style shown above to generate the ‘href’ part of the link, the common idiom in rails is to use this:

  <%= link_to article.title, :action => 'show', :id => article %>
  <%= link_to 'Destroy', { :action => 'destroy', :id => article }, :confirm => 'Are you sure?' %>  

This brings it closer to one of the things I like about the Seaside experience: you’re not writing into a string to make a link, you’re linking to objects and methods directly. Not only that, you’re using other options (like shown in the second example above) to do things like common Javascript events, or to do tricks to make a regular hyperlink perform an HTTP POST request. Ultimately, they make the development experience feel more like actual application programming. For the record, there’s a fairly direct Python implementation of many of Rails helpers available too.

Some of the things I’ve done for my Zope 3 development lately has been to take ideas from these systems. I’ve started building my own helpers system, implemented as a Zope adapter for the Request. Since the Request has ‘skin’ information in it, it’s possible to extend the helper with specialized functions for a particular skin, knowing that it might have certain javascript libraries or DOM identifiers / CSS stylings available. I also have a simple HTML builder system, inspired by Nevow’s Stan, but without the more advanced features like mutation, macros, or.. well, pretty much anything advanced. Applying a python getitem trick and making an object that generates any kind of Tag object by name, I can spit out pretty much anything.

With Zope 3.2 and the introduction of viewlets and content providers, I’ve been focusing on building smaller and smaller view components. I’ve stopped thinking of pages in the standard macro + content or header + content + footer sense and have gotten back into thinking of a page as a collection of connected yet independent view objects.

  html = api.htmlHelperFor(self)
  ul = T.ul(class_='menu')
  for item in self.listItems():
      ul << T.li[ html.linkTextTo(item.name, item) ]
  return unicode(ul)

  # Or, for more fun
  html = api.htmlHelperFor(self)
  return unicode(T.ul(class_='menu')[(
      T.li[html.linkTextTo(item.name, html.viewURL(item, 'delete'), confirm='Are you sure?', post=True)]
      for item in self.listItems()
  )])

Added to that, zope.formlib delivers form components with ‘Actions’ bound intelligently to methods. zope.formlib handles dispatching to the proper action instead of just having Zope’s publisher go to a method / view directly to handle forms:

    @form.action(u"Import Articles", condition=form.haveInputFields,
                 failure='handleFailure')
    def importArticles(self, action, data):
        # This gets called when the 'import articles' button is clicked, and
        # only if validation succeeds. The button only gets rendered if there
        # are input fields - a form can be generated to display data on a field
        # by field basis according to security, which can set up 'view' 
        # situations for unprivileged users - and you wouldn't want to have
        # an input button when there are no input fields, right?
        # 'data' has all of the validated form input, and can be easily
        # applied like this:
        form.applyChanges(self.context, self.form_fields, data)

    def handleFailure(self, action, data, errors):
        # Optionally, individual actions can have other methods associated
        # with them to handle action-specific features for validation,
        # validation failure, and so on.

Being actual objects, they can be referred to in the view or view’s template and filtered out if unavailable:

  controls = T.div(class_="controls")
  for action in self.actions:
      if not action.available():
          continue
      controls << action.render()
      controls << '&nbsp;&nbsp;'

I like having tools like this available. It really makes the application feel a lot stronger. ‘formlib’ is pretty simple, conceptually, while providing a lot of flexibility and features and strength on the ‘web application as actual application’ front. That’s the nice thing with Seaside and Rails’ controller/view layer and its helpers too: a lot of tools, some simple concepts, a lot of power.

This was the opening line in one of those oil scam spams: Let me crave your indulgence to introduce myself to you.

My brain is now stuck in a locked groove.

Somehow, Griddle Noise got selected as a spam blog or offensive blog or something along those lines. I'm trying to decide whether I should don the dynamite hat of paranoia or not. It could be fun: add to my crankiness a fear of everyone. "They know I'm right, dammit, and they're trying to shut me up?" "Who?" "THEM!" "Who?" "Them!"

If it wouldn't be so mind-crushingly depressing (and wasn't so out of budget), I think I could use a vacation. Remember that week I spent in a shot glass in the bay? Me neither.

I actually do have a small chance coming up to get away, but it's not all stress free like I need. It's sortof work. I'll be performing at No Fun Fest 2006, March 17th, down in the basement, around 8-ish I think, somewhere in Brooklyn where the trains don't go.

ls4+2. Hearts and springs are stupid things.

By the way, the crisis of faith mentioned in my last post may be minor. There is a lot about Zope 3 that I like. There’s a lot about classic Zope that I’ve liked. But I feel frustrated. That frustration is felt on more than one level too. I’ll get to the positives soon enough.

Again, I’ll mention that I’m not necessarily a good person. I haven’t contributed any actual code for more than a couple of years now. I don’t even run checkouts from Subversion. I must deliver results to my customers on as much trusted, supported, and released software as possible. I try to keep up on the general discussions on the mailing lists, but can only keep up on a few usually.

But I do have a passion for this. I want beauty, I want happiness, I want Python, I want Zope. I want others to want this and have it. But if I’m having frustrations, what are they going to feel?

Gripe 1, Message message message. There is no message.

Well, if there’s a message, good luck finding it. Now, a lot of people say “Python has a lot of web frameworks”. Well, I wouldn’t call every Python programmer’s one-off a real “framework”. There are a lot of options. But there are only two others that have a lot of buzz and momentum behind them, are usable today as complete options, and look like they might actually make it to a 1.0 release and beyond. Let’s look at what they offer (keeping in mind of course that it may change after the time of this writing):

TurboGears has a nice site with a narrative text and clear links to what’s important. You can quickly learn what Turbogears is about, and can quickly see tutorials that deliver something usable and understandable. I think what I like best about the current version of the TurboGears site is that the text is readable. It’s prose, but full of links. It’s not just bullet points (we’ll get to this in a minute).

Django also has a great site. The main focus is on some great introductory text (again, with links) about where Django came from. Elsewhere on the page, in nicely and subtly separated section, are links and text (with links!) about the essentials of the framework, documentation, showcase sites, and weblog entries about the project itself. The weblog side is small (title, description only). It shows that there is movement and momentum in the project, but it doesn’t overwhelm the page as the central piece of information. The newspaper mentality of the core Django developers shows through here. Like the front page of any good newspaper or news site (emphasis on the ‘good’ part), there are headlines and columns and information that opens up into inner pages, and it all seems pretty appropriately weighted.

And while it’s not a Python framework, a quick visit to the framework that has made the dynamic language set in vogue again, Ruby on Rails. It’s opinionated. It’s got big buttons – “Get Excited” – promotional screencasts, presentations, and more; “Get Started” – the download link (big clear ‘1.0’ with a release date underneath the big shiny button); “Get Better” – API references, books, documentation, etc, for both Rails and the Ruby language; “Get Involved” – community resources.

And what about Zope 3? Zope.org does have a nice introductory paragraph. Personally, I find it a little hard to focus on with a noisy lists and lists and lists of links design. Beyond the introductory paragraph, there’s not much real text on the page. “Zope 3” is buried down in the “Developers” section in the list of links on the left. On a 1024×768 screen, it may or may not even make it “above the fold”.

The Zope 3 page is full of many sins. It’s basically the development wiki for the project. Quick – can you spot the download link? Visit the ComponentArchitectureOverview page. Is it up to date with how things have worked since Zope 3.1? Back to the front page – is there anything on there your eyes are actually happy to land on and spend a few minutes soaking in?

The sites for TurboGears, Django, and Ruby on Rails all have wikis, but they’re not the prominent interface. They’re the backup resource. I believe they also all use Trac. The Zope 3 project is older than Trac. But I often wish it would find a way to move to it. Trac’s unification of subversion browsing, bug tracking, milestone managing, and more is a great feature. I love the turbogears roadmap which is just a Trac generated page. “what’s going on with the 0.9 release? Oh, there it is.” The Zope 3 Roadmap? – Nothing. It’s a good attempt at providing useful information.

Some systems provide proposal tracking, where proposals are documents with metadata that are attached to a milestone and release. Granted, this information is only as useful as people keep it up to date. But man, I wish I knew what is coming in Zope 3.3, or might be coming, and the status of it. A wiki can’t get me that information. It’s a terrible interface for information like that. How can I get excited? Excited enough to actually get involved and contribute? Excited enough to say “well, Zope 3.2 didn’t quite do what I wanted, but if that new thing is coming in 3.3 and it will be released this summer, I’ll definitely come back and check it out then.”

But can you get any of that information out of the Zope 3 Proposals page? No. What about the zope 3 subprojects page? Nope. What about the projects that have been discussed (or sometimes just mentioned in passing) on the mailing lists? And why should they go in the Wiki anyways? It’s hard to navigate. It’s hard to query. It’s hard to make information stand out. It’s hard to group associated tickets, subversion branches, etc in right with it.

Of course, I’m biased against wikis in general. With very few exceptions, I’m screaming and cursing within the first few links into any wiki. And contributing / editing is just as bad, as most of the “structured texts” of the world tend to really bite. The question marks that show up unwanted in text, the FunNyCapitAliZation (especially of words like ZoPe and PyThon) that can show up makes me cringe, and makes any text very hard to read as my brain has to stop and take the word apart and put it back together. Augh. Frustrating.

The thing is: the whole Zope 3 community knows this. There’s no good web site that gets people excited, shows motion, offers easy and obvious download(s), offers introductory tutorials, or takes a stand and says “Zope 3 Kicks Ass”.

And, well, there is no message. There’s no stand. Debates are still going on about whether to give Zope a new name. Whether to use silly release/code names (a terrible concept, I think). Whether Zope 3 means the core set of technologies, whether it means the full stack and half assed content pieces that ship with the current Zope 3 download (which basically provides the “Zope 3 Application Server”), or whether it means something on par with Zope 2 and offers a future for Zope 2 and Zope 3 as we may (or may not) know them today.

So no one can say “this is what Zope means”. They don’t know. I don’t know. And we can’t agree. Zope 2 offered so many ways of working with it that it was hard to agree. “You can develop your web site like this… Or you could do it like this… Or you could do it like this…”. Not that there is anything wrong with that kind of flexibility. But it’s ultimately too many features, too many options, too many different ways of doing things that aren’t compatible. And we suffer from this in Zope 3 already. Not as badly as in Zope 2. But … I don’t know. I really don’t.

I don’t know why whenever I start a new project / application I’m not excited. I always feel like I’m having to copy / load / configure / tweak / copy quite a bit, still, to get started. It doesn’t take that long, and I’m sure I can extract it out into a usable starting kit framework. But even for me, after all of these months working with Zope 3 and years of working with Bobo/Principia/Zope 1 and 2, it’s hard knowing the best way to start. There are “easy” features we could use. For immediate results, we have a couple of different options. But none of them grow easily into a real application.

Having those is a distraction. There’s a user set that I’m sure they’re useful for. But as I mentioned in the last post, I’d love not only the core simple concepts of Bobo (Python object publishing). I’d love one that was stripped of these distractions, but was otherwise Zope as we know it: easy container/containment relationships. Pure persistent Python content objects. But most of all, it’d be one that could be installed and used with a tutorial that didn’t have to spend time explaining “this ZMI thing” or “that i18n file add menu item”. The slate would be blank, the possibilities wide open. Concepts that don’t apply to the situation don’t need to be learned. For me, things that don’t concern me don’t need to be pushed out of the way. With this tool, you’d still be signing up for certain aspects of the Zope experience: the object database (or maybe a really well integrated object-relational mapper), etc. But you wouldn’t be signing up for the bigger set of tools that build on that. Put that in another release.

And a message. We need a message. We need to prove that message in code examples, tutorials, etc. The fact that it seems to be so hard to even come up with this message is not good. And after a night of wrestling with decoding ZCML directives and vocabulary lookup and binding to write a test to prove (and then fix) a bug in my code – many many many many lines of setup support in a test harness (setup support that acts nothing like it does when used in the application environment). The bug fix? 3 lines of Python plus 1 line of adapter registration. That seemed very disproportionate. And as I grumpily sat at work the next day, the lines from David Heinemeier Hansson’s presentation about Ruby on Rails at Snakes and Rubies kept running through my mind (especially as I started reading their PDF book, Getting Real): Beauty leads to Happiness, Happiness leads to Productivity. I’ve spent just enough time wrestling with things that I shouldn’t this past week for that to be very appealing.

For the first time since spring of 1997, I find myself having a crisis of faith regarding Zope. I’m not sure it remembers where it came from. I’m not sure it knows its own future. It’s suffering from a crazy identity crisis. And there seems to be a core effort to take it in directions that I really wish it wouldn’t go. Is it a good tool? Oh hell yeah. Ever since I wrote my first serious Bobo web application, spending a spring day salvaging a nowhere-going Python socket server / Java client application by putting a web interface on it, I’ve made my living off of Zope and the technologies and systems that preceded it.

Back in 1996, if not earlier, Jim Fulton wrote ‘Bobo’. The legend goes that right after being hired by Digital Creations, he was flown out to give a talk about Python CGI / web programming, which he had never really done. Jim is a very smart man, and the legend (as I remember hearing it) is that he found CGI programming to be a terrible experience. He sketched out the concepts for Bobo on the flight back, and changed many lives forever.

At the time, CGI was the dominant web programming way. FastCGI and other tools were available to work around CGI’s shortcomings, but working with any of them required changing core code. One of the concepts of Bobo was to abstract out the “object publishing” process: Bobo dealt with the protocol/interface specifics. You just wrote regular Python code following a couple of simple rules, and you had a web application. Bobo was, in essence, an ORB (Object Request Broker). It turned an HTTP Request into an object call, typically following the path based on the URL:

    /greyhounds/betty_joan/editForm
       |
       |
       V
    greyhounds['betty_joan'].editForm()

For example. You could customize traversal on any object along the path, and in essence get any kind of system you want. The default behavior was just to follow Python attribute or ‘getitem’ access. To just start getting something up on the web, you could just use ‘Bobo’ and what you already had, more or less.

Bobo also took care of another problem that plagued CGI programmers. I started doing Python CGI programming not long after picking up the language in 1996, and I found that I spent more of my time trying to get data out of the request and getting it ready to pass into my program than I did doing my little programs themselves. It was very frustrating. Most of the errors would occur in the input / output phases of the CGI scripts I was writing. If I had a field like ‘races_won’ that I expected to come in as an integer, I had to convert it myself (at that time in Python, one had to use string.atoi(), int() wouldn’t parse strings), or deal with it not being in the request, and so on. Multiply this up for larger forms, and it was quite the annoyance. Form libraries didn’t really exist at the time either. So I had a script expecting an integer to come in, and I had all this code in front of it to make sure that (a) something came in, and (b) it was an integer. It wasn’t a hard thing to do, just frustrating. Since Bobo was about turning web requests into Python object calls, it had a way of marshalling the data in the form. With Bobo, I could have races_won:int as the name of a text field, and have a form action of ‘setRacesWon’. On a GET, the URL and resulting object call might look like this:

    /greyhounds/betty_joan/setRacesWon?races_won:int=8
      |
      |
      V
    greyhounds['betty_joan'].setRacesWon(races_won=8)

This was more impressive with larger inputs, like lists and (later on) records. And especially impressive when stacked up, so preferred_powerball_numbers:list:int would always bring in a list, whether there was one value submitted or many. This was all to shorten the space between the web and your own Python code.

Bobo also featured an easy entry point. Actually, it was a little tricky how you’d use the cgi module publisher (you’d basically do a symbolic link in your CGI directory to cgi_module_publisher, which was the thin wrapper that would publish your application by looking up the module name (packages didn’t really exist then either) based on the symbolic link). But! In your module, you could provide a couple of entry points: an object named bobo_application which would be the root of the application and where traversal would start from, or a dictionary called web_objects, with names (keys) mapping to objects/methods to publish. That was easy! That was awesome. In the application that I wrote that Spring 1997 day, I did that to make my front end. I already had this application written that searched a selectable list of different Glimpse text indexes and returned objects pointing to the found file, as well as matching line (and surrounding lines) information. All I really had to do was:

1. Make a page with a text field and some checkboxes.


2. Have that call my search method. My search method returned the HTML with the results.


3. Load and display the matched files.

For my situation, it was the view/controller layer straight up. I didn’t need another model layer: I already had a model of custom Python objects that bridged to this third party data.

So I made a DocumentTemplate for the search form, and had a function in my application module that ran the search by going through the selected search targets, based on the input. And I could publish it with the following in the top level of the module:

    def glimpseSearch(searchfor='', engines=[]):
        """ Search for the submitted text across engines """ 
        #.... python code

    def getFile(path=''):
        """ Get the published file from the relative path """ 
        #.... python code

    searchform = DocumentTemplate('searchform.dtml')

    web_objects = {
        'index_html': searchform,
        'search': glimpseSearch,
        'getfile': getFile,
    }

That was about it. I got a quick search form and quick results template together, and spent the first half of my day working on the search method and discarding some of the old socket-server specific code. Then I spent the second half of the day making the HTML look pretty. I still look at that and go “now there’s a starting point that I could teach somebody”. Zope has long lost that easy starting point. There are many many many starting point options. And even Zope 3, which is supposed to be stripped down and simpler in its concepts, cannot be so easily explained.

I was very impressed by that. I had my own “web framework” going at the time. Python’s always made it easy to write your own, be it 1996 or 2006. But after taking this death march project I had been working on and giving it new life by putting it on the web in a matter of hours, I completely abandoned my own foolish ideas and continued to build on Bobo.

At the time, there was a full “back to front” stack available for Bobo, but you could use all of these parts completely on their own with no dependency on the others:

1. bobo, aka the python object publisher: publishing objects on the web, marshalling requests. Oh, and it offered security too. Could be used on its own as the foundation for building a web application. Generally stayed out of your way otherwise.


2. DocumentTemplate: also known as DTML. Before Principia/Zope, DTML was really quite a nice template language. It did not have any Python expressions at all, and thus could not be used for programming. It had a lot of built in features for displaying and formatting data, especially when working with sequences. It was purely, purely, purely about inserting data into a string, and was quite nice to use at the time.


3. BoboPOS: This eventually became the ZODB. This was a persistent Python object storage. When used with Bobo, the Bobo publisher would start a transaction at the beginning of a web request, and commit it at the end if no errors were raised. I only used it lightly, but loved it when I did. Prior to this, I always had the problem (during development) of having code blow up, leaving me with partially written data that I’d have to clean by hand while also fixing my bugs.

You could use these individually, or all together, or in any combination.

The nice thing was – it was so easy! At least, after learning a couple of relatively simple concepts. There was an easy entry point to basically say ‘for this name, call this function’ or ‘start traversing this object’ and start writing a web application. The job of the FCGI Publisher, or CGI Publisher, was to find the name of the module to publish and pass the module into the publisher core, and that would look for these entry points.

Zope is just a Bobo application. Seriously! Zope 2 is still published as a ‘bobo_application’ object in a module. It just happens to be an object that wraps a ZODB instance. But at its heart, this is all it is.

But being able to so easily start writing a dumb simple web application based on other Python code you might have is not so obvious any more. Bobo got swallowed up into Zope 2. The core pieces are theoretically available again in Zope 3 as independently usable pieces, particularly zope.publisher. And there’s been a zope.bobo effort to restore that ease of use. But the effort hasn’t had much work. And I don’t contribute code to much these days.

I’m a bad person in this respect. I just bitch. But I’ve got a job, a dog, and art projects I’m neglecting that I shouldn’t be.

Anyways, I’ve been in discussions lately advocating a product strategy that offers up the core of the Zope 3 architecture. No object database. Just the component architecture (a pretty simple system on its own) and zope.publisher, which is the descendent of Bobo. Something that could be used to make a web application. Or not. But something that was separate, documented, exciting, etc. Something where you could actually do the “20 minute wiki” tutorial without spending the 20 minutes talking about the zope management interface, about skins, about different development options, about zope.conf and site.zcml files.

I’ve also been advocating a “Zope without all of Zope” release. One that has the Zope Object Database (it’s a better solution than relational databases for many web-targeted projects, and no translation layer like an object-relational mapper), and many of the items that are in the Zope 3 download you can get right now. But it wouldn’t have the “Zope Management Interface” that Zope 3 has. It wouldn’t have the half-assed content objects that are like weaker and harder to use and understand versions of their Zope 2 counterparts (things like page templates, python scripts, SQL scripts that are stored in the ZODB). They’re a terrible distraction. They’re hard to explain. I’ve seen very very smart Zope developers get confused about what to do and whether to use those things. And for writing custom applications, they’re completely unnecessary.

There’s a lot that I love in Zope 3. But there are also just enough frustrating things that I’m running into that, combined with the continued debating over whether or not to rename Zope 3 or what Zope 2 features should come in or god knows what, is having me looking at one of the Ruby on Rails mantras longingly:

Beauty Leads To Happiness

I’m still not as happy as I want to be. And I nearly came unglued when confronted by someone (someone I respect too) advocating a single “everything and the kitchen sink” distribution. I don’t care if that’s available. But I just want a couple of pipes and a wrench. If I have to keep running with this analogy by talking about all of the walls I need to knock down in order to understand why the values for the ‘select’ widget aren’t being saved properly, I’m just not sure this is a house I can stay in.

Ruby On Rails sticks to a strong opinion. Django and Turbogears in the Python world do too. They have a message that they all seem to believe in and love.

And I’m starting to realize that there’s never been a single Zope message that everyone has been able to rally behind. Is this why Zope 3 seems timid to even say it exists? It’s such a good architecture! There are some GREAT tools shipping with it (zope.formlib – the best form library I’ve ever worked with and it absolutely kills anything else I’ve seen out in the Python community), and some GREAT tools available for it (if you fish through Subversion repositories. zc.table looks like it’s the equivalent of ‘formlib’ for doing comprehensive and intelligent data grids of many styles, and I hope to take a close look at it very soon now; ‘hurry’ collects many useful little tools. It’s also pretty easy to start writing your own reusable supporting toolkit, so easy that you don’t even realize it’s happening). It has a lot of potential.

But it really feels like Zope doesn’t know where it wants to go. There’s an existing system to support, Zope 2, that has a lot of good work and maturity behind it. There’s a new system that – my gods – has the potential to just kick ass. And I think it could kick more ass if we could start breaking it out and making it simpler, smaller, and easier to use for developers while still providing larger options for those that like the complete stack or the larger feature set of Zope 2.

But… I don’t know which way it’s going. And there are still a lot of frustrating pieces in the code. And while some people are doing valiant work simplifying some core concepts, the meaning of it all if we can’t talk about it is beyond me.

I still have a hard time knowing how best to start a new ‘application’ in Zope 2 or 3. There are too many options, with too much work to go along with each one. And there are plenty of documents out there that will argue one way as being the best, and one will argue a completely different way as being the best.

And I see the quick start tutorials that other projects have and go “that makes sense” and as they mature, I really start to wonder if this Zope boat is the one I want to stay on if it can’t stick to a direction, communicate its intent, and (if it does pick a direction) heads back towards the land of more features that I’ve been struggling to get away from.