apk zipalign

So, my recent problems with Zope 3 that were making me want to resume smoking? Yeah, I’m a smoker again. Two months clean. The stress just got to be too much.

I reported my problems to the community, with long and detailed explanations and explorations and pleas for help. The responses? Very few. Far fewer than I would have hoped. I had to scrap upgrading a customer to Zope 3.3 after losing a day of work trying to upgrade their database, unsuccessfully. Back to 3.1 they go. And now maybe we’ll forget about this for another year and have an even harder time upgrading next year.

Combined with another day lost chasing an obscure bug (moral of the story - sometimes saying that you implement the interface you implement can be bad, at least in cases where people do identity checks against self when attempting to retrieve keys out of a system), I’m pretty pissed. I really want to leave it all behind. I can’t. But right now, I’d like to. Even when Zope 3 is working, there are some common activities that seem to take way too long to accomplish. I feel like I’ve been spending a vast chunk of my time in recent months focusing on building tools and classes on top of Zope 3 that could bring at least some of what I see and like in Rails and Pylons while taking advantage of Zope 3’s strengths.

I think Zope 3 can be good for CMS type applications. And even for big business and public applications, it can be a good system. The so-called “MVC” stack of the new breed of systems bugs me. I have mixed feelings on templates these days. I think that with good objects, one can generate simple HTML out of Python (or any other programming language) which can then be styled up with CSS. It’s not appropriate for all situations, but it can be a great tool.

But, I must admit, I like what I’ve seen of Pylons. I installed it yesterday and ran through a couple of the tutorials (not even getting to the Wiki tutorial). Last night, I decided to convert the Basic To-do Application that I wrote up last year to Pylons, using SQLAlchemy as the storage mechanism. Fortunately I had some recent (and inspired, if I do say so) SQLAlchemy work done for Zope 3 whose ideas I was able to borrow to get the basic Alchemy session and engine management working in my application. I had just a single table and thus a single model class (keys: ‘id’, ‘description’, and ‘done’).

I decided to go for the same minimum of functionality that I had gone for in the Basic To-do scenario: have a single page that lists completed and incomplete items, with a small form to add new items to the system.

I got this done - including the writing of the simple SQLAlchemy bridge - in about an hour or hour and a half. No time wasted writing ZCML. No time wasted turning off the ZMI. It was generally quite easy. I made a few faltering steps along the way, but not many.

I think Pylons may be the sleeper hit out of all of the Python “web frameworks”. TurboGears got to be overwhelming, immediately. I think that I’ve just never been a fan of its core components. No offense to their authors. Django is also a bit overwhelming. It’s more like Zope: more of a complete environment. That can be great for many uses, but it can be a hindrance for others. I think Django is a terrific system from what I’ve seen of it, but it’s too big for me. I have one big system in my life already.

I have a bit more admiration for the core bits of Pylons. While I’m not a big fan of its syntax, I do think that Myghty (a port of Mason) has a substantial amount to offer beyond just being a templating language: namely lots of caching options and intelligence. SkunkWeb was the only other Python web system that I’ve seen that seems to make caching such a priority. The alternate templating languages one can use in Pylons - Kid and Cheetah - are not at all appealing to me.

WebHelpers are alright as a collection of useful functions for use in templates and controllers. I’ve had a blast taking the helpers concept and implementing it in our new Zope 3 based sites. It makes a big difference. With a good base set of helpers that are easy to expand on, it becomes much more desirable to use Python expressions straight up in templates. When I saw the ERB code in many Rails projects, I was impressed - it didn’t look like the typical mess of common PHP / Classic ASP style development; nor did it have to create yet-another-language on top of its base with hard to learn bind / call semantics. One doesn’t have to translate::

<% h.text_field(name='description', size=35, maxlength=100) %>

into “what Python or template or other thing does that there code call?” compared to::

<f:text name=quest label="Question:" />

The latter coming from Spyce, from an entry on the Spyced Blog.

Actually, I was starting to want to use Spyce in my Zope projects in spaces where we had Helpers doing many common tasks (like seen above) and ZPT was way way way too verbose:

<input tal:replace="structure python:view.h.text_field(...)" />

But Spyce has grown up into a richer system, where all I really want is a good basic Python Server Pages style system. But I’ve never come across one that I’ve liked: something small, simple, fast, easy to bind, meant to work with others (ie - not meant to be standalone pages with big blocks of Python, but rather to work with a view/controller class and helpers).

Back to what I like about Pylons - the last two are Routes, which does the URL dispatch (writing maps like in Rails and Django); and the aggressive use of WSGI / Paste. These seem to be bringing the Component Architecture dream of Zope 3 closer to reality as there are additional filters and middleware one can put around one’s application. Paste provides all sorts of cool features. There seem to be a lot of options for interchangeability.

I’m hoping that Zope 3’s “eggification” will make it easy to write Zope 3 style applications without having to use Zope 3. There are a lot of great libraries and toolkits in the heart of Zope 3 whose liberation would be most welcome. Even without that, I hope to look into what might happen if Zope 3’s libraries were just added to the Python path so that a Pylons application might be written up, using Zope 3’s concept of Views (Multi-adapters) and Transactions with objects pulled from SQLAlchemy without all of Zope’s security and nonsense.

There’s a great set of lines delivered by Lloyd Bridges in Airplane:

• Looks like I picked the wrong week to quit drinking.


• Looks like I picked the wrong week to quit smoking.


• Looks like I picked the wrong week to quit sniffing glue.


• Looks like I picked the wrong week to quit amphetamines.

Sometimes I love Zope 3. Other times, it makes me start to sound like McCroskey. I haven’t smoked now in two months, but I just smashed my head into another foolish and crappy scenario in Zope 3 (after discovering an incredibly crappy scenario earlier this week). Too stressed and confused to think clearly and engineer / hack my way around this latest situation, I just want to grab a fresh pack of Lucky Strikes (unfiltered, of course).

Waaa.

Ian Bicking wrote a Zope 3 Critique a couple of weeks or so ago, partially in response to a critique by Jean-Marc Orliaguet. I’ve wanted to write a response of my own, but time has not allowed it. Now, I’d like to use a few posts to post my own responses to a few items.

The first point I’d like to address comes under headline 3, The importance of standards:

I personally don’t care much about namespaces. Is zope.formlib any more “right” than zope_formlib? Is it any more hierarchical? I don’t think so. I have one namespace package, but it’s kind of a historical accident. I wouldn’t (and don’t) put any more libraries under that namespace. Each library should stand on its own, so why create these hierarchies? People read too much into those namespaces, which is happening right here.

Ugh. zope.formlib is far more “right” than zope_formlib. Granted, this is largely opinion. But man, do I love namespaces. zope_formlib looks like one of those crap-ass hacks and tricks used by the litany of languages who have no namespaces and have to fake it.

First off - I hate the underscore, and believe it should never enter package / module names. Mixed case package/module names are also evil. PEP 8 recommends this:

Modules should have short, lowercase names, without underscores.
…
Like modules, Python packages should have short, all-lowercase names, without underscores.

While I don’t follow PEP 8 too religiously, I’ve found my life to be generally better by trying to stay close to it. I believe that short lowercase names without underscores are a nice constraint, requiring one to think a little bit about what a module/package will entail and how it will be used. If it’s a module that will be reused frequently, it’s good to have a name that’s easy to remember and type.

At least we don’t see as much of the following any more:

from DocumentTemplate.DocumentTemplate import DocumentTemplate

ugh.

Finally, Zope 3 is a large system. I make fairly large systems using many of its pieces, some more than others. Some of those are quite nice to use from an import standpoint when one just imports the sub-package:

from zope import component, interface

from interfaces import IBar

class IFoo(interface.Interface):
    def newbar():
        """ returns 'ha!' """

class BarAdapter(object):
    interface.implements(IFoo)
    component.adapts(IBar)

    ...

That’s a trivial and nearly pointless example, but it’s a pattern I find myself using more and more. I’d be a much less happy mac if I had to do the following:

import zope_component
import zope_interface
import sqlalchemy_sql

Ugh. Why not just introduce crap like ‘require’ and ‘use’ and leave this modularism behind?

There has been one problem, historically, with Python’s packages: unlike Perl, Ruby, Java, etc, the namespaces in Python are based exclusively on filenames and file system hierarchies. That has meant that a “namespace package” like zope.* would have to include everything. It couldn’t be distributed and used as smaller chunks easily, whereas if this were Java the packages/modules would use the ‘package’ keyword:

package org.zope.component;

But now SetupTools allows one to break these pieces up and distribute them separately while still getting the organizational and conceptual benefits of big hierarchical namespaces.

I know that I haven’t posted much lately. I do have a couple of long draft pieces saved that I haven’t had time to finish. I don’t know if I’ll get back to them, but I’ll try.

I’ve been looking at a lot of different things recently, looking for new inspirations, tools, and/or chances to widen my understanding. I find myself growing increasingly fond of a couple of things in particular: Seaside on the web development side, and Smalltalk, Scheme/LISP, Io, and Javascript (particularly with the Prototype.js library) on the programming language side.

Seaside interests me on many levels. I’ve been staring at it with a lustful eye for some time now. There is definitely a place for heavily stateful web apps. Seaside and WebObjects make this a part of their core, where a developer can make an application in a style that is more like a desktop application: less dealing with sessions, freedom from the typical request-response cycle, etc.

But Seaside is more than continuations and callbacks. I love how you generate HTML in it: NO TEMPLATES!!! In a post titled “Turtles All The Way Down”, Ramon Leon writes:

Seaside lets me work in Smalltalk, at every level, all the way from programmatic generation of the HTML and Javascript to configuration of the application. No HTML, XML, XSLT, or SQL is necessary to build a web application, just simple, pure Smalltalk. Programming’s never been more fun!

I’ve made some in-house tools that get us closer to that goal when using Zope 3 and Python. I’ve started to hate template languages again. Well - I don’t mind using them for layout intensive work. But as I get more proficient with CSS, even that is changing.

[Edit: Moved side rant about Zope and Views and Templates out... This isn't the post for it]

It still stare lustfully at Seaside code, especially things like this very basic example:

renderContentOn: html
    html heading: count.
    html anchor callback: [self increase]; text: '++'.
    html space.
    html anchor callback: [self decrease]; text: '--'.

Ooo. Two links are made with anonymous functions that call self increase and self decrease respectively. There’s no “absolute URL”-ness going on. There’s no “how do I make it so that increase can be called through the web?”. No exposure decorator. No seven lines of XML code each for ‘increase’ and ‘decrease’ to be made into separate views. And no template. Just a solid binding between a link on a form and real code.

Seaside is a testament to the power of Smalltalk. People think that Seaside is all about continuations, and if they get just that part down, they’ve made something to compete. But it’s more than that: Seaside is a tool for building web applications in pretty much the same way one makes a desktop application. It’s not the tool or style for every job, but it definitely has its place.

In a recent bit of work we've done in Zope 3, I would have loved to have something like Seaside, especially for the checkout process. I've tried to engineer that process as well as I could, with extra preconditions and redirectors on pages to ensure that all desired information is supplied (ie - so that one can't jump from step 1 to step 4). With Seaside I could have encompassed the in a WATask component, expressing the checkout workflow as normal Python code: while shippingAddressRequired and (not order.shippingAddress): yield self.askForShippingAddressForm() (or something like that). The shopper could never accidentally (or purposefully) get past that point without supplying the right information. Doing this with preconditions, sessions, hidden form fields, cookies, etc, is all possible. But it can be a lot more code overhead.

Back to the other interests listed, really quickly: I’ve used LISP on very rare occasions (for customizing EMacs, basically, back in the day). I’ve never quite understood the appeal that LISP / Scheme have for some people until recently. While I enjoy and will probably continue to work in general purpose languages like Python, I’ve started to grow fond of these languages that are built on very simple precepts. Smalltalk has practically no syntax: it’s all messages. And I really like that Squeak is written in itself - there’s just something impressive about that. On a related topic, I checked on the progress of PyPy and was very impressed. Like Squeak, which has a small subset-of-smalltalk-to-C translator for generating the core pieces from Smalltalk itself, PyPy is Python written in Python, with translators to lower-level languages.

Regarding Scheme, I found a Scheme translator for Smalltalk and have spent a small amount of time playing with it this weekend. I came away impressed with how easy it is to parse and process Scheme, and how easy it is to implement it and its primitives in quite easily readable Smalltalk. The “everything is a list” (‘car’ and ‘cdr’) power finally came through to me as I browsed through the Smalltalk code to see how things that are keywords and operators in other languages (‘+’, ‘if’, etc) were implemented by just using car and cdr (the head and remainder of any list/pair, on which all things are based).

I’ve also become very interested in prototype-based programming. With prototype based programming, one has Objects but without classes and metaclasses and on and on. I’ve known about prototype based programming for some time, but I’ve only recently started to appreciate it. It’s radically simple. Looking around at Prototype based languages, I came across the relative newcomer Io. Io has no keywords - which, again, I find quite interesting and provocative. And powerful. It’s message based, which I like, and it has no classes. You just clone something to make something else. A simple object example from the sample code page:

Account := Object clone do(
    balance := 0
    deposit  := method(v, balance = balance + v)
    withdraw := method(v, balance = balance - v)
    show := method(writeln("Account balance: $", balance))
)

myAccount := Account clone
myAccount show
"Depositing $10\n" print
myAccount deposit(10)
myAccount show

Oh yeah - and the minimalist in me swoons over its web site.

There it is - a scattering of a portion of the thoughts that occupy my brain when I have the chance to take a breath at work.

I’ve surprised myself by starting to enjoy doing browser-side development in CSS and JavaScript more than I enjoy the server side. Prototype, Scriptaculous, LowPro, and a comfortably growing in-house library have gone a long way towards helping with this shift, along with finally getting down into the guts of CSS and learning about the box model. Tools like FireBug and the Web Developer Toolbar for Firefox have helped out tremendously, as well as native Mac OS X tools like XyleScope and direct linking to the CSS specs from within TextMate (very helpful when wanting to learn the real details about any given property).

I saw a link today from Hack The Planet titled “What’s new in JavaScript 1.7: Lisp/Python!”, pointing to a “new in JavaScript 1.7” page at developer.mozilla.org.

So, of course I had to follow this and see what he meant. Sure enough, JavaScript 1.7 starts to look a bit more like Python in some places. From the “What’s new” document, this is an Array Comprehensions example:

var ten_squares = [i * i for (i in range(0, 10))];

Take out the word var, and that’s 100% valid Python (although Python can get by here with less parentheses and without the semicolon).

Iterators in JavaScript 1.7 are also very similar to Python’s, with objects providing an optional __iterator__ slot. Iterators just implement a next() method, and raise StopIteration when done. Generators turn functions into iterators by using the yield keyword.

Very interesting. For a while, I was a fan of the MochiKit JavaScript libraries. MochiKit takes a lot of inspiration from Python and some Python toolkits like Twisted. Among other things, it provides MochiKit.Iter, which brings a lot of the Python itertools functions as well as basic Iterator concepts to current JavaScript, if desired.

I must admit, however, that while I like MochiKit for its general philosophy and the fact that it’s one of the best documented ‘New Age’ JavaScript libraries/frameworks/toolkits around, I’ve gone back to using Prototype.js for most of my work. MochiKit is big. It’s powerful, but big. Yet I found myself straining to copy over some really useful little functions from Prototype, some done while trying to make the Ajax calls a little easier to work with. Ultimately, I find that my Prototype.js based code tends to be easier to read, write, and comprehend. I’m a JavaScript outsider, and Prototype.js pulls enough tricks to make programming much more familiar and convenient. While it’s not well documented, its source code is actually quite easy to read. Both toolkits are nice, but I now reserve MochiKit for heavy-lifting situations only.

Congratulations to Allan Odgaard and everyone else involved with TextMate on winning an Apple Design Award for Best Mac OS X Developer Tool. TextMate has so greatly improved my development life, finally allowing me to (where possible) work on my desktops / laptop instead of on a development server as I’ve done for so many years.

A little bit of my development day today: Using Mac OS X terminal to test a new date-range formatting function. The results were clipped directly into a Circus Ponies Notebook page without leaving Terminal via Mac OS X Services. All of those clippings were then brought into Hog Bay Software’s excellent WriteRoom where I went into full screen mode to turn the test results into documentation that could run via Python’s super-cool Doctest tool. Using WriteRoom for this allowed me to focus on writing the documentation without distractions: it was like being back in the old old 70’s terminals: green text, black screen. No syntax coloring, no modes, nothing.

A couple of times while writing the documentation, I was able to use the globally installed Edit in TextMate… command that lets just about any Cocoa text field / area / etc be editable in a TextMate window. I used this during the main writing to do some big find/replace runs (some names that I had imported in the terminal were redundant in the test documentation). So, from WriteRoom, I hit Edit in TextMate… (actually, I just hit control-command-e) and a new TextMate window opens up with the contents of my WriteRoom document. I used TextMate’s nicer search/replace tools to do some mass fixing, hit ‘save’ to send the results back to WriteRoom, and closed the window - which caused a direct return to WriteRoom.

Again, this was nice for this situation since I could use a specialized no-frills tool to focus on the words I wanted to write. When I was done with the writing, I copied everything out of WriteRoom and went back to TextMate to paste the documentation in the docstring of this function. At that point I could use TextMate’s tools and powers to format the results (wrap / indent properly, apply some reStructuredText styling, etc).

What a cool little collection of tools.

And happily, when the documentation test was executed, it ran on the first pass.

These are absolutely amazing. Phill Ryu held a Fake Leopard Screenshot Contest in anticipation of next week’s Apple Developer Conference which will unveil the next version of Mac OS X. Some of these people put a lot of time into these designs and some of theme are quite interesting. The winning shots by Eric Patterson are quite nice, going for user experience: more views of data “your way”, nice collections of information. Some of it reminds me of the pie-in-the-sky Longhorn screen shots showcasing all of the power of Windows Vista.. well, all the power that was hoped… And Eric’s tabbed finder is about the only mock-up that I’ve seen that I would use.

Honestly - why are people so obsessed with a tabbed Finder? I like the Finder browser (metal) windows, I like Column view, but I don’t want tabs. What I do want is “Open Folder in New Browser Window” so I can branch off at any time. What I would like - a shelf. The sidebar in the Finder is nice. I use it like Tabs anyways, but it’s better. It’s so easy to throw folders on there for one time use for easy copying. And it’s useful when working on long-term projects for rapid access to documents. But it doesn’t deal with drives going away and coming back very well.

Anyways, back to the fake screenshots. The final runner-up is acknowledged as being the most out there, and the most controversial. I like this guy for - ahem - thinking differently. Stephen Sciliano wraps everything up into a single window interface, with multiple panes. I’ve seen some Unix window managers that do similar, with a heavy focus on being text-only. I think I know some people who are big fans of it. Sciliano’s mock-up reminds me a bit of OpenDoc gone wild, and reminds me of Microsoft’s new UI for Office 2007.

Another runner-up, Nathan Ziarek, focuses heavily on metadata - particularly on making metadata easy and obvious. Among other things, his shots showcase being able to associate files and other items with To Do items in iCal, and associating files with contacts so that viewing them from the contact view can link to associated files and other objects.

And that brings me back to one of my favorite Longhorn images (which I haven’t been able to find) was one that showed a Contact. Well, not a contact, but everything associated with that contact: IM chats, phone calls, files, events, etc. In one window, nice and clean. I think it was a WinFS showcase item. I haven’t seen anything like it in Longhorn / Vista screen shots over the past couple of years. That’s something I would like: ways to just group a lot of different information together without requiring a third party app.

I want this most in the home studio. I have lots of audio files and (now) a rapidly growing iPhoto library. The audio files at home that I’m concerned about are not the songs in my iTunes library, but all of the work for Eucci, aodl, etc. Most of them are raw little audio bits, not fit for a loop library or management through a dedicated audio program: they’re AIFF’s of various lengths in various states of processing. I use labels (the little color codes in the Finder) to highlight some of them, such as completed pieces or raw sources (usually the two kinds worth keeping around), but they still tend to get lost, especially if I’ve been away for a while. It would be nice if it were easier to tag those files, to group them dynamically, to associate them with a particular contact to whom I’ve promised an album, to associate them with a to-do so that I have a better sense of what I was doing if I have been away for a while. “Oh yeah, I need to just finalize the levels on that track and I can send it to …”.

The staggering array of personal shit management (PSM) tools for Mac OS X seems to be growing, constantly. There are Outliners, Notebooks, personal hierarchical databases, and so on.

Many of the tools may look similar on the surface. Besides downloading all of the trials or crawling through the archives of About This Particular Outliner, how can you decide which tool may be right for you? Many companies do nothing but expound on their particular feature set, but when there are many similar features between applications, it’s not always easy to filter out the two or three items that make a particular application different from substantially different from its competitors. Some companies have comparison tables between their product and one or two (usually popular) competitors. But these tables often show the main company’s product as doing more! more! more! for the same or lower price. And while that can be helpful when evaluating your options, it’s not exactly unbiased research that you’re reading.

That’s why I was impressed yesterday when I took a glance at the page for Hog Bay Software’s Mori. The lead-in paragraph and accompanying screen shot didn’t sell me on it. I have too many “digital notebooks” as it is, and the screen shot reminded me of DEVONThink. But I scrolled down the page, skimming the text and looking at screen shots, and at the bottom of the page I found a paragraph with a lead-in Is Mori the best choice for you?

Buying a note manager can be a daunting process with many choices. On one hand you have great outlining programs. If your goal is lists and outlines, try OmniOutliner. On the other hand you have powerful and complex note databases. If your goal is to create reference database of “everything” try DevonThink. But if you need something in-between, give Mori a try. We think it’s the ideal place for your day to day notes, projects, and activities.

Mori Product Information Page, Hog Bay Software, viewed July 26 2006

Wow. I think that’s refreshingly honest. It gives Mori a good scope, and it differentiates itself by saying “if you need more than lists, but less than a database of everything, we’re your app. But if you need those things, by all means, go next door and get them.”

Hog Bay Software now has my attention. Looking at their other offerings I noticed the application Clockwork, a computer timer program. You know, the buzz-me with an alarm, remind of this, etc, kind. Its page also has a “right choice for you?” paragraph. They also are the developers of WriteRoom, a “distraction free full screen writing environment.” WriteRoom is one of those applications like 37Signals’ Writeboard. Simple plain text writing, no formatting toolbars, no wysiwyg (which is still painful in most browsers), just words. 37Signals has a nice post about the difference between word processing and writing. They are different tools with different scopes. For the time in writing when the words matter more than layout and formatting, it’s wonderful to be distraction free.

And it’s refreshing to look at a software page and wonder “but why would I use that if there’s this tool and that tool and that other tool available?” only to find an answer that says “hey, that other tool might work better for you if you’re looking for something that does feature X really well.”

A post that’s been making the rounds lately, it seems, is Tim Bray’s “On Ruby”. Bray praises Ruby as being “remarkably, perhaps irresistibly, attractive,” especially for people (like Bray) who are proficient in Perl and Java. As some see Bray’s post as a pro-dynamic-language post as much as (or more than) a pro-Ruby post, I finally decided to take a read.

As I was reading, this paragraph jumped out at me:

Maybe the single biggest advantage is readability. Once you’ve got over the hump of the block/yield idiom, I find that a chunk of Ruby code shouts its meaning out louder and clearer than any other language. Anything that increases maintainability is a pearl beyond price.

While I’ve still done very little actual coding in Ruby, I have found the above to be true with most code that I’ve come across. I love reading source. Well, perhaps read is not the right term. I love to look at it, just as I love to look at screen shots when reading about any particular piece of desktop software. I have a deep love for design, and much of my own personal artwork is grounded more in design aesthetics than conventional art aesthetics, so I often take a visual approach to code. And what appeals to me at any given moment often shifts.

Ruby code tends to be quite attractive to me these days. I’m not sure why, entirely. But I think Bray hits it on the head in his next paragraph. Funny thing: when I was reading the paragraph quoted above - regarding readability - the thought “hmm, what about Python?” ran through my head. That, too, is answered in the following.

In theory, Python ought to do better, lacking all those silly end statements cluttering up the screen. But in practice, when I look at Python code I find my eyes distracted by a barrage of underscores and double-quote marks. Typography is an important component of human communication, and Ruby’s, on balance, is cleaner.
Tim Bray, “On Ruby”, Jul 24 2006

Typography. Could that really be it? You know, he may be right. Phillip Eby recently had a post about Python Based DSLs. He brings up a couple of the elements that seem to give Ruby the power to make nice little domain-specific-languages in, which projects like Rails use to great advantage.

However, Ruby’s advantage in this area basically boils down to two things: being able to apply functions without parentheses, and having code blocks. The first is nice because it makes it possible to create commands or pseudo-statements, and the second is a necessity because it allows those pseudo-statements to encompass code blocks. Without at least the second of these features, Python is never going to be suitable for heavy-duty DSL construction.
Phillip J Eby, “Schema Analysis and the need for Python-based DSLs”, Jul 15 2006

I think Eby captures a couple of the more common elements that make this work for Ruby. However, I think there are a couple of other elements that make Ruby look particularly good in this area. The first is Ruby’s use of Symbols. One description I found for Symbols (I forget the source) is that they’re like Strings that you’ll never show to the user. Therefor they don’t need to be printed. They also seem to work with the Ruby’s equivalent (as far as I understand it) of Python’s identity comparison (in python: obj is None, foo is marker). I’d always seen Symbols in Ruby code, but it was Rails that really made me take notice, as they’re used all over the place. Symbols in Ruby are those things that begin with a colon:

finished = Todo.find :all, :include => [:something]
render :action => 'error'

There is something about that that just looks kindof… nice. :all. In my editor, Symbols are colorized differently than strings, which helps them stand out even more. find :all. Not findAll() or find(all=True) or find('all') like one might have in Python (all of which are OK solutions, but man.. those Symbols).

Now as much as the blocks, Symbols, and optional parenthesis seems to help Ruby - it’s the meta-programming that really seems to stand out. This is some ActiveRecord code from Tracks, a GTD-focused task management application written in Rails:

class Project < ActiveRecord::Base

  has_many :todos, :dependent => true
  has_many :notes, :dependent => true, :order => "created_at DESC"
  belongs_to :user
  acts_as_list :scope => :user

  attr_protected :user

  # Project name must not be empty
  # and must be less than 255 bytes
  validates_presence_of :name, :message => "project must have a name"
  validates_length_of :name, :maximum => 255, :message => "project name must be less than %d"
  validates_uniqueness_of :name, :message => "already exists", :scope =>"user_id"

  ...

These are statements in the class level that read… easily. A project has many todos and notes and belongs to a user. It must have a unique name for an individual user that is less than 255 characters. Wow. That is a lot of information in a small space. And you don’t think of has_many as a function or method or something that’s pulling of crazy metaclass trickery: it reads like a statement. I see this done so often in Ruby that writing things like has_many are not that difficult (I’m not talking about the implementation behind it, since O-R mapping is always a beast; I’m talking about how easy it is to operate on the Project class in-line like that).

Now this is doable in Python. Zope 3 has applied similar things, albeit sparingly (and with good cause). You can’t really monkey with the Project class in Python while inside the class statement without resorting (at the least) to execution frame tricks. But one you’re comfortable with them - you can start doing some nice things. In the early days of Zope’s Interface system, declaring support for what was implemented by class instances looked like this:

class Project(Base):
    __implements__ = IProject
    __used_for__ = IUser

Now it can be this:

class Project(Base):
    implements(IProject)
    adapts(IUser)

In a SQLAlchemy based system I’ve worked on for a major project I’m involved with, I’ve applied some of these techniques to yield code like this:

class FeeScale(Base):
    act.baseTable(
        fee_scale, classify='type',
        order_by=fee_scale.c.price_floor,
    )

    @classmethod
    def getFeeForPrice(cls, price, owner, default=None):
        fee = cls.selectFirst(
            (cls.c.price_floor < price)
            & (price <= cls.c.price_ceil)
            & (cls.c.owner_id == owner.id)
        )
        if fee:
            return fee.service_fee
        return default

class RetailerFeeScale(FeeScale):
    act.whenClassified('retailer')
    implements(interfaces.IRetailerFeeScale)
    classProvides(interfaces.IQueryableFeeScale)

    owner = props.One(ILinkedFee['owner'])

class WholesalerFeeScale(FeeScale):
    act.whenClassified('wholesaler')
    implements(interfaces.IWholesalerFeeScale)
    classProvides(interfaces.IQueryableFeeScale)

    owner = props.One(ILinkedFee['owner'])

It doesn’t read quite as nice as the Ruby example above, and it’s from a different problem domain. But it’s not bad… I had to move my validation and relationship-related items into Zope Schema fields which are about specifying design and contracts. Since I use them heavily in order to do things like form generation, ILinkedFee['owner'] contains the information about the relationship, props.One is a Python descriptor that uses that field information in conjunction with SQLAlchemy’s properties to handle relationships between two mapped objects.

class ILinkedFee(Interface):
    owner = schema.HasOne(
        title=u"Owner",
        model=Ref.relative('.base.Owner'),
        options=dict(lazy=True),
    )

There are upsides and downsides to this setup. One really big downside is that there’s no single place to see everything going on with RetailerFeeScale. Earlier versions of this code had functions that played tricks inside of a class suite to spell things like hasOne(...) without assignment or using a second mechanism such as Interface schemas. That code, however, was playing too many tricks and causing too much trouble. And this setup works well with Zope 3.2 and the latticework I’ve put in place to support it.

But it seems like a lot of work, learning metaclasses and descriptors and decorators and getframe() trickery. However, looking at this mammoth project’s code tonight, I’m impressed by how concise and readable most of it is. Ruby and Smalltalk have both been major influences on me lately, even if I seldom get to actually use those languages directly. God I wish for blocks though. I think that the with statement in Python 2.5 will help. A place where Blocks seem especially powerful is their ability to construct an object, yield it to the block, let the block do things with it (set attributes, etc), and then the thing which yielded control can do more things with the object. Hmm. That’s vague. Here’s one case: explicitly saving (flushing) a SQL Alchemy object after doing a bunch of updates. It’s one of those lines that bugs me: it’s something important (it should happen), but it’s not necessarily important to the business logic. It’s small, and it often blends in with its surroundings:

fee = Fee.get(1)
fee.floor = 0.0
fee.ceil = 10.0
fee.price = 1.25
fee.save()

Small annoyance, really. But it does bug me. I believe that the with statement might help out, ensuring that the save happens immediately:

with savable(Fee.get(1)) as fee:
    fee.floor = 0.0
    fee.ceil = 10.0
    fee.price = 1.25

And with that, I put to bed this evening’s incredibly long and pointless rambling, whose beginnings I scarcely remember. Oh yeah. Ruby’s punctuation and typography is beautiful. It really is. And I envy it.

I’m not sure what this indicates about me, but I felt such an overwhelming air of excitement and relief that I almost - almost! - had a little tear in my eye when I saw this announcement from the OmniGroup. Omni’s new secret product that’s no longer a secret is (oh boy oh boy oh boy!): OmniPlan. Project management software.

Why am I so excited about project management software? Because there’s very little project management software for the Mac. I know this, because I recently looked for some. We were using an OmniOutliner document to try to capture all of the known things that we needed to do to get this big enormous project out the door, and to come up with some estimations on the time involved. OmniOutliner was OK for this, to some extent: it can have multiple columns, and the columns can be different types including durations. The duration column can be configured to calculate things in working days or 24 hour days, so you can say “that’s going to take 10 hours” and the document can turn that into “1 day 2 hours”. You can also have summaries, where a parent outline element can add (or average or mean) the values of the child nodes. However, with the way that we were entering data, we couldn’t take advantage of this.

As we were fleshing out the development plan, we were unable to easily spell dependencies. We got around this by adding another column, but it was just plain text. One couldn’t look at the list of things to be done and easily decide to do “that big scary thing that needs to be done so we can do these four other things.”

Turning that into a calendar so we could plan time and other commitments appropriately didn’t work out either. We did it manually, on paper, and then turned some of the big milestones into milestone entries in Basecamp. And while Basecamp excels at communication (insofar as everyone involved actually uses it), this level of task / timeline / dependency management is outside of its scope. And I’m glad for it. We seldom need this kind of MS Project style project management… Until this project came along.

So I downloaded a couple of demo versions of different Mac OS X project managers to see how easily I could turn our Plan into something that could better display time estimates, dependencies, and so on.

How easy was it? Not very. I didn’t want deep project management features. I wanted to be able to enter tasks as quickly and easily as I had with OmniOutliner and have the Plan start coming together on its own. But in everything that I tried, there were too many steps, too many dialog boxes, and so on. I don’t mind dealing with inspectors and details later, but I hate when they get in the way of doing simple data entry. And I also hate the tools whose simple data entry is too simple, so that if you enter a lot of items rapidly you then have to spend a lot of time in dialog boxes, tabs, etc, turning it into something real.

I don’t know what OmniPlan is going to look like, or cost, or anything. But Omni’s record of building good solid NeXTStep and Mac OS X applications that are fast, fun, intuitive, and flexible is nearly unbroken. I wish this application had been available a month or two ago, but better now than never. I expect we’ll see a tool that humble programmers such as myself can use without a degree in business or project management.

If you ban the burning of the American Flag because it supposedly desecrates what our soldiers have died to protect (funny - I don’t see soldiers out protecting the flag over the supermarket from lightning), shouldn’t there also be a ban on using the flag to bring everyone down to “celebrate their independence from high prices at your local car dealer and/or furniture warehouse Fourth of July Independence Day Sale Spectacular”?

How about a ban on Memorial Day sales pitches? “Remember all those who have actually died in the service of this country, often in hostile environments most of you would have a heart attack in, by getting a Free Hot Dog and a chance to win a new sofa set on us!”

I know that this kind of commercialism is here to stay. Personally, I think that it really does cheapen the meaning behind some of our holidays. Why aren’t more people up in arms about big business’s war on memorial day? Unlike the bullshit crusades against the so-called “war on Christmas”, some of these holidays actually have a potentially good national meaning that is completely - COMPLETELY - washed over. “Support our troops and celebrate your freedom by getting a new Honda at Zero Percent Financing, this weekend only!”

These proposed constitutional amendments to protect the flag and protect marriage do nothing of the sort. They’re stupid ass wedge issues. If you want to really protect marriage, do something about the divorce rate in this country; shut down the 24 Hour shotgun chapels in Vegas (does that not make more of a mockery of this “blessed institution” (cough) than anything?); boycott Budweiser for cheapening marriage with its image of a man getting married over and over in Vegas for a free case of beer; boycott television sitcoms; get “Desperate Housewives” off the air; support federal aid policies that help low income families so that financial stress is less likely to be a factor in separations and divorce; hell, why not make adultery laws with severe punishments like some countries that wobble daily between friend and foe?

Is a constitutional amendment defining marriage as being a union between a man and a woman really going to do anything to this country’s divorce rate? Is it going to suddenly put an end to single parent homes? Is it going to cut down on adultery? Murder? Foster care?

Not one goddamn bit.

And it’s the same goddamn thing with the flag burning amendment. I personally believe there are many other ways to desecrate the flag, our troops, and our history, and it’s all accepted as perfectly legitimate as long as there are free balloons for the kids.

Stupid, stupid, stupid.

Terrific targeted advertising: The current ad that appears on the main page for Sci-Fi channel’s excellent new Battlestar Galactica series is for Microsoft’s Visual Studio 2005.

The ad? It just says “Cylons. Why debugging matters.”

I think that’s the first good ad I’ve seen placed by Microsoft in a long time. For those unfamiliar with Battlestar Galactica, especially the new series, the basic premise is "The Cylons were originally created by man. They evolved. They rebelled." The miniseries that brought the story back to the air (in a much more rich and intelligent flavor than the old series, which had a good idea but had poor execution and network interference) starts with the near annihilation of the human race in a massive coordinated Cylon attack. Yeah, debugging matters. Great placement. Although, of course, you could see variations on this for Terminator: "SkyNet. Why debugging matters." And for The Matrix. Which is not to say that Galactica is anything like those series (I think the nearest it could be compared to is Bladerunner). Just that someday, these robots will come back and kill us all. Sony's cute little QRIO has its own emotions and can express them in different ways, including changing eye color. You just know one night you're going to wake up and see this cute little robot at the foot of the bed staring at you with those red eyes... And then you'll never wake up again...

Sorry, off track.

While on the concept of ads, I’ve been enjoying one of the latest from Apple’s “Get a Mac” campaign. The commercial is Out of the Box and features the Mac and PC in their boxes talking about what they’re going to do. If you haven’t seen this series, they follow the sparse white backgrounds and straight-on shots of the older “Switch” commercials (and, in fact, most of Apple’s modern non-iPod commercials). Instead of the “real people” stories that were the center of “Switch”, “Get a Mac” features a humanized PC and Mac. The PC is played by John Hodgman (now a Daily Show staple), and the Mac by Justin Long (an actor in “Dodgeball”, which is a movie I unexpectedly fell in love with).

In Out of the Box, the PC and Mac are sitting in boxes - the PC’s is plain brown, the Mac’s is plain white. They introduce themselves as they do in every commercial - “Hi, I’m a Mac” “And I’m a PC”. The Mac starts getting out of his box and says “ready to get started?” to which the PC responds “well, not quite… what’s your big plan?” The Mac sits and says “well, I might make a movie, create a web site, try out my built in camera.. I can do it all right out of the box. So what about you?” The PC responds with “well first I got to download those new drivers and then erase all the trial software that came on my hard drive…”

The ad ends with the Mac getting out of the box and leaving frame while the PC says “actually I can’t go yet… the rest of me is in other boxes.”

Now this is a great ad. It’s short, funny, and quite charming. While the content of the ads is competitively aggressive, the PC and Mac are quite nice to each other (in one ad the Mac even says that the PC is great at business). Overall the series is doing a good job, in very short segments, of highlighting the cultural differences. Some ads play up to the “if you like the iPod or iTunes, we’ve got even more of that!” message, while others cover other differences. This one, however, I particularly enjoyed.

We recently purchased a new Dell machine at the office to run accounting software. A few days later, there is still a cadre of boxes in our lobby with CDs scattered everywhere. The machine is installed, and it installed fairly quickly. But what I remember most is how much my boss kept yelling at all of the trial software, and how much time he spent un-installing things. “Something new pops up every time!”

I imagine that this is how Dell and the like can offer their machines for such low prices. They stuff it with all sorts of trial software - often very annoying trial software (it bugs you more than it helps you) - which is just more advertising, basically, subsidizing the lower cost.

It’s amazing. I have a windows machine at my desk that I use on occasion (it’s actually been off for a couple of months), and I hated having to restart it because there were so many little balloons and bubbles that kept popping up for the first few minutes. “I did this” “You should upgrade to this” “Buy this” “Your machine is at risk - upgrade to the full version of bla bla antivirus for $79.95 today!” and on and on. Half of these interrupts were modal, taking away focus from what I was trying to get back to doing. Part of me is amazed that this passes as OK in the Windows world, but part of me looks at advertising today (and through time) and realizes that it’s just the norm of our society.

I always get entertained at list once during local coverage of the Utah Jazz at all of the little inserts that get slipped in, and how casually the announcer puts it in. Granted, a lot of these same games are being simulcast on radio (some games are only broadcast on radio), so perhaps it makes a bit more sense. But it’s just things like “Malone drives it up the floor and it’s knocked out of bounds. This portion of the game is brought to you by Bla Bla Bla, reminding you that it’s always a good time to Bla bla” or “brought to you by Bla Bla. Come down and see the professionals at Bla Bla, quality service guaranteed. Stockton takes it inbound…”

It’s all around us.

Anyways, I just found the Out of the Box commercial funny, having watched someone just recently go through the experience of spending his time removing all this junk software before he could set the machine up to do its actual job. It’s not like the Mac is devoid of trial software - a .Mac trial period is pimped, trials for both Apple’s iWork and Microsoft’s Office are also on board. But to my recollection, no bubbles ever popped up and interrupted my work telling me to use / install / upgrade either of those. In fact, I didn’t even notice they were on my new iBook. They didn’t hinder performance any, and - most importantly - never got in the way. Dragging them to the trash was all that was needed when I did notice they were there.

So, good ad, sells the point well. I’ve been impressed with how well all of my Macs have worked out of the box - even discovering that a new laptop had a decent charge in its battery when it arrived!

A nice post from 43Folders came across my desk today pointing towards a post by Eric Schmidt about Using OmniOutliner Pro and Kinkless GTD in Law School. Schmidt first covers how he uses OmniOutliner Pro to take and manage notes for his law courses. One significant Pro feature in OmniOutliner is the ability to have Sections - a navigable list of top level outline elements. Kinkless GTD uses these to separate the Inbox, Projects, Actions, Archive, Templates, and Setting portions of the kGTD document. As I’ve started to work on larger outlines again in OmniOutliner, I’ve started to make use of this feature in other documents.

Now, there are numerous outliners available for the Mac OS. A glance through the archives of About This Particular Outliner shows comprehensive coverage of various outlining and mindmapping tools. I’ve used quite a few of these over the years, often just in evaluation. The only additional ones I’ve purchased and gotten any real use out of have been Aquaminds NoteTaker, DEVONThink Personal, and Tinderbox.

NoteTaker is a decent application. I got a lot of great use out of it in a school setting as a math notebook. Using EquationService, I was able to take notes in a full Notebook paradigm and enter the math formulas in LaTeX and convert them to inline PDFs, with no knowledge of LaTeX or equations being required of NoteTaker. I’ve also used NoteTaker on a couple of customer projects that had clear releases. NoteTaker’s structure has many pages, grouped by sections. A section might be “WebSite 1.6”, with a page devoted to each feature request made by the customer for that next version. NoteTaker’s pages are outlines, but it often works best with big blocks of text. No problems - that worked great for these books. But while NoteTaker is steeped in Mac OS X’s NeXTStep heritage, it has a bit of a strange feel. More NeXTStep-ish still than Mac OS-ish.

Interestingly, NoteTaker has a very very very similar competitor: Circus Ponies NoteBook. They both have the notebook metaphor - the windows look like ringed notebooks (by default), they have index tabs on the side, and the document is a collection of pages. There are differences between the two, but they both actually share a common ancestry. AquaMinds (NoteTaker) and Circus Ponies (NoteBook) were started by two different principals that had previously made a similar [product for NeXTStep][http://www.simson.net/nextworld/93.2/93.2.ApMay.Notebook.html].

DEVONThink Personal is not so much an outliner as a personal database. NoteTaker and NoteBook position themselves that way too, but DEVONThink is much more like a specialized Finder implementation than an outliner. DEVONThink has a sort-of outline “mode”, but it’s nowhere near as fast or natural as OmniOutliner or NoteTaker. Instead, DEVONThink shines as a place to collect PDFs, web pages/archives, text notes (rich or plain), etc. It has excellent cross-referencing and indexing capabilities so any document can have a “See Also” drawer which shows contextually similar documents in the system. This is great for building reference systems, especially since you can drop a new note, text, web archive (great for keeping documentation around), etc, in and then use the classification tools. Classification works similarly to the “See Also”, but shows folders instead. Folders are matched based on their contained contents. I use DEVONThink at work to collect odds and ends from various projects, and (more often) to keep and archive web pages containing documentation for tools like SQLAlchemy, Markdown, etc. On my laptop I use it to keep archives of web pages I want to read in my spare time, whether or not I have internet access.

Tinderbox is an interesting system. Part of me thinks it could be so much more if only it were modernized. It just doesn’t feel that natural on Mac OS X. It’s such a powerful and flexible system, with the ability to think and work in different modes - outline, treemaps, visual maps - with multiple windows open in the same document even! Custom attributes, agents, and rules can make for pretty powerful and personal systems. But using it is like using [Squeak][http://squeak.org/]. It’s really cool and really powerful, but it’s also its own self-contained world. I had a hard time integrating Tinderbox into my day-to-day operations as it works less and less like the Mac OS X I know and love. No Services support, changing a font requires using a traditional font menu (which is a nightmare with my sprawling font collection). Hell - you can’t even save a file with a name longer than 30 characters! I don’t think I’m in Tinderbox’s target market though. If I wrote as much as I often say i’d like to write, I’m sure it would be a great tool. I’ve wanted to use it to start mapping and managing a lot of characters and places from real life and mythic life. But I have a hard time going into Tinderbox land.

OmniOutliner, on the other hand, is tremendous. Especially as of version 3 (and even more so with version 3.5). It is so deceptively simple. A fresh document is just a basic outline. Start typing, indenting, structuring, etc. Often that’s all you need to take some notes down or think something through. But it also has excellent multi-column support. The columns can have different value types with the ability to summarize values. In evaluating whether I could afford an impromptu weekend trip, I used OmniOutliner with two columns - Topic (the main outline/text column) and Price, with Price formatted as dollars and with a “Total” summary. The top level of the outline was “San Francisco Trip”, and then inside of it I had lines for flight, hotel, tickets to an event I wanted to go to, boarding the dog, train, etc. Very quickly I added borders on rows and columns, and within seconds I had a very small and simple document that many people use spreadsheets like Excel for - but with absolutely none of the complexity or long arguments that are involved when I use Excel (or any spreadsheet, for that matter). This was just a quick “I wonder if I could…” experiment - could I make a quick list and see the estimated total? could I afford to go? (The answers were ‘yes’ and ‘no’, respectively).

But OmniOutliner does even more, and all naturally. By using Mac OS X’s text formatting system, it gets access to the Ruler (NoteTaker and NoteBook get this too). The Ruler makes it possible for just about any Rich Text input area to be at least as powerful as a tool like WordPad on Windows. Named styles, tab stops, left / right / justification alignment, paragraph indentation, spacing, and more. OmniOutliner doesn’t show the ‘Lists’ option, but NoteTaker does, so bulleted and numbered lists inside of the text are available just about anywhere.

This is what amazes me about OmniOutliner: it can be such a fast and simple basic outliner, requiring practically no training or documentation to use. It’s smooth and fast and looks great to boot. But with just a little exploration of the menus, toolbars, and inspectors, it can be unleashed. Some sample documents look closer to a full Word or Pages produced document: the outline handles and checkboxes are hidden, the indention settings are massaged, the fonts and styles applied well. Other sample documents show its usefulness as a simple spreadsheet / database system: the kind where the main text is delightfully easy to enter and formats well against the other columns (spreadsheets do not do this well at all), and the supporting columns can have intelligent value entry and handling. The date support is wonderful - “tomorrow”, “monday”, “the 3rd”, “last friday”, are all valid values. Duration columns can be set with how to measure hours - is it a work day? is the work day 8 hours? then a total of 9.5 hours immediately displays as 1 day 1.5 hr.

The section support in OmniOutliner Pro invades some of the territory enjoyed by NoteTaker and NoteBook and the like. Suddenly a single large outline can become very navigable. I exported some of my NoteTaker books out to OPML and into OmniOutliner Pro and had a system almost as nice up and running within moments (aside from some lost formatting). Nicer still is that a co-worker has started using OmniOutliner pretty heavily and it’s nice to be able to settle on a single tool and share. I’m just amazed that this tool can be so simple and useful for the quick-notes while becoming powerful enough to support a system like Kinkless GTD.

Can’t live with ‘em, can’t live without ‘em. This is about the Mac OS X applications I can’t live without, and some mentions of ones that I’ve still never found use for.

Can’t Live Without

Quicksilver

The ultimate quick launch and system control tool, Quicksilver is like the perfect marriage of graphical and command like control. Before Quicksilver I was an avid user of Launchbar. At heart, both of these products are just launchers - quick access to applications, bookmarks, etc. Launchbar allowed me to never be bothered by the Mac OS X Dock: I use the Dock to house applications and documents I use frequently, and Launchbar let me quickly get to ones that I use only on occasion. What makes both of these products stand out is that they learn. One might bring up the tool and type the letters A G L. The first hit might be a bookmark to an agile development article. But hitting the spacebar a couple of times to highlight Adobe GoLive will cause both LaunchBar and Quicksilver to increase the weight of that result. Within a couple of uses, CONTROL-SPACE A G L becomes the fastest way to launch GoLive.

Both tools offer access to more than just applications and bookmarks. Quicksilver started to differentiate itself from Launchbar (as of Launchbar version 3 - it’s now at version 4.1) as a more thorough system control tool with a lot of AppleScript support and a wide (and growing) range of plug-ins. Quicksilver could be used to control iTunes, to put the computer to sleep, and more.

I’m not sure when or why I made the switch. As Mac OS X matured, I used Launchbar less and less. When Quicksilver came on the scene, I didn’t pay much attention to it. But I kept hearing good things about it, and eventually got around to checking it out myself as it was a big hit amongst the 43Folders crowd. As I started to use Quicksilver, I found it insanely useful. I could pause and resume iTunes with a few keystrokes without having to change focus from what I was doing. I could start doing some “fire and forget until later” actions, such as emailing a quick note or to-do to a Backpack page or sending an action to my Kinkless GTD document. I could take quick advantage of Mac OS X services such as Make Sticky or Take Plain Note in DEVONThink, even in applications (like Tinderbox) that don’t support the Services system.

This is most handy when doing something very focused, such as a debugging session, when I’d notice something else I might need to come back and revisit. I don’t want to take the time to change focus to OmniOutliner or some web project tool or iCal to make the note to “come back and fix the call to …” when I’m chasing some obscure issue. Quicksilver lets me make that note almost invisibly - I don’t even have to take my eyes off of the current debugger line. Priceless.

Small feature that I absolutely love - Large Type. One of the default text actions in Quicksilver, this is damn handy when making phone calls. Apple’s Address Book, and some other applications, support this for values they have in the system. But sometimes you need to call a phone number and read off an account number that isn’t going to be in Address Book. With Quicksilver and a couple of tuning options, you can get this down to a single chord or just a couple of keystrokes, and BAM! Screen-size type. Doesn’t interfere with anything else - next keystroke or mouse click makes it go away.

TextMate

A native Mac OS X text editor, TextMate has become my new killer application. For the first time ever, I’m doing most of my Zope and what-have-you development on my desktop. Historically I’ve always done my web development on servers, using XEmacs and then VIM as my comprehensive editor of choice. Even when I’d do development on the desktop, I’d often use VIM. But TextMate has changed that.

With support for lots of languages and commands, it’s become indispensable. The killer feature was the ability to install an Edit in TextMate… command in all Cocoa applications. Now I can edit the contents of any textarea in Safari in TextMate - whether it’s GMail, a bulletin board system, Basecamp, a Zope application, a classic Zope page template, I can pop over to TextMate. Even now, I’m using MarsEdit to make this post. I’m writing it in Markdown and will convert it to HTML from within TextMate before posting it.

Over the years, I’ve found long-time Mac staple BBEdit to be too expensive, and ultimately underpowered, for my needs. BBEdit still bears too much of its classic Mac OS heritage, I feel, and some of its keyboard commands and preference screens started to feel really unnatural. Granted, I was still using an old version. But even the newer free ‘lite’ version of BBEdit, TextWrangler, felt uncomfortable. I used it for some development in the past year, but TextMate feels a lot more natural to me in most regards.

There are some features that TextWrangler / BBEdit has that are still missing from TextMate - integrated FTP support is one, split windows is another. The one feature that I can’t believe TextMate doesn’t have yet is the ability to split editing windows! But the number of plug-ins available, the “Edit in TextMate” feature, the relative ease with which new syntaxes and commands can be added, far outweigh some of the holes.

Small feature that I absolutely love - Find in Project. Any folder can be a project (without all of the project overhead imposed by IDE’s). In the Zope 3 development that I’ve been doing, this has made refactoring easy as I could rename or move a class or interface and could quickly find all uses of it. TextMate also supports many of the Cocoa find/replace features and commands (and builds on them). One handy command is “Use Selection for Find (command-E)”. Safari, Mail, MarsEdit, OmniOutliner, etc, all use this command. Select a word or phrase, hit ‘command E’ and then ‘command G’ (find again) to search for that word without bringing up the ‘Find’ dialog box, without using up the clipboard, etc. This is great with the Find in Project feature in TextMate when I’m thinking of renaming or moving a class, or just want to see where it’s used, such as when exploring other people’s code (such as the Rails or Zope frameworks themselves).

DarwinPorts

DarwinPorts has made it possible to do the kinds of development I now do on my desktops / laptop. This is where I get my Python 2.4.x with readline, Ruby 1.8.4, Docutils, subversion, etc.

OmniOutliner Pro / Kinkless GTD

OmniOutliner has been in my toolbox since version 1, back in the early Mac OS X days. I’ve gone through other outliners and information databases, but now I’m using OmniOutliner Pro pretty heavily again. Part of this is because it’s still the fastest, easiest to use, and best looking outliner on Mac OS X. A third party set of AppleScripts turns it into a powerful planning / task management system, as Kinkless GTD. I’ve tried some other systems such as GTDRules on Tinderbox, which I’ve compared before.

Kinkless GTD features a Quicksilver action called ‘Send to kGTD’ which parses text into an action and sends it to the kGTD document. If the document’s not open, it opens it. The parsing revolves around being able to specify a project and/or context to place an action in. For me, this is useful for that ‘fire and forget until later’ moment that comes up from time to time. I capture something that needs to be done, but I don’t let it consume my mind while I focus on the task at hand. When I have a moment, I can go to Kinkless GTD and file it appropriately or figure out what needs to be done for that note I scribbled off. This shortens the amount of inboxes that I have, and shortens the number of places I have to think about when I want to send myself a “deal with this later” note.

iDisk / CVS

Apple’s iDisk (part of their .Mac offerings) has a nice feature: the ability to work offline with transparent synchronization. That synchronization can be automatic or manual. This is how I keep track of my Kinkless GTD document, and any other notes and documents I want to work on without worry of location. No need to have a Lotus Notes size system just to manage a small set of personal information across three machines, especially when one of them spends half of its time offline (and one is spending ALL of that time offline these days). iDisk lets me save any document of any type, synchronize the disk to my laptop, take the laptop home, work on it there (without internet, which hopefully will get fixed soon), come back, synchronize back. I’ve thought of using USB “key drives” to house information like this, but I often fear damaging or forgetting the drive.

We still use CVS on our development servers at the office. We’re too small and busy to have time to change to subversion when CVS is doing its job (mostly) well. Some people use CVS to manage their home directories, documents, etc. I just use it to bring code home in case I want / need to work there while I’m disconnected. It’s hardly worth mentioning, except for how every work day ends with ensuring the iDisk and source directories are up to date on the laptop.

Can’t Live With

There’s really only one thing I can think of here, and it’s one thing that’s always bothered me: Virtual Desktops. I’m glad Mac OS X doesn’t support this directly. My main problem with Virtual Desktops is the ease with which important windows get lost. With Mac OS X, I use “Hide Others” and window minimization heavily when I need to focus and I have too many distractions on screen. A quick “Show All” can bring all the windows back on screen and into the dock (if minimized), and Exposé is, at that point, the best way to find a particular window when needed. With Virtual Desktops, I was always playing the game of “is it on 1? nope 2? nope 3?”. Some people establish systems - web browser always here, email always here, main editor always here - but I never was able to. I’d just lose things and get frustrated. Between Quicksilver and Mac OS X’s native app / window switching features, I have adequate control over my desktop and what populates it, I feel.

A couple of months ago, we started work on a major rewrite of one of our core sites. What started as a small and specialized e-commerce service a few years ago has grown mightily over those same years. The architecture, however, has hit a limit - especially when compared to the large push for what we want to do (and can do) next. The current site works great and handles its load and duties very well. But there are small annoyances internally with some things, and there is some growth and market potential that it cannot satisfy.

The first version of the site’s primary work was done in less than a month. The main focus of that version was the visitor’s experience. The original HTML mockups drove a lot of the site, and the implementation was primarily done in Zope 2, through the web, using templates, scripts, and SQL Methods. It was a nice enough system for the time - we were very fast with it, we could tweak elements in each other’s offices, etc. It ended with an all-night debug-and-deploy session, wrought with all sorts of strange problems and angst and fun. But it worked. We continued to work on it over the next few weeks, of course, filling in some major holes that couldn’t be filled in initial development and responding to site usage.

The biggest problem with that implementation was the administration side. Since we were the only users of the administration screens, we punted on that issue. There was no form validation. All of the database CRUD statements were entered manually. Zope’s SQL Methods made this a bit easier, but their main optimization is for read-queries. Any new field required numerous code updates all over the place. The whole business was a little messy back then - requiring us to print out certain reports and delivering them to our providers on time-constrained schedules.

The site was never empty, but it got perilously close a few times during the lifetime of version 1. But there was enough interest to keep it going. With eyes on expanding and getting new providers, with providing new services (including physical goods), and with running the business side of things better, we went into version 2.

As I saw all of the HTML forms and database statements I’d have to change just to accommodate the expanding and changing data requirements, I started looking at options for easing that pain. I ultimately rolled my own system focused primarily on the common CRUD operations. There was no object-relational mapping solution at the time that I liked. A problem, at that time, was that we had many queries in place that would not work in the object-relational tools available.

What I really wanted was an architecture that:

1. Helped me get important / core business and administration logic out of the ZODB. We wanted to have as much of the core software on disk and under source control as possible.


2. Dynamic form and SQL generation for common tasks (primarily in the administration section). I wanted this so that adding new fields to the database schema required only one or two changes.


3. Still allowed us to use Zope’s SQL Methods for generating complex queries.


4. Could be introduced in a way that the public application might not even be aware: instead of a folder full of Python scripts in the ZODB, there’d be a single persistent instance of a class that had many of those scripts as methods.

The resulting framework was essentially a service layer comprised of very rough Table Data Gateways. You couldn’t load an object out and do a jeff.hair = 'bleached' and have it save. The get/create/save statements basically took a dictionary (hash table) of values to save and would flush it out to the database, after harvesting some information. It was expected that the gateways would have the data prepared before passing it off to the lowest level interface, handle_op, which would perform the requested operation.

When dealing with fairly set data, such as a form, this didn’t really matter. I cared about loading and binding the data to the widgets when a page was loaded, and then parsing/validating/converting that data on save. For that, this system worked like a champ. For some other pieces of business logic, it did OK. It wasn’t great, but it was better than what we had before - especially because it didn’t require specialized update SQL to be written for those situations where just a couple of values needed to be changed.

That site has served us fine, and now the offerings on the site are quite full (many many pages). The content providers have access to their items and reports, so we no longer have to run around town delivering reports and lists.

Both versions were deployed, in full, and then continuously massaged over time. There was no ‘beta’. Some features sneaked in after the deployment deadlines. Some specialized parts of the system were overhauled a couple of times to deal with scaling issues - ones that we could never have predicted.

We’ve kept laundry lists over the years of the things we wanted to do, but couldn’t under the first two designs. Some of these came from dealing with some hacks pulled together for a couple of special customers, some came from wishful thinking, some came from complicated set-ups required to make certain rare items available for purchase in the existing system. In a getting real sense, this was OK, but not great. It limited our ability to start working with larger and more specialized providers.

Architectures have continued to improve over the past couple of years as well. When Zope 3, version 3.1 specifically, came out, we were able to do a major rewrite of a content system for one of our oldest customers. We were even able to pick up additional related customers as a result. We’ve delivered other solutions on top of Zope 3 and the ZODB (Zope Object Database) that have been both impressive and fun to work on, although sometimes Zope 3 can still cause me to go off in a huge screaming match.

I had hoped (and still wish) that for this particular site’s version 3 implementation we could ditch the relational database completely and just use the ZODB. Instead of classic Zope ZODB usage, where scripts and templates fill up the database, Zope 3 makes it much easier to keep that stuff OUT, and then easier to keep real data in. This is one of the nicer parts about working with the Zope 3 / ZODB stack: Python is just Python is just Python. There’s no translation layer, no tricks, trying to fake inheritance. There’s no real worry about translating statements into a query language - Python is the query language (with tools like the Catalog providing application level indexes for larger queries). I had an application that I was working on in Zope 3 that I thought would be a good testbed for writing as a Rails app - until I realized that Active Record’s inheritance model could never match my object model, at least as of Rails 1.0. I also played a little bit with Python’s Turbogears and SQLObject stacks, and they had me feeling pissed and frustrated within seconds. No offense to their authors, but they just did NOT work for me.

But the decision was made that for this site, we would continue with using a relational database. Well, I knew the strengths and limitations of our current systems. And I admit, I was envious of Ruby on Rails. In fact, I even suggested that we ditch Zope for this project and use Rails instead! But it was feared that there was no time on the schedule to learn Ruby. We did, however, agree on wanting a real object-relational system, or at least something more object oriented than what we used for Version 2. Version 3 has richer business logic requirements, and I’m happiest when that code can look as clean and natural as possible. I also wanted it to be easy to define new pages and views and to have to go through less guess-and-hope work than Zope 3 (as of 3.2) typically requires.

So we’ve birthed yet another in-house framework because there’s little out there that’s satisfactory. The one thing that is satisfactory is SQLAlchemy. I built a base storage framework on top of that which allowed us to use SQL Alchemy fairly transparently from within Zope. Fields and properties now let us define and use relationships fairly transparently. Items get bound to their context, are easily traversable, and can even masquerade as Zope containers without interfering with SQLAlchemy’s on system of managing work. We also birthed a base web framework that provides some useful base classes for constructing the kinds of pages and sub-views that we need with ease. These base classes corral some core Zope 3 features and some other in-house features together to ease our web development.

It’s all pretty cool, for what it is. But it’s going so slowly that I can’t help but wonder if we would have been better off working in Rails. I still haven’t found anything in the Python world that I’d leave Zope for, but Zope 3 continues to jump between being insanely cool and powerful and flexible, and being incredibly frustrating and aggravating. That I’ve been able to pull many of the tricks I’ve already been able to pull is a testament to the better parts of its design.

Even with all of my tricks and base classes, it can sometimes still take an entire morning just to get what seems like a simple page together and rendering. On the other hand, once you get pieces in place they’re pretty sturdy. There’s low likelihood of accidentally overriding a critical method name or a view/page that applies to a different context or interface layer.

But as it’s been going so seemingly slow, this development, I start to wonder why it feels that way. I really was trying to give myself a system where I could enjoy the benefits of Zope 3’s Component Architecture with an intelligent database / object-relational mapper while also enjoying the benefits of Getting Real. “Getting Real” isn’t anything really new to me (or many people). It just helps give you the OK to say “no”. Martin Fowler’s book, Refactoring, made me realize that bad smells are OK now, so long as they’re cleaned up later. In the interest of getting something done, it’s hard to go for purity. Or in the case of Python’s “import this”:

Special cases aren’t special enough to break the rules.

Although practicality beats purity.

So, what is the problem with me then?

I think it comes down to a very special case: the major rewrite. This is not a new application for us, nor is it a moderate upgrade or maintenance release. We’ve acknowledged that the existing architecture has issues when it comes to certain growth paths. To this point, we’ve also been good in saying “no” to those growth paths. But business situations and opportunities have changed.

One problem with the rewrite is the knowledge that it’s already been done. “We already did this three years ago, why can’t you show me it working in the new system already?” There senses of worry, fear, excitement all change. This is further exaggerated by the fact that these kind of rewrites are often architectural, perhaps deeply architectural. All of those agile decisions made in earlier versions no longer apply, since many of those decisions may have led to the limitations you’re trying to overcome. Those decisions were right at the time, and have been right for the past few years. But now you realize that being able to do more promotions, discount options, special offers, are among the things you really want to do with the cart. You also need to handle more delivery options instead of the two you’ve used for the past two years. You fight and push and think and tango with the cart, just focusing on shipping, with coupons in the back of the mind. Part of you thinks “I just want to add a shipping column / attribute here and be done with it, and we’ll expand it later…. Oh wait… This IS later… Oh yeah, we need to support more options… Oh yeah, this is one of the growth areas we’re focusing on. Crap!”

It’s an interesting struggle now, those two sides in my mind. In practical terms, I just want to add an extra column or attribute and be done. But I know in a week it’ll already be pushed to its limits. Better to apologize for lack of screens and work this the hell out now. Nothing fancy - keep the interfaces and collaborators simple, and expand on them if needed when more data presents itself.

Last night I floated like a ghost past the strip club, under the freeway, over the railroad tracks, past the big electric sign and refrigeration companies and worn out corners until I got to the super-center. I needed milk, inner tubes for my bike, and dog treats. Of course, my cart started filling up with much more than just that. And in the middle of my 11pm shopping excursion I stumbled across the music/video section. As I stared (half-angrily) at the so-called “top sellers” in the country department (it’s where my cart stopped), I remembered coming across a new Johnny Cash collection on the iTunes Music Store.

So I now have Personal File. It’s a collection of recordings Johnny made (circa 1973-1980) in his studio. It’s just Johnny and his guitar: the same kind of intimate, personal, and raw sound that made the first American Recordings album such a standout. Some of these pieces are recognizable, such as When it’s Springtime in Alaska, It’s 40 Below and Jim, I Wore a Tie Today. Many songs have stories attached to them - why Johnny remembers it, why he wrote it, why he sings it, etc. It was a style seen on some of his commercial albums from that time, although those were accompanied by a larger band and sound.

The second disc of Personal File hasn’t grabbed me much. It’s fairly religious, and while I respect Johnny Cash’s religious strength, I have little interest in it. There are some nice songs on that disc though. But the first disc is the good one. It’s from a good time in Johnny’s life - settling into his life with June, his drug days behind him. The voice is powerful, most of the songs quite rich. A very enjoyable listen.