When I was fresh out of college, my first job was working for the OpenGroup, and my command line looked something like this:

In 2009, fired by the white heat of years of cutting edge research and development, my command line looks a lot more like this:

To be fair, we've now got transparency, anti-aliasing, our shells now do better completion. But that's about it for changes. Considering how much the rest of the industry has changed, that's shockingly glacial.

For the past week or so, I've had the privilege of working on the Bespin command line, and I think there's a lot that we can do to make things better.

This is what my Bespin command line looks like now:

This post is all about how we can make command lines better.

The core of the unix command line philosophy seems to be: small programs and pipe. There’s a lot to be said for loose coupling but there are some drawbacks as well...

Zsh

I love Zsh, and love the fancy completion that it does. Basic file completion has been around since ksh, IIRC:

$ ls [TAB]
Applications/  Mesh/       Projects/
Desktop/       Movies/     Public/
Documents/     Music/      Sites/
Downloads/     Parallels/  backup/
Library/       Pictures/   bin/

But zsh can complete against more than just files:

$ telnet [TAB]
192.168.0.1
192.168.0.12
download.directwebremoting.org
download.dojotoolkit.org
getahead.org
hg.mozilla.com
localhost
people.mozilla.com
sandbox.dojotoolkit.org
svn.directwebremoting.org

Zsh knows that you don’t telnet to a file, so it completes against hosts (Does anyone know where that host list came from BTW?)

Zsh also knows about other commands:

$ git [TAB]
add                 -- add paths to the index
apply               -- apply patch on a git index file and a work tree
applymbox           -- apply patches from a mailbox
applypatch          -- apply one patch extracted from an e-mail
archimport          -- import an Arch repository into git
bisect              -- find the change that introduced a bug
branch              -- create and show branches
checkout            -- checkout and switch to a branch
cherry              -- find commits not merged upstream
cherry-pick         -- cherry-pick the effect of an existing commit
clone               -- clones a repository into a new directory
commit              -- record changes to the repository
...

That list of commands is the result of a 'git help' command, but that's not what I typed. Zsh has been doing something funky behind my back when I pressed [TAB].

Zsh can do different things depending on where in a command you are:

$ git add [TAB]
Applications/  Mesh/       Projects/
Desktop/       Movies/     Public/
Documents/     Music/      Sites/
Downloads/     Parallels/  backup/
Library/       Pictures/   bin/

My point, is that, whilst all of this is very clever, the implementation is also insane.

In order to make this work, the zsh guys have had to reverse engineer every single unix command (or at least all most of the ones that I use). And any time those commands change, then zsh is going to have to change too.

It's very useful, and I'm glad that zsh has done it, and they probably didn't have much of a choice, but this way of going about things is not the Best Idea ever.

I'm sure somewhere the bash guys are working on the same feature themselves, or maybe they've already done it. I don’t care - it just strengthens the point about the lunacy of the situation.

The Unix philosophy of small programs and pipe has a lot to be said for it, but there are serious drawbacks too.

Here's some of the ways that we're innovating when it comes to the Bespin command line, and benefiting from being about to integrate things better.

GUI integration

Traditionally you've had a set of shells (bash, zsh, etc) that work in a set of terminal emulators (xterm, terminal, gnome-terminal, konsole, etc) But there's so much more you could do if you had one program do both.

In Bespin:

• If you click on an old command, it gets copied to the prompt (the blue highlight is a mouseover).
• If you double click it's re-executed.
• Output is not restricted to ascii in a grid. We (predictably) use HTML.
• Error output is visually different to normal output
• You can minimize or maximize output (the help output has been minimized above) - I hate it when a vcs diff operation takes up 4k lines of scrollback obliterating what I wanted to see.

There's no reason why you couldn't do this in Unix (or even Windows). I don't think you’d want such a gui-shell as your login shell, but since its use is very much from within a GUI, that's OK - you’d probably launch it from the GUI anyway.

In Bespin we don't have pipes yet, but would it be neat to be able to inspect a pipeline visually as it's working, I'm sure it's possible with a gui-shell.

Linking Output to Commands

We link the output from a command to the command that created it. The obvious effect is that output from slow running commands no longer gets confused with what's happening now.

The 'slow' command is just something I hacked up to demonstrate asynchronous behavior.

A nice upshot of this is that we can implicitly add & to the end of every command, and you no longer have to wait for anything.

In essence, each command has it’s own separate stdout, which goes to it’s own div. Maybe there is a reason why unix shells don’t do something similar. Would it mean shells being curses based or something?

Command Meta-Data

With the exception of the reverse-engineering of the command line structure done by zsh, the integration between commands and the shell is limited. In Bespin, a command is a structure like this

{
    name: 'createproject',
    takes: ['projectname'],
    preview: 'create a new project',
    usage: '[newprojectname]',
    execute: function(commandLine, projectname) {
        /* code to do the work */
    }
}

This means that we can provide help as the user is typing. The idea is that TAB will guess what you want to do at all times.

This ties in a lot with GUI integration too. If the command line knows that the next parameter is a file, when the user presses TAB without typing anything to complete on, we can offer a dialog to allow selection of a file, and so on. This isn’t about forcing the user to switch to the mouse, but it is about allowing better feedback from the system about what options are available.

We've only really got started with this part - there’s a lot more work to do. I hope we'll be able to do some of the funky GUI integration that Ubiquity has been pioneering.

Attention Guides

There's a lot to look at on any command line, I want to find ways to keep the most important bits in front of your eyes at all times. I'd like to expire older commands more intelligently than a standard terminal (oldest dies first), Commands are more important if:

• They are still in progress
• They keep being re-executed
• They are waiting for input
• etc

We've only basically started on this right now, with the ability to hide/show output, but there's potential for it to make the command line lots more usable.

Plug-ins

Linking everything together is easy if you make one big monolithic lump. That's not the plan - we're evolving a plug-in strategy as we go, and planning on making as much of Bespin itself use the plug-in system. The key to coupling is to get the right level of coupling.

Status Messages

My other integration obsession is about status messages. There's a huge amount we could do to integrate the familiar concept of "what are you doing now" with commit messages, bug trackers, chat, and team tracking. More soon.

What other things could we do to trick out the command line?

Responses[24]

Posted by Joe Walker on 27 May 2009 20:08:19 BST #