A handy bash function for creating "file://" URLs

I often find myself loading test files from my local filesystem into Firefox with "file://" URLs.  Unfortunately Firefox on OS X (or at least my local build of 3.1) is maddeningly inconsistent about recognizing an argument as a filesystem path or a URL.  So for example "ffz test.html" ("ffz" is my launch script) usually opens "file:///Users/cbartley/Dev/mozilla/test.html" (which I want) but sometimes tries to open "http://www.test.html/" (which doesn't even make sense).

It seemed sensible to switch to simply supplying a full file URL on the command line so there's no confusion.  The downside is that file URLs need to be absolute and constructing an absolute file path on the fly is a pain.  Fortunately, I can make the computer do this work for me with a handy bash function.  The "fileurl" function below takes a path to a file (usually a relative path) and prints the corresponding "file://" URL for it.  So for example:
  fileurl ../test.html
might print
  file:///Users/cbartley/dev/mozilla/test.html

I can invoke my test build with something like
  ffz `fileurl ../test.html`
and have it reliably open the file every time.

I haven't yet rolled fileurl into the ffz script, but that's the logical next step.  I should also acknowledge that there may well be better ways to do this than the approach I've used -- I am no bash expert.  Finally, for all those other amateur bash programmers out there, the magical Google phrase for finding documentation on the weird things you can do with variables in bash is "variable mangling".

# Given a path to a file (relative or absolute), print a "file://" URL for
# that file.
fileurl()
{
  # Split the directory name out of the argument path.
  #   "/dir/subdir/file" ==> "/dir/subdir"
  #   "dir/subdir/file" ==> "dir/subdir"
  #   "subdir/file" ==> "subdir"
  #   "file" ==> "."
  TEMP="/$1"                            # Hack: Prepend a slash so there's at least one
  TEMP="${TEMP%/*}"                     # Chop off the trailing "/file"
  TEMP="${TEMP#/}"                      # Remove the leading slash if it's stil there
  DIRNAME="${TEMP:-.}"                  # If DIRNAME is empty, set it to "."

    # Get the base file name from the argument path.
  BASENAME="${1##*/}"                   # Remove everything up to the last slash, inclusive

    # Convert the directory name to an absolute path.
  ABSDIRNAME=$(cd "$DIRNAME"; pwd)

    # Echo the file URL built from the components.
  echo "file://$ABSDIRNAME/$BASENAME"
}

Firebug UI: Tabs on the Top

In a previous post I explored the idea of moving Firebug's main panel bar tabs to the bottom of the UI.  One of my teammates pointed out that a common scenario in Firebug is to select a tab and then immediately click in the toolbar to make a panel specific adjustment.  For example, it's common to click on the Script tab and then to click the files drop down to select a specific JavaScript file.  With the current arrangement, the toolbar and the panel tabs are close together regardless of Firebug's window size.  If Firebug's window is opened to a large size, for example showing 40 or 50 lines of source code, then this two step usage scenario is going to be rather inconvenient for the user.

If this is a common scenario (and I suspect that it is), then this argues for a tabs-on-top design, which is what I'm going to look at in this post. 

There's a link to the prototype at the end of the post if you want to try it out.  In the meantime, here's a screenshot.


A screenshot of the Firebug UI with panel tabs at the top.  The Script panel is selected.

One of my goals with the previous tabs-on-the-bottom design was to reduce the "busyness" of having the toolbar and main panel tabs right next to each other.  This design doesn't do much to address the busyness angle, but it still seems cleaner than the current Firebug layout.  For example, in this design the tab strip for the main panel bar is at the top of Firebug's UI, and all it has on it is tabs, and the tabs are largely static -- the selected tab can change, but that's it.  It's a much simpler UI element than the toolbar.

The toolbar is placed such that it appears to be inside the currently selected panel, which seems logical since so many of the toolbar controls are dependent on which panel that is.  In this particular variation, the toolbar extends all the way across the window, so I call it the "long toolbar" version.  One negative of this design is that the side panel looses a line of vertical real estate.

There's another advantage of having the tabs on top.  In Firebug, the major functional units correspond to individual panels in the main panel bar.  In the tabs-on-top layout, the tabs for these panels is the most obvious thing you see.  I think that's probably a good thing.


Moving the Search Box


A mockup with the Search Box and window controls moved from the toolbar to the tab strip.

One possible variation on the tabs-on-top layout is to move the Search Box and adjacent window buttons from the toolbar to the right end of the tab strip.  Since the Search Box and window buttons don't change with the active panel, it makes sense to pull them "out" of the panel.  This clutters up the tab strip a little bit, but as long as there's some reasonable amount of blank space between the rightmost tab and the search box, it doesn't really seem to hurt that much.  One advantage is that it substantially reduces the clutter on the toolbar and another is that it frees up a lot of space for the "object path", which, unfortunately, you can't see in this screenshot.  (In the Script panel the object path is the call stack in horizontal layout.  In the HTML panel, the object path is the HTML containment hierarchy.)


The short toolbar version.



With the Search Box and window controls on the tab strip, we can also do a "short toolbar" layout as mentioned above.  In this layout, the toolbar is only as long as the left sub-panel.  The side panel bar (containing the Watch, Stack, and Breakpoints panels) now tops out just below the tab strip, reclaiming that line of vertical real estate that we'd lost in the first two layouts.  Of course the short toolbar reduces area available to the object path by quite a bit.

Conclusion

I think putting the tabs on top and above the toolbar makes for a much nicer UI.  Although I think my earlier tabs-on-the-bottom UI offered some advantage by completely separating the toolbar and the tab strip, I think the tabs-on-top design offers some clear advantages.

  • The tabs are the major organizational principle in the Firebug UI and putting them on top makes them stand out as the most prominent UI element. 
  • The toolbar appears like it belongs to the current tab page, almost like you're tabbing between different toolbars as well as different panels.  This seems logical since the toolbar changes with the panel anyway (The ever present Firebug-icon-menu and Inspect button do muddy the water a bit, though).
  • Selecting a tab and then selecting something on the toolbar (e.g. selecting the Script tab and then selecting a JavaScript file from the drop down) is still convenient.  It feels a little more natural to select something on the tab strip and then move down one line to carry out the next step.  In the current UI with the toolbar on top, the flow is up rather than down.
I haven't tested it in a prototype yet, but after looking at the mockups, I think moving the search box and window controls to the right end of the tab strip is a good idea.  I'm less certain about whether the long toolbar or the short toolbar is the best approach, although I feel like the short toolbar wins on simple aesthetics.

Try it out
A link to the Firebug "tabs-on-top" prototype extension is at the bottom of this post.  The standard disclaimer applies.  It's prototype code built on 1.3 pre-release code, so although it's OK to play around with, you probably don't want to leave it installed.



A Firebug User Interface Proposal

One of my biggest complaints about Firebug is the way its tabbed UI works.  Firebug organizes its features into six major functional units: Console, HTML, CSS, Script, DOM, and Net.  Each unit gets a "panel" and these panels are organized in a tabbed display called a "panel bar".  There's also a smaller panel bar that sometimes appears to the right of the main panel bar depending on which primary panel is selected.  The main Firebug toolbar appears just above the panel bars.  Below is a screenshot from Firebug and an annotated version of the same screenshot.



A screenshot from Firebug 1.2.1.  The Script panel is currently selected.




The same screenshot as above with the major UI components labeled.

One of the things I don't like about the current UI is just how busy it looks.  The toolbar is already pretty complex, and it's stacked right on top of the tabs for the two panel bars.  That's not the biggest problem however.  Take a look at this next screenshot.



A screenshot from Firebug 1.2.1.  The HTML panel is currently selected.

In this screenshot, the HTML panel has been selected.  Look carefully at the toolbar, and compare it with the first screenshot above where the Script panel was selected.  The Inspect button and the find textbox are the same, but everything in between has changed.  Now look at the side panel bar.  That's right, it's completely different.  Changing panels in the main panel bar essentially causes the whole Firebug UI to change, not just the main panel.

I think you can make a case that the toolbar and the side panel bar belong inside the current panel since they are so dependent on which panel that is.  The argument is stronger for the side panel bar since it is completely dependent on the main panel bar.  The argument is weaker for the toolbar since the toolbar doesn't completely change, but I think you can still make a case there.

Tabs on the bottom

So I have two problems that I'd like to solve.  I'd like to make the user interface less "busy", especially with regard to the toolbar.  I'd also like to promote the illusion that both the toolbar and the side panel bar are inside the currently selected main panel, since both those UI elements are so dependent on which panel that is.

I've built an experimental variant of Firebug to test out an alternative UI that I think addresses these issues.

To address the busyness issue I've moved the main panel tabs from the top of the panel bar to the bottom.  This also helps differentiate the main panel bar from the side panel bar, where I've retained the tabs on the top.  I think this helps a little bit with the illusion that the side panel bar is actually inside the main panel since the two panel bars now look different.  Then I've tweaked the UI so that the main panel bar tabs appear to extend all the way across the window and all the way around both panels and the toolbar.  This effect is perhaps more subtle than I'd like, but it still feels like an improvement.

Below is a screenshot of this alternative UI.



A screenshot from an experimental version of Firebug with the main panel bar tabs on the bottom.

It might also be useful to evaluate the design where the main panel bar tabs are moved above the toolbar.  This won't do anything for the busyness issue but it might make a stronger statement that the toolbar is dependent upon the currently selected panel.  (Firebug issue 222 suggests making exactly this change.)

Take it for a spin
I've attached the XPI file for my experimental Firebug build so you can try it out yourself.  That will give you a much better feel for the proposed change than just looking at static screenshots.  The experimental build is probably a little bit buggy, both due to my changes and the fact that it's built on Firebug 1.3 pre-release code.  It should be OK to take for a test-drive, but you probably don't want to leave it installed for a prolonged period of time.





Using MemTrack to instrument Firefox's memory management


MemTrack is a library for tracking memory allocations in C++ programs.  I had two goals with MemTrack.  The first was a simple demonstration of the classic C/C++ preprocessor-based technique for instrumenting memory allocations, which I'm sure a lot of people have done, but few people have documented.  The second was as a testbed for a rather novel technique for tagging each memory allocation with type information, something that is not normally possible with the preprocessor macro-based approach.

The macro-based approach has some serious limitations, but for some codebases it can work quite well.  I tested against my FH-reader library (a library for reading Macromedia FreeHand files) which was an ideal test case.  All I needed to do was add a '#include "MemTrack.h"' directive in a strategic location and then add one call to the test harness to dump the gathered information after the test run was complete.

I was curious how well the technique would work for a large real world codebase, so I tried it out on the Mozilla Firefox codebase, which is on the order of 2 million lines of C++.  This was kind of a brutal exercise, but I was able to get some potentially useful data out of the effort.  The Firefox team has already used other tools for the substantial memory usage optimizations that were done for Firefox 3.  MemTrack's memory tracking would be a poor substitute for what those tools would provide.  However, MemTrack's ability to annotate allocations with C++ type information might be a useful extension.  My understanding is that type information was already available for XPCom objects but not for other C++ types.

In order to use MemTrack at all on the Firefox codebase, I had to resort to a static pre-processing step.  One purpose of this step is to insert a '#include "MemTrack.h"' directive at the beginning of every .cpp file.  The other major purpose of the static pass is to temporarily suspend the new macro in places where it's going to cause problems.  A common case is any class that declares and defines a custom operator new.  The static preprocessing code is simply a python script which relies on a rather scary regular expression which identifies individual lines in the source code as being problematical.  The script simply undefines the new macro above the problematical line and redefines the macro below it.

For exampe:

#ifdef MEMTRACK_NEW        //:memtrack!  #undef new               //:memtrack!#endif                     //:memtrack!  void* operator new(size_t aSize, nsIPresShell* aHost) {#ifdef MEMTRACK_NEW        //:memtrack!  #define new MEMTRACK_NEW //:memtrack!#endif                     //:memtrack!

There aren't that many files that declare or define operator new but for expediency I ended up wrapping any preprocessor directives that include system headers.  This is probably not a big deal either way, since one would rarely do much debugging or development against an instrumented build.  Nevertheless the amount of explicit instrumentations could probably be reduced, and that would be a good thing.

To sum up the current state of this exercise: I have to statically pre-process the Firefox codebase in order to use MemTrack.  This preprocessing step is pretty much completely automatic on the Mac although it's still pretty crude.  To make it actually useful, MemTrack's tracking layer should be stripped out and the MemTrack instrumentation layer should be piggy-backed on top of the existing memory instrumentation code.  The required static preprocessing step is somewhat of a hassle, but it's not really all that onerous.  If one has to do a special build of Firefox to instrument memory usage anyway, it's probably not a big deal to include a static source preprocessing step as well.  The key question is whether the extra type information that can be gathered by MemTrack is sufficiently useful.  I think that it might be, but I'll need to do some more investigation before I can say for sure.

For future reference, the regular expression I'm using is:

    newIgnorePat = re.compile(r"operator new|include NEW_H|NS_DECL_AND_IMPL_ZEROING_OPERATOR_NEW|::new[ (]|include [<]|[)] *new[ (]|[*] new[ (]|!new[ (]|include.*nsILocalFileMac|include.*nsIInternetConfigService|new PathChar")

This pattern includes common cases like occurrences of "operator new" (not just "new") and include directives for system headers as well as some special cases like "new PathChar", which is specific to Firefox on the Mac.

Firebug UI Notes: Is it a button or is it a menu?

Firebug crams a lot of disparate UI into a relatively small amount of screen real estate, and consequently it can't help but be noisy and confusing.  However, I think there's some obvious areas for improvement, and I wanted to document some of the problems I've been noticing as I go along, and maybe propose some solutions while I'm at it.

First up: Is it a button or is it a menu?

One of the most notable aspects of the Firebug UI is that it wants to stick menus all over the place, including on some of the tabs.  Firebug is not very consistent about how it does this, and so the user is left wondering: Is it a menu, or just a button?  For example, in the screenshot below, you can see two Firebug bug icons -- one on the right end of the Firefox status bar at the bottom of the window and another one (slightly larger but otherwise identical) as the first control on the Firebug toolbar.  The small icon behaves as a simple button -- it toggles Firebug on and off.  The larger icon actually triggers a menu.

Next on the toolbar is the Inspect button, which always appears in the location, regardless of the current tab.  The Inspect button triggers a mode where you can click on elements in an HTML page and have them show up in Firebug.  Continuing to the right, the "all" button actually controls a menu, and there's a helpful arrow next to it to indicate that there's a menu there.  Next is the button that's labeled "slideshow.html".  It's also a menu, only this time there's no helpful arrow to indicate that.

One simple improvement that can be made here is to consistently use the arrow next to all menu items.  I'd also consider replacing the Inspect label with an icon, and the Firebug icon with just "Firebug", so the button is an icon, and the menu has a label.  This sounds more natural to me, although in terms of actual usability, I don't know if it would make a big difference.

This is just scratching the surface, but it seemed like a good place to start.

Firefox weirdness on OS X when launching from the command-line

Mozilla issued me a new MacBook, and I've been trying to get it set up for development.  In particular I wanted to get it set up for Firebug development since that's my primary focus right now.  This should be easy -- Firebug is just an extension written in JavaScript.  However, one thing I wanted to do is set up a build script that will automatically launch Firefox using a specific profile just for testing.  It's not at all hard to launch Firefox from the command-line in OS X, except that I was getting really bizarre behavior: A Firefox window would appear, but there would be no menu bar, and indeed, Firefox didn't show up in the applications list when you hit command-tab.  Even worse, there was no way to get keyboard focus into the window.  You could close the window by clicking on it, but then you'd have a zombie Firefox process that you couldn't quit, and you couldn't Force-Quit it, because it didn't show up in that list either.  It was necessary to kill it from the command-line.

I tried everything I could think of, and a handful of times, everything worked, and then it went back to being broken.  Eventually I discovered that the path to the executable that I typed out differed from the path in the filesystem by the case of a single letter.  I had /Applications/Firefox.app/Contents/MacOs/firefox-bin, when what I really wanted was /Applications/Firefox.app/Contents/MacOS/firefox-bin (MacOs ==> MacOS).  That was it.

Addendum: This is now bug #453913 in Bugzilla.

My first day working for Mozilla

Today was my first day working for Mozilla Corporation.  It was pretty uneventful, which is no real surprise since I'm working from home in Seattle rather than from Mountain View or Toronto.  I will be splitting my time between Firefox UI and Firebug, although I'm not sure what that means yet. In the short term I'm looking at Firebug, which has the benefit of being a much smaller project than Firefox, both in terms of source code and number of people working on it.  If that means I can make a useful contribution in weeks rather than months, then I'm all for that.

Deep Search: Extending Google search into the web page

My most recent project has been a Firefox extension called "Deep Search". The idea is to extend Google search one level "deeper", into the returned web pages themselves. This is by no means a new idea. Google already does this for cached pages -- if you view a cached page the search terms will be highlighted. Google can't do this for the real web pages, of course, but the Deep Search Firefox extension can.

Deep Search also attaches decorations ("marks") to the scroll bar to show at a glance where in the web page the search terms were found. One of the limitations of the current Deep Search extension is that the marks appear next to the scroll bar rather than directly on it. This is not ideal but it's acceptable for a proof-of-concept. You can click on a particular mark to immediately scroll to that search term match.

I've attached a screenshot below. I've circled the changes made by Deep Search in red. The search term is highlighted and the "mark" is right next to the scrollbar thumb in the lower right corner. The gray line at the top of the mark shows the precise position of the search term relative to the scrollbar. The red area below it is, not coincidentally, roughly the same size as the thumb. It shows where the thumb needs to be positioned so that the search term corresponding to this mark appears at the top of the window.

Crane City

When I was in Indonesia last year, one of the things I remember is how many buildings were under construction in Jakarta, each one with a tower crane. Then I got back to Seattle. I just shot this picture from my building's rooftop deck. There are three other cranes nearer by that I couldn't capture without a wideangle lens. It's pretty fuzzy, so I think I'm doing something wrong.

Who do I talk to about Firefox UI ideas

I have an idea for a Firefox feature that I think would be useful. I'll get into the idea itself in a later post. In the meantime, I've got a Firefox extension that I've written as a proof of concept, and I wasn't clear on what to do next. So I emailed my contact on the Firefox development team for his advice. Since I expect this advice will be useful for others, I'm repeating it here.

  • Upload the extension to the AMO (addons.mozillla.org) sandbox.
  • Blog about it, pointing to the AMO page.
  • Post to DAF (mozilla.dev.apps.firefox newsgroup or its mirror mailing list).
  • Go to #ux on irc.mozilla.org, point to your post, and ask for feedback.