After spending significant time today upgrading the handling of my email, I would like to share some of the techniques I use to manage my email over many separate email addresses. My focus here is on getting emails automatically sorted so that certain categories can be ignored and others seen immediately as circumstances require. Ignoring things that are ignorable is difficult when all of your email is jumbled together. And missing emails that demand attention is not acceptable.

As the post goes on it will become more and more specific to the mail reader I use, Apple Mail 4.0. I do not address spam filtering in this post.

TIP 1: Use separate email addresses for different kinds of emails

For example, I use the address consult@crotown.com for emails that are sent by the contact form on this web site and for subscriptions to various web services that I use for Cronen-Townsend Consulting. And the address steve@templeisraelgreenfield.org is for emails relating to my volunteer work for the local synagogue. In my mail reader, emails are pre-sorted, and I never need to open the steve@templeisraelgreenfield.org mailbox during consulting work, for example. The number of unread emails there may increment, but since I know that a new email there can wait by virtue of the email address it was sent to, it will wait appropriately for my attention until after the paying work ends for the day.

TIP 2: Use Smart Mailboxes for cross-mailbox categories

Smart Mailboxes are a term Apple uses to denote mailboxes which are full of pointers to email messages in regular mailboxes, and are filled according to a definable set of rules.

Smart mailboxes are useful since they allow one to create something that appears with the list of mailboxes, but goes from no icon next to it to a one, as soon as you receive an email fitting its criteria.

I have one smart mailbox for each client and one for my wife. I don't want to miss an email about dinner arrangements for tonight, because it was mixed in with all my other emails and I didn't notice when a mailbox count went from 101 to 102. When I find an email that was not included in a smart mailbox that I think it should be in, I add to or edit the existing rules defining that smart mailbox until it is included. Your own individual experimentation will be required, but you get the idea.

TIP 3: Use Rules to move emails from each mailing list to its own mailbox

Rules are a term Apple uses to denote filtering criteria that trigger certain actions on email, when they are received (or later).

Even after applying the first two tips in this post, you may find yourself missing mail in a certain mailbox because it is overwhelmed by mail from a certain mailing list.

For example, I subscribe to the groups.drupal.org OpenPublish group, which has bursts of traffic. So, in order not miss other emails during burst, I have a Rule that automatically moves any email whose subject contains "groups.drupal.org OpenPublish" to mailbox called "OpenPublish" that lives on my Mac. Then I can read it an appropriate time, when I have time to read it, and more pressing email that comes to the same mailbox will still be seen during a burst of traffic from that group.

As our use of email grows, the need for techniques to manage it so it remains an effective tool for us increases. Hopefully these three tips contribute in a small way to that ongoing discussion.

As you may have noticed, I recently honed the 'mission statement' for Cronen-Townsend Consulting from "Building custom Web architecture with Drupal" to "Building custom Web platforms based on Drupal". Here is the story behind the change.

Recently, I listened to the Acquia webinar Amazon Web Services Building Blocks for Drupal Applications and Hosting. The first part of the presentation was by Jeff Barr, Senior Web Services Evangelist at Amazon, who explained Amazon Web Services. In his presentation he used the terms IaaS and PaaS, for "Infrastructure as a Service" and "Platform as a Service." respectively. The talk was followed by an explanation by my friend Barry Jaspan of Acquia's Drupal hosting environment, which is built on Amazon Web Services.

The term inrfrastructure was used for rather low level services, and it occurred to me that Acquia's hosting environment, may be described as architecture building on the IaaS Amazon provides. On the other hand, higher level structure was called "platforms" and "applications".

So I decided to upgrade Cronen-Townsend Consulting's mission statement to use the term "platforms" instead of "architecture". When I build a single web site based on Drupal, what I am really delivering is a customized platform which is a good foundation for future growth. This hits the real strength of Drupal-based solutions. And the word "platforms" also hints at the way that I can deliver, using Drupal's installation profiles, a package that can be used to install a single website each time it is executed. And the nearly limitless possibilities provided by things like multiple web sites running on the same Drupal PHP code and sharing a common database, and on and on...

Here is a little post to help you try Apture searches. This is going to be easy since I have Apture 2.0 installed on this very web site.

Suppose you are reading my previous post and you see the phrase "Having Apture integration built-in to a site's infrastructure lowers the barrier to trying it out" and you are wondering what exactly the term infrastructure means. So, highlight it (select it).

A search button should pop-up as shown when you click here. Go ahead and hit the search button.

You will be rewarded by the appearance of the Apture Magic Search Bar and an Apture window.

Scroll and select the link you want (probably the dictionary.reference.com link, in this example). And there you have it — anything becomes a link, according to your own needs.

You can make the toolbar appear by simply scrolling down any page until the web site's header is off the screen. Then you can type anything you want Apture to search for. Try something from popular culture and browse videos. Have fun!

We recently switched the new site development for Boston Review(BR) from a bare Drupal underpinning to an OpenPublish underpinning. In this post I share my early impressions from a the perspective of a developer and site-builder.

[OpenPublish is a Drupal distribution from Phase2 Technologies based (currently) on Drupal 6.x with many module choices, configuration, and default theming intended to be useful to publishers.]

1. Installation is the same as with Drupal

Obviously, as a Drupal developer, it's important to me that OpenPublish is based on Drupal. But I was pleasantly surprised when I didn’t have to learn anything new to install OpenPublish.

The fact that I got over sixty compatible installed modules with lots of basic configuration and default theming right out-of-the-box, really piqued my interest. I knew that if it was consistent with what I wanted to build, it was going to save me immense amounts of time.

2. Streamlined administration

Take a look at the Drupal reports page in OpenPublish:

The well-designed icons are the first thing I noticed, but the way the whole administrative experience is streamlined also hit me after I got used to the administrative menus that appear to the left of each page (thanks to the Admin Module). One of the many benefits of this is that clients get the impression “Hey, an OpenPublish-based site is actually usable by me”. This is very important, since many potential clients have anxieties about taking care of a new site – and an OpenPublish test site looks (and is) manageable.

3. Power “under the hood”

One of the first things I noticed about OpenPublish is that the platform makes the same choice of fundamental modules that I already had chosen for the data-, image-, and multimedia-handling needs of the Boston Review. I’m speaking here of things like Views, CCK, ImageField, ImageCache, and Embedded Media Field. I knew that these were very solid choices in terms of upgrade-path for Drupal 7 and beyond. But I was also taken by some of the other advanced possibilities in OpenPublish, like Apache Solr search integration, that also matched my choices. So I suspected, right away, that the various semantic technologies also built into OpenPublish (see later entries on OpenCalais and Apture) might be appropriate for BR as well.

4. Attractive default theming

Take a look at this article view from my Boston Review development site:

This article, despite the fact that it can’t display custom Boston Review fields like “discussed works”, gives a good idea how BRs content will eventually look on their new site. So it is important that the default layout is good, so clients can get an idea of a new site before a designer and themer get to work on it. People wanting a new site just respond better to a site that is easy to look at.

5. Attention to detail: Author creation and display

A issue that publishers are faced with in setting up a Drupal site is that the actual author of a given piece of content does not generally have an account on the Drupal site. So Drupal’s out-ot-the-box authorship mechanisms do not suffice. But in OpenPublish each actual author is an object of type Author, and these objects are created and displayed flexibly and easily.

When a person creates a new article on an OpenPublish site, the part of the form where one sets the author looks like this:

One great thing about this is that if the article is by a new article, one can simply hit the “plus” icon, create the new author in a pop-up window, and then automatically reference the newly created Author instance. This sort of attention to detail carries through to viewing the article in the default OpenPublish theme.

So, for example, if I create myself as the second author of this piece, the end of the article will look like this:

Note that the panel says “About the authors...” automatically and lists us both. What a pleasure.

6. Organization with the Features module

OpenPublish organizes related views, content types fields, imagecache settings, and the like into “features” which can be enabled or disabled as a group, and are version-controlled on http://drupal.org. This emerging module provides a needed and sensible level of structure to the complex platform.

7. Bidirectional links with the Relationships Module

Articles have Authors in OpenPublish, but Authors at the same time refer back to the articles they have written. One manifestation of this is that viewing an author automatically shows the articles they have written:

And using this capability in my own custom node-reference fields has been a snap.
NB: I have never actually co-authored anything with Craig Morgan Teicher

8. Search with Apache Solr

Once I set up a Solr Search server one of my first tasks was to turn on a block for automatically sorting search results different ways. Once I learned that block visibility is controlled by the Context Module in OpenPublish, and is not set on the usual blocks page, it was a snap to produce the following search results page.

9. Automated linking (and more) with Apture

With OpenPublish, one can automatically choose a link with the web service Apture. For example, when entering a story about the world bank, one can highlight the words "world bank" and hits the Apture link icon.

This gets you a pop-up that asks you what kind of link you want:

And when you click the Create Link button the appropriate HTML code is inserted in your article.

It's as easy as that. This barely scratches the surface of the services available through Apture which include embedding content and automatic links to your content.

10. Automated tagging (and more) with OpenCalais

OpenCalais is a service powered by Thompson Reuters which automatically tags content. The text of content of types you choose is sent to OpenCalais in the background, semantic analysis is performed there, and appropriate tags arereturned to your site and added to your article. For example, default tagging of articles produced all of the following tags except "On poetry" and "January February 2010".

This addresses a fundamental problem in making the web more semantic, and a daily struggle of editors everywhere - "Whose job is it to tag the content and maintain the taxonomy?"

As you can see, OpenPublish is allowing me to leverage vast efforts on the part Phase2 Technologies and the Drupal community effectively for clients in publishing. And in very practical terms, it is helping me deliver a new site to Boston Review with capabilities that neither of us dreamed of at the start of our project. Capabilities that will bring tremendous value to the Boston Review and tremendous connection to the Boston Review readership.

At Cronen-Townsend Consulting, I am building the back-end of a new web site for the Boston Review using the Open Source content management framework Drupal. The site will incorporate current standard features, like a block showing the most recent comments, sharing with all popular social-network sites, and a block showing today's most popular content.

Because the new site is being built on top of the powerful content management framework Drupal it will be able to easily incorporate future web innovations as the Boston Review community evolves. There are currently Drupal modules that implement all web technologies (at around 4000 contributed modules, and growing rapidly). And with the Drupal community doubling in size each year all web-technologies should be implemented for the foreseeable future. (See the recent keynote at DrupalCon 2010 by project-founder Dries Buytaert).

In addition, I am designing a process that can automatically post the content of an entire issue of BR on a private preview site for the editors of BR, save an archival XML version of the content and associated files, and push any content to the live site when and if the BR staff chooses. The archival form is being designed to be easily-translated to future format (such as new eBook formats), so this will put the BR content on a solid foundation for the future.

The new site is going to live on a Rackspace cloud server, which will allow it to accommodate spikes in demand, feature sophisticated Apache Solr searching, and position it to incorporate new hosting technologies for Drupal soon after they are developed. (Rackspace was a platinum sponsor of DrupalCon 2010 and I learned a lot about cloud hosting at the conference. They also partnered recently with Dries Buytaert's company Acquia).

All together these activities comprise what I call architecting the future of the Boston Review on the web, one of the most exciting projects I have been involved in.

UPDATE May 10, 2010
The new site will be on OpenPublish! I already have some OpenPublish test sites up and have one configured to search with a separate Apache Solr server and sort the results flexibly. Now I am converting my code which parses an entire issue of Boston Review and imports it to work with OpenPublish. I will update this article with progress.

UPDATE June 2, 2010
Boston Review staffers are playing with a draft of the new OpenPublish-based site that auto-imports content from the most recent three issues. Even with just the default OpenPublish theme the new site is a beauty!

The focus of the Boston Review is its print edition. So one task in creating their new web site has been creating an automated system for parsing the InDesign files that lay out an issue of the print edition, and creating the files needed to post the entire issue, if they desire, to their new web site.

The best answer to this task is clearly not one huge script, but a collection of smaller scripts that call one another, pass variables, and share one common log file (for my own sanity when debugging). In this article, I list some of the issues I have overcome in setting up such a collection of JavaScript files, rather than one enormous JavaScript, to script InDesign CS2.

Setting up a library

The first thing I wanted to do was set up a library, which is to say a collection of JavaScript functions in a common file, say, that other JavaScripts use. To do this, I used the ExtendScript directive #include to grab the library, which just has a lot of common functions, like so:

The full header of my files

There are a couple other important pieces at the top of all my scripts in this collection, so the complete header looks like this:

The '#strict on' turns on strict error checking, which always seems like a good idea top me, having lots of experience in C/C++. The '#target indesign' says the JavaScript refers to the InDesign application and its Document Object Model, so a dialog will pop up when you run a script and there is no InDesign instance running. And the directive '#engine importerToDrupal' makes all the scripts into one group.

Things to watch out for

There are a number of important issues that came up for me around this modular programming environment.

The first is that the representation of the engine (your collection of scripts) happens within InDesign, so the first time you invoke one of your scripts must be done through InDesign's "Window -> Automation -> Scripts" pane for your grouping of scripts to be recognized as such. This should cause "importerToDrupal" to become a choice in ExtendScripts dropdown list of available scripting engines. Unfortunately, in my case, I must switch the target to something else and then back to InDesign CS2 in order for the dropdown list of scripting engines to be updated. After my "scripting engine" importerToDrupal is selected in the ExtenScript Toolkit (ESTK) things will continue to function smoothly even if I instantiate one of the scripts directly from ESTK, rather than from InDesign, in order to debug it.

This leads directly to the next important consideration, which is that any included files (like importWebLibrary.jsxinc) do not appear in the debugger. So I generally get new functions working tacked on to the source file I am debugging before I move them into the library file itself.

Calling another script, and passing variables

You may be wondering, if you've hung in to this point, how to call one script from another and pass variables. Here is a code fragment from my code demonstrating the way I do this:

Note that I am executing a one-line script to pass the variable DocumentPageRange. It does the job, but if you have a cleaner way, please let me know.

By now you may have realized that 'log()' is the function that was the subject of my previous post. And since it automatically tacks the passed string onto the end of the log file, all I need to do is pass it the same logFile variable to get the one log file behavior I wanted, regardless of where in the collection of scripts I begin execution.

Since processForumFile.jsx acts on the active InDesign script I can execute it directly through ESTK in order to debug it and it still functions correctly. All I have to do, in my case, is make sure that processForumFile.jsx does something reasonable if the variable 'DocumentPageRange'.

Taking all of these things together makes a reasonable system for architecting a collection of modular JavaScripts to perform a complex task, and be clear to future programmers who may have to maintain your solution.

The upcoming DrupalCon in San Francisco from April 19-21, 2010 will be my first. I'm thrilled to return to the West Coast as a Drupal developer, having fond memories of the Human Language Technologies 2002 conference in San Diego (as a computer scientist) and the Materials Research Society's Spring meetings in San Francisco in the mid-90's (as a physicist).

Meeting many collaborators on Drupal "in real life" (IRL) for the first time will mark quite an important career milestone for me. To make the most of this opportunity I plan to participate in the code sprints the day before and the day after the main conference day. So see you in SF April 18-22!

UPDATE May 14, 2010
Participating in DrupalCon SF 2010 was amazing. I caught up with an old physicist friend of mine, jhodgdon, at the sprint on the April 18, 2010. She's a colleague, again, now that I am part of the Drupal movement. At Dries' State of Drupal keynote the next day I saw her handle projected on HUGE screens - she is the #7 top contributor to Drupal 7! The combination of Dries' talk and Tim O'Reilly's keynote the next day was very powerful for me, as I came to grips with my small place in the movement at the same time I was introduced to the notion that the movement as a whole is playing a important role in keeping part of the evolving "Internet Operating System" free. Heady stuff. Then I worked on a Drupal 7 core issue on the 22nd. Back in the trenches, so to speak, but with a new sense of the whole endeavor.

Today I got my environment for debugging PHP web pages working much more snappily, and I'm excited to share my tips. My experience only applies directly to you if you are running Eclipse and debugging PHP on a Macintosh with OS X.

Last weekend I scored a free VGA LCD monitor from my brother-in-law who didn't take it to law school with him and is now loaning it to me for at least a couple years. I was thinking "Free screen real estate, what could be better? I should be able to put the debugging web browser on one and screen the rest of my debugging windows on the other." After I got the right adapter to connect it, that is.

Once it was connected, the new monitor was somewhat convenient, but I quickly discovered that I couldn't put my Eclipse internal web browser in it's own window and move it to the other monitor. For some reason, Eclipse kept insisting on opening a new internal browser in the pane with the editing windows even if I had already opened the internal web browser in it's own window with "Window | Show View | Other... | Internal Web Browser" before I started the debugging session. And I was not allowed to "detach" the internal web browser tab from that pane and make it a separate window.

So I tried setting Eclipse to use an external browser for debugging at "Eclipse | Preferences... | General | Web Browser", reasoning that I could surely leave the external web browser's window on a different monitor from the Eclipse window I was using. Well, after a lot of time trying things I was left with only Firefox seeming to work. But it seemed slow and if Eclipse was running, Firefox was *always* starting the debugger in Eclipse and being very slow. Firefox is my browser of choice because of the Firebug plug-in, and Eclipse is about always running on my system. I just couldn't stand my main tool always being slowed to a crawl.

So I started trying every other Mac browser I could find until I discovered Camino. And it worked and even got invoked by the Eclipse debugger at the right time, but unfortunately it gave me errors like:

So here the main tip. Edit the settings for launching Camino at "Eclipse | Preferences... | General | Web Browser". In the Parameters edit box add the command-line option "-url %URL%" and Camino will start working as your Eclipse debugging browser.

The coolest thing is that my debugging between breakpoints started going several times faster. I was thinking that perhaps I needed a new computer to get faster debugging. But, it seems, that Eclipse's internal web server really bogs it down. So not only do I get more space for my other debugging panels with the new scheme I get a much faster PHP debugging system overall. Good deal! It would have been a good idea to do this even without getting a second monitor. If only I had known sooner that my setup for PHP debugging could be so quick and responsive. But at least I know now, -- and hopefully reading my story will help some fellow developers.

When setting up debug configurations for Eclipse, I often have my files living in a subdirectory of my web server root. For example, at directory drydock/ I have a full replica of whatever live site I am working on.

On my hosting, I have the same directory structure and a subdomain set up so that drydock.crotown.com maps to the drydock subdirectory. Then the site drydock.crotown.com lets me put changes on the web for testing and approval, before the actual live site is touched.

On my local machine, until recently, my only domain was http://localhost:8888. So for debugging the page users/crotown, for example, I would have the "file" set to /drydock/index.php and the "URL" set to drydock/users/crotown. This worked fine if I was debugging the generation of that page, but any redirects would not work right since the base URL was http://localhost:8888 and NOT http://localhost:8888/drydock as it should have been. So the URL for redirects would be missing the "drydock/" part and cause a "Page not found" error. Also, paths to image files would be missing the "drydock/" part resulting in lots of missing images when the site was viewed.

The solution? I created a virtual domain called drydock.local so that the URL http://drydock.local:8888 will automatically map to the drydock/ directory and the URLs would not have to have have this part added to work correctly. To do this, I followed the instructions here and here on the MAMP Forum.

All I had to do was add the new virtual server to Eclipse at "Preferences-->PHP-->PHP Servers". Now when I visit "Run-->Debug Configurations..." I can select the new virtual domain in the "PHP Server" drop down menu. Then I can set the URL at the bottom of the Server pane, without the "drydock/" and everything, including all the redirects and images, works correctly in my debugging instance. Well, everything except whatever I'm debugging...

Chatting is something that makes a community site seem so much more vibrant and dynamic. I had looked into chatting options using contributed modules to Drupal 6 at one point but never implemented any of them.

But recently I was contracted to write a simple RealChat module for Drupal 6 that allows users of a Drupal 6 site to chat easily using a RealChat server. On clicking a link, users automatically start a chat session with their Drupal username as their "nickname", Drupal picture for "avatar", and profile on the Drupal site linked to their chatting account. The chat link is made authenticatable if the chat server's authentication key is entered on the ReaChat administrative settings page. This allows the chat server to trust chat sessions from your Drupal site and reject others.

My plans are to have this be my first contributed Drupal module and to implement it with the free 5-user test chat server on Composers' Village. I will update this story with progress on these fronts.

UPDATE 1: (July 19, 2009) See the RealChat module at drupal.org.

UPDATE 2: (October 9,2009) Chatting enabled with the RealChat module can now be used at Composers' Village. The chat server is running on a VPS using Ubuntu 8 and LAMP since Cronen-Townsend Consulting's shared hosting could not handle this use. To have your name and picture automatically set you must get a Composers' Village account.

Once MIDI files were reliably playing in the browser (see the previous post), the next thing I wanted was to have them play in little popup windows, instead of making a blank new page and playing there. Having to hit the browser's "Back button" after playing any MIDI was not acceptable.

The first way I did this was to put the Javascript code to make the popup window inside the link (with an onclick attribute). The problem with that method is that the link is completely broken when Javascript is not supported. Clearly a new page to play the file is the best option when Javascript is not supported.

So, the question is, how can one get a popup when Javascript is supported, but have the link still go to a new page and load the file when it is not?

The answer is to use Javascript (and the JQuery library) to make an appropriately tagged link popup if Javascript is fully supported, otherwise do nothing.

The first step is tagging the link. In my case, I'm using FileField for the MIDI file links so I overrode the theme_fiefield_file in the template.php of my composers_village_theme:

The next step is to implement the Javascript. This was a little tricky, for reasons noted at http://api.drupal.org/api/file/developer/topics/javascript_startup_guide.... Another starting point is http://drupal.org/node/121997. But what actually oriented me was the introductory JQuery material in Matt Butcher's book Learning Drupal 6 Module Development. The easiest way was to put the following code in a file called script.js in my composers_village_theme:

1. This line checks that Javascript is fully supported for Drupal.
2. This line waits for the document to be fully loaded.
3. This jQuery line adds a click handler popup to links of class "popup-link"
4. This line actually creates the popup window.

For more information, consult references, but this recipe may get you started. It's what worked for me.

Setting up my Drupal development environment based on Eclipse with PDT and the Zend debugger took me a huge amount of time. Searching on drupal.org yields a lot of configuration details for Eclipse, and Web-wide searches yield detailed setup instructions for various combinations of software versions and operating systems. But what finally helped me over my last, and most time-consuming, challenges was the workflow discussion at http://groups.drupal.org/node/2663.

So, in this article, I emphasize two main points that cost me huge amounts of time getting my development environment set up. I currently use Eclipse Ganymede 3.4.2 with PDT 2.0.0.

1. Stable PHP projects must be created with "New->Project->PHP project" in Eclipse/PDT

   This point was buried in the discussion referenced above, but I hope that emphasizing it here will save somebody a lot of time. My mistake was that I kept checking out Drupal, contributed modules, and my own custom modules from CVS repositories. These projects looked like PHP projects for awhile, but over time they would decay on me, and eventually forget the files they contained. To avoid this problem, I start each new PHP project with "New->Project->PHP Project" and then import the desired code from another location. When I do the import I do not overwrite the ".project" file that is already there. When downloading contrib modules, I establish them first in the main Eclipse workspace as discussed, and then I use Eclipse's "Refactor->Move" command to put put them in the correct subdirectory under my Drupal install.

2. Breakpoints must be set in code as opened by PHP Explorer under the drupal directory you are debugging.

   This was an important point for me once I had the previous point figured out and was making sure that I could debug my custom modules and step into Drupal code as well. Each project appears at the main level of the Eclipse workspace, as well as again under the main Drupal install if it is a module (and was put there with "Refactor->Move"). For example, my custom module for Composers' Village appears once at composers_village and again at drupal/sites/all/modules/composers_village. Breakpoints set in the main workspace one will never be hit, although they will look like they are set correctly. To have your breakpoints be hit, they must be set in the version of the file that appears under the Drupal being debugged. If your debug configuration points to drupal/index.php then your breakpoint must be set at drupal/sites/all/modules/composers_village, for example. This is my experience, anyway, and I hope it proves helpful. If this article helps you or if you have questions please contact me.