Drupalist

Blog posts

  • Building a Developer Community around Acquia Dev Cloud

    For the last three months I have been providing dedicated support for Acquia Dev Cloud, a platform for putting Drupal-based web sites on-line with superior performance. The service is largely aimed at Drupal developers and is based on best-practices for working efficiently and safely with the sprawling complex systems that are modern dynamic web sites. Subscriptions to our service include standard professional development tools and many extra tools for evaluating, improving, and managing web sites, such as New Relic, Insight, and Mollom and some level of Acquia support, depending on the subscription purchased.

    Since all Acquia subscriptions include support through posts to our forums, one of my tasks has been making sure posts to the new Acquia Dev Cloud forum are responded to reasonably quickly. I'm most proud of the community that is growing around Dev Cloud and both appreciates my work and often helps support each other.

    Being a part of a company supporting the web presence of the largest independent newspaper in Egypt, Al Masry–Al Youm at this most-crucial time in Egypt's political history makes me particularly proud and there are many such projects that make me proud of being at Acquia. Here is a recent group picture with my picture displayed on a laptop in the center of the picture in the front row.

  • Steve joins Acquia

    I am thrilled to announce that on August 22, 2011 I joined Acquia's Client Advisory Team in a new role dedicated to Acquia Dev Cloud support. Being part of Acquia's amazingly strong Drupal team is pure joy. Additionally, I am working closely with my friend and mentor Barry Jaspan who designed and built Acquia's hosting infrastructure, to improve the Drupal developer experience, which is a cause I love to champion.

    My first two weeks in the new job have been quite eventful including Hurricane Irene sweeping up the East Coast of the United States and Acquia moving from its main office from Woburn, MA to Burlington, MA. And three (more) of my Drupal heroes have joined Acquia recently: Angela Byron, Moshe Weitzman, and Greg Knaddison. This is a team I am very proud to be joining!

  • The New Cronen-Townsend Consulting


    • The name of my site has changed back to Cronen-Townsend Consulting to emphasize the fact that I am taking new clients again.
    • The logo has been redone by artist Ryan Mackinnon and now shows three eyes against a dark background and stays sharp at larger sizes.
    • My clients page has been redone to show Imbee as a new client and list a current project for the Ecotarium
  • CTC again taking new clients

    As of today my former employer, Imbee, has become the first new client of a revitalized Cronen-Townsend Consulting(CTC).

    Working closely with a small development team on imbee.com has honed my Drupal skills greatly. Imbee.com is an extremely complex Drupal 6 site that uses layers of permissions to implement parental controls over kids' social networking activity. My experience now covers virtually every game-changing Drupal module including Panels, Spaces, Contexts, Features, and Organic Groups as well as state-of-the-art methods for managing version control; pushing code between development, staging, and production servers; rapid debugging techniques for Drupal-based sites; and more. I have also expanded my versatility to cases where the overhead of booting Drupal for each request would be a hindrance, by implementing a compact, efficient music search server using an object-oriented PHP-based system based on a MVC design pattern. In addition, I experienced Acquia Managed Cloud hosting from a customer perspective and have just started on Acquia's new Dev Coud.

    I am excited to be able bring my new level of skills to my on-going clients and to new clients, as well, as my consultancy continues to expand. Please contact me if you want to explore the possibility of using my services.

  • Don't leave behind .htaccess

    When copying the files of a Drupal install, do not forget to copy Drupal's .htaccess file. If you've had the vexing problem where a site you've just manipulated has clean URLs turned on but they suddenly don't work, this may well be the reason. It took me fixing this problem many times before I realized what was going on.

    Scenario

    You make a directory for a new site, cd to the new directory, and use drush to download Drupal.

    % mkdir mynewsite
    % cd mynewsite
    % drush dl drupal

    Now, you do an ls command and notice that drush put the files in a subdirectory of the current directory.

    % ls -F

    drupal-6.20/

    "No, problem," you think, you will just move those files up a level. So you

    cd to the directory and copy the files up a level.

    % cd drupal-6.20
    % mv * ..

    The Problem

    You think you copied all the files up a level, but the Unix wildcard "*" does not match .htaccess so you left the .htaccess file behind in the subdirectory. Your out-of-the box Drupal site is not going to work right without it (once you connect it properly to a blank database).

    The Proof

    If you still do not believe that you left behind the .htaccess do an ls -a to check. You think there is nothing left in this directory but...

    % ls -a

    .		..		.htaccess

    The easiest thing is to recognize that it was left behind and do another command

    % mv .htaccess ..

    Then clean up with

    % cd ..
    % rm -r drupal-6.20

    Conclusion

    Strange but true, the Unix wildcard "*" does not match hidden files, whose names begin with ".". But the file >.htaccess is important to most Drupal installs, do not leave it behind when manipulating a site to switch hosting, or whatever. I'm pretty sure that there are other ways, with various file-handling GUIs to leave behind the hidden ".htaccess" fille. So, whatever tools you use to manipulate, move, or copy a site's files, just make sure that .htaccess gets the same treatment as all the other files.

  • RealChat 7.x-1.0-beta2 Release

    I just made a release of the RealChat Module on Drupal.org that is fully compatible with Drupal 7.0. Due to a small change in Drupal 7 core before release, RealChat 7.x-1.0-alpha2 developed an error in detecting user picture use on the Drupal site that caused it to crash when you hit the "Chat Now!" link. The new release no longer has this problem and is now recommended for Drupal 7. If you have one of the 80 sites that officially use this module, I'd love to hear from you about your use of this module. Thanks!

  • The Importance of Preserving Context in Web Development

    This morning, when I got down to the particular work task at hand for Imbee, I switched to Eclipse and found it stopped at a breakpoint I had set when debugging a certain crucial issue in my local copy of imbee.com. This was exactly my context from last night. Seeing where the debugger was suspended and being able to study the values of variables set at that point in my testing instantly brought me right back into the task of fixing the problem at hand and making decisions about whether to "step over" or "step into" for my next move to find the cause of the particular problem and other things that would lead me to an understanding and fix of the particular issue.

    This made me remember back to my first job as a software engineer, where working from home (on occasion) meant I had to log into a server at our offices over a very slow VPN connection, and take a tremendous amount of effort and time to get back to where I was the night before. So it might have taken an hour to get back to the same point I was at the night before in debugging, if I had even recorded the information I needed to return to the same spot correctly(more overhead!).

    Since each day I worked from home saved 2 hours of commuting by car, this overhead is not as ridiculous as it might seem at first. But my point is simply this: it is crucially important to consider the overhead involved in getting back to the same context you were in when you stopped debugging the night before. Whatever your commuting situation, software stack used, etc... is not important, but the actual result of how much overhead you have *is* important.

    The low overhead I have in working from home for Imbee is a large part of what makes me productive, and feel successful, and love the job. And that matters.

  • Getting phpinfo() output from any Drupal site

    Seeing the output of phpinfo() is as easy as going to URL /admin/reports/status/php (e.g. http://www.example.com/admin/reports/status/php) on any Drupal site, when logged in to administer the site. This is quicker than writing a one-line PHP program yourself to call this function as suggested in a previous post on finding, changing, and verifying changes to your PHP initialization file.

    [I Learned this from Rob of Left Click at last night's Western Mass. Drupal meetup. (Here's Rob's presentation on the Features Module at last month's Drupal Camp.)]

  • Unsticking Yourself as a Drupal Developer: Googling & Getting Feedback

    Intro

    This post is being written for a presentation at the Western Massachusetts Drupal Camp on Jan. 22, 2011. It aims to share tips on getting yourself out of specific problems everyone encounters working as a developer with Drupal sites. Your presenter, Steve Cronen-Townsend a.k.a. crotown, was a Drupal freelancer for almost two years, and a few weeks ago joined Imbee as a full-time software engineer developing their Drupal-based web site.

    Case study: Turning off Drupal's login block

    One of the first things a person learns about using core Drupal, is going to the blocks page at /admin/build/block (navigation Administer->Site building->Blocks) and enabling, disabling, and moving blocks around on the pages of a freshly-installed Drupal web site. Disabling the login block and not knowing how to log in to administration account (the first account created) is a right-of-passage I suspect all Drupallers go through.

    Image you are locked out of your newly-installed Drupal site because you turned of the login block and you don't know how to log in. Suppose you try this search on Google. I've used the query "User login block Drupal" [note: capitalization does not matter for Google searches]. Note that the second hit in the list, How do I get the User Login block back, is exactly what we wanted to know. If we go to that page, we discover that one can always log on to an (unmodified!) Drupal site by visiting "example.com/?q=user". I will address how I arrived at that particular search in the next section. But first, and most importantly, let me continue with an important principle.


    Principle #1

    Realize that many other people have been in the exact same situation, and have published their solution on the web.


    Part 1: Searching How-To

    Goal: Search for the "exact" issue, concisely

    • Specific terms / keywords
      1. Precise terms. (e.g. check /admin/build/block for precise name "User login block")
      2. Add "Drupal" to web-wide searches
      3. Learn keywords from early search results, improve search query
      4. Ask friends for the right keyword (e.g. FOUC for "Flash of Unstyled Content")
    • Jump close to the answer
      1. Project pages on drupal.org (modules and more), http://drupal.org/project/[name-of-project], e.g. http://drupal.org/project/admin_menu
      2. Drupal handbook pages at http://drupal.org/documentation
      3. API Documentation at http://api.drupal.org
      4. PHP Documentation at http://php.net.

      Jumping is often inferior to Google search. But at least recognize the above places as authoritative.

    • Anything you can learn about searching is worthwhile (e.g. Query syntax. Useful sites.)

    Part 2: Crashes and errors

    The topic here is debugging and I want to start it with the most important thing I can say about it:


    Principle #2

    Demand feedback on crashes. Starting point of any fix is always exact knowledge of what is happening.


    Part of the reasoning behind adhering to this principle is that once you know exactly is going on you can conduct an effective Web search to find the answers others have published on the same situation (Part 1 of this talk). The other main idea behind this principle is that web sites are such complex systems (lots of code files, several languages, different layers of software, possibility of communicating with other web sites, etc...) that trying to fix a problem by changing something that you think may help the problem often leads to greater confusions and problems, rather than fixing the original issue. Let your experience guide you, but my experience says that nearly always you should get exact feedback on the error occurring before proceeding to fix it. As we all know, it is often challenging when you start with full feedback.

    The feedback you get often tells you that PHP has an issue with a certain line of code, identified by source file path and line number. Follow my final principle of unsticking yourself:


    Principle #3

    Read the source code.


    You have access to all the source code. Take full advantage of this and read it when you have the slightest question about how it works.

    Feedback from Drupal

    By feedback from Drupal, I mean feedback you can get from any Drupal sight by logging in as a user with administration permissions and visiting certain web pages.

    The first of these is the Status page at /admin/reports/status and navigated to with Administer >> Reports >> Status report. This page provides a wealth of information that is particularly handy when first setting up a Drupal site. The idea here is that each of the colored lines should be green, unless you have a good reason for it not to be. For example, having modules that need a security update will make the appropriate line turn red, and you should always apply security fixes to make that same line turn (at least yellow). Yellow means, in this case, that some of your modules are out-of-date. You may well know that updating a certain module makes it incompatible with another module you use. If that is the case, you should probably be satisfied with the yellow line.

    The other main way of getting feedback from the Drupal site is the "watchdog" error log accessed at /admin/reports/dblog and navigated with Administer >> Reports >> Recent log entries. Obviously this site is confused about a file that is expected to have the path /sites/crotown.com/files/submit-button.png but does not (ever since this site was converted from being part a multisite to being a stand-alone site.)

    Feedback from PHP

    By this I mean four things: information about the PHP install, the PHP error log and error pages, debugging print statements, and source-level debugging:

    1. information about the PHP installation your web server is using

      For this case, you may use PHP's built-in phpinfo() command as described in this blog post. Sometimes you can simply ask your environment to display the output of phpinfo() for you by clicking a button. In any case, the HTML page you get will look something like this. This feedback may direct you, for example, to install a certain PHP module. Use the first principle and conduct a web search, and add the name of your operating system to the search query.

    2. The PHP error log and error web pages

      The simplest and most helpful thing is often to turn on PHP's error web pages. This will usually replace a White Screen of Death (WSoD) with an error page from PHP telling you exactly what went wrong and caused PHP to crash. As usual, everything you need to know as a Drupaller about WSoD is on drupal.org:

      To enable error reporting, temporarily edit your index.php file (normally located in your root directory) directly after the first opening PHP tag (do not edit the actual file info!) to add the following:

      <?php
       
      error_reporting(E_ALL);
      ini_set('display_errors', TRUE);
      ini_set('display_startup_errors', TRUE);
       
      // $Id: index.php,v 1.94 2007/12/26...

      You will now be able to see any errors that are occurring directly on the screen. Memory problems may still not be displayed, but it's the first step in a process of elimination.

      This just a small sampling of the massive amount of debugging wisdom at http://drupal.org/node/158043. Reading it thoroughly will save you huge amounts of time as a Drupal developer.

      The other important thing to know about PHP errors is that they can be logged but that often you will have to turn on that logging and, obviously, you will have to know where that log is stored.

      Use the techniques above see the output of phpinfo() for the PHP that your web server uses. Find the line called "Configuration File(php.ini) Location" and edit that file. Look for the following lines containing error and make sure the two lines given are set correctly for your system:

      ; Default Value: E_ALL & ~E_NOTICE                                              
      ; Development Value: E_ALL | E_STRICT                                           
      ; Production Value: E_ALL & ~E_DEPRECATED                                       
      ; <a href="http://php.net/error-reporting" title="http://php.net/error-reporting">http://php.net/error-reporting</a>                                                
      error_reporting = E_ALL | E_STRICT

      ; Besides displaying errors, PHP can also log errors to locations such as a     
      ; server-specific log, STDERR, or a location specified by the error_log         
      ; directive found below. While errors should not be displayed on productions    
      ; servers they should still be monitored and logging is a great way to do that.
      ; Default Value: Off                                                            
      ; Development Value: On                                                         
      ; Production Value: On                                                          
      ; <a href="http://php.net/log-errors" title="http://php.net/log-errors">http://php.net/log-errors</a>                                                     
      log_errors = On

      ; Log errors to specified file. PHPs default behavior is to leave this value   
      ; empty.                                                                        
      ; <a href="http://php.net/error-log" title="http://php.net/error-log">http://php.net/error-log</a>                                                      
      ; Example:                                                                      
      error_log = "/var/log/apache2/php_errors.log"

      This should get you started. Remember that the web server must have permission to write the log where you put it. Search (Principle #1) when you have questions.

    3. Debugging print statements

      Putting debugging print statements in your code is the simplest way to get that exact feedback you need about what is going when PHP code executes. In increasing order of sophistication you have, basically, four choices:

      • print_r from PHP.
      • drupal_set_message() from Drupal core.
      • dpm() from the Devel module.
      • Your own custom wrapper function for dpm() that checks if the Devel module is enabled, to avoid PHP errors if you turn it off.

    4. Source-level debugging

      Get this capability. Remember that you must become all-powerful with PHP as a Drupal developer. You sometimes need to seed the code run line-by-line, change the values of variables in midstream, etc... To get this capability, start by asking around and search the web to find others who have succeeded setting it up (or buying it) with a system equivalent to yours.

    Feedback from Apache

    Here, we are talking about just getting logging to occur. As in the case of PHP, you may have to turn the logging on in order to get it. The ErrorLog directive is key. For Apache Version 2, start at http://httpd.apache.org/docs/2.0/logs.html.

    As an example, on my server running Ubuntu 10.04 I did the following command to see where this directive was set:

    cd /etc/apache2
    egrep -r  ErrorLog *

    In the example of my Ubuntu-powered server the output of the above command has the answer all over it:

    ErrorLog /var/log/apache2/error.log

    Feedback from MySQL

    This case is not so crucial, since one can generally get the necessary feedback from the Devel module by visiting /admin/settings/devel and turning on Devel's "Collect query info" and "Display query log" and then visiting the page in question. The error log on my Ubuntu server has the default location of /var/log/mysql.err. And on my Mac using MAMP it's at /Applications/MAMP/logs/mysql_error_log.err. If you ever need to set up MySQL logging yourself, the information is at here and many other places on the Web.

    Summary

    The main insight of this presentation is that there are three principles underlying unsticking yourself as a Drupal developer. To recap, they are:

    • Principle #1: Realize that many other people have been in the exact same situation, and have published their solution on the web.
    • Principle #2: Demand feedback on crashes. Starting point of any fix is always exact knowledge of what is happening.
    • Principle #3: Read the source code.

    Remembering these three principles when facing issues with Drupal will give you a solid foundation for solving those issues.

    Acknowledgement

    A special thanks to Imbee for supporting my completion of this presentation.

  • Steve joins imbee.com

    I am excited to announce that I have joined imbee.com as a full-time employee, as of today. Imbee.com is a secure social network for kids, and I will be helping grow imbee's technology platform, which is based on Drupal.

    So, as of the this first work day of 2011, Cronen-Townsend Consulting is no longer accepting new clients. On-going projects will be completed using weekend-time in 2011.

  • Western Massachusetts Drupal Camp

    The Western Massachusetts Drupal Group, is sponsoring a Drupal Camp to be held Jan. 22, 2011 at UMass Amherst's modern two-year-old Integrated Science Building.

    I am excited to be part if the planning for this event and happy to announce that one of the greatest Drupal developers, Moshe Weizman (creator of Drush), will be attending and presenting.

    The event is appropriate for all sorts of expertise levels, from those comparing different Content Management Systems and perhaps still grappling with the question "What is Drupal?", to those administering Drupal-Based sites, to seasoned Drupal/PHP developers. The event features various tracks and a "Genius Bar" where folks can get expert advice for getting through their Drupal conundrums.

    Get current information on the camp site, and hope to see you next month.

  • This site is in a cloud

    This site, along with all others in the crotown.com domain, is now on a Rackspace Cloud Server.

  • A Smooth Transition for Boston Review

    [This article describes, in overview, two recent migrations Cronen-Townsend Consulting carried out for Boston Review(BR).]

    In order to host their upcoming OpenPublish-based site, BR started renting a Rackspace Cloud Server two months ago. This server will become the server for the new BR site, allows efficient changes to site structure, and can be scaled-up easily to support both their growing average needs and any anticipated spikes in traffic.

    But their contract with their previous hosting provider expires on Nov. 31, 2010 and involves a two-year commitment. So, when the new web site was not yet ready to take over in early November, the decision was made to let the old contract lapse and port the old site to the new server. This solution avoided entering another two-year hosting contract to cover the time until the new site is completed, but created a new technical challenge: moving the current site to the new host, so that their site would continue to function (on the new host) when the contract for the old host expires a couple days from now.

    Two migrations were actually necessary, one of all email accounts, and another of the web site itself. The email migration was accomplished smoothly earlier this month and involved moving all bostonreview.net email to Google Apps, which provides up to 50 GMail accounts and other services to small organizations for free. Both migrations, as of today, have been accomplished successfully, leaving no possibility of any disruptions as their previous hosting contract lapses. This post focuses on the web site migration.


    Before: on Bluehost

    After: on Rackspace Cloud

    Well before the planned migration I made a script of all actions I was going to follow during the web migration. A key part of this was a separate Test Plan which is a script telling me exactly how to test all the functions of the web site so I could know when something is broken. Carrying out the test plan and fixing the things that were broken, was the last step before taking the Rackspace site live.

    On Wednesday I began following the script. (Actually, I had already completed a couple of steps ahead of time, so I could save time over the holiday weekend.) The first step was to get an alert posted informing visitors that the migration was happening, was going to turn off the ability to comment and make submissions over the holiday, and invite the users to hit a "notify us" link in case of any problems. That link emailed directly to me. After that, I actually turned off the submissions system and hid the comment form, so that the databases could no longer be updated by users. Then I directed their old host to make a full backup of their server. That file, created by their old host as I was traveling, formed the basis of the identical site on BR's Rackspace cloud server.

    Then Wednesday night, when I reached my holiday destination, I started using Unix's "secure copy" utility to transfer the backup from the old host to Rackspace.

    On Thursday I celebrated Thanksgiving.

    On Friday, I carried out the bulk of the steps in my script, which included reconstituting the files from their old server, reconnecting databases for comments and submissions, and relocating the directory that holds the submitted files to somewhere accessible to the new server without more reconfiguring. After this the server's configuration was changed from settings typical of a development web server to settings typical of a production web server. As the final phase in the cloning, I turned on the submission system back on and unhid the comment form on the new site. After this, I carried out the test plan and fixed everything that was broken. This involved recreating all their old server's URL redirections, which was not fully anticipated but was uncovered by the tests.

    On Friday night I logged into the domain name registrar BR uses and pointed it to the Rackspace DNS servers. One of the steps I had carried out before last Wednesday, was setting up these DNS servers to direct web traffic to the new site and pass mail on to Google's servers. Then, over the next 48 hours, the DNS system refreshed itself and the name "bostonreview.net" resolved to the new Rackspace server by last night. I checked periodically from my location in NYC, and I "bostonreview.net" began hit the new server when I checked Saturday morning. (The reason for this behavior is that all DNS servers cache their information on where to point a certain domain's traffic, so they do not have to check every time they are asked, which would be too slow and create too much traffic for the top-level machines in the system.) Recall that I turned submissions on and unhid the comment form on the new site, so visitors could use those services as soon as the DNS servers they used updated themselves.

    By the time I checked this morning their had been 5 new comments made on the site (and they were all about BR content) and 34 new submissions to the site. (And no one ever clicked on the "notify us" link!) So the transition had gone off without a hitch. This afternoon I re-established the SFTP access the BR staff was familiar with so they publish things to the new site in the same way they were used to. An uneventful transition is a good good thing.

  • Changing PHP's configuration file, php.ini

    [This post is about making changes in PHP's configuration files when you have full command-line access to the underlying Ubuntu 10 server. The commands should be identical for other Unix-based web servers.]

    An important step in setting up hosting of any web site is tailoring PHP to the needs of the code you will be running. You do this by changing PHP's initialization files which have the extension .ini. There is at least one initialization file, called php.ini, and there may be others.

    Step 1: Use phpinfo() to find the .ini files being used

    With any text editor, create a file called phpinfo.php in your web root directory, which I will assume is /var/www. Here is the contents of /var/www/phpinfo.php:

    <?php
    phpinfo();
    ?>

    [Note: Remove this file when finished with it, or before the server goes live. You want to use it to give you information (and not hackers) about how your server initializes PHP.]

    Go to http://example.com/phpinfo.php with any web browser. Going to this page should cause the script phpinfo.php to run, giving you something that looks like this. The four entries "Configuration file (php.ini) path", "Loaded Configuration File", "Scan this dir for additional .ini files" and "Additional .ini files parsed" give you the information you need.

    Step 2: Change the appropriate .ini file

    Now you need to find the .ini file that has the parameter you want to change. This will usually be php.ini, but keep in mind that you the parameter you want to change might be in one of the other .ini files, or that you may want to make a new .ini file to hold the new setting.

    Suppose we want to change memory_limit. Here I assume the standard case of a scan directory link in the directory of the loaded configuration file and give a command which returns the file name and line number of any match.

    cd /etc/php5/apache2
    egrep -rHn memory_limit *.ini

    Edit the appropriate .ini file and make the change, remembering that you need to become root to do this.

    sudo nano /etc/php5/apache2/php.ini

    Step 3: Give the web server permission to read the .ini files

    The server may have been unable to read the .ini file before you edited it (!). And even if it could before, your editing this file as root may have made the web server unable to read it. The easiest make the file readable by the web server is to change its group to the web server's group. This is setting is in /etc/apache2/envvars and is www-data by default.

    sudo chgrp www-data php.ini

    Back to Step 1 to make sure the change took effect

    You need to go back to Step 1 and look for the new value you just entered being used. The new value must appear in the HTML output of your phpinfo.php script when run by your web server, or it is not being used when your web server starts PHP. Please comment and share your experience.

  • Cloning Drupal sites - Part II: with root command-line access

    This article is about making an exact duplicate of any live Drupal 6.x site onto one's local machine. It assumes you have a web server installed on your local machine and have root command-line access to the web server of the target site. It is the second and final part in a series of two articles. Part I addresses the case where you have (or use) only CPanel access.

    A simple site to clone

    In the first installment, I demonstrated cloning on a site called http://simplesite.crotown.com. For this installment, we will clone the same site living on a Rackspace Cloud Server and called http://simplesite.crotownconsult.com. [Advanced note: to do this I simply cloned the site for the previous installment onto the Rackspace Cloud Server where I have root command-line access, and edited the site title and the first Story so they list the correct domain name.]

    Cloning how-to: root command-line access

    Making and downloading a dump of the target site's database is still the first and most important task.

    1. Get a dump of the site's database onto the local computer with the following steps, starting from a command line on the local machine:
      ssh -l root crotownconsult.com
      cd /var/www/simplesite
      cat sites/default/settings.php | grep db_url | grep -v \*

      where you will change crotownconsult.com to the domain name or IP address of the target server, and you will change /var/www/simplesite to the root directory of the Drupal install for the site you wish to clone.

      Read off the [database_name] from the line of output:

       $db_url = 'mysqli://.../[database_name]'
      and do
      cd
      mysqldump -u [user_name] -p [database_name] | gzip > [dump_name].sql.gz

      where [user_name] is your database user name for accessing the site's database. A simple choice is to have the dump come out with the name of the database plus file extensions, by setting [dump_name] = [database_name].

    2. Download and uncompress the dump with the following command lines on the local machine:
       cd
      scp root@crotownconsult.com:[dump_name].sql.gz .
      gzip -d [dump_name].sql.gz
    3. Install the database on your local machine with
       mysql -p
      create database [database_name];
      use [database_name];
      source [dump_name].sql
      exit

      Note that your root password on your local machine is 'root' by default.
    4. Archive and compress the files with the following commands on the target machine:
      cd /var/www
      tar -cvf [archive_name].tar simplesite
      gzip [archive_name].tar
      mv [archive_name].tar.gz ~

      where you replace simplesite with your document root and may choose [archive_name] = [database_name] for simplicity.
    5. Download on unpack the files with the following commands on your local machine:
       cd /Applications/MAMP/htdocs
      scp root@crotownconsult.com:[archive_name].tar.gz .
      gzip -d [archive_name].tar.gz
      tar -xvf [archive_name].tar
    6. Reset Drupal settings with the following commands on the local machine:
      cd simplesite/sites/default
      cp default.settings.php settings.php

    Now point a web browser at http://localhost:8888/simplesite and attach the drupal install to the database. Remember that the local database username and password are 'root' and 'root'. You will be presented with "View your existing site" link. When you go there you should see this:

    Note that setting up a sub-domain, as in this previous blog post will restore the logo, leaving you with an exact clone of the live site, simplesite.crotownconsult.com.

    Please comment to this post to help me improve it. Cloning Drupal sites is a vast topic, and I am sure that this site does not meet everyone's needs, but I am also sure that it can improve greatly with enough feedback. If you are feeling shy about commenting publicly please send me an email using my contact form.

  • Twitter: What's the use?

    Intro

    This post is being written for a presentation at the Western Massachusetts Drupal Users Group on Sept. 21, 2010. It is intended to share how I have been using Twitter as a Drupal developer in a way that might inspire others to use Twitter or use it in new ways, and to promote discussions about Twitter.

    Anatomy of a Tweet

    • A 140-character message that generally goes to all of your followers and the general timeline.
    • May have
      1. mentions of another user e.g. "Having lunch with @rgem"
      2. hashtags (keywords) e.g. "Working on D6 version of #drupal Versus module."
    • May be a ...
      1. retweet - e.g. "RT @crotown Having lunch with @rgem"
      2. direct message to a follower -- e.g. "d @rgem Lunch tomorrow?"

    Followers vs. Friends

    I follow @eaton, but he does not (necessarily) follow me.

    How to tweet

    Case Studies

  • Cloning Drupal sites - Part I: with CPanel access

    This article is about making an exact duplicate of any live Drupal 6.x site onto one's local machine. It assumes you have a web server installed on your local machine, and have CPanel or access for a Drupal site on the web. Part II will address the case where you have root command-line access to the web server.

    First I discuss how I use site cloning in my work and the many uses of this capability. Then I move on to instructions on cloning a typical Drupal 6.x site that uses MySQL in the case where you have (or use) only CPanel access to the server.

    Why clone a Drupal site?

    Recently, while I was assisting a client with a large data migration, the client's server suddenly developed multiple software issues that kept it from serving web pages the night before the deadline. We had both been working on that same server. What to do? To be as efficient as possible, I downloaded a dump of the site's MySQL database to my laptop and made the 17,000 Drupal users from legacy data locally, while the client dealt with the errant server and rebooted it. When the web server was back up I transferred a dump from my laptop to the live web server . The site we continued working on had one big difference to the site before the glitch -- it had 17,000 Drupal users. This was possible because I had earlier copied the site's PHP files to my laptop, in order to be able to trace the PHP execution line-by-line as a page was built, and get at the root causes of a software issue.

    That anecdote sums up the two main reasons to clone a Drupal site:

    • To get the site on a development machine, in order to debug PHP line-by-line
    • To have another option of a machine to do site configuration on.

    There are clearly, millions of variations of these reasons. As a Drupal developer, let me say that it has always been worthwhile to a client, to allow me to make a temporary clone of the client's web site on my laptop.

    A simple site to clone

    I put a simple web on-line in order to demonstrate cloning. The site is bare Drupal 6.19 install with the following configuration:

    • site information set
    • one Story created
    • core Upload module enabled
    • two file attachments (centipede image and README) added to the one Story
    • anonymous user granted permission to view file attachements.
    • added and enabled contrib module Administration Menu (from http://drupal.org/project/admin_menu)
    • added and enabled contributed theme sky (from http://drupal.org/project/sky)
    • created a sub-theme of sky called ctc_sky, with:
      1. The CTC logo at sites/all/themes/ctc_sky/logo.gif
      2. An template override templates/page-front.tpl.php that hard codes logo IMG tag with src="sites/all/themes/ctc_sky/logo.gif"
      3. A style.css override at sites/all/themes/ctc_sky/style.css that centers the logo properly by including a 1.5 em top-margin on elements of id="logo".

    The site is installed in subdirectory "simple" of my document root on a shared server. I created the subdomain simplesite.crotown.com to point at that directory, so site can be accessed at http://simplesite.crotown.com. Here is a screenshot when I browse to this site with Firefox.

    Cloning how-to: CPanel access

    The first (and most important) task in cloning a site is downloading a database dump and installing it on your local machine. The tool I will use on both ends to accomplish this is phpMyAdmin, so I will assume you can run that through your CPanel on your web host and on your local machine (I use MAMP).

    1. Get a dump of the sites database on the local computer with the following steps:
      • Get the sites' [database_name] by looking at sites/default/settings.php in the web site's document root directory. And looking at the end of the path given in the line that begins with "$db_url =".
        $db_url = 'mysqli://.../[database_name]'
      • Open phpMyAdmin on your web host using CPanel.
      • Open [database_name] in phpMyAdmin.
      • Hit the "export" tab.
      • Scroll down and hit "Go" in the "Save as file"-section.
      • Locate the file called "[database_name].sql" on your local computer, in the directory where you browser puts downloaded files (e.g. your desktop or "Downloads" folder).
    2. Import the database onto the local machine using the following steps: (note MAMP-specific screenshots)
      • Open phpMyAdmin on your local machine.
      • Create a local database of the name [database_name].
      • Hit "Import" tab.
      • Hit "Browse" in the "File to import" section.
      • Select the dump in Step 1 and hit "Import".
        You should receive confirmation that the database was imported, similar to this:

        You should see the left sidebar list all the tables in your newly-imported Drupal database, like this. Congratulations, you now have the database of your web site cloned on your local machine.

    3. Transfer the files of the remote site to the local machine with the following steps:
      • Open CPanel File Manager and navigate to the root directory of the Drupal installation for the remote web site.
      • Check the box next to the name of the root directory and hit "Compress".
      • Select a "GZiped Tar Archive" and hit "Compress File(s)"
      • Dismiss the "Compression results" dialog.
      • Note the location of the new file called "simple.tar.gz"
      • Open your ftp client (I use FileZilla) and connect with sftp to the remote site. Navigate to simple.tar.gz on the remote site. Your window should look something like this.
      • Transfer the simple.tar.gz to your local web root directory (e.g. /Applications/MAMP/htdocs or /var/www) by dragging it across from the remote side to the local side or by selecting transfer from a menu.
      • Decompress and unarchive the file on your local machine (e.g. by opening it with StuffIt Expander.app).
      • Observe the local directory simple. It should look like this. Congratulations, you now have the files of the remote Drupal site on your local machine.
    4. Connect the database to your local copy of the site's files by the following steps:
      • Copy simple/sites/default/default_settings.php over simple/sites/default/settings.php. [Note that you may have to change permissions to do this, as you might in a regular Drupal install.]
      • Visit the URL that points to the local site, e.g. http://localhost:8888/simple
      • Choose "English" on Drupal's Choose a language page.
      • Set the database. Note that the default MySQL access is username "root" and password root. Hit "Save and continue".
      • Hit "Visit existing site" link.
      • Note that clone site is all correct, except for the logo (whose path was hard-coded into the theme ctc_sky).
    5. Make a virtual domain simplesite.local, as I described in a past blog post, so that the site can be accessed at http://simplesite.local:8888 and note that this cures the hard-coded path problem. The clone is perfectly identical now.

    Where to go from here

    A logical next step is to investigate cloning sites from the command-line entirely. This streamlines the process and will be covered in Part II of this post. There are also ways to clone using scripts, Drush, and Aegir. See my post about handling large databases with phpMyAdmin on shared hosts. And there is an excellent post (and comments) on dealing with large MySQL databases from the command-line.

    Cloning is essential to any development/staging/live-type environment, so there is lots of experience using it in the real-world. You may investigate posts on various problems cloning, because there are lots of things that can go wrong. But handling whatever comes up while using this technique is a mark of a experienced Drupal developer. Rest assured, we Drupalistas can clone any Drupal site. It is an amazing fact.

    ADDENDUM [last updated Sept. 10, 2010]
    Here are some notes about the cloned site and dealing with things going worng on it. I will expand and improve this section occassionally

    • The first time you log into your cloned site - remember that your credentials will be identical to those of the on-line; the cloned site has an identical users table in its database.
    • If the cloned site will not boot Drupal, it is important not to settle for a White Screen of Death and merely guess what is causing the problem. Add the three lines:
      error_reporting(E_ALL);
      ini_set('display_errors', TRUE);
      ini_set('display_startup_errors', TRUE);

      at the top of index.php so that PHP will try to give to give you error page when it crashes.
    • If a crontrib module is crashing you can disable it easily by renaming its directory from sites/all/modules/[MODULE] to anything else, like sites/all/modules/OLD_[MODULE].
  • Getting control of your email in Apple Mail

    After spending significant time today upgrading the handling of my email, I would like to share some of the techniques I use to manage my email over many separate email addresses. My focus here is on getting emails automatically sorted so that certain categories can be ignored and others seen immediately as circumstances require. Ignoring things that are ignorable is difficult when all of your email is jumbled together. And missing emails that demand attention is not acceptable.

    As the post goes on it will become more and more specific to the mail reader I use, Apple Mail 4.0. I do not address spam filtering in this post.

    TIP 1: Use separate email addresses for different kinds of emails

    For example, I use the address consult@crotown.com for emails that are sent by the contact form on this web site and for subscriptions to various web services that I use for Cronen-Townsend Consulting. And the address steve@templeisraelgreenfield.org is for emails relating to my volunteer work for the local synagogue. In my mail reader, emails are pre-sorted, and I never need to open the steve@templeisraelgreenfield.org mailbox during consulting work, for example. The number of unread emails there may increment, but since I know that a new email there can wait by virtue of the email address it was sent to, it will wait appropriately for my attention until after the paying work ends for the day.

    TIP 2: Use Smart Mailboxes for cross-mailbox categories

    Smart Mailboxes are a term Apple uses to denote mailboxes which are full of pointers to email messages in regular mailboxes, and are filled according to a definable set of rules.

    Smart mailboxes are useful since they allow one to create something that appears with the list of mailboxes, but goes from no icon next to it to a one, as soon as you receive an email fitting its criteria.

    I have one smart mailbox for each client and one for my wife. I don't want to miss an email about dinner arrangements for tonight, because it was mixed in with all my other emails and I didn't notice when a mailbox count went from 101 to 102. When I find an email that was not included in a smart mailbox that I think it should be in, I add to or edit the existing rules defining that smart mailbox until it is included. Your own individual experimentation will be required, but you get the idea.

    TIP 3: Use Rules to move emails from each mailing list to its own mailbox

    Rules are a term Apple uses to denote filtering criteria that trigger certain actions on email, when they are received (or later).

    Even after applying the first two tips in this post, you may find yourself missing mail in a certain mailbox because it is overwhelmed by mail from a certain mailing list.

    For example, I subscribe to the groups.drupal.org OpenPublish group, which has bursts of traffic. So, in order not miss other emails during burst, I have a Rule that automatically moves any email whose subject contains "groups.drupal.org OpenPublish" to mailbox called "OpenPublish" that lives on my Mac. Then I can read it an appropriate time, when I have time to read it, and more pressing email that comes to the same mailbox will still be seen during a burst of traffic from that group.

    As our use of email grows, the need for techniques to manage it so it remains an effective tool for us increases. Hopefully these three tips contribute in a small way to that ongoing discussion.

  • Two tricks for debugging a Drupal WSoD

    When working on custom Drupal modules in PHP, one commonly experiences the dreaded White Screen of Death (WSoD). The Drupal community has documented the Drupal WSoD extensively at http://drupal.org/node/158043.

    Typically, everything seems fine, and then the Drupal-based web site you are working on (on your local machine) suddenly becomes completely broken, it gives a blank page for any request. Usually I cannot remember even what inconsequential-seeming things I have just changed. The tricks of this post are for directly tracking the down the error with a debugger, by watching it occur. [There is an even better method for this example, described at the end of this post.]

    Recently I was working on the versus module and had to track down such an error. I came to a section of Drupal 6.17 code in drupal_load() in bootstrap.inc that looks like this

      if ($filename) {
        include_once "./$filename";
        $files[$type][$name] = TRUE;
     
        return TRUE;
      }

    After tracking the crash down to roughly here with the debugger, I wanted to set a breakpoint that was only hit when the versus module was the next to be included with include_once and verify that the crash occurred when the module was included.

    A conditional breakpoint would do the trick, but conditional breakpoints do not work for me in Eclipse Ganymede with PDT. [And they don't work in other PHP debugging environments for other developers I have asked].

    So the first trick is to add code so that you can set a breakpoint that is only hit in the case you care about. Basically, if conditional breakpoints don't work in your environment, then add the condition to the code.

      if ($filename) {
        if ($name == 'versus') {
          $i = 0;
        }
        include_once "./$filename";
        $files[$type][$name] = TRUE;
     
        return TRUE;
      }

    [Since this debugging change to core is destined to be overwritten, and does not effect code function, it is not "hacking core".]

    So, after placing a breakpoint on the bogus

    $i = 0;

    line, running the code, and then stepping past the breakpoint, I learned that PHP was indeed crashing on the line

    include_once "./$filename";

    when $filename was sites/all/modules/versus/versus.module.

    Then what?

    Here's the second trick: Use command line PHP to read the file in question and tell you the error, with a command like:

    php < sites/all/modules/versus/versus.module

    when you are in the Drupal site's root directory.

    You get something like

    Fatal error: Cannot redeclare versus_voting_form_submit() (previously declared in /Applications/MAMP/htdocs/drupal6x/sites/all/modules/versus/-:195) in /Applications/MAMP/htdocs/drupal6x/sites/all/modules/versus/- on line 221

    which tells you exactly what the error is and lets you fix it and move on quickly.

    As I was researching this post, I found http://drupal.org/node/158043 again and studied it carefully. It turns out if you turn on error reporting at the start of index.php as described there you see this page, in the example above. This technique essentially replaces the WSoD with a page containing the fatal PHP error.

    So I recommend always pre-pending the index.php for a development site with the three lines of code that turn PHP error reporting on. I'm sure that skills at direct tracking of bugs will still be necessary for difficult debugging tasks, but they are not necessary in the above example.

  • CTC's changing mission statement

    As you may have noticed, I recently honed the 'mission statement' for Cronen-Townsend Consulting from "Building custom Web architecture with Drupal" to "Building custom Web platforms based on Drupal". Here is the story behind the change.

    Recently, I listened to the Acquia webinar Amazon Web Services Building Blocks for Drupal Applications and Hosting. The first part of the presentation was by Jeff Barr, Senior Web Services Evangelist at Amazon, who explained Amazon Web Services. In his presentation he used the terms IaaS and PaaS, for "Infrastructure as a Service" and "Platform as a Service." respectively. The talk was followed by an explanation by my friend Barry Jaspan of Acquia's Drupal hosting environment, which is built on Amazon Web Services.

    The term inrfrastructure was used for rather low level services, and it occurred to me that Acquia's hosting environment, may be described as architecture building on the IaaS Amazon provides. On the other hand, higher level structure was called "platforms" and "applications".

    So I decided to upgrade Cronen-Townsend Consulting's mission statement to use the term "platforms" instead of "architecture". When I build a single web site based on Drupal, what I am really delivering is a customized platform which is a good foundation for future growth. This hits the real strength of Drupal-based solutions. And the word "platforms" also hints at the way that I can deliver, using Drupal's installation profiles, a package that can be used to install a single website each time it is executed. And the nearly limitless possibilities provided by things like multiple web sites running on the same Drupal PHP code and sharing a common database, and on and on...

  • Loading large MySQL databases on a shared Web host

    Writing a blog post about loading large databases on a shared host has been in the back of my mind for almost a year. It all started when I wanted to put a replica of MTTProNetwork.com on the web, for testing purposes. Then my client could do testing herself and only instruct me to go live with the changes when she was certain nothing had broken. (Note: As of this writing these changes are not yet live, for external reasons).

    As a step in putting the replica on-line I had to put the site's Drupal database on-line. So I made a new blank database by going to my CPanel and clicking the MySQL Databases icon and went to PHPMyAdimin, selected the database, and then chose "Import". [Note: In order to make things more secure when I set up the site, I created a new database user and gave that user privileges only on the new database, but that's not part of this post.]

    Importing the large database with PHPMyAdmin failed on my host. I ended up successfully importing that database dump with bigdump.php. So I was thinking that this post would be about those methods. [Basic instruction on PHPMyAdmin for import and export are also at drupal.org].

    But when I was researching this post and tried importing that same large database onto my shared host it just worked. So what had changed? After a little poking around I determined that I had increased PHP's memory_limit to 128M as I described in a previous blog post. [In that post I actually used the limit 98M, but I had settled on 128M for myself.]

    To make a long story short, I placed a copy of my host's php.ini with the line:

    memory_limit = 128M

    In my account's web root in directory public_html. See that previous blog post for details. That change apparently improved the ability of PHPMyAdmin in my CPanel to import large databases.

    There is one other possibility for something that changed to cause my CPanel's PHPMyAdmin to become more capable. I have another php.ini in my home directory that is being used when command-line PHP starts on my web host. I increased the memory_limit in that one as well. I did this in following the plans in the first comment to that previous post to get the command-line tool drush working on my shared host.

    Even though I have learned all these tricks to set up capable Drupal sites on my shared host, the bottom line is that it is still annoyingly time-consuming to manipulate Drupal sites on shared hosting, and I need to be able to set up whatever extra servers (e.g. RealChat for chatting or Apache Solr for facetted search) at will. Because of this, I have long ago learned to set up virtual Linux servers from scratch for Drupal and other things; I do most oif my development on a MAMP server on my laptop, and put things live for current clients on a VPS host rented from VPS.net. The only reason crotown.com is still on shared hosting is lag (a.k.a. "inertia") due to time-constraints. My plan is to move crotown.com to a Rackspace Cloud Server as my next big project for my business. I will blog about the change when it happens.

    UPDATE Sept. 1, 2010:
    See this excellent related blog post for details of command-line MySQL dumps and restores for large databases.

  • Give Apture a try at crotown.com

    Here is a little post to help you try Apture searches. This is going to be easy since I have Apture 2.0 installed on this very web site.

    Suppose you are reading my previous post and you see the phrase "Having Apture integration built-in to a site's infrastructure lowers the barrier to trying it out" and you are wondering what exactly the term infrastructure means. So, highlight it (select it).

    A search button should pop-up as shown when you click here. Go ahead and hit the search button.

    You will be rewarded by the appearance of the Apture Magic Search Bar and an Apture window.

    Scroll and select the link you want (probably the dictionary.reference.com link, in this example). And there you have it — anything becomes a link, according to your own needs.

    You can make the toolbar appear by simply scrolling down any page until the web site's header is off the screen. Then you can type anything you want Apture to search for. Try something from popular culture and browse videos. Have fun!

  • Apture Part I: Introduction

    Building an OpenPublish (2.0)-based site has introduced me to Apture, a technology that is built into each OpenPublish install. (My recent post "My first Impression of OpenPublish: 10 Joys" gives a tiny introduction to Apture.)

    Having Apture integration built-in to a site's infrastructure lowers the barrier to trying it out, for sure. All a site developer has to do is surf to apture.com, get an API key, and enter the key into the appropriate configuration page on the OpenPublish site. Then they are ready to experiment.

    But what is Apture? Apture is a web service that allows you to enhance your web site's content in many ways. And it does it in such a way as to encourage people to stay on (and come back to) your site.

    (Frankly, it is so fun and amazing it will make visitors want to experience your site. I got "distracted" several times in writing this post by Apture suggesting relevant things to link to. So they weren't really distractions, they were deeper engagement with this material. It has been an amazing experience, and I want to visit more sites with this capability, and I want to help spread this sort of capability by building sites that use it).

    Apture helps content creators and editors search for and add links and embeds to textual content easily, with dynamic pop-up windows that communicate with apture.com. It also has the ability to automatically enhance existing links on your site.

    Apture helps visitors to your site find relevant content elsewhere on the web relating to any words they put in the Apture Magic Search Bar. Your visitors can even highlight any words in your content and do an Apture search on them without any typing. The Apture search bar only appears when a user scrolls too far down a page to see any header information of yours, reminds them that they are on your site, and provides a place where they can type an Apture search. But your visitors can hide it if they do not like it.

    And Apture helps site owners by reporting back useful statistics on what users are sharing and searching and linking. This helps them stay on top of emerging trends in the interests of their site visitors. Gone are the days when a site's owner had little feedback on what users really care about on a site.

    My plan for this series of posts is that Part 2 of this series of posts will be about Apture links and embeds, and part 3 will be on the Apture Magic Bar. And part 4 will be on reporting. I hope I can meaningfully treat this important topic in such a short series! And I'm going to write a prequel, Part 0: Apture and Drupal Technical Issues, next.

    So what is the easiest way have an Apture experience and really get what it is about? It seems to me that going to a site using the Apture Magic Search Bar and trying it out from an anonymous site vistor's point-of-view. Well, as you may have noticed, this site now uses the Apture Magic Search Bar so highlight any terms in this article you are curious about, select the search button that should pop up, and you'll be on your way to understanding Apture.

    There is one more issue I want to tackle in this introduction: How does the Apture company make money providing this service? The Apture FAQ helped me understand this:

    Publishers pay Apture monthly subscription fees for premium features, integration of custom content sources, customizations, and priority support, while the basic blogger version is free. We also have revenue-sharing relationships to split revenue with large publishers that derive extra advertising revenue (from video pre-rolls, for example) or page views through Apture’s technology. Currently, Apture has relationships with some of the web’ largest global publishing brands including the BBC, Washington Post, SFGate, and the New York Times. If you are interested in our paid publisher services, please contact us.

    So, basically, web sites that use of the service heavily pay a subscription fee for having the service on their sites, but people who add the service to their blog, like me, (and individual end users) do not. That piece is important to get out in the open, so there are no suspicions about what is going on with payments. Have fun trying out Apture!

    UPDATE July 6, 2010:
    The Apture FAQ also has an answer to the question "How much does Apture cost?":

    If you are a blog or website with less than five million page views per month, then Apture is free! If you have a website with more than five million page views per month, you should contact us, but you’re welcome to try it out on your own test site while you’re waiting to hear back from us.

    That's pretty clear.

  • My First Impressions of OpenPublish: 10 Joys

    We recently switched the new site development for Boston Review(BR) from a bare Drupal underpinning to an OpenPublish underpinning. In this post I share my early impressions from a the perspective of a developer and site-builder.

    [OpenPublish is a Drupal distribution from Phase2 Technologies based (currently) on Drupal 6.x with many module choices, configuration, and default theming intended to be useful to publishers.]

    1. Installation is the same as with Drupal

    Obviously, as a Drupal developer, it's important to me that OpenPublish is based on Drupal. But I was pleasantly surprised when I didn’t have to learn anything new to install OpenPublish.

    The fact that I got over sixty compatible installed modules with lots of basic configuration and default theming right out-of-the-box, really piqued my interest. I knew that if it was consistent with what I wanted to build, it was going to save me immense amounts of time.

    2. Streamlined administration

    Take a look at the Drupal reports page in OpenPublish:

    The well-designed icons are the first thing I noticed, but the way the whole administrative experience is streamlined also hit me after I got used to the administrative menus that appear to the left of each page (thanks to the Admin Module). One of the many benefits of this is that clients get the impression “Hey, an OpenPublish-based site is actually usable by me”. This is very important, since many potential clients have anxieties about taking care of a new site – and an OpenPublish test site looks (and is) manageable.

    3. Power “under the hood”

    One of the first things I noticed about OpenPublish is that the platform makes the same choice of fundamental modules that I already had chosen for the data-, image-, and multimedia-handling needs of the Boston Review. I’m speaking here of things like Views, CCK, ImageField, ImageCache, and Embedded Media Field. I knew that these were very solid choices in terms of upgrade-path for Drupal 7 and beyond. But I was also taken by some of the other advanced possibilities in OpenPublish, like Apache Solr search integration, that also matched my choices. So I suspected, right away, that the various semantic technologies also built into OpenPublish (see later entries on OpenCalais and Apture) might be appropriate for BR as well.

    4. Attractive default theming

    Take a look at this article view from my Boston Review development site:

    This article, despite the fact that it can’t display custom Boston Review fields like “discussed works”, gives a good idea how BRs content will eventually look on their new site. So it is important that the default layout is good, so clients can get an idea of a new site before a designer and themer get to work on it. People wanting a new site just respond better to a site that is easy to look at.

    5. Attention to detail: Author creation and display

    A issue that publishers are faced with in setting up a Drupal site is that the actual author of a given piece of content does not generally have an account on the Drupal site. So Drupal’s out-ot-the-box authorship mechanisms do not suffice. But in OpenPublish each actual author is an object of type Author, and these objects are created and displayed flexibly and easily.

    When a person creates a new article on an OpenPublish site, the part of the form where one sets the author looks like this:

    One great thing about this is that if the article is by a new article, one can simply hit the “plus” icon, create the new author in a pop-up window, and then automatically reference the newly created Author instance. This sort of attention to detail carries through to viewing the article in the default OpenPublish theme.

    So, for example, if I create myself as the second author of this piece, the end of the article will look like this:

    Note that the panel says “About the authors...” automatically and lists us both. What a pleasure.

    6. Organization with the Features module

    OpenPublish organizes related views, content types fields, imagecache settings, and the like into “features” which can be enabled or disabled as a group, and are version-controlled on http://drupal.org. This emerging module provides a needed and sensible level of structure to the complex platform.

    7. Bidirectional links with the Relationships Module

    Articles have Authors in OpenPublish, but Authors at the same time refer back to the articles they have written. One manifestation of this is that viewing an author automatically shows the articles they have written:

    And using this capability in my own custom node-reference fields has been a snap.
    NB: I have never actually co-authored anything with Craig Morgan Teicher

    8. Search with Apache Solr

    Once I set up a Solr Search server one of my first tasks was to turn on a block for automatically sorting search results different ways. Once I learned that block visibility is controlled by the Context Module in OpenPublish, and is not set on the usual blocks page, it was a snap to produce the following search results page.

    9. Automated linking (and more) with Apture

    With OpenPublish, one can automatically choose a link with the web service Apture. For example, when entering a story about the world bank, one can highlight the words "world bank" and hits the Apture link icon.

    This gets you a pop-up that asks you what kind of link you want:

    And when you click the Create Link button the appropriate HTML code is inserted in your article.

    It's as easy as that. This barely scratches the surface of the services available through Apture which include embedding content and automatic links to your content.

    10. Automated tagging (and more) with OpenCalais

    OpenCalais is a service powered by Thompson Reuters which automatically tags content. The text of content of types you choose is sent to OpenCalais in the background, semantic analysis is performed there, and appropriate tags arereturned to your site and added to your article. For example, default tagging of articles produced all of the following tags except "On poetry" and "January February 2010".

    This addresses a fundamental problem in making the web more semantic, and a daily struggle of editors everywhere - "Whose job is it to tag the content and maintain the taxonomy?"

    As you can see, OpenPublish is allowing me to leverage vast efforts on the part Phase2 Technologies and the Drupal community effectively for clients in publishing. And in very practical terms, it is helping me deliver a new site to Boston Review with capabilities that neither of us dreamed of at the start of our project. Capabilities that will bring tremendous value to the Boston Review and tremendous connection to the Boston Review readership.

  • Beware of extra Commas: PHP Function Argument Lists are not like Arrays

    Take another look at the Drupal community's coding standard on Arrays.

    A couple months ago I was working on an importer for a Drupal site where the originating data could be scripted in JavaScript. So I was writing PHP code (to upload the Drupal nodes) using JavaScript code (to access the original data). It occurred to me just how convenient it is to be able to write every array element the same way, with a comma after it and not have to write conditional code that makes the last entry comma-less.

    Well, yesterday, I had the realization that adding a comma after the last argument in a function call causes PHP (5.2.11) to crash. This is particularly annoying if you can not debug the code because it is being executed by drupal_execute in a separate process. So, write PHP function calls like this:

    bropimporter_create_image(
    "january_february_2010 :: essays_35-1 :: img2",
    $IMPORT_ROOT,
    "images/january_february_2010/essays_35-1",
    "img2.jpg",
    "january/february 2010"
    );

    NEVER like this

    $image_nids[] = bropimporter_create_image(
    "january_february_2010 :: essays_35-1 :: img2",
    $IMPORT_ROOT,
    "images/january_february_2010/essays_35-1",
    "img2.jpg",
    "january/february 2010",
    );

    A comma after the last argument is a bad thing.

  • Architecting the future of BR on the web

    At Cronen-Townsend Consulting, I am building the back-end of a new web site for the Boston Review using the Open Source content management framework Drupal. The site will incorporate current standard features, like a block showing the most recent comments, sharing with all popular social-network sites, and a block showing today's most popular content.

    Because the new site is being built on top of the powerful content management framework Drupal it will be able to easily incorporate future web innovations as the Boston Review community evolves. There are currently Drupal modules that implement all web technologies (at around 4000 contributed modules, and growing rapidly). And with the Drupal community doubling in size each year all web-technologies should be implemented for the foreseeable future. (See the recent keynote at DrupalCon 2010 by project-founder Dries Buytaert).

    In addition, I am designing a process that can automatically post the content of an entire issue of BR on a private preview site for the editors of BR, save an archival XML version of the content and associated files, and push any content to the live site when and if the BR staff chooses. The archival form is being designed to be easily-translated to future format (such as new eBook formats), so this will put the BR content on a solid foundation for the future.

    The new site is going to live on a Rackspace cloud server, which will allow it to accommodate spikes in demand, feature sophisticated Apache Solr searching, and position it to incorporate new hosting technologies for Drupal soon after they are developed. (Rackspace was a platinum sponsor of DrupalCon 2010 and I learned a lot about cloud hosting at the conference. They also partnered recently with Dries Buytaert's company Acquia).

    All together these activities comprise what I call architecting the future of the Boston Review on the web, one of the most exciting projects I have been involved in.

    UPDATE May 10, 2010
    The new site will be on OpenPublish! I already have some OpenPublish test sites up and have one configured to search with a separate Apache Solr server and sort the results flexibly. Now I am converting my code which parses an entire issue of Boston Review and imports it to work with OpenPublish. I will update this article with progress.

    UPDATE June 2, 2010
    Boston Review staffers are playing with a draft of the new OpenPublish-based site that auto-imports content from the most recent three issues. Even with just the default OpenPublish theme the new site is a beauty!

  • How to tell Ads from Photos and other Images

    When I was a child my great Aunt Cornelia gave me a copy of Robert Williams Wood's How to tell the Birds from the Flowers: And Other Woodcuts. The joke behind every poem and illustration is the same - you can't. Check out Amazon's cover shot to see that a Pansy looks very much like a Chim-pansy, for example. In writing the package of JavaScripts that parses the InDesign files for an entire issue of the Boston Review into the files needed to upload all of the issue's content into the new Drupal-based web site I am building for them, I was faced with a very similar question: How do you tell an ad from a photo?.

    Although I still can't tell a bird from a flower, I'm glad to report that I have found a trick that allows me (and my code) to tell an ad from a photo every time -- at least in the Boston Review's current format.

    Here's the basis of the trick: ads are not attributed and photos and other images are attributed, every time. So if one's code can programmatically detect the attribution, it can tell the difference between an ad and a photo or other image, every time. If it has an attribution, it is a photo or other image, and if it does not have an attribution it is an ad.

    In the current format of the Boston Review an attribution is given as a sideways piece of text next to the photo that reads from bottom to top. In InDesign, it's implemented as a very tall and thin TextFrame, the height holding the length of the attribution and the width holding the text's height. So my idea is to go through all the PageItems on the same layout page as the image in question and look for a tall thin TextFrame next to the image.

    So here is the code for my checkIfAd() routine, that identifies Boston Review ads, every time. That way my code can skip the ads on import. A very useful trick.

    // checkIfAd() <br />
    // Given all the pageitems on a page and the item index of a graphic, checks if there <br />
    // are any tall and skinny TextFrames on its right side.  If so, it is judged not <br />
    // to be an ad (since it has an attribution) and if no attribution TextFrame is found <br />
    // it is judged to be an ad <br />  
    function checkIfAd(pageItems, itemIndex) { <br />
    	t = pageItems[itemIndex].geometricBounds[0]; <br />
    	l = pageItems[itemIndex].geometricBounds[1]; <br />
    	b = pageItems[itemIndex].geometricBounds[2]; <br />
    	r = pageItems[itemIndex].geometricBounds[3]; <br />
    	isAttribution = false; <br />
    	var i; <br />
    	for (i=0; i<pageItems.length; i++) { <br />
    		if (i != itemIndex ) { <br />
    			if (checkIfAttribution( pageItems[i])) { <br />
    				pi_t = pageItems[i].geometricBounds[0]; <br />
    				pi_l = pageItems[i].geometricBounds[1]; <br />
    				pi_b = pageItems[i].geometricBounds[2]; <br />
    				pi_r = pageItems[i].geometricBounds[3]; <br />
    				if (pi_t>t && pi_b<=1.1*b && (pi_r-r)/(r-l)<0.1) { <br />
    					isAttribution = true; <br />
    				} <br />
    			} <br />
    		} <br />
    	} <br />
    	return !isAttribution; <br />
    } <br />

  • Modular programming for InDesign JavaScripts using ExtendScript

    The focus of the Boston Review is its print edition. So one task in creating their new web site has been creating an automated system for parsing the InDesign files that lay out an issue of the print edition, and creating the files needed to post the entire issue, if they desire, to their new web site.

    The best answer to this task is clearly not one huge script, but a collection of smaller scripts that call one another, pass variables, and share one common log file (for my own sanity when debugging). In this article, I list some of the issues I have overcome in setting up such a collection of JavaScript files, rather than one enormous JavaScript, to script InDesign CS2.

    Setting up a library

    The first thing I wanted to do was set up a library, which is to say a collection of JavaScript functions in a common file, say, that other JavaScripts use. To do this, I used the ExtendScript directive #include to grab the library, which just has a lot of common functions, like so:

    #include  "importWeblibrary.jsxinc"

    The file importWebLibrary.jsxinc is in the InDesign scripts directory located as Presets/Scripts/ under the Adobe InDesign CS2 directory.

    The full header of my files

    There are a couple other important pieces at the top of all my scripts in this collection, so the complete header looks like this:

    #strict on
    #target indesign
    #engine importerToDrupal
    #include  "importWeblibrary.jsxinc"

    The '#strict on' turns on strict error checking, which always seems like a good idea top me, having lots of experience in C/C++. The '#target indesign' says the JavaScript refers to the InDesign application and its Document Object Model, so a dialog will pop up when you run a script and there is no InDesign instance running. And the directive '#engine importerToDrupal' makes all the scripts into one group.

    Things to watch out for

    There are a number of important issues that came up for me around this modular programming environment.

    The first is that the representation of the engine (your collection of scripts) happens within InDesign, so the first time you invoke one of your scripts must be done through InDesign's "Window -> Automation -> Scripts" pane for your grouping of scripts to be recognized as such. This should cause "importerToDrupal" to become a choice in ExtendScripts dropdown list of available scripting engines. Unfortunately, in my case, I must switch the target to something else and then back to InDesign CS2 in order for the dropdown list of scripting engines to be updated. After my "scripting engine" importerToDrupal is selected in the ExtenScript Toolkit (ESTK) things will continue to function smoothly even if I instantiate one of the scripts directly from ESTK, rather than from InDesign, in order to debug it.

    This leads directly to the next important consideration, which is that any included files (like importWebLibrary.jsxinc) do not appear in the debugger. So I generally get new functions working tacked on to the source file I am debugging before I move them into the library file itself.

    Calling another script, and passing variables

    You may be wondering, if you've hung in to this point, how to call one script from another and pass variables. Here is a code fragment from my code demonstrating the way I do this:

    if(sectionTypeGroup.selectedButton == 1) {
         log(logFile, "OPENING file #" + index + ' AS FORUM : ' + contentsItem.fullName + "...");
         app.open(contentsItem.fullName,true);
         var thescript = new File("/Applications/Adobe%20InDesign%20CS2/Presets/Scripts/processForumFile.jsx");
         var setpagerange = "var DocumentPageRange='"+contentsItem.documentPageRange+"';";
         app.doScript(setpagerange); 
         app.doScript(thescript);
         app.activeDocument.close();
    }

    Note that I am executing a one-line script to pass the variable DocumentPageRange. It does the job, but if you have a cleaner way, please let me know.

    By now you may have realized that 'log()' is the function that was the subject of my previous post. And since it automatically tacks the passed string onto the end of the log file, all I need to do is pass it the same logFile variable to get the one log file behavior I wanted, regardless of where in the collection of scripts I begin execution.

    var filePath = "~/Desktop/tempLogFile.txt";
    var logFile = new File(filePath);

    Since processForumFile.jsx acts on the active InDesign script I can execute it directly through ESTK in order to debug it and it still functions correctly. All I have to do, in my case, is make sure that processForumFile.jsx does something reasonable if the variable 'DocumentPageRange'.

    Taking all of these things together makes a reasonable system for architecting a collection of modular JavaScripts to perform a complex task, and be clear to future programmers who may have to maintain your solution.

  • Robust File-writing from InDesign JavaScripts

    Recently I've been scripting InDesign with JavaScript to parse an entire issue of the print version of the Boston Review and produce the files needed for the new Drupal site I am building for them.

    After getting oriented in the Adobe ExtendScript scripting environment, I found that outputting all my debugging print statements to the JavaScript console with $.write() was really slowing things down. Since I was going to have to write files eventually anyway, I was thrilled to find Dave Saunder's code for logging from InDesign JavaScripts. Using this logging function sped up my code greatly versus writing everything to the console.

    So, what is wrong with the code? Well, it makes the mistake of not checking for errors in file-handling calls. I know that I, for one, have made this "mistake" many times purposefully since it is just hard to write good error handling code without examples of failures to catch. And, after all, why worry about something failing when it has never failed? Maybe it will never fail on you and the coding-time will be wasted, or maybe the error-catching code itself will fail since you have no error examples to test it on.

    Well, the original logging code failed seriously on me in a way that I just had to track done to keep my project on-track. So I thought I would share my experiences and my additions to the logging routine that make it report the error message when it fails. And with a real failure to test on, I had no excuse to not improve the code.

    What I was noticing is that when I called the log() routine to see the contents of the array activeDocument.stories for a certain document the output for certain stories would always be mysteriously missing from the log file. So even though there was supposed to be one entry per story, occasionally output would just be missing for some stories.

    When I tracked it down, I learned that the error message (from the File object) I needed to see was "Character conversion error." Here is my version of the log() routing that writes the error message to the JavaScript console once the error occurs:

    function log(aFile, message, header, incDate) {
    	var today = new Date();
    	if (!aFile.exists) {
    		// make new log file
    		aFile.open("w");
    		if (incDate) {
    			aFile.write(String(today));
    		}
    		if (header != null) {
    			aFile.write(header);
    		}
    		aFile.close();
    	}
     
    	if (aFile.open("e")) {
    		if (aFile.seek(0,2)) {
    			if (aFile.write("\n" + message)) {
    				if (aFile.close()) {
    					return true;
    				}
    			}
    		}
    	}
    	$.writeln("ERROR='"+aFile.error+"', Log() failed to write '"+ message + "' to log file.");
    	aFile.close();
    	return false;
    }

    But why was this error happening? And why was it only happening only when trying to write the contents of certain stories? Well, it turns out that some of the stories are picture captions, or the name of the author, or other short things that have no exotic characters in them. But the longer entries, that are whole narratives written by someone all have some exotic characters in them, since they *all* end with a solid diamond character.

    So when the File object's write method was passed certain strings to write to the file, it was trying to convert things like the solid diamond to some standard text characters in MACROMAN encoding and it had no instruction on how to convert them, so it failed with a "Character conversion error."

    This is an interesting case, since I did not even imagine that character conversions were a possible source of failure in the File object's write method. I have to admit that if I had been writing the original logging routine, I probably would have reasoned wrongly that the only way a write can fail is that there is not enough disk space, and I'm never going to be close to that point, and not attempted error-catching code, either.

    So how did I fix it? Easy, just tell the file object to use the UTF-16 encoding when it writes the file, rather than using the default encoding on my system. So my code, when opening the log file (or writing it for the first time) would read:

    var filePath = "~/Desktop/tempLogFile.txt";
    var logFile = new File(filePath);
    logFile.encoding = "UTF-16";

    This all ties into my next blog topic, which will be modular scripting for Adobe InDesign CS2. The most compelling reason for me to write exploratory output to a log file, came when I wanted break my process into separate scripts and simply append onto the end of one ongoing log file, and begin execution lower down (via the ExtendScript Toolkit) or call the top level script from In Design and still be able to check my single log file.

  • CTC at DrupalConSF 2010

    The upcoming DrupalCon in San Francisco from April 19-21, 2010 will be my first. I'm thrilled to return to the West Coast as a Drupal developer, having fond memories of the Human Language Technologies 2002 conference in San Diego (as a computer scientist) and the Materials Research Society's Spring meetings in San Francisco in the mid-90's (as a physicist).

    Meeting many collaborators on Drupal "in real life" (IRL) for the first time will mark quite an important career milestone for me. To make the most of this opportunity I plan to participate in the code sprints the day before and the day after the main conference day. So see you in SF April 18-22!

    UPDATE May 14, 2010
    Participating in DrupalCon SF 2010 was amazing. I caught up with an old physicist friend of mine, jhodgdon, at the sprint on the April 18, 2010. She's a colleague, again, now that I am part of the Drupal movement. At Dries' State of Drupal keynote the next day I saw her handle projected on HUGE screens - she is the #7 top contributor to Drupal 7! The combination of Dries' talk and Tim O'Reilly's keynote the next day was very powerful for me, as I came to grips with my small place in the movement at the same time I was introduced to the notion that the movement as a whole is playing a important role in keeping part of the evolving "Internet Operating System" free. Heady stuff. Then I worked on a Drupal 7 core issue on the 22nd. Back in the trenches, so to speak, but with a new sense of the whole endeavor.

  • Optimizing Drupal sites on shared hosting: A Case Study

    The first web hosting I signed up for was ANHosting's "All Inclusive Mega Package" shared hosting. Going with an $8/per month plan made a lot of sense while I was learning to build web sites. I chose ANHosting after finding it recommended for Drupal sites on John Forsythe's blog. Over my first year building web sites I have built many sites on that hosting (and many others including some similar hosts, Pair Networks hosting, and a VPS I rent from vps.net).

    Recently I learned how to optimize my sites on the shared host by using a custom php.ini developed from the default php.ini my host uses for my account. This was not a trivial thing to learn how to do, since I originally thought (mistakenly) that I was restricted to CPanel access that did not allow me to access the server's php.ini.

    So the first part of this tip, is to investigate whether you are actually are allowed any ssh (command-line) access to your account. If you really do not have this access to your hosting, then the rest of this post probably does not apply to you.

    If you are still reading, log into your shared hosting with a command-line like (ignore the line number "1."):

    1. ssh -l username mydomain.com

    Enter your password when asked. At this point, you should be presented with some sort of UNIX prompt. Even if you have this much access, you probably do not have the permissions needed to use the very handy command-line tool drush. But, for the purposes of this article, we just need to find the php.ini used to set up PHP. And that does not take a high level of permissions.

    The task now, is to locate the php.ini that is used when PHP starts up on your host. I first found the php.ini by hunting around using the UNIX commands ls and cd. But then I discovered that I could run PHP from the command line. So try something like:

    1. php -i | less

    The "php" runs the PHP command, the "-i" is a flag to PHP to output information about its configuration, and the "| less" is a UNIX way of getting the output to be paginated (and going on to the next page with space, etc...) and not just letting the output run by, leaving you looking at the last lines of a long output.

    What you are looking for is a line in the output like this:

          Loaded Configuration File => /usr/local/lib/php.ini

    [Note: Experienced UNIX users will immediately see many precise ways to search for such a line (e.g. with the command grep, but the commands are much harder to type correctly for those not experienced in UNIX).]

    Now all one has to do is to transfer that file to the root of one's Drupal install (where my host, at least, will use it in place of the default one. Then you can modify whatever you need to to make Drupal run smoothly.

    To transfer it there, try returning to you local machine's command line with:

    and then use the command:

    sftp username@mydomain.com

    to start the secure version of the ftp command.

    At the sftp prompt you would use a series of commands like:

          get /usr/local/lib/php.ini
          cd public_html/drupal
          put php.ini

    The last task is to modify the file you have copied into the root of the Drupal install the way you need. I use CPanel's "File Manager" and its "Edit" button to do this. The lines I change look like the following:

    memory_limit = 98M
    upload_max_filesize = 10M
    post_max_size = 20M

    That gets my Drupal sites on my shared host humming along. The beauty of this process is that you can modify any of PHP's starting configuration and keep all the good things that your hosting already had set up for you in the default php.ini. I found out the hard way that the php.ini I first put at the document root of the Drupal install overrides the one your host provides, and I needed some of the default configuration. This is the real reason to find the host's default php.ini and start from there to make a custom one for your Drupal site. Please contact me for assistance with setting up Drupal on your shared host, or post a comment to this article.

  • Speeding up Eclipse PHP debugging by using an external web browser

    Today I got my environment for debugging PHP web pages working much more snappily, and I'm excited to share my tips. My experience only applies directly to you if you are running Eclipse and debugging PHP on a Macintosh with OS X.

    Last weekend I scored a free VGA LCD monitor from my brother-in-law who didn't take it to law school with him and is now loaning it to me for at least a couple years. I was thinking "Free screen real estate, what could be better? I should be able to put the debugging web browser on one and screen the rest of my debugging windows on the other." After I got the right adapter to connect it, that is.

    Once it was connected, the new monitor was somewhat convenient, but I quickly discovered that I couldn't put my Eclipse internal web browser in it's own window and move it to the other monitor. For some reason, Eclipse kept insisting on opening a new internal browser in the pane with the editing windows even if I had already opened the internal web browser in it's own window with "Window | Show View | Other... | Internal Web Browser" before I started the debugging session. And I was not allowed to "detach" the internal web browser tab from that pane and make it a separate window.

    So I tried setting Eclipse to use an external browser for debugging at "Eclipse | Preferences... | General | Web Browser", reasoning that I could surely leave the external web browser's window on a different monitor from the Eclipse window I was using. Well, after a lot of time trying things I was left with only Firefox seeming to work. But it seemed slow and if Eclipse was running, Firefox was *always* starting the debugger in Eclipse and being very slow. Firefox is my browser of choice because of the Firebug plug-in, and Eclipse is about always running on my system. I just couldn't stand my main tool always being slowed to a crawl.

    So I started trying every other Mac browser I could find until I discovered Camino. And it worked and even got invoked by the Eclipse debugger at the right time, but unfortunately it gave me errors like:

    So here the main tip. Edit the settings for launching Camino at "Eclipse | Preferences... | General | Web Browser". In the Parameters edit box add the command-line option "-url %URL%" and Camino will start working as your Eclipse debugging browser.

    The coolest thing is that my debugging between breakpoints started going several times faster. I was thinking that perhaps I needed a new computer to get faster debugging. But, it seems, that Eclipse's internal web server really bogs it down. So not only do I get more space for my other debugging panels with the new scheme I get a much faster PHP debugging system overall. Good deal! It would have been a good idea to do this even without getting a second monitor. If only I had known sooner that my setup for PHP debugging could be so quick and responsive. But at least I know now, -- and hopefully reading my story will help some fellow developers.

  • Flexibly titled pop-up windows

    Here's a tip to add to the previous articles here about pop-up windows. When you invoke a pop-up with Javascript, the window is automatically given a title which is the URL of the page it is viewing. But often one wants a simpler title like, say, "Play audio", instead of the URL of the audio file. What to do?

    An easy solution I found was to put the file within its own Javascript IFrame(). That way, the invisible title of the IFrame is the file URL, while the window can have whatever title one sets.

    Here's the sample Javascript code:

    if (Drupal.jsEnabled) {
    	$(document).ready(function(){
    		$("a.popup-link").click(function() {
    			newwindow2=window.open('','myPopup','height=100,width=400');
    			if (window.focus) {newwindow2.focus()}
    			var tmp = newwindow2.document;
    			tmp.write('<html><head><title>Play audio</title>');
    			tmp.write('<link rel="stylesheet" href="js.css">');
    			tmp.write('</head><body>');
    			tmp.write('<iframe src=',this.href,' width="100%" height="100%"> </iframe>')
    			tmp.write('</body></html>');
    			tmp.close();
    			return false;
    		});
    	});
    };

    If you are having trouble placing this code, see the earlier article for a more complete explanation.

    The tradeoff here is that the IFrame appears within a sunken border, but it was great for my purposes. Here's my final result:

  • Debugging locally using Eclipse, MAMP, and virtual domains

    When setting up debug configurations for Eclipse, I often have my files living in a subdirectory of my web server root. For example, at directory drydock/ I have a full replica of whatever live site I am working on.

    On my hosting, I have the same directory structure and a subdomain set up so that drydock.crotown.com maps to the drydock subdirectory. Then the site drydock.crotown.com lets me put changes on the web for testing and approval, before the actual live site is touched.

    On my local machine, until recently, my only domain was http://localhost:8888. So for debugging the page users/crotown, for example, I would have the "file" set to /drydock/index.php and the "URL" set to drydock/users/crotown. This worked fine if I was debugging the generation of that page, but any redirects would not work right since the base URL was http://localhost:8888 and NOT http://localhost:8888/drydock as it should have been. So the URL for redirects would be missing the "drydock/" part and cause a "Page not found" error. Also, paths to image files would be missing the "drydock/" part resulting in lots of missing images when the site was viewed.

    The solution? I created a virtual domain called drydock.local so that the URL http://drydock.local:8888 will automatically map to the drydock/ directory and the URLs would not have to have have this part added to work correctly. To do this, I followed the instructions here and here on the MAMP Forum.

    All I had to do was add the new virtual server to Eclipse at "Preferences-->PHP-->PHP Servers". Now when I visit "Run-->Debug Configurations..." I can select the new virtual domain in the "PHP Server" drop down menu. Then I can set the URL at the bottom of the Server pane, without the "drydock/" and everything, including all the redirects and images, works correctly in my debugging instance. Well, everything except whatever I'm debugging...

  • ShareLinkField is born

    Have you ever posted a link on Facebook? An image and the site description is automatically scraped from the site, and you can select an image from the site without a page reload. When you are done the result is posted in one unit; it is very slick.

    Currently, one project of mine is to recreate this ability for Drupal sites. To add such fields flexibly to nodes I decided to create a custom CCK field modeled on Jennifer Hodgdon's article and using Karen Stevenson's recent post as a reference.

    The result is so far is called ShareLinkField and is a compound field consisting of a link (made by the link module) an imagefield, and a site description field. There is no scraping, yet... If you are interested in the status of this project please contact me.

  • Drupal 6 RealChat module

    Chatting is something that makes a community site seem so much more vibrant and dynamic. I had looked into chatting options using contributed modules to Drupal 6 at one point but never implemented any of them.

    But recently I was contracted to write a simple RealChat module for Drupal 6 that allows users of a Drupal 6 site to chat easily using a RealChat server. On clicking a link, users automatically start a chat session with their Drupal username as their "nickname", Drupal picture for "avatar", and profile on the Drupal site linked to their chatting account. The chat link is made authenticatable if the chat server's authentication key is entered on the ReaChat administrative settings page. This allows the chat server to trust chat sessions from your Drupal site and reject others.

    My plans are to have this be my first contributed Drupal module and to implement it with the free 5-user test chat server on Composers' Village. I will update this story with progress on these fronts.

    UPDATE 1: (July 19, 2009) See the RealChat module at drupal.org.

    UPDATE 2: (October 9,2009) Chatting enabled with the RealChat module can now be used at Composers' Village. The chat server is running on a VPS using Ubuntu 8 and LAMP since Cronen-Townsend Consulting's shared hosting could not handle this use. To have your name and picture automatically set you must get a Composers' Village account.

  • Working on the Drupal 7 Field API

    I have recently started working on the Drupal 7 Field API. In Drupal 7 the Content Creation Kit module (CCK) is mainly a user interface that uses the new Field API to do the behind-the-scenes work of attaching fields to objects.

    We have lots of coding and testing to do before the code freeze coming up in September 2009 and it's great to be building the future of Drupal. The first issue I took on involved adding the call to hook_field_create_field() to the end of the core function field_create_field(). That invocation was simply left out of the implementation of field_create_field(). In addition, I added an automated test of the hook getting called (using the Drupal 7 core module SimpleTest). You can see this work at http://drupal.org/node/489438. My patch was immediately applied to the Drupal 7 code at testing.drupal.org and run through the battery of automated tests (including my new one). When human reviewers first look at my code, they will already know that it passes those tests.

    As a former software engineer, I am particularly heartened by the way the Drupal community continually adopts and evolves the best software engineering practices. Being involved in the Drupal project is a real joy.

    [UPDATE: I am happy to report that my change was made part of Drupal 7 core on July 2, 2009. Thanks to bjaspan, webchick, and (of course) Dries for helping me become a Drupal core developer!]

  • Robust popup links in Drupal 6

    Once MIDI files were reliably playing in the browser (see the previous post), the next thing I wanted was to have them play in little popup windows, instead of making a blank new page and playing there. Having to hit the browser's "Back button" after playing any MIDI was not acceptable.

    The first way I did this was to put the Javascript code to make the popup window inside the link (with an onclick attribute). The problem with that method is that the link is completely broken when Javascript is not supported. Clearly a new page to play the file is the best option when Javascript is not supported.

    So, the question is, how can one get a popup when Javascript is supported, but have the link still go to a new page and load the file when it is not?

    The answer is to use Javascript (and the JQuery library) to make an appropriately tagged link popup if Javascript is fully supported, otherwise do nothing.

    The first step is tagging the link. In my case, I'm using FileField for the MIDI file links so I overrode the theme_fiefield_file in the template.php of my composers_village_theme:

    /**
     * Theme function for the 'generic' single file formatter.
     */
    function composers_village_theme_filefield_file($file) {
    ...
      if ($file['filemime']=='audio/midi') {
        // mark MIDI link so that the javascript script.js can find it
        //   and make a popup-window
        $options['attributes']['class'] = 'popup-link';
      }
      return '<div class="filefield-file clear-block">'. 
                 $icon . l($link_text, $url, $options) .'</div>';
    }

    The next step is to implement the Javascript. This was a little tricky, for reasons noted at http://api.drupal.org/api/file/developer/topics/javascript_startup_guide.... Another starting point is http://drupal.org/node/121997. But what actually oriented me was the introductory JQuery material in Matt Butcher's book Learning Drupal 6 Module Development. The easiest way was to put the following code in a file called script.js in my composers_village_theme:

    if (Drupal.jsEnabled) {
    	$(document).ready(function(){
    		$("a.popup-link").click(function() {
    			window.open(this.href, "popup", "width=250,height=50,menubar=0,toolbar=0,resizable=1,scrollbars=0");
    			return false;
    		});
    	});
    };

    1. This line checks that Javascript is fully supported for Drupal.
    2. This line waits for the document to be fully loaded.
    3. This jQuery line adds a click handler popup to links of class "popup-link"
    4. This line actually creates the popup window.

    For more information, consult references, but this recipe may get you started. It's what worked for me.

  • MIDI links playing in the browser

    Getting MIDI links at Composers' Village to play in people's browsers has been an interesting challenge. After I first noticed that the links were prompting file downloads, investigation led me to the "Content-Disposition" header that FileField puts on files downloaded from a filefield. (FileField is the name of the Drupal module that I am using to implement the file links for the Village, and the particular container is a filefield.)

    Making FileField output "Content-Disposition: inline" as opposed to "Content-Disposition: attachment" makes the browser look for a plug-in that handles that type of content and play it, rather than downloading the corresponding file.

    After initially correcting the problem in the FileField module's code at first, I wanted to make the fix in a modular way, that did not require changing FileField's code. When I looked at the Drupal 6 code further, however, I discovered that any implementation of hook_file_download I did in my composers_village module could only add to the headers put there by FileField's implementation and could never take away what FileField's implementation had added.

    So, the remaining option was to contribute my fix of the problem to the FileField project at drupal.org. In researching similar issues in the drupal.org issue queue, I learned that the same problem had just been fixed for Flash content in a filefield. Since MIDI is a relatively rare file format and it required the "inline" designation, I fixed the problem in a way that created an administrative settings page for FileField where an administrator can set additional file types to be downloaded as "inline". This community discussion is at http://drupal.org/node/485336 and is ongoing. The next release of FileField should contain some version of my fix.

    [UPDATE 9/2/09: The change has been committed to FileField head for some time but has not yet appeared in a release.]

    [UPDATE 12/23/09] Just searched for filefield_inline_types in version 6.x 3.2 of FileField and can confirm that this patch is part of this version.]

  • Drupal, Eclipse, and PDT development workflow -- the big picture

    Setting up my Drupal development environment based on Eclipse with PDT and the Zend debugger took me a huge amount of time. Searching on drupal.org yields a lot of configuration details for Eclipse, and Web-wide searches yield detailed setup instructions for various combinations of software versions and operating systems. But what finally helped me over my last, and most time-consuming, challenges was the workflow discussion at http://groups.drupal.org/node/2663.

    So, in this article, I emphasize two main points that cost me huge amounts of time getting my development environment set up. I currently use Eclipse Ganymede 3.4.2 with PDT 2.0.0.

    1. Stable PHP projects must be created with "New->Project->PHP project" in Eclipse/PDT

      This point was buried in the discussion referenced above, but I hope that emphasizing it here will save somebody a lot of time. My mistake was that I kept checking out Drupal, contributed modules, and my own custom modules from CVS repositories. These projects looked like PHP projects for awhile, but over time they would decay on me, and eventually forget the files they contained. To avoid this problem, I start each new PHP project with "New->Project->PHP Project" and then import the desired code from another location. When I do the import I do not overwrite the ".project" file that is already there. When downloading contrib modules, I establish them first in the main Eclipse workspace as discussed, and then I use Eclipse's "Refactor->Move" command to put put them in the correct subdirectory under my Drupal install.

    2. Breakpoints must be set in code as opened by PHP Explorer under the drupal directory you are debugging.

      This was an important point for me once I had the previous point figured out and was making sure that I could debug my custom modules and step into Drupal code as well. Each project appears at the main level of the Eclipse workspace, as well as again under the main Drupal install if it is a module (and was put there with "Refactor->Move"). For example, my custom module for Composers' Village appears once at composers_village and again at drupal/sites/all/modules/composers_village. Breakpoints set in the main workspace one will never be hit, although they will look like they are set correctly. To have your breakpoints be hit, they must be set in the version of the file that appears under the Drupal being debugged. If your debug configuration points to drupal/index.php then your breakpoint must be set at drupal/sites/all/modules/composers_village, for example. This is my experience, anyway, and I hope it proves helpful. If this article helps you or if you have questions please contact me.

  • Hiding the clutter on "Create content" and other forms

    One of the first things that struck me about the Drupal 6 out-of-the-box experience is that there are an overwhelming number of choices when one clicks Create content and creates ones's first Story and Page nodes. The first user account is privileged and hence you see many options a typical user would not when you are using the first account.

    When I moved on to custom types and considering actual users of Composers' Village there were still fields on some forms that I wanted to hide when a typical user was logged in.

    Here is some Drupal 6 sample code that turns off access to the personal contact form settings, so users of Composers' Village cannot turn off their own contact forms. The sample code also turns off access to a hidden field (filed_o0) of my CCK type Tune that is set programmatically later in the flow and hence is not meant for humans at all. The strategy is to implement hook_form_alter and set the "#access" property to false for elements you wish to turn off.

    /**
     * An implementation of hook_form_alter
     */
    function composers_village_form_alter(&$form, $form_state, $form_id) {
      global $user;
     
      $is_admin = false;
      if (user_access('access administration pages')) {
        $is_admin = true;
      }
     
      if ($form_id == 'user_profile_form' && !$is_admin) {
        // make a typical user unable to hide their personal contact page
        if (isset($form['contact'])) {
          $form['contact']['#access'] = false;
        }
      }
     
      // turn off all access to the owner field,
      //   since it is set by nodeapi in 'presave'
      if (isset($form['field_o0'])) {
        $form['field_o0']['#access'] = false;
      }
    }

  • Programmatically Creating Instances of CCK Types - Drupal 6

    During the design phase of Composers' Village I devised a database schema for the musical data consisting of three CCK types: Villager, Tune, and Project. The reason for the Villager type was to contain a user's musical information, e.g., what Tunes they had contributed in a way that could persist after that user's account had been deleted. I basically wanted the database to stay consistent and well-linked even after a contributing user left and their account was deleted.

    The difficulty of this design was that in early implementations the first thing a user had to do after setting up their account information was go to Create content and make a VIllager to represent themselves musically. It would hold a text blurb which was their musical bio but was mostly an empty container for information about any future musical contributions to Composers' Village. This was a usability problem because it was a real nuisance to have to go through this extra step when joining Composers' Village.

    So I realized that the Villager for a user had to be created automatically for them when the account was made. In the Getting started page I could explain how to fill out the musical bio after creating account but having to teach the new user how to create an instance of Villager, as well, was definitely too much.

    How to create an instance of a CCK type programmatically? What I did was implement hook_user() in the composers_village module. In my composers_village.module I added the following code:

    function composers_village_user($op, &$edit, &$account, $category = NULL) {
      global $user;
     
      if ($op=='insert' && $account->uid) {   
        // a new user is being added, create a villager for them 
        $node = new stdClass();
        // leaving the $nid out will tell node_save() it is a new node
        $node->type = 'villager';
        $node->language = '';
        $node->uid = $account->uid;
        $node->status = '1';
        $node->created = time();
        $node->changed = $node->created; 
        $node->comment = '0';
        $node->promote = '1';
        $node->moderate = '0';
        $node->sticky = '0';
        $node->tnid = '0';
        $node->translate = '0';
        $node->revision_uid = $node->uid;
        $node->title = $account->name;
        $node->body = '';
        $node->teaser = '';
        $node->log = '';
        $node->revision_timestamp = $node->created;
        $node->format = '';
        $node->picture = '';
        $node->field_a4 = array();
        $node->field_c1 = array(array('value'=>null,'format'=>null));
        $node->field_a5 = array(array('nid'=>null));
        $node->field_a8 = array(array('nid'=>null));
        $node->field_a7 = array(array('nid'=>null));
        $node->field_c2 = array(array('url' => null,'title'=>null));
        $node->last_comment_timestamp = $node->created;
        $node->last_comment_name = null;
        $node->comment_count = '0';
        $node->taxonomy = array();
        node_save($node);
      }
    }

    My friend Barry Jaspan recommended this approach and tells me it does not work for all field types in Drupal 6 CCK. But, since I basically create an empty Villager, it works perfectly -- saving a step for new users of Composers' Village. Using CCK's ability to set permission by user role, I turned off the ability to create a VIllager for normal users of the site. Thus user of the site need not be concerned about the Village-type and there is always exactly one Villager submitted by a certain user (programmatically).

    What I did to figure out what I structure I needed to create was to put in a temporary line of code like $test_node = node_load(193); (where 193 is the node-id of a some pre-existing Villager node) and a breakpoint just after this line. When I hit the breakpoint I looked at the structure of $test_node. Then all I had to do was recreate that structure in my new node and then do a node_save() on it. Voila!

Drupal SEO