How to automatically replace files when updating WordPress

WordPress is quick and easy to install and update, but the quicker you can make these operations the better, right? If you have shell access to the server where your WordPress copy is installed, it is possible to perform all the operations in Step 1 of the Manual WordPress Update Procedure with the shell script below. It will save you a few minutes, which may seem too little, but is great if you maintain more than one copy of WordPress. That, however, is not the main reason to use a script like this. Its bigger advantage is reducing the possibility of human error by doing things by hand, at the prompt or with the mouse is the same.

The scripts takes two parameters: the first is the complete URL of the zipped WordPress files that you need in order to upgrade. The second is the directory, called WP_ROOT, where the WordPress installation you need to upgrade lives. You will note that in my script $WP_ROOT is only the name of that directory, not the complete path to it, which in the script is $HTML_ROOT/$WP_ROOT. I have do this because I have several, independent copies of WordPress running on the same server, all living in different subdirectories of my $HTML_ROOT, named with the name of each blog. Therefore, this choice of variables sounds more natural to me. When I want to update http://stop.zona-m.net, I will type `upgrade_wordpress.sh http://wordpress.org/wordpress-3.1.1.zip stop`. When I want to update http://strider.zona-m.net, instead, I will type “strider” instead of “stop”, and so on. Here is the script. Enjoy it, but remember to read the warnings at the end of this page first!


  #! /bin/bash
  # upgrade_wordpress.sh
  # This script performs automatically the operations described in the section
  # "Manual Update - Step 1" of http://codex.wordpress.org/Updating_WordPress
  # Copyright 2011 M. Fioretti http://stop.zona-m.net
  # Released under GPL V2 license
  #
  # $1 URL of latest WordPress version, zipped
  # $2 Root directory of the WordPress installation that must be upgraded

  WP_ROOT=$2
  HTML_ROOT=/var/www/html/wp
  TEMP_DIR=/tmp/temp_wp_update

  # Step 1.1 and 1.2: get latest WordPress, unpack it in temporary directory
  mkdir $TEMP_DIR
  cd    $TEMP_DIR
  wget  $1
  unzip -q wordpress*.zip

  # Step 1.3: Delete the old wp-includes and wp-admin directories
  rm -rf $HTML_ROOT/$WP_ROOT/wp-includes $HTML_ROOT/$WP_ROOT/wp-admin

  # Step 1.4: "upload" the new wp-includes and wp-admin directories
  cp -r -p $TEMP_DIR/wordpress/wp-includes $TEMP_DIR/wordpress/wp-admin $HTML_ROOT/$WP_ROOT

  # Step 1.5: "Upload" the individual files from the new wp-content folder
  # to your existing wp-content folder, overwriting existing files.

  cd $TEMP_DIR/wordpress/wp-content/
  find . -type f | sort  > $TEMP_DIR/new_wp_content_files_list
  tar cf $TEMP_DIR/wp-content-files-to-replace.tar -T $TEMP_DIR/new_wp_content_files_list
  cd  $HTML_ROOT/$WP_ROOT/wp-content
  tar xf $TEMP_DIR/wp-content-files-to-replace.tar

  # Step 6: "Upload" all new loose files from the root directory of the new
  # version to your existing wordpress root directory

  cd     $TEMP_DIR/wordpress
  tar cf $TEMP_DIR/loose_files_in_wp_rootdir.tar license.txt readme.html *php
  cd     $HTML_ROOT/$WP_ROOT
  tar xf $TEMP_DIR/loose_files_in_wp_rootdir.tar

  # Final step: remove temporary files, remember to check the configuration

  rm -rf $TEMP_DIR
  echo "you should take a look at the wp-config-sample.php file, to see if any new settings have been introduced that you might want to add to your own wp-config.php"



Warnings

If you reload the WordPress admin page just after running the script you’ll get the “let’s upgrade the database” button, and everything should be OK after that. I have tested this script myself to update both http://tips.zona-m.net/ and this website from WordPress 2.8.1 on a Centos VPS and as far as I can see it worked just fine.

As everything done with shell scripts, howeverm this thing is powerful enough to both save you time and to hurt your blog if there is some bug or if something changes in the official WordPress instructions before I notice it and update this page. Of course, the script is provided as is, without warranties, and you should (because you should do it anyway!!!) back everything up right before running it.

The script could have been even shorter. I deliberately wrote it in that way to have the same steps as the procedure described at WordPress.org, and make it easier to update if/when something changes in WordPress. If you find errors or ways to improve it, please write them as a comment and I will update the page as soon as possible. Thanks.

How to make a “multilingual” WordPress blog without multilingual plugins

Around October 2010 I migrated from Drupal to WordPress my bilingual websites Stop and Strider. Eighteen months later, WordPress has confirmed to be better than Drupal for my own needs, as far as those two websites are concerned (I still stick to Drupal for other websites).

There is, however, one part of those WordPress websites that has just become a big problem for me, and is how to keep them (looking) multilingual. I think I have found a solution for it and have explained it here for two reasons. First if I am right, this trick will be probably useful to many other WordPress users. The other is caution. I said I think I have a solution: I have done a little testing on a dummy installation (see below) and everything seems to work. However, before implementing it on the actual blogs, I’d really like to have some feedback from the community, just to make sure I am not forgetting something important (which I may have very well done).

When I started those two websites, I wanted them to have one name/base URL as simple and short as possible, so that I could just tell people “go to stop.zona-m.net” and then they would immediately find the version in their language, versus “go to {it,en}.stop.zona-m.net”. English content would have URLs like stop.zona-m.net/2010/11/article-title/, while all Italian content would have the /it/ suffix, as in stop.zona-m.net/it/2010/11/article-title/ and there would be automatic links from each version to the other. Drupal 6 does this, no problem.

Do you REALLY need a multilanguage blog?

When I moved to WordPress the best way to keep this architecture with one WordPress install seemed, and very likely is, the WPML plugin with English as primary language. After a while, however, I realized that WPML doesn’t seem to do all I need and had in Drupal. This may be my own fault but, for example, I found no way in WPML to:

  • use language-dependent widgets, ie how to tell WPML that a certain widget, e.g. an RSS feed, must be shown only in the English or Italian version of each page (update 2011/04/15: Amir, one of the WPML maintainers, points out in the comments to this post that this is now possible with WPML and it is explained here)
  • link to each other, as Drupal lets you do, the different names in each language of each category, in order to have automatic links between, for example, http://stop.zona-m.net/category/digiworld/ and its italian equivalent http://stop.zona-m.net/it/category/digimondo/, or to have automatic preselection of the right categories in the new target language when you translate an existing article.

In addition to this, I also realized something more important. Very likely, no matter how cool it looks, I don’t really need the links with the nice flags between english and italian versions of each page. By this I mean that my readers don’t really need it and almost never use it, because almost all of them only use the website in only one language.

Finally, at the beginning of 2011, WPML announced it would become a commercial/proprietary product. Putting all this together made me decide to find another solution.

Making two copies of WordPress look like one bilingual website

Those two websites run on a Virtual Private Server, on which I can add databases, change apache settings and so on as much as I want. This and all I explained in the previous section made me consider splitting each one of those websites in two independent, monolanguage WordPress installs. The result would be something a little bit less convenient to manage for me than it was with WPML, but nothing substantial, and almost transparent for all my readers.

As long, of course, as no URL changes in the migration! I want all the English URLs to remain the same (no problem) but I also do NOT want to “lose” or change the italian URLs. Here’s how I have achieved this on a test install, please tell me if there are weaknesses!

I have set up a test bilingual blog at bits.zona-m.net, with the same versions of WordPress and all the plugins as the real websites, in the /var/www/html/wp/tips directory, associated to the wp_tips MySql database and with this Apache server configuration:


  <VirtualHost *:80>
      ServerAdmin marco@digifreedom.net
      DocumentRoot /var/www/html/wp/tips
      ServerName tips.zona-m.net
      AccessFileName .htaccess
      CustomLog logs/tips.zona-m.net.access.log combined
      ErrorLog  logs/tips.zona-m.net.error.log
  </VirtualHost>


Then I created a few posts in it, both in English and Italian, with links to each other, to have something meaningful on which to test the following method.

In order to split that WordPress bilingual blog in two independent ones without breaking any URL already created, first I cloned its MySql database:


  # mysqldump -u root -p wp_tips > ~/dumb_wptips_bilingual.sql
  # mysql -u root -p
  > create database wptips_it;
  > use wptips_it;
  > source /root/dumb_wptips_bilingual.sql;
  > GRANT ALL PRIVILEGES ON wptips_it.* TO "sameuser_ofwptips"@"localhost" IDENTIFIED BY "password";


then I duplicated the whole, already working WordPress installation of tips.zona-m.net to another directory:


  cp -r -p /var/www/html/wp/tips /var/www/html/wp/tips_it


and modified the wp-config.php file in tips_it to point at the wptips_it database. Next, I modified the Apache configuration to “fetch” all and only the URLs starting with http://tips.zona-m.net/it from that second directory. In order to do this it is necessary to create an alias in the httpd.conf file:


  <VirtualHost *:80>
      ServerAdmin marco@digifreedom.net
      DocumentRoot /var/www/html/wp/tips
      ServerName tips.zona-m.net
      Alias /it /var/www/html/wp/tips_it
      AccessFileName .htaccess
      CustomLog logs/tips.zona-m.net.access.log combined
      ErrorLog  logs/tips.zona-m.net.error.log
  </VirtualHost>


and to modify the .htaccess files in the two folders as follows:


  # more /var/www/html/wp/tips/.htaccess
  <IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /
  RewriteRule ^index.php$ - [L]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule . /index.php [L]
  </IfModule>
  # more /var/www/html/wp/tips_it/.htaccess
  <IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteBase /it/
  RewriteRule ^index.php$ - [L]
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule . /it/index.php [L]
  </IfModule>
  #


At that point, “all” was left was to convince the cloned WordPress installation in tips_it and its MySql database wptips_it that they now live at another base URL with the it/ suffix, that is tips.zona-m.net/it. It seemed to me that the simplest/safest method in this case among those recommended in the official Change the WordPress URL page is the one called “Edit functions.php”. So I added these two lines to the functions.php file of the active theme:


  # cat /var/www/html/wp/tips_it/wp-content/themes/twentyten/functions.php
  <?php
  /*
  update_option('siteurl','http://tips.zona-m.net/it');
  update_option('home','http://tips.zona-m.net/it');
  */
  /**
   * TwentyTen functions and definitions
   *


then, as the WordPress page above recommends, I loaded the admin page a couple of times and immediately after I removed those two lines. Last but not least, I deactivated English in WPML in the “Italian” blog and Italian in the “English” one. Done! Apparently, everything works, as you can check by yourself by going to http://tips.zona-m.net and http://tips.zona-m.net/it. All the URL created when there was only one bilingual WPML/WordPress install are still valid with the same website URL: you can still visit both http://tips.zona-m.net/2011/04/here-we-go-with-my-second-english-post/ and http://tips.zona-m.net/it/2011/04/versione-italiana-del-secondo-post/, even if they are now served by different copies of WordPress. The only thing that has been lost, but as I said I do wonder if I really needed it, are the links between the several versions of each page.

If this works, I will migrate the real websites to this structure, deactivate/remove WPML completely, then upgrade both of them to the last version of WordPress, customized their layout separately and go on. The only things I should lose are, I believe:

  • possibility to handle posts in both languages from the same browser window. Worth the hassle for me, if it both maintains the old URLs and lets me customize each language at will
  • clear signs for Italian visitors that there is also an English version of each website and vice-versa. I feel that just adding a link in the top menu bar to the home pages of each language should be enough, since I’m pretty confident that almost nobody so far actually used both two languages
  • (totally wild guess, not sure if this really is an existing issue) search engines temporarily pissed off and de-ranking each post because it doesn’t have links from the version in the other language???

What do you think? Have I missed something? Is there any security hole or other weakness that will come and bite me if I migrate the real website? Should I do something else? Please let me know!

How to post content to a WordPress blog from the command line

WordPress is a great publishing system, but managing it manually can be a very time consuming process. This is especially true when you want to upload lots of posts, or if you would like to write content in your preferred, full-blown text editor and then have it “magically” appear online.

WordPress takes care of these needs allowing remote posting via email or the WordPress XML-RPC interface (if you enable the WordPress, Movable Type, MetaWeblog and Blogger XML-RPC checkbox in goinig to Settings > Writing > Remote Publishing). The first method is explained here and in other places, but requires setting up a dedicated email account. For several reasons I preferred not to do it that way, so I looked at the other system.

PHP scripts for this purpose are here and here. The first uses the cURL capabilities of PHP to send the data over SSL. The second uses IXR, the Incutio XML-RPC Library for PHP, to “incorporate both client and server classes, as it is designed to hide as much of the workings of XML-RPC from the user as possible”.

Both those scripts work, and the second can also be used to edit existing posts or get lists of the latest published articles. However I was looking for something that didn’t require PHP (personal preference, really). Eventually, I found the WordPress-CLI utility by Leo Charre and have already used it successfully to upload hundreds of posts to several of my WordPress websites (see bottom of this page for examples). Here’s how I did it.

WordPress-CLI installs like any other Perl Module, see the instructions in the README file. In order for it to work, however, I also had to download and install from CPAN the Perl Modules Getopt-Std-Strict-1.01 and LEOCHARRE-Debug-1.03.

Once everything is installed, you’ll have in your path a script called wordpress-upload-post. Run it at the command prompt or in a script, giving as options the name of the HTML file containing the post, plus its required publication date, title and category, as well as your user name and password, and you’ll have your post online.

To make things faster, I use wordpress-upload-post inside this script:


  #! /bin/bash
  # usage: post2wp.sh postfile blogname
  # postfile: text file containing post content in txt2tags format
  # blogname: name of blog

  POST=$1
  BLOGNAME=$2

  HTML=/tmp/tmp_wordpress_post.html
  ACCOUNTS_DIR= "$HOME/.blog_accounts"

  #########################################################################
  #extract title, category and publication date of the post
  TITLE=`grep '%TITLE: ' $POST | cut -c9-`
  CATEGORY=`grep '%CATEGORY: ' $POST | cut -c12-`
  DATE=`grep '%DATE: ' $POST | cut -c8-`

  YEAR=`echo $DATE | cut -c1-4`
  MONTH=`echo $DATE | cut -c5-6`
  DAY=`echo $DATE | cut -c7-8`
  HOUR=`echo $DATE | cut -c9-10`
  MIN=`echo $DATE | cut -c11-12`
  DATE="$HOUR:$MIN $YEAR/$MONTH/$DAY"

  rm -f $HTML.tmp*
  txt2tags -t xhtml --no-headers -i $POST -o $HTML

  ###########################################################################
  # source blog parameters: user name, password, XML_RPC url

  if [ -e $ACCOUNTS_DIR/$BLOGNAME ]
  then
    source $ACCOUNTS_DIR/$BLOGNAME
  else
    echo "Error! $BLOGNAME account file doesn't exist!"
    exit
  fi

  ###########################################################################
  # upload post to blog
  WP_OPTS="-D '$DATE' -t '$TITLE'  -c '$CATEGORY'  -u $USER -p '$PW' -x $XMLRPC_URL"
  WP_CMD="wordpress-upload-post $WP_OPTS $HTML"
  eval $WP_CMD
  exit


I use txt2tags as source format for most things I publish online. It is a very simple ASCII markup format, that I customize adding to each article to be published on WordPress comments like these:


  %TITLE: Is it OK for a School or Charity to accept software donations?
  %CATEGORY: Digiworld
  %DATE: 200707010900


The script extracts these variables from the source text and reformats them in the way required by wordpress-upload-post.

Next, it converts the article from the ASCII markup to HTML calling the txt2tags Python Script. Password, user name and the URL of the wordpress blog to use to upload the HTML post are in a separate file ($ACCOUNTS_DIR/$BLOGNAME) that has this format:


  USER='your user name'
  PW='your password'
  XMLRPC_URL='http://your.blog.home.page/xmlrpc.php'


and is read right after generating the HTML version of the post. The last part of the script, “upload post to blog” concatenates all the parameters in one option string WP_OPTS, build the publishing command WP_CMD and evaluates, thus publishing your post online. Enjoy! Of course, if you only post once in a while you won’t save much time, but if you ever need, as I did, to post lots of stuff, try this!

What’s missing

wordpress-upload-post and this script work great (see bottom of page), but they aren’t perfect. The biggest limit right now of WordPress-CLI is that you can’t specify WordPress tags or, if you have a multilingual blog, the language of the current post. I’d also like to use it to add comments to existing posts, but that’s not essential really. I discussed these things with Leo. His answer is that “some of these things simply won’t work. For example- adding tags- because xmlrpc.php does not implement a call to add a tag. I’ve made some hacks to be able to do so- but this works on a local/server level. [also] I can’t recall right now if they have a comment call”. Leo also asked me to forward to all readers of this page this invitation:

  I'd be open to actually share in maintenance of things like WordPress::XMLRPC and a WordPress::CLI revised version. If you want to do this level of changes/additions, we could set up a branch off some cvs server... and... implement bugzilla on some server- to keep track of changes and todos.

Another thing I’d like to figure out is a way to upload images to WordPress so that they are associated to that post and get thumbnails. If that were possible, it would be easy to figure out in advance the right URLs for both the images and their thumbnails, and add them to the txt2tags source. As it is today, in the rare cases where I need to upload with this script a post that does have images, I add them in a second moment by hand. Suggestions?

How I’ve used this script

If you are curious to what real posts written in txt2tags format, then converted to HTML and automatically uploaded to a WordPress blog look like, and don’t mind a bit of self-promotion, have a look at the following links. I have already used the script above to: