The Road to Pelican 3.2

2013-05-02

Page content

Over the weekend, I updated my Pelican installation from v2.8 to v3.2. I tested the new version for a couple of days and was anxious to get on the new hotness. Previously, I tried to go to Pelican 3.0 but site regeneration was painfully slow for me. Version 3.2 handled my 1800 posts like a champ.

I thought I’d share some tips and describe (again) how I’m using Pelican to run this site.

Background

Pelican is a static blogging system written in Python. Markdown files go in and HTML files come out. In the middle there are Jinja2 templates, JS and CSS that make everything look pretty. When a new post is added to a Pelican blog, the entire site is regenerated. This is necessary (right now) because the index and archive needs to be recreated from scratch. Typically, rebuilding a Pelican blog takes less than 10 seconds.

Many Pelican users only rebuild their site when they add some new content. Most of them do this from their Mac and upload the updated files to their host. I don’t work that way.¹ I like to write from anywhere that I can get to my Dropbox folder of text files. That also means working from iOS.²

Virtualenv

I did future-me a huge favor. I decided to use Python’s Virtualenv for Pelican 3.2. I figured this would make upgrades much easier.³ I could install an entirely new version of Pelican alongside my existing version without interuption. When I want to switch-over, I just change my build script to use the new virtualenv.

Setting up virtualenv is pretty straight forward. First you install the tool and then you install the environment. There are plenty of resources around to figure this out. I performed the following installation on my host, Webfaction, through a SSH session.

:::bash
pip install virtualenv
mkdir virt_env
virtualenv virt_env/pelican3.2
cd virt_env/pelican3.2
source bin/activate

This uses your current default Python version to create an entirely new installation of Python. There are several options for this part if you want to create a clean Python installation or you just want to make use all of your exsiting libraries with this new Python instance.

After the source bin/activate is executed, the command prompt will change to indicate that a virtualenv is referenced.

Go ahead and do a which python to put your mind at ease. It should reference the Python in your virtualenv.

Now get to installing anything else you need. I needed Pelican and Markdown. I also went ahead and installed Typogrify even though I’m not using it right now.

:::bash
pip install pelican
pip install markdown
pip install typogrify

That’s it. Pelican 3.2 is installed and ready to run. Try a which pelican to prove that you’re running what you think you are.

Configuration

Pelican 3.2 has some small, yet critical, changes to the configuration file from the earlier Pelican 2.8. The most important changes are around RSS and link formatting.

Here’s my config file (minus some stuff you don’t need to see):

:::python
#!/usr/bin/env python
# -*- coding: utf-8 -*- #
from __future__ import unicode_literals
AUTHOR = u"Gabe"
SITENAME = u"Macdrifter"
SITEURL = 'http://www.macdrifter.com'
FEED_MAX_ITEMS = 5

DISQUS_SITENAME = "macdrifter"
DISQUS_SHORTNAME = "macdrifter"
TIMEZONE = 'America/New_York'
FALLBACK_ON_FS_DATE = "True"
DEFAULT_DATE = 'fs'
# DELETE_OUTPUT_DIRECTORY = "True"
DEFAULT_LANG='en'

# Blogroll
LINKS =  (
    ('Archives', 'archives.html'),
         )

# Social widget
SOCIAL = (
          ('twitter', 'http://twitter.com/macdrifter'),
         )

DEFAULT_PAGINATION = 10
REVERSE_ARCHIVE_ORDER = 1

DISPLAY_PAGES_ON_MENU = "True"
DEFAULT_CATEGORY = ('Articles')
MD_EXTENSIONS = ['codehilite','extra']
MARKUP = ('rst', 'md')
ARTICLE_EXCLUDES = ('pages',)
#from pelican.plugins import related_posts
PLUGIN_PATH = '/home/macdrifter/virt_env/pelican3.2/plugins/pelican-plugins-master'
PLUGINS = ['sitemap',]
DIRECT_TEMPLATES = ('index', 'tags', 'categories', 'archives', 'highlights')
TAG_CLOUD_STEPS = 4
TAG_CLOUD_MAX_ITEMS = 50
STATIC_PATHS = ['images', 'js', 'css', 'macdrifter-logo-art']
OUTPUT_PATH = '/home/macdrifter/webapps/pelican/'
#ARTICLE_PERMALINK_STRUCTURE = '/%Y/%m/'
ARTICLE_URL = '{date:%Y}/{date:%m}/{slug}.html'

ARTICLE_SAVE_AS = '/home/macdrifter/webapps/pelican/{date:%Y}/{date:%m}/{slug}.html'
ARTICLE_EXCLUDES = ('pages',)
PAGE_URL = 'pages/{slug}.html'

#CATEGORY_FEED_ATOM = 'feeds/%s.atom.xml'
FEED_ATOM = 'feeds/all.atom.xml'

#TAG_FEED_ATOM = 'feeds/%s.atom.xml'

FILES_TO_COPY = (('extra/.htaccess', '.htaccess'),)
COPYRIGHT = 'Copyright Vagabond Industries LLC, 2012'
LOGOTEXT = 'Macdrifter'
THEME = '/home/macdrifter/virt_env/pelican3.2/macdrifter-neue'
LOGOIMAGE = '/theme/macdrifter-logo-art/macdrifter-logo_280px.png'
PATH = '/home/macdrifter/pelican_raw/'
OUTPUT_SOURCES = 'True'
OUTPUT_SOURCES_EXTENSION = '.txt'
SITEMAP = {
    'format': 'xml',
    'priorities': {
        'articles': 0.5,
        'indexes': 0.5,
        'pages': 0.5
    },
    'changefreqs': {
        'articles': 'daily',
        'indexes': 'daily',
        'pages': 'monthly'
    }
}

I’ll point out the most import bits.

The plugins are now a separate download and the plugin path needs to be set in the config file. I’ve enabled the new sitemap generator plugin.

:::python
PLUGIN_PATH = '/home/macdrifter/virt_env/pelican3.2/plugins/pelican-plugins-master'
PLUGINS = ['sitemap',]

The sitemap configuration is at the end:

:::python
SITEMAP = {
    'format': 'xml',
    'priorities': {
        'articles': 0.5,
        'indexes': 0.5,
        'pages': 0.5
    },
    'changefreqs': {
        'articles': 'daily',
        'indexes': 'daily',
        'pages': 'monthly'
    }
}

The feeds are also configured different in Pelican 3.2:

:::python
FEED_ATOM = 'feeds/all.atom.xml'

The ARTICLE_PERMALINK_STRUCTURE configuration has been replaced by ARTICLE_URL.

There are also now configurations to specify the theme directory and raw source file directory.

A simple test run showed me that this new configuration file was working and Pelican 3.2 was live.

Plain Text, Like Grandpa Used To Do

I’m taking advantage of Pelican’s new OUTPUT_SOURCES and OUTPUT_SOURCES_EXTENSION settings to add some Markdown nerdery to Macdrifter. This combination copies my source files into my Apache directory for serving up to the world. I just made one little change to my article_info.html template file to add an [mmd] link on each article:

:::html
<span class="mmdtext"> &#124; <a href="{{ SITEURL }}/{{ article.date|strftime('%Y/%m') }}/{{ article.slug }}.txt" title="Raw Markdown for {{ article.title}}">[mmd]</a> &#124;</span>

Pretty cool right? Jinja is pretty fun stuff. Go ahead click the link and come back.

Cronic Pain

I was feeling pretty good. A little too good. Reality walked up and put a foot right in my omelette.

I just needed to setup my fancy new Pelican build command as a cron job. I whipped up a little shell script (siteregen2.sh) to load the virtualenv and run Pelican.

:::bash
#!/bin/bash
### Switch to the virtualenv and activate
cd /home/macdrifter/virt_env/pelican3.2/
source bin/activate
### Write to a log file. Let's log which version of Pelican is doing the work.
which pelican >> /home/macdrifter/macdrifter_cron.log 2>&1
pelican --version >> /home/macdrifter/macdrifter_cron.log 2>&1
### Kick that Pelican into action by pointing it at the config file and log the output
/home/macdrifter/virt_env/pelican3.2/bin/pelican -s /home/macdrifter/virt_env/pelican3.2/pelican.conf.py >> /home/macdrifter/macdrifter_cron.log 2>&1
### Timestamp that bad boy.
echo "siteregen2.sh: $(date)" >> /home/macdrifter/macdrifter_cron.log 2>&1

I added a line to my crontab so the site regenerates every 5 minutes:

:::bash
*/5 * * * * sh ~/pelican_config/siteregen2.sh

Now, let’s bask in the glow of our cleverness:

Some bad encoding

Nope. That’s not right. Let’s try 100’s of other things, break all of our cron jobs and contact Webfaction support to display the depths of our ignorance. Check.

Finally, I took a trip over to the infinitely helpful Pelican IRC channel.

Salvation

One thing I love about Pelican, is the development team. They are friendly and eager to help. Joining the IRC discussion feels like chatting at the bar. No criticsm, trolling or belittling. Just a bunch of people that like the same kind of things.

Within a couple of minutes of posting my error message several people joined in to offer suggestions. “Avaris”, on the channel, figured it out right away.⁴ Cron knows shit about where it’s running. It doesn’t even know its locale.

One line addition to the top of the script fixed the issue:

:::bash
#!/bin/bash
export LANG=en_US.UTF-8
### Switch to the virtualenv and activate
cd /home/macdrifter/virt_env/pelican3.2/
source bin/activate
### Write to a log file. Let's log which version of Pelican is doing the work.
which pelican >> /home/macdrifter/macdrifter_cron.log 2>&1
pelican --version >> /home/macdrifter/macdrifter_cron.log 2>&1
### Kick that Pelican into action by pointing it at the config file and log the output
/home/macdrifter/virt_env/pelican3.2/bin/pelican -s /home/macdrifter/virt_env/pelican3.2/pelican.conf.py >> /home/macdrifter/macdrifter_cron.log 2>&1
### Timestamp that bad boy.
echo "siteregen2.sh: $(date)" >> /home/macdrifter/macdrifter_cron.log 2>&1

Conclusion

You’re looking at the conclusion. This site is now running on Pelican 3.2.

It rebuilds every 5 minutes. Regenerating ~1800 articles takes about 30 seconds. Counting indexes, articles, category and tag pages, there are around 3000 files generated each time. CPU and memory usage on my host is negligable.

I’ve become a bigger fan of Pelican since I first started using it. Pelican enables me to write for almost anywhere. I write from my Mac in Sublime Text. I write from a PC in Notepad++. I write from my iPhone on the can. Anything that can create text and get an Internet connection can be a blogging tool.

Maybe it’s because I like to be different. Maybe I just like to make things hard for myself. ↩︎
For some previous posts about my Pelican setup, see here, here, and here. ↩︎
Right now Pelican works perfectly on Python2.7. Some day it may require Python3. Virtualenv would make that switch pretty easy. It only takes one extra flag and parameter to build a new virtualenv with a different version of Python. ↩︎
This was the first time I used Gittip. This is why Gittip exists. ↩︎