Diff: DovecotXuidBug

Differences between version 2 and previous revision of DovecotXuidBug.

Other diffs: Previous Major Revision, Previous Author

version 2

Dealing with the Dovecot X-UID Problem

How I got bit by the dovecot X-UID bug and how I solved it.

Versions of the dovecot IMAP server prior to .99.15 contained a bug which would cause the insertion of too-large X-UID headers in mbox-format mail spool files. This causes Thunderbird to choke with a message like "The mail server responded: Invalid messageset: -2147483648". In a normal mail spool file the first delivered message should have an X-UID header value of 1, and for each message the X-UID header is incremented by one. If you examine the spool file manually in this case you will see X-UID values like 4270588972, which would not be possible unless a user had received more than four billion messages!

This bug potentially affects a large number of installed systems because even the most recent CentOS 4 (4.7, as of February 2009) ships with dovecot .99.13. Even worse, this bug can cause mailbox open failures indefinitely because the invalid X-UID headers are left in the spool file even when you upgrade to a newer version of dovecot. To completely fix this problem you have to both upgrade dovecot and fix existing spool files.

If you are running CentOS 4 or any .99.X version of dovecot on any platform, first upgrade to the latest stable dovecot (1.1.11 as of this writing). If you are on CentOS 4 you have to either build dovecot yourself or obtain a prebuilt rpm from a third party site. I used the rpm from ATrpms and it worked fine.

Here's a relevant blog post detailing the problem and a manual fix. Here's a Redhat bugzilla report on the issue.

Finally here is my cleanup script, which does three key things:

• Stops mail delivery via imap and smtp for duration of cleanup
• Removes all X-UID and X-IMAPbase headers
• Removes all user imap index files

Note that if you do not remove both the X-UID and X-IMAPbase headers dovecot will not regenerate the X-UID headers. Also my fix users formail to parse the mailbox, which means that messages embedded in other messages as MIME attachments can still have bogus X-UID headers.

#!/bin/bash

# script to clean bad X-UID and X-IMAPbase headers from
# username spool files, and erase dovecot imap index files.
# all of this stuff gets regenerated the next time dovecot
# accesses the mailbox.

daemons="dovecot sendmail"

# remember to including trailing slash if spooldir is a symlink!
spooldir="/var/mail/"

[ $(id -u) -eq 0 ] || \
  { echo "this script must be run as root, aborting" >&2; exit 1; };

[ -r $spooldir ] || \
  { echo "$spooldir not readable, aborting" >&2; exit 1; };

# we can't be delivering mail while we are cleaning up spool files.
for daemon in $daemons
do
  echo "shutting down $daemon"
  if ps -C $daemon >/dev/null
  then
    /etc/init.d/$daemon stop
  fi
  sleep 5
  if ps -C $daemon >/dev/null
  then
    { echo "$daemon still running, aborting" >&2 && exit 1; };
  fi
done

for file in $(find $spooldir -type f -maxdepth 1)
do
  username=$(basename $file)
  if id $username >/dev/null
  then
    # user exists, continue
    spooltemp=$(mktemp) || \
      { echo "problem creating tempfile, aborting." >&2 && exit 1; };

    echo "removing X-UID and X-IMAPbase headers from $file"
    if ! formail -I X-UID: -I X-IMAPbase: -s < $file >$spooltemp
    then
      echo \
  "not replacing $file with $spooltemp because formail reported an error"
    else
      # cat file to preserve permissions. this is kind of slow.
      cat $spooltemp >$file
      rm $spooltemp

      echo "removing dovecot index directory /home/$username/mail/.imap"
      rm -rf /home/$username/mail/.imap
    fi
  else
    echo "skipping $file because user $username doesn't exist."
  fi
done

# all done, start daemons back up to resume mail delivery.
for daemon in dovecot sendmail
do
    /etc/init.d/$daemon start
done

exit 0