Upgrading From Previous Versions

Upgrading from 2.4.15

      * The tools/rehash utility has been fixed, we recommend ALL sites
        with the fulldirhash option enabled to run tools/rehash on their
        mailspools. 

Upgrading from 2.4.14

      * We recommend that ALL sites with fulldirhash enabled run
        tools/rehash on their mail spools after upgrading from 2.4.14.
        There were serious 32 vs 64 bit hashing bugs which were made worse
        with 2.4.14. NOTE: the syntax of tools/rehash has changed. Run it
        without arguments to see a usage statement. 

Upgrading from 2.4.12

      * some LOG_DEBUG level messages are no longer created by default in
        the duplicate delivery system, so you won't see them in syslog any
        more 
      * mailboxes in the DELETED. heirarchy are now always suppressed for
        non-admin users, even if delayed delete is not enabled (see Bug
        #3604). If you happen to be using this namespace at your site for
        something else, you're crazy - but you can set "deletedprefix" in
        imapd.conf to an invalid value (e.g. '-') if you need. 

Upgrading from 2.4.11

      * To support bug-interoperability with older versions of Cyrus, the
        quota command now supports "-1" as a synonym for unlimited storage 

Upgrading from 2.4.10

      * Duplicate database formats have changed, so any duplicate or
        vacation data will be lost - meaning that vacation responses will
        be sent again, and duplicates just across the upgrade time will
        both get delivered. 

Upgrading from 2.4.9

      * quota -f now works correctly. If you are upgrading from earlier
        than 2.4.9 we recommend that you run quota -f to ensure that all
        quotas are now correct. There is a slight race condition in the
        quota command, so it's safest to run with the server shut down, but
        this has always existed. 
      * On the topic of quota -f, the documentation has been updated to
        make it quite clear that you need to give "complete" prefixes if
        using quotalegacy, as it doesn't implement quite the same semantics
        as other databases. If you run 'quota -f a' and you have domains
        starting with 'a', you will be quite sad at the results. So don't
        do that. Running quota -f with no argument, or with a full domain
        or full user specification is perfectly safe. 

Upgrading from 2.4.8

      * CYRUS_PREFIX environemnt varabile. If you have an environment
        variable "CYRUS_PREFIX" then config files will be searched for in
        there first, so for example the file /var/cyrus/etc/imapd.conf will
        override /etc/imapd.conf if your CYRUS_PREFIX is /var/cyrus. This
        is to make things easier for sites with multiple installs on a
        single machine. You will want to check that there aren't unexpected
        files in those locations when upgrading. 
      * New config options: *_db_path allow moving individual DB files to
        different partitions - for example you may want to put the
        deliver.db onto tmpfs to improve performance. 
      * KNOWN BUG: quota -f can double ALL quota usage values for all
        users. This is particularly bad because of the bug fixed in this
        release where quotaroots were not correctly updated on folder
        rename. Recommended workaround: run quota -f twice. 

Upgrading from 2.4.7

      * New config option: proc_path allows setting the path to the proc
        directory onto tmpfs more easily 

Upgrading from 2.4.6

      * New config option: failedloginpause allows you to change the pause
        after a failed login from the existing default of 3 seconds 
      * On Solaris: now getpassphrase is used, so passwords longer than 8
        characters will work with management tools 
      * $confdir/proc now gets created automatically if it doesn't exist,
        which may impact init script design 
      * If you have damaged mailboxes that weren't previously fixable, a
        reconstruct of those mailboxes may be advised 

Upgrading from 2.4.5

      * New config option: suppress_capabilities, which takes a space
        separated list of capabilities which will NOT be given in any imap
        capability response. This can be used on frontends to not display
        capabilities that older backends don't have, so clients don't get
        confused 

Upgrading from 2.4.4

      * sync_client no longer forks two process - it just pre-forks a
        single process in rolling replication mode and uses it "forever".
        This is to avoid the bug where it used to open the BDB environment
        once before forking, then close it in every child, causing the
        reference count to go negative. This also means you can start it
        without '-o' and still have the master start without the replica
        already running. 

Upgrading from 2.4.3

      * Actually, it's more upgrading directly to here from 2.3.x or older.
        ALL upgrades from previous versions now incur a full re-parse of
        all messages, which may take a while. In exchange you get reliable
        upgrades though, and guaranteed still-searchable cyrus.cache
        records. 

Upgrading from 2.4.2

      * The AFS ptloader configure options have changed. If you were
        previously specifying --with-afs when you built, you'll now want to
        use --enable-afs instead. Additionally, you may now specify
        --with-afs-libdir and --with-afs-incdir to facilitate finding the
        AFS libraries and header files. 
      * The sync_log_names option has been replaced with sync_log_channels
        and the documentation and tools updated to match. This is to use a
        less overloaded term and allow multi-channel replication to be
        used. 
      * Cyrus 2.4.3 can now XFER mailboxes back to earlier version (at
        least as far back as Cyrus 2.2.12) in a murder configuration. 
      * sync_client in rolling mode now only connects after forking, so it
        no longer blocks startup. You can remove the '-o' option from your
        cyrus.conf entry if you want, and not have to start sync_client
        manually afterwards! 

Upgrading from 2.4.0

      * The response to the "ID" command has changed slightly to include
        git metadata rather than CVS metadata - so your in-house tools
        expecting a particular value may need changing (very unlikely) 

Upgrading from 2.3.16

      * There is a new index format, so downgrades are not possible without
        a reconstruct. We recommend that you take a full backup before
        upgrading, and perform an upgrade with a small test server if
        possible to make sure everything will work for your site. 
      * MURDER does work, but you can't rename a mailbox back to a previous
        version backend because the index was upgraded. 
      * There is a new replication protocol, so you can't replicate between
        2.4.0 and previous versions in either direction. You need to
        upgrade both before you can restart repliation. 
      * The default value for delayed_expunge has changed to provide better
        QRESYNC and replication support. The default is still pretty
        efficient, but you may want to change it to 'immediate', the
        previous default 
      * The default type for all databases is now skiplist which is very
        reliable now, all the bugs are ironed out! Because ctl_cyrusdb -r
        automatically converts databases between known types, you shouldn't
        need to do anything, but if you want to keep the old defaults,
        you'll need to make them explicit in your imapd.conf as follows: 

        duplicate_db: berkeley-nosync
        ptscache_db: berkeley
        statuscache_db: berkeley-nosync
        tlscache_db: berkeley-nosync

      * There is a new lock folder which defaults to configdir/lock/ and
        contains one zerobyte file per mailbox. These can get pretty hot,
        and don't need to persist over reboots (they will be auto-created
        when needed) - so you may want to define mboxname_lockpath to be on
        tmpfs or ramfs or similar. It certainly makes sense to clean it out
        on restart, because names will persist in there forever otherwise.
        Even on mailbox delete these files aren't removed (to avoid
        potential race conditions) 
      * The make_md5 and make_sha1 utilities no longer exist. If you want
        to check message file integrity, the best way is reconstruct -r -n
        -G. This will parse every message file, check the sha1 against the
        value stored in the cyrus.index, and tell you about mismatches
        without fixing them. 
      * Speaking of reconstruct, the switches have changed, as has the
        implementation. It fixes the mailbox "in place" a lot more rather
        than creating a new mailbox, meaning that IMAP semantics won't be
        broken. It's more likely to abort rather than complete if your
        filesystem permissions are wrong - fix them first! 

        -kRemoved, expunge data is always kept

        -gRemoved, GUIDs are always stored

        -GChanged, now means "re-parse the message". If not specified,
        reconstruct only checks the message size and assumes the rest of
        the stuff in the index record is OK (pretty safe because we do
        CRC32 checks on index records now)

        -sAdded, skip the stat call, so reconstruct doesn't even check the
        message file size - just that a file with the correct name exists
        at all

        -nAdded, don't make any changes, just report what would have been
        done if -n wasn't specified

        If you try to use -k or -g, you will get a warning but the
        reconstruct will still run. 

Upgrading from 2.3.15

      * The SQL detection code in configure has been reworked. Separate
        options for the include directories and lib directories have been
        added. Previous SQL options may not work as expected. 

Upgrading from 2.3.10

      * STARTTLS is now allowed for externally preauth'd LMTP connections.
        If you don't want STARTTLS to be advertised and used by preauth'd
        clients, you can set <service_name>_tls_cert_file: disabled in
        imapd.conf. 

Upgrading from 2.3.9

      * The method used for generating Globally Unique IDentifiers used for
        replication has been changed to be the SHA1 hash of the messages.
        If you wish to upgrade the existing GUIDs in particular mailbox(es)
        or the entire server, perform the following steps in the listed
        order. Note that is is NOT REQUIRED that existing GUIDs be
        upgraded. 

         1. Zero GUIDs on the replica (reconstruct -g) 
         2. Regenerate GUIDs on the master (reconstruct -G) 
         3. Regenerate GUIDs on the replica (reconstruct -G) 

Upgrading from 2.3.8

      * You must install and configure Cyrus SASL version 2.1.17 or later
        to use Cyrus IMAP 2.3.9 and later. You can download SASL at
        http://www.cyrusimap.org/mediawiki/index.php/Downloads#SASL_Library. 
      * The default value of the allowplaintext option has been changed to
        disabled (0). If you need to allow cleartext passwords on the wire,
        then you will have to explicitly enable the allowplaintext option
        in imapd.conf. 

Upgrading from 2.3.3 or later (64-bit machines)

      * Due to byte alignment issues in cyrus.index, all mailboxes will
        have to be reconstructed. 

Upgrading from 2.3.4 or 2.3.5

      * Any mailboxes which had messages appended/delivered/copied with a
        2.3.4 service or copied with a 2.3.5 imapd MUST be reconstructed in
        order for the new messages to be displayed by clients. 

Upgrading from 2.2.x or earlier

      * If you wish to use separate metadata partition(s), you MUST first
        shut down Cyrus and then perform the following: 

         1. Set the metapartition-* and metapartition_files options to suit
            your configuration. For a full description of these options,
            see the imapd.conf(5) man page. 
         2. Create the metadata partition directory(s) listed in the
            metapartition-* option(s), setting the ownership and
            permissions in same fashion as step 6 of install-configure. 
         3. Run the tools/migrate-metadata script (as the cyrus user) to
            move the metadata files listed in the metapartition_files
            option from the spool partition(s) to the new metadata
            partition(s). This script may take a long time to run depending
            on the number of mailboxes on the server, but presumably the
            metadata partitions are located on high speed storage, so the
            writes should be relatively fast. 
         4. Restart Cyrus. 

Upgrading from 2.2.2 or earlier

      * The Cyrus database backend configuration is now handled at runtime
        using imapd.conf options. If you are not using the default backend
        for any of the databases, make sure that you specify the correct
        backend(s) in appropriate option(s). 
      * The format of the newspeer option has been changed. The existing
        format will still be parsed, but the option should be upgraded to
        use the new format (see imapd.conf(5) for details). 

Upgrading from 2.2.1 or earlier

      * The sieve bytecode format has changed again to correct an issue
        with the short circuiting of the allof and anyof operators. To
        upgrade existing scripts (outside of home directories), you can run
        the tools/masssievec perl script included with the distribution. It
        requires a path to your sievec binary. This should also upgrade
        scripts that have already been compiled to bytecode. For example: 

        masssievec /usr/src/cyrus/sieve/sievec

Upgrading from 2.2.0 or earlier

      * The improved directory hashing (fulldirhash) is now a runtime
        configuration option. If you are currently using this feature, then
        make sure that you enable the fulldirhash option in imapd.conf. 
      * The format of mailbox index files has changed. They are upgraded on
        the fly, so you need to do nothing to upgrade. However, to
        downgrade them you will need to remove the cyrus.index files, and
        reconstruct the mailboxes, otherwise the index files will be
        invalid. 
      * ctl_deliver -E has been deprecated in favor of cyr_expire -E. This
        new tool does both duplicate delivery database pruning as well as
        message expunging. You should replace the appropriate EVENTS entry
        in cyrus.conf with one of those in the sample configurations in the
        master/conf directory. 
      * The sieve bytecode format has changed. The new format is encoded in
        network byte order, and will be transferable between architechures.
        To upgrade existing scripts (outside of home directories), you can
        run the tools/masssievec perl script included with the
        distribution. It requires a path to your sievec binary. This should
        also upgrade scripts that have already been compiled to bytecode.
        For example: 

        masssievec /usr/src/cyrus/sieve/sievec

Upgrading from 2.1.x or earlier

General information (ALL SITES)

      * The default database formats for the mailbox list and the seen
        state databases has been changed to the skiplist backend. There are
        two ways of dealing with this if you have been using the defaults. 

         1. Specify --with-mboxlist-db=berkeley and --with-seen-db=flat to
            configure. This will instruct Cyrus to continue to use the
            previous defaults. 
         2. Use the cvt_cyrusdb program to directly convert the databases.
            This should be done with the server down, and with the binaries
            from the new Cyrus distribution. Change any paths that do not
            match your configuration.
            For the mailbox list, the command looks like: 

            /usr/cyrus/bin/cvt_cyrusdb /var/imap/mailboxes.db berkeley /var/imap/mailboxes.db.new skiplist
            mv /var/imap/mailboxes.db.new /var/imap/mailboxes.db

            Note that the use of full paths to the database files is
            important. You should also backup your old mailboxes database
            before moving the new one in.
            For the seen state databases, the command to get them all in
            one fell swoop looks like: 

            find /var/imap/user -name \*.seen -exec /usr/cyrus/bin/cvt_cyrusdb \{\} flat \{\}.new skiplist \; -exec mv \{\}.new \{\} \;

            The slashes are important for shell escaping. Again, you should
            back up the contents of your /var/imap/user directory before
            executing this command. These commands may take some time to
            complete, especially if your databases are large. 

        We believe that skiplist offers considerable performance advantages
        for these two databases over the previous defaults. 
      * Sieve scripts are now compiled into bytecode. The program sievec is
        provided to do this process manually (timsieved will compile
        submitted sieve scripts as they are uploaded). To upgrade existing
        scripts (outside of home directories), you can run the
        tools/masssievec perl script included with the distribution. It
        requires a path to your sievec binary. For example: 

        masssievec /usr/src/cyrus/sieve/sievec

        Note that this will fail for scripts that use the "envelope"
        extention but do not require it. Cyrus 2.1's timsieved did not do
        appropriate checking that the optional envelope test was required
        before it was used. 
      * Configuration subsystem changes: 

          * The tls_[service]_* configuration options have been removed.
            Now use [servicename]_tls_*, where servicename is the service
            identifier from cyrus.conf for that particular process. 
          * The admins and lmtp_admins configuration options no longer
            union. Per-service options completely override the default
            value when they are specified. 
          * lmtp_allowplaintext is no longer a defined parameter and must
            be specified using the service name of your lmtp process if you
            require a specific value. 

Specialized information (Murder, AFS, etc.)

      * The IMAP IDLE command is now supported by proxyd and is controlled
        by the imapidlepoll option, which is enabled by default (60
        seconds). To disable IMAP IDLE in proxyd, set imapidlepoll to 0. 
      * User moves via RENAME and XFER are now controlled by the
        allowusermoves option, which defaults to off. 
      * If you use ptloader, it now runs as a regular cyrus service. This
        means that you will need master to acquire and maintain AFS tokens
        for it. You will also need to create the ptclient directory under
        your imap configdirectory, to hold the PTS cache (now a
        full-fledged cyrusdb) and UNIX socket. In cyrus.conf, ptloader
        should be setup to listen on <configdirectory>/ptclient/ptsock. See
        the master/test/cmu-backend.conf example configuration file. 
      * Also, ptloader has been given a generic interface. You should now
        specify "--with-auth=pts" (instead of "--with-auth=krb_pts") to
        configure. There is also a --with-pts= configure option that
        defaults to afskrb (Kerberos Canonicalization, AFS PTS Groups).
        There is also an experimental ldap module. Note also that if
        ptloader fails the lookup, authorization (and therefore
        authentication) will now fail, as canonicalization is done inside
        of ptloader. 
      * The format of sieve referrals has changed to be more consistant
        with the current managesieve draft, this may cause interoperability
        problems when using managesieve clients and servers from different
        cyrus versions. 
      * Clients that use old-style ACL commands that include the "MAILBOX"
        directive will no longer function. We do not know of any clients
        that have this problem currently. 
      * Any applications that link libcyrus.a now need to link
        libcyrus_min.a as well. 

Upgrading from 2.1.13 or earlier

      * We are now more forgiving of MIME boundry headers generated by
        earlier versions of eudora. However, if you have messages already
        in the mailstore that you want to fix you will need to reconstruct
        the affected mailboxes to regenerate the cached bodystructure data
        to take this into account. Nothing needs to be done for new
        messages to be treated in this way. 

Upgrading from 2.1.12 or earlier

      * timsieved was corrected to behave properly in the altnamespace
        configuration. However, this means that it was previously looking
        for sieve scripts in "user.name" format instead of the (correct)
        "user^name" format. A sample script to do this (which should be run
        in the top level of the sieve directory) is in
        tools/convert-sieve.pl. Note that this is only needed if you are
        running with altnamespace turned on. 

Upgrading from 2.1.3 or earlier

      * If you use notifications (previously notify_zephyr or notify_unix)
        this functionality has been seperated out to notifyd. See the
        notifyd manpage and example entries in master/conf. 

Upgrading from 2.1.2 or earlier

      * Sieve has been updated to be compliant with RFC 3028 and
        draft-martin-sieve-notify-01. All notify actions and any fileinto
        and/or redirect actions using stringlists will have to be
        updated/changed. 

Upgrading from 2.0.16 or earlier

      * You must install and configure Cyrus SASL version 2 to use Cyrus
        IMAP 2.1 and later. You can download SASL at
        http://www.cyrusimap.org/mediawiki/index.php/Downloads#SASL_Library. 
      * If you use timsieved to manage Sieve scripts, and have enabled the
        alternate namespace and/or the Unix hierarchy separator, run the
        script "tools/translatesieve". This script will translate the
        folder names in fileinto actions. 
      * Cyrus now uses the service name "sieve" instead of "imap" for the
        SASL profile of timsieved. If you use timsieved to manage Sieve
        scripts, be sure to update your password checking mechanism
        appropriately, 
      * If you have enabled the improved directory hashing scheme, run the
        script "tools/rehash full". This script will rehash your existing
        directories. 
      * The hashed deliver databases (used for duplicate delivery
        suppression and Sieve) have been merged into a single deliver.db
        database. You can safely remove the entire /var/imap/deliverdb
        directory structure after shutting down the server. 
      * All of the Cyrus databases have been unified under a single BDB
        environment. A new ctl_cyrusdb tool is now used for database
        recovery and checkpointing instead of ctl_mboxlist and ctl_deliver.
        You should replace the appropriate START and EVENTS entries in
        cyrus.conf with those in the sample configurations in the
        master/conf directory. 
      * Cyrus now caches SSL/TLS sessions in an external database. If you
        have support for SSL/TLS, and haven't disabled session caching (see
        imapd.conf(5)), you should add a line like the following to the
        EVENTS section of cyrus.conf to prune expired sessions from the
        database: 

           # this is only necessary if caching TLS sessions
           tlsprune      cmd="tls_prune" period=1440

Upgrading from 2.0.6, 2.0.7, 2.0.8, or 2.0.9 or earlier

      * If you use timsieved to manage Sieve scripts, run the script
        "tools/upgradesieve". timsieved now uses symlinks instead of hard
        links. 

Upgrading from a previous 2.0 version to 2.0.6

    Warning: You do not need to follow these instructions if you're
    upgrading from version 1.6. 

      * You can now pick whether to use Berkeley db to store seen state,
        the subscription files, and the mailboxes file or a flat text file,
        at compile time only. (Look in imap/seen_db.c and imap/mboxlist.h.) 
      * The format of the mailboxes file and seen state has changed. It is
        not possible to preserve seen state, but upgrade the mailboxes file
        as follows: 

         1. Run ctl_mboxlist -d > mboxlist.temp to dump existing mailboxes. 
         2. Remove old database files: rm mailboxes.db db/* user/*/*.seen 
         3. With the new version of ctl_mboxlist, run ctl_mboxlist -u <
            mboxlist.temp. 

Upgrading from 1.6.22 or 1.6.24

    Warning: Cyrus imapd 2.0 will automatically convert on-disk file
    formats as the server is used. It is not possible to run 1.6 after 2.0
    has been used on a mail spool without reconstructing every mailbox. 

      * Create some extra directories and remove the duplicate delivery
        database: 

           mkdir /var/imap/db
           mkdir /var/imap/socket
           chown cyrus /var/imap/db /var/imap/socket
           rm -rf /var/imap/deliverdb

      * Convert mailboxes file to Berkeley DB: 

           su cyrus
           cd /var/imap
           ctl_mboxlist -u < mailboxes
           ctl_cyrusdb -c

        Please keep a backup of your mailboxes file. You can dump an
        old-style mailboxes file by using ctl_mboxlist -d. 
      * remove "/etc/inetd.conf" entries. The imap and popd3d lines need to
        be removed from /etc/inetd.conf and inetd needs to be restarted. 
      * master process configuration: You'll need to configure the master
        process Cyrus process and ensure that it starts on boot. see this
        section of the configuration instructions. 
      * MTA configuration. You will have to reconfigure your MTA to speak
        to lmtpd. See this section of the configuration document. 
      * cyrus.seen conversion. The cyrus.seen file will be automatically
        upgraded as users read mail. After some time, you might want to
        delete the cyrus.seen file in each mailbox; it is superceded by the
        user/joe.seen file. 
      * cyrus.index conversion. The cyrus.index file will be automatically
        upgraded the first time each mailbox is SELECTed. 
      * Netnews conversion. The netnews programs are no longer built. If
        you are using netnews, you will need to apply the diff in the
        netnews/ directory to INN or see if INN is now distributing those
        changes. You will also want to run remotepurge on a regular basis
        to purge old netnews posts. 

Upgrading from 1.6.13

      * Upgrading from the Cyrus IMAP server version 1.6.13 or earlier: if
        you use Sieve, you should run the "tools/upgradesieve" script, as
        the format of the "/var/sieve" directory has changed slightly. 

        timsieved, included in this release, will handle maintenance of
        Sieve scripts. 
      * Upgrading from the Cyrus IMAP server version 1.6.10 or earlier: if
        you export news via the IMAP server, you'll have to change your
        "newsfeeds" file to contain 

        collectnews!:*:Tf,WR:collectnews

        The format of the input to collectnews has changed. 

        Duplicate delivery suppression is now required for Sieve. 
      * Upgrading from the Cyrus IMAP server version 1.6.1 or earlier
        (including 1.5.x!): the quota and user directories are now hashed
        by the first character of the username. This is to reduce the
        number of entries in any given directory. It doesn't do a great job
        (and in some cases it will do a really poor job) but as a quick
        hack it shouldn't make things worse. Optionally, the data
        partitions can also be hashed by enabling the "hashimapspool"
        option. 

        You must hash your directories using the "dohash" script in the
        tools subdirectory. (If you want to hash your mail spool, be sure
        to set "hashimapspool" before running "dohash".) This must be run
        as the Cyrus user. Be sure to stop mail service while converting.
        Doing this in single user mode is probably the safest. 

Upgrading from 1.5

      * Upgrading from the Cyrus IMAP server version 1.5 or earlier:
        libsasl is now required. Configuring SASL to work may be a chore,
        especially if you use shadow passwords. 
      * An ANSI C compiler is now required. gcc should work fine and can be
        acquired from http://www.gnu.org/software/gcc/gcc.html. 
      * Make sure to read the upgrading instructions under 1.6 above. 
      * Upgrading from 1.5.14 or earlier requires deleting the delivered
        database. Remove the file delivered.db in the configdirectory and
        make a directory called "deliverdb" in the configdirectory. This
        may cause some duplicates to get through. 
      * Upgrading from 1.5.14 or earlier requires removing the PTS cache
        database (if the AFS PTS group support is used, which is not the
        default). The PTS cache is in /var/ptclient/ptscache.db, and you
        should remove it. This is because the format for the PTS cache for
        IMSP has changed. If you use AFS ACLs, IMSPd, and IMAPd on the same
        machine, make sure you have version 1.5a5 of the IMSP server for
        this version of the IMAP server. (If you don't have IMSP, or AFS,
        don't worry about it.) 

