Upgrade iRedMail from 0.9.6 to 0.9.7

Upgrade iRedMail from 0.9.6 to 0.9.7

Warning

THIS IS A DRAFT, DO NOT APPLY ANY STEPS MENTIONED IN THIS TUTORIAL.

Paid Remote Upgrade Support

We offer remote upgrade support if you don't want to get your hands dirty, check the details and contact us.

ChangeLog

May 30, 2017: Fixed: incorrect freshclam parameter UpdateLogFile.
May 22, 2017: Fixed improper Fail2ban filter for Dovecot, add new filter for Roundcube.
May 15, 2017: SQL structure change in vmail.alias SQL table
May 3, 2017: Fixed: improper order of Postfix HELO restriction rules.
Apr 13, 2017: Fixed: incorrect owner and permission for rotated Dovecot log files
Mar 22, 2017: New backup script for SOGo.
Mar 16, 2017: Fixed: Avoid possible backdooring mysqldump backups
Mar 8, 2017: [RHEL/CentOS][Nginx] Fix incorrect session.save_path in php-fpm pool config file.

General (All backends should apply these steps)

Update `/etc/iredmail-release` with new iRedMail version number

iRedMail stores the release version in /etc/iredmail-release after installation, it's recommended to update this file after you upgraded iRedMail, so that you can know which version of iRedMail you're running. For example:

0.9.7

Upgrade Roundcube webmail to the latest stable release (1.2.5)

There're several security fixes in Roundcube 1.2.4 and 1.2.5, all users are encouraged to upgrade it as soon as possible. For more details about this release, please check Roundcube release notes:

1.2.4

1.2.5

Please follow Roundcube official tutorial to upgrade Roundcube webmail to the latest stable release immediately:

How to upgrade Roundcube.

Fixed: improper order of Postfix HELO restriction rules.

iRedMail-0.9.6 and earlier releases didn't configure Postfix to apply custom HELO restriction rule before FQDN helo hostname check and DNS verification, this way you cannot whitelist some bad HELO hostnames. Please follow steps below to fix it.

Open file /etc/postfix/main.cf (Linux/OpenBSD) or /usr/local/etc/postfix/main.cf (FreeBSD), find parameter smtpd_helo_restrictions like below:

smtpd_helo_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    reject_non_fqdn_helo_hostname
    reject_unknown_helo_hostname
    check_helo_access pcre:/etc/postfix/helo_access.pcre

Move the check_helo_access line after permit_sasl_authenticated:

smtpd_helo_restrictions =
    permit_mynetworks
    permit_sasl_authenticated
    check_helo_access pcre:/etc/postfix/helo_access.pcre
    reject_non_fqdn_helo_hostname
    reject_unknown_helo_hostname

Reloading or restarting Postfix service is required.

Fixed: incorrect owner and permission for rotated Dovecot log files

iRedMail-0.9.6 and earlier releases have an incorrect logrotate setting for Dovecot log file, it causes all Dovecot log files are empty due to no required permission to open log files. Please follow steps below to fix it.

Please open file /etc/logrotate.d/dovecot, find line below:

    create 0600 vmail vmail

Remove above line and save the change.

Fixed: incorrect session.save_path in php-fpm pool config file on RHEL/CentOS

Attention

This is applicable to RHEL/CentOS system, and Nginx web server.

iRedMail-0.9.6 doesn't set path for session.save_path parameter in php-fpm pool config file /etc/php-fpm.d/www.conf, please fix it with steps below:

Open file /etc/php-fpm.d/www.conf, find line:

php_value[session.save_path] = "/var/lib/php/session"

The directory name should be sessions (ends with s), not session. So please change it to:

php_value[session.save_path] = "/var/lib/php/sessions"

Restarting php-fpm service is required:

service php-fpm restart

Fixed: incorrect freshclam setting `UpdateLogFile`

With iRedMail-0.9.6, freshclam program cannot update ClamAV signatures due to improper log file permission, please open its config file /etc/freshclam.conf, comment out setting UpdateLogFile to use syslog for logging.

Fail2ban: fixes an improper filter and add new filter rule

iRedMail-0.9.7 fixes an improper filter for Dovecot log file which may cause incorrect ban, and adds a new filter for Roundcube log file to help ban bad client while Roundcube is running behind a proxy server.

On Linux:

cd /etc/fail2ban/filter.d/
rm -f dovecot.iredmail.conf roundcube.iredmail.conf
wget https://bitbucket.org/zhb/iredmail/raw/default/iRedMail/samples/fail2ban/filter.d/dovecot.iredmail.conf
wget https://bitbucket.org/zhb/iredmail/raw/default/iRedMail/samples/fail2ban/filter.d/roundcube.iredmail.conf

Restarting Fail2ban service is required.

NEW: New backup script for SOGo

Attention

This is not applicable to SOGo-2.x because it doesn't support backing up all users' data with command sogo-tool backup /path/to/backup/dir ALL.

iRedMail has script /var/vmail/backup/backup_mysql.sh (or backup_pgsql.sh) to backup SOGo database by dumping whole database to a plain SQL file as backup. It's not ideal because:

it's hard to restore single user's data with a plain SQL file.
if SOGo changes some SQL structure, it's hard to restore all data.

This new script does backup with sogo-tool backup command to avoid issues mentioned above, you can restore a single user's data or all users data with sogo-tool restore.

Please follow steps below to setup this daily cron job.

Download backup script. We store it under /var/vmail/backup, if you prefer a different directory, feel free to change the directory name used in commands below:

cd /var/vmail/backup/
wget https://bitbucket.org/zhb/iredmail/raw/default/iRedMail/tools/backup_sogo.sh
chmod +x backup_sogo.sh

This script will create new directory under /var/vmail/backup like below to store backup files:

/var/vmail/backup
            |- sogo/
                |- 2017/                # <- year
                    |- 03/              # <- month
                        |- 22.tar.bz2   # <- day (file name is: <day>.tar.bz2)

If you prefer a different backup root directory, please open
`backup_sogo.sh`, update variable `BACKUP_ROOTDIR` with the new directory.

Open file backup_sogo.sh, modify
Run command crontab -e -u root to setup root user's cron job. Add content below as new job:

# SOGo: backup all users' data at 3:05AM everyday.
5   3   *   *   *   bash /var/vmail/backup/backup_sogo.sh

OpenLDAP backend special

Fixed: Avoid possible backdooring mysqldump backups

For more details about this backdooring mysqldump backup issue, please read blog post:

[CVE-2016-5483] Backdooring mysqldump backups.

Steps to fix it:

Open the daily MySQL backup script, it's /var/vmail/backup/backup_mysql.sh by default. if you use different storage directory during iRedMail installation, you can find the base directory with command postconf virtual_mailbox_base.
Find variable name CMD_MYSQLDUMP like below:

export CMD_MYSQLDUMP="mysqldump ..."

Make sure it has argument --skip-comments like below:

export CMD_MYSQLDUMP="mysqldump ... --skip-comments"

Save your change. That's it.

MySQL/MariaDB backend special

SQL structure change in `vmail.alias` SQL table

We've made some changes to vmail.alias SQL table for easier account management, you can find details about this change here: SQL structure changes in vmail.alias table.

This change introduces 2 new SQL tables (forwardings, alias_moderators), and (optionally) dropped few columns in vmail.alias table.

iRedAPD and iRedAdmin (and iRedAdmin-Pro) have been upgraded to use this new SQL structure.

Create required new SQL tables

Please connect to MySQL server as MySQL root user, and execute SQL commands below to create required new tables:

USE vmail;

CREATE TABLE IF NOT EXISTS alias_moderators (
    id BIGINT(20) UNSIGNED AUTO_INCREMENT,
    address VARCHAR(255) NOT NULL DEFAULT '',
    moderator VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    PRIMARY KEY (id),
    UNIQUE INDEX (address, moderator),
    INDEX (domain)
) ENGINE=InnoDB;

CREATE TABLE IF NOT EXISTS forwardings (
    id BIGINT(20) UNSIGNED AUTO_INCREMENT,
    address VARCHAR(255) NOT NULL DEFAULT '',
    forwarding VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    -- defines whether it's a standalone mail alias account. 0=no, 1=yes.
    is_list TINYINT(1) NOT NULL DEFAULT 0,
    -- defines whether it's a mail forwarding address of mail user. 0=no, 1=yes.
    is_forwarding TINYINT(1) NOT NULL DEFAULT 0,
    -- defines whether it's a per-account alias address. 0=no, 1=yes.
    is_alias TINYINT(1) NOT NULL DEFAULT 0,
    active TINYINT(1) NOT NULL DEFAULT 1,
    PRIMARY KEY (id),
    UNIQUE INDEX (address, forwarding),
    INDEX (domain),
    INDEX (is_list),
    INDEX (is_alias)
) ENGINE=InnoDB;

Migrate mail accounts

Please download script used to migrate mail accounts, and run it directly:

cd /root/
wget https://bitbucket.org/zhb/iredmail/raw/default/iRedMail/tools/migrate_sql_alias_table.py
python migrate_sql_alias_table.py

Note: It will try to read iRedAdmin config file from one of paths below, and connects to SQL server as user vmailadmin:

/opt/www/iredadmin/settings.py (Debian/Ubuntu)
/var/www/iredadmin/settings.py (CentOS/OpenBSD)
/usr/share/apache2/iredadmin/settings.py (Debian/Ubuntu with old iRedMail releases)
/usr/local/www/iredadmin/settings.py (FreeBSD)

Update Postfix config files

Please run shell commands below to tell Postfix to use new SQL tables.

Notes: on FreeBSD, the path is /usr/local/etc/postfix/mysql.

cd /etc/postfix/mysql/
perl -pi -e 's#alias\.address#forwardings.address#g' *.cf
perl -pi -e 's#alias\.goto#forwardings.forwarding#g' *.cf
perl -pi -e 's#alias\.active#forwardings.active#g' *.cf
perl -pi -e 's#alias\.domain#forwardings.domain#g' *.cf
perl -pi -e 's#alias,#forwardings,#g' *.cf

Restarting Postfix service is required.

[OPTIONAL] Drop unused SQL columns in `vmail.alias` table

Warning

Please upgrade iRedAPD and iRedAdmin-Pro also, they need the new SQL structure also.

After migration, few columns in vmail.alias table are not used anymore. it's ok to drop them. But it's strongly recommended to keep them for few more days until you can confirm all features are working as expected.

Please connect to MySQL server as MySQL root user, then execute SQL commands below:

USE vmail;

-- Remove non-mail-alias account
DELETE FROM alias WHERE islist <> 1;

-- per-domain catch-all account
DELETE FROM alias WHERE address=domain;

-- Drop unused columns
ALTER TABLE alias DROP COLUMN goto;
ALTER TABLE alias DROP COLUMN moderators;
ALTER TABLE alias DROP COLUMN islist;
ALTER TABLE alias DROP COLUMN is_alias;
ALTER TABLE alias DROP COLUMN alias_to;

Fixed: Avoid possible backdooring mysqldump backups

For more details about this backdooring mysqldump backup issue, please read blog post:

[CVE-2016-5483] Backdooring mysqldump backups.

Steps to fix it:

Open the daily MySQL backup script, it's /var/vmail/backup/backup_mysql.sh by default. if you use different storage directory during iRedMail installation, you can find the base directory with command postconf virtual_mailbox_base.
Find variable name CMD_MYSQLDUMP like below:

export CMD_MYSQLDUMP="mysqldump ..."

Make sure it has argument --skip-comments like below:

export CMD_MYSQLDUMP="mysqldump ... --skip-comments"

Save your change. That's it.

PostgreSQL backend specific

SQL structure change in `vmail.alias` SQL table

We've made some changes to vmail.alias SQL table for easier account management, you can find details about this change here: SQL structure changes in vmail.alias table.

This change introduces 2 new SQL tables (forwardings, alias_moderators), and (optionally) dropped few columns in vmail.alias table.

iRedAPD and iRedAdmin (and iRedAdmin-Pro) have been upgraded to use this new SQL structure.

Create required new SQL tables

Please connect to PostgreSQL server as vmailadmin user, then execute SQL commands below to create required new tables:

su - postgres
psql -U vmailadmin -d vmail

CREATE TABLE forwardings (
    id SERIAL PRIMARY KEY,
    address VARCHAR(255) NOT NULL DEFAULT '',
    forwarding VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT '',
    -- defines whether it's a standalone mail alias account. 0=no, 1=yes.
    is_list INT2 NOT NULL DEFAULT 0,
    -- defines whether it's a mail forwarding address of mail user. 0=no, 1=yes.
    is_forwarding INT2 NOT NULL DEFAULT 0,
    -- defines whether it's a per-account alias address. 0=no, 1=yes.
    is_alias INT2 NOT NULL DEFAULT 0,
    active INT2 NOT NULL DEFAULT 1
);
CREATE INDEX idx_forwardings_address ON forwardings (address);
CREATE INDEX idx_forwardings_forwarding ON forwardings (forwarding);
CREATE UNIQUE INDEX idx_forwardings_address_forwarding ON forwardings (address, forwarding);
CREATE INDEX idx_forwardings_domain ON forwardings (domain);
CREATE INDEX idx_forwardings_is_list ON forwardings (is_list);
CREATE INDEX idx_forwardings_is_forwarding ON forwardings (is_forwarding);
CREATE INDEX idx_forwardings_is_alias ON forwardings (is_alias);

CREATE TABLE alias_moderators (
    id SERIAL PRIMARY KEY,
    address VARCHAR(255) NOT NULL DEFAULT '',
    moderator VARCHAR(255) NOT NULL DEFAULT '',
    domain VARCHAR(255) NOT NULL DEFAULT ''
);
CREATE INDEX idx_alias_moderators_address ON alias_moderators (address);
CREATE INDEX idx_alias_moderators_moderator ON alias_moderators (moderator);
CREATE INDEX idx_alias_moderators_domain ON alias_moderators (domain);
CREATE UNIQUE INDEX idx_alias_moderators_address_moderator ON alias_moderators (address, moderator);

-- Grant required privilege to vmail user
GRANT SELECT ON TABLE forwardings to vmail;
GRANT SELECT ON TABLE alias_moderators to vmail;

Migrate mail accounts

Please download script used to migrate mail accounts, and run it directly:

cd /root/
wget https://bitbucket.org/zhb/iredmail/raw/default/iRedMail/tools/migrate_sql_alias_table.py
python migrate_sql_alias_table.py

Note: It will try to read iRedAdmin config file from one of paths below, and connects to SQL server as user vmailadmin:

/opt/www/iredadmin/settings.py (Debian/Ubuntu)
/var/www/iredadmin/settings.py (CentOS/OpenBSD)
/usr/share/apache2/iredadmin/settings.py (Debian/Ubuntu with old iRedMail releases)
/usr/local/www/iredadmin/settings.py (FreeBSD)

Update Postfix config files

Please run shell commands below to tell Postfix to use new SQL tables.

Notes: on FreeBSD, the path is /usr/local/etc/postfix/pgsql.

cd /etc/postfix/pgsql/
perl -pi -e 's#alias\.address#forwardings.address#g' *.cf
perl -pi -e 's#alias\.goto#forwardings.forwarding#g' *.cf
perl -pi -e 's#alias\.active#forwardings.active#g' *.cf
perl -pi -e 's#alias\.domain#forwardings.domain#g' *.cf
perl -pi -e 's#alias,#forwardings,#g' *.cf

Restarting Postfix service is required.

[OPTIONAL] Drop unused SQL columns in `vmail.alias` table

Warning

Please upgrade iRedAPD and iRedAdmin-Pro also, they need the new SQL structure also.

After migration, few columns in vmail.alias table are not used anymore, it's ok to drop them. But it's strongly recommended to keep them for few more days until you can confirm all features are working as expected.

su - postgres
psql -U vmailadmin -d vmail

-- Remove non-mail-alias account
DELETE FROM alias WHERE islist <> 1;

-- per-domain catch-all account
DELETE FROM alias WHERE address=domain;

-- Drop unused columns
ALTER TABLE alias DROP COLUMN goto;
ALTER TABLE alias DROP COLUMN moderators;
ALTER TABLE alias DROP COLUMN islist;
ALTER TABLE alias DROP COLUMN is_alias;
ALTER TABLE alias DROP COLUMN alias_to;

Upgrade iRedMail from 0.9.6 to 0.9.7

ChangeLog

General (All backends should apply these steps)

Update /etc/iredmail-release with new iRedMail version number

Upgrade Roundcube webmail to the latest stable release (1.2.5)

Fixed: improper order of Postfix HELO restriction rules.

Fixed: incorrect owner and permission for rotated Dovecot log files

Fixed: incorrect session.save_path in php-fpm pool config file on RHEL/CentOS

Fixed: incorrect freshclam setting UpdateLogFile

Fail2ban: fixes an improper filter and add new filter rule

NEW: New backup script for SOGo

OpenLDAP backend special

Fixed: Avoid possible backdooring mysqldump backups

MySQL/MariaDB backend special

SQL structure change in vmail.alias SQL table

Create required new SQL tables

Migrate mail accounts

Update Postfix config files

[OPTIONAL] Drop unused SQL columns in vmail.alias table

Fixed: Avoid possible backdooring mysqldump backups

PostgreSQL backend specific

SQL structure change in vmail.alias SQL table

Create required new SQL tables

Migrate mail accounts

Update Postfix config files

[OPTIONAL] Drop unused SQL columns in vmail.alias table

Update `/etc/iredmail-release` with new iRedMail version number

Fixed: incorrect freshclam setting `UpdateLogFile`

SQL structure change in `vmail.alias` SQL table

[OPTIONAL] Drop unused SQL columns in `vmail.alias` table

SQL structure change in `vmail.alias` SQL table

[OPTIONAL] Drop unused SQL columns in `vmail.alias` table