Part 2 - IMAP and TLS Setup

This is part 2 of building your own secure email server on Debian from scratch tutorial series. In previous part, we showed you how to set up a basic Postfix SMTP server. In this tutorial, we are going to configure the email server so that we can receive and send emails using a desktop email client like Mozilla Thunderbird or Microsoft Outlook.

To be able to send emails using a desktop email client, we need to enable the submission service in Postfix. To receive emails using a desktop email client, we can install an open-source IMAP server named Dovecot on the Debian server. And to encrypt our communications, we can install a free TLS certificate issued by Let’s Encrypt.

This part is focused only on Canonical users (use only local OS user as mailbox login)

Debian doesn’t enable firewall by default. If you have enabled the UFW firewall, then you need to run the following command to open email related ports in firewall.

sudo ufw allow 80,443,587,465,993/tcp

If you use POP3 to fetch emails (I personally don’t), then also open port 110 and 995, you may also open unsecure IMAP port 143 (I personnaly don't).

sudo ufw allow 110,143,995/tcp

When we configure our desktop email clients, It’s always a good idea to enable TLS encryption to prevent hackers from snooping on our emails. We can easily obtain a free TLS certificate from Let’s Encrypt. Issue the following commands to install Let’s Encrypt client (certbot) on Debian server from the default software repository.

sudo apt update
 
sudo apt dist-upgrade
 
sudo apt install certbot

If you don’t have a web server running yet, I recommend you install one (Apache or Nginx), because it’s easier to obtain and install TLS certificate with a web server than using other methods. And in a later tutorial, I will show you how to set up webmail, which requires running a web server.

If you choose to use Apache web server, you need to install the Apache plugin. (The following command will install Apache web server if it’s not already installed on your system.)

sudo apt install python3-certbot-apache

If you choose use Nginx web server, then install the Nginx plugin. (The following command will install Nginx web server if it’s not already installed on your system.)

sudo apt install python3-certbot-nginx

We create an Apache virtual host for mail.example.com before obtaining Let’s Encrypt TLS certificate. Create the virtual host file:

sudo nano /etc/apache2/sites-available/mail.example.com.conf

Then paste the following text into the file.

<VirtualHost *:80>        
        ServerName mail.example.com
 
        DocumentRoot /var/www/html/
</VirtualHost>

Save and close the file. Enable this virtual host.

sudo a2ensite mail.example.com.conf

Then disable the default virtual host, because it might interfere with other virtual hosts.

sudo a2dissite 000-default

Reload Apache for the changes to take effect.

sudo systemctl reload apache2

Once the virtual host is created and enabled, run the following command to obtain Let’s Encrypt TLS certificate.

sudo certbot certonly -a apache --agree-tos --no-eff-email --staple-ocsp --email you@example.com -d mail.example.com

Where:

Substitute the “example.com” fields with your actual data. You should see the following which means the certificate is successfully obtained. You can also see the directory under which your cert is stored.

We create an Nginx virtual host for mail.example.com before obtaining Let’s Encrypt TLS certificate. Create the virtual host file:

sudo nano /etc/nginx/conf.d/mail.example.com.conf

Next, paste the following text into the file.

server {
      listen 80;
      listen [::]:80;
      server_name mail.example.com;
 
      root /usr/share/nginx/html/;
 
      location ~ /.well-known/acme-challenge {
         allow all;
      }
}

Save and close the file. Make sure the /usr/share/nginx/html/ directory exists on your server.

sudo mkdir -p /usr/share/nginx/html/

Reload Nginx for the changes to take effect.

sudo systemctl reload nginx

Once the virtual host is created and enabled, run the following command to obtain Let’s Encrypt certificate with Nginx plugin.

sudo certbot certonly -a nginx --agree-tos --no-eff-email --staple-ocsp --email you@example.com -d mail.example.com

Where:

You should see the following which means the certificate is successfully obtained. You can also see the directory under which your cert is stored.

To send emails from a desktop email client, we need to enable the submission service of Postfix so that the email client can submit emails to Postfix SMTP server. Edit the master.cf file.

sudo nano /etc/postfix/master.cf

In submission section, uncomment or add the following lines. Please allow at least one whitespace (tab or spacebar) before each -o. In postfix configurations, a preceding whitespace character means that this line is continuation of the previous line. (By default the submission section is commented out. You can copy the following lines and paste them into the file, so you don’t have to manually uncomment or add new text.)

submission     inet     n    -    y    -    -    smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_tls_wrappermode=no
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_relay_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth

Some explainations

submission     inet     n    -    y    -    -    smtpd
  :
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  :

The above configuration enables the submission daemon of Postfix and requires TLS encryption. So later on our desktop email client can connect to the submission daemon in TLS encryption. The submission daemon listens on TCP port 587. STARTTLS is used to encrypt communications between email client and the submission daemon.

Microsoft Outlook mail client only supports submission over port 465. If you are going to use Microsoft Outlook, then you also need to enable submission service on port 465 by adding the following lines in the file.

smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth

Save and close the file.

Next, we need to specify the location of TLS certificate and private key in Postfix configuration file. Edit main.cf file.

sudo nano /etc/postfix/main.cf

Edit the TLS parameter as follows. Remember to replace mail.example.com with your real hostname.

# TLS parameters
# Let's Encrypt certificate
smtpd_tls_cert_file=/etc/letsencrypt/live/gallifrey.nox-rhea.org/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/gallifrey.nox-rhea.org/privkey.pem
smtpd_tls_CAfile=/etc/letsencrypt/live/gallifrey.nox-rhea.org/chain.pem
 
smtpd_tls_security_level=may 
smtpd_tls_loglevel = 1
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
 
#Enable TLS Encryption when Postfix sends outgoing emails
smtp_tls_security_level = may
smtp_tls_loglevel = 1
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
 
#Enforce TLSv1.3 or TLSv1.2
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1

Your Let’s Encrypt certificate and private key are stored under /etc/letsencrypt/live/mail.example.com/ directory.

Save and close the file. Then restart Postfix.

sudo systemctl restart postfix

If you run the following command, you will see Postfix is now listening on port 587 and 465.

sudo ss -lnpt | grep master

Enter the following command to install Dovecot core package and the IMAP daemon package on Debian server.

sudo apt install dovecot-core dovecot-imapd

If you use POP3 to fetch emails, then also install the dovecot-pop3d package.

sudo apt install dovecot-pop3d

Check Dovecot version:

sudo dovecot --version

Sample output:

2.3.13 (89f716dc2)

By default, Postfix will forward mail from clients in authorized network blocks to any destination.

The current default is to authorize the local machine only.

Prior to Postfix 3.0, the default was to authorize all clients in the IP subnetworks that the local machine is attached to.

Sometime you need to “allow” some specific IP (or subnet) to be considered as “trusted”, for receiving email “from” you.

the correct parameter is

mynetworks = [list of IPs separated by comma]

Edit the main config file.

sudo nano /etc/dovecot/dovecot.conf

Add the following line to enable IMAP protocol.

protocols = imap

If you use POP3 to fetch emails, then also add POP3 protocol.

protocols = imap pop3

Save and close the file.

By default, Postfix and Dovecot use mbox format to store emails. Each user’s emails are stored in a single file /var/mail/username. You can run the following command to find the mail spool directory.

sudo postconf mail_spool_directory

Sample output:

mail_spool_directory = /var/mail

However, nowadays it’s almost always you want to use the Maildir format to store email messages. The config file for mailbox location is /etc/dovecot/conf.d/10-mail.conf.

sudo nano /etc/dovecot/conf.d/10-mail.conf

The default configuration uses mbox mail format.

mail_location = mbox:~/mail:INBOX=/var/mail/%u

Change it to the following to make Dovecot use the Maildir format. Email messages will be stored under the Maildir directory under each user’s home directory.

mail_location = maildir:~/Maildir

Save and close the file. Then add dovecot to the mail group so that Dovecot can read the INBOX.

sudo adduser dovecot mail

Although we configured Dovecot to store emails in Maildir format, by default, Postfix uses its built-in local delivery agent (LDA) to move inbound emails to the message store (inbox, sent, trash, Junk, etc), and it will be saved in mbox format.

We need to configure Postfix to pass incoming emails to Dovecot, via the LMTP protocol, which is a simplified version of SMTP, so incoming emails will saved in Maildir format by Dovecot. LMTP allows for a highly scalable and reliable mail system. It also allows us to use the sieve plugin to filter inbound messages to different folders.

sudo apt install dovecot-lmtpd

Edit the Dovecot main configuration file.

sudo nano /etc/dovecot/dovecot.conf

Add lmtp to the supported protocols.

protocols = imap lmtp

Save and close the file. Then edit the Dovecot 10-master.conf file.

sudo nano /etc/dovecot/conf.d/10-master.conf

Change the lmtp service definition to the following. Be careful about the syntax. Each opening bracket needs to be paired with a closing bracket.

service lmtp {
 unix_listener /var/spool/postfix/private/dovecot-lmtp {
   mode = 0600
   user = postfix
   group = postfix
  }
}

Save and close the file.

Next, edit the Postfix main configuration file.

sudo nano /etc/postfix/main.cf

Add the following lines at the end of the file. The first line tells Postfix to deliver incoming emails to local message store via the Dovecot LMTP server. The second line disables SMTPUTF8 in Postfix, because Dovecot-LMTP doesn’t support this email extension.

mailbox_transport = lmtp:unix:private/dovecot-lmtp
smtputf8_enable = no

Save and close the file.

Edit the authentication config file.

sudo nano /etc/dovecot/conf.d/10-auth.conf

Uncomment the following line.

disable_plaintext_auth = yes

It will disable plaintext authentication when there’s no SSL/TLS encryption. Then find the following line,

#auth_username_format = %Lu

Uncomment it and change its value to %n.

auth_username_format = %n

By default, when Dovecot tries to find or deliver emails for a user, it uses the full email address. Since in this part, we only set up canonical mailbox users (using OS users as mailbox users), Dovecot can’t find the mailbox user in full domain format (username@example.com), so we need to set auth_username_format = %n to drop the domain part, then Dovecot should be able to find the mailbox user. This also allows us to use the full email address (username@example.com) to log in.

Next, find the following line.

auth_mechanisms = plain

This line only enables the PLAIN authentication mechanism. LOGIN is another authentication mechanism you probably want to add to support older email clients.

auth_mechanisms = plain login

Save and close the file.

Edit SSL/TLS config file.

sudo nano /etc/dovecot/conf.d/10-ssl.conf

Change ssl = yes to ssl = required to enforce encryption.

ssl = required

Then find the following lines.

ssl_cert = </etc/dovecot/private/dovecot.pem
ssl_key = </etc/dovecot/private/dovecot.key

By default, Dovecot uses a self-signed TLS certificate. Replace them with the following values, which specify the location of your Let’s Encrypt TLS certificate and private key. Don’t leave out the < character. It’s necessary.

ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem

Find the following line.

#ssl_prefer_server_ciphers = no

It’s a good practice to prefer the server’s order of ciphers over client’s. So uncomment this line and change the value to yes.

ssl_prefer_server_ciphers = yes

Then find the following line.

#ssl_min_protocol = TLSv1

Change it to the following to disable insecure SSLv3, TLSv1, and TLSv1.1 protocols.

ssl_min_protocol = TLSv1.2

Save and close the file.

Edit the following file.

sudo nano /etc/dovecot/conf.d/10-master.conf

Change service auth section to the following so that Postfix can find the Dovecot authentication server. Please be careful about the syntax. Every opening bracket should be terminated by a closing bracket.

service auth {
    unix_listener /var/spool/postfix/private/auth {
      mode = 0660
      user = postfix
      group = postfix
    }
}

Save and close the file.

After you save and close all the above config files, restart Postfix and Dovecot.

sudo systemctl restart postfix dovecot

Dovecot will be listening on port 143 (IMAP) and 993 (IMAPS), as can be seen with:

sudo ss -lnpt | grep dovecot

If there’s a configuration error, dovecot will fail to restart, so it’s a good idea to check if Dovecot is running with the following command.

sudo systemctl status dovecot

Now open up your desktop email client such as Mozilla Thunderbird. Go to Edit → Account Settings → Account Actions → Add Mail Account to add a mail account.

In the incoming server section, select IMAP protocol, enter mail.your-domain.com as the server name, choose port 143 and STARTTLS. Choose normal password as the authentication method. In the outgoing section, select SMTP protocol, enter mail.your-domain.com as the server name, choose port 587 and STARTTLS. Choose normal password as the authentication method.

debian-postfix-dovecot-letsencrypt

You should now be able to connect to your own email server and also send and receive emails with your desktop email client!

We use local Unix accounts as email addresses, as we did in previous part. For example, if you have a user named user1 on your Debian server, then you have an email address: user1@example.com, and the password for the email address is the same password for the user1 user. To create a local Unix account, run

sudo adduser user1

Note: Dovecot doesn’t allow you to log in with the root account. You need to create separate user accounts.

You can list all available mailbox users with:

sudo doveadm user '*'

It’s recommended to restart Dovecot after adding users, so Dovecot can recognize new mailbox users.

sudo systemctl restart dovecot

As a rule of thumb, you should always check the mail log (/var/log/mail.log) on your mail server when an error happens. The following is a list of specific errors and troubleshooting tips.

If you can’t log into your mail server from a desktop mail client, scan your mail server to find if the ports (TCP 587, 465, 143, and 993) are open. Note that you should run the following command from another Linux computer or server. If you run it on your mail server, then the ports will always appear to be open.

sudo nmap mail.example.com

And check if Dovecot is running.

systemctl status dovecot

You can also check the mail log (/var/log/mail.log), which may give you some clues. If Dovecot fails to start, the error might not be logged to the /var/log/mail.log file, you can run the following command to see what’s wrong.

sudo journalctl -eu dovecot

For example, some folks may have the following error in the journal.

doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/10-master.conf line 78: Unknown setting

Most of the time, it’s a simple syntax error, like a missing curly bracket. Open the configuration file, go to the specified line and fix the error.

If you find the following error message in the mail log

imap-login: Error: Failed to initialize SSL server context: Can't load DH parameters: error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small

Then open the Dovecot TLS configuration file.

sudo nano /etc/dovecot/conf.d/10-ssl.conf

Add the following line in this file.

ssl_dh = </etc/dovecot/dh.pem

Save and close the file. Then generate the DH parameter file with:

sudo openssl dhparam -out /etc/dovecot/dh.pem 4096

Restart Dovecot for the changes to take effect.

As I said in previous part, if you use Cloudflare DNS service, you should not enable the CDN (proxy) feature when creating DNS A record and AAAA record for the hostname of your mail server. Cloudflare doesn’t support SMTP or IMAP proxy.

If you see the “relay access denied” error when trying to send emails from a mail client, it’s most likely that you use port 25 as the SMTP port in your mail client. As I said a while ago, you should use port 587 or 465 as the SMTP port in mail clients (Mozilla Thunberbird, Microsoft Outlook, etc) to submit outgoing emails. Port 25 should be used for SMTP server to SMTP server communications.

postfix_dovecot_relay_access_denied

If you see the following “relay access denied” error in the /var/log/mail.log file when trying to send emails from other mail services like Gmail to your own mail server, it’s likely that yourdomain.com is not in the list of $mydestination parameter.

NOQUEUE: reject: RCPT from mail-il1-f180.google.com[209.85.166.180]: 454 4.7.1 <xiao@inuxbabe.com>: Relay access denied; from=<someone@gmail.com> to=<frater@example.com> proto=ESMTP helo=<mail-il1-f180.google.com>

You can display the current value of $mydestination with:

postconf mydestination

Some folks might not have the main domain name in the list like so:

mydestination = $myhostname, localhost.$mydomain, localhost

Then run the following command to add the main domain name to the list.

sudo postconf -e "mydestination = example.com, \$myhostname, localhost.\$mydomain, localhost"

Reload Postfix for the changes to take effect.

sudo systemctl reload postfix

If you see the following error message in the mail log (/var/log/mail.log), it’s likely that you forgot to set auth_username_format = %n In /etc/dovecot/conf.d/10-auth.conf file.

mail postfix/lmtp[2256]: 68E00FC1A5: to=, relay=mail.example.com[private/dovecot-lmtp], delay=509, delays=509/0.03/0.03/0.02, dsn=5.1.1, status=bounced (host mail.example.com[private/dovecot-lmtp] said: 550 5.1.1  User doesn't exist: user1@example.com (in reply to RCPT TO command))

If you use the iOS Mail app to log into your mail server and encounter the following error.

You can try to fix it by enforcing SSL encryption, for both SMTP and IMAP.

If you encounter the “No password provided” error in the iOS Mail app, it’s likely that you have a typo when entering the username in the Mail account settings, or you didn’t enable SSL in the Mail account settings.

If you can’t receive emails from Gmail, Hotmail, Yahoo Mail, etc, here are the possible causes:

1. Your MX record is wrong, or not propagated to the Internet yet.
2. Your mail server hostname doesn’t have DNS A record, or not propagated to the Internet yet.
3. Your firewall doesn’t allow incoming connection to port 25. Maybe your mail server is behind a NAT?
4. Postfix isn’t listening on the public IP address.
5. Check the mail log (/var/log/mail.log) to find out if there are other errors in your Postfix and Dovecot configuration.

You can use the Network Tools Email Checker to test if your SMTP server is reachable from the Internet. Just enter your domain email address and click the Go button. As you can see from the screenshot below, it successfully found my domain’s MX record and my SMTP server is reachable from the Internet.

email_checker

If your SMTP servers isn’t reachable from the Internet, then you have a problem in the first 4 items. If your SMTP server is reachable from the Internet, but you still can’t receive emails, check the mail log (/var/log/mail.log) to find out if there is any errors in your Postfix and Dovecot configuration.

You can create Cron job to automatically renew TLS certificate. Simply open root user’s crontab file.

sudo crontab -e

If you use Apache web server, add the following line at the bottom of the file.

@daily certbot renew --quiet && systemctl reload postfix dovecot apache2

If you are using Nginx web server, then add the following line.

@daily certbot renew --quiet && systemctl reload postfix dovecot nginx

Reloading Postfix, Dovecot and the web server is necessary to make these programs pick up the new certificate and private key.

If for any reason your Dovecot process is killed, you need to run the following command to restart it.

sudo systemctl restart dovecot

Instead of manually typing this command, we can make Dovecot automatically restart by editing the dovecot.service systemd service unit. To override the default systemd service configuration, we create a separate directory.

sudo mkdir -p /etc/systemd/system/dovecot.service.d/

Then create a file under this directory.

sudo nano /etc/systemd/system/dovecot.service.d/restart.conf

Add the following lines in the file, which will make Dovecot automatically restart 5 seconds after a failure is detected.

[Service]
Restart=always
RestartSec=5s

Save and close the file. Then reload systemd for the changes to take effect.

sudo systemctl daemon-reload

To check if this would work, kill Dovecot with:

sudo pkill dovecot

Then check Dovecot status. You will find Dovecot automatically restarted.

systemctl status dovecot