LINUX, PROXMOX, PVE, LXC, CONTAINER, POSTFIX, HOMELAB, MAIL, NOTIFICATION

Setting up Postfix Mail Relay with Mailgun on Proxmox LXC Container

I’ve been running various services in my homelab and needed a reliable way to get email notifications from different systems. Instead of dealing with SPF records, DKIM, and all the complexity of running a full mail server, I decided to set up a dedicated mail relay using Postfix with Mailgun on a lightweight LXC container.

This setup gives me a centralized mail relay that any service in my homelab can use to send notifications through Mailgun’s free tier, which is perfect for homelab use.

Overview

The setup consists of:

  • Proxmox VE host running the LXC container
  • Debian 12 LXC container (mailrelay - 192.168.7.10) with Postfix configured as SMTP relay
  • Mailgun as the external SMTP service for reliable delivery
  • Local services connecting to the relay for outbound mail
+---------------------------+
|     Proxmox VE Host       |
|                           |
|  +---------------------+  |
|  | LXC: mailrelay      |  |     +--------------+
|  | IP: 192.168.7.10    |  |     |              |
|  | Debian 12 + Postfix |  |---->|   Mailgun    |
|  +---------------------+  |     |   SMTP       |
|                           |     |              |
+---------------------------+     +--------------+
             ^
             |
    +----------------+
    | Homelab        |
    | Services       |
    | (monitoring,   |
    |  backups, etc.)|
    +----------------+

Prerequisites

  • Proxmox VE environment
  • Mailgun account (free tier works fine for homelab use)
  • Basic understanding of LXC containers and Postfix

Container Setup

First, I created a new LXC container in Proxmox with these specs:

  • Template: debian-12-standard
  • CPU: 1 core
  • RAM: 512 MB
  • Storage: 8 GB
  • Network: 192.168.7.10/24

Installation and Configuration

Update the container and install required packages: 

apt update && apt upgrade -y
apt install postfix libsasl2-modules swaks -y

Package explanations:

  • postfix - The mail transfer agent (MTA)
  • libsasl2-modules - SASL authentication modules for Mailgun
  • swaks - SMTP test tool (Swiss Army Knife for SMTP) - useful for testing

During the Postfix installation, choose “Internet Site” and set the system mail name to something like mailrelay.local.

Main Postfix Configuration

I created a clean main.cf configuration in /etc/postfix/main.cf:

# ============================================================================
# Postfix Mail Relay Configuration with Mailgun
# Security-hardened configuration for homelab use
# ============================================================================

# ---------------------------------------------------------------------------
# BASIC SETTINGS
# ---------------------------------------------------------------------------
smtpd_banner = $myhostname ESMTP
biff = no
append_dot_mydomain = no
readme_directory = no
compatibility_level = 3.6

# ---------------------------------------------------------------------------
# TLS CERTIFICATES - INCOMING CONNECTIONS
# 
# WARNING: Self-signed certificates (snakeoil) are suitable for testing only!
# For production use, obtain certificates from:
#   - Let's Encrypt (free, automated): https://letsencrypt.org
#   - Internal PKI/CA (for corporate environments)
#   - Commercial certificate authorities
# 
# Self-signed certs will trigger warnings on clients but work for internal use.
# ---------------------------------------------------------------------------
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key

# Require TLS for incoming connections (optional - may break old clients)
# Set to 'may' if you have legacy clients that don't support TLS
smtpd_tls_security_level = may

# ---------------------------------------------------------------------------
# TLS FOR OUTBOUND CONNECTIONS (Your Server -> Mailgun)
# 
# CRITICAL: Always enforce TLS to protect Mailgun credentials in transit.
# 'encrypt' prevents fallback to plaintext authentication.
# ---------------------------------------------------------------------------
smtp_tls_security_level = encrypt
smtp_tls_CApath = /etc/ssl/certs
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# Verify remote certificates to prevent MITM attacks
smtp_tls_verify_cert_match = hostname, nexthop

# ---------------------------------------------------------------------------
# NETWORK CONFIGURATION
# ---------------------------------------------------------------------------
myhostname = mailrelay.local
mydestination = $myhostname, localhost.localdomain, localhost
relayhost = [smtp.mailgun.org]:587

# Allowed client networks - ADJUST THESE FOR YOUR ENVIRONMENT!
# Only hosts in these networks can relay mail through this server.
# Examples below - replace with your actual internal networks:
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 \
    192.168.7.0/24 \
    10.0.0.0/24 \
    10.0.0.0/24

inet_interfaces = all
inet_protocols = ipv4  # Change to 'all' if you need IPv6 support

# ---------------------------------------------------------------------------
# MAILGUN SASL AUTHENTICATION
# ---------------------------------------------------------------------------
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous, noplaintext
smtp_sasl_tls_security_options = noanonymous
smtp_sasl_mechanism_filter = plain, login

# ---------------------------------------------------------------------------
# RATE LIMITING (Prevent abuse and resource exhaustion)
# ---------------------------------------------------------------------------
smtpd_client_connection_rate_limit = 30
smtpd_client_message_rate_limit = 20
anvil_rate_time_unit = 60s

# ---------------------------------------------------------------------------
# SIZE LIMITS
# ---------------------------------------------------------------------------
mailbox_size_limit = 0
message_size_limit = 52428800  # 50MB max message size
recipient_delimiter = +

# ---------------------------------------------------------------------------
# SECURITY HARDENING
# ---------------------------------------------------------------------------

# Disable VRFY command (prevents email address harvesting by spammers)
disable_vrfy_command = yes

# Require HELO/EHLO command (prevents some automated spam)
smtpd_helo_required = yes

# Delay rejections until after RCPT TO (prevents information leakage)
smtpd_delay_reject = yes

# ---------------------------------------------------------------------------
# ACCESS RESTRICTIONS (Control who can send mail)
# ---------------------------------------------------------------------------

# Relay restrictions: Who can use this server to send mail
smtpd_relay_restrictions = 
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_unauth_destination

# Recipient restrictions: Validate recipients
smtpd_recipient_restrictions = 
    permit_mynetworks,
    permit_sasl_authenticated,
    reject_unauth_destination,
    reject_unknown_recipient_domain,
    reject_non_fqdn_recipient

# Sender restrictions: Prevent spoofing
smtpd_sender_restrictions = 
    permit_mynetworks,
    reject_non_fqdn_sender,
    reject_unknown_sender_domain

# HELO restrictions: Validate client hostnames
smtpd_helo_restrictions = 
    permit_mynetworks,
    reject_invalid_helo_hostname,
    reject_non_fqdn_helo_hostname

# ---------------------------------------------------------------------------
# ALIASES
# ---------------------------------------------------------------------------
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases

Enable Port 587 (Submission)

By default, Postfix only listens on port 25. To enable port 587 for secure mail submission, edit /etc/postfix/master.cf:

# Open master.cf
nano /etc/postfix/master.cf

Find and uncomment (or modify) the submission line. It should look like this:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_tls_wrappermode=no
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_sasl_security_options=noanonymous
  -o smtpd_sasl_local_domain=$myhostname
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Note: Since we’re using mynetworks for authorization (not SASL), a simpler approach is to just uncomment the basic submission line:

submission inet n       -       y       -       -       smtpd

This enables port 587 with the same restrictions as port 25 (based on your mynetworks configuration).

Restart Postfix after making changes:

systemctl restart postfix

Verify both ports are listening:

ss -tlnp | grep -E ':25|:587'

Security Considerations

Important Security Warnings:

  • Self-Signed Certificates: The snakeoil certificates are pre-generated self-signed certs included with Debian for testing. They provide encryption but no authentication - clients will see certificate warnings. For production, replace with certificates from Let’s Encrypt or your internal CA.
  • mynetworks Configuration: The mynetworks parameter defines which IP addresses can relay mail through your server. Never include public internet IPs here unless you want to run an open relay (which will get abused by spammers within hours). Only include your trusted internal networks.
  • TLS Enforcement: We set smtp_tls_security_level = encrypt to force TLS when connecting to Mailgun. This prevents your credentials from being sent in plaintext if TLS negotiation fails. If Mailgun TLS breaks, mail will queue rather than send insecurely.
  • Rate Limiting: The configuration includes rate limiting (30 connections and 20 messages per minute per client) to prevent accidental mail loops or abuse from compromised clients.
  • SASL Security: We disable plaintext authentication except over TLS (noplaintext option) and require non-anonymous mechanisms.

Understanding the Configuration

Key differences between smtpd_* and smtp_* settings:

  • smtpd_* = Settings for incoming connections (clients connecting TO your server)
    • smtpd_tls_*: TLS for clients connecting to your relay
    • smtpd_relay_restrictions: Who can relay through you
  • smtp_* = Settings for outbound connections (your server connecting TO Mailgun)
    • smtp_tls_*: TLS when connecting to Mailgun
    • smtp_sasl_*: Authentication to Mailgun

How the mail flow works:

  1. Your application → connects to your relay on port 587 (recommended) or port 25
  2. Postfix → requires STARTTLS (port 587) or offers it (port 25), your cert encrypts the connection
  3. Postfix → checks if client IP is in mynetworks (security check)
  4. Postfix → accepts mail and queues it
  5. Postfix → connects to Mailgun (smtp.mailgun.org:587) with TLS (using trusted CA certs)
  6. Postfix → authenticates using credentials from sasl_passwd
  7. Mailgun → delivers mail to final destination

Port Selection Guide:

I recommend using Port 587 (Submission) for all new setups:

  • Port 587 (Recommended) - “Submission” port
    • Requires STARTTLS encryption immediately
    • Modern standard for mail clients submitting mail
    • Better security posture - no plaintext fallback
    • Uses certificate for TLS encryption
    • Use this for all new applications
  • Port 25 (Legacy) - Standard SMTP
    • Accepts plaintext initially, upgrades to TLS via STARTTLS
    • Backwards compatible with older devices
    • Certificate used during STARTTLS upgrade
    • Use only if your application doesn’t support port 587

Why port 587 is better: Port 587 enforces immediate TLS negotiation, ensuring credentials and message content are always encrypted in transit. Port 25 allows plaintext connections initially, which could expose sensitive data if STARTTLS fails or is not supported by the client.

Self-signed certificates and port usage: Your self-signed certificate is used regardless of which port you choose. The certificate is presented during the TLS handshake (after STARTTLS on both ports). Clients connecting to either port 25 or 587 will see certificate warnings since the certificate isn’t trusted, but the traffic will be encrypted.

Mailgun Authentication Setup

Create the SASL password file with your Mailgun credentials:

# Create the password file
cat > /etc/postfix/sasl_passwd << 'EOF'
[smtp.mailgun.org]:587 postmaster@sandbox[YOUR_DOMAIN].mailgun.org:[YOUR_API_KEY]
EOF

# Generate hash database and set secure permissions
postmap /etc/postfix/sasl_passwd
chmod 600 /etc/postfix/sasl_passwd*
chown root:root /etc/postfix/sasl_passwd*

Note: After any change to sasl_passwd, you must run postmap to rebuild the hash database.

Configure Mail Aliases

Set up mail forwarding for system accounts:

echo "root: your-email@domain.com" >> /etc/aliases
newaliases

Restart and Test the Service 

# Verify configuration syntax
postfix check

# Restart and enable service
systemctl restart postfix
systemctl enable postfix
systemctl status postfix

Testing Mail Delivery

Test mail delivery on port 587 (recommended) or port 25:

# Test via port 587 (recommended - requires STARTTLS support)
swaks --to root --from test@mailrelay.local --server 192.168.7.10:587 --tls

# Test via port 25 (fallback option)
swaks --to root --from test@mailrelay.local --server 192.168.7.10:25 --tls

# Alternative using mail command (sends via port 25 by default)
# Note: Some mail implementations may not support STARTTLS
echo "Test from mailrelay container" | mail -s "Test Mail" root

# Test with specific recipient
echo "Test message body" | mail -s "Test Subject" -r "from@yourdomain.com" "recipient@example.com"

# Check queue status
mailq

# View recent logs
journalctl -u postfix -n 50 --no-pager

Installing swaks (recommended testing tool):

apt install swaks

The swaks tool is excellent for testing SMTP servers as it supports STARTTLS and shows detailed protocol information.

Testing TLS Configuration

Verify TLS works to Mailgun:

# Test TLS connection to Mailgun
openssl s_client -connect smtp.mailgun.org:587 -starttls smtp

# In the openssl prompt, type:
# EHLO test
# QUIT

You should see a certificate chain and 250 STARTTLS in the response.

Verification and Monitoring

To check if everything is working correctly:

# Check mail queue
mailq

# Watch mail logs in real-time
tail -f /var/log/mail.log

# Check for authentication errors
grep "SASL" /var/log/mail.log

# Check for TLS errors
grep "TLS" /var/log/mail.log

# View service status
systemctl status postfix

# Test configuration
postconf -n  # Shows non-default settings

Log entries to watch for:

  • status=sent - Mail delivered successfully
  • status=deferred - Temporary failure (will retry)
  • status=bounced - Permanent failure
  • SASL authentication failed - Check your Mailgun credentials

Integration with Other Services

Now that the mail relay is running, other services in my homelab can use it by pointing their SMTP configuration to your relay IP. This works particularly well with:

  • Proxmox backup notifications - Configure in Datacenter → Notification
  • Monitoring systems like Zabbix, Nagios, or Uptime Kuma
  • Application logs and alerts from Docker containers
  • Script-based notifications from backup jobs or cron
  • Network devices (switches, routers, firewalls that support SMTP alerts)

Recommended configuration (Port 587 with STARTTLS):

Use port 587 for all new applications. Most modern systems support it and it provides better security.

Benefits of This Setup

This approach gives me several advantages:

  • Centralized mail handling - all homelab mail goes through one point
  • Reliable delivery through Mailgun’s infrastructure and reputation
  • No SPF/DKIM complexity - Mailgun handles email authentication
  • Lightweight - minimal resource usage in an LXC container (~512MB RAM)
  • Easy troubleshooting - centralized logs and configuration
  • Security - credentials not stored on multiple systems
  • Rate limiting - built-in protection against mail floods

Troubleshooting

If you run into issues, check these common problems:

Authentication Failures

  • Symptom: SASL authentication failed in logs
  • Solution:
    • Verify Mailgun API key and domain in /etc/postfix/sasl_passwd
    • Ensure you ran postmap /etc/postfix/sasl_passwd after editing
    • Check file permissions: chmod 600 /etc/postfix/sasl_passwd*

TLS/SSL Errors

  • Symptom: SSL3_GET_RECORD:wrong version number or TLS handshake failures
  • Solution:
    • Verify libsasl2-modules is installed: apt install libsasl2-modules
    • Check if Mailgun TLS is working: openssl s_client -connect smtp.mailgun.org:587 -starttls smtp
    • Ensure system time is correct (TLS requires valid time)

Connection Timeouts

  • Symptom: Mail stuck in queue, connection timeouts
  • Solution:
    • Verify container has internet access: ping smtp.mailgun.org
    • Ensure port 587 is not blocked by firewall
    • Check DNS resolution: nslookup smtp.mailgun.org

Permission Errors

  • Symptom: warning: problem talking to server private/anvil
  • Solution:
    • Check that sasl_passwd files have correct ownership: chown root:root /etc/postfix/sasl_passwd*
    • Verify Postfix can read the files: ls -la /etc/postfix/sasl_passwd*

Mail Not Delivered (Deferred)

  • Symptom: status=deferred in logs, mail stays in queue
  • Solution:
    • Check full error in logs: tail -f /var/log/mail.log
    • Force immediate delivery attempt: postqueue -f
    • Check if destination domain has MX records: dig MX example.com

“Relay Access Denied” Errors

  • Symptom: Clients get 554 5.7.1 <recipient>: Relay access denied
  • Solution:
    • Verify client IP is in mynetworks in main.cf
    • Check client is connecting on correct interface
    • Review smtpd_relay_restrictions rules

The setup has been running smoothly in my homelab, and I’m getting reliable notifications from all my services. It’s a simple but effective solution that avoids the complexity of running a full mail server while still providing the functionality I need for homelab monitoring and alerting.