Postfix SMTP Access Policy Delegation

Purpose of Postfix SMTP access policy delegation

The Postfix SMTP server has a number of built-in mechanisms to block or accept mail at specific SMTP protocol stages. In addition, the Postfix SMTP server can delegate decisions to an external policy server (Postfix 2.1 and later).

With this policy delegation mechanism, a simple greylist policy can be implemented with only a dozen lines of Perl, as is shown at the end of this document. A complete example can be found in the Postfix source code, in the directory examples/smtpd-policy.

Another example of policy delegation is the SPF policy server at http://www.open-spf.org/Software/

Policy delegation is now the preferred method for adding policies to Postfix. It's much easier to develop a new feature in few lines of Perl, Python, Ruby, or TCL, than trying to do the same in C code. The difference in performance will be unnoticeable except in the most demanding environments. On active systems a policy daemon process is used multiple times, for up to $max_use incoming SMTP connections.

This document covers the following topics:

Protocol description

The Postfix policy delegation protocol is really simple. The client sends a request and the server sends a response. Unless there was an error, the server must not close the connection, so that the same connection can be used multiple times.

The client request is a sequence of name=value attributes separated by newline, and is terminated by an empty line. The server reply is one name=value attribute and it, too, is terminated by an empty line.

Here is an example of all the attributes that the Postfix SMTP server sends in a delegated SMTPD access policy request: 

Postfix version 2.1 and later:
request=smtpd_access_policy
protocol_state=RCPT
protocol_name=SMTP
helo_name=some.domain.tld
queue_id=8045F2AB23
sender=foo@bar.tld
recipient=bar@foo.tld
recipient_count=0
client_address=1.2.3.4
client_name=another.domain.tld
reverse_client_name=another.domain.tld
instance=123.456.7
Postfix version 2.2 and later:
sasl_method=plain
sasl_username=you
sasl_sender=
size=12345
ccert_subject=solaris9.porcupine.org
ccert_issuer=Wietse+20Venema
ccert_fingerprint=C2:9D:F4:87:71:73:73:D9:18:E7:C2:F3:C1:DA:6E:04
Postfix version 2.3 and later:
encryption_protocol=TLSv1/SSLv3
encryption_cipher=DHE-RSA-AES256-SHA
encryption_keysize=256
etrn_domain=
Postfix version 2.5 and later:
stress=
Postfix version 2.9 and later:
ccert_pubkey_fingerprint=68:B3:29:DA:98:93:E3:40:99:C7:D8:AD:5C:B9:C9:40
Postfix version 3.0 and later:
client_port=1234
Postfix version 3.1 and later:
policy_context=submission
Postfix version 3.2 and later:
server_address=10.3.2.1
server_port=54321
Postfix version 3.8 and later:
compatibility_level=major.minor.patch
mail_version=3.8.0
[empty line]

Notes:

The following is specific to SMTPD delegated policy requests:

The policy server replies with any action that is allowed in a Postfix SMTPD access(5) table. Example: 

action=defer_if_permit Service temporarily unavailable
[empty line]

This causes the Postfix SMTP server to reject the request with a 450 temporary error code and with text "Service temporarily unavailable", if the Postfix SMTP server finds no reason to reject the request permanently.

In case of trouble the policy server must not send a reply. Instead the server must log a warning and disconnect. Postfix will retry the request at some later time.

Simple policy client/server configuration

The Postfix delegated policy client can connect to a TCP socket or to a UNIX-domain socket. Examples: 

inet:127.0.0.1:9998
unix:/some/where/policy
unix:private/policy

The first example specifies that the policy server listens on a TCP socket at 127.0.0.1 port 9998. The second example specifies an absolute pathname of a UNIX-domain socket. The third example specifies a pathname relative to the Postfix queue directory; use this for policy servers that are spawned by the Postfix master daemon. On many systems, "local" is a synonym for "unix".

To create a policy service that listens on a UNIX-domain socket called "policy", and that runs under control of the Postfix spawn(8) daemon, you would use something like this: 

 1 /etc/postfix/master.cf:
 2     policy  unix  -       n       n       -       0       spawn
 3       user=nobody argv=/some/where/policy-server
 4 
 5 /etc/postfix/main.cf:
 6     smtpd_recipient_restrictions =
 7         ... 
 8         reject_unauth_destination 
 9         check_policy_service unix:private/policy 
10         ...
11     policy_time_limit = 3600
12     # smtpd_policy_service_request_limit = 1

NOTES: 

 1 /etc/postfix/master.cf:
 2     127.0.0.1:9998  inet  n       n       n       -       0       spawn
 3       user=nobody argv=/some/where/policy-server
 4 
 5 /etc/postfix/main.cf:
 6     smtpd_recipient_restrictions =
 7         ... 
 8         reject_unauth_destination 
 9         check_policy_service inet:127.0.0.1:9998
10         ...
11     127.0.0.1:9998_time_limit = 3600
12     # smtpd_policy_service_request_limit = 1

Configuration parameters that control the client side of the policy delegation protocol:

Configuration parameters that control the server side of the policy delegation protocol:

Advanced policy client configuration

The previous section lists a number of Postfix main.cf parameters that control time limits and other settings for all policy clients. This is sufficient for simple configurations. With more complex configurations it becomes desirable to have different settings per policy client. This is supported with Postfix 3.0 and later.

The following example shows a "non-critical" policy service with a short timeout, and with "DUNNO" as default action when the service is unvailable. The "DUNNO" action causes Postfix to ignore the result. 

1 /etc/postfix/main.cf:
2     mua_recipient_restrictions = 
3         ...
4         reject_unauth_destination
5         check_policy_service { inet:host:port, 
6             timeout=10s, default_action=DUNNO
7             policy_context=submission }
8         ...

Instead of a server endpoint, we now have a list enclosed in {}.

Inside the list, syntax is similar to what we already know from main.cf: items separated by space or comma. There is one difference: you must enclose a setting in parentheses, as in "{ name = value }", if you want to have space or comma within a value or around "=". This comes in handy when different policy servers require different default actions with different SMTP status codes or text: 

1 /etc/postfix/main.cf:
2     smtpd_recipient_restrictions = 
3         ...
4         reject_unauth_destination
5         check_policy_service {
6           inet:host:port1, 
7           { default_action = 451 4.3.5 See https://www.example.com/support1 }
8         } 
9         ...

Example: greylist policy server

Greylisting is a defense against junk email that is described at https://www.greylisting.org/. The idea was discussed on the postfix-users mailing list one year before it was popularized (alternative version).

The file examples/smtpd-policy/greylist.pl in the Postfix source tree implements a simplified greylist policy server. This server stores a time stamp for every (client, sender, recipient) triple. By default, mail is not accepted until a time stamp is more than 60 seconds old. This stops junk mail with randomly selected sender addresses, and mail that is sent through randomly selected open proxies. It also stops junk mail from spammers that change their IP address frequently.

Copy examples/smtpd-policy/greylist.pl to /usr/libexec/postfix or whatever location is appropriate for your system.

In the greylist.pl Perl script you need to specify the location of the greylist database file, and how long mail will be delayed before it is accepted. The default settings are: 

$database_name="/var/mta/greylist.db";
$greylist_delay=60;

The /var/mta directory (or whatever you choose) should be writable by "nobody", or by whatever username you configure below in master.cf for the policy service.

Example: 

# mkdir /var/mta
# chown nobody /var/mta

Note: DO NOT create the greylist database in a world-writable directory such as /tmp or /var/tmp, and DO NOT create the greylist database in a file system that may run out of space. Postfix can survive "out of space" conditions with the mail queue and with the mailbox store, but it cannot survive a corrupted greylist database. If the file becomes corrupted you may not be able to receive mail at all until you delete the file by hand.

The greylist.pl Perl script can be run under control by the Postfix master daemon. For example, to run the script as user "nobody", using a UNIX-domain socket that is accessible by Postfix processes only: 

 1 /etc/postfix/master.cf:
 2     greylist  unix  -       n       n       -       0       spawn
 3       user=nobody argv=/usr/bin/perl /usr/libexec/postfix/greylist.pl
 4 
 5 /etc/postfix/main.cf:
 6     greylist_time_limit = 3600
 7     smtpd_recipient_restrictions =
 8         ... 
 9         reject_unauth_destination 
10         check_policy_service unix:private/greylist
11         ...
12     # smtpd_policy_service_request_limit = 1

Notes:

With Solaris < 9, or Postfix < 2.10 on any Solaris version, use inet: style sockets instead of unix: style, as detailed in the "Policy client/server configuration" section above. 

 1 /etc/postfix/master.cf:
 2     127.0.0.1:9998  inet  n       n       n       -       0       spawn
 3       user=nobody argv=/usr/bin/perl /usr/libexec/postfix/greylist.pl
 4 
 5 /etc/postfix/main.cf:
 6     127.0.0.1:9998_time_limit = 3600
 7     smtpd_recipient_restrictions =
 8         ... 
 9         reject_unauth_destination 
10         check_policy_service inet:127.0.0.1:9998
11         ...
12     # smtpd_policy_service_request_limit = 1

Greylisting mail from frequently forged domains

It is relatively safe to turn on greylisting for specific domains that often appear in forged email. At some point in cyberspace/time a list of frequently forged MAIL FROM domains could be found at https://web.archive.org/web/20080526153208/http://www.monkeys.com/anti-spam/filtering/sender-domain-validate.in. 

 1 /etc/postfix/main.cf:
 2     smtpd_recipient_restrictions =
 3         reject_unlisted_recipient
 4         ...
 5         reject_unauth_destination 
 6         check_sender_access hash:/etc/postfix/sender_access
 7         ...
 8     smtpd_restriction_classes = greylist
 9     greylist = check_policy_service unix:private/greylist
10 
11 /etc/postfix/sender_access:
12     aol.com     greylist
13     hotmail.com greylist
14     bigfoot.com greylist
15     ... etcetera ...

NOTES:

Greylisting all your mail

If you turn on greylisting for all mail you may want to make exceptions for mailing lists that use one-time sender addresses, because each message will be delayed due to greylisting, and the one-time sender addresses can pollute your greylist database relatively quickly. Instead of making exceptions, you can automatically allowlist clients that survive greylisting repeatedly; this avoids most of the delays and most of the database pollution problem. 

 1 /etc/postfix/main.cf:
 2     smtpd_recipient_restrictions =
 3         reject_unlisted_recipient
 4         ...
 5         reject_unauth_destination 
 6         check_sender_access hash:/etc/postfix/sender_access
 7         check_policy_service unix:private/policy
 8         ...
 9 
10 /etc/postfix/sender_access:
11     securityfocus.com OK
12     ...

NOTES:

Routine greylist maintenance

The greylist database grows over time, because the greylist server never removes database entries. If left unattended, the greylist database will eventually run your file system out of space.

When the status file size exceeds some threshold you can simply rename or remove the file without adverse effects; Postfix automatically creates a new file. In the worst case, new mail will be delayed by an hour or so. To lessen the impact, rename or remove the file in the middle of the night at the beginning of a weekend.

Example Perl greylist server

This is the Perl subroutine that implements the example greylist policy. It is part of a general purpose sample policy server that is distributed with the Postfix source as examples/smtpd-policy/greylist.pl. 

#
# greylist status database and greylist time interval. DO NOT create the
# greylist status database in a world-writable directory such as /tmp
# or /var/tmp. DO NOT create the greylist database in a file system
# that can run out of space.
#
$database_name="/var/mta/greylist.db";
$greylist_delay=60;

#
# Auto-allowlist threshold. Specify 0 to disable, or the number of
# successful "come backs" after which a client is no longer subject
# to greylisting.
#
$auto_allowlist_threshold = 10;

#
# Demo SMTPD access policy routine. The result is an action just like
# it would be specified on the right-hand side of a Postfix access
# table.  Request attributes are available via the %attr hash.
#
sub smtpd_access_policy {
    my($key, $time_stamp, $now);

    # Open the database on the fly.
    open_database() unless $database_obj;

    # Search the auto-allowlist.
    if ($auto_allowlist_threshold > 0) {
        $count = read_database($attr{"client_address"});
        if ($count > $auto_allowlist_threshold) {
            return "dunno";
        }
    }

    # Lookup the time stamp for this client/sender/recipient.
    $key =
        lc $attr{"client_address"}."/".$attr{"sender"}."/".$attr{"recipient"};
    $time_stamp = read_database($key);
    $now = time();

    # If new request, add this client/sender/recipient to the database.
    if ($time_stamp == 0) {
        $time_stamp = $now;
        update_database($key, $time_stamp);
    }

    # The result can be any action that is allowed in a Postfix access(5) map.
    #
    # To label the mail, return ``PREPEND headername: headertext''
    #
    # In case of success, return ``DUNNO'' instead of ``OK'', so that the
    # check_policy_service restriction can be followed by other restrictions.
    #
    # In case of failure, return ``DEFER_IF_PERMIT optional text...'',
    # so that mail can still be blocked by other access restrictions.
    #
    syslog $syslog_priority, "request age %d", $now - $time_stamp if $verbose;
    if ($now - $time_stamp > $greylist_delay) {
        # Update the auto-allowlist.
        if ($auto_allowlist_threshold > 0) {
            update_database($attr{"client_address"}, $count + 1);
        }
        return "dunno";
    } else {
        return "defer_if_permit Service temporarily unavailable";
    }
}