WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING
===============================================================
The content filtering approach described in this document is suitable
only for low-traffic sites. See the "Pros ans Cons" section below
for details.

Purpose of the before-queue content filter feature
==================================================

The Postfix SMTP server can be configured to forward all incoming
mail to a content filter (for example, a real-time SPAM filter)
that inspects all mail BEFORE it is stored in the Postfix mail
queue.

The before-queue content filter is meant to be used as follows:

                        BEFORE                               / smtp
    Internet -> smtpd -> QUEUE -> smtpd -> cleanup -> queue -> local
               Postfix  FILTER   Postfix                     \ virtual etc.

The before-queue content filter is not to be confused with the
approach described in the FILTER_README document, where mail is
filtered AFTER it is stored in the Postfix mail queue.

Principles of operation
=======================

The filter receives unfiltered SMTP mail from Postfix and does one
of the following:

1 - Re-inject the mail back into Postfix via SMTP, perhaps after
    changing content.

2 - Reject the mail by sending a suitable SMTP status code back
    to Postfix. Postfix passes the status back to the remote SMTP
    client. This way, Postfix does not have to send a bounce message.

3 - Send the mail somewhere else, or discard the mail.

The before-queue content filter functions just like the after-queue
content filter. In many cases you can use the same content filtering
software, within the limitations as discussed in the "Pros and Cons"
section below.

Using the SMTP pass-through proxy feature
=========================================

In the following example, the Postfix SMTP server gives mail to a
content filter that listens on localhost port 10025, and receives
mail from the content filter via localhost port 10026. From then
on mail is processed as usual.

The result looks as follows:

    Internet
	-> Postfix SMTP server on port 25
	    -> filter on localhost port 10025
		-> Postfix SMTP server on localhost port 26
		    -> cleanup
			-> queue

This is configured by editing the master.cf file:

/etc/postfix/master.cf:
    # =============================================================
    # service type  private unpriv  chroot  wakeup  maxproc command
    #               (yes)   (yes)   (yes)   (never) (100)
    # =============================================================
    #
    # Before-filter SMTP server. Receive mail from the network and
    # pass it to the content filter on localhost port 10025.
    #
    smtp      inet  n       -       n       -       20      smtpd
        -o smtpd_proxy_filter=localhost:10025
	-o smtpd_client_connection_count_limit=10
    #
    # After-filter SMTP server. Receive mail from the content filter
    # on localhost port 10026.
    #
    :10026    inet  n       -       n       -        -      smtpd
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
        -o receive_override_options=no_unknown_recipient_checks

Note: do not specify spaces around the "=" or "," characters.

The before-filter SMTP server entry is a modified version of the
default Postfix SMTP server entry that is normally configured at
the top of the master.cf file:

- The number of SMTP sessions is reduced from the default 100 to
  only 20. This prevents a burst of mail from running your system
  into the ground with too many content filter processes.

- The "-o smtpd_client_connection_count_limit=10" prevents one
  SMTP client from using up all 20 SMTP server processes.  This
  limit is not necessary if you receive all mail from a trusted
  relay host.

- The "-o smtpd_proxy_filter=localhost:10025" tells the before
  filter SMTP server that it should give incoming mail to the
  content filter that listens on localhost port 10025.

The after-filter SMTP server is a new master.cf entry:

- The ":10026" makes the after-filter SMTP server listen on the
  localhost address only, without exposing it to the network.
  NEVER expose the after-filter SMTP server to the Internet :-)

- The "-o smtpd_authorized_xforward_hosts=127.0.0.0/8" allows the
  after-filter SMTP server to receive remote SMTP client information
  from the before filter SMTP server, so that the after-filter
  Postfix daemons log the remote SMTP client information instead
  of logging localhost[127.0.0.1].

- The other after-filter SMTP server settings avoid duplication of
  work that is already done in the "before filter" SMTP server.

The filter itself is not described here. You can use any filter
that is SMTP enabled. For non-SMTP capable content filtering
software, Bennett Todd's SMTP proxy implements a nice PERL/SMTP
content filtering framework. See: http://bent.latency.net/smtpprox/

By default, the filter has 100 seconds to do its work. If it takes
longer then Postfix gives up and reports an error to the remote
SMTP client. You can increase this time limit (see configuration
parameter section below) but doing so is pointless because you
can't control when the remote SMTP client times out.

Pros and Cons
=============

The before-queue content filter allows Postfix to reject mail before
the incoming SMTP mail transfer completes, so that Postfix does
not have to send rejected mail back to the sender.  Mail that is
not accepted remains the responsibility of the remote SMTP client.

The problem with before-queue content filtering is that the remote
SMTP client expects an SMTP reply within a deadline. As the system
load increases, fewer and fewer CPU cycles remain available to
answer within the deadline, and eventually you either have to stop
accepting mail or you have to stop filtering the mail.

Another problem is that content filtering software can use lots of
memory resources. In order to not run out of memory you have to
reduce the number of before-filter SMTP server processes so that
a burst of mail will not drive your system into the ground with
too many content filter processes. This, in turn, means that SMTP
clients have to wait for a long time before they receive service.

How Postfix talks to the before-queue content filter
==================================================

The before-filter Postfix SMTP server connects to the content
filter, delivers one message, and disconnects.  While sending mail
into the content filter, Postfix speaks ESMTP but uses no command
pipelining.  Postfix generates its own EHLO, XFORWARD (for logging
the remote client IP address instead of localhost[127.0.0.1]), DATA
and QUIT commands, and forwards unmodified copies of all the MAIL
FROM and RCPT TO commands that the before-filter Postfix SMTP server
didn't reject itself.  The SMTP proxy server should accept the same
MAIL FROM and RCPT TO command syntax as the Postfix SMTP server.
Postfix sends no other SMTP commands.

The content filter is expected to pass on unmodified SMTP commands
from a before-filter Postfix SMTP server to an after-filter Postfix
SMTP server that usually listens on a non-standard port.  When the
filter rejects content, it should send a negative SMTP response
back to the before-filter Postfix SMTP server, and it should abort
the connection with the after-filter Postfix SMTP server without
completing the SMTP conversation with the after-filter Postfix SMTP
server.

More details on the postfix-to-proxy interaction is at the end of
this document, in the section titled "Transparency".

Configuration parameters
========================

Parameters that control proxying:

smtpd_proxy_filter (syntax: host:port)

    The host and TCP port of the before-queue content filter.  When
    no host or host:  is specified, localhost is assumed.

smtpd_proxy_timeout (default: 100s)

    Timeout for connecting to the before-queue content filter and
    for sending and receiving commands and data.  All proxy errors
    are logged to the maillog file. For privacy reasons, all the
    remote SMTP client sees is "451 Error:  queue file write error".

smtpd_proxy_ehlo (default: $myhostname)

    The hostname to use when sending an EHLO command to the
    before-queue content filter.

Transparency
============

The before-filter Postfix SMTP server forwards the MAIL FROM, RCPT
TO and DATA commands that it has approved, but it does not forward
other commands such as TLS or SASL commands.  It can therefore not
be transparent.

The real-time content filter, on the other hand, has to be transparent.
In order to support non-transparent real-time content filters,
Postfix would have to reconcile the before-filter Postfix ESMTP
feature set with the feature set that Postfix receives from the
real-time content filter.

- When a future Postfix version supports DSN, but the content filter
  does not announce DSN support in the EHLO reply, then the
  before-filter SMTP server would have to either 1) suppress the
  DSN feature in its EHLO announcement, or 2) duplicate all the
  work that needs to be done when delivering DSN-aware mail to a
  non-DSN destination.

- When the content filter does not announce 8BITMIME support in
  the EHLO reply, then the before-filter SMTP server would have to
  either 1) suppress the 8BITMIME feature in its EHLO announcement,
  or 2) convert the content to quoted-printable before giving it
  to the content filter.

- Performance: when Postfix has to suppress elements from the
  before-filter EHLO reply because they are incompatible with the
  real-time content filter, then Postfix has to connect to the
  content filter as soon as the client sends a valid EHLO command.
  This wastes a lot of resources when all the MAIL FROM or RCPT TO
  commands are rejected.

Therefore, the Postfix SMTP server cannot be transparent with
respect to the before-queue content filter.
