Spam Vigilante - Mail Filter Virtual Appliance
Table of Contents
Sections
1. Introduction
2. Installation
3. Spam Viewer
4. Updates
Appendices
A. Integration Notes
B. Important Files
C. Software Used
D. References
E. Disclaimer
Section 1 - Introduction
This virtual machine mail filter appliance is designed
to give small businesses and organizations the ability to deploy spam and
virus checking for their e-mail communications in only minutes. It
does this work by acting as a preprocessor, retrieving and scanning all
incoming mail and then forwarding it to its final destination.
It offers the unique benefits of allowing users to view and delete quarantined
messages through a custom, easy-to-use web interface and a level of integration
with Microsoft Exchange via LDAP, if that server is also being used.
Open source and free software is used throughout, so no licensing costs
are incurred.
The mail filter is not the mail server that users will access
directly. Because there is no user interaction, its deployment can be completely transparent, and no
user account details need to reside on the appliance, allowing it to be integrated
equally well into
both new and existing mail infrastructures and to be compatible with any mail system
which can accept SMTP connections (virtually any Internet-capable mail system).
Section 2 - Installation
Choosing a Layout:
The mail filter appliance is not a mail server in the traditional sense
of the term. It is intended as a complement to an existing service,
supplementing it by pre-filtering virus-infected or spam messages, and
processing them as directed. By default this means marking spam
messages, and quarantining viruses. Though no traditional mail server
is included in this appliance, if it is also required, one may wish to look
into hMailServer, an excellent open source mail server for Windows operating
systems, Postfix, the SMTP server used by this appliance, and Dovecot, an
open source IMAP/POP3 mail server for UNIX-like operating systems, including
Linux. Links to all three are provided in Appendix D.
The mail filter is designed to operate in two primary modes, either as a
standard SMTP server, shown in Figure 1.1, or as a "fetching" mail server,
shown in Figure 1.2.
SMTP is the traditional form of mail server operation.
When a message is sent by another party, their SMTP server consults DNS to find the appropriate mail exchange (MX) records for
the recipient -- in this case, the mail filter appliance. The
sending server connects to the recipient server and submits the message, which
is delivered to a user's mailbox, completing the process. The
mail filter appliance works in this chain as a proxy, receiving the incoming
message from SMTP, processing it, and then forwarding it again to the "real"
SMTP server which contains the user's mailbox. This approach requires
registering the IP address and hostname of your appliance in DNS, so that
other parties on the Internet can locate it, and a high-level of
availability from your equipment, so messages cannot be missed or bounced.
It is best suited for organizations that need absolute control over all
parts of their mail system and that can provide the level of maintenance and
expertise needed to administer it.
By contrast, the fetching layout for the mail appliance does not require
registering
DNS or allowing inbound connections from the
Internet. Instead, a hosting provider is used to host the public SMTP
server and an e-mail tool is used to retrieve the mail from the
hosting company's server using POP3 or IMAP4, similar to a user checking
mail with a mail client. The messages are downloaded and injected into
the appliance's SMTP server for processing. After that, the chain is identical to the SMTP layout; the mail is forwarded
to the "real" mail server on the network and is delivered into the
receiving user's mailbox. This layout is ideal for many small
businesses and organizations, as it allows a high level of control over
e-mail services, while providing protection against incoming mail
being bounced because of a local hardware or connection failure.
That is important because many organizations do not have the fully
redundant hardware, power, and Internet connectivity that a quality hosting
provider can supply.
Figure 1.1 - SMTP Layout |
Figure 1.2 - Fetchmail Layout |
Starting:
Once you've chosen a layout, to begin installation simply boot the
virtual machine. When presented with the login prompt, login using the
username 'root' and the password 'password', and a setup wizard will launch.
Below is an example setup session using the fetchmail layout. In the
example, the internal network has a domain name of 'company.local' and the
domain used for mail (hosted by a hosting provider) is 'company.com'.
The internal network in 192.168.100.0. The final destination server
for the processed messages is named mail.company.local and is running
Microsoft Exchange. Default values are shown by the wizard within
brackets ("[]") and examples within parentheses ("()"). Additional
comments are added where needed.
Network configuration:
Enter the fully qualified hostname
(eg, mailfilter.company.com) []: fetcher.company.local
Use DHCP to configure network settings? (Yes/No) [no]:
Enter the IP address to use
(eg, 192.168.100.10) []: 192.168.100.2
Enter the subnet mask to use
(eg, 255.255.255.0) []: 255.255.255.0
Enter the address of the default gateway
(eg, 192.168.100.1) []: 192.168.100.1
Enter the address of the primary DNS server
(eg, 192.168.100.1) []: 192.168.100.1
Enter the address of the secondary DNS server
(eg, 192.168.100.2) (blank for none) []:
Fetchmail configuration:
Use fetchmail or use as SMTP server (fetchmail/smtp) [fetchmail] ?
Would you like to enter account information to be used by fetchmail?
If there are more than a few accounts it is recommended to edit the
fetchmail configuration file (/usr/local/etc/fetchmail.cf)
directly (Yes/No) [no]: yes
Would you like to keep messages on the servers after fetching?
Strongly recommended when first configuring a system. (Yes/No) [yes]:
Enter a server to poll (blank to quit): mail.company.com
Enter a username/address for this server (blank to quit): user_one@company.com
Enter a password for this account: password_one
Enter the local address to take delivery: user_one@company.local
Enter a username/address for this server (blank to quit): user_two@company.com
Enter a password for this account: password_two
Enter the local address to take delivery: user_two@company.local
Enter a username/address for this server (blank to quit):
Enter a server to poll (blank to quit):
E-mail setup:
External mail domain (eg, redbudcomputer.com) []: company.com
Internal mail domain (eg, redbudcomputer.local).
Very often this is the same as the external []: company.local
Enter the full name or address of the mail server to take
final delivery after filtering is complete []: mail.company.local
Enter a comma separated list of domains this server
should service / relay ['company.com', 'company.local']:
Enter 'host,' subnet,' or a comma separated list of
networks (eg, 192.168.100.0/24) from which this server
should allow clients. The default option is usually best unless
one is sure they have other requirements ['host']:
Spam configuration:
Spam message destiny (D_PASS, D_BOUNCE, or D_DISCARD).
It is safest to use D_PASS, and allow the clients to filter
on the X-Spam-Status or X-Spam-Level headers. [D_PASS]:
Virus message destiny (D_PASS, D_BOUNCE, or D_DISCARD).
It is strongly recommended to use D_BOUNCE or D_DISCARD so
that infected messages cannot be delivered to clients. [D_BOUNCE]:
Banned attachement message destiny (D_PASS, D_BOUNCE, or D_DISCARD).
D_BOUNCE or D_DISCARD should be used so disallowed attachments (eg, .exe)
are not delivered to clients. [D_BOUNCE]:
Bad header message destiny (D_PASS, D_BOUNCE, or D_DISCARD).
Leave this at the default if delivering a message with a malformed
header is OK. [D_PASS]:
Spam level - the score required to mark a message as spam
(X-Spam-Status: Yes) and trigger corresponding action. [6.31]:
Allow banned attachments if they are inside a .zip file?
Eg, if .exe attachments are banned, they can still be received
if the are packaged inside a .zip attachment. The full list of affected
archive types is: zip, bz2, gz, tgz, tbz. (Yes/No) [no]: yes
SA maximum mail body size (in bytes). Messages larger than this
size will be scanned for viruses, but not for spam. [524288]:
Maximum size of the quarantine mail folder (in MB).
When the quarantine folder reaches this size, the oldest
messages are deleted [500]:
Use extra spam checking rules from RulesDuJour? Recommended
unless the server is under a high load. (Yes/No) [yes]:
Integration configuration:
Enable web interface for viewing quarantined
messages? The URL is https://<server>/viewer/ (Yes/No) [yes]:
Use Microsoft Exchange recipient integration?
This provides automated construction of an allowed recipients
map by querying the Exchange server. If you selected to enable
the web viewer, it allows users to login to it using their Exchange
address and password (Yes/No) [yes]:
By default the integration tools will attempt to establish
a secure connection to the Exchange server, and if that fails
will attempt an insecure connection. If it is never allowable
to try an insecure connection you should change the value of
LDAP_ALLOW_INSECURE in /usr/home/spamviewer/config.py to 'False'.
Enter the hostname of the Exchange-enabled
Active Directory server to connect to []: mail.company.local
Enter an Active Directory username to use to connect.
The username can be in the form of an LDAP DN or @.
The user only needs read access to the part of directory to be scanned
for accounts. Do not use a privileged account! [] sync@company.local
Enter the password for the account []: password
Enter the base DN of the topmost container to scan.
This can be the top of the domain such as "DC=company,DC=local"
or a sub-container like "CN=Users,DC=company,DC=local" []: OU=MyBusiness,DC=company,DC=local
Review:
Network configuration:
Hostname: fetcher.company.local
Use DHCP: no
IP Address: 192.168.100.2
Subnet Mask: 255.255.255.0
Default Gateway: 192.168.100.1
Primary DNS: 192.168.100.1
Secondary DNS:
Mail configuration:
Destination mail server: mail.company.local
Fetchmail or SMTP: fetchmail
mail.company.com:
user_one@company.com password_one user_one@company.local
user_two@company.com password_two user_two@company.local
Ext. mail domain: company.com
Int. mail domain: company.local
Spam destiny: D_PASS
Banned destiny: D_BOUNCE
Bad header destiny: D_PASS
Virus destiny: D_BOUNCE
Spam level: 6.31
Allow banned inside zip?: yes
Max message size to scan for spam: 524288
Use extra spam rules?: yes
Allowed network: host
Relay domain: company.com
Relay domain: company.local
Integration:
Use web quarantine viewer?: yes
Use Exchange: yes
Exchange server: mail.company.local
Active Directory user: sync@company.local
Active Directory pw: password
Active Directory base DN: OU=MyBusiness,DC=company,DC=local
Configuration OK? (Yes/No) [Yes]:
...
It is recommended that you restart your system soon with
'shutdown -r now' to engage all of the configuration changes. Before
restarting, however, this is also an excellent time to change the root
password, using 'passwd,' and enter any additional fetchmail configuration
information before rebooting.
To reconfigure your machine at any time simply execute: 'touch /reconfigure'
and reboot!
Good hunting! |
That is it! Configuration of an SMTP style server is very similar,
except for entering fetchmail account details.
Likewise, entering information for a server without Exchange integration is
the similar, except that the sections dealing with the Active Directory
user name, base DN, and other details are skipped.
To activate all the settings,
reboot the appliance by executing 'shutdown -r now'. During boot, a
message may be shown stating that the virus database is older than seven
days. This is not an error. It will be automatically updated,
and kept up to date, from now on since the network settings have been
configured. As soon as it has rebooted, the appliance is ready to handle mail. Try sending and receiving test
messages to confirm, and in particular examine the headers of incoming
messages for an entry such as "X-Virus-Scanned: amavisd-new at company.com."
If any problems are encountered, try adjusting the configuration through the
setup wizard by running "python /usr/local/etc/conftemplates/spam_configure.py"
or by executing "touch /reconfigure" and rebooting. In the latter
case, the setup wizard
will present itself again at login.
Note: Though the appliance has SSH access, configuring over SSH can be
troublesome as the network interfaces are restarted during the final stage.
Section 3 - The Spam Viewer
Introduction:
The Spam View web application was specifically made for this appliance. If enabled, the Spam Viewer can be used to view any quarantined messages
through a web interface. The administrator account ('vadmin')
can view the messages for any user. Users may also login to view their
own messages.
Login operates in two modes, LDAP login and local login. The
default login method is dependent on whether Exchange integration was
chosen. If it was selected, LDAP login is used, allowing users to
login to the viewer using their Exchange e-mail account and password.
The standard login method requires that user accounts be created by 'vadmin'
before those may login.
Note: The login method may be selected manually by setting
the value of the LOGIN_METHOD variable in config.py.
Administration
To begin administering the viewer, login with the username 'vadmin' and
the password 'password'. A screen similar
to the one in Figure 2.1 will open.
Note: If the LDAP login
method is being used, the "Add User" link will not be present.
Figure 2.1 - Administration Console |
From this console you can change the password for the 'vadmin' user,
clear the quarantine/spam folder, and manage user accounts for the
viewer.
User Accounts
Before users can login to view their messages, accounts must
exist for them in the viewer's database. User accounts in the system
consist of two primary parts: the username and a list of associated e-mail addresses. Each e-mail address may only be associated
with one user, but each user can have multiple e-mail addresses. This is
particularly useful with a fetchmail-style layout where a single user can be
receiving mail from multiple external addresses (e.g.
where mail for user@domain_one.com and for user@domain_two.com
is being delivered to user@company.com). Each user can only
view messages addressed to an account contained within their list of allowed
addresses.
Adding Users
If user accounts are being maintained locally rather than using the LDAP login
method, user accounts will need to be added manually.
Selecting 'Add User' from the Administration Console will show a page
similar to the one if Figure 2.2. Enter the username and one e-mail
address per line in the 'Address' box. Press 'Save' to store the user.
Figure 2.2- New User
|
If the LDAP login method is being used instead, users accounts are drawn
from the remote server. They are updated daily by the script '/usr/local/sbin/maintain-spamviewer.sh'
so that new user accounts will be found automatically and e-mail addresses
found in the directory are assigned to the users. You may also execute
this script manually to perform an update immediately. By default,
accounts are not altered once they are created. To perform a full
update that clears existing user information entirely and re-draws it from
the directory server execute '/usr/local/sbin/maintain-spamviewer.sh' with
the '-c' option.
Editing a User
The Users page, accessible from the Administration Console, will
present a list of all the users in the system similar to Figure 2.3.
Figure 2.3 - Users |
Clicking on "Delete" will remove the user account and clicking "Messages"
will allow you to view that user's message list. Clicking on the
username in the Users page will allow editing of the user's details. From
here, you can change the e-mail addresses associated with an account and, if
the account information is local, the user's password. Figure 2.4
shows the Edit User page.
Note: If you are using LDAP and the list of users
is empty, you may wish to execute the '/usr/local/sbin/maintain-spamviewer.sh'
script. This script is executed periodically, but if you have recently
installed the system it may not have been executed yet.
Figure 2.4 - Edit User |
Viewing Messages
The message view shows the basic message details, such as the subject and
the sender. The messages are sorted by date. It also
displays the reason for quarantine -- VIRUS, SPAM, BADH[eader], or BANNED.
Messages may be filtered using the form in the upper corner. The
search phrase is used to search the Subject, From, and To fields, but does
not search the message body.
Clicking upon the subject will display the contents of the message in a
safe, plain-text view. The Download link allows the original
message to be retrieved in the message/rfc822 MIME-type. By default,
the Download link is not available for messages infected with a virus but
is for all other types, including those with banned attachments. This
can be changed by altering the value of the ALLOWED_DOWNLOAD_TYPES variable
in config.py.
The Delete All link will remove all quarantined messages for the user.
Added in an update to the original package, it is also possible to
release a message from the quarantine to resume normal delivery. This
option takes the form of a red right-arrow (not pictured) next to the
download link.
Figure 2.5 - Messages |
Section 4 - Updates
Updates and fixes:
As new features, updates, or bug fixes are released, downloads and
information will be made available at
http://www.redbudcomputer.com/downloads.htm.
Appendix A - Integration Notes
In order to integrate the Postfix recipients list and Spam Viewer user
account database with information from Exchange, an account needs to be
available on the Exchange/Active Directory server that can be used to
retrieve the information. This user should have no special privileges,
as all that is required is read access on the items within the parts of the
directory tree that are to be searched.
One option is to create a dedicated user account so that
access can be audited and controlled. A sample structure is
illustrated in Figure B.1, where the user account has been added to a group
with restricted login privileges. User Rights Assignments within a Group Policy
Object are used to prevent any member of the group from logging in locally
to any machines in the Active Directory domain. The key properties of
the restriction are listed in Figure B.2. The group should not be
added to the the "Deny access to this computer from the network" property,
or the user will not be able to perform the necessary queries.
Figure B.1 - GPO |
Figure B.2 - GPO Style |
Additional security can be added by controlling the allowed login hours
for the user to those used by the automated task
and by implementing auditing of user logon events. Both topics, and instructions on how to create
the group policy object are beyond the scope of this documentation.
Please refer to
http://www.microsoft.com/windowsserver2003/technologies/management/grouppolicy/default.mspx
(group policy),
http://support.microsoft.com/?kbid=814595 (auditing), and
http://technet2.microsoft.com/WindowsServer/en/Library/f94917d2-660a-48a0-a1e6-9a67252c6dd71033.mspx
(logon hours) for more information.
Appendix B - Important Files
The locations of important configuration files are listed below.
Amavis (Mail Scanner):
/usr/local/etc/amavisd.conf
/var/virusmails
Apache (Web Server):
/usr/local/etc/apache2/http.conf
/usr/local/etc/apache2/ssl.conf
/usr/local/etc/apache2/Includes/spamviewer.confClamAV (Anti-virus):
/usr/local/etc/clamd.conf
/usr/local/etc/freshclam.conf
Exchange Integration:
/usr/home/spamviewer/config.py
Fetchmail (Mail Agent):
/usr/local/etc/fetchmail.cf
Postfix (SMTP):
/usr/local/etc/postfix/main.cf
/usr/local/etc/postfix/master.cf
/usr/local/etc/postfix/transport
/usr/local/etc/postfix/exchange_recipients (optional)
SpamAssassin (Spam Processing):
/usr/local/etc/mail/spamassassin/local.cf
/etc/rulesdujour/config
Spam Viewer:
/usr/home/spamviewer/config.py
Configuration Template Files:
/usr/local/etc/conftemplates/
Appendix C - Software Used
| amavisd-new 2.4.1 |
GNU Public License |
| Apache 2.0.58 |
Apache License |
| clamav 0.88.2 |
GNU Public License |
| Fetchmail 6.3.2 |
GNU Public License |
| FreeBSD 6.1 |
BSD License |
| mod_python 3.2.8 |
Apache License v2.0 |
| OpenSSH 4.2 |
BSD License |
| OpenSSL 0.9.7e |
OpenSSL (Apache-style) License |
| Perl 5.8.8 |
Artistic License |
| Postfix 2.2.9 |
IBM Public License v1.0 |
| pysqlite 2.1.3 |
pysqlite Open Source License |
| Python 2.4.2 |
Python 2.4 License |
| python-ldap 2.2.0 |
python-ldap (Python-style) License v1.1 |
| Quixote 2.4 |
CNRI Open Source License |
| Razor 2.81 |
Artistic License |
| session2 |
MIT License |
| SpamAssassin 3.1.1 |
Apache License |
| Spam Viewer |
Creative Commons Attribution License v2.5 |
Appendix D - References
Appendix E - Disclaimer
The information in this document is provided "as-is." No warranty
is expressed or implied and no guarantees are made to its accuracy.
Any loss of information, data, or revenue resulting from the use of the
information contained in this document is not the responsibility of the
author. Unless specifically stated, the author of this document does
not claim to be the original author of any works or products used in this
appliance. All copyrights are the property of their respective owners.
Copyright (C) Thomas E Lackey 2006
Licensed under the Creative Commons Attribution License v2.5
