Introduction

This section describes how to install an “administrative command handler”, something you can email list management commands to, just like emailing commands to LISTSERV. If you never used the Listserv email interface then this entire section may be unnecessary for you.

Background

One of the first things I did when investigating Mailman was try to find out how to send mailing list management commands by email.

That was important because in administering our 70 email lists over the years, we’d developed a set of helper scripts and modules to make list management actions like subscribing/unsubscribing easier.

If someone wanted to subscribe to our lists they’d put their name and email into a form, check off the lists they wanted, and submit. Then behind the scenes our scripts would send an email to LISTSERV with the appropriate commands to make it happen.

This was all aimed at 1) allowing subscribers to manage their own subscriptions on the web, and 2) allowing list admins to easily manage subscribers over a lot of lists.

Unfortunately, it soon became apparent that Mailman has no administrative email interface. Instead, Mailman provides a simple but clean web-based interface for list admins to manage users on each mailing list.

Going to each list’s web interface to manage a given subscriber is fine for one or a few mailing lists, but it’s completely unmanageable at 10, 20, or 70 lists—you need a simple one-stop place to set options, change addresses, etc. for a subscriber on all lists at once.

This meant I either had to abandon Mailman as the mailing list software in favor of something which did have an administrative email interface, or stick with Mailman and program it myself. Since Mailman seemed like the only viable replacement for Listserv, I decided to scratch my own itch.

Fortunately, I already knew Python, which most of Mailman is written in, so programming an administrative email interface which could call built-in Mailman code to do its work was actually possible.

Issues Setting Up Admin Command Handler

I’d noticed when I created the “test-list” on my development box with the newlist command that it told me I still needed to create some email aliases for various list-related addresses to pipe messages into mailman for handling:

To finish creating your mailing list, you must edit your
/etc/aliases (or equivalent) file by adding the following
lines, and possibly running the `newaliases' program:

## test-list mailing list
test-list:              "|/path/to/mailman post test-list"
test-list-admin:        "|/path/to/mailman admin test-list"
test-list-bounces:      "|/path/to/mailman bounces test-list"
...

This got me to thinking that I’d need an email address to receive the Mailman admin commands. My first thought, based on the above, was something like test-list-admin-cmd, but I didn’t want to have to make a new admin alias for every new list.

I really wanted something like our old listserv@lists.ourhost.com, which all our web and shell scripts were sending admin command messages to.

So I created a command handler called mailman_admin_cmd_handler.py and in the control panel for the development box set it so that mail sent to mailman-admin-cmd@lists.ourhost.com would get piped into mailman_admin_cmd_handler.py

(Our host’s control panel was cPanel, and I used the Forwarders option in the Mail section to “Add a Forwarder”, then “Address to Forward” of mailman-admin-cmd, then “Advanced Options”, then “Pipe to a Program”, then /path/to/mailman_admin_cmd_handler.py)

If you use /etc/aliases for Sendmail or Exim, as the newlist output suggests, then you’d probably do something like:

## admin interface for all mailing lists
mailman-admin-cmd:   "|/path/to/mailman_admin_cmd_handler.py"

Actually that’s a simplification of what I really did. I really used procmail because that’s what handled all our other mail forwarding needs. So in “Pipe to a Program” I piped it to |/usr/bin/procmail -m /home/ouruser/.procmail/rc.mailman-admin-cmd and then in that procmail recipe file I put:

:0 H
* ^TOmailman-admin-cmd@lists.ourhost.com
| $HOME/.procmail/mailman_admin_cmd_handler.py

(actually I had a few other things in that file, but a procmail tutorial is beyond the scope of this guide—type “man procmailrc” or “man procmailex” at a shell for more info)

This was all on our dev box, and part of why I stuck mailman_admin_cmd_handler.py in ~/.procmail was that my goal was not to make any modifications to the base Mailman installation in case our production list host didn’t let us muck with those files.

This placement had a few minor consequences. One is that I had to create symbolic links for two mailman “bin” programs, list_members and clone_member, into the same directory as the handler script, adding .py to the symbolic links so the handler script could import them. For example:

cd /home/ouruser/.procmail
ln -s /path/to/mailman/bin/clone_member ./clone_member.py
ln -s /path/to/mailman/bin/list_members ./list_members.py

I also had to install a helper module (ListStream.py) and configure mailman_admin_cmd_handler.py with custom settings like the Mailman installation directory, the SMTP server, an optional string/password for the subject line of admin command messages (for added security), etc.

Even if you can put mailman_admin_cmd_handler.py in the mailman bin directory itself, you’ll still need to symlink clone_member to clone_member.py (and the same for list_members) within that directory so mailman_admin_cmd_handler.py can import and use those scripts, you’ll still need to save ListStream.py there, etc.

(But I’d still recommend keeping these custom scripts outside the installed Mailman tree, even if you have permissions to modify those files, simply because if you ever upgrade your Mailman installation, you don’t want to inadvertently delete/move the custom files and break a lot of stuff. Plus it’s just nice to have standard-install Mailman stuff in one area, and custom add-on stuff in another.)

I should also mention that when we finally chose a list host, they didn’t use alias forwarding/piping or procmail at all, they used a specialized postfix-to-mailman.py script so that no aliases needed to be made when new lists were created.

(If you’re using Postfix with Mailman see the GNU Mailman Installation Manual, particularly the Integrating Postfix and Mailman section; see also a related discussion on FreeBSDDiary.)

Downloading the Files

Here are the basic files for setting up the administrative email interface; click to view, right-click and Save to download:

• mailman_admin_cmd_handler.py – the main handler program
• ListStream.py – a helper module used by mailman_admin_cmd_handler
• clone_member.py.txt – a reminder that you need to symlink clone_member
• list_members.py.txt – a reminder that you need to symlink list_members
• test_mailman_admin_cmd_handler.py – optional unit tests of the handler

test_mailman_admin_cmd_handler.py is completely optional, but I’d recommend downloading and running it just to check that you made all the necessary customization changes in mailman_admin_cmd_handler.py. Just be sure to look through it thoroughly before running it, because you’ll need to choose your own test values.

IMPORTANT NOTE: I’ve tried to make sure that any code I’m providing has the word INSTALL in comments to mark any places you’ll need to modify the code for your own installation (i.e., places where your installation will probably differ from mine). So you need to go through every file you download and search/grep for INSTALL and update the corresponding settings.

Installing

The following are the steps I used to install the administrative email command handler on our production host. In some cases your environment may differ, such as the difference between using Sendmail/procmail filtering (on our dev box) and using Postfix (on our production box) as described above.

First, I created a directory to hold all custom Mailman related files, ~/mailman.

I also noted the locations of the system (installed) Mailman files, which seemed to be /etc/mailman, /var/lib/mailman, and /usr/lib/mailman. In particular, I needed to know the path to the directory tree holding the Mailman bin scripts and helper objects, which the admin command handler uses. (For us, those were /usr/lib/mailman/bin/ and /usr/lib/mailman/Mailman/ respectively.)

After downloading the files in the “Downloading the Files” section above into the ~/mailman directory I needed to modify the /etc/mailman/postfix-to-mailman.py file in order to allow commands sent to mailman-admin-cmd@lists.ourhost.org to get sent to the handler script:

                  ("/usr/sbin/sendmail", MailmanOwner))
         sys.exit(0)

+    # Local customization
+    elif local == 'mailman-admin-cmd':
+        mm_admin_pgm = '/home/adf/mailman/mailman_admin_cmd_handler.py'
+        os.execv(mm_admin_pgm, [])
+
     # Assume normal posting to a mailing list
     mlist, func = local, 'post'

(If you’re not familiar with diff output, that means I added the four lines with the pluses to the area between the sys.exit(0) line and the # Assume lines.)

At this point I really wanted to send a command to see what would happen, but I thought it would be better to run the handler’s unit tests first to see if I’d missed something basic.

To do this, I needed to search the downloaded files for server-specific things to edit that had been flagged with INSTALL comments (such as the location of our Mailman files in /usr/lib/mailman).

I also had to, as described above, create the list_members.py and clone_member.py symlink files in ~/mailman pointing to list_members and clone_member in /usr/lib/mailman/bin.

The unit tests assume a test list called test-list, so I had to use newlist test-list in /usr/lib/mailman/bin/ to make it (you can delete it later with rmlist if you wish).

The unit tests in test_mailman_admin_cmd_handler.py also assume there is a test user already on the test list, so I did this with:

echo "test@ourhost.org" | /usr/lib/mailman/bin/add_members -r - test-list

Once I’d done that, and made sure all the other settings in test_mailman_admin_cmd_handler.py were correct (especially in the setUp function), I ran test_mailman_admin_cmd_handler.py at the command line and verified that all the unit tests passed.

(If you do this and the tests don’t pass, you’ll need to look at the tests that failed in test_mailman_admin_cmd_handler.py and figure out what went wrong and why.)

Testing

Now, having verified that the code was set up correctly via the unit tests in test_mailman_admin_cmd_handler.py, I wanted to send a test command by email.

I should mention that I hit a snag at this point that you might not.

I had decided, as part of the overall migration strategy, to have our new Mailman list host be lists2.ourhost.org while continuing to run our lists on the soon-to-be-old Listserv lists.ourhost.org.

When Mailman was set up properly, including creating all lists and copying all users and their options, so that the two hosts were in sync, the plan for the cutover was to just change where lists.ourhost.org pointed to. This worked out pretty smoothly in the end, by the way, but I wanted to explain the lists2.ourhost.org thing now.

So I sent a command to mailman-admin-cmd@lists2.ourhost.org, putting help in the subject line and body.

That should have generated an error because the default setup requires any emailed admin commands to have a password in the subject line for security, and of course most list-related commands require their own passwords too (the list password, the site admin password, or the list owner’s password if the list owner is on the list and the email is sent from that account).

Unfortunately that simple “help” test produced the following error:

550-Verification failed for <mailman-admin-cmd@lists2.ourhost.org>
550-The mail server could not deliver mail to
mailman-admin-cmd@lists2.ourhost.org. The account or domain may not
exist, they may be blacklisted, or missing the proper dns entries.
550 Sender verify failed (in reply to RCPT TO command))

Also, the error only happened when I sent from myaddress@ourhost.org; when I sent the help test message from anotheracct@gmail.com (or any domain which was not ourhost.org) it worked fine and produced the expected “Required password not present in subject line.”

The admin for our new list provider said, “Because this test domain, lists2.ourhost.org, does not have Sender Verify set up on it, it’s failing the check and your ourhost.org mailserver is rejecting it.” I researched this and found that what he called “sender verify” refers to the Sender Policy Framework, aka SPF record, an anti-spam related DNS record.

Eventually our old host (which was still running the DNS for ourhost.org) solved the problem and when I asked them how, they wrote:

I’m not sure I remember all the steps along the way. I originally
created an A record for lists2.ourhost.org, and an MX record that
referred to the same IP address. When all was said and done I had
an A record, an MX record that referred to lists2.ourhost.org (instead
of to its IP address) and a TXT field with the following contents:

spf1 a mx a:208.85.173.148 -all

Here are the tinydns source records that I used:
+lists2.ourhost.org:208.85.173.148:3600
@lists2.ourhost.org::lists2.ourhost.org:20
‘lists2.ourhost.org:v=spf1 a mx a\072208.85.173.148 -all:3600

(I’ve altered the IP addresses for the privacy of the org I did this for, btw.)

I wish I could give more information about this Sender Verify problem and what really solved it, but at least if you encounter the same thing this info might help.

Anyway, having solved the Sender Verify problem, I re-sent the test message to mailman-admin-cmd@lists2.ourhost.org with help in the subject and body and got the expected “Required password not present in subject line” response.

Then I sent another test message, this time with help PASSWORD in the subject (no quotes, where PASSWORD was the value of SUBJECT_PASSWD in mailman_admin_cmd_handler.py) and help in the body, and got the expected response, “Valid commands:” followed by a list of valid commands for the admin command handler.

I didn’t do an exhaustive test of every command, since the unit tests in test_mailman_admin_cmd_handler.py sort of do that, but I did try one more command besides “help” just to make sure the basic mechanism was working.

I tried sending the “who” command to see who was subscribed to the test list I’d set up. Specifically, I sent a message with a subject of who test-list PASSWORD (where PASSWORD was the SUBJECT_PASSWD value in mailman_admin_cmd_handler.py) and a body of who test-list LISTPASSWORD (where LISTPASSWORD was the password I set for the list when creating it).

(Note: You can actually use anything in the subject line as long as it has the admin command handler password, but I like to keep it as descriptive as possible, and for a single command that usually means using the command itself minus the list password.)

After a few issues involving list ownership (I’d mistakenly sent the command from an account that wasn’t one of the list owners for the test list), I finally got a roster of subscribers for test-list so I felt a little more confident that the basic system was working correctly.

Next: Creating Mailman Lists Based On Listserv Lists or Up: Table of Contents