Fetchmail and Sieve with Virtual Mail on Debian Etch

When it comes to mail servers, I really like the setup Christoph Haas describes in his Document Howto: ISP-style Email Server with Debian-Etch and Postfix 2.3. One thing I was missing on a server was the ability to automatically generate config files for fetchmail and sieve from the database in order to get mail from other servers and being able to apply server side filters on incoming mail. This howto is based on the mentioned tutorial.

The additional setup is quite simple: 2 more database tables hold the data for fetchmail and sieve rules and a set of PHP scripts called by cron every few minutes fetches the data and writes it into the appropriate config files. For fetchmail, a script creates a .fetchmailrc file in /home/vmail/. For sieve, another script creates a .dovecot.sieve config file for every user who got sieve rules in the database.

Dovecot configuration

To enable per-user sieve scripts, a virtual mail user has to have a home directory where dovecot/sieve can find the user-specific config file. This is given by the existing setup, but I didn’t want to write directly to the Maildir structure. So I moved the user’s mail in a subdirectory called – guess what – mail.

So the original setup looks like this: user home and mail location are the same directory

/home/vmail/%d/%n/

After editing I got this directory structure:

home: /home/vmail/domains/%d/%n/
mail: /home/vmail/domains/%d/%n/mail/

Edit /etc/dovecot/dovecot.conf and change the following entry:

mail_location = maildir:/home/vmail/domains/%d/%n/mail

Remember that if you already got existing mailboxes, you have to move them into the new subdirectories.

After a reload, dovecot does now look for a file named .dovecot.sieve in a user’s virtual home directory. You can try creating a sieve config file and sending a mail to that user to see if sieve works. Dovecot should also create a file named .dovecot.sievec and eventually .dovecot.sieve.err. If it doesn’t work, please make sure you loaded the cmusieve plugin in dovecot’s config. You can learn more on dovecot’s sieve implementation on the dovecot wiki.

Alter the database

To hold fetchmail and sieve records, it’s necessary to add two more tables to the mailserver database. I will also add two views which will make it easier to query the record sets.

First, let’s add the fetchmail table. The fields should be pretty straightforward.

field description
active if set to 1, the rule gets used
options options for fetchmail; for example keep would not delete mails from the source server when fetching them – see man fetchmail for more information
remoteserver, remoteuser, remotepass login credentials for the external POP3 server
CREATE TABLE `virtual_fetchmail` (
`id` int(11) NOT NULL auto_increment,
`user_id` int(11) NOT NULL,
`active` tinyint(1) NOT NULL default '1',
`options` varchar(50) NOT NULL,
`remoteserver` varchar(50) NOT NULL,
`remoteuser` varchar(50) NOT NULL,
`remotepass` varchar(50) NOT NULL,
PRIMARY KEY  (`id`),
FOREIGN KEY (user_id) REFERENCES virtual_users(id) ON DELETE CASCADE
) ENGINE=InnoDB;
Please be aware that login credentials for external servers are stored in plain text in the database. This is OK for me, but might not be for you. So take care about passwords and access rights and think about a way of encrypting passwords stored in the database.

Same for the sieve table:

field description
active if set to 1, the rule gets used
rules rule/rules which should be added to the user’s sieve config file
CREATE TABLE `virtual_sieve` (
`id` int(11) NOT NULL auto_increment,
`user_id` int(11) NOT NULL,
`active` tinyint(1) NOT NULL default '1',
`rules` text NOT NULL,
PRIMARY KEY  (`id`),
FOREIGN KEY (user_id) REFERENCES virtual_users(id) ON DELETE CASCADE
) ENGINE=InnoDB;

The mentioned views:

CREATE VIEW view_fetchmail AS
SELECT
virtual_fetchmail.id,
virtual_fetchmail.active,
virtual_fetchmail.remoteserver,
virtual_fetchmail.remoteuser,
virtual_fetchmail.remotepass,
virtual_fetchmail.options,
CONCAT(virtual_users.user, '@', virtual_domains.name) AS destination
FROM virtual_fetchmail
LEFT JOIN virtual_users ON(virtual_fetchmail.user_id = virtual_users.id)
LEFT JOIN virtual_domains ON(virtual_users.domain_id = virtual_domains.id);
CREATE VIEW view_sieve AS
SELECT
virtual_sieve.id,
virtual_sieve.active,
virtual_users.user,
virtual_domains.name AS domain,
virtual_sieve.rules
FROM virtual_sieve
LEFT JOIN virtual_users ON(virtual_sieve.user_id = virtual_users.id)
LEFT JOIN virtual_domains ON(virtual_users.domain_id = virtual_domains.id);

Install fetchmail

Easy step, thanks to apt :)

$ aptitude install fetchmail

Install scripts

As I mentioned in the introduction, I wrote a set of small PHP scripts which fetch the data from the database and write it into the config files. The scripts rely on PEAR::DB, so you need to have PHP and PEAR installed. If you don’t have done so yet, install them with apt:

$ aptitude install php5-cli php5-mysql php-pear php-db

Next install the scripts – I put them into a /home/vmail/mailserverscripts/. You can download them from my SVN repository or simply check them out with SVN:

svn co http://svn.ailoo.net/dev/public/mailserverscripts/

Then, copy config.example.inc.php to config.inc.php and edit it according to your needs. There shouldn’t bee too much to change, just the database credentials, some default options and possibly the paths to config files.

It’s important to change file permissions as all scripts should run under the user vmail and nobody should be able to read the config file.

chown vmail.vmail /home/vmail/mailserverscripts -R
chmod 0600 /home/vmail/mailserverscripts/config.inc.php

Test the setup

Now that all is in place, try adding some records to the database and executing the scripts as user vmail:

$ su vmail
$ php /home/vmail/mailserverscripts/fetchmail.php
$ php /home/vmail/mailserverscripts/sieve.php

Then check if the config files .fetchmailrc in vmail’s home and .dovecot.sieve in the virtual mail user’s home were generated.

Running scripts as cronjob

To generate the config files automatically every few minutes, set up cronjobs for the user vmail. Edit the vmail’s crontab with crontab -e. My crontab looks like this:

*/5 * * * * php /home/vmail/mailserverscripts/fetchmail.php
*/5 * * * * php /home/vmail/mailserverscripts/sieve.php
*/2 * * * * /usr/bin/fetchmail > /dev/null 2>&1

Fetchmail and sieve config files are regenerated every 5 minutes, fetchmail looks for new mail on foreign servers every 2 minutes. This values depend on the amount of mail accounts and sieve rules you have to configure.

Conclusion

It’s a quite simple setup which does its work. However, this may not be useful for bigger installations, as the scripts regenerate the config files without checking for modifications. So if you got many rules to fetch, it would make sense to extend the scripts by some sort of caching. Another aspect is security: fetchmail passwords are stored in plain text in the database and in the .fetchmailrc file, so take care of setting the right file permissions. The sieve script loads the three extensions ["fileinto", "reject", "vacation"] by default, which may not be very efficient and could be replaced by some sort of mechanism which checks for the needed extensions.

Another thing you may notice: if you got sieve rules set for a user and delete all of them, on the next run the script will not operate on the user’s config and therefore will not clean the config file. This results in old rules remaining set until you create new rules for that user. A simple workaround for this bug is creating an empty rule and waiting until the script ran at least once, which will create an empty config file. This and other bugs/errors may be corrected in future versions.

Any ideas and suggestions are welcome :)

License for the scripts: GNU General Public License (GPL)

  • Toby G.

    Great tutorial, didn’t try it tough. stumbpled across it while searching for a sieve server.

    Just to let you know what I m doing:

    I will probably use your fetchmail part. I use horde webmail and somtimes squirrelmail. So I liked to generate filter rule with horde.

    The problem with horde for me was that I m running apache with suexec/fastcgi and I didnt want to mess with chmods/chowns. So I couldnt use the file driver/backend from hord.

    So I installed pysieved because the debian etch dovecot package doesnt ship with the managesieve protocol. (pysieded is from a dovecot developer btw). Now I m using the sieveserver to write .dovecot.sieve

    I also think about using horde as the DB frontend for fetchmail.

    If your interrested in my pysieved config let me know and I will write a write up. took me some time to figure this out.

    Toby

  • Toby G.

    Great tutorial, didn’t try it tough. stumbpled across it while searching for a sieve server.

    Just to let you know what I m doing:

    I will probably use your fetchmail part. I use horde webmail and somtimes squirrelmail. So I liked to generate filter rule with horde.

    The problem with horde for me was that I m running apache with suexec/fastcgi and I didnt want to mess with chmods/chowns. So I couldnt use the file driver/backend from hord.

    So I installed pysieved because the debian etch dovecot package doesnt ship with the managesieve protocol. (pysieded is from a dovecot developer btw). Now I m using the sieveserver to write .dovecot.sieve

    I also think about using horde as the DB frontend for fetchmail.

    If your interrested in my pysieved config let me know and I will write a write up. took me some time to figure this out.

    Toby

  • Toby G.
  • Toby G.
  • Hi and thanks for your comment. Yes, actually I’d be very interested in the pysieved setup. This sounds like a way better approach for Sieve management, especially because I still didn’t write any real frontend for the DB ;)

    Mathias

  • Hi and thanks for your comment. Yes, actually I’d be very interested in the pysieved setup. This sounds like a way better approach for Sieve management, especially because I still didn’t write any real frontend for the DB ;)

    Mathias

  • Hi,

    I wrote an interface based on Christoph Haas’s and your MySQL tables.
    You can find it at openMM.de.
    I’d be happy to hear your thoughts about it.

    Greetings,
    Rainer

  • Hi,

    I wrote an interface based on Christoph Haas’s and your MySQL tables.
    You can find it at openMM.de.
    I’d be happy to hear your thoughts about it.

    Greetings,
    Rainer

  • Mun

    Hi, I tried your write up in setting up fetchamil together with virtual user (which I personally feel it is a very good idea).

    However, the fetchmail part did not work. Mails from the remote server did not fetch into my local server virtual user.

    Below is my fetchmailrc and mail.log

    Nov 17 23:15:31 server1 fetchmail[6765]: 103 messages (103 seen) for remote_email at pop3.remotemailserver.
    Nov 17 23:15:32 server1 fetchmail[6765]: sleeping at Monday 17,November,2008 11:15:32 PM SGT for 60 seconds
    Nov 17 23:16:32 server1 fetchmail[6765]: awakened at Monday 17,November,2008 11:16:32 PM SGT

    fetchmailrc

    set syslog
    set daemon 60
    set postmaster postmaster@localserver
    set no bouncemail

    defaults:
    timeout 60

    poll pop3.remotemailserver:
    with nodns,
    with protocol pop3,
    user “remoteuser@remotemailserver” is localuser@localserver here,
    with password password;

    I really appreciate you can assist

    Thank you in advance.

  • Mun

    Hi, I tried your write up in setting up fetchamil together with virtual user (which I personally feel it is a very good idea).

    However, the fetchmail part did not work. Mails from the remote server did not fetch into my local server virtual user.

    Below is my fetchmailrc and mail.log

    Nov 17 23:15:31 server1 fetchmail[6765]: 103 messages (103 seen) for remote_email at pop3.remotemailserver.
    Nov 17 23:15:32 server1 fetchmail[6765]: sleeping at Monday 17,November,2008 11:15:32 PM SGT for 60 seconds
    Nov 17 23:16:32 server1 fetchmail[6765]: awakened at Monday 17,November,2008 11:16:32 PM SGT

    fetchmailrc

    set syslog
    set daemon 60
    set postmaster postmaster@localserver
    set no bouncemail
    
    defaults:
     timeout 60
    
    poll pop3.remotemailserver:
            with nodns,
            with protocol pop3,
            user "remoteuser@remotemailserver" is localuser@localserver here,
            with password password;

    I really appreciate you can assist

    Thank you in advance.

  • @Mun: did you follow the guide and use the scripts or did you create the fetchmail config yourself (seems to use another syntax than my script’s one).

    Here’s an example .fetchmailrc which is created by my script and run by the user vmail (local domain example.com, remote domain example.net):

    set syslog
    set postmaster postmaster@example.com
    set no bouncemail

    defaults:
    timeout 300

    # ————————-

    poll mail.example.net
    proto pop3
    user “john.smith@example.net”
    pass “secret”
    is john.smith@example.com

  • @Mun: did you follow the guide and use the scripts or did you create the fetchmail config yourself (seems to use another syntax than my script’s one).

    Here’s an example .fetchmailrc which is created by my script and run by the user vmail (local domain example.com, remote domain example.net):

    set syslog
    set postmaster postmaster@example.com
    set no bouncemail
    
    defaults:
    timeout 300
    
    # -------------------------
    
    poll mail.example.net
            proto pop3
            user "john.smith@example.net"
            pass "secret"
            is john.smith@example.com
  • michael

    Hello

    when i run the fetchmail.php skript i become an empty .fetchmailrc in the vmail dir.

    I have filled the tabel virtual_fetchmail with the remoteserver, user and password.

    My view_fetchmail table:
    Field name Type Allow nul Key Default Extras
    id int(11) No Primary auto_increment
    user_id int(11) No Indexed
    active tinyint(1) No None 1
    options varchar(50) No None keep
    remoteserver varchar(50) No None pop.gmx.de
    remoteuser varchar(50) No None xxUserxx
    remotepass varchar(50) No None xxpassxx

    When i start mysql consol and run view_fetchmail i see nothing.

    Could you explane it to me. I´m not the best in mysql and php.

  • michael

    Hello

    when i run the fetchmail.php skript i become an empty .fetchmailrc in the vmail dir.

    I have filled the tabel virtual_fetchmail with the remoteserver, user and password.

    My view_fetchmail table:
    Field name Type Allow nul Key Default Extras
    id int(11) No Primary auto_increment
    user_id int(11) No Indexed
    active tinyint(1) No None 1
    options varchar(50) No None keep
    remoteserver varchar(50) No None pop.gmx.de
    remoteuser varchar(50) No None xxUserxx
    remotepass varchar(50) No None xxpassxx

    When i start mysql consol and run view_fetchmail i see nothing.

    Could you explane it to me. I´m not the best in mysql and php.

  • Junta

    Hi,

    first off, thank you Mathias for making this great how-to. It works flawless for what’s it supposed to do.

    I do have an issue to use this in combination with ISP-mail server how to. I have some virtual users on my mailserver which need to receive pop3 mail which is fetched by Fetchmail. If I use your script however, it just pulls all mail in from the external pop3 box and delivers everything to ALL virtual users, so not just the intended recipient. I’ve googled all over the place, but can’t find a solution to this…

    maybe you have some experience with this? It’s driving me mad :D

    Many thansk in advance

    Kind regards,
    Junta

  • Hi,

    first off, thank you Mathias for making this great how-to. It works flawless for what’s it supposed to do.

    I do have an issue to use this in combination with ISP-mail server how to. I have some virtual users on my mailserver which need to receive pop3 mail which is fetched by Fetchmail. If I use your script however, it just pulls all mail in from the external pop3 box and delivers everything to ALL virtual users, so not just the intended recipient. I’ve googled all over the place, but can’t find a solution to this…

    maybe you have some experience with this? It’s driving me mad :D

    Many thansk in advance

    Kind regards,
    Junta

  • Tobias

    Well done you’re implementation it’s running very good. But I have a question about the sieve scripts:

    Where could I easy test them? (Or better – how to test sieve scripts …)

    For example easy filter methods are not the problem. I have problems with conntecting 2 or more conditions with “allof” / “anyof” – any hints?

    Tobias

  • Tobias

    Well done you’re implementation it’s running very good. But I have a question about the sieve scripts:

    Where could I easy test them? (Or better – how to test sieve scripts …)

    For example easy filter methods are not the problem. I have problems with conntecting 2 or more conditions with “allof” / “anyof” – any hints?

    Tobias

  • Jens Kowalski

    I think the following VIEW for sieve can solve the “cleaning the sieve config files after deleting all sieve rules”-BUG:

    CREATE VIEW `view_sieve` AS select `virtual_sieve`.`id` AS `id`, COALESCE(`virtual_sieve`.`active`,1) AS `active`,`virtual_users`.`user` AS `user`,`virtual_domains`.`name` AS `domain`,COALESCE(`virtual_sieve`.`rules`,'') AS `rules` from ((`virtual_sieve` right join `virtual_users` on((`virtual_sieve`.`user_id` = `virtual_users`.`id`))) left join `virtual_domains` on((`virtual_users`.`domain_id` = `virtual_domains`.`id`)));

    It works fine for me…

  • Jens Kowalski

    I think the following VIEW for sieve can solve the “cleaning the sieve config files after deleting all sieve rules”-BUG:

    CREATE VIEW `view_sieve` AS select `virtual_sieve`.`id` AS `id`, COALESCE(`virtual_sieve`.`active`,1) AS `active`,`virtual_users`.`user` AS `user`,`virtual_domains`.`name` AS `domain`,COALESCE(`virtual_sieve`.`rules`,'') AS `rules` from ((`virtual_sieve` right join `virtual_users` on((`virtual_sieve`.`user_id` = `virtual_users`.`id`))) left join `virtual_domains` on((`virtual_users`.`domain_id` = `virtual_domains`.`id`)));

    It works fine for me…

  • Tobias

    I have installed everything like the tutorial of Christoph Haas and your tutorial but i have some problems:

    First in my dovecot-deliver.log I got an error like this:
    “deliver(user@example.com): “2009-06-11 17:19:03 “Error: chdir(/home/vmail/domains/example.com/usermail) failed: No such file or directory
    deliver(user@example.com): “2009-06-11 17:19:03 “Info: msgid=: saved mail to INBOX”

    There is one “/” missing between “user” and “mail” -> in my config is everything fine:
    “mail_location = /home/vmail/domains/%d/%u/mail”

    Where could be the error?

    And the second problem regards the .dovecot.sieve is not compiled and recognized … (pehaps there is a relation between the error described above).

  • Tobias

    I have installed everything like the tutorial of Christoph Haas and your tutorial but i have some problems:

    First in my dovecot-deliver.log I got an error like this:
    “deliver(user@example.com): “2009-06-11 17:19:03 “Error: chdir(/home/vmail/domains/example.com/usermail) failed: No such file or directory
    deliver(user@example.com): “2009-06-11 17:19:03 “Info: msgid=: saved mail to INBOX”

    There is one “/” missing between “user” and “mail” -> in my config is everything fine:
    “mail_location = /home/vmail/domains/%d/%u/mail”

    Where could be the error?

    And the second problem regards the .dovecot.sieve is not compiled and recognized … (pehaps there is a relation between the error described above).

  • Tobias

    Okay, first problem is solved. It was in dovecot.conf -> the mail_location was correct, but there is also in the section “userdb static” a part where to configure the mail location.

    This works now fine.

    But the other problem still exists:
    /home/vmail/domains/example.de/user/.dovecot.sieve is created correct – but if I send a mail there is no compiled file of .dovecote.sieve or an error file. – What should I do now?

    • Tobias, sorry for the delay – have been quite busy for the past days. Have you activated the Sieve plugin (cmusieve) in Dovecot’s config? From the original tutorial:

      protocol lda {
      log_path = /home/vmail/dovecot-deliver.log
      auth_socket_path = /var/run/dovecot/auth-master
      postmaster_address = postmaster@example.com
      mail_plugins = cmusieve
      global_script_path = /home/vmail/globalsieverc
      }

  • Tobias

    Okay, first problem is solved. It was in dovecot.conf -> the mail_location was correct, but there is also in the section “userdb static” a part where to configure the mail location.

    This works now fine.

    But the other problem still exists:
    /home/vmail/domains/example.de/user/.dovecot.sieve is created correct – but if I send a mail there is no compiled file of .dovecote.sieve or an error file. – What should I do now?

    • Tobias, sorry for the delay – have been quite busy for the past days. Have you activated the Sieve plugin (cmusieve) in Dovecot’s config? From the original tutorial:

      protocol lda {
          log_path = /home/vmail/dovecot-deliver.log
          auth_socket_path = /var/run/dovecot/auth-master
          postmaster_address = postmaster@example.com
          mail_plugins = cmusieve
          global_script_path = /home/vmail/globalsieverc
      }
  • Tobias

    Yes, I did – but i found the error a few days ago …
    I used the folder structure of the original tutorial “/home/vmail/%d/%n/” and then I did not only change “mail_location”. I changed also the home part in the following code – but this was definitly wrong.

    userdb static {
    args = uid=5000 gid=5000 home=/home/vmail/domains/%d/%n/ allow_all_users=yes
    }

    No everything works again fine – but it was hard to find this error. I started the original tutorial again from beginning to find this problem. :)

  • Tobias

    Yes, I did – but i found the error a few days ago …
    I used the folder structure of the original tutorial “/home/vmail/%d/%n/” and then I did not only change “mail_location”. I changed also the home part in the following code – but this was definitly wrong.

    userdb static {
    args = uid=5000 gid=5000 home=/home/vmail/domains/%d/%n/ allow_all_users=yes
    }

    No everything works again fine – but it was hard to find this error. I started the original tutorial again from beginning to find this problem. :)

  • Pingback: ISPmail tutorial for Debian Lenny @()

  • hsw

    Thanks for the how to. This is exactly what I needed. I’ve been looking for one for getmail but didnt find any.

    Can I ask you 1 question though? I’ve followed the how to from end to end. I’ve read it several times, and it seems that my setup is working except that I’m getting (in /var/log/mail.log):
    fetchmail: Server CommonName: my-ssh.com:mydomain.com ….
    and ended up with AUTHFALIED everytime I run fetchmail.

    Can you tell me what I’m, doing wrong? I’ve also tried this on gmail, but was unable to retrieve anything.

    • @hsw Thanks for the feedback. I was experimenting with getmail too, but this was the solution I got working at last.

      Regarding you fetchmail problem: I was experiencing problems with a specific host (an exchange server with a self-signed certificate) too. Altough the login data was correct and I could login via telnet or a mail client without problems the server still claimed the username/password was wrong and the fetchmail call ended up with AUTHFAILED. The only solution I found to solve this was adding the line auth password (as found on this german thread: http://www.ureader.de/msg/1198736.aspx) to the account config. Unfortunately, the script can’t do this at the moment, but as soon as I got some spare time I’ll add the functionality to the script. Meanwhile you could try to add the line manually to your generated .fetchmailrc to try if this makes the difference. If not, please try to post the full (anonymized) log from a fetchmail call. I’m not really familiar with fetchmail either, just got this to work as wanted with the described config. But maybe we could investigate further with the logfile.

      Concerning the general howto: I just implemented sieve rule management with Dovecot’s ManageSieve server and Horde/Thunderbird as frontend…works really nice and is much more comfortable than the solution descibed here. I’ll post a howto on this blog as soon as I find the time to document the setup.

      — Mathias

    • FYI: I just managed to fetch from my gmail account with the following config (and to spam my inbox with about 250 unread messages from the past year ;)).

      set syslog
      set postmaster postmaster@example.com
      set no bouncemail

      defaults:
      timeout 300
      sslproto ssl23

      # ————————-

      poll pop.googlemail.com
      proto pop3
      port 995
      user “username@gmail.com”
      pass “password”
      is username@example.com
      ssl

      Meanwhile you can help yourself by editing the .fetchmailrc manually, I’ll update the script as soon as possible (or don’t hesitate to send me your patches if you get your hands dirty ;))

      — Mathias

  • hsw

    Thanks for the how to. This is exactly what I needed. I’ve been looking for one for getmail but didnt find any.

    Can I ask you 1 question though? I’ve followed the how to from end to end. I’ve read it several times, and it seems that my setup is working except that I’m getting (in /var/log/mail.log):
    fetchmail: Server CommonName: my-ssh.com:mydomain.com ….
    and ended up with AUTHFALIED everytime I run fetchmail.

    Can you tell me what I’m, doing wrong? I’ve also tried this on gmail, but was unable to retrieve anything.

    • @hsw Thanks for the feedback. I was experimenting with getmail too, but this was the solution I got working at last.

      Regarding you fetchmail problem: I was experiencing problems with a specific host (an exchange server with a self-signed certificate) too. Altough the login data was correct and I could login via telnet or a mail client without problems the server still claimed the username/password was wrong and the fetchmail call ended up with AUTHFAILED. The only solution I found to solve this was adding the line auth password (as found on this german thread: http://www.ureader.de/msg/1198736.aspx) to the account config. Unfortunately, the script can’t do this at the moment, but as soon as I got some spare time I’ll add the functionality to the script. Meanwhile you could try to add the line manually to your generated .fetchmailrc to try if this makes the difference. If not, please try to post the full (anonymized) log from a fetchmail call. I’m not really familiar with fetchmail either, just got this to work as wanted with the described config. But maybe we could investigate further with the logfile.

      Concerning the general howto: I just implemented sieve rule management with Dovecot’s ManageSieve server and Horde/Thunderbird as frontend…works really nice and is much more comfortable than the solution descibed here. I’ll post a howto on this blog as soon as I find the time to document the setup.

      — Mathias

    • FYI: I just managed to fetch from my gmail account with the following config (and to spam my inbox with about 250 unread messages from the past year ;)).

      set syslog
      set postmaster postmaster@example.com
      set no bouncemail
      
      defaults:
      timeout 300
      sslproto ssl23
      
      # -------------------------
      
      poll pop.googlemail.com
              proto pop3
              port 995
              user "username@gmail.com"
              pass "password"
              is username@example.com
              ssl

      Meanwhile you can help yourself by editing the .fetchmailrc manually, I’ll update the script as soon as possible (or don’t hesitate to send me your patches if you get your hands dirty ;))

      — Mathias

  • hsw

    “I’m not really familiar with fetchmail either” ??? You’re kidding me right? I think I need to adjust my understanding on what “familiar” is….

    And Thank You for the google script too. I’ll give this a go.

    • Well I’m familiar with some kind of software when I don’t have to lookup every config option on the web or on manpages :D. On fetchmail it’s more trial and error than knowing what I’m doing ;)

      — Mathias

  • hsw

    “I’m not really familiar with fetchmail either” ??? You’re kidding me right? I think I need to adjust my understanding on what “familiar” is….

    And Thank You for the google script too. I’ll give this a go.

    • Well I’m familiar with some kind of software when I don’t have to lookup every config option on the web or on manpages :D. On fetchmail it’s more trial and error than knowing what I’m doing ;)

      — Mathias

  • hsw

    Hi Again Mathias,

    I’m a bit thick when it comes to this linux thing (I’ve been using linux for almost a year now)…

    I’m having issues with the permission of .dovecot.sieve that keeps being denied. I checked using “ln -l” and it says the file is owned by root.
    Should I change it? What should I change it to?

    The fetchmail & sieve cron is also generating permission error. I’m getting these errors via email It says:

    fopen(/home/vmail/mydomain/myuser/.dovecot.sieve): failed to open stream:
    Permission denied in /home/vmail/mailserverscripts/sieve.php on line 37

    fwrite(): supplied argument is not a valid stream resource in
    /home/vmail/mailserverscripts/sieve.php on line 38

    fclose(): supplied argument is not a valid stream resource in
    /home/vmail/mailserverscripts/sieve.php on line 39

    chmod(): Operation not permitted in /home/vmail/mailserverscripts/sieve.php
    on line 41

    Thanks so much for helping

  • hsw

    Hi Again Mathias,

    I’m a bit thick when it comes to this linux thing (I’ve been using linux for almost a year now)…

    I’m having issues with the permission of .dovecot.sieve that keeps being denied. I checked using “ln -l” and it says the file is owned by root.
    Should I change it? What should I change it to?

    The fetchmail & sieve cron is also generating permission error. I’m getting these errors via email It says:

    fopen(/home/vmail/mydomain/myuser/.dovecot.sieve): failed to open stream:
    Permission denied in /home/vmail/mailserverscripts/sieve.php on line 37

    fwrite(): supplied argument is not a valid stream resource in
    /home/vmail/mailserverscripts/sieve.php on line 38

    fclose(): supplied argument is not a valid stream resource in
    /home/vmail/mailserverscripts/sieve.php on line 39

    chmod(): Operation not permitted in /home/vmail/mailserverscripts/sieve.php
    on line 41

    Thanks so much for helping

  • hsw

    Ah, never mind. The problem was I initially run the php command using root, so perhaps that was why the .dovecot.sieve file was owned by root and not vmail.

    So I deleted .dovecot.sieve and re-run the php commands under vmail and it works. No more error.

    The last thing (I hope) that needed to be fixed is that spams are still being quarantined and not sieved to spam folder (as it was supposed to according to the ISP Mailserver’s globalsieverc script… Any ideas?

  • hsw

    Ah, never mind. The problem was I initially run the php command using root, so perhaps that was why the .dovecot.sieve file was owned by root and not vmail.

    So I deleted .dovecot.sieve and re-run the php commands under vmail and it works. No more error.

    The last thing (I hope) that needed to be fixed is that spams are still being quarantined and not sieved to spam folder (as it was supposed to according to the ISP Mailserver’s globalsieverc script… Any ideas?

  • Tobias

    Is it possible to use this setup with Debian Lenny? (For example the ISPmail lenny tutorial did not use the views like in the etch version … and virtual_users.user did not exists – but I know that virtual_users.email is correct.)

    • TobyG

      What for? I m not sure, but wasnt the problem with the etch setup, that the dovecot etch package had no sieve extension? I figure that should be changed by now in lenny.
      I havent upgraded my server yet, so I cant really tell!

    • It is possible, but (as you mentioned) the SQL schema has changed in the Lenny tutorial. I’m still using the old schema on my Lenny server, so I can’t really tell you what you have to change. However, the scripts used to generate the config files are quite simple, so you should be able to change them easily to match your setup. I guess the easiest solution would be to update the views used in this tutorial to reflect the new DB schema.

      — Mathias

  • Tobias

    Is it possible to use this setup with Debian Lenny? (For example the ISPmail lenny tutorial did not use the views like in the etch version … and virtual_users.user did not exists – but I know that virtual_users.email is correct.)

    • TobyG

      What for? I m not sure, but wasnt the problem with the etch setup, that the dovecot etch package had no sieve extension? I figure that should be changed by now in lenny.
      I havent upgraded my server yet, so I cant really tell!

    • It is possible, but (as you mentioned) the SQL schema has changed in the Lenny tutorial. I’m still using the old schema on my Lenny server, so I can’t really tell you what you have to change. However, the scripts used to generate the config files are quite simple, so you should be able to change them easily to match your setup. I guess the easiest solution would be to update the views used in this tutorial to reflect the new DB schema.

      — Mathias

  • Tobias

    The problem isn’t sieve … I would like to fetch mails from internet server to my intranet server and then distribut it to local clients.

    So I’ trying at the moment to use only the fetchmail options ;-)

    • TobyG

      Sorry dude, I forgot what this post is about. good luck on that!

  • Tobias

    The problem isn’t sieve … I would like to fetch mails from internet server to my intranet server and then distribut it to local clients.

    So I’ trying at the moment to use only the fetchmail options ;-)

    • TobyG

      Sorry dude, I forgot what this post is about. good luck on that!

  • Great Tutorial Dude!

    I was lookin’ for a fetchmail-mysql-backend, but your way is nice, too.

    This made my day :-)

  • Great Tutorial Dude!

    I was lookin’ for a fetchmail-mysql-backend, but your way is nice, too.

    This made my day :-)

  • andrew

    Thanks for your great tutor.
    I am using this method for my server now and managing more than 10 accounts so far.

    Recently, I found a problem when I try to use postfix as an smtp server in the same server.

    Say, there are some email accounts below I am handling in this server:
    andy@gmail.com
    peter@gmail.com
    mary@gmail.com

    When I send email to andy@gmail.com via the smtp server in the same server, the email can reach immediately without going to gmail official server.

    That’s great with this kind of instand action.

    However, when I send email, via the smtp server, to other account other than the account i manage in this server, say watermelon@gmail.com, I got an error message something like:
    “User unknown in virtual mailbox table.”

    If I send to watermelon@hotmail.com, there is no problem.

    In my understand, after the setting, my server treat “gmail.com” is my own domain. For all the email sending via my server to gmail.com, it will directly deliver to the user in my server.

    This make me headache.
    At this moment, I am not able to send email via my server to any other friends’ gmail account.

    Are there any way to solve this problem?

  • andrew

    Thanks for your great tutor.
    I am using this method for my server now and managing more than 10 accounts so far.

    Recently, I found a problem when I try to use postfix as an smtp server in the same server.

    Say, there are some email accounts below I am handling in this server:
    andy@gmail.com
    peter@gmail.com
    mary@gmail.com

    When I send email to andy@gmail.com via the smtp server in the same server, the email can reach immediately without going to gmail official server.

    That’s great with this kind of instand action.

    However, when I send email, via the smtp server, to other account other than the account i manage in this server, say watermelon@gmail.com, I got an error message something like:
    “User unknown in virtual mailbox table.”

    If I send to watermelon@hotmail.com, there is no problem.

    In my understand, after the setting, my server treat “gmail.com” is my own domain. For all the email sending via my server to gmail.com, it will directly deliver to the user in my server.

    This make me headache.
    At this moment, I am not able to send email via my server to any other friends’ gmail account.

    Are there any way to solve this problem?

  • toni

    i can’t access the SVN repository.
    it asks for username and pass.
    pls help

    toni

    • Ah sorry i was playing around with the repository last weekend, didn’t fix the permissions yet. I’ll fix this soon.

      Regards

    • Alright, the SVN is up and running again.

  • toni

    i can’t access the SVN repository.
    it asks for username and pass.
    pls help

    toni

    • Ah sorry i was playing around with the repository last weekend, didn’t fix the permissions yet. I’ll fix this soon.

      Regards

    • Alright, the SVN is up and running again.

  • I’m trying to install the script on a debian squeeze, but when i write the sql command VIEW for view_sieve, i have this error message:

    ERROR 1054 (42S22): Unknown column ‘virtual_users.user’ in ‘field list’

    What’s wrong?!?!

    • I’m sorry.. the name of the table was workng… in the new tutorial it is “email” and not “user” anymore. So you only need to substitute the
      “virtual_users.user,” with “virtual_users.email,

    • However, the script doesn’t work :(


      vmail@serverino:/home/stefano$ php /var/vmail/mailserverscripts/fetchmail.php
      PHP Fatal error: Uncaught exception 'DB_DSN_Exception' with message 'DB Error: connect failed' in /var/vmail/mailserverscripts/db.inc.php:35
      Stack trace:
      #0 /var/vmail/mailserverscripts/db.inc.php(18): MailserverDB->__construct()
      #1 /var/vmail/mailserverscripts/config.inc.php(14): MailserverDB::getInstance()
      #2 /var/vmail/mailserverscripts/fetchmail.php(4): require_once('/var/vmail/mail...')
      #3 {main}
      thrown in /var/vmail/mailserverscripts/db.inc.php on line 35
      vmail@serverino:/home/stefano$

    • Hey Stefano,

      yeah, the DB structure changed from the Etch to the Lenny ISPMail tutorial, so some changes in the db queries are necessary. I’m still running this setup and didn’t have time yet to write a follow-up tutorial.

      To the script issue: it’s using the standard PEAR_Db class, which should work fine, db.inc.php is just a simple wrapper around the class. Are you sure your connection string is correct?

      Regards,
      Mathias

    • Hi mathias,

      thanks for your quick reply.
      I don’t know if the connection string is correct.. how can i check it?
      I only know that dbuser, dbname, dbpass and other variables for connection are ok…

  • I’m trying to install the script on a debian squeeze, but when i write the sql command VIEW for view_sieve, i have this error message:

    ERROR 1054 (42S22): Unknown column ‘virtual_users.user’ in ‘field list’

    What’s wrong?!?!

    • I’m sorry.. the name of the table was workng… in the new tutorial it is “email” and not “user” anymore. So you only need to substitute the
      “virtual_users.user,” with “virtual_users.email,

    • However, the script doesn’t work :(


      vmail@serverino:/home/stefano$ php /var/vmail/mailserverscripts/fetchmail.php
      PHP Fatal error: Uncaught exception 'DB_DSN_Exception' with message 'DB Error: connect failed' in /var/vmail/mailserverscripts/db.inc.php:35
      Stack trace:
      #0 /var/vmail/mailserverscripts/db.inc.php(18): MailserverDB->__construct()
      #1 /var/vmail/mailserverscripts/config.inc.php(14): MailserverDB::getInstance()
      #2 /var/vmail/mailserverscripts/fetchmail.php(4): require_once('/var/vmail/mail...')
      #3 {main}
      thrown in /var/vmail/mailserverscripts/db.inc.php on line 35
      vmail@serverino:/home/stefano$

    • Hey Stefano,

      yeah, the DB structure changed from the Etch to the Lenny ISPMail tutorial, so some changes in the db queries are necessary. I’m still running this setup and didn’t have time yet to write a follow-up tutorial.

      To the script issue: it’s using the standard PEAR_Db class, which should work fine, db.inc.php is just a simple wrapper around the class. Are you sure your connection string is correct?

      Regards,
      Mathias

    • Hi mathias,

      thanks for your quick reply.
      I don’t know if the connection string is correct.. how can i check it?
      I only know that dbuser, dbname, dbpass and other variables for connection are ok…

  • Is the svn working? because i’m trying to checkout and i get timeout.

    thanks!

    • Hey,

      yes it’s working – maybe you have a problem with my self-signed certificate on the SVN server?

      Cheers