[imapfilter-devel] imapfilter stressing mail server

[Josh Cronemeyer]

On Sat, 2005-05-21 at 15:41 +0300, Lefteris Chatzibarbas wrote:
> On Fri, May 20, 2005 at 12:17:51PM -0500, Josh Cronemeyer wrote:
> > Hi,
> > 
> > Recently i've moved over to an amd64 macine.  I'm running fc3, imapfilter v. 
> > 1.0.1.  Previously I was running a 32 bit debian macine but the same version 
> > of imapfilter.
> > 
> > After the switch I noticed that my filters were taking much more time to 
> > finish.  After running with the -d switch I got the following output:
> > 	1013 OK [READ-WRITE] Completed^M
> > 
> > 	sending command (4):
> > 
> > 	516E70 (null)SEARCH ^M
> > 
> > 	getting response (4):
> > 
> > 	516E70 BAD Null command^M
> > 
> > 	getting response (4):
> > 
> > 	* OK heracles Cyrus IMAP4 v2.1.9 server ready^M
> > 
> > 	sending command (4):
> >
> > At the point where the null search is attempted the process pauses for a long 
> > time.  I have run the same config.lua on my old machine and that null search 
> > never happens.  When I remove that line from the config.lua I get the null 
> > search somewhere else later on, so I am pretty confident it isn't a problem 
> > with my config file.    
> > 
> > Also we have recently begun to have problems with the load on our mailserver.  
> > This problem appeared about the same time as our staff were moving over to 
> > amd64 workstations.  It seems to me that when imapfilters hangs for a minute 
> > or so it is pulling alot of cycles on the mailserver.
> > 
> > Any ideas?  My only thought is that there is some 64 bit issue happening here 
> > (*sigh* it is always some 64 bit issue these days).
> > 
> > thanks in advance
> 
> That "516E70" command tag and the "(null)" string there don't make
> sense... Hmm...
> 
> First of all, can you send me (in private mail) the whole debug file and
> maybe the configuration file you are using?  Although, I don't think it
> will help much, considering the above strange output...
> 
> Is anyone in the list using imapfilter on amd64+linux machines? Any
> problems?
> 
> I could only help if I had access to your IMAP server, something like a
> guest account to test imapfilter with your amd64+linux IMAP server.  And
> maybe next a shell account to an amd64+linux machine to run imapfilter
> from there.  Of course, you could always dive in yourself and try to
> figure what goes wrong... :-/
> 
> 

Ok,  

A colleague of mine figured out how to prevent That "516E70" command tag
and the "(null)" string from showing up.  By keeping filter definitions
less than 20 lines long you don't have any problems.  When a filter has
more than 20 lines of conditions OR'd together it causes that null
search condition to occur.  Example of a bad filter.

-- My Apps!!!  FLAG the hell out of em.
legislative = {
        invert=true,
        {'to "legis at xxx.xxx"'},
        {'subject "legisjrnl:"', 'to "kic-error at xxx.xxx"'},
        {'subject "legisjrnl:"', 'to "kic-notice at xxx.xxx"'},
        {'subject "legisregis:"','to "kic-error at xxx.xxx"'},
        {'subject "legisregis:"','to "kic-notice at xxx.xxx"'},
        {'subject "legiskars:"','to "kic-error at xxx.xxx"'},
        {'subject "legiskars:"','to "kic-notice at xxx.xxx"'},
        {'subject "ethicsopn:"','to "kic-error at xxx.xxx"'},
        {'subject "ethicsopn:"','to "kic-notice at xxx.xxx"'},
        {'subject "legisportal:"','to "kic-error at xxx.xxx"'},
        {'subject "legisportal:"','to "kic-notice at xxx.xxx"'},
        {'subject "legiscalen:"','to "kic-error at xxx.xxx"'},
        {'subject "legiscalen:"','to "kic-notice at xxx.xxx"'},
        {'subject "statutes:"','to "kic-error at xxx.xxx"'},
        {'subject "statutes:"','to "kic-notice at xxx.xxx"'},
        {'subject "legbill:"','to "kic-error at xxx.xxx"'},
        {'subject "legbill:"','to "kic-notice at xxx.xxx"'},
        {'subject "house:"','to "kic-error at xxx.xxx"'},
        {'subject "house:"','to "kic-notice at xxx.xxx"'},
        {'subject "senate:"','to "kic-error at xxx.xxx"'},
        {'subject "senate:"','to "kic-notice at xxx.xxx"'},
        {'subject "statutes:"','to "kic-error at xxx.xxx"'},
        {'subject "statutes:"','to "kic-notice at xxx.xxx"'},
        {'subject "legbtrack:"','to "kic-error at xxx.xxx"'},
        {'subject "legbtrack:"','to "kic-notice at xxx.xxx"'},
        {'subject ":Legis File Updating"','to "kic-notice at xxx.xxx"'}
}


if i break that in half into say, legislativeA and legislativeB
everything runs just fine.  When I was running this on a 32 bit system I
wasn't having any problems.  What could be causing this?  I am running
imapfilters 1.0.1 and lua 5.0.2 i believe.

I also don't know exactly what is happening between my machine and the
imap server when that null search is running.  

Thanks