[imapfilter-devel] INBOX confusion

[Lefteris Chatzimparmpas]

Happy new year everybody!

Ezsra McDonald wrote:
> Greetings imapfilter gurus,
> 
> I have been using the basic config for imapfilter for a couple years.
> I have a problem that I need to work out. I was hoping you could help.
> Please keep in mind that I am not a programmer.
> 
> When I run filters on my inbox some messages get filtered incorrectly.
> I think I have traced the problem to messages that get delivered while
> a filter is being processed. If the message arraves at a point after
> which it would have been filtered correctly it will get picked up by
> my "junk" filter.
> 
> The question is, how can I get a "snapshot" of the messages in the
> inbox at the time of execution and apply all filters only to this
> snapshot of messages?

Hello Ezsra,

I have two or three solutions to suggest...

> What I have done in the past was move all my msgs to a ztmp folder and
> process them from there. The problem is that now I have a PDA and our
> sysadmin only allows pop from outside of our network. This means all
> of my interesting messages must remain in the inbox rather than some
> other folder of my choice.

The first solution is a variation of your solution.  You move all
messages currently in "INBOX" to some other folder, for example a folder
named "tmp", you do all the filtering in this snapshot folder called
"tmp", and then you move the remaining (interesting) messages back to
"INBOX".

Another solution which does not require moving messages to some other
folder is you take a snapshot of any new messages inside "INBOX" and you
store them to a Lua table (I will show you how).  Every time you
move/delete messages, you subtract the messages you moved/deleted from
the original snapshot table.  Thus in the end, when you move/delete all
the remaining junk messages to another folder, you just move those
messages that remain in the snapshot table and not all the messages
currently in "INBOX" (which may include messages that came after you
started processing the messages).

You just have to define a new function in your configuration file:

   function subtract(all, part)
       for key, value in all do
           for k, v in part do
               if (value == v) then
                   table.remove(all, key)
               end
           end
       end
   end

Then you can use this function in your config file like this:

   -- take a snapshot of current messages
   snapshot = match(myaccount, 'INBOX', {})

   -- move messages that match a filter as normal
   result = match(myaccount, 'INBOX', myfilter1)
   move(myaccount, 'INBOX', myaccount, 'foo', result)
   subtract(snapshot, result)

   -- move messages that match another filter as normal
   result = match(myaccount, 'INBOX', myfilter2)
   move(myaccount, 'INBOX', myaccount, 'bar', result)
   subtract(snapshot, result)

   -- now move the rest of the messages from the snapshot (eg. junk)
   move(myaccount, 'INBOX', myaccount, 'junk', snapshot)


Last, you may find useful the urgent/special flag that you can set on 
messages.  For example, you may set initially this flag to all messages 
currently in INBOX, then apply filters only to messages that have this 
flag set, and unset this flag before moving messages.  This way, in the 
end, only the messages that have this flag will be deleted as junk (so 
new messages are not considered).  You can set this flag like this:

   -- mark special all current messages
   all = match(myaccount, 'INBOX', {})
   flag(myaccount, 'INBOX', 'add', { 'flagged' }, all)

   -- apply filter only to messages marked special
   myfilter = { 'flagged', 'subject "foo"' }
   result = match(myaccount, 'INBOX', myfilter)
   flag(myaccount, 'INBOX', 'remove', { 'flagged' }, result)
   move(myaccount, 'INBOX', myaccount, 'foo', result)

   -- move the rest of the messages that are marked special (eg. junk)
   result = match(myaccount, 'INBOX', { 'flagged' })
   flag(myaccount, 'INBOX', 'remove', { 'flagged' }, result)
   move(myaccount, 'INBOX', myaccount, 'junk', result)


You may also use user keywords (custom flags) if the IMAP server permits 
it (see the flag() command in the imapfilter_config(5) man page).

I guess you can customize these techniques to achieve the result you 
desire...

L.