UCEprotect & Iverson

Continuing with the theme of bringing spam related issues from nanae and
blogs to the group....

Claus v. Wolfhausen runs the blocklist uceprotect
http://www.uceprotect.net/en/index.php

He has a pretty vocal and sometimes argumentative presence on nanae.

Al Iverson is another interesting internet personality, who has had some
'engagements' and disagreements and makeups with spamcop over the years.
Currently he has a blog and also is involved with publishing stats which
result from his analysis of various blocklists.   http://www.dnsbl.com/
What is a DNSBL? - Which ones work well? - Blacklist Statistics Center

Claus had this to say about a recent (special or 'personal') evaluation
of combining the 3 uceprotect lists.


From: Claus v. Wolfhausen
Newsgroups: news.admin.net-abuse.email
Subject: Union of UCEPROTECT became the most successfull blocklist
Date: Wed, 16 Apr 2008 16:32:23 +0000 (UTC)
Message-ID: <fu59mn$v1$1@ulm.shuttle.de>

According to AL IVERSON's stats the Union of UCEPROTECT (Levels 1 + 2 +
3) is the most succesfull blocklist if we calculate average blocking
over the last 13 weeks ... <snip>


... and then he goes on to 'extract' a comparison between the uce123
with spamhaus, spamcop, cbl, psbl, derived from this iverson site page
http://stats.dnsbl.com/uceany.html Note: This data does not actually
represent a distinct zone. This is a union of all 3 UCEPROTECT lists.
There is not actually any "UCEANY" list.

Even if you consider Wolfhausen a 'loudmouth', those are nice stats and
have a very respectable low false positive rate;  and the stats service
Iverson is providing is useful.

--
Mike Easter
kibitzer, not SC admin

VanguardLH wrote:
> As I recall, UCEprotect is a replacement for SPEWS when that blacklist
> went dead.

APEWS is the replacement for spews, not uceprotect.  Wolfhausen &
uceprotect previously published apews, but... (don't anymore)^0  That
footnote is an iverson review of apews.  Iverson hasn't done a review on
uceprotect, but I would think he might, considering this stats issue.

> The same criteria were used in UCEprotect as for SPEWS so
> it was not a good personal-use list to use for filtering if it was
> simply used to mark "good" or "bad" an e-mail as spam
but might have
> some value if used to score the spamminess of an e-mail (but
> definitely not to score it at 100% if on their list).

The listing process for uce's 1, 2, & 3 is described at the site.  1 is
single IPs, 2 is escalated blocks, 3 is ASNs based on 1&2.  SpamCop once
tried a scheme for handling block related issues, but it didn't work out
very well.  Claus's scheme appears to have favorable stats considering
the percentages and the low percentage false positives as seen at the
uceany link^1.


^0 http://www.dnsbl.com/2007/08/apews-news-and-commentary-roundup.html
Claus V. Wolfhausen, maintainer of UCEPROTECT, another German-run
blacklist, indicates that UCEPROTECT will no longer publish the APEWS
blacklist zones. (Previously: Claus warned that unless APEWS were to
make immediate, significant changes to its policies, UCEPROTECT will no
longer publish the APEWS blacklist zones.)

^1  http://stats.dnsbl.com/uceany.html  The folks behind UCEPROTECT
asked me what it would look like if I were using all three UCEPROTECT
backlist zones together. I thought it was a neat idea and decided to
share the results publicly.


--
Mike Easter
kibitzer, not SC admin

As I recall, UCEprotect is a replacement for SPEWS when that blacklist
went dead.  The same criteria were used in UCEprotect as for SPEWS so it
was not a good personal-use list to use for filtering if it was simply
used to mark "good" or "bad" an e-mail as spam but might
have some value
if used to score the spamminess of an e-mail (but definitely not to

Sofa King Tyred of Lar Ting wrote:
> Mike Easter wrote:
>> Even if you consider Wolfhausen a 'loudmouth', those are nice stats
>> and have a very respectable low false positive rate;  and the stats
>> service Iverson is providing is useful.
>
> I contacted Wolfhausen about backscatter analysis, since he claims to
> have vast quantities of it on the UCEPROTECT network. Thanks to the
> 900 messages I was getting/day and a small Java tool, I was able to
> do a small study that showed that in real-time, 19% of the IP
> addresses (zombies) injecting spams that resulted in backscatter were
> not on any of the CBL, SCBL, DNSBL, etc.

Did you check them to see if they were positive on the backscatterer.org
list?

> The only thing Wolfhausen offered me was to move my domain to his
> space on UCEPROTECT (for free). Since I use an alias on bigfoot.com,
> that was impossible. It's too bad his system (and mind?) is closed
> because his backscatter (backscatterer.org) data is beyond anything I
> could dream of crunching and would surely make for great additions to
> DNS block lists.

I don't understand exactly what you are saying.  The backscatterer.org
data is publicly available just like all of the public blocklists.
Here's a test:

dns 2.0.0.127.backscatterer.org
Canonical name: 2.0.0.127.backscatterer.org
Addresses:
  127.0.0.2

> Lastly analyzing backscatter for zombie detection is peculiar because
> there's no question of spam vs. ham or false positives. Any NDR/bounce
> you get for a message you didn't send is a result of a forged sender
> and definitely unsolicited (spam). It might be question of random
> spam vs. joe-job, but it's always spam.

I'm not convinced that analyzing backscattered spam is any more valuable
than analyzing ordinary spam.  There's a little problem of not being
inside the 'head' (or algo) of the spam propagator.  Did the propagator
(for any given particular spam) hit a backscattering server on purpose
or just as a part of the broadcast process?

--
Mike Easter
kibitzer, not SC admin

Mike Easter wrote:
> Even if you consider Wolfhausen a 'loudmouth', those are nice stats and
> have a very respectable low false positive rate;  and the stats service
> Iverson is providing is useful.

I contacted Wolfhausen about backscatter analysis, since he claims to 
have vast quantities of it on the UCEPROTECT network. Thanks to the 900 
messages I was getting/day and a small Java tool, I was able to do a 
small study that showed that in real-time, 19% of the IP addresses 
(zombies) injecting spams that resulted in backscatter were not on any 
of the CBL, SCBL, DNSBL, etc. These days, my backscatter rate is only 
several hundred per day (I suspect it's because my alias doesn't forward 
it all to me anymore, maybe because they use a DNSBL or two).

The only thing Wolfhausen offered me was to move my domain to his space 
on UCEPROTECT (for free). Since I use an alias on bigfoot.com, that was 
impossible. It's too bad his system (and mind?) is closed because his 
backscatter (backscatterer.org) data is beyond anything I could dream of 
crunching and would surely make for great additions to DNS block lists.

I'm sure that backscatter analysis would make spam detection better for 
several reasons:

0. Backscatter is likely on the rise because of SAV.
1. The IPs of the zombies are easy to spot if the backscatter has 
received headers inside. There is no question of where the zombie is.
2. The subjects of the enclosed spams might be useful if you want to try 
improving filtering (although I think the zombie IPs are an order of 
magnitude more useful, since most spams are sent by zombies).

Lastly analyzing backscatter for zombie detection is peculiar because 
there's no question of spam vs. ham or false positives. Any NDR/bounce 
you get for a message you didn't send is a result of a forged sender and 
definitely unsolicited (spam). It might be question of random spam vs.