Main | Download | History | Mailing List | License | FAQ
This is version 2 of the FAQ and is more applicable to
versions from 1.2.0 onwards. It has been updated to reflect the
new regular expression and scoring filtering methods. Obsolete
installation instructsions have been removed. Questions that are
new or that have been updated are marked with
Nfilter is a supplemental filtering program to your existing newsreader. It is designed to run transparently alongside any client for Windows such as Outlook Express, Agent, Netscape, etc.
It is a 32-bit windows console application which will run on Windows '95, '98 or NT. It is especially useful if your newsreader doesn't have any filtering capabilities (eg. Netscape). Even if your news reader has filtering capabilities (eg. Agent), the filter can perform more advanced filtering on fields that are not normally available to your newsreader.
Nfilter is capable of filtering on any fields that appear in the header of a news article, not just fields in the overview (normally news clients can only filter on author, subject, etc). For example, it can filter on path (eg. all posts from news.foo.net), newsgroups (eg. all posts cross-posted to alt.foo), nntp-posting-host, etc, etc.
Nfilter does this by acting as a local news proxy. It takes commands from your news client and passes them on to your news server. It analyses the list of message headers returned by your news server and, based on the filter rules that you write, either drops the headers so that you don't see them or marks them with a flag to let you know they are filtered.
These comments from thynkr@aol.com, thanks thynkr:
"Access to an NNTP port on your news server. This rules out AOL users who read Usenet from AOL servers, sorry. However you do have the option of subscribing and connecting to any commercial news service, using any newsreader that will run on your computer. Information on how to do this may be found on news:aol.newsgroups.help."
Okay, the information can be found at http://home1.gte.net/docthomp/AOL.htm. I am not afiliated with the person who maintains that site, but that is where I found out how to do it.
There is a modern urban legend / old wives tale afoot which holds that access to nntp servers is somehow blocked by AOL. This is not true. In reality, only AOL subscribers can access *AOL's* news servers, and only using the (abysmal) AOL news client. This is simply because AOL's news servers are a) non-nntp-compliant and b) firewalled. It's a pity that AOL refuses to set up true nntp servers for the use of its customers, because AOL's newsfeed ranks among the best in the world. Essentially, all one has to do is:
These comments from roland.rashleigh-berry@virgin.net, thanks Roland:
alt.religion.scientology is under major spam attack. I have been maintaining an nfilter.dat file to filter out all the spam. I post new versions daily to the newsgroup news:alt.religion.scientology.xenu.
Also check out http://www.hotel.wineasy.se/xemu/index.html for Roland's latest nfilter.dat file.
Under normal circumstances, nfilter should have absolutely no effect on the speed of your newsreader. However, if you have lots of filter rules set up, then there will be a slight performance hit when downloading headers. This will not effect downloading article bodies or posting.
Performance can be optimised in a few ways:
That is my intention. Ideally, I'm hoping this will become an open project with several contributors. The source code is available under the GNU General Public License (GPL) with this in mind.
I don't mind contributing myself so long as it doesn't take up too much of my time. I can only afford to spend a few hours a week on this, so if you can help out it would be appreciated.
Currently, in approximate order of priority, I am thinking about the following:
Support for 16-bit Windows has been put on the long finger for now. It looks like it would be more difficult to do than I first thought.
Unix support is also at a fairly low priority. Most users seem to be on Windows.
Yes.
Things that can be done to help:
Newer versions of nfilter (now called NewsProxy) are self-installing using InstallShield. Just download the installation file from the web page and run the application. It will take care of the rest of the process for you.
After installation, you need to do some configuration of nfilter.
For most purposes, the only information you need to change in the configuration file is the "NewsServer" entry. Set this to the name of the news server you normally read news from.
See 2.5 for more information on configuring.
If you selected the defaults while installing nfilter, then a short cut has automatically been created from your start menu. Click "start", select "programs", then "Newsproxy" and then click the newsproxy icon. Before you use NewsProxy you should confired the name of your news server in the configuration/network panel. The fields are as follows:
server - The news server to connect to (eg. news.foo.com)
port - This optional field specifies what port to connect to on the remote server. By default this is the nntp port (119).
listen port - This optional field specified which local port to listen to. The default is nntp (119), but if you wish to run multiple proxies (eg. for multiple news servers), you should set each one to listen to a different local port.
There are now four action modes for each filter. You can elect to drop an article, to flag an article's subject, to flag an article's author or to change a score for an article.
In drop mode, the header will be removed completely from the list of headers given to your newsreader so you will not even know the article exists. This mode is suitable for news clients that have no filtering capabilities.
The second mode marks the subject line of a filtered article before presenting it to your newsreader. In this mode, the subject line will be modified so that it is preceded by the text you specify. You must specify marker text to use when using the flag mode. For example:
The third mode, author flagging, is very similar to the second except it modifies the From header instead of the Subject header.
The fourth mode, scoring, is discussed in 3.10
You need to create or edit the file called nfilter.dat in the nfilter directory. The easiest way to do this is to select "Edit" then "Filters" from NewsProxy's drop down menus.
This file contains filtering rules. Each line is one rule. The format of each line is:
NOTE: If you enable regular expressions, only one header is allowed per line. This is new as of release 1.2.0, previous releases allowed multiple headers. This change has been made in order to accomodate regular expression matches.
Wildcards are allowed in newsgroup and value. All others must be constant strings. action may be flag, aflag, drop or score. If action is flag or aflag, it must be followed by a colon and a subject marker (eg. FLAG:FILTER-ME-).
Configuration of nfilter can be done in two ways. The easiest is to use the menus: pull down the "Edit" menu and select "Configuration". You may also edit the file NewsProxy.INI directly but be advised that it is not a standard Windows INI file. (See 2.6.)
In the INI file, blank lines and lines beginning with a '#' are ignored. All other lines must be in the format:
Values can be string, integer or boolean depending on the configuration item. Strings can be any text. Integers can be any integer value. Booleans can be True, False, T, F, Yes, No, Y or N. Case is ignored.
Because then it wouldn't be portable. Functions such as ReadPrivateProfileString are not available on all platforms so it is easier to actually read and parse the config file myself.
With Agent, you need to change the configuration to point to "localhost". To do this, select the "Options" menu, then the "User and System Profile" option. Change the value in the first text box (News Server) to "localhost" (without the quotes).
If you were on-line, press Ctrl-O in Agent to go offline, and then again to go online.
No other changes are required.
Pull down the "File" menu. Select "open news host" and type in "localhost" as the server.
No other changes are required.
You can filter any message with a supersedes header with the following line:
Be advised that many FAQs and other regularly posted articles use the supersedes header and not all supersedes are rogue.
If someone morphs to avoid kill-files, then you need to find something consistent about their headers that doesn't change from post to post. You may need to combine two or more common elements in their header to achieve this.
Say I changed my From field on every post, but you know I always post from sexzilla and I always post with Agent 1.5/32.451. You also know that nobody else on sexzilla uses this version of Agent. In this case you could filter me using the rule:
At present, the forgery flood (sporgeries) are not trivially filterable. There is an option in the configuration to enable forgery flood filtering but it does not work.
The only current method for filtering the flood is to use the current flooding path host and/or NNTP-Posting-Host. This method is not as straightforward as the flooders move ISP almost every day so keeping the list updated is not trivial.
Yes.
Use the action AFLAG instead of FLAG.
Yes. See 3.10
This can be done in a slightly roundabout way. You have to count the commas in the newsgroups line. It is more efficient to use the Xref: header however and you can filter crossposts on this header by counting colons. For example:
You can filter all posts from an entire site, regardless of what line appears in the From: header by either filtering on the NNTP-Posting-Host: header or by a known element in the Path: header.
For example, to filter all posts from iol.ie, you could use either:
or
You could also use scoring to combine the two, for example:
These may not work on all servers, see question 4.1 nfilter isn't filtering properly
Yes.
Filters can be combined in several ways. If two fields appear on a single line in nfilter.dat, they are treated as a logical AND condition. For example, the line:
On the other hand, fields that are on separate lines are treated as a logical OR condition. The two lines:
Nathaniel Eliot has also come up with a trick where you can make use of the fact that nfilter stops processing once it has met a rule that matches. For example, if I am a morpher that you want to kill-file and you know that everything from sexzilla using Agent 1.5/32.451 is from either me or from Fred. You could create a line to "pass" posts from Fred by flagging with them with no text. For example:
From version 1.2.0 onwards you can elect to use regular expressions for header matches. Go to the "General" pane of the configuration property sheet and ensure the "Use Regular Expressions" check box is checked.
(This option is off by default for backward compatibility with older filter files. NOTE: If you enable this option, you can not have multiple headers on one line.)
Once this option is checked, all header comparisons will use a regular expression comparison. For example, the following will drop any message from Mark whether the first letter is uppercase or lowercase.
Regular expressions are very powerful. For example, the following regexp could be used is a basic sporgery filter. (Note that this should only be used as an indicator, as it will generate many false positives) :
Spaces are now also allowed in header comparison fields.
(If anyone knows of a good online regular expression tutorial, please let me know and I will link to it here.)
From version 1.2.0 onwards you can keep a running score for an article. To do this, you use an action of "score" along with an integer value (which may be positive or negative). An action of score does not halt filtering the way that flag or drop do. Instead it continues until a further filter is executed or the article passes. For example:
The third line checks if the score is greater or equal to 150. If it is the article is dropped. Filtering halts at this point.
The fourth line checks if the score is greater or equal to 100. If it is the article is dropped. Again, filtering halts at this point.
Note that you must check higher score values first, as nfilter will stop processing the filters as soon as an action is reached. For example:
The most likely reason is that your news server is not giving you the headers you request. Very often, a news server will not give you a header unless it is in the overview.
To find a list of overview fields, connect to your news server on port 119 using telnet. Type "list overview.fmt" and press enter. You should see a response like this:
You can filter on any of the headers listed.
This is usually the same reason as for question 4.1. However, there is a work-around and that is to use the XREF header instead. The XREF header contains the list of newsgroups that the mesage was stored in on your news server. This may be slightly shorter than the original newsgroups line if your server doesn't carry all the groups that the message was posted to.
Also, the format of XREF is different in that it also contains the number of the article in each group, so be sure to use wildcards in between newsgroup names. For example:
You can either run nfilter with the -d option (eg. nfilter -d server) or edit the config file and set DebugInfo = Yes.
The file debug.log is always created. It normally only contains startup and shutdown messages.
This is a bug that crops up from time to time. It is due to variances in how news servers return the list of headers. There seems to be an almost infinite variety that these can take. If you get this error message, drop me an email and I'll try to fix the problem.
This is a message from your news server to say that the article has either been deleted or has expired from the news spool. It is possibly a spam article that was cancelled or it could just be an old article. There is nothing that nfilter can do about this message, it would occur without nfilter as well. nfilter doesn't know that the article is gone until it asks for it, at which point all it can do is return the error.
This is from people who use nfilter following up to articles that have been marked by nfilter. If you do follow-up to articles which have been marked, please make sure to remove the pre-fix from the subject before posting your article.
It shouldn't. If you are on a LAN and can ordinarily read news, then nfilter will make no difference whatsover.
There are some firewalls that run on a local machine that need to be told about nfilter.
JOWazzoo has contributed this list of actions required:
It doesn't have any proxy support. If you are behind a socks proxy, this unfortunately means it won't work.
If you get the error message
It is also possible to get this message if there are too many concurrent connections to nfilter. Since this limit is currently set at 20, it is very unlikely you will see this message for that reason, unless you are using nfilter as a news gateway.
If you get the error message
If it happens repeatedly or for any other reason, then this probably indicates a bug which you should let me know about.
$Id: faq.html,v 1.2 2001/04/15 11:30:02 mark Exp $