Hi all

I don't recall if I've brought this up again but are there echomail import limits that may not be explictly configurable? I ask because I noticed again (on my v0.92.0 system at 1:120/544) that although a rescan of an echo to my uplink resulted in a message noting that 400 messages were scanned but my system only ended up importing 47. Need to figure out where that is...


Jame


--- GoldED+/LNX 1.1.5--b20111217
* Origin: RJC eeePC (1:120/419)

Hello RJ!

Thursday May 09 2013 11:21, RJ Clay wrote to All:

I don't recall if I've brought this up again but are there echomail import limits that may not be explictly configurable? I ask because I noticed again (on my v0.92.0 system at 1:120/544) that although a
rescan of an echo to my uplink resulted in a message noting that 400 messages were scanned but my system only ended up importing 47. Need
to figure out where that is...

Is it possible the others were discarded as duplicates?

Andrew

---
* Origin: Bits & Bytes BBS * V.Everything! * 860/535-4284 (1:320/119)

I don't recall if I've brought this up again but are there
echomail import limits that may not be explictly configurable? I
ask because I noticed again (on my v0.92.0 system at 1:120/544)
that although a rescan of an echo to my uplink resulted in a
message noting that 400 messages were scanned but my system only
ended up importing 47. Need to figure out where that is...

Is it possible the others were discarded as duplicates?

they would have been logged as dupes, wouldn't they? ;)

)\/(ark

---
* Origin: (1:3634/12.42)

Hi Andrew!

10 May 13 17:05, you wrote to me:

Thursday May 09 2013 11:21, RJ Clay wrote to All:

I don't recall if I've brought this up again but are there
echomail import limits that may not be explictly configurable? I
ask because I noticed again (on my v0.92.0 system at 1:120/544)
that although a rescan of an echo to my uplink resulted in a
message noting that 400 messages were scanned but my system only
ended up importing 47. Need to figure out where that is...

Is it possible the others were discarded as duplicates?

Shouldn't have been; I'd just added the echo to the configuration...




Jame



--- GoldED+/LNX 1.1.5--b20111217
* Origin: RJC eeePC (1:120/419)

I don't recall if I've brought this up again but are there
echomail import limits that may not be explictly configurable? I
ask because I noticed again (on my v0.92.0 system at 1:120/544)
that although a rescan of an echo to my uplink resulted in a
message noting that 400 messages were scanned but my system only
ended up importing 47. Need to figure out where that is...

Is it possible the others were discarded as duplicates?

Shouldn't have been; I'd just added the echo to the
configuration...

that depends on the tosser's method of dupe detection... some of them allow the
exact same message in more than one area because their dupe base is for each area... others use one dupe base for all areas...

i can't say which i feel is best... both have their pluses and minuses...

)\/(ark

---
* Origin: (1:3634/12.42)

Hi mark!

11 May 13 16:58, you wrote to me:

I don't recall if I've brought this up again but are there
echomail import limits that may not be explictly configurable?
I ask because I noticed again (on my v0.92.0 system at
1:120/544) that although a rescan of an echo to my uplink
resulted in a message noting that 400 messages were scanned but
my system only ended up importing 47. Need to figure out where
that is...

Is it possible the others were discarded as duplicates?

Shouldn't have been; I'd just added the echo to the
configuration...

that depends on the tosser's method of dupe detection...

True but in this case none of the messages would be in any other echo...



Jame


--- GoldED+/LNX 1.1.5--b20111217
* Origin: RJC eeePC (1:120/419)

Hi! Jame,

In a message to Mark Lewis you wrote:

Is it possible the others were discarded as duplicates?

Shouldn't have been; I'd just added the echo to the
configuration...

that depends on the tosser's method of dupe detection...

True but in this case none of the messages would be in any other echo...

Smack me if I'm wrong... I think this aspect is weak point for MBSE: the handling of dupes/bad mail. IIRC there ain't no option to 'toss from bad', after configuring new areas. So, you are at the mercy of any dupes in the dupe
database.

It would be nice to get some advice from Michiel or Vince on this...

Cheers,
Paul.

... Gravity isn't easy, but it's the law.
--- Paul's Win98SE VirtualBox
* Origin: Quinn's Post - Maryborough, Queensland, OZ (3:640/384)

On Mon, 13 May 2013, RJ Clay wrote to mark lewis:

I don't recall if I've brought this up again but are there
echomail import limits that may not be explictly configurable? I
ask because I noticed again (on my v0.92.0 system at 1:120/544)
that although a rescan of an echo to my uplink resulted in a
message noting that 400 messages were scanned but my system only
ended up importing 47. Need to figure out where that is...

Is it possible the others were discarded as duplicates?

Shouldn't have been; I'd just added the echo to the
configuration...

that depends on the tosser's method of dupe detection...

True but in this case none of the messages would be in any other echo...

understandable... but that doesn't mean that the MSGID hasn't been seen before and it is possible that there may be a CRC16 or CRC32 collision if the tosser uses such methods and then that also depends on what data in the message they use for those calculations... some take only the header information... some take the header information plus 40 bytes or so to try to include some of the hidden control lines... they are hoping to catch the MSGID line in there but if
the line is too far away that that won't always work...

duplicate detection is definitely an art and generating messages to pass the various duplicate detection schemes can be even more of an art... however, in some cases, the duplicate detection is very brain dead (eg: header only or message body only) and no amount of crafting will get a valid new message by them... sbbsecho is one that insists that a message with an identicle body is a
duplicate even thought the header plainly shows that it is a new posting... DB is another one with a similar method of duplicate detection...

)\/(ark

--- FMail/Win32 1.60
* Origin: (1:3634/12.71)

On Tue, 14 May 2013, Paul Quinn wrote to Rj Clay:

that depends on the tosser's method of dupe detection...

True but in this case none of the messages would be in any other echo...

Smack me if I'm wrong... I think this aspect is weak point for
MBSE: the handling of dupes/bad mail. IIRC there ain't no option
to 'toss from bad', after configuring new areas. So, you are at
the mercy of any dupes in the dupe database.

exactly and this isn't the only system aflicted with this problem...

synchronet (actually the sbbsecho tosser) is another one... it deletes what it considers to be duplicates with no facility for saving them to a special area for human perusal... the only real solution to this is to use a script that deletes the dupe databases before each round of tossing performs the toss and then deletes the dupe bases again after the toss... this second deletion so that other methods of message importing (QWK, news) don't get caught with possibly false dupes... it barely has the ability to save messages for areas not defined and no good way to move them into the proper area once it is defined... i'm ever hopefuly that this will be remedied... i'd contribute if i were a C coder but since i'm not, i can't... all i can do is point out what is desired and how it would be done...

i'm sure there are other tossers with similar aflictions, as well...

)\/(ark

--- FMail/Win32 1.60
* Origin: (1:3634/12.71)

Hello Paul!

14 May 13 08:32, you wrote to Rj Clay:

that depends on the tosser's method of dupe detection...

True but in this case none of the messages would be in any
other echo...

Smack me if I'm wrong... I think this aspect is weak point for MBSE:
the handling of dupes/bad mail. IIRC there ain't no option to 'toss
from bad', after configuring new areas. So, you are at the mercy of
any dupes in the dupe database.

It would be nice to get some advice from Michiel or Vince on this...

For badmail that I have created a echo area for I just use golded to mark them and them move to the new area.

On dupes I have it set to 16k and have not noticed any issues but I also use age, so that any msgs over 30 days old get junked.

Hope that helps in some way.


Vince

--- Linux/Mbse/GoldED+/LNX 1.1.5-b20120229
* Origin: Air Applewood, The Linux Gateway to the UK (2:250/1)

Hi! Vince,

On Tue, 14 May 13, you wrote to me:

For badmail that I have created a echo area for I just use golded to
mark them and them move to the new area.

Neat, and cunning. I like it! ;-)

On dupes I have it set to 16k and have not noticed any issues but I
also use age, so that any msgs over 30 days old get junked.

I have some large-ish echoes, some going back a decade, so I use 50,000 records
in FastEcho's dupe database.

Hope that helps in some way.

I did. Thank you.

Cheers,
Paul.

... Hey SysOp! You'd better upgrade me or el%$^&%NO CARRIER
--- Paul's Win98SE VirtualBox
* Origin: Quinn's Post - Maryborough, Queensland, OZ (3:640/384)