in any case, dupe checking in FTN is not done my /just/ detecting duplicate MSGIDs and rejecting the others... the header, including the time stamp, as well as the message body should be taken into
it should also be said that CRC16/CRC32 on the message bodies is also
not sifficient... even with filtering out white space and the various EoLs... this because, and most programmers know this, there's a
limited supply of CRC values in the tables and it is all too easy to
find "hash clashes"... CRC16 has only 65536 values... CRC32 has only 4294967296 values... the "Birthday Problem" also comes into play...
these days, MD5 and SHA1 are also out due to defects in them...
SHA256 would be the first really useful algorithm or SHA512...
but the real key is to filter out the stuff that can change and hash
only that which won't...
anyway, i'm done with this topic in this area... the discussion really belongs elsewhere for those that are truely interested in implementing proper duplicate detection in FTNs...
|Location:||Riverside County, California|
|Nodes:||15 (1 / 14)|