For those of you that use BBBS, and run into this, this is what I put in inet.bbb:
[bbbsd]
!ftp 66.249.65/$
!www 66.249.65/$
!tcpip 66.249.65/$
!binkp 66.249.65/$
!FTP 66.249.65.83/10
!HTTP 66.249.65.83/10
!raw 66.249.65.83/10
!TCPIP 66.249.65.83/10
!FTP 66.249.65.83/10
!HTTP 66.249.65.83/10
!raw 66.249.65.83/10
!TCPIP 66.249.65.83/10
Good idea; I'll just block the entire block from coming in. My web server is professionally hosted so there's no need for Google to go poking around.
I should just kill them period even on port 80 for apache, and I still
may do so <grin>. I have free-find indexing my site automatically,
and they do it in such a way that it doesn't interfere with anything..
so it's not like google-bot couldn't do it that way if they wanted to!
I should just kill them period even on port 80 for apache, and I still
may do so <grin>. I have free-find indexing my site automatically,
and they do it in such a way that it doesn't interfere with anything..
so it's not like google-bot couldn't do it that way if they wanted to!
I had to block that Chinese search engine, Baidu, from my site-their bot would
hit it nearly every hour...and it was killing my bandwidth.
so the idiots at google-bot were trying anyway they could..
I laughed when I saw them try the port on 24555 (my bbbs binkp port)..
Hello, Janis.
Friday August 20 2010 at 14:29, you wrote to All:
!FTP 66.249.65.83/10Good idea; I'll just block the entire block from coming in.
!HTTP 66.249.65.83/10
!raw 66.249.65.83/10
!TCPIP 66.249.65.83/10
My web server is professionally hosted so there's no need
for Google to go poking around.
Are you sure that it was actually googlebot? It only follows http links,
which won't lead to that binkp port... (How'd the IP show up in a reverse dns?)
Also; do you have a robots.txt? I use it to disallow "/bbbs"...
I'm not familiar with that one.. I should do a grep for them to see if they've hit over here as well <grin>
Hey, last night I had the greatest chat with a user on my system..
he's from Zone 6 (he's in Tokyo) - and he wonders what happened to
Z6.. sad :( Well, he is looking to get it going again.. Super I think
:)
Do keep in mind that the IPs have a tendancy to change...
My web server is professionally hosted so there's no need
for Google to go poking around.
I don't see what the first thing as to do with the second...?
I'm not familiar with that one.. I should do a grep for them to see if
they've hit over here as well <grin>
Yeah, it's called BaiduSearch or BaiduBot; I've seen both of them in my logs before.
Hey, last night I had the greatest chat with a user on my system..
he's from Zone 6 (he's in Tokyo) - and he wonders what happened to
Z6.. sad :( Well, he is looking to get it going again.. Super I think
:)
That's great! It'd be nice to see Z6 up and running again.
I'm not familiar with that one.. I should do a grep for them to see if
they've hit over here as well <grin>
Yeah, it's called BaiduSearch or BaiduBot; I've seen both of them in
my logs before.
Wow Baidubot is surely hitting on BBBS' webserver over here :( Oh
well.. off to inet.bbb to get rid of them haha
I mean, I really don't mind if they hit the main web server here..
but bbbs is limited to only those 7 nodes.
Hey, last night I had the greatest chat with a user on my system..
he's from Zone 6 (he's in Tokyo) - and he wonders what happened to
Z6.. sad :( Well, he is looking to get it going again.. Super I think
:)
That's great! It'd be nice to see Z6 up and running again.
Yeah really :) He's got some friends who'll be coming in, and if
we count the z6 folks that we've got listed in Z3 right now, it
should make a good basis for reinstating that zone.
Wow Baidubot is surely hitting on BBBS' webserver over here :( Oh
well.. off to inet.bbb to get rid of them haha
you could just deny them access to those links in BBBS that lead to
places like the messages, files and games... they are robots.txt friendly...
Yeah, it's called BaiduSearch or BaiduBot; I've seen both of them in
my logs before.
Wow Baidubot is surely hitting on BBBS' webserver over here :( Oh
well.. off to inet.bbb to get rid of them haha
you could just deny them access to those links in BBBS that lead to places lik
the messages, files and games... they are robots.txt friendly...
I mean, I really don't mind if they hit the main web server here..
but bbbs is limited to only those 7 nodes.
i assume by that you mean that it is similar to apache in that there's seven
HTTP handlers that are allowed to run and you don't allow any more than that??
That's great! It'd be nice to see Z6 up and running again.
Yeah really :) He's got some friends who'll be coming in, and if
we count the z6 folks that we've got listed in Z3 right now, it
should make a good basis for reinstating that zone.
that might be a good thing... especially considering the reasons why Z6 went away last time it was operational...
just to follow up and clarify this... an example is my gallery site... i may not want the spiders trapsing thru the exhibits and looking at the actual photos available... only the microthumbs and thumbs, sure so i just set up a
disallow for the exhibits area and let them wander over the others... then, if
i see them in the exhibits areas, then i know that they're not following
robots.txt and i can then contact their admins with my complaint or just block
them at the perimeter and deny them access to everything... depending on my mood, of course ;)
Wow Baidubot is surely hitting on BBBS' webserver over here :( Oh
well.. off to inet.bbb to get rid of them haha
you could just deny them access to those links in BBBS that lead to
places like the messages, files and games... they are robots.txt friendly...
Well, that's not exactly the problem.. the content is so not
'private' in other words.. but when they tie up my bbbs
web/telnet/binkp nodes, 'real fido people' can't connect.. that is
a drag <g>
I mean, I really don't mind if they hit the main web server here..
but bbbs is limited to only those 7 nodes.
i assume by that you mean that it is similar to apache in that
there's seven HTTP handlers that are allowed to run and you don't
allow any more than that??
Yes, that's right, but it's not that I don't allow any more than 7,
it's because the way bbbs works is you register the number of nodes
you want for the bbbs daemons.. so I have 7 nodes registered...
that's 1 phone-modem node, 6 http instances, 6 telnet nodes, 6
binkp nodes, etc.
I also run BinkD stand-alone mailer on the standard binkp port to
pick up more binkp connections since I know the bbbs binkp nodes
get a bit busy with the number of downlinks here <g>. BBBS's
binkp daemon runs on port 24555.
I guess what really bugged me was that these spiders were hitting
all the ports here.. not just 80 :( I mean, what could a spider
get out of attemping repeated connections to my binkp port on
24555?? <bg>
Great thing this week: when I contacted Kim Heino (he's the
author of bbbs) about my dead motherboard and all that, etc., (he
uses ftp to connect so we needed to set that up on the main ftp
server), he sent me a beta 64 bit version of BBBS.. Really cool :)
It's running really well.
That's great! It'd be nice to see Z6 up and running again.
Yeah really :) He's got some friends who'll be coming in, and if
we count the z6 folks that we've got listed in Z3 right now, it
should make a good basis for reinstating that zone.
that might be a good thing... especially considering the reasons why
Z6 went away last time it was operational...
Understand.. these kinds of things take time, but we're hoping.
'private' in other words.. but when they tie up my bbbs
web/telnet/binkp nodes, 'real fido people' can't connect.. that is
a drag <g>
yeah, i can see that with a server that has limited handlers allowed... but th
idea was to limit what they have access to so they're in and out as fast as possible ;)
I mean, I really don't mind if they hit the main web server here..
but bbbs is limited to only those 7 nodes.
i assume by that you mean that it is similar to apache in that
there's seven HTTP handlers that are allowed to run and you don't
allow any more than that??
Yes, that's right, but it's not that I don't allow any more than 7,
it's because the way bbbs works is you register the number of nodes
you want for the bbbs daemons.. so I have 7 nodes registered...
that's 1 phone-modem node, 6 http instances, 6 telnet nodes, 6
binkp nodes, etc.
yup... pretty much the same idea... close enough for the analogy ;)
pick up more binkp connections since I know the bbbs binkp nodes
get a bit busy with the number of downlinks here <g>. BBBS's
binkp daemon runs on port 24555.
i'd really hate it if the bots started hitting the telnet and binkd stuff... i
think that google does the http and ftp stuff now... i know i've seen references to it in my ftp logs somewhere...
I guess what really bugged me was that these spiders were hitting
all the ports here.. not just 80 :( I mean, what could a spider
get out of attemping repeated connections to my binkp port on
24555?? <bg>
trying to connect to a web server that it thinks is running there... if it is
doing that, lodge a complaint and/or block it at the perimeter and don't let i >in at all ;) depending on the methods, it could be blocked at the perimeter fo
all ports except for 80 :P
Great thing this week: when I contacted Kim Heino (he's the
author of bbbs) about my dead motherboard and all that, etc., (he
uses ftp to connect so we needed to set that up on the main ftp
server), he sent me a beta 64 bit version of BBBS.. Really cool :)
It's running really well.
i saw reference to you running 64bit earlier... that's a GoodThing<tm> ;)
That's great! It'd be nice to see Z6 up and running again.
Yeah really :) He's got some friends who'll be coming in, and if
we count the z6 folks that we've got listed in Z3 right now, it
should make a good basis for reinstating that zone.
that might be a good thing... especially considering the reasons why
Z6 went away last time it was operational...
Understand.. these kinds of things take time, but we're hoping.
word up! :P
with anything.. so it's not like google-bot couldn't do it that way
if they wanted to!
with anything.. so it's not like google-bot couldn't do it that way
if they wanted to!
time to get a mirror :)
but:
----- .htaccess begins -----
RewriteEngine on
# Allow only GET and POST verbs
RewriteCond %{REQUEST_METHOD} !^(GET|POST)$ [NC,OR]
# Ban Typical Vulnerability Scanners and others
# Kick out Script Kiddies
RewriteCond %{HTTP_USER_AGENT} ^(java|curl|wget).* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*(libwww-perl|curl|wget|python|nikto|wkito|pi
to|scan|acunetix).* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*(winhttp|HTTrack|clshttp|archiver|loader|ema
l|harvest|extract|grab|miner). * [NC,OR]
# Ban Search Engines, Crawlers to your administrative panel
# No reasons to access from bots
# Ultimately Better than the useless robots.txt
# Did google respect robots.txt?
# Try google: intitle:phpMyAdmin intext:"Welcome to phpMyAdmin *.*.*" intext:"Log in" -wiki -forum -forums -questions intext:"Cookies must be enabled"
RewriteCond %{HTTP_USER_AGENT} ^.*(AdsBot-Google|ia_archiver|Scooter|Ask.Jeeve >|Baiduspider|Exabot|FAST.Enter prise.Crawler|FAST-WebCrawler|www\.neomo\.de|Gi >abot|Mediapartners-Google|Goog le.Desktop|Feedfetcher-Google|Googlebot|heise-I >-Markt-Crawler|heritrix|ibm.co m\cs/crawler|ICCrawler|ichiro|MJ12bot|MetagerBo >|msnbot-NewsBlogs|msnbot|msnbo t-media|NG-Search|lucene.apache.org|NutchCVS|Om >iExplorer_Bot|online.link.vali dator|psbot0|Seekbot|Sensis.Web.Crawler|SEO.sea >ch.Crawler|Seoma.\[SEO.Crawler \]|SEOsearch|Snappy|www.urltrends.com|www.tkl.i >s.u-tokyo.ac.jp/~crawler|Synoo Bot|crawleradmin.t-info@telekom.de|TurnitinBot| >oyager|W3.SiteSearch.Crawler|W 3C-checklink|W3C_Validator|www.WISEnutbot.com|y
cybot|Yahoo-MMCrawler|Yahoo\!. DE.Slurp|Yahoo\!.Slurp|YahooSeeker).* [NC] RewriteRule .* - [F]
----- .htaccess ends -----
if bbbs http server cant do this make it as proxy in apache !
Sysop: | digital man |
---|---|
Location: | Riverside County, California |
Users: | 1,033 |
Nodes: | 15 (0 / 15) |
Uptime: | 215:21:58 |
Calls: | 529 |
Calls today: | 23 |
Files: | 95,154 |
D/L today: |
4,424 files (585M bytes) |
Messages: | 298,866 |
Posted today: | 12 |