Robot protection

[cools]

For the past week we've been bombarded by far higher than usual traffic from a wide variety of sources.

The server has held up, but you will have noticed instances where it's responded far more slowly than usual as a result.

I've now implemented various strategies to mitigate this. The only one that may affect real users is a daily request rate limit. I'm completely open to raising this one if it affects anyone - if you notice the site is unavailable and you've been pretty active, please email contact@arcadeotaku.com (ideally including your IP address) and I'll take a look - it's a temporary block and you will regain access after 24 hours anyway!

This isn't perfect for people behind CGNat and other proxy type systems but it's what it is for now.

Incidentally, if anyone knows how to feed requests for invalid phpBB sessions into fail2ban, please let me know. I think that will further help.

[cools]

https://www.abuseipdb.com/check/20.171.207.18

[geotrig]

was this causing the outages the last few days ? great work cools

[cools]

Part of it. I think I've got most of it controlled now.

Pleasingly, we get a fair chunk of bot traffic from Japanese search engines.

[SuperPang]

おいグーグル。一体誰が俺らのゲームを全部買ってるんだ？

[cools]

Nevermind the Japanese. Someone (on TalkTalk) is refreshing Arcade Games & Artwork every six seconds

[cools]

Finished with what I was aiming for. The major trouble was coming from some Asian clouds, so those are now perma blocked. As well as Meta, which is satisfying. Tempted also to block Google on volume of requests but they are at least throttling whatever they do, so not causing any problems.

Some automatic actions are in place to try and mitigate general reoccurrence. Because we're a quiet place and heavily optimised for performance, normal people shouldn't get blocked as I've set the thresholds way above what anyone would reach. Except for whoever is refreshing the marketplace - you might want to curb that to once every couple minutes.

Stats for the past week if anyone is curious. https://www.arcadeotaku.com/stats.html

[SuperPang]

Forum for sale. 1.8m unique visitors a week. Enquire within.