Please use our links: LEGO.com • Amazon
Brickset.com is a participant in the Amazon Services LLC Associates Program, the Amazon.com.ca, Inc. Associates Program and the Amazon EU Associates Programme, which are affiliate advertising programs designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.
As an Amazon Associate we earn from qualifying purchases.
i agree with other comments the german ip sounds like hes using scraper, and the fact there is network issues is probably inadvertently causing his crappy scraper to make too many requests. if the network was stable he probably would have scraped the entire catalogue overnight and you would be none the wiser unless you study the logs like a hawk each morning
plus if its only a single ip once you banned his entire subnet shouldnt things return to normal?
new.brickset.com seems wonderfully whizzy and working like a charm.
Are both sites hosted on the same server?
Could you do a 302 or 307 from the old to the new. If any traffic is then seen on the IP for the old site you know it's suspicious (and if the new starts having issues, then it's probably a bad scraper which is following the redirect). Could also try disallowing all in robots.txt but it's unlikely a custom scraper would pay any attention to it.
All IPs except those originating via CloudFlare have been blocked. Cloudflare reports security threat originating countries and many are coming from Singapore and China. I have thus blocked them at CloudFlare for now, but will re-enable when this over.
The reason many pages are not loading is because ORI have put a 1 second timeout at the firewall on requests to try and stop the rogue traffic. But that is not enough for many of the pages to load so I've asked it to be increased.
It's tempting to think that rehosting will solve the problems but if it's due to traffic targeted at Brickset directly then obviously that will continue wherever it's hosted.
What I might do tomorrow is sign up for the other hosting package and transfer traffic to it, if only to see what happens. It will be expensive, but if it does turn out to solve the problem it will be worth it.
Just for info, keep up the good work !!
Taking this theory back to the Brickset issues. It's very unusual for a small site like Brickset to be targeted by a DDos attack. And if it was just Brickset being targeted, I'm fairly sure that Huw's web host would immediately say that it was all the fault of his site clogging down their bandwidth and shut his site down to keep all of their other customers happy.
However, if it was their entire network being targeted, they'd be extremely reluctant to admit this to any of their customers because they'd face a mass exodus.
It's just a theory, but if I were to have a bet, I'd say its' the entire server not just Bricklink being targeted.
BTW, I keep getting empty RSS feeds with a 'The Brickset.com RSS feeds have moved' title.
Also, I don't think that it is too much 'legit' traffic going through the page and that the provider isn't able to handle it. Bricklink used to be hosted with the same people until last summer, and BL has a shedload more of traffic (I think!).
The RSS has been moved to http://www.brickset.com/feed to align it with the normal location for wordpress-like site feeds.
Have to say, I really like it - don't be in too much of a rush to change it, @Huw !
Categories on the left I like!
But maybe that's just me.
I can't say I like the notification and inbox pop ups you get with the default either. I want all my PM's / notifications to appear with one click like it did in the old version.
But again, maybe that's just me.
Edit: Sorry, I thought you were talking about the main site!
^ No, i wouldn't make it match. I like the distinction between the two. The forum needs to be simple, as the chat is the attraction. If it looks all glitz and bling, it'll detract from the threads/be less functional, if that makes sense?
^^^ I don't think there is any need to... What, do you kids need bright colors to entertain yourselves? ;oP
Recent discussions link now at the bottom, and the header is 'Brickset blue'.
I think that'll do for now, don't you?