Last week i was facing high resource usage on my server and causing outage to the main website and other services hosted. When i looking for the culprit i find out that The Piratebay Feed builder is the one eating up my resource. I was getting around 1500-2000 hits in 30 minutes. When i wrote this app in the first place i never thought i will never get this much request. Also the unavailability and slow response time from the upstream pbay server caused more cpu load and php execution time. Because of this reason i had to temporally stop service and planning to move pbay service alone to some other server. I tried so many free php hosting service and all of them complained about resource use above free account level and service got suspended in hours.This forced me to move to another solution. What i did is ported the entire php app to java and created a spring web app. I used Google app engine to host the app. Within one hour it consumed 50% frontend instance hours. So i had to deploy the same app in 5 different Google App Instance and wrote a simple php load balancer app to equaly distribute the load among the google apps. The service is up and running now. During my troubleshooting and analysis i found out some IP addresses making over 200 requests in 10 minutes. Current setup barely support the current user load, front end instance hours reaches up to 85% in all the five servers at the end of day. If traffic will continue to increase service might go down again. Because of these reason I placing some restrictions on the service. Don’t worry the service is still free but i am currently banning some ip address which makes 100+ requests in 30 minutes. As of writing this post banning is done manually, but i am planning to make banning automatic.So if you are banned you will get a http response code of 403 with message ‘I dont like you’.
If you are using this service please consider donating to cause and keep the service available and free. Happy torrenting :).
if you would like to share your script, that would be awesome!
as of now i am not planning to
The code is far from rocket science.. put this together in 10 min and has pretty much the same effect: http://pastie.org/8470809
how do you use this? I mean I have it setup on a server, and the code changed to my specifications, I think.. yet it doesnt print the xml?
Hope i am not banned by mistake. I am using opera mail RSS reader and it can be a bit finnicky, but it does not request too often. Mostly i do it manually.
pbay.maxters.net not resolving in dns at all, is it down for good?
OOPs sorry it is there just refusing port 80
Love the pbay feedservice. tnx.
the last couple of days i am getting an error in my rss reader (poll once every 24h) :
“failed to open stream: HTTP request failed! HTTP/1.1 403 FORBIDDEN”
but when i open the exact same (feed)url in a browser, it works fine…..
any ideas?
which rss reader are you using?
I am busy or something went wrong on Upstream
user is anonymouspackage is Anonymousquota exceeded
hello hello fail fail
Your Package is Anonymous
Your Quota per Day is 50000
Total request today 50000
Total Usage 100.0
Link Generator
PirateBayUrl
Feed Url
So, for anyone using a VPN this won’t work, because many are probably using that VPN thus it is banned because that IP makes too many requests.
can you opensource the code, so someone with more resources can host this?
Service is down, can you please let us know when it is going to work again?
i’m actually getting a bad gateway error 502. just letting you know.. Thanks for your hard work