Pbay FeedService Updates

feed

Last week i was facing high resource usage on my server and causing outage to the main website and other services hosted. When i looking for the culprit i  find out that The Piratebay Feed builder is the one eating up my resource. I was getting around 1500-2000 hits in 30 minutes. When i wrote this app in the first place i never thought i will never get this much request. Also the unavailability and slow response time from the upstream pbay server caused more cpu load and php execution time. Because of this reason i had to temporally stop service and planning to move pbay service alone to some other server. I tried so many free php hosting service and all of them complained about resource use above free account level and service got suspended in hours.This forced me to move to another solution. What i did is ported the entire php app to java and created a spring web app. I used Google app engine to host the app. Within one hour it consumed 50% frontend instance hours. So i had to deploy the same app in 5 different Google App Instance and wrote a simple php load balancer app to equaly distribute the load among the google apps. The service is up and running now. During my troubleshooting and analysis i found out some  IP addresses making over 200 requests in 10 minutes. Current setup barely support the current user load, front end instance hours reaches up to 85% in all the five servers at the end of day. If traffic will continue to increase service might go down again. Because of these reason I placing some restrictions on the service. Don’t worry the service is still free but i am currently banning some ip address which makes 100+ requests in 30 minutes. As of writing this post banning is done manually, but i am planning to make banning automatic.So if you are banned you will get a http response code of 403 with message ‘I dont like you’.

If you are using this service please consider donating to cause and keep the service available and free. Happy torrenting :).

  • lander

    if you would like to share your script, that would be awesome!

    • maxters

      as of now i am not planning to

    • ties

      The code is far from rocket science.. put this together in 10 min and has pretty much the same effect: http://pastie.org/8470809

      • Jojhn

        how do you use this? I mean I have it setup on a server, and the code changed to my specifications, I think.. yet it doesnt print the xml?

  • Egon Ruuda

    Hope i am not banned by mistake. I am using opera mail RSS reader and it can be a bit finnicky, but it does not request too often. Mostly i do it manually.

  • Disturbed

    pbay.maxters.net not resolving in dns at all, is it down for good?

  • Disturbed

    OOPs sorry it is there just refusing port 80

  • Raymond99

    Love the pbay feedservice. tnx.

    the last couple of days i am getting an error in my rss reader (poll once every 24h) :
    “failed to open stream: HTTP request failed! HTTP/1.1 403 FORBIDDEN”

    but when i open the exact same (feed)url in a browser, it works fine…..

    any ideas?

    • maxters

      which rss reader are you using?

  • Disturbed

    I am busy or something went wrong on Upstream

  • Disturbed

    user is anonymouspackage is Anonymousquota exceeded

  • Disturbed

    hello hello fail fail

    Your Package is Anonymous

    Your Quota per Day is 50000

    Total request today 50000

    Total Usage 100.0

    Link Generator

    PirateBayUrl

    Feed Url

  • Guest123

    So, for anyone using a VPN this won’t work, because many are probably using that VPN thus it is banned because that IP makes too many requests.

  • northernViking

    can you opensource the code, so someone with more resources can host this?

  • Netsmile

    Service is down, can you please let us know when it is going to work again?

  • jamie337nichols

    i’m actually getting a bad gateway error 502. just letting you know.. Thanks for your hard work