Last week i was facing high resource usage on my server and causing outage to the main website and other services hosted. When i looking for the culprit i find out that The Piratebay Feed builder is the one eating up my resource. I was getting around 1500-2000 hits in 30 minutes. When i wrote this app in the first place i never thought i will never get this much request. Also the unavailability and slow response time from the upstream pbay server caused more cpu load and php execution time. Because of this reason i had to temporally stop service and planning to move pbay service alone to some other server. I tried so many free php hosting service and all of them complained about resource use above free account level and service got suspended in hours.This forced me to move to another solution. What i did is ported the entire php app to java and created a spring web app. I used Google app engine to host the app. Within one hour it consumed 50% frontend instance hours. So i had to deploy the same app in 5 different Google App Instance and wrote a simple php load balancer app to equaly distribute the load among the google apps. The service is up and running now. During my troubleshooting and analysis i found out some IP addresses making over 200 requests in 10 minutes. Current setup barely support the current user load, front end instance hours reaches up to 85% in all the five servers at the end of day. If traffic will continue to increase service might go down again. Because of these reason I placing some restrictions on the service. Don’t worry the service is still free but i am currently banning some ip address which makes 100+ requests in 30 minutes. As of writing this post banning is done manually, but i am planning to make banning automatic.So if you are banned you will get a http response code of 403 with message ‘I dont like you’.
If you are using this service please consider donating to cause and keep the service available and free. Happy torrenting :).