Rate Limiting with WordPress’ Transient API

I run a web app called leenk.me, it’s a Social Media Optimization application for WordPress. Basically it publishes your WordPress content to Twitter / Facebook / Google Buzz whenever you publish new content to your website. There are a lot of “advertisers” who have been signing up for the service and one in particular has been hitting the service hard — really hard — about 4000 API requests in an hour.

This is a pretty big problem, all these social networks rate limit their connections (close to 300 requests per hour). So making 4000 requests in an hour could cause them to ban the user, or worse, ban leenk.me. So I’ve had to implement a rate limit of my own. I thought about doing this with iptables to block anyone who exceeds a certain number of requests per hour, but the violating users wouldn’t know what happened. I really wanted needed a way to prevent them from over-connecting to the social networks.

This is where the WordPress Transients API comes in. The transients API is very similar to the Options API but with the added feature of an expiration time. This is basically how the transients API works in WordPress:

The only problem with WP Transients is that there isn’t a good way to create “rolling transients” (as I call them). A rolling transient is a transient that rolls in time. In other words, I want to rate-limit a users connection by 1 hour from their current API call, but if the users does not make any API calls within the hour the transient should expire.

This is how I implemented “rolling transients”:

With this code, you now have rolling transients… if your users exceed the number of calls within the past hour they will be rejected. If not, the transient will expire as it should. Let me know if you found this useful or if you have any tips for making it better (it seems a little “hacky” to me).

Comments

  1. If you are already tracking the number of requests made in the last hour, why not push overflow requests onto a heap and if the quota has room, ie. traffic lightens up, pull requests from the heap and post them?

    Since you have serialized data already, it should be pretty cheap (resource wise) to write the queued requests (and timestamp) to a temp file using the hostname.

    To make it more efficient, it might be worthwhile to move on to other users’ queues. That way you could make the most of every run.