Decorator for Memcache Get/Set in python

I have suggested some time back that you could modularize and stitch together fragments of js and css to spit out in one HTTP connection. That makes the page load faster. I also indicated that there ways to tune them by adding cache-control headers.

On the server-side however, you could have a memcache layer on the stitching operation. This saves a lot of Resources (CPU) on your server. I will demonstrate this using a python script I use currently on my site to generate the combined js and css fragments.

So My stitching method is like this

@memize(region="jscss")
def joinAndPut(files, ext):
    res = files.split("/")
    o = StringIO.StringIO()
    for f in res:
        writeFileTo(o, ext + "/" + f + "." + ext)  #writes file out
    ret = o.getvalue()
    o.close()
    return ret;

The method joinAndPut is *decorated* by memize. What this means is, all calls to joinAndPut are now wrapped (at runtime) with the logic in memize. All you want do is use the someregion+files+ext as your key in memcache. I extended an example in App Engine CookBook.

def memize(region="", time=3600):
    """Decorator to memoize functions using memcache.
       The calling function must give a memizekey named parameter
    """
    def decorator(fxn):
        def wrapper(*args, **kwargs):
    
            if (not "memizekey" in kwargs):
                raise Exception("memize-d function does not send key")
            key = region + kwargs["memizekey"]
            del kwargs["memizekey"]

            data = memcache.get(key)
            if data is None or not settings.DEBUG:
                data = fxn(*args, **kwargs)
                memcache.set(key, data, time)
                logging.info("Cached");
            else: logging.info("Cache hit");
            
            return data
        return wrapper
    return decorator

This version of memize takes a region instead of key. The intention is a key is generally given at runtime, and is controlled by application logic, so it goes into the *actual* call (ex given below). But the region is (generally a prefix) decided at coding time to avoid runtime collisions. You could also update memcache by force at runtime. A sample call would be something like

class MainPage(webapp.RequestHandler):
  def get(self, files, ext):
      if ext == "css":
          self.response.headers['Content-Type'] = 'text/css'
      elif ext == "js":
          self.response.headers['Content-Type'] = 'text/javascript'
      
      forceUp = self.request.get("force");
      if 'true' == forceUp:
          forceUp = True
      else:
          forceUp = False
          self.response.headers["Cache-Control"] = "public, max-age=432000"
      
      con = joinAndPut(files, ext , memizekey=files + "." + ext, memizeforce=forceUp) 
      self.response.out.write(con);
  

For people coming from java background, this is similar to an around Advice in Aspect Oriented Programming. If I get time, may be I'll post another sample in AOP :)