Skip to main content

Decorator for Memcache Get/Set in python

I have suggested some time back that you could modularize and stitch together fragments of js and css to spit out in one HTTP connection. That makes the page load faster. I also indicated that there ways to tune them by adding cache-control headers.

On the server-side however, you could have a memcache layer on the stitching operation. This saves a lot of Resources (CPU) on your server. I will demonstrate this using a python script I use currently on my site to generate the combined js and css fragments.

So My stitching method is like this
def joinAndPut(files, ext):
    res = files.split("/")
    o = StringIO.StringIO()
    for f in res:
        writeFileTo(o, ext + "/" + f + "." + ext)  #writes file out
    ret = o.getvalue()
    return ret;

The method joinAndPut is *decorated* by memize. What this means is, all calls to joinAndPut are now wrapped (at runtime) with the logic in memize. All you want do is use the someregion+files+ext as your key in memcache. I extended an example in App Engine CookBook.
def memize(region="", time=3600):
    """Decorator to memoize functions using memcache.
       The calling function must give a memizekey named parameter
    def decorator(fxn):
        def wrapper(*args, **kwargs):
            if (not "memizekey" in kwargs):
                raise Exception("memize-d function does not send key")
            key = region + kwargs["memizekey"]
            del kwargs["memizekey"]

            data = memcache.get(key)
            if data is None or not settings.DEBUG:
                data = fxn(*args, **kwargs)
                memcache.set(key, data, time)
            else:"Cache hit");
            return data
        return wrapper
    return decorator

This version of memize takes a region instead of key. The intention is a key is generally given at runtime, and is controlled by application logic, so it goes into the *actual* call (ex given below). But the region is (generally a prefix) decided at coding time to avoid runtime collisions. You could also update memcache by force at runtime. A sample call would be something like
class MainPage(webapp.RequestHandler):
  def get(self, files, ext):
      if ext == "css":
          self.response.headers['Content-Type'] = 'text/css'
      elif ext == "js":
          self.response.headers['Content-Type'] = 'text/javascript'
      forceUp = self.request.get("force");
      if 'true' == forceUp:
          forceUp = True
          forceUp = False
          self.response.headers["Cache-Control"] = "public, max-age=432000"
      con = joinAndPut(files, ext , memizekey=files + "." + ext, memizeforce=forceUp) 

For people coming from java background, this is similar to an around Advice in Aspect Oriented Programming. If I get time, may be I'll post another sample in AOP :)

Popular posts from this blog

Javascript: Convert Strings to Binary (and representing in a nerdy way!)

I follow those GoogleDevelopers Videos . Sometime back, in one of the presentations on GoogleIO, there was this interesting string of dots at the bottom of each page of the presentation . They looked like random big and small dots. A similar bunch of dots were also on the T-shirt of a presenter was wearing in another presentation . While it seemed something in the pattern, I could not find what it was. Finally, another presenter cleared the matter that those dots are just binary representation of "GOOGLEIO" (So much for advertizing Google IO, Impressive!). So I wanna do it. Takes me back to days of those DSP classes at school. Nerdy me had to churn some old brain cells. I remember those first programming language classes in Pascal and C when you were asked to do fibonacci series and converting a binary string to ascii codes. That *experience* came handy here: Check it out! Text to Binarize: For those who came to copy the javascript code to convert string to binary,

MySql Copying Table Structures.

Some times you need to copy only table structures across databases. This article describes two ways of doing it. If the whole database schema need to be exported, mysqldump is very effective. A --nodata flag will dump all tables' schema. Like this. mysqldump --nodata -p -u username databaseName But if you want to copy a specific table, individually, you could use "create table like" feature. You could create it even from a different database. However it must be on the same mysqld instance. Like this. create table newtable like oldtable; --Or from a table in other database create table mytable like otherdatabase.tablename;

Add jquery in Chrome console

Many a time, a page you are debugging doesnot have jquery. This simple js will add(or prompt you to overwrite) jquery to any page from chrome console. javascript:if(!window.jQuery||confirm('Overwrite\x20current\x20version?\x20v'+jQuery.fn.jquery))(function(d,s){s=d.createElement('script');s.src='';(d.head||d.documentElement).appendChild(s)})(document);