Skip to main content

Decorator for Memcache Get/Set in python

I have suggested some time back that you could modularize and stitch together fragments of js and css to spit out in one HTTP connection. That makes the page load faster. I also indicated that there ways to tune them by adding cache-control headers.

On the server-side however, you could have a memcache layer on the stitching operation. This saves a lot of Resources (CPU) on your server. I will demonstrate this using a python script I use currently on my site to generate the combined js and css fragments.

So My stitching method is like this
def joinAndPut(files, ext):
    res = files.split("/")
    o = StringIO.StringIO()
    for f in res:
        writeFileTo(o, ext + "/" + f + "." + ext)  #writes file out
    ret = o.getvalue()
    return ret;

The method joinAndPut is *decorated* by memize. What this means is, all calls to joinAndPut are now wrapped (at runtime) with the logic in memize. All you want do is use the someregion+files+ext as your key in memcache. I extended an example in App Engine CookBook.
def memize(region="", time=3600):
    """Decorator to memoize functions using memcache.
       The calling function must give a memizekey named parameter
    def decorator(fxn):
        def wrapper(*args, **kwargs):
            if (not "memizekey" in kwargs):
                raise Exception("memize-d function does not send key")
            key = region + kwargs["memizekey"]
            del kwargs["memizekey"]

            data = memcache.get(key)
            if data is None or not settings.DEBUG:
                data = fxn(*args, **kwargs)
                memcache.set(key, data, time)
            else:"Cache hit");
            return data
        return wrapper
    return decorator

This version of memize takes a region instead of key. The intention is a key is generally given at runtime, and is controlled by application logic, so it goes into the *actual* call (ex given below). But the region is (generally a prefix) decided at coding time to avoid runtime collisions. You could also update memcache by force at runtime. A sample call would be something like
class MainPage(webapp.RequestHandler):
  def get(self, files, ext):
      if ext == "css":
          self.response.headers['Content-Type'] = 'text/css'
      elif ext == "js":
          self.response.headers['Content-Type'] = 'text/javascript'
      forceUp = self.request.get("force");
      if 'true' == forceUp:
          forceUp = True
          forceUp = False
          self.response.headers["Cache-Control"] = "public, max-age=432000"
      con = joinAndPut(files, ext , memizekey=files + "." + ext, memizeforce=forceUp) 

For people coming from java background, this is similar to an around Advice in Aspect Oriented Programming. If I get time, may be I'll post another sample in AOP :)

Popular posts from this blog

Javascript: Convert Strings to Binary (and representing in a nerdy way!)

I follow those GoogleDevelopers Videos . Sometime back, in one of the presentations on GoogleIO, there was this interesting string of dots at the bottom of each page of the presentation . They looked like random big and small dots. A similar bunch of dots were also on the T-shirt of a presenter was wearing in another presentation . While it seemed something in the pattern, I could not find what it was. Finally, another presenter cleared the matter that those dots are just binary representation of "GOOGLEIO" (So much for advertizing Google IO, Impressive!). So I wanna do it. Takes me back to days of those DSP classes at school. Nerdy me had to churn some old brain cells. I remember those first programming language classes in Pascal and C when you were asked to do fibonacci series and converting a binary string to ascii codes. That *experience* came handy here: Check it out! Text to Binarize: For those who came to copy the javascript code to convert string to binary,

MySql Copying Table Structures.

Some times you need to copy only table structures across databases. This article describes two ways of doing it. If the whole database schema need to be exported, mysqldump is very effective. A --nodata flag will dump all tables' schema. Like this. mysqldump --nodata -p -u username databaseName But if you want to copy a specific table, individually, you could use "create table like" feature. You could create it even from a different database. However it must be on the same mysqld instance. Like this. create table newtable like oldtable; --Or from a table in other database create table mytable like otherdatabase.tablename;

javascript maxlength for textarea with \r\n breaks in java (esp Firefox)

Textareas allow new lines to enter. These are represented by \n (1) or \r\n (2) characters. But when you save to DB you have a limit to certain length of chars. There is no maxlength attribute in HTML that will stop you from entering data. This is generally acomplished by Javascript. You do a onkeyup hook and stop event or trim after textarea.value.length > maxlength. There are many other solutions out there.. But.. Here is the problem that most of those solutions overlook, How do you deal with the count on \n and \r\n representations. Lets first see how it matters. If the text entered has new lines, the length is calculated differently in Firefox and IE. When you enter a Text like 01234 567890 You expect the textarea.value.length to be 11. (10 chars + new line).On the backend, however, java would recieve it as 12 chars (10 chars + \r\n) (this is irrespective of FF or IE). So you are effectively saving 12 chars to DB. Worse yet, IE seems to figure textarea.value.length as 12 (