Sarath Chandra Pandurangi's Blog

Application cache (appcache) is a powerful feature in HTML5. However, it does come with baggage. Many (see links below) advocated ferociously against it due to tricky issues it comes with. For someone who is just testing waters, these issues may throw them off grid. Knowing them before hand helps reduce some unpredictable effects.

This site rendered js and css (now even images) dynamically. Contents of files are stored in Google BigTable Datastore and are served by a DynServlet. The DynServlet maps the name of the file to content of the file in DB. This was slowly using up a lot of resources. Adding a layer of memcache, saved some Datastore API calls. But the memcache calls were still significant. So an ETag mechanism is introduced. A Global ETag is generated on any file posted or updated, and (G-ETag is) stored in Datastore. To save App Engine resources (which are counted to cost), the ETag header is obtained from each request checked with GDTS from backend. If it is the same a "304 – Not Modified" response is sent back, asking the client to use the local cache copy. If it is not exactly same, a the request is passed through and rendered dynamically. This guarantees all content will be served fresh after every deployment. Requests in between deployments may be served from client-side cache, because ...

Java on Google appengine has always been an matter of concern, especially with low traffic. People have expressed their anxiety with their application's performance. This site/blog runs on blogger. However the images and JavaScript are dynamically served from Google's Big table.   This site was earlier on Google appengine  - Py. A much performant environment. It has its owns issues. However, relatively it was a more worry free environment for small applications. But as an enterprise language, java was to be used for this site. The migration was finally done after three unsuccessful attempts. And eventually, with an extremely slim home grown framework – smvc, a simple mvc with dependency injection, the site is run on java.   Initially there were resource intensive cold startups. The smvc framework was an answer to it. But lately it appears that the appengine java is plaguing the site with a bunch of 500 errors, more frequently. This behavior started around Oct 27 and persi...

here is what happens on twitter.. @antderosa: Zuck isn't turning Facebook back on unless he gets final edit on The Social Network @emilcardell: You know your a geek if you think that it is worse that #github is down then #facebook @sargenthouse Facebook is down, you know what that means right? NO POKING. @opb: BREAKING NEWS: Facebook is down. Worker productivity rises. U.S. climbs out of recession. @sacca: When Facebook goes down, everyone comes to Twitter to talk about it. When Twitter goes down, the world falls mute. @jlnd: Don't worry, if Facebook doesn't come back you can retrieve all your private data from any of the companies they sold it to. @LouBrutus: DNS FAILURE: Facebook is down which means 9 months from today, many children will born. @irockiroll the day has finally come. cloverfield ate facebook @willmckinley: First they cut Facebook. Then Social Security. Then Medicare. @kris10turner OMG I just paid my cable bill thinking that's why my ...

Recently, a heavy burst of traffic into our production webservice caused this error. As with all Mysql errors, this shows little information. Like any sane person would think, we checked if the filesystem was full. We found the /tmp has 98% free space. Permissions are all fine. No locks on table. Increasing /tmp partition - NoGo. Just crazy error message. However, the issue was beating us time and again. It took us a couple of days and we realized the cause was a select query. The query was fetching from a table of about 2 million records. Although it was fetching a limited subset, the column in the order by had no index. This was causing mysql to load records into pagefile in tmp for sorting, apparently. Created the index on the sort column seemed to have solved the problem. We also augmented it with the following options in my.cfg sort_buffer_size=2M table_cache=1800 thread_cache_size=384 tmp_table_size=64M

IaaS : Infrastructure as a Service eg: Amazon EC2, Rackspace, Verizon Private Cloud. PaaS : Platform as a Service eg: App Engine, Azure. SaaS : Software as a Service eg: Google Docs, Acrobat.com. Here is a collection of slideshows about IaaS, Paas and SaaS.

I was at Google App Engine IRC Chat today. Following a lot of concerns of others and of my own , I wanted to put forth my ideas to them about cold startups. I tried to promote my ideas of different approach to classloading, osgi, profiling the runtime itself etc. But time was not enough. I will probably talk again next time. For the most part, It looks like Google is almost ready to come up with *reserved instances* approach. But here is what they have to say. (9:59:52 PM) sar4j: This is about cold startups for java. I run my website sarathonline.com on appegnine py. But I am a java developer. So I have to get reasonabl undersanding about how gae/j works. To simulate the same performance, I am adding a test.js script to exisitng website. So every time a visit comes to gae/py a hit happens to gae/j. I still see that gae/j is very aggressively spinned down. (given number of requests are the same) is there any algorithm on how and when spinning down happens? ..... (10:04:17 PM) api...

This is connected to the post in Google App Engine Group .  Although Google Appengine gives out a java container, I had to roll back to the python code I used for version SarathOnline 3.1 – for a sole reason – perfromance. My (blog) site has a very low traffic (compared to what google thinks a regular traffic application). Averaging ~100 visits/day. All the css and js is dynamically generated. My current python code is doing pretty well servicing these requests @ a maximum of 2.something secs

I am back from a big vacation. I found this decent presentation for Cloud Computing from sun. This gives a very basic but good explanation about how cloud benefits Enterprise. The presentation can also be downloaded as a mp4 here src: https://slx.sun.com/node/1179275657

After looking at a way you can do a waited lock on a synchronised monitor in java, it needs a much more object oriented way of handling multiple scenarios. The sychronised-wait-notify paradigm works if you wait and notify on the monitor based on one condition (wait before borrow if not available). The Give back doesnot wait ever, It just gives back. The pool kills any over flow. What if you need to put a conditional wait on that too? something like (give back only if object pool is not full and WAIT until space is available) Here is where you have Java 5's Lock - Conditions paradim comes handy. A lock is objectification of the monitor, Threads that need to be synchronised on one monitor is can go through Locks instead. However, Locks are more useful if you have multiple conditions on which you what these threads to wait and be notified. Take the above scenario, which is the case of a Bounded buffer where there are two conditions on which threads need to wait, empty and ful...

By now, I have worked enough on OpenSessionInViewFilter , Hibernate and Spring in some projects. Hibernate 3 uses Lazy Loading by default . So, We use for maintaining lazy loaded collections to display in view. But, lately, as I mentioned in earlier posts, we were having some issues with connections that were not returned to pool. To understand where connections were held up we need to see how OSIVfilter, Session, HibernateTransactionManager and Spring work together - Where and when connections are picked and released. OSIVFilter guarentees that each request will have one and only one Hibernate Session. A Session is a lightweight abstraction of data in the database pulled into vm converted as objects. So for a session to fetch data from db, it needs connections. This connection is available from the datasource through a TransactionManager. In general, if you have one database, You would use Springs HibernateTransactionManager. A Session generally contains one Transaction. However, ...

In terms of scalability, a frequent term used is "Resource Contention". Most commonly this means, there is a shared object, file handle, db connection, or similar for which Threads are trying to get ownership. Until they obtain this ownership they either wait or exit on a timeout mechanism. In normal flow, they simply wait - This state is called blocked state. In an asynchronous mode (generally meaning a spawned child thread), there can be a Time-Out flow that can be defined to control the maximum length of time the thread can take to complete a task. In a blocked state, There may be "Resource Contention" in one thread that is avoiding other threads to proceed. In a JEE environment, This Resource Contention is caused commonly by Transactions(, Synchronized blocks, Locks). Since User Threads are discouraged in JEE, Synchronized blocks and Locks are less often encountered as issues. That's theory. If you dint understand much, Dont feel despair. I come from the ver...

A little more than 3 weeks ago, I was asked to rewrite a Multi-threading component that *pooled* connections to Mainframe HTTP Gateway. There was an existing implementation with synchronized array of connections which was initialization with a configurable size. The problem was this implementation was home grown way back in history, and the confidence level in that code was very low. Especially because its going into a JEE container and code-managed threads were not scaling atone to the container. My first instinct was to use Apache Commons pooling . For one, this would have been much more QA-ed than what I would have implemented. The primary logic would be approximately the same, create-pool > borrow-object > return object. Everything could be done with commons pooling, using the GenericObjectPool . On a finer level, I need to manage the life of the objects themselves - make sure the encapsulated connection is not stale etc, in which case invalidate it A sample example (as in Ja...

I have been interested on scalability of Humongous apps recently. Google declared the first ever free real time stock quotes provision today. So Tomorrow morning, If you are on Google Finance, You are not watching the 15-20 minute delayed results. For an average Joe, this might be just another thing google can do. But for Technology enthusiasts, This is a much interesting step. Just give you an idea, a few days back Facebook opened up its chat feature to its ~70 million strong member base. Google does a similar feat with its Chat integration for its, I don't even know how many, million users. IMs are the least of issues. Presence Notification packets, Security, Keeping so many connections open, memory to handle all these are of highest consideration. It would be interesting how Google will explain feed polling from NASDAQ and delivery and still keep it realtime for Gigillian hits google.com/finance gets every day. What is even harder to imagine is how an open webapp(even availab...