Skip to main content

AppEngine Cold Startups - Rendezvous with Google App Engine Team

I was at Google App Engine IRC Chat today. Following a lot of concerns of others and of my own, I wanted to put forth my ideas to them about cold startups. I tried to promote my ideas of different approach to classloading, osgi, profiling the runtime itself etc. But time was not enough. I will probably talk again next time. For the most part, It looks like Google is almost ready to come up with *reserved instances* approach. But here is what they have to say.

(9:59:52 PM) sar4j: This is about cold startups for java. I run my website on appegnine py. But I am a java developer. So I have to get reasonabl undersanding about how gae/j works. To simulate the same performance, I am adding a test.js script to exisitng website. So every time a visit comes to gae/py a hit happens to gae/j. I still see that gae/j is very aggressively spinned down. (given number of requests are the same) is there any algorithm on how and when spinning down happens?
(10:04:17 PM) apijason_google: sar4j: We're definitely aware of the issues that Java developers are facing with cold startup times, especially since many Java frameworks exaggerate the loading time. We have reserved instances on the roadmap, which should allow you to pay to keep a minimum number of app instances warm, and we're working on other general improvements to increase performance. Right now, the only general way to ensure that your application is not
(10:04:18 PM) apijason_google: cycled out is to have steady traffic. However, this should be genuine traffic -- our systems know when you're just using a cron job to keep your app alive.
(10:05:14 PM) cgrinds: apijason_google, knows and doesn't keep it hot?
(10:05:56 PM) ikai_google: cgrinds: Not necessarily, but we discourage using cron jobs
(10:06:36 PM) sar4j: apijason_google: So you advice me to try a full blown move to gae/java for a few days and see? Will your algorithm then spin down the app at longer intervels?
(10:06:37 PM) ikai_google: cgrinds: *using cron jobs purely as a mechanism to keep applications loaded
(10:06:51 PM) cgrinds: ikai_google, yep understood
(10:08:25 PM) prencher: apijason_google: consider this to me my lobbying request for you to lobby that it should be hot spares, not minimum instances (minimum instances just reduces the problem - hot spares eliminate it, mostly)
(10:08:38 PM) apijason_google: sar4j: If you have steady traffic, then your application should stay loaded. Are you using the Python runtime already?
(10:09:11 PM) sar4j: yes. it is much more realtime - than my study on gae/j
(10:09:38 PM) sar4j: my apps : sar (py) and sarjava
(10:10:12 PM) apijason_google: prencher: Understood. I think the work for this is largely underway already, but I'll pass on your request.
(10:10:18 PM) sar4j: so now if I want to move to java, I am a little hesitant only because of cold startups.
(10:11:37 PM) apijason_google: sar4j: With steady, real-time traffic, your Java app should perform just fine and shouldn't be cycled out. I'm interested in hearing your impressions if you decide to deploy a Java app vs. your already deployed Python app.
(10:11:39 PM) ikai_google: sar4j: You should base your solution on which language or tool fits you best. For many developers, this is Python. It sounds like you've got code written already
(10:13:56 PM) Wesley_google: sar4j: we're curious... what are the diffs in startup times b/w your Py vs. Java apps?
(10:19:11 PM) sar4j: Well, (steady, real-time traffic, ) is a little abstract. I get real traffic (regular search generated traffic, etc about 1 hit evry 10 min or less). For my blog and site (they run resources and dynamic content off of sar.appspot, the python version - right now). The python runtime give sub 2 sec response consistently, and most times sub second. While different versions of java apps (with, without spring, slim based, and ust servlet based) all spin down agressively and give 3-12 secs of response time EVERY time. they some times even time out. I understand what I am currently doing is as good as running a cron job. But I want to benchmark before I move to java. So, my question is, If this traffic can be considered steady, real-time traffic, and that my app will spin down less (not like live forever - but atleast not go down for almost every request) then I am willing to go for it.
(10:22:15 PM) sar4j: Also I noticed that the time taken after the webapp gets the hit (real time of response) is very little(in msecs for bare minimum app and about 1-2 secs for a spring app). Its the runtime bootup that takes a most of the time.
(10:22:39 PM) apijason_google: sar4j: In that case, I would probably stick with your Python version, at least until reserved instances are available. At 1 hit every 10 minutes, it's likely that your Python app is getting spun down too. But the extra complexity and weight of the JVM means that Python instances spin up faster than Java instances, even discounting any frameworks you might be using.
(10:23:29 PM) sar4j: apijason_google:thanks. I guess I will do the same.
(10:25:15 PM) apijason_google: Five more minutes left -- get your questions in! :)
(10:25:26 PM) sar4j: However, If I may ask, I would like to know, how the jetty run time work. Does it create a seperate java process everytime spindown/up happens or just the webapp is turned off and on?
(10:26:24 PM) sar4j: What takes the most time for the runtime? the classloading, or the file io for reading the jars, etc. do y have any such profiling info?
(10:27:03 PM) sar4j: the appstats gives profiling AFTER the runtime is up,
(10:28:20 PM) frew_google: sar4j: The same process is reused sometimes (you can see this experimentally by checking whether or not your global variables are still around from last time).
(10:28:55 PM) yodler12: Just wanted to comment that I am very excited for whatever upcoming new features you'll be announcing at IO. Having you google nerds working on that stuff reminds me why I chose App Engine - it's almost like having your own Googlers working for you!
(10:29:22 PM) sar4j: I ask because, if its the classloading or the library over head that is slowing down the runtime's bootup, may be you should think in the direction of OSGi.. most libraries are same. if an OSGi model is adopted, the runtime bootup takes less time and less CPU
(10:29:38 PM) apijason_google: sar4j: We don't have any public profiling info. currently. Maybe we can put this together at some point.
(10:29:41 PM) sar4j: saves green and saves the hassle of reserved instances.
(10:32:45 PM) frew_google: sar4j: You might want to look at (not written by a Googler, and I haven't had a chance to verify the numbers, but the methodology of adding/removing libraries and checking cold start times is pretty sound if you're worried about class loading times).
(10:35:23 PM) apijason_google: OK Everyone, thanks for a great session. It's past 10:00, so today's office hour is officially complete, though a few of us may stick around for a few more minutes. The next chat session is in two weeks, Wednesday, May 5th, from 7:00-8:00 p.m. PST. Cheers!
(10:36:36 PM) sar4j: thanks frew_google = I went even further. I unjarred libraries and put them directly under classes to check. But I still got a range of response times (2-8) secs. If you can access my logs, you can look at sarjava logs. different versions. the cpu time hangs around 1000msec but the total response time varies drastically.

Popular posts from this blog

Being a Vegetarian

I am a Proud Vegetarian. I don't eat Meat or Eggs. People say its hard here in US to be one. I beg to differ. The mere fact that I am hail and healthy these 4 years is a definitive proof. Apart from being bullied and trash talked by The Meat-Eaters, There is really nothing that makes this choice of mine any more than a debatable issue at a lunch or dinner. Other things aside, I am writing this blog having watched a PETA Video. Before you click on the play button, I ask you - If you are a vegetarian : Dont watch it. If you are not : Dare to watch it till the end. If you think going veg is just a fashion, think again . Even if you just want to do it for Fashion . Do it. Go Vegetarian. And Feel better asking the waiter for a Vegetarian Entrée in your next lunch.

Using Equinox CommandProvider to make OSGi console interactive.

After fiddling with the First Bundles that "Hello World"-ed upon Activation, You want to see more interactivity in OSGi. Although Using OSGi for an interactive Command Line Application would be like this one would be, well, a callable over-kill, I am going to start with an example and Expand it in later posts. So, please Welcome CommandProvider. CommandProvider is an EQUINOX specific API for extending the Console. This basic Example illustrates how to get a command from console and do something in java and also gets your feet wet on Service Registry package; import org. eclipse .osgi.framework.console .CommandInterpreter; import org.eclipse.osgi.framework.console.CommandProvider; public class DisplayMessageCommand implements CommandProvider { public void _say(CommandInterpreter ci) { ci.print("You said:" + ci.nextArgument()); } @Override public String getHelp() { return "\tsay - repeats what you say\n"; } }

How to Make a Local (Offline) Repository in Ubuntu / Debian

If you are in a place where you dont have internet (or have a bad one) You want to download .deb packages and install them offline. Each deb file is packaged as a seperate unit but may contain dependencies (recursively). apt-get automagically solves all the dependencies and installs all that are necessary. Manually install deb files one by one resolving each dependency would be tedious. A better approach is to make your own local repository. Before you actually make a repo, You need *all* deb files. You dont practically have to mirror all of the packages from the internet, but enough to resolve all dependencies. Also, You have to make sure, you are getting debs of the correct architecture of your system (i386 etc) # 1. make a dir accessible (atleast by root) sudo mkdir /var/my-local-repo # 2. copy all the deb files to this directory. # 3. make the directory as a sudo dpkg-scanpackages /var/my-local-repo /dev/null > \ /var/my-local-repo/Packages # 4. add the local repo to sour