Skip to main content

ManagedConnection has no connection handle - ORA-02396: exceeded maximum idle time

If there's not any endgame, we're in quicksand. We take one more step, and we're still there, and there's no way out. - Richard Shelby
We've had yet another outage this week. We were exceeding connection idle time limit in Oracle. with almost 270 connections blocked by the application just waiting idly. There was something that was holding up the connections, so we rebounded the server. We thought that was an off shoot. However this repeated on another instance. Around 90 minutes into this blocked state, we got this stack trace.

org.springframework.jdbc.UncategorizedSQLException: Hibernate operation: could not inspect JDBC autocommit mode; uncategorized SQLException for SQL [???]; SQL state [null]; error code [0]; Connection handle is not currently associated with a ManagedConnection; nested exception is java.sql.SQLException: Connection handle is not currently associated with a ManagedConnection

[org.hibernate.util.JDBCExceptionReporter] ORA-02396: exceeded maximum idle time, please connect again

The first thing that came to our *brilliant* minds was - In Jboss, there is an idle-timeout-minutes setting for local-tx-datasource configuration - Why is the max idle time out coming from oracle (which in our case is way larger than idle-timeout-minutes specified in oracle-ds.xml).

A little research showed that the idle-timeout-minutes, is for connections that have been used by the app and returned to the pool. So, there are connections held up in the application. We started looking at where we have long standing transactions. To help us, The admin took a thread dump. all of them were held in a Mainframe call. The call was improperly configured to 90 minutes of time out. Fixed the time out. So far, we are good :)
Labels:

Popular posts from this blog

Powered By

As it goes, We ought to give thanks to people who power us. This page will be updated, like the version page , to show all the tools, and people this site is Powered By! Ubuntu GIMP Firebug Blogger Google [AppEngine, Ajax and other Apis] AddtoAny Project Fondue jQuery
Read More >>

Decorator for Memcache Get/Set in python

I have suggested some time back that you could modularize and stitch together fragments of js and css to spit out in one HTTP connection. That makes the page load faster. I also indicated that there ways to tune them by adding cache-control headers. On the server-side however, you could have a memcache layer on the stitching operation. This saves a lot of Resources (CPU) on your server. I will demonstrate this using a python script I use currently on my site to generate the combined js and css fragments. So My stitching method is like this @memize(region="jscss") def joinAndPut(files, ext): res = files.split("/") o = StringIO.StringIO() for f in res: writeFileTo(o, ext + "/" + f + "." + ext) #writes file out ret = o.getvalue() o.close() return ret; The method joinAndPut is * decorated * by memize. What this means is, all calls to joinAndPut are now wrapped (at runtime) with the logic in memize. All you wa...
Read More >>

How to Make a Local (Offline) Repository in Ubuntu / Debian

If you are in a place where you dont have internet (or have a bad one) You want to download .deb packages and install them offline. Each deb file is packaged as a seperate unit but may contain dependencies (recursively). apt-get automagically solves all the dependencies and installs all that are necessary. Manually install deb files one by one resolving each dependency would be tedious. A better approach is to make your own local repository. Before you actually make a repo, You need *all* deb files. You dont practically have to mirror all of the packages from the internet, but enough to resolve all dependencies. Also, You have to make sure, you are getting debs of the correct architecture of your system (i386 etc) # 1. make a dir accessible (atleast by root) sudo mkdir /var/my-local-repo # 2. copy all the deb files to this directory. # 3. make the directory as a sudo dpkg-scanpackages /var/my-local-repo /dev/null > \ /var/my-local-repo/Packages # 4. add the local repo to sour...
Read More >>