Immortal application in a mortal server..

I remember few years back when I was running the Websphere, it was very difficult to kill a instance. On killing of one instance, a deamon was spawning another. Killing that one, one more was coming up. I had tried "kill -9", "kill -KILL" - No help. When one was going down, other was coming up. It is not like, I have 2 process instances running and on killing of both, the application  is completely down. Rather, it is only one and taking birth each time it is murdered.

Why not this happens for a server. If I consider that is possible, virtually the application is immortal. 

[I have taken this section out from my regular writting as I want to highlight on one particular issue. Recently I had mapped a production instance to amazon EC2 instance in web DNS mapping. I issued a shutdown command on EC2 machine by mistake. It just took me 2 mins to bring up a snapshot of the last instance as I was taking the snapshot. Then I pointed to this new server address in web DNS mapping. This process took a good 30mins and the system remained down for so long :(]

Consider a machinsm is available, where web DNS mapping is instantaneous to the depployed server. In this spectrum, I have an application which transcends through server instances. The possibilities lies on the snapshot creation. Before death can I take an exact snapshot and all next instances starts from there. Amazon EC2 has infiniband networking to S3. So, network latency is not my concern. Hadoop is there for my rescue to transfer the big files. The big file is the tar ball of the complete mnt. If you are reading my previous posts, you know you can shove everyting to mnt. That means the complete harddisk portion of my interest to the S3. This helps on saving the database files, other contents, application state and everything else.

During shutdown,

1. Create a snapshot tar ball of /mnt of the portion of harddisk of my interest. - 30sec
2. Copy this to S3.
3. Start another instance.
4. When the next instance comes up, it self configures itself during start up taking the file from S3.

I have taken snapshot and created a new instance quickly but manually. It takes just a minute. How can I do it automatically?

First I looked for linux shutdown hooks like init. No luck. If you are aware of any, please let me know. Now I am going to explore by creating a blank java application and starting it during init and putting a shutdown hook there. This shutdown hook will grap the linux shutdown. I am concluding this reading the post at http://forum.java.sun.com/thread.jspa?threadID=504889 . Once I finish this experiement, I blog about my findings :) This will open up following possibilities in EC2:

1. Immortal application in a mortal server
2. No information loss when instance shutted down.
3. Many app instance creation when load increases. [What about the database??? I have a big blog coming on this. I am half way through]

Don't forget to help me on implementing a linux hook of shutdown rather than going via the java way...  A work around not a clean solution :(