1 post tagged “load balancing”

We advocate stateless architecture as an architecture for its numerous benefits. When combined with cloud computing from Amazon and failover/load balancing features built into the client it offers a very efficient way to handle sporadic heavy loads. Take cricket sites such as cricinfo, cricketnext or stock information sites such as moneycontrol which experience heavy loads during certain times on certain days. In all these cases a server has to push information constantly to the client.

Amazon gives the ability to start additional server instances on the fly. These can be started / shutdown as the load increases. Essentially an an infinite data center at a cost of Rs4 per hour per server for a 2Gig, 2CPU instance. As clients don't have affinity to the server any server can service any client. Who will handle the load balancing? Normally it is a reverse proxy sitting at the server end, sort of a gatekeeper. This can get overwhelmed if a million connections come in. We propose to do load balancing on the client. A client is given a bunch of server urls to connect to on its first request. Client tries out alternate servers till it finds an available server. A server can decide to service a request based on its load.

There is one more aspect in this scenario. A browser client has to make requests to the server to get up-to-date information. This creates a significant overhead in connection management on the server. Persistent connections can be used to solve this issue - details in a later post.