Data / Cluster Synchronization

From SuperbHosting Support Wiki

Jump to: navigation, search

When multiple servers are clustered together to run a single application, there has to be some means of synchronization.


Contents

Tiers

Most large applications are broken down into tiers, to separate the areas of responsibility for the different nodes in the cluster. Two tiers is the most common now:

Two Tier

  1. Web Server Tier
  2. Database Tier


Web Servers

Static Content

Web servers always have some content (HTML files, images, CSS files, etc) that does not change very often. Since it does not change very often, you can just copy the files to each server. Or you can manually use rsync to synchronize the files to the other servers, whenever there is an update to the static content.


Uploaded Files

So web applications accept uploaded files. If those files are stored into the local filesystem, then they can only be served by the cluster node that received the upload. Depending on what these files are used for, and how many need to be kept, rsyncing each node to each other node can work for small amounts of files (< 500MB).

But generally, it is better that the files be stored on some sort of shared storage, where it is visible to all of the nodes. A database BLOB field is a good option. See File Servers below.


Sessions

What are Sessions?

Web applications that need to store per-user state, need track sessions. Any application that has a Login page, has to create a session for that user. Plus, some web applications that need to track the click-trail through a website create anonymous sessions.

A problem arises, if a user logins into one web node, but the load balancer sends successive requests to a different node, which they have not logged into. There are two solutions:

1. Use sticky sessions. The first time a new source IP hit the cluster, stick that IP to a specific node, so all successive requests go to the same node. The problem is that the load balancer hardware has to use a lot of memory to track all of these sticky sessions. The table can grow quite large. But as long as the sticky session timeout is set to the same as the session timeout on the node, then it works ok. However, some applications need sessions to last a very long time. But sticky sessions don't really scale beyond an hour.

2. Layer 7 Switching. The first time a new user hit hits the cluster, add a Cookie to the the nodes reply. This contains the node ID of the node that sent the reply. Then every time that user hits the cluster again, their browser will include that cookie in the request. And the load balancer will use that to direct the request to the same node. This method works very well, as is quite scalable. The drawback is, for HTTPS, the load balancer needs to decrypt all of the traffic in order add and read the Cookie. This adds some load to the load balancer.


Session Replication Solutions

Both of these solutions have the drawback that sessions only exist on a single node. If your application needs to keep sessions for a long time, and you need the flexibility to change node servers, the above methods may not work for the customer. In which case, they need session synchronization. This somewhat depends on the platform:

1. Tomcat (or another J2EE server): Session synchronization is built in. But it needs to be configured. Everytime a new session is created on a node, that node broadcasts the details over the network to the other nodes.


2. IIS: Very similar to the Tomcat situation. Session synchronization is available, but needs to configured.


3. PHP: By default PHP stores sessions in local files. You need to configure it for either storing the sessions in a database, or in memcached (http://www.php.net). memcached needs to be installed. Normally in a small cluster, memcached is installed on each web node. See memcached section below. We recommend that customers develop their PHP applications with a framework that has support for this.


4. Ruby, Python: Similar to PHP. The best choices are either a database, or memcached. If you are using a framework like Rails, it support for storing sessions in the datatabase or memcached is built-in, but it does need to be configured.


Databases

Databases are difficult to synchronize since the rate of change is often very high. Plus, a database system has to ensure the database maintains consistency. If you have multiple copies of the database being updated at the same time, maintaining consistency becomes difficult. So the typical pattern is:

  • Small Config: A single master database node that handles all database requests. All database changes are replicated to a standby server, that does nothing but wait for the master to fail. If the master fails, the standby server takes over. This type of configuration works up to about 2 to 4 web servers. At that point, the database requests from the web servers usually becomes too great for a single master db node.
  • Medium Config: A master and standby node as with the Small configuration, but with the addition of some read-only database nodes. Then the application is configured to send changes (DELETE, INSERT, UPDATE) to the master node, and lookups (SELECT) to the read-only servers. The read-only servers can be put behind the load balancer too. Given that most applications generate database requests that are 75 to 90% lookups, this scales out quite well. Most applications could scale up to 50 servers with this configuration.
  • Large Config: Partition the database, and setup multiple master databases. Plus, everything from the medium configuration.


MS-SQL

There are really too many options. MS-SQL supports replication to a standby server (one server is active, and one is standby). It also supports multi-master replication (both servers are active).


MySQL

MySQL supports active-standby replication. However, there is no automated way to promote the standby server to a master, if the master fails. Some people have built scripts to do this for them, but it is not very smooth.

The most desirable configuration is using DRBD and Hearbeat for MySQL failure instead. This works quite smoothly. One server is active, and one is standby.

If the single master server becomes a performance bottleneck, setting up regular MySQL replication two one or more read-only servers, works well.


File Servers

Some applications need to work with files, and the files can't be stored in a database. And the rate of change of the files is quite large, and there are a large number of files. Examples, are images for a photo gallery site.

In this case, using a master-standby NFS servers is probably the best bet.

For Windows, you can use Windows Cluster services to create a CIFS cluster, but it requires a SAN. Windows Cluster services works quite well.


memcached

memcached acts as a very simple distributed cache. It is not replicated across multiple servers though. So don't store anything in memcached that you can't afford to lose. Many sites use memcached to store sessions. If a memcached server dies, all of the users with their sessions on that server, are logged out. For many sites, that is acceptable.

memcached can also be used as database cache. Before a request is sent to the database tier, check memcached. If there is nothing in the cache, then query the database. memcached in this mode, can greatly extend your database resources. This also works well, if you are storing sessions in the database, as memcached automatically caches this too then.

memcached is the secret sauce that powers some of the largest sites on the Internet.

Retrieved from "http://www.superbhosting.net/support-wiki/Data_/_Cluster_Synchronization"
Personal tools