 Nobody likes waiting for a website to load into a browser. When a site takes more than a few seconds to start loading users will leave. So how do you make a web site that loads fast? Here are techniques that I have developed over a number of years to improve website response, particularly for database-driven sites. 1.0 Hardware
Moore's law (processor speed doubling every 18 months) has given everyone access to lots of CPU power at an affordable cost.
For a site getting 20,000 hits/day I recommend at least 2 GHz processor, and preferably two processors in an SMP configuration.
For database-driven sites I recommend at least 512MB to 1GB RAM. I generally allocate at least 50% of RAM to database buffers. If I'm running a proxy server to cache "hot" objects I generally allocate it at least 10% of RAM.
I prefer SCSI disks, at least 10,000 RPM. Try using a SCSI controller with at least 64MB RAM. An alternative to SCSI is 10,000 RPM SATA discs. If you use IDE disks then buy the 8MB RAM models.
The following checklist can be used as a guide
- choose SCSI over IDE disks
- choose faster disk spindle speed (RPM)
- choose SCSI controllers with RAM >= 64MB
- if using IDE drives, choose models with an 8MB buffer
- use hardware RAID if its available
- use software RAID if hardware RAID is not available
- get a fast net connection; or if necessary use colocation service
2.0 Software
2.1 Tuning the Operating System
Tuning memory/IO/networking can help improve system performance.
The best advice I have for Linux users is to upgrade to a recent 2.6 kernel. This kernel has new scheduler algorithms that improve interactive response time compared to earlier 2.4 kernels.
This article discusses the features of the 2.6 kernel.
The following
Redhat guide, while somewhat old, contains much useful information on Linux tuning. The guide can be used for just about any Linux system, not only Redhat
I have found that increasing the TCP/IP window size can improve networking performance. It helps increase download speed, especially for users with fast connections, by increasing the number of TCP/IP packets that can transmitted before a packet acknowledgment is expected. For linux the following line can be added to the boot script:
echo 131072 > /proc/sys/net/core/wmem_max
2.2 Use static HTML in preference to CGI
Web servers process static HTML faster then cgi. So write HTML!
For dynamic web sites you can try to "precompute" dynamic content into static content. The X-Cart shopping cart allows its product pages to be generated as static HTML.
Notice how fast the product pages load at www.eyo.com.au. The product pages have been converted to static HTML using a proprietary system.
2.3 Compress the http stream
The http protocol provides for http compression using gzip.
Is your site compressed? Check whether your site uses gzip compression at Lonkor.com
If you are running Apache 2 you can turn on compression using mod_deflate and adding the following lines to the Apache configuration file (httpd.conf):
LoadModule deflate_module modules/mod_deflate.so
AddOutputFilterByType DEFLATE text/html text/plain text/xml
2.4 Remove HTML whitespace in cgi
Extra whitespace will increase load time for a page. So it makes sense to remove whitespace to improve response time.
This Perl script gives a quick amd simple method to remove most redundant whitespace from HTML. It works with dynamic HTML and can be easily adapted for generating static HTML.
2.5 Use large database buffers
As mentioned previously, I usually allocate at least 50% of RAM to database buffers.
Mysql comes with several sample configuration files. The main parameter to consider is "key_buffer". Conside a mysql configuration file like the following: (/etc/my.cnf)
skip-locking
set-variable = key_buffer=256M
set-variable = max_allowed_packet=1M
set-variable = table_cache=64
set-variable = sort_buffer=3M
set-variable = net_buffer_length=8K
set-variable = myisam_sort_buffer_size=8M
2.6 Optimise SQL statements
Make sure your are familiar with your database system's optimisation features. Optimising SQL statements to use appropriate indexes can lead to significant performance improvements.
Mysql provides various optimisation tools and techniques in Chapter 7 of the Mysql manual.
The mysql "explain" command can be use to determine which indexes are being used in a select statement. If the wrong indexes are being used, or worse still a full table scan is being used, the select statement will need to be rewritten
Additional information can be gained using the 'analyse table' command. It is used to determine the distribution of keys in a table and the results can be useful in designing joins.
2.7 Use a transparent proxy server
A transparent proxy server will speed up access to static objects like static HTML and gifs/jpegs. The proxy server delivers static content faster than a web server and can be configured to keep objects in RAM. Transparency means that web browser clients do not know about the proxy server and do not need to be reconfigured
The Squid proxy server is often included in Linux distribtions.
This guide shows how to configure Squid as a transparent proxy.
3.0 Conclusions
This guide presents a number of techniques for designing fast loading websites. The techniques focus on LAMP (Linux/Apache/Mysql/Perl) but may be applicable to other systems.
The techniques have been used in a number of production systems to improve response time for web services. |
