• one of your services (mysql, apache, nginx , php-fpm or whatever is of a vital importance) goes down
• one of your services (again, mysql, nginx, apache, php-fpm, etc) is overloaded, meaning that CPU used by the process is almost 100%

In this case your web application won’t be available any more. In a good scenario, you’ll notice the outage yourself or one of the angry users will drop you a line via email or twitter. In a bad scenario your website won’t be available for hours. So what do you do?

1. First of all, make sure that all processes that are needed to run your website smoothly are added to the autoload. Meaning, that after server reboot they are available automatically and there is no need to manually start them. How to do that, really depends on your server OS. Just google “add apache to autoload ***your OS name goes here***”.
2. I would recommend installing and configuring some kind of software that would be monitoring processes running on your server and in case some of them go down or are consuming too much of resources, start/restart them automatically. There is a good open source solution for *nix systems called Monit. It is easy to install and configure and even sends you emails when something goes wrong. Don’t forget to add Monit to the server boot load list.

The post Do you monitor your server? appeared first on cloudinservice.com.

nano avmem.sh

In the window opened, paste the following text:

echo “Enter the process name (i.e. apache2):”
read process
ps -ylC $process | awk ‘{x += $8;y += 1} END {print “Process Memory Usage (MB): “x/1024; print “Average Proccess Size (MB): “x/((y-1)*1024); print “Total Number of Processes: “(y-1)}’

Then press Ctrl+x and answer yes when prompted.

Your script is ready! You can now run it by executing the following in the command line:

./avmem.sh

It’ll ask you to enter the name of a process (i.e. apache2 for Apache, mysqld for MySQL, nginx for nginx, you name it) and give you information about number of processes running, total amount of memory consumed by these processes and average amount of memory consumed by one process instance.

Hope it helps!

The post How to calculate average amount of memory for a process appeared first on cloudinservice.com.

Nginx is also a web server (as Apache and IIS) and can be used on its own, but often it is used just as a proxy server. Since it is much more light weight than Apache (meaning it uses just a fraction of resources Apache uses to perform the same task), it makes sense to let nginx serve all static content (images, javascript, css, html, css, etc) to offload Apache. Apache then will be busy with more important jobs, such as generating dynamic content.

You can ask, why not to use nginx to serve everything if it needs much less resources? My understanding is that nginx is not designed to perform complicated tasks, it’s job is to do simple things, but fast and resource-wise cheap. On this schema nginx is solely responsible for delivering all static content to the user, and also being a proxy for the dynamic content generated by apache:

Of course it is possible to configure nginx to serve static content not only for Drupal sites, but below nginx config files are Drupal-specific. You can use this configuration for the case you host only one site on your server/VPS and also for the case when there are several websites (domains) on this server (which can be configured as virtual hosts on Apache level).

So let’s start:

1. Install nginx, the easiest way is to use packages, for example for Ubuntu the command will be

apt-get install nginx

2. Make sure Apache is configured to run on some other port, i.e. port 81 (I’ll be using 81 in all my examples below). Depending on your installation and configuration, you may need to change one or more files, but usually it is either apache2.conf, httpd.conf or ports.conf. Just look for the following line of code:

NameVirtualHost *:80
Listen 80

If you are using Virtual Hosts, the port should be changed there as well.

<VirtualHost *:80>

…

</VirtualHost>

80 should be changed to 81 in all the above examples.

3. Nginx main configuration file is called nginx.conf, but it is more logical to split the configuration into several files:

nginx.conf includes general server-wide configuration that should apply to all sites hosted on this machine

user www-data;
worker_processes  8;  #one process per CPU, so if your server has 8 CPUs, write 8 here.
timer_resolution 100ms;
worker_rlimit_nofile 16384;

error_log  /var/log/nginx/error.log; #error log location
pid        /var/run/nginx.pid;

events {
worker_connections  2048;
use epoll;
}

http {
include       /etc/nginx/mime.types;
default_type  application/octet-stream;
access_log  off;
sendfile        on;
keepalive_timeout  65;
tcp_nodelay        on;

gzip  on;
gzip_types text/plain application/xml+rss text/javascript application/javascript application/x-javascript text/css text/xml application/xml;

server_tokens off;
client_max_body_size 1000M;

include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}

sites-enabled/00-nginx-default – default server, this configuration will be used in case current domain does not match specific domains. Using this configuration nginx will pass ALL requests to Apache, doing basically nothing. It can be useful, if for example you’d like to install phpmyadmin or some other tools on a different domain/subdomain for security purposes and easier administration (which means that there won’t be big load passing through this configuration).

server {
listen 80 default; #this is here it says nginx is running on port 80
server_name _;
root /var/www;

access_log  /var/log/nginx/default.access.log;

location /nginx_status {
stub_status on;
access_log  off;
allow 127.0.0.1;
deny all;
}

location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://localhost:81; #apache is on port 81
}
}

Domain specific configuration will go into a separate file for each domain:

server {
server_name mysite.com;

root /var/www/mysite; #document root for your site
access_log off; #for production environment; for testing you can enable it

location ~* \.(htm|html)$ {
error_page    404 = @imagecache;
proxy_cache_valid  200 302  10m;
proxy_cache_valid  404  1m;
access_log off;
expires 2m;
}

# serve static files directly
location ~* \.(gif|jpg|jpeg|png|wmv|avi|mpg|mpeg|mp4|js|css|mp3|swf|ico|flv|htm)$ {
error_page    404 = @imagecache;
proxy_cache_valid  200 302  10m;
proxy_cache_valid  404  1m;
access_log off;
expires 30d;
}

# ImageCache work arounds
location ~ ^/sites/.*/files/imagecache/ {
try_files $uri @rewrite;
}

location @imagecache {
client_max_body_size    50m;
client_body_buffer_size 128k;
proxy_send_timeout   90;
proxy_read_timeout   90;
proxy_buffer_size    4k;
proxy_buffers     16 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_connect_timeout 30s;

#in case image cache is not generated yet (drupal is generating them on first access, send request to apache

proxy_redirect  http://mysite.com:81   http://mysite.com;
proxy_pass   http://localhost:81;
proxy_set_header   Host   $host;
proxy_set_header   X-Real-IP  $remote_addr;
proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ /\.ht {
deny  all;
}

location / {
client_max_body_size    1000m;
client_body_buffer_size 128k;
proxy_send_timeout   90;
proxy_read_timeout   90;
proxy_buffer_size    4k;
proxy_buffers     16 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_connect_timeout 30s;
proxy_redirect  http://mysite.com:81   http://mysite.com;
proxy_pass   http://localhost:81/;
proxy_set_header   Host   $host;
proxy_set_header   X-Real-IP  $remote_addr;
proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
}
#this file should exist on your disk, it will be shown in case apache is overloaded and nginx is not able to ping it
error_page 403 /misc/mysiteovercapacity.htm;
error_page 502 /misc/mysiteovercapacity.htm;
error_page 503 /misc/mysiteovercapacity.htm;
error_page 504 /misc/mysiteovercapacity.htm;
}

So, in case you have more than one site running on the server, you’ll have to create a separate configuration file for every site (otherwise all of them will go to 00-default-nginx and it won’t bring any value, just on the contrary – additional layer when delivering content).

Enjoy!

The post configure nginx proxy for drupal site appeared first on cloudinservice.com.

To do this, just include the below lines into your httpd.conf apache configuration file:

<IfModule mpm_prefork_module>
StartServers 2
MinSpareServers 2
MaxSpareServers 5
MaxClients 200 #must be customized
ServerLimit 200 #must be customized
MaxRequestsPerChild 100
</IfModule>

KeepAlive Off

Some explanations are here:

• StartServers – this is how many apache instances should start at the very beginning when apache is started. I set it to 2, and it works fine for me.

• MinSpareServers – minimum number of spare servers that should be running waiting for potential requests. MinSpareServers=2 worked fine for me too.

• MaxSpareServers – maximum number of spare servers that should be running waiting for potential requests, obviously >= MinSpareServers. In my working example MaxSpareServers=5.

• MaxClients & ServerLimit. You can use this shell script to determine an average amount of memory consumed by one Apache process. In addition to that it’ll show total amount of memory consumed by all Apache processes. Just unzip and execute as follows:
  

  wget http://cloudinservice.com/wp-content/uploads/2011/03/ap.sh.zip

  unzip ap.sh.zip

  sh ap.sh

  The output will be something like that:

  Apache Memory Usage (MB): 1372.6
  Average Proccess Size (MB): 54.9041

  Try to execute it several times to compare the numbers; good results will be shown when server is under a heavy load. Now when you know average amount of memory consumed by Apache and total amount of memory of your server, it is possible to calculate value to be used for MaxClients setting. For example, if in average one your Apache process consumes 50MB RAM and server RAM is 2GB, and you want to leave 512MB for the rest processes, then:
  MaxClients = (2GB – 512MB)/50MB = 30.72 ~ 30.
  ServerLimit is, as I understand, the same thing, but while MaxClient setting can be changed on the go without a need to restart Apache, for new ServerLimit value to take effect Apache restart is required. MaxClients should always be <= ServerLimit. To make it easy, I set ServerLimit = MaxClients calculated by above formula.

• By default MaxRequestsPerChild = 0, which means that httpd process will never expire. However, it can happen that there are some memory leaks in your PHP scripts (especially if you are using lot’s of third-party contributed modules). To protect yourself from accidental memory leaks, you can set how many requests should be processed by Apache process before it dies. In my example I used MaxRequestsPerChild = 100 and it worked fine. You can experiment to find a value that suits you better.

• KeepAlive directive description is pretty straight forward on Apache site. If KeepAlive is On, it allows multiple requests to be sent over the same TCP connection, which can boost site performance up to 50% for some sites. This mostly works for pages that have lot’s of images. However, I’ve noticed that it can lead to the situation when processes never die and that can load the server a lot. Instead, I’m using a proxy server to serve images and other static content (I’ll make another post to clarify it) and in my configs KeepAlive is set to Off.

The post tune apache performance using mpm prefork module appeared first on cloudinservice.com.