Denie's Tech Blog » MySQL

Scripting a MySQL InnoDB Engine Conversion

Denie — Sat, 10 Jul 2010 19:50:08 +0000

0) Backup your database.
You should probably be doing this already. Now’s a good time to make sure that your backups ran.

1) Create the script.
You’ll need the correct permissions to query the database. Here’s the command. Be sure to change as it fits.

# mysql -p -e "show tables in ;" | \ tail --lines=+2 | \ xargs -i echo "ALTER TABLE {} ENGINE=INNODB;" > alter_table.sql

2) Run the script.

# mysql --database= -p < alter_table.sql

3) Verify it by running this command in mysql:

mysql> show table status;

SOURCE

Share this on :

Simple MySQL Backup with auto 3 days old file deletion

Denie — Sun, 04 Jul 2010 13:57:27 +0000

/usr/local/bin/mysql.backup.sh

#!/bin/bash
NOW=$(date +"%m-%d-%Y")
OLD=$(date +"%m-%d-%Y" --date="3 days ago")
PROJECT="project_name"
LOCATION="/home/backup"
FILE="$PROJECT.$NOW.sql"
FILE2="$FILE.gz"
FILEOLD="$PROJECT.$OLD.sql.gz"
EMAIL="youremail@domain.com"
$SQLUSER="username"
$SQLPASS="password"
$SQLNAME="database_name"

cd $LOCATION ; \
rm -f $FILEOLD ; \
mysqldump -u $SQLUSER --password=$SQLPASS $SQLNAME > \
$LOCATION/$FILE ; \
gzip $LOCATION/$FILE ; \
echo "Backup location is in $LOCATION/$FILE2" | \
mail -s "[$PROJECT] MySQL Backup" $EMAIL

Then you can put it on your cron (background process)

Below cron will execute the script on Saturday at 12AM:

0 0 * * 6 /usr/local/bin/mysql.backup.sh

Good luck!

Share this on :

How to tune Apache and Mysql

Denie — Fri, 24 Jul 2009 17:29:43 +0000

By default, Apache comes preconfigured to serve a maximum of 256 clients simultaneously. This particular configuration setting can be found in the file /etc/httpd/conf/httpd.conf

If your server has 2 GB of RAM, and you’re sharing your server with MySQL(true in my case), you’ll want to reserve about half of it for Apache (1 GB)

MaxClients: here is the process of determining MaxClients. type

ps -U apache -u apache u

See the number of apache process running in you command prompt.

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
apache 7694 0.0 0.3 42704 6680 ? S 18:30 0:00 /usr/sbin/httpd

The above indicates that a single httpd process is using 6.6 MB of RSS (Resident Set Size) memory (or non-swapped physical memory) and that it is using 42 MB of VSZ (Virtual Size) memory. This depends on the number of modules you have loaded and running in Apache.

As shared libraries are included in this number, it’s not 100 percent accurate. We can assume that half the RSS number is “real” memory. Let’s assume that each httpd process is using (6.6/2=3.3) 4 MB of memory. So if you have 1 GB ram then divide it with 4 MB of memory, which leaves room for around 256 concurrent httpd processes.

Set MaxClients 256

Somebody prefers to set MaxClients using following rule

MaxClients = 150 x RAM (GB)

So for example if you have 2 GB RAM (dedicated for apache) set this value to 300. In my case IT WILL BE 150

Some individuals maintain that each httpd thread uses about 5 MB of “real” memory. So they determine by the following way..

MaxClients = RAM(MB)/5

So for example if you have 2 GB RAM (dedicated for apache) set this value to 409. In my case IT WILL BE 204(1 GB for apache)

Note: There is no reason for you to set it any higher unless you have a specific problem with this value. A high value can lead to a complete server hang in case of a DOS attack. A value too low can create timeout problems for your clients if the limit is reached

StartServers – Sets the number of child server processes created on startup. This setting depends greatly on the type of webserver you run. If you run low traffic websites on that server set it low to something like 5. If you have resource intensive websites on that server you should set it close to MaxClients.

MaxRequestsPerChild – Controls the number of request the a child serves before the child is killed. This should not be set too low as it will put an unnecessary load on the apache server to recreate the child. I suggest setting it to 1000.

But we are going to use 2000 for handling heavy traffic load properly.

MinSpareServers and MaxSpareServers – MaxSpareServers and MinSpareServers control how many spare (unused) child-processes Apache will keep alive while waiting for more requests to put them to use. Each child-process consumes resources, so having MaxSpareServers set too high can cause resource problems. On the other hand, if the number of unused servers drops below MinSpareServers, Apache will fork. Leave those values to: MinSpareServers 5 MaxSpareServers 10

ServerLimit: Its better to keep Server limit same as the value of MaxClients.

MaxRequestsPerChild: I’ve Kept default apache value for this one.

So few changes need to be made in httpd.conf file which is located in /etc/httpd/conf/ directory

StartServers     140
MinSpareServers    5
MaxSpareServers   10
ServerLimit      150
MaxClients       150
MaxRequestsPerChild 4000

[Note]: Response time depends on MaxClients. If you increase the MaxClients number, server will response more quickly for each request but a high value can lead to a complete server hang.

Ab is a tool for benchmarking the performance of your Apache HyperText Transfer Protocol (HTTP) server. It does this by giving you an indication of how many requests per second your Apache installation can serve.

uptime command in your root login should not yield a load average above 1, and the server should respond to commands quickly

ab -n 10000 -c 200 -k http://your_url
-c = concurrent connections
-t = time limit
-n = # of requests

Keep tuning until you hit your maximum desired load average. For servers used interactively often, having a load above 3 is way too much to use the server comfortably. For servers used mostly as real servers, a maximum load average of 10 should be acceptable. More than that, and you’ll find yourself needing to reboot the server when experiencing heavy traffic conditions, because no terminal or remote console will respond quickly to commands, and managing the server will be impossible.

How to configure few things in php.ini file for supporting huge traffic

* Enable the compression of HTML by putting in your php.ini:

output_handler = ob_gzhandler

** Switch from file based sessions to shared memory sessions. Compile PHP with the –with-mm option and

set session.save_handler=mm

Configure mysql. Change my.cnf file for better performance.

The database parameters are tuned for systems with 1 GB RAM (for ISO CD images). If you have higher RAM, please change the following in the “my.cnf” MySQL configuration file under /etc/mysql or /etc directory.

For a machine running with 512 MB of RAM, you can set these to:

key_buffer=128M table_cache=1024 sort_buffer=64M read_buffer=2M record_buffer=4M

For a machine running with 1 GB of RAM, you can set these to:

key_buffer=256M table_cache=2048 sort_buffer=128M read_buffer=2M record_buffer=8M

For a machine running with 2 GB of RAM, you can set these to:

key_buffer=512M table_cache=3072 sort_buffer=256M read_buffer=2M record_buffer=8M

For a machine running with 4 GB of RAM, you can set these to:

key_buffer=1G table_cache=4096 sort_buffer=512M read_buffer=2M record_buffer=8M

Original Post

MySQL Master-Slave Replication

Denie — Mon, 08 Jun 2009 16:20:43 +0000

To setup Master-Slave Replication the first thing you need to do is create a user on the Master server that allows replication.

# mysql -u root -p
mysql> grant replication slave on *.* TO repl@”%” identified by ‘[repl password]‘;
mysql> quit

Be sure to replace [repl password] with the actual password you want to use. Also, you must ensure that your firewall has port 3306:tcp open, the default port for the mysql server service.

Next, exit your ini file (typically /etc/my.cnf on Linux servers) to start binary logging of the Master server. You may or may not want to use the last line to ignore changes to the mysql database since that is the database used for mysql configuration and permissions.

Under the [mysqld] heading add the following lines:

log-bin=mysql-bin
server-id=1
binlog-ignore-db=”mysql”

Restart your mysql server service.

Before we start copying changes, we want to make sure the data on each server is the same, so dump the data from the Master server and add it to the Slave server. This can easily be performed using mysqldump as follows:

# mysqldump -u root -p[password] [database]>/home/[user]/[database].sql

Be sure again to replace [password] with the actual password, [database] with the actual name of each database, one at a time.

Now that you have a snapshot of the data, get the binary position of the log file.

# mysql -u root -p[plain-text password]
mysql> SHOW MASTER STATUS;

The output should look something like this:

Next, copy the databases to your Slave server.

# scp /home/[user]/*.sql [Slave IP]:/home/[user]/

At this point we are done with the Master server. Now for the Slave.

Edit the Slave server’s mysql configuation file (typically /etc/my.cnf on Linux servers) to identify its server number, master host and user.

Under the [mysqld] heading add the following lines:

server-id=2
master-host = [IP of Master Server]
master-user = repl
master-password = [repl password]
master-port = 3306

Again, ensure you replace [repl password] with the actual repl user password, and [IP of Master Server] with the IP address of the Master server.

Insert the data from the Master server into the Slave server databases for each of the databases.

# mysql -p[password] [database] < /home/[user]/[database].sql

Be sure to replace [password] with the root users password, and [database] with each database’s name, one at a time.

Restart the mysql server service.

Now log in to Mysql and configure the Slave replication.

# mysql -u root -p
mysql> CHANGE MASTER TO MASTER_LOG_FILE=’[Filename written down]‘,
MASTER_LOG_POS=[Position written down];
mysql> START SLAVE;
mysql> SHOW SLAVE STATUS\G

Slave_IO_State status information should identify “Waiting for master to send event”. If it stops at “Connecting to Master” check your log file.