Aditya Kothadiya's Blog

Entrepreneurship, programming, design, productivity, books, philosophy and more.

Automating Your MySQL Database Backup On Media Temple’s Grid-Server

Aditya May 25th

Comments

Let me admit this – I did not backup the Tweeght’s database until today. Somehow I kept procrastinating it for no reason. Since Tweeght is growing consistently, it was crucial for me to backup its database on daily basis and prepare myself from loosing all the crucial data in some sort of catastrophic event.

I use Media Temple’s Grid-Server for Tweeght. MT has good knowledge base article on – How can I Backup and Restore a MySQL database?.

But this article just mentions about how to backup your database for once. I wanted to automate this backup process on daily basis. So I Googled a little-bit, and came across this wonderful script. So I took that script, updated it for MT’s Grid-Server infrastructure, and now I’m all set within 20 minutes to take daily backups of my crucial databases.

Here are the steps I followed:

1. Go to /data directory in your account. I preferred this over /domains directory because this is accessible only by you and not by public. You don’t want the public to accidentally access dump of your database.

2. Create /db_backup directory. Go to /db_backup directory and create three new directories – /scripts , /daily and /recent.

3. Go to /scripts directory and open up your favorite editor. Copy and paste following code -

#!/bin/bash
# Set a value that we can use for a datestamp
DATE=`date +%Y-%m-%d`
# Our Base backup directory
BASEBACKUP="/home/XXXXX/data/db_backup/daily"

for DATABASE in `cat /home/XXXXX/data/db_backup/scripts/db_list.txt`
do
# This is where we throw our backups.
FILEDIR="$BASEBACKUP/$DATABASE"

# Test to see if our backup directory exists.
# If not, create it.
if [ ! -d $FILEDIR ]
then
mkdir -p $FILEDIR
fi

echo -n "Exporting database:  $DATABASE"
mysqldump --add-drop-table -h internal-db.sXXXXX.gridserver.com -udbXXXXX -pPASSWORD $DATABASE | gzip -c -9 > $FILEDIR/$DATABASE-$DATE.sql.gz
echo "      ......[ Done Exporting to local backup, now exporting for remote backup] "
cp $FILEDIR/$DATABASE-$DATE.sql.gz /home/XXXXX/data/db_backup/recent/$DATABASE.sql.gz
echo "      .......[Done]"
done

# AutoPrune our backups.  This will find all files
# that are "MaxFileAge" days old and delete them.
MaxFileAge=4
find $BASEBACKUP -name '*.gz' -type f -mtime +$MaxFileAge -exec rm -f {} \;

4. Change few things in above code -

  • Replace all the instances of ‘XXXXX’ with your gridserver account number.
  • Replace ‘PASSWORD’ with your SSH password

5. Now save this file as something like ‘db_backup.sh’

6. Open up your editor one more time, and write the names of databases you want to back up on each single line -

dbXXXXX_proj1
dbXXXXX_proj2

7. Now save this file as something like ‘db_list.txt’

8. Basically, the script pulls a list of databases to be backed up from a file called db_list.txt, this file takes 1 database name per line. It then exports the database and compresses, then saves it with a date-stamped filename to a directory called ../daily/<db_name>/.

Then, the script copies a non-date-stamped filename to a directory called ../recent/, thus overwriting the previous file stored there.

Finally it goes through the daily directory structure and deletes any files older than 4 days.

9. Now you can test your backup script by running following command -

./db_backup.sh

10. The script will echo out some progress messages. If no error message, then hopefully your databases should be backed up.

To see if it was successful execution, go to ../daily directory. You will see directories created with <db_name> names. Under these directories, you’ll see compressed version of sql dumps. Similarly, go to ../recent directory, and you’ll see the similar compressed version of sql dump.
To automate this backup process, go to your MediaTemple’s admin section of your primary domain and click the “Cron Jobs” link.

11. Click on the ‘Add a new cron job’ button.

12. Add your email address in the “Notification Email” field. This will send you emails whenever the cron job runs and will let you know if it was successful or not.

13. Also add path of your script in the “Command or script to execute” field. Enter: /home/XXXXX/data/db_backup/scripts/db_backup.sh. Again, replace ‘XXXXX’ with your gridserver account number.

14. On the same page, you can specify how often you’d like your script to run. This entire section is up to you requirements. I’ve set mine to run daily at 3:00 AM when traffic on my site is slower.

15. Click Save and you’re all set to backup your databases on daily basis!

I hope this will be useful to you. If you see any issues, please let me know in the comment section.

Posted in ProgrammingTagged with , , ,

  • niquito
    For some old MT accounts that used to be (ss), the main db user isn't necessarily 'dbxxxx', it could be the original main MySQL account username.
  • I get a "permission denied" error... not sure what permissions to change, any suggestions? SSH is enabled.
  • I second Clint's error.

    That's what I keep getting.
  • Clint
    Thanks for this. I followed directions, but I keep getting this error when I try to test:

    ./db_backup.sh
    : bad interpreter: No such file or directory


    If I try "bash db_backup.sh" I get this error:
    : command not found6:
    'b_backup.sh: line 8: syntax error near unexpected token `do
    'b_backup.sh: line 8: `do


    Any suggestions? I can't wait to get this working. Thanks again.
  • Clint,
    Not sure if you figured it out, but basically what you want to do is make sure that the line-endings are in the Unix-friendly 'LF' format.

    That's what fixed it for me.

    Also, another issue I found with the script that I fixed was the path. It kept giving me a 'file or folder cannot be found' error. All I did was change the XXXXX from my gridserver/account number to my primary domain name. It seems that MediaTemple has both stored - I guess the gridserver number works for some clients, but for me only the primary domain worked.

    Hope that works for you.
  • Marc, thanks! I'll give that a try. Appreciate the tips. Clint
  • thelittlemermaid22
    I have used this script on my mediatemple server and it works great. A many thanks for writing this article. Its really very useful. I have made(and want to make) a few modifications to this script. For eg. I have modified the date variable to timestamp like this - DATE=`date +%Y-%m-%d.%H%M%S`. Now I want that it should delete the last 4 days backup, nut thats not happening. I need to go and manually delete the backup files. Can you please tell me how should I do it?

    Many thanks.
  • Al
    Everything seems to be working except the date stamping, all the databases are output dbname.sql.gz
  • Al
    Please ignore this comment, i figured it out
  • This worked great for me to back up my (mt) databases. Thank you!!
  • Mike Marley
    You might be able to make this a bit sweeter. Wouldn't it be nice if you didn't have to know the names of all your databases. Perhaps changing line 7 to this instead:

    for DATABASE in `mysqlshow -udbXXXXX -pPASSWORD -h internal-db.sXXXXX.gridserver.com | grep '^|'|grep -v Databases | cut -f 2 -d ' '`

    No guarantees but that might work.

    Mike Marley
    Technical Communications
    (mt) Media Temple
blog comments powered by Disqus