Server Hot Backup

12006

1. Introduction

In the previous lesson, we learned how to migrate server data. In this lesson, we will introduce how to use a standby server to synchronize server data in real-time, allowing for quick service recovery on the standby server in the event of a failure of the primary server.

2. Preparing the Server

Preparing the Standby Server Environment:

Prepare two identical virtual machines with the same hardware and software configuration, and configure the network environment accordingly.
Initial data synchronization: Generate a migration package on the primary server and restore it on the standby server.
Generate the SSH key required for data synchronization:
- On the primary server, run the Jianghu panel script tool, and sequentially select "2. Generate SSH Key" --> "2. Generate Server Synchronization SSH Key" to create the key standby_sync needed for file synchronization, which will be stored in the /root/.ssh directory.
- After the key is generated, the public key standby_sync.pub will be transferred to the standby server via rsync and added to the authorized_keys in the /root/.ssh directory of the standby server. This way, there will be no need to enter a password during subsequent data synchronization.

3. Daily Backup

Data and files on the primary server need to be synchronized in real-time to the backup server. In this lesson's backup plan, we will synchronize the database backup files and user-uploaded data to the standby server using the rsyncd plugin of the Jianghu panel for recovery during server switching.

The main operations for daily backup include:

Setting Up Database Backups: On the primary server, the following database backups need to be enabled:

Full and incremental backups using the Xtrabackup incremental plugin: Full backup once a day, incremental backup every half hour.
Xtrabackup backup: Once a day.
Database mysqldump backup: Backup all databases every 2 hours.

Configuring Real-Time Data and File Synchronization:

Create real-time synchronization tasks for the following directories in the rsync plugin:
- /www/backup
- /www/wwwstorage
The above directories will be synchronized to the corresponding directories on the backup server.
During configuration, the Rsyncd plugin will automatically select standby_sync as the SSH key.

Updating Server, Website, and Project Settings:

Website and JianghuJS manager configuration: Use the migration package tool for synchronization and recovery.
Project directory: Synchronize the /www/wwwroot directory to the standby server.

Daily Checks: Check the backup and synchronization status in the server report daily to ensure that daily backups and synchronization are functioning properly.

4. Switching Process

When it is necessary to enable the standby server and disable the primary server, the following operations need to be performed:

Check Synchronization Backup Status:

On the primary server, check the lsyncd.status log in the /www/server/rsyncd/logs directory to ensure that the log shows no delays in any synchronization tasks. This indicates that the latest files have been synchronized to the standby server.

Restore Database:

On the standby server, execute the incremental recovery task of the Xtrabackup incremental version to restore the database.
After restoration, obtain the database checksum reports from both the primary and standby servers to ensure there are no significant differences between the two databases.

Restore Website Data and Plugin Data:

If website data or JianghuJS plugin data needs to be updated, use the Jianghu panel script tool to generate a migration data package, selecting only the website data and JianghuJS plugin data, and then restore it on the backup server.
Note: JianghuJS plugin data will overwrite the original data on the backup server, so please proceed with caution.

Switch Server Status:

On both the primary and standby servers, use the Jianghu panel script tool to generate offline and online scripts, confirm the content, and run them to switch the server status.
After switching, modify the DNS resolution records of the website to access the website and services through the standby server.

Set Up Backup Tasks to the Primary Server on the Standby Server:

After switching server status, the standby server becomes the new primary server, and its data and files also need to be backed up.
We need to start a server as the new standby server. Typically, if the primary server is functioning normally, we will use the original primary server as the standby server.
Once the new standby server is prepared, you need to perform the same operations as in the daily backup tasks in Section 3 on the new primary server.

Ensure One-Way Data Synchronization Mechanism:

In daily backups, the public key standby_sync.pub generated on the primary server will be sent and added to the standby server's /root/.ssh/authorized_keys to ensure that daily file synchronization can proceed normally.
When switching server status,
- In the online script of the standby server, the record of standby_sync.pub is removed from authorized_keys, so the Rsyncd synchronization from the primary server can no longer be sent to the standby server.
- In the offline script of the primary server, the record of standby_sync.pub is added to authorized_keys, allowing the standby server to perform Rsyncd file synchronization to the primary server.

## 1. Introduction

In the previous lesson, we learned how to migrate server data. In this lesson, we will introduce how to use a standby server to synchronize server data in real-time, allowing for quick service recovery on the standby server in the event of a failure of the primary server.

## 2. Preparing the Server

**Preparing the Standby Server Environment:**  
- Prepare two identical virtual machines with the same hardware and software configuration, and configure the network environment accordingly.  
- Initial data synchronization: Generate a migration package on the primary server and restore it on the standby server.  
- Generate the SSH key required for data synchronization:  
  - On the primary server, run the Jianghu panel script tool, and sequentially select "2. Generate SSH Key" --> "2. Generate Server Synchronization SSH Key" to create the key `standby_sync` needed for file synchronization, which will be stored in the `/root/.ssh` directory.  
  - After the key is generated, the public key `standby_sync.pub` will be transferred to the standby server via `rsync` and added to the `authorized_keys` in the `/root/.ssh` directory of the standby server. This way, there will be no need to enter a password during subsequent data synchronization.

## 3. Daily Backup

Data and files on the primary server need to be synchronized in real-time to the backup server. In this lesson's backup plan, we will synchronize the database backup files and user-uploaded data to the standby server using the rsyncd plugin of the Jianghu panel for recovery during server switching.

The main operations for daily backup include:

**Setting Up Database Backups:** On the primary server, the following database backups need to be enabled:  
- Full and incremental backups using the Xtrabackup incremental plugin: Full backup once a day, incremental backup every half hour.  
- Xtrabackup backup: Once a day.  
- Database mysqldump backup: Backup all databases every 2 hours.

**Configuring Real-Time Data and File Synchronization:**  
- Create real-time synchronization tasks for the following directories in the rsync plugin:  
  - /www/backup  
  - /www/wwwstorage  
- The above directories will be synchronized to the corresponding directories on the backup server.  
- During configuration, the Rsyncd plugin will automatically select `standby_sync` as the SSH key.

**Updating Server, Website, and Project Settings:**  
- Website and JianghuJS manager configuration: Use the migration package tool for synchronization and recovery.  
- Project directory: Synchronize the /www/wwwroot directory to the standby server.

**Daily Checks:** Check the backup and synchronization status in the server report daily to ensure that daily backups and synchronization are functioning properly.

## 4. Switching Process

When it is necessary to enable the standby server and disable the primary server, the following operations need to be performed:

**Check Synchronization Backup Status:**  
- On the primary server, check the `lsyncd.status` log in the `/www/server/rsyncd/logs` directory to ensure that the log shows no delays in any synchronization tasks. This indicates that the latest files have been synchronized to the standby server.

**Restore Database:**  
- On the standby server, execute the incremental recovery task of the Xtrabackup incremental version to restore the database.  
- After restoration, obtain the database checksum reports from both the primary and standby servers to ensure there are no significant differences between the two databases.

**Restore Website Data and Plugin Data:**  
- If website data or JianghuJS plugin data needs to be updated, use the Jianghu panel script tool to generate a migration data package, selecting only the website data and JianghuJS plugin data, and then restore it on the backup server.  
- **Note:** JianghuJS plugin data will overwrite the original data on the backup server, so please proceed with caution.

**Switch Server Status:**  
- On both the primary and standby servers, use the Jianghu panel script tool to generate offline and online scripts, confirm the content, and run them to switch the server status.  
- After switching, modify the DNS resolution records of the website to access the website and services through the standby server.

**Set Up Backup Tasks to the Primary Server on the Standby Server:**  
- After switching server status, the standby server becomes the new primary server, and its data and files also need to be backed up.  
- We need to start a server as the new standby server. Typically, if the primary server is functioning normally, we will use the original primary server as the standby server.  
- Once the new standby server is prepared, you need to perform the same operations as in the daily backup tasks in Section 3 on the new primary server.

**Ensure One-Way Data Synchronization Mechanism:**  
- In daily backups, the public key `standby_sync.pub` generated on the primary server will be sent and added to the standby server's `/root/.ssh/authorized_keys` to ensure that daily file synchronization can proceed normally.  
- When switching server status,  
  - In the online script of the standby server, the record of `standby_sync.pub` is removed from `authorized_keys`, so the Rsyncd synchronization from the primary server can no longer be sent to the standby server.  
  - In the offline script of the primary server, the record of `standby_sync.pub` is added to `authorized_keys`, allowing the standby server to perform Rsyncd file synchronization to the primary server.