In fact, Redis has provided us with a persistence mechanism, namely RDB and AOF two ways, followed by me to see how these two tips are how to ensure the persistence of data.
Because Redis is an in-memory database, our data is not secured when a server fails.
At this time, it is necessary to store the data in memory to the disk, and when our server is restarted, the data can be recovered through the disk, which is called Redis persistence.
The full name of RDB is Redis Database Backup file (Redis data backup file), which can also be called Redis data snapshot.
The RDB file is a compressed binary file (default: dump.rdb);
RDB files are saved on the hard disk;
Record database state by saving key-value pairs in the database.
When Redis is persisted, the program saves the current in-memory database state to disk.
There are two main Redis commands for creating an RDB file: SAVE and BGSAVE.
Synchronous operations, when executing commands, block the Redis server process and reject command requests sent by clients.
Code example:
Icon:
Asynchronous operations, when the command is executed, the child process performs the save work, and the server can continue to let the main thread process the command request sent by the client.
Code example:
Icon:
Onboarding is performed automatically when the server starts.
The server remains blocked while the RDB file is being loaded until the loading work is complete.
Redis allows users to have the server automatically execute the BGSAVE command at regular intervals by setting the save option for the server configuration.
The following configurations are provided:
In this case, the BGSAVE command is executed as long as one of the following conditions is true:
The server has made at least 1 modification to the database within 900 seconds;
The server made at least 10 modifications to the database within 300 seconds.
The server program sets the saveparams property of the server state redisServer structure according to the save conditions set by the save option.
The saveparams property is an array;
Each element in the array is a saveparam structure;
Each saveparam structure holds a save condition set by the save option.
The dirty counter records how many modifications (including writes, deletes, updates, and so on) the server has made to the database state since the last successful execution of the SAVE command or BGSAVE command.
is a UNINX timestamp that records the last successful execution of the SAVE command or BGSAVE command on the server.
The serverPeriodic action function serverCron (which maintains the running server) executes every 100 milliseconds by default, and one of the jobs is to check that the save conditions set by the save option have been met, and if so, execute the BGSAVE command.
Code example:
The default configuration of the RDB file is as follows:
The full name of AOF is Append Only File. The log is a post-write log, and Redis executes a command to write data to memory before logging the log.
Log the database state by saving the write commands executed by the Redis server;
All commands written to the AOF file are saved in the format of the Redis command request protocol.
AOF persistence process implementation is mainly achieved through the following processes:
If AOF persistence is turned on, after the server executes a command, it appends the executed write command in protocol format to the end of the aof_buf buffer in the server state.
Each time the server ends an event loop, it calls the flushAppendOnlyFile function, which considers whether the contents of the aof_buf buffer need to be written to and saved to the AOF file.
The flushAppendOnlyFile function performs the following process:
WRITE: Write the cache in aof_buf to the AOF file according to the conditions;
SAVE: Depending on the condition, call the fsync or fdatasync function to save the AOF file to disk.
This function is influenced by the three values of appendfsync configured by the server: always, everysec, no, also known as the three policies.
Each command fsyncs to the hard disk so that redis write data is not lost.
The buffer is flushed to the hard disk every second (the default).
Deciding when to flush to the hard disk according to the rules of the current operating system does not require us to think about it.
Create a pseudo-client without a network connection;
Parse and read a write command from the AOF file;
Use a pseudo-client to execute a read write command;
Steps 2 and 3 are executed until all write commands in the AOF file have been processed.
Why file rewrite is required:
To solve the problem of volume expansion of AOF files;
Replace the existing AOF file by overriding to create a new AOF file, and the new AOF file does not contain any space-wasting redundant commands.
How file rewriting works:
No operations are required on existing AOF files;
Read the value of the key directly from the database;
Record a key-value pair with a single command, replacing the previous multiple commands that logged the key-value pair.
To not block the parent process, Redis places the AOF override program in the child process for execution.
During an AOF override by a child process, the server process needs to perform three processes:
Execute the command sent by the client;
Append the executed write command to the AOF buffer;
Appends the executed write command to the AOF rewrite buffer.
The default configuration of the AOF file is as follows:
Through the above introduction, I think everyone has a general understanding of Redis persistence, so how do we choose these two ways?
For large and medium-sized applications, we want to ensure both data integrity and efficiency, we should use a combination of RDB and AOF two ways;
If it is only necessary to ensure the integrity of the data and protect the data from loss, then the AOF method is preferred;
If you are dealing with large-scale data recovery and pursuing higher and faster efficiency, the RDB method is preferred.
You can also refer to the following illustration to make a selection: