diff --git a/book/installation/build-for-arm-devices.md b/book/installation/build-for-arm-devices.md index 0f86d95ab..d07d4cb49 100644 --- a/book/installation/build-for-arm-devices.md +++ b/book/installation/build-for-arm-devices.md @@ -62,12 +62,12 @@ A simple and safe approach to achieve this is to use the Armbian build framework To be able to build an Armbian image and set those values, you will need to: - Clone the Armbian build framework repository -```shell +```bash git clone https://github.com/armbian/build cd build ``` - Run the compile script with the following parameters: -```shell +```bash ./compile.sh \ BUILD_MINIMAL=yes \ BUILD_DESKTOP=no \ diff --git a/book/run/troubleshooting.md b/book/run/troubleshooting.md index e810c9b55..41dcdcae3 100644 --- a/book/run/troubleshooting.md +++ b/book/run/troubleshooting.md @@ -6,6 +6,64 @@ This page tries to answer how to deal with the most popular issues. ## Database +### Slow database inserts and updates + +If you're: +1. Running behind the tip +2. Have slow canonical commit time according to the `Canonical Commit Latency time` chart on [Grafana dashboard](./observability.md#prometheus--grafana) (more than 2-3 seconds) +3. Seeing warnings in your logs such as + ```console + 2023-11-08T15:17:24.789731Z WARN providers::db: Transaction insertion took too long block_number=18528075 tx_num=2150227643 hash=0xb7de1d6620efbdd3aa8547c47a0ff09a7fd3e48ba3fd2c53ce94c6683ed66e7c elapsed=6.793759034s + ``` + +then most likely you're experiencing issues with the [database freelist](https://github.com/paradigmxyz/reth/issues/5228). +To confirm it, check if the values on the `Freelist` chart on [Grafana dashboard](./observability.md#prometheus--grafana) +is greater than 10M. + +Currently, there are two main ways to fix this issue. + + +#### Compact the database +It will take around 5-6 hours and require **additional** disk space located on the same or different drive +equal to the [freshly synced node](../installation/installation.md#hardware-requirements). + +1. Clone Reth + ```bash + git clone https://github.com/paradigmxyz/reth + cd reth + ``` +2. Build database debug tools + ```bash + make db-tools + ``` +3. Run compaction (this step will take 5-6 hours, depending on the I/O speed) + ```bash + ./db-tools/mdbx_copy -c $(reth db path) reth_compact.dat + ``` +4. Stop Reth +5. Backup original database + ```bash + mv $(reth db path)/mdbx.dat reth_old.dat + ``` +6. Move compacted database in place of the original database + ```bash + mv reth_compact.dat $(reth db path)/mdbx.dat + ``` +7. Start Reth +8. Confirm that the values on the `Freelist` chart is near zero and the values on the `Canonical Commit Latency time` chart +is less than 1 second. +9. Delete original database + ```bash + rm reth_old.dat + ``` + +#### Re-sync from scratch +It will take the same time as initial sync. + +1. Stop Reth +2. Drop the database using [`reth db drop`](../cli/db.md#reth-db-drop) +3. Start reth + ### Database write error If you encounter an irrecoverable database-related errors, in most of the cases it's related to the RAM/NVMe/SSD you use. For example: @@ -33,6 +91,6 @@ Caused by: git clone https://github.com/paradigmxyz/reth cd reth make db-tools - db-tools/mdbx_chk $(reth db path)/mdbx.dat | tee mdbx_chk.log + ./db-tools/mdbx_chk $(reth db path)/mdbx.dat | tee mdbx_chk.log ``` If `mdbx_chk` has detected any errors, please [open an issue](https://github.com/paradigmxyz/reth/issues) and post the output from the `mdbx_chk.log` file. \ No newline at end of file