jleony/nanoreth
1
0
Fork 0
mirror of https://github.com/hl-archive-node/nanoreth.git synced 2025-12-06 19:09:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

perf(storage, mdbx): set rp augment limit (#5614)

Browse Source
This commit is contained in:
Alexey Shekhirin
2023-11-29 08:47:47 +00:00
committed by GitHub
parent 9fe3b02c21
commit 07265d9787
1 changed files with 26 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
crates/storage/db/src/implementation/mdbx/mod.rs
View File

@ -99,6 +99,32 @@ impl DatabaseEnv {
}); });
// configure more readers // configure more readers
inner_env.set_max_readers(DEFAULT_MAX_READERS); inner_env.set_max_readers(DEFAULT_MAX_READERS);
// This parameter sets the maximum size of the "reclaimed list", and the unit of measurement
// is "pages". Reclaimed list is the list of freed pages that's populated during the
// lifetime of DB transaction, and through which MDBX searches when it needs to insert new
// record with overflow pages. The flow is roughly the following:
// 0. We need to insert a record that requires N number of overflow pages (in consecutive
// sequence inside the DB file).
// 1. Get some pages from the freelist, put them into the reclaimed list.
// 2. Search through the reclaimed list for the sequence of size N.
// 3. a. If found, return the sequence.
// 3. b. If not found, repeat steps 1-3. If the reclaimed list size is larger than
// the `rp augment limit`, stop the search and allocate new pages at the end of the file:
// https://github.com/paradigmxyz/reth/blob/2a4c78759178f66e30c8976ec5d243b53102fc9a/crates/storage/libmdbx-rs/mdbx-sys/libmdbx/mdbx.c#L11479-L11480.
//
// Basically, this parameter controls for how long do we search through the freelist before
// trying to allocate new pages. Smaller value will make MDBX to fallback to
// allocation faster, higher value will force MDBX to search through the freelist
// longer until the sequence of pages is found.
//
// The default value of this parameter is set depending on the DB size. The bigger the
// database, the larger is `rp augment limit`.
// https://github.com/paradigmxyz/reth/blob/2a4c78759178f66e30c8976ec5d243b53102fc9a/crates/storage/libmdbx-rs/mdbx-sys/libmdbx/mdbx.c#L10018-L10024.
//
// Previously, MDBX set this value as `256 * 1024` constant. Let's fallback to this,
// because we want to prioritize freelist lookup speed over database growth.
// https://github.com/paradigmxyz/reth/blob/fa2b9b685ed9787636d962f4366caf34a9186e66/crates/storage/libmdbx-rs/mdbx-sys/libmdbx/mdbx.c#L16017.
inner_env.set_rp_augment_limit(256 * 1024);
if let Some(log_level) = log_level { if let Some(log_level) = log_level {
// Levels higher than [LogLevel::Notice] require libmdbx built with `MDBX_DEBUG` option. // Levels higher than [LogLevel::Notice] require libmdbx built with `MDBX_DEBUG` option.