jleony/nanoreth
1
0
Fork 0
mirror of https://github.com/hl-archive-node/nanoreth.git synced 2025-12-06 10:59:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: rework book (#2566)

Browse Source
This commit is contained in:
Georgios Konstantopoulos
2023-05-04 20:28:33 -07:00
committed by GitHub
parent 70bcd76032
commit 52f2bc5ebc
32 changed files with 537 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
book/README.md
View File

@ -1,20 +0,0 @@
# reth Book
<img src="https://raw.githubusercontent.com/paradigmxyz/reth/main/assets/reth.jpg" style="border-radius: 20px">
<!-- Add a quick description about Reth, what it is, the goals of the build, and any other quick overview information -->
The book is continuously rendered [here](https://paradigmxyz.github.io/reth/)!
> 📖 **Contributing**
>
> You can contribute to this book on [GitHub][gh-book].
### Sections
**[Getting Started]()**
To get started with Reth, install, configure and sync your node.
* To install and build reth, you can use the following [installation instruction](./getting_started/installation.md).
[gh-book]: https://github.com/paradigmxyz/reth/tree/main/book

40
book/SUMMARY.md
View File

@ -1,17 +1,27 @@
# Reth Book
## Getting Started
- [Introduction](./README.md)
- [System requirements](./getting_started/system_requirements.md)
- [Installation](./getting_started/installation.md)
- [Running Reth](./getting_started/running.md)
## Fundamentals
- [Node](./fundamentals/node.md)
- [Command-Line Options](./fundamentals/command-line_options.md)
- [Logs](./fundamentals/logs.md)
## Monitoring
- [Metrics](./monitoring/metrics.md)
## [FAQ](./faq/README.md)
1. [Introduction](./intro.md)
1. [Installation](./installation/installation.md)
1. [Pre-Built Binaries](./installation/binaries.md)
1. [Docker](./installation/docker.md)
1. [Build from Source](./installation/source.md)
1. [Run a Node](./run/run-a-node.md)
1. [Mainnet or official Testnets](./run/mainnet.md)
1. [Local Testnet](./run/local_testnet.md)
1. [Observability with Prometheus & Grafana](./run/observability.md)
1. [Configuring reth.toml](./run/config.md)
1. [Interacting with Reth over JSON-RPC](./jsonrpc/intro.md)
1. [admin](./jsonrpc/admin.md)
1. [eth](./jsonrpc/eth.md)
1. [engine](./jsonrpc/engine.md)
1. [tracing](./jsonrpc/tracing.md)
1. [CLI Reference](./cli/cli.md)
1. [reth node](./cli/node.md) -->
1. [reth db](./cli/db.md)
1. [reth p2p](./cli/p2p.md)
1. [reth stage](./cli/stage.md)
1. [reth dump-stage](./cli/dump-stage.md)
1. [reth drop-stage](./cli/drop-stage.md)
1. [Developers](./developers/developers.md)
1. [Contribute](./developers/contribute.md)
1. [Architecture](./developers/architecture.md)

1
book/cli/cli.md Normal file
View File

@ -0,0 +1 @@
# CLI Reference

1
book/cli/db.md Normal file
View File

@ -0,0 +1 @@
# reth db

1
book/cli/drop-stage.md Normal file
View File

@ -0,0 +1 @@
# reth drop-stage

1
book/cli/dump-stage.md Normal file
View File

@ -0,0 +1 @@
# reth dump-stage

1
book/cli/node.md Normal file
View File

@ -0,0 +1 @@
# reth node

1
book/cli/p2p.md Normal file
View File

@ -0,0 +1 @@
# reth p2p

1
book/cli/stage.md Normal file
View File

@ -0,0 +1 @@
# reth stage

1
book/developers/architecture.md Normal file
View File

@ -0,0 +1 @@
# Architecture

3
book/developers/contribute.md Normal file
View File

@ -0,0 +1,3 @@
# Contribute
<!-- TODO: Add various debugging tips and tricks we use, ways to configure logging, unwinding a node etc. -->

1
book/developers/developers.md Normal file
View File

@ -0,0 +1 @@
# Developers

30
book/getting_started/installation.md
View File

@ -1,30 +0,0 @@
## Installation
reth is currently unstable and does not have officially cut releases. To install reth, you must either install it from source or using Docker.
### From Source
To build from source, first install [Rust](https://rustup.rs). You also need some basic build tools for our C libraries:
- For Ubuntu: `apt-get install libclang-dev pkg-config`
- For Arch: `pacman -S base-devel`
- For macOS: `brew install llvm pkg-config`
Then clone the repository and build the binary:
```console
git clone https://github.com/paradigmxyz/reth
cd reth
cargo install --locked --path bin/reth --bin reth
```
The binary will now be in a platform specific folder, and should be accessible as `reth` via the command line.
### Using Docker
Clone the repository and build the image:
```console
git clone https://github.com/paradigmxyz/reth
docker build -t paradigmxyz/reth .
```

7
book/getting_started/introduction.md
View File

@ -1,7 +0,0 @@
# Introduction
Reth is a user-friendly, modular, fast, and efficient Ethereum full node implementation that is compatible with the [Engine API](https://github.com/ethereum/execution-apis/tree/main/src/engine). It allows users to connect to the Ethereum network and interact with the blockchain, including sending and receiving transactions and accessing smart contracts. Reth is built and maintained by [Paradigm](https://www.paradigm.xyz/) and is licensed under the Apache and MIT licenses.
## Status
The project is not ready for use. We hope to have full sync implemented sometime in January/February 2023, followed by optimizations. In the meantime, we're working on making sure every crate of the repository is well documented, abstracted and tested.

145
book/getting_started/running.md
View File

@ -1,145 +0,0 @@
# Running Reth
This chapter will provide a few runbooks for getting the node up and running.
This goal of this is not to expose all of the options available to you for running Reth (e.g. config file options, CLI flags, etc.) - those will be documented in the other chapters - but rather showcase some examples of different ways in which one could run the node.
## Run on w/ Prometheus + Grafana for Metrics
This will provide a brief overview for how to run Reth locally.
First, ensure that you have Reth installed by following the [installation instructions][installation].
### Basic operation
The most basic way to run it is by using the following command:
```console
reth node
```
You will likely see a large number of error logs to the tune of:
```bash
ERROR Disconnected by peer during handshake: Too many peers
```
These simply indicate that the peer you discovered is already connected to too many other peers and cannot accept a new connection with you. Nothing to worry about here, this is part of the peering process.
Believe it or not, your node is already running and attempting to sync.
When playing around with sync, you may want to set a threshold block that you'd like to sync up to instead of going for a full sync from the jump.
To do so, add the `--debug.tip` flag with the block hash that you'd like to sync to:
```bash
reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e
```
This will sync to block 1M.
That's great and all, but wouldn't it be nice to get a deeper view into what's going on?
### Adding more logs to the mix
Let's try again, now with the following command:
```bash
RUST_LOG=info,sync::stages=trace,downloaders=trace reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e
```
This will dump info-level logs throughout the node's operation, as well as trace-level logs for the pipeline stages and the downloaders (which fetch data over the P2P network). Check out the [docs][docs] for more info on these!
Now, you'll probably see a _lot_ of warning logs stemming from attempting to connect to peers, including but not limited to:
- `error=Some(Eth(P2PStreamError(HandshakeError(NoResponse))))`
- This means that the peer you're attempting to do a devp2p handshake with didn't send you any data
- `error=Some(Eth(EthHandshakeError(MismatchedGenesis { expected: 0x..., got: 0x... })))`
- This means you have a different genesis block from the peer, which is likely because they are a node in a different devp2p network (e.g. a testnet, or Polygon, etc.)
- `error=Some(Eth(P2PStreamError(HandshakeError(NoSharedCapabilities))))`
- This means the peer supports a devp2p subprotocol your node does not, e.g. they might be a light client (which Reth doesn't yet support).
- `error=Some(Eth(P2PStreamError(Disconnected(ClientQuitting))))`
- This means the peer you're trying to connect to is in the process of shutting down.
These warnings are also nothing to worry about, all of this is part of the normal peering process. But, this gives you a view into how difficult finding good peers is!
You may want to keep these logs around outside of your terminal. To accomplish this, let's run:
```bash
RUST_LOG=info,sync::stages=trace,downloaders=trace reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e --log.directory ./
```
Here, adding `--log.directory` specifies a location to which the logs will be saved (in a file named `reth.log`) so that you can view them with a tool like `more`, `less`, or `tail`.
Now, trying to get a sense of sync progress by scanning through the logs is quite painful. Let's start consuming some metrics.
### Exporting metrics
Reth exposes a number of metrics, which are listed [here][metrics]. We can serve them from an HTTP endpoint by adding the `--metrics` flag:
```bash
RUST_LOG=info,sync::stages=trace,downloaders=trace nohup reth node --debug.tip 0x8e38b4dbf6b11fcc3b9dee84fb7986e29ca0a02cecd8977c161ff7333329681e --metrics '127.0.0.1:9000' > reth-out.txt &
```
Now, as the node is running, you can `curl` the endpoint you provided to the `--metrics` flag to get a text dump of the metrics at that time:
```bash
curl 127.0.0.1:9000
```
The response from this is quite descriptive, but it can be a bit verbose. Plus, it's just a snapshot of the metrics at the time that you `curl`ed the endpoint.
You can run the following command in a separate terminal to periodically poll the endpoint, and just print the values (without the header text) to the terminal:
```bash
while true; do date; curl -s localhost:9000 | grep -Ev '^(#|$)' | sort; echo; sleep 10; done
```
We're finally getting somewhere! As a final step, though, wouldn't it be great to see how these metrics progress over time (and generally, in a GUI)?
### Consuming metrics with Prometheus & Grafana
We're going to be using Prometheus to collect metrics off of the endpoint we set up, and use Grafana to scrape the metrics from Prometheus and define a dashboard with them.
Let's begin by installing both Prometheus and Grafana, which one can do with e.g. Homebrew:
```bash
brew update
brew install prometheus
brew install grafana
```
Then, kick off the Prometheus and Grafana services:
```bash
brew services start prometheus
brew services start grafana
```
After this, let's run Prometheus to start collecting metrics. In the root of the `reth` repo, run:
```bash
prometheus --config.file etc/prometheus/prometheus.yml
```
This uses the example Prometheus configuration file in the repo. Most importantly, be sure that the URL in the `targets` field of this config file matches the endpoint that you supplied to the `--metrics` flag.
Next, open up "localhost:3000" in your browser, which is the default URL for Grafana. Here, "admin" is the default for both the username and password.
Once you've logged in, click on the gear icon in the lower left, and select "Data Sources". Click on "Add data source", and select "Prometheus" as the type. In the HTTP URL field, enter "http://localhost:9090", this is the default endpoint for the Prometheus scrape endpoint. Finally, click "Save & Test".
As this might be a point of confusion, "localhost:9000", which we supplied to `--metrics`, is the endpoint that Reth exposes, from which Prometheus collects metrics. Prometheus then exposes "localhost:9090" (by default) for other services (such as Grafana) to consume Prometheus metrics.
To configure the dashboard in Grafana, click on the squares icon in the upper left, and click on "New", then "Import". From there, click on "Upload JSON file", and select the example file in `reth/etc/grafana/overview.json`. Finally, select the Prometheus data source you just created, and click "Import".
And voilá, you should see your dashboard! If you're not yet connected to any peers, the dashboard will look like it's in an empty state, but once you are, you should see it start populating with data.
### Conclusion
In this runbook, we took you through starting the node, exposing different log levels, exporting metrics, and finally viewing those metrics in a Grafana dashboard.
This will all be very useful to you, whether you're simply running a home node and want to keep an eye on its performance, or if you're a contributor and want to see the effect that your (or others') changes have on Reth's operations.
[installation]: ./installation.md
[release-profile]: https://doc.rust-lang.org/cargo/reference/profiles.html#release
[docs]: https://github.com/paradigmxyz/reth/tree/main/docs
[metrics]: https://github.com/paradigmxyz/reth/blob/main/docs/design/metrics.md#current-metrics

19
book/getting_started/system_requirements.md
View File

@ -1,19 +0,0 @@
# System Requirements
**Disclaimer**: The information provided is only a rough estimate based on current node implementations, and may not be representative of all node implementations. We plan to conduct benchmarking and optimization tests to ensure the best performance for each specific audience.
## Hardware
- 64-bit architecture.
- We recommend a minimum of 8GB RAM
- We recommend using an SSD or NVMe, as an HDD may cause the node to fall behind the chain tip. Note that SSD performance can deteriorate when close to capacity.
### Archive node
- TODO: We will update the disk usage estimate once it is ready.
### Full node
- TODO: We will update the disk usage estimate once it is ready.
## Software (for building)
- rustc 1.66.0 (69f9c33d7 2022-12-12)
- libclang-dev
- pkg-config

57
book/installation/binaries.md Normal file
View File

@ -0,0 +1,57 @@
# Binaries
Precompiled binaries are available from the [GitHub releases page](https://github.com/paradigmxyz/reth/releases).
These are better managed by using [rethup](#using-rethup).
## Rethup
rethup is the Reth installer. It is a wrapper around the GitHub releases page, and allows you to install Reth from a specific branch, commit, or pull request from your terminal, easily.
Open your terminal and run the following command:
```sh
curl -L https://reth.paradigm.xyz | bash
```
This will install rethup, then simply follow the instructions on-screen,
which will make the `rethup` command available in your CLI.
Running `rethup` by itself will install the latest (nightly) precompiled binary for `reth`.
See `rethup --help` for more options, like installing from a specific version or commit.
> ℹ️ **Note**
>
> If you're on Windows, you will need to install and use [Git BASH](https://gitforwindows.org/) or [WSL](https://learn.microsoft.com/en-us/windows/wsl/install),
> as your terminal, since rethup currently does not support Powershell or Cmd.
You can use the different rethup flags to install reth from a specific branch, pull request, or path.
```sh
rethup --branch master
rethup --path path/to/reth
rethup --pr 1234
```
## From Github Releases
Alternatively, you can download the binaries from the [GitHub releases page](https://github.com/paradigmxyz/reth/releases).
Binaries are supplied for four platforms:
- `x86_64-unknown-linux-gnu`: AMD/Intel 64-bit processors (most desktops, laptops, servers)
- `x86_64-apple-darwin`: macOS with Intel chips
- `aarch64-unknown-linux-gnu`: 64-bit ARM processors (Raspberry Pi 4)
- `x86_64-windows`: Windows with 64-bit processors
Each binary is contained in a `.tar.gz` archive. For this example, lets assume the user needs
a `x86_64` binary:
1. Go to the [Releases](https://github.com/paradigmxyz/reth/releases) page and
select the latest release.
1. Download the `reth-${VERSION}-x86_64-unknown-linux-gnu.tar.gz` binary. For example, to obtain the binary file for v0.0.1 (the latest version at the time of writing), a user can run the following commands in a linux terminal:
```bash
cd ~
curl -LO https://github.com/paradigmxyz/reth/releases/download/v0.0.1-alpha/reth-v0.0.1-alpha-x86_64-unknown-linux-gnu.tar.gz
tar -xvf reth-v0.0.1-alpha-x86_64-unknown-linux-gnu.tar.gz
```
1. Test the binary with `./reth --version` (it should print the version).
1. (Optional) Move the `reth` binary to a location in your `PATH`, so the `reth` command can be called from anywhere. For example, to copy `reth` from the current directory to `usr/bin`, run `sudo cp reth /usr/bin`.

47
book/installation/docker.md Normal file
View File

@ -0,0 +1,47 @@
# Docker
There are two ways to obtain a Reth Docker image:
1. [Docker Hub](#docker-hub), or
2. By [building a Docker image from source](#building-the-docker-image).
Once you have obtained the docker image via one of these methods, proceed to [Using the Docker
image](#using-the-docker-image).
## Docker Hub
Reth maintains the [paradigmxyz/reth][docker_hub] Docker Hub repository which provides an easy
way to run Reth without building the image yourself.
Obtain the latest image with:
```bash
docker pull paradigmxyz/reth
```
Download and test the image with:
```bash
docker run paradigmxyz/reth reth --version
```
If you can see the latest [Reth release](https://github.com/paradigmxyz/reth/releases) version
(see example below), then you've successfully installed Reth via Docker.
## Building the Docker Image
To build the image from source, navigate to
the root of the repository and run:
```bash
docker build . -t reth:local
```
The build will likely take several minutes. Once it's built, test it with:
```bash
docker run reth:local reth --help
```
[docker_hub]: https://hub.docker.com/repository/docker/paradigmxyz/reth/

41
book/installation/installation.md Normal file
View File

@ -0,0 +1,41 @@
# Installation
Reth runs on Linux and macOS (Windows tracked).
There are three core methods to obtain Reth:
* [Pre-built binaries](./binaries.md).
* [Docker images](./docker.md).
* [Building from source.](./source.md).
## Hardware Requirements
The hardware requirements for running Reth depend on the node configuration and can change over time as the network grows or new features are implemented. The most important requirement is by far the disk, whereas CPU and RAM requirements are relatively flexible.
### Disk
There are multiple types of disks to sync Reth, with varying size requirements, depending on the [syncing mode](../run/sync-modes.md):
* Archive Node: At least 2.5TB is required to store
* Full Node: TBD
The time to sync also varies depending on the node type. NVMe drives are recommended for the best performance, however they can get expensive. SSDs are a good alternative, and HDDs are the cheapest option, but they will take the longest to sync.
### CPU
Most of the time in Ethereum is spent executing transactions, which is a single-threaded operation due to potential state dependencies of a transaction on previous ones. As a result, the number of cores matters less, but in general higher clock speeds are better. More cores are better for parallelizable [stages, like Senders Recovery, or Bodies](../developers/architecture.md), but these stages are not the bottleneck for syncing.
### Memory
It is recommended to use at least 8GB of RAM.
Most of Reth's components tend to consume a low amount of memory, unless you are under heavy RPC load, so this should matter less than the other requirements.
Higher memory is generally better as it allows for better caching and less stress on the disk.
### Bandwidth
A stable and dependable internet connection is crucial for both syncing a node from genesis and for keeping up with the chain's tip.
Note that due to Reth's staged sync, you only need an internet connection for the Headers and Bodies stages. This means that the first 1-3 hours (depending on your internet connection) will be online, downloading all necessary data, and the rest will be done offline and does not require an internet connection.
Once you're synced to the tip you will need a reliable connection, especially if you're operating a validator. A 24MBps connection is recommended, but you can probably get away with less. Make sure your ISP does not cap your bandwidth.

121
book/installation/source.md Normal file
View File

@ -0,0 +1,121 @@
# Build from Source
You can build Reth on Linux, macOS, and Windows WSL.
# Dependencies
First, **install Rust** using [rustup](https://rustup.rs/)：
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
The rustup installer provides an easy way to update the Rust compiler, and works on all platforms.
> Tips:
>
> - During installation, when prompted, enter `1` for the default installation.
> - After Rust installation completes, try running `cargo version` . If it cannot
> be found, run `source $HOME/.cargo/env`. After that, running `cargo version` should return the version, for example `cargo 1.68.2`.
> - It's generally advisable to append `source $HOME/.cargo/env` to `~/.bashrc`.
With Rust installed, follow the instructions below to install dependencies relevant to your
operating system:
- **Ubuntu**: `apt-get install libclang-dev pkg-config build-essential`
- **macOS**: `brew install llvm pkg-config`
# Build Reth
With Rust and the dependencies installed, you're ready to build Reth. First, clone the repository:
```bash
git clone https://github.com/paradigmxyz/reth
cd reth
```
Then, install Reth into your path directly via:
```bash
cargo install --locked --path bin/reth --bin reth
```
The binary will now be accessible as `reth` via the command line, and exist under your default `.cargo/bin` folder.
Alternatively, you can build yourself with:
```bash
cargo build --release
```
This will place the reth binary under `./target/release/reth`, and you can copy it to your directory of preference after that.
You can also build Reth with the following rustflags for utilizing faster CPU instructions available on your machine:
```bash
RUSTFLAGS="-C target-cpu=native" cargo build --release
```
Compilation may take around 10 minutes. Installation was successful if `reth --help` displays
the [command-line documentation](../cli/cli.md).
If you run into any issues, please check the [Troubleshooting](#troubleshooting) section.
out to us on [Telegram](https://t.me/paradigm_reth).
## Update Reth
You can update Reth to a specific version by running the commands below. The `reth`
directory will be the location you cloned reth to during the installation process.
`${VERSION}` will be the version you wish to build in the format `vX.X.X`.
```bash
cd reth
git fetch
git checkout ${VERSION}
cargo build --release
```
## Compilation Profiles
You can customise the compiler settings used to compile Reth via
[Cargo profiles](https://doc.rust-lang.org/cargo/reference/profiles.html).
Reth includes several profiles which can be selected via the `--profile` cargo parameter.
* `release`: default for source builds, enables most optimisations while not taking too long to
compile.
* `maxperf`: default for binary releases, enables aggressive optimisations including full LTO.
Although compiling with this profile improves some benchmarks by around 20% compared to `release`,
it imposes a _significant_ cost at compile time and is only recommended if you have a fast CPU.
You can also use `RUSTFLAGS="-C target-cpu=native"` to enable CPU-specific optimisations. In order to get
the highest performance out of your build:
```bash
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf
```
## Troubleshooting
### Command is not found
Reth will be installed to `CARGO_HOME` or `$HOME/.cargo`. This directory
needs to be on your `PATH` before you can run `$ reth`.
See ["Configuring the `PATH` environment variable"](https://www.rust-lang.org/tools/install) for more information.
### Compilation error
Make sure you are running the latest version of Rust. If you have installed Rust using rustup, simply run `rustup update`.
If you can't install the latest version of Rust you can instead compile using the Minimum Supported
Rust Version (MSRV) which is listed under the `rust-version` key in Reth's
[Cargo.toml](https://github.com/paradigmxyz/reth/blob/main/Cargo.toml).
If compilation fails with `(signal: 9, SIGKILL: kill)`, this could mean your machine ran out of
memory during compilation. If you are on Docker, consider increasing the memory of the container, or use a [pre-built
binary](../binaries.md).
If compilation fails with `error: linking with cc failed: exit code: 1`, try running `cargo clean`.
_(Thanks to Sigma Prime for this section from [their Lighthouse book](https://lighthouse-book.sigmaprime.io/installation.html)!)_

60
book/intro.md Normal file
View File

@ -0,0 +1,60 @@
# Reth Book
_Documentation for Reth users and developers._ [![Telegram Chat][tg-badge]][tg-url]
Reth (short for Rust Ethereum, [pronunciation](https://twitter.com/kelvinfichter/status/1597653609411268608)) is an **Ethereum full node implementation that is focused on being user-friendly, highly modular, as well as being fast and efficient.**
<img src="https://raw.githubusercontent.com/paradigmxyz/reth/main/assets/reth.jpg" style="border-radius: 20px">
<!-- Add a quick description about Reth, what it is, the goals of the build, and any other quick overview information -->
# What is this about?
[Reth](https://github.com/paradigmxyz/reth/) is an Execution Layer (EL) and is compatible with all Ethereum Consensus Layer (CL) implementations that support the [Engine API](https://github.com/ethereum/execution-apis/tree/59e3a719021f48c1ef5653840e3ea5750e6af693/src/engine). It is originally built and driven forward by [Paradigm](https://paradigm.xyz/), and is licensed under the Apache and MIT licenses.
As a full Ethereum node, Reth allows users to connect to the Ethereum network and interact with the Ethereum blockchain. This includes sending and receiving transactions/logs/traces, as well as accessing and interacting with smart contracts. Building a successful Ethereum node requires creating a high-quality implementation that is both secure and efficient, as well as being easy to use on consumer hardware. It also requires building a strong community of contributors who can help support and improve the software.
# What are the goals of Reth?
1. **Modularity**: Every component of Reth is built to be used as a library: well-tested, heavily documented and benchmarked. We envision that developers will import the node's crates, mix and match, and innovate on top of them. Examples of such usage include but are not limited to spinning up standalone P2P networks, talking directly to a node's database, or "unbundling" the node into the components you need. To achieve that, we are licensing Reth under the Apache/MIT permissive license.
2. **Performance**: Reth aims to be fast, so we used Rust and the [Erigon staged-sync](https://erigon.substack.com/p/erigon-stage-sync-and-control-flows) node architecture. We also use our Ethereum libraries (including [ethers-rs](https://github.com/gakonst/ethers-rs/) and [revm](https://github.com/bluealloy/revm/)) which we’ve battle-tested and optimized via [Foundry](https://github.com/foundry-rs/foundry/).
3. **Free for anyone to use any way they want**: Reth is free open source software, built for the community, by the community. By licensing the software under the Apache/MIT license, we want developers to use it without being bound by business licenses, or having to think about the implications of GPL-like licenses.
4. **Client Diversity**: The Ethereum protocol becomes more antifragile when no node implementation dominates. This ensures that if there's a software bug, the network does not finalize a bad block. By building a new client, we hope to contribute to Ethereum's antifragility.
5. **Used by a wide demographic**: We want to solve for node operators that care about fast historical queries, but also for hobbyists who cannot operate on large hardware. We also want to support teams and individuals who want both sync from genesis and via "fast sync". We envision that Reth will be configurable enough for the tradeoffs that each team faces.
# Who is this for?
Reth is a new Ethereum full node that allows users to sync and interact with the entire blockchain, including its historical state if in archive mode.
- Full node: It can be used as a full node, which stores and processes the entire blockchain, validates blocks and transactions, and participates in the consensus process.
- Archive node: It can also be used as an archive node, which stores the entire history of the blockchain and is useful for applications that need access to historical data.
As a data engineer/analyst, or as a data indexer, you'll want to use Archive mode. For all other use cases where historical access is not needed, you can use Full mode.
# Is this secure?
Reth implements the specification of Ethereum as defined in the [ethereum/execution-specs](https://github.com/ethereum/execution-specs/) repository. To make sure the node is built securelty, we run the following tests:
1. EVM state tests are run on every [Revm Pull Request](https://github.com/bluealloy/revm/blob/main/.github/workflows/ethereum-tests.yml)
2. Hive tests are [run every 24 hours](https://github.com/paradigmxyz/reth/blob/main/.github/workflows/hive.yml) in the main Reth repository.
3. We regularly re-sync multiple nodes from scratch.
4. We operate multiple nodes at the tip of Ethereum mainnet and various testnets.
5. We extensively unit test, fuzz test and document all our code, while also restricting PRs with aggressive lint rules.
We intend to also audit / fuzz the EVM & parts of the codebase. Please reach out if you're interested in collaborating on securing this codebase.
# Sections
Here are some useful sections to jump to:
- Install Reth by following the [guide](./installation/installation.md).
- Sync your node on any [official network](./run/run_a_node.md), or [spin up your own testnet](./run/local_testnet.md)!
- View [statistics and metrics](./run/observability.md) about your node
- Query the [JSON-RPC](./api/api.md) using Foundry's `cast` or `curl`.
- Set up your [development environment and contribute](./contribute.md)!
> 📖 **About this book**
>
> The book is continuously rendered [here](https://paradigmxyz.github.io/reth/)!
> You can contribute to this book on [GitHub][gh-book].
[tg-badge]: https://img.shields.io/endpoint?color=neon&logo=telegram&label=chat&url=https%3A%2F%2Ftg.sumanjay.workers.dev%2Fparadigm%5Freth
[tg-url]: https://t.me/paradigm_reth
[gh-book]: https://github.com/paradigmxyz/reth/tree/main/book

1
book/jsonrpc/admin.md Normal file
View File

@ -0,0 +1 @@
# admin

1
book/jsonrpc/engine.md Normal file
View File

@ -0,0 +1 @@
# Configuring the Engine API

1
book/jsonrpc/eth.md Normal file
View File

@ -0,0 +1 @@
# eth

25
book/jsonrpc/intro.md Normal file
View File

@ -0,0 +1,25 @@
# JSON-RPC Namespaces
Reth supports all standard Ethereum JSON-RPC API methods. The API methods are grouped into namespaces, which are listed below:
* [admin](./admin.md): Administrative APIs for the node. DO NOT expose these APIs to the public internet.
* [eth](./eth.md): Ethereum APIs for interacting with the Ethereum blockchain.
* [tracing](./tracing.md): APIs for tracing the execution of transactions, compatible with the popular [Parity Tracing module](https://openethereum.github.io/JSONRPC-trace-module).
* [debug](./debug.md): APIs for debugging the node, originally created by Geth.
<!-- TODO: add missing ones -->
One can easily interact with these APIs just like they would with any Ethereum client. You can use curl, a programming language with a low-level library, or a tool like Foundry to interact with the chain at the exposed HTTP or WS port. As a reminder, you need to run the command below to enable all of these apis:
```bash
RUST_LOG=info reth node --http --http.api "eth,admin,debug,tracing"
```
> The IPC transport is also supported with `--ipc`!
This allows you to then call:
```bash
cast block-number
cast rpc admin_nodeInfo
cast rpc debug_traceTransaction
cast rpc trace_replayBlockTransactions
```

1
book/jsonrpc/tracing.md Normal file
View File

@ -0,0 +1 @@
# tracing

15
book/monitoring/metrics.md
View File

@ -1,15 +0,0 @@
# Metrics
Reth integrated with Prometheus allows users to measure various metrics to gain insight into the status and performance of a Reth node. For example, users can track the network status, transaction pool status, and other metrics to understand how the node is functioning.
## Stage Header
- **headers_counter**: Number of headers successfully retrieved
- **timeout_errors**: Number of timeout errors while requesting headers
- **validation_errors**: Number of validation errors while requesting headers
- **unexpected_errors**: Number of unexpected errors while requesting headers
## Transaction pool
- **inserted_transactions**: Number of transactions inserted in the pool
- **invalid_transactions**: Number of invalid transactions
- **removed_transactions**: Number of removed transactions from the pool

2
book/run/config.md Normal file
View File

@ -0,0 +1,2 @@
# Configuring Reth.toml
<!-- TODO: Explain how to configure reth.toml -->

3
book/run/local_testnet.md Normal file
View File

@ -0,0 +1,3 @@
# Local Testnet
<!-- TODO: Explain how to run 2 CL/EL stacks and connect them together via admin_addPeer etc. -->

62
book/run/mainnet.md Normal file
View File

@ -0,0 +1,62 @@
# Running Reth on Ethereum Mainnet or testnets
Reth is an [_execution client_](https://ethereum.org/en/developers/docs/nodes-and-clients/#execution-clients). After Ethereum's transition to Proof of Stake (aka the Merge) it became required to run a [_consensus client_](https://ethereum.org/en/developers/docs/nodes-and-clients/#consensus-clients) along your execution client in order to sync into any "post-Merge" network. This is because the Ethereum execution layer now outsources consensus to a separate component, known as the consensus client.
There are multiple choices of consensus clients, but in this guide we will assume usage of Lighthouse 🦀.
## Running the Reth Node
First, ensure that you have Reth installed by following the [installation instructions][installation].
Now, start the node as follows:
```bash
RUST_LOG=info reth node
```
> Note that this command will not open any HTTP/WS ports by default. You can change this by adding the `--http`, `--ws` flags, respectively and using the `--http.api` and `--ws.api` flags to enable various [JSON-RPC APIs](../api/api.md). For more commands, see the [`reth node` CLI reference](../cli/node.md).
The EL <> CL communication happens over the [Engine API](https://github.com/ethereum/execution-apis/blob/main/src/engine/common.md), which is by default exposed at `http://localhost:8551`. The connection is authenticated over JWT using a JWT secret which is auto-generated by Reth under `~/.local/share/reth/mainnet/jwt.hex` in Linux (`/Users/<NAME>/Library/Application Support/reth/mainnet/jwt.hex` in Mac).
You can override this path using the `--authrpc.jwtsecret` option. You MUST use the same JWT secret in BOTH Reth and the chosen Consensus Layer. If you want to override the address or port, you can use the `--authrpc.addr` and `--authrpc.port` options, respectively.
So one might do:
```bash
RUST_LOG=info reth node \
--authrpc.jwtsecret /path/to/secret \
--authrpc.addr 127.0.0.1 \
--authrpc.port 9999
```
At this point, our Reth node has started discovery, and even discovered some new peers. But it will not start syncing until you spin up the consensus layer!
## Running the Consensus Layer
First, make sure you have Lighthouse installed. Sigma Prime provides excellent [installation](https://lighthouse-book.sigmaprime.io/installation.html) and [node operation](https://lighthouse-book.sigmaprime.io/run_a_node.html) instructions.
Assuming you have done that, run:
```bash
RUST_LOG=info lighthouse bn \
--checkpoint-sync-url https://mainnet.checkpoint.sigp.io \
--execution-endpoint http://localhost:8551 \
--execution-jwt ~/.local/share/reth/jwtsecret/jwt.hex
```
Your Reth node should start receiving "fork choice updated" messages, and begin syncing the chain.
## Verify the chain is growing
You can easily verify that by inspecting the logs, and seeing that headers are arriving in Reth. Sit back now and wait for the stages to run!
In the meantime, consider setting up [observability](./observability.md) to monitor your node's health or [test the JSON RPC API](../api/api.md).
<!-- TODO: Add more logs to help node operators debug any weird CL to EL messages! -->
[installation]: ./../installation//installation.md
[docs]: https://github.com/paradigmxyz/reth/tree/main/docs
[metrics]: https://github.com/paradigmxyz/reth/blob/main/docs/design/metrics.md#current-metrics
## Running without a Consensus Layer
We provide a method for running Reth without a Consensus Layer via the `--debug.tip <HASH>` parameter. If you provide that to your node, it will simulate sending a `engine_forkChoiceUpdated` message _once_ and will trigger syncing to the provided block hash. This is useful for testing and debugging purposes, but in order to have a node that can keep up with the tip you'll need to run a CL alongside it. At the moment we have no plans of including a Consensus Layer implementation in Reth, and we are open to including light clients other methods of syncing like importing Lighthouse as a library.

67
book/run/observability.md Normal file
View File

@ -0,0 +1,67 @@
# Observability with Prometheus & Grafana
Reth exposes a number of metrics, which are listed [here][metrics]. We can serve them from an HTTP endpoint by adding the `--metrics` flag:
```bash
RUST_LOG=info reth node --metrics 127.0.0.1:9000
```
Now, as the node is running, you can `curl` the endpoint you provided to the `--metrics` flag to get a text dump of the metrics at that time:
```bash
curl 127.0.0.1:9000
```
The response from this is quite descriptive, but it can be a bit verbose. Plus, it's just a snapshot of the metrics at the time that you `curl`ed the endpoint.
You can run the following command in a separate terminal to periodically poll the endpoint, and just print the values (without the header text) to the terminal:
```bash
while true; do date; curl -s localhost:9000 | grep -Ev '^(#|$)' | sort; echo; sleep 10; done
```
We're finally getting somewhere! As a final step, though, wouldn't it be great to see how these metrics progress over time (and generally, in a GUI)?
### Prometheus & Grafana
We're going to be using Prometheus to collect metrics off of the endpoint we set up, and use Grafana to scrape the metrics from Prometheus and define a dashboard with them.
Let's begin by installing both Prometheus and Grafana, which one can do with e.g. Homebrew:
<!-- TODO: Provide cross-platform guidance -->
```bash
brew update
brew install prometheus
brew install grafana
```
Then, kick off the Prometheus and Grafana services:
```bash
brew services start prometheus
brew services start grafana
```
This will start a Prometheus service which by default scrapes the metrics exposed at `localhost:9000`. If you launched reth with a different `--metrics` endpoint, you can change the Prometheus config file at `/usr/local/etc/prometheus/prometheus.yml` to point to the correct endpoint, and then restart the Prometheus service. You can also stop the service and launch Prometheus with a custom `prometheus.yml` like the one provided under [`etc/prometheus/prometheus.yml`](https://github.com/paradigmxyz/reth/blob/main/etc/prometheus/prometheus.yml) in this repo.
Next, open up "localhost:3000" in your browser, which is the default URL for Grafana. Here, "admin" is the default for both the username and password.
Once you've logged in, click on the gear icon in the lower left, and select "Data Sources". Click on "Add data source", and select "Prometheus" as the type. In the HTTP URL field, enter `http://localhost:9090`, this is the default endpoint for the Prometheus scrape endpoint. Finally, click "Save & Test".
As this might be a point of confusion, `localhost:9000`, which we supplied to `--metrics`, is the endpoint that Reth exposes, from which Prometheus collects metrics. Prometheus then exposes `localhost:9090` (by default) for other services (such as Grafana) to consume Prometheus metrics.
To configure the dashboard in Grafana, click on the squares icon in the upper left, and click on "New", then "Import". From there, click on "Upload JSON file", and select the example file in `reth/etc/grafana/overview.json`. Finally, select the Prometheus data source you just created, and click "Import".
And voilá, you should see your dashboard! If you're not yet connected to any peers, the dashboard will look like it's in an empty state, but once you are, you should see it start populating with data.
### Conclusion
In this runbook, we took you through starting the node, exposing different log levels, exporting metrics, and finally viewing those metrics in a Grafana dashboard.
This will all be very useful to you, whether you're simply running a home node and want to keep an eye on its performance, or if you're a contributor and want to see the effect that your (or others') changes have on Reth's operations.
[installation]: ./installation.md
[release-profile]: https://doc.rust-lang.org/cargo/reference/profiles.html#release
[docs]: https://github.com/paradigmxyz/reth/tree/main/docs
[metrics]: https://github.com/paradigmxyz/reth/blob/main/docs/design/metrics.md#current-metrics

11
book/run/run-a-node.md Normal file
View File

@ -0,0 +1,11 @@
# Run a Node
Congratulations, now that you have installed Reth, it's time to run it!
In this chapter we'll go through a few different topics you'll encounter when running Reth, including:
1. [Running on mainnet or official testnets](./mainnet.md)
1. [Setting up a local testnet](./local_testnet.md)
1. [Logs and Observability](./observability.md)
1. [Configuring reth.toml](./config.md)
In the future, we also intend to support the [OP Stack](https://stack.optimism.io/docs/understand/explainer/), which will allow you to run Reth as a Layer 2 client. More there soon!