mirror of
https://github.com/hl-archive-node/nanoreth.git
synced 2025-12-06 10:59:55 +00:00
docs: typos & cleanup (#4296)
This commit is contained in:
Bjerg
@ -8,7 +8,7 @@ The database is a central component to Reth, enabling persistent storage for dat
|
||||
|
||||
Within Reth, the database is organized via "tables". A table is any struct that implements the `Table` trait.
|
||||
|
||||
[File: crates/storage/db/src/abstraction/table.rs](https://github.com/paradigmxyz/reth/blob/main/crates/storage/db/src/abstraction/table.rs#L56-L65)
|
||||
[File: crates/storage/db/src/abstraction/table.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/storage/db/src/abstraction/table.rs#L55-L82)
|
||||
|
||||
```rust ignore
|
||||
pub trait Table: Send + Sync + Debug + 'static {
|
||||
@ -23,7 +23,7 @@ pub trait Table: Send + Sync + Debug + 'static {
|
||||
}
|
||||
|
||||
//--snip--
|
||||
pub trait Key: Encode + Decode + Ord {}
|
||||
pub trait Key: Encode + Decode + Ord + Clone + Serialize + for<'a> Deserialize<'a> {}
|
||||
|
||||
//--snip--
|
||||
pub trait Value: Compress + Decompress + Serialize {}
|
||||
@ -32,38 +32,42 @@ pub trait Value: Compress + Decompress + Serialize {}
|
||||
|
||||
The `Table` trait has two generic values, `Key` and `Value`, which need to implement the `Key` and `Value` traits, respectively. The `Encode` trait is responsible for transforming data into bytes so it can be stored in the database, while the `Decode` trait transforms the bytes back into its original form. Similarly, the `Compress` and `Decompress` traits transform the data to and from a compressed format when storing or reading data from the database.
|
||||
|
||||
There are many tables within the node, all used to store different types of data from `Headers` to `Transactions` and more. Below is a list of all of the tables. You can follow [this link](https://github.com/paradigmxyz/reth/blob/main/crates/storage/db/src/tables/mod.rs#L36) if you would like to see the table definitions for any of the tables below.
|
||||
There are many tables within the node, all used to store different types of data from `Headers` to `Transactions` and more. Below is a list of all of the tables. You can follow [this link](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/storage/db/src/tables/mod.rs#L161-L188) if you would like to see the table definitions for any of the tables below.
|
||||
|
||||
- CanonicalHeaders
|
||||
- HeaderTD
|
||||
- HeaderNumbers
|
||||
- Headers
|
||||
- BlockBodies
|
||||
- BlockBodyIndices
|
||||
- BlockOmmers
|
||||
- BlockWithdrawals
|
||||
- TransactionBlock
|
||||
- Transactions
|
||||
- TxHashNumber
|
||||
- Receipts
|
||||
- Logs
|
||||
- PlainAccountState
|
||||
- PlainStorageState
|
||||
- Bytecodes
|
||||
- BlockTransitionIndex
|
||||
- TxTransitionIndex
|
||||
- AccountHistory
|
||||
- StorageHistory
|
||||
- AccountChangeSet
|
||||
- StorageChangeSet
|
||||
- HashedAccount
|
||||
- HashedStorage
|
||||
- AccountsTrie
|
||||
- StoragesTrie
|
||||
- TxSenders
|
||||
- Config
|
||||
- SyncStage
|
||||
- SyncStageProgress
|
||||
- PruneCheckpoints
|
||||
|
||||
<br>
|
||||
|
||||
## Database
|
||||
|
||||
Reth's database design revolves around it's main [Database trait](https://github.com/paradigmxyz/reth/blob/0d9b9a392d4196793736522f3fc2ac804991b45d/crates/interfaces/src/db/mod.rs#L33), which takes advantage of [generic associated types](https://blog.rust-lang.org/2022/10/28/gats-stabilization.html) and [a few design tricks](https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats#the-better-gats) to implement the database's functionality across many types. Let's take a quick look at the `Database` trait and how it works.
|
||||
Reth's database design revolves around it's main [Database trait](https://github.com/paradigmxyz/reth/blob/eaca2a4a7fbbdc2f5cd15eab9a8a18ede1891bda/crates/storage/db/src/abstraction/database.rs#L21), which takes advantage of [generic associated types](https://blog.rust-lang.org/2022/10/28/gats-stabilization.html) and [a few design tricks](https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats#the-better-gats) to implement the database's functionality across many types. Let's take a quick look at the `Database` trait and how it works.
|
||||
|
||||
[File: crates/storage/db/src/abstraction/database.rs](https://github.com/paradigmxyz/reth/blob/main/crates/storage/db/src/abstraction/database.rs#L19)
|
||||
[File: crates/storage/db/src/abstraction/database.rs](https://github.com/paradigmxyz/reth/blob/eaca2a4a7fbbdc2f5cd15eab9a8a18ede1891bda/crates/storage/db/src/abstraction/database.rs#L21)
|
||||
|
||||
```rust ignore
|
||||
/// Main Database trait that spawns transactions to be executed.
|
||||
|
||||
@ -5,16 +5,20 @@ The `discv4` crate plays an important role in Reth, enabling discovery of other
|
||||
## Starting the Node Discovery Protocol
|
||||
As mentioned in the network and stages chapters, when the node is first started up, the `node::Command::execute()` function is called, which initializes the node and starts to run the Reth pipeline. Throughout the initialization of the node, there are many processes that are are started. One of the processes that is initialized is the p2p network which starts the node discovery protocol amongst other tasks.
|
||||
|
||||
[File: bin/reth/src/node/mod.rs](https://github.com/paradigmxyz/reth/blob/main/bin/reth/src/node/mod.rs#L95)
|
||||
[File: bin/reth/src/node/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/bin/reth/src/node/mod.rs#L314-L322)
|
||||
```rust ignore
|
||||
pub async fn execute(&self) -> eyre::Result<()> {
|
||||
//--snip--
|
||||
let network = config
|
||||
.network_config(db.clone(), chain_id, genesis_hash, self.network.disable_discovery)
|
||||
.start_network()
|
||||
.await?;
|
||||
let network = self
|
||||
.start_network(
|
||||
network_config,
|
||||
&ctx.task_executor,
|
||||
transaction_pool.clone(),
|
||||
default_peers_path,
|
||||
)
|
||||
.await?;
|
||||
|
||||
info!(peer_id = ?network.peer_id(), local_addr = %network.local_addr(), "Started p2p networking");
|
||||
info!(target: "reth::cli", peer_id = %network.peer_id(), local_addr = %network.local_addr(), "Connected to P2P network");
|
||||
|
||||
//--snip--
|
||||
}
|
||||
@ -22,7 +26,7 @@ As mentioned in the network and stages chapters, when the node is first started
|
||||
|
||||
During this process, a new `NetworkManager` is created through the `NetworkManager::new()` function, which starts the discovery protocol through a handful of newly spawned tasks. Lets take a look at how this actually works under the hood.
|
||||
|
||||
[File: crates/net/network/src/manager.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/manager.rs#L147)
|
||||
[File: crates/net/network/src/manager.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/manager.rs#L89)
|
||||
```rust ignore
|
||||
impl<C> NetworkManager<C>
|
||||
where
|
||||
@ -37,6 +41,7 @@ where
|
||||
mut discovery_v4_config,
|
||||
discovery_addr,
|
||||
boot_nodes,
|
||||
dns_discovery_config,
|
||||
//--snip--
|
||||
..
|
||||
} = config;
|
||||
@ -50,16 +55,18 @@ where
|
||||
disc_config
|
||||
});
|
||||
|
||||
let discovery = Discovery::new(discovery_addr, secret_key, discovery_v4_config).await?;
|
||||
let discovery =
|
||||
Discovery::new(discovery_addr, secret_key, discovery_v4_config, dns_discovery_config)
|
||||
.await?;
|
||||
|
||||
//--snip--
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
First, the `NetworkConfig` is deconstructed and the `disc_config` is updated to merge configured [bootstrap nodes](https://github.com/paradigmxyz/reth/blob/main/crates/net/discv4/src/bootnodes.rs#L8) and add the `forkid` to adhere to [EIP 868](https://eips.ethereum.org/EIPS/eip-868). This updated configuration variable is then passed into the `Discovery::new()` function. Note that `Discovery` is a catch all for all discovery services, which include discv4, DNS discovery and others in the future.
|
||||
First, the `NetworkConfig` is deconstructed and the `disc_config` is updated to merge configured [bootstrap nodes](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/primitives/src/net.rs#L120-L151) and add the `forkid` to adhere to [EIP 868](https://eips.ethereum.org/EIPS/eip-868). This updated configuration variable is then passed into the `Discovery::new()` function. Note that `Discovery` is a catch all for all discovery services, which include discv4, DNS discovery and others in the future.
|
||||
|
||||
[File: crates/net/network/src/discovery.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/discovery.rs#L51)
|
||||
[File: crates/net/network/src/discovery.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/discovery.rs#L53)
|
||||
```rust ignore
|
||||
impl Discovery {
|
||||
/// Spawns the discovery service.
|
||||
@ -70,6 +77,7 @@ impl Discovery {
|
||||
discovery_addr: SocketAddr,
|
||||
sk: SecretKey,
|
||||
discv4_config: Option<Discv4Config>,
|
||||
dns_discovery_config: Option<DnsDiscoveryConfig>,
|
||||
) -> Result<Self, NetworkError> {
|
||||
let local_enr = NodeRecord::from_secret_key(discovery_addr, &sk);
|
||||
|
||||
@ -86,6 +94,20 @@ impl Discovery {
|
||||
(None, None, None)
|
||||
};
|
||||
|
||||
// setup DNS discovery
|
||||
let (_dns_discovery, dns_discovery_updates, _dns_disc_service) =
|
||||
if let Some(dns_config) = dns_discovery_config {
|
||||
let (mut service, dns_disc) = DnsDiscoveryService::new_pair(
|
||||
Arc::new(DnsResolver::from_system_conf()?),
|
||||
dns_config,
|
||||
);
|
||||
let dns_discovery_updates = service.node_record_stream();
|
||||
let dns_disc_service = service.spawn();
|
||||
(Some(dns_disc), Some(dns_discovery_updates), Some(dns_disc_service))
|
||||
} else {
|
||||
(None, None, None)
|
||||
};
|
||||
|
||||
Ok(Self {
|
||||
local_enr,
|
||||
discv4,
|
||||
|
||||
@ -12,7 +12,7 @@ This crate can be thought of as having 2 components:
|
||||
## Types
|
||||
The most basic Eth-wire type is an `ProtocolMessage`. It describes all messages that reth can send/receive.
|
||||
|
||||
[File: crates/net/eth-wire/src/types/message.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/types/message.rs)
|
||||
[File: crates/net/eth-wire/src/types/message.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/types/message.rs)
|
||||
```rust, ignore
|
||||
/// An `eth` protocol message, containing a message ID and payload.
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
@ -50,7 +50,7 @@ pub enum EthMessageID {
|
||||
Messages can either be broadcast to the network, or can be a request/response message to a single peer. This 2nd type of message is
|
||||
described using a `RequestPair` struct, which is simply a concatenation of the underlying message with a request id.
|
||||
|
||||
[File: crates/net/eth-wire/src/types/message.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/types/message.rs)
|
||||
[File: crates/net/eth-wire/src/types/message.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/types/message.rs)
|
||||
```rust, ignore
|
||||
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct RequestPair<T> {
|
||||
@ -62,7 +62,7 @@ pub struct RequestPair<T> {
|
||||
Every `Ethmessage` has a correspoding rust struct which implements the `Encodable` and `Decodable` traits.
|
||||
These traits are defined as follows:
|
||||
|
||||
[Crate: crates/common/rlp](https://github.com/paradigmxyz/reth/blob/main/crates/common/rlp)
|
||||
[Crate: crates/rlp](https://github.com/paradigmxyz/reth/tree/1563506aea09049a85e5cc72c2894f3f7a371581/crates/rlp)
|
||||
```rust, ignore
|
||||
pub trait Decodable: Sized {
|
||||
fn decode(buf: &mut &[u8]) -> Result<Self, DecodeError>;
|
||||
@ -93,7 +93,7 @@ The items in the list are transactions in the format described in the main Ether
|
||||
|
||||
In reth, this is represented as:
|
||||
|
||||
[File: crates/net/eth-wire/src/types/broadcast.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/types/broadcast.rs)
|
||||
[File: crates/net/eth-wire/src/types/broadcast.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/types/broadcast.rs)
|
||||
```rust,ignore
|
||||
pub struct Transactions(
|
||||
/// New transactions for the peer to include in its mempool.
|
||||
@ -103,7 +103,7 @@ pub struct Transactions(
|
||||
|
||||
And the corresponding trait implementations are present in the primitives crate.
|
||||
|
||||
[File: crates/primitives/src/transaction/mod.rs](https://github.com/paradigmxyz/reth/blob/main/crates/primitives/src/transaction/mod.rs)
|
||||
[File: crates/primitives/src/transaction/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/primitives/src/transaction/mod.rs)
|
||||
```rust, ignore
|
||||
#[main_codec]
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash, AsRef, Deref, Default)]
|
||||
@ -131,7 +131,7 @@ impl Decodable for TransactionSigned {
|
||||
// Implementation omitted for brevity
|
||||
//...
|
||||
}
|
||||
|
||||
}
|
||||
```
|
||||
Now that we know how the types work, let's take a look at how these are utilized in the network.
|
||||
|
||||
@ -146,7 +146,7 @@ The lowest level stream to communicate with other peers is the P2P stream. It ta
|
||||
Decompression/Compression of bytes is done with snappy algorithm ([EIP 706](https://eips.ethereum.org/EIPS/eip-706))
|
||||
using the external `snap` crate.
|
||||
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/p2pstream.rs)
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/p2pstream.rs)
|
||||
```rust,ignore
|
||||
#[pin_project]
|
||||
pub struct P2PStream<S> {
|
||||
@ -164,7 +164,7 @@ pub struct P2PStream<S> {
|
||||
To manage pinging, an instance of the `Pinger` struct is used. This is a state machine which keeps track of how many pings
|
||||
we have sent/received and the timeouts associated with them.
|
||||
|
||||
[File: crates/net/eth-wire/src/pinger.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/pinger.rs)
|
||||
[File: crates/net/eth-wire/src/pinger.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/pinger.rs)
|
||||
```rust,ignore
|
||||
#[derive(Debug)]
|
||||
pub(crate) struct Pinger {
|
||||
@ -190,7 +190,7 @@ pub(crate) enum PingState {
|
||||
|
||||
State transitions are then implemented like a future, with the `poll_ping` function advancing the state of the pinger.
|
||||
|
||||
[File: crates/net/eth-wire/src/pinger.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/pinger.rs)
|
||||
[File: crates/net/eth-wire/src/pinger.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/pinger.rs)
|
||||
```rust, ignore
|
||||
pub(crate) fn poll_ping(
|
||||
&mut self,
|
||||
@ -218,12 +218,12 @@ pub(crate) fn poll_ping(
|
||||
```
|
||||
|
||||
### Sending and receiving data
|
||||
To send and recieve data, the P2PStream itself is a future which implemenents the `Stream` and `Sink` traits from the `futures` crate.
|
||||
To send and receive data, the P2PStream itself is a future which implements the `Stream` and `Sink` traits from the `futures` crate.
|
||||
|
||||
For the `Stream` trait, the `inner` stream is polled, decompressed and returned. Most of the code is just
|
||||
error handling and is omitted here for clarity.
|
||||
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/p2pstream.rs)
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/p2pstream.rs)
|
||||
```rust,ignore
|
||||
|
||||
impl<S> Stream for P2PStream<S> {
|
||||
@ -250,7 +250,7 @@ impl<S> Stream for P2PStream<S> {
|
||||
Similarly, for the `Sink` trait, we do the reverse, compressing and sending data out to the `inner` stream.
|
||||
The important functions in this trait are shown below.
|
||||
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/p2pstream.rs)
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/p2pstream.rs)
|
||||
```rust, ignore
|
||||
impl<S> Sink<Bytes> for P2PStream<S> {
|
||||
fn start_send(self: Pin<&mut Self>, item: Bytes) -> Result<(), Self::Error> {
|
||||
@ -287,7 +287,7 @@ impl<S> Sink<Bytes> for P2PStream<S> {
|
||||
## EthStream
|
||||
The EthStream is very simple, it does not keep track of any state, it simply wraps the P2Pstream.
|
||||
|
||||
[File: crates/net/eth-wire/src/ethstream.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/ethstream.rs)
|
||||
[File: crates/net/eth-wire/src/ethstream.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/ethstream.rs)
|
||||
```rust,ignore
|
||||
#[pin_project]
|
||||
pub struct EthStream<S> {
|
||||
@ -298,7 +298,7 @@ pub struct EthStream<S> {
|
||||
EthStream's only job is to perform the RLP decoding/encoding, using the `ProtocolMessage::decode()` and `ProtocolMessage::encode()`
|
||||
functions we looked at earlier.
|
||||
|
||||
[File: crates/net/eth-wire/src/ethstream.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/ethstream.rs)
|
||||
[File: crates/net/eth-wire/src/ethstream.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/ethstream.rs)
|
||||
```rust,ignore
|
||||
impl<S, E> Stream for EthStream<S> {
|
||||
// ...
|
||||
@ -341,7 +341,7 @@ To perform these, reth has special `Unauthed` versions of streams described abov
|
||||
|
||||
The `UnauthedP2Pstream` does the `Hello` handshake and returns a `P2PStream`.
|
||||
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/p2pstream.rs)
|
||||
[File: crates/net/eth-wire/src/p2pstream.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/p2pstream.rs)
|
||||
```rust, ignore
|
||||
#[pin_project]
|
||||
pub struct UnauthedP2PStream<S> {
|
||||
@ -370,6 +370,6 @@ impl<S> UnauthedP2PStream<S> {
|
||||
}
|
||||
|
||||
```
|
||||
Similary, UnauthedEthStream does the `Status` handshake and returns an `EthStream`. The code is [here](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/ethstream.rs)
|
||||
Similarly, UnauthedEthStream does the `Status` handshake and returns an `EthStream`. The code is [here](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/ethstream.rs)
|
||||
|
||||
|
||||
|
||||
@ -33,7 +33,7 @@ The `"node"` CLI command, used to run the node itself, does the following at a h
|
||||
|
||||
Steps 5-6 are of interest to us as they consume items from the `network` crate:
|
||||
|
||||
[File: bin/reth/src/node/mod.rs](https://github.com/paradigmxyz/reth/blob/main/bin/reth/src/node/mod.rs)
|
||||
[File: bin/reth/src/node/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/bin/reth/src/node/mod.rs)
|
||||
```rust,ignore
|
||||
let network = start_network(network_config(db.clone(), chain_id, genesis_hash)).await?;
|
||||
|
||||
@ -84,7 +84,7 @@ pipeline.run(db.clone()).await?;
|
||||
|
||||
Let's begin by taking a look at the line where the network is started, with the call, unsurprisingly, to `start_network`. Sounds important, doesn't it?
|
||||
|
||||
[File: bin/reth/src/node/mod.rs](https://github.com/paradigmxyz/reth/blob/main/bin/reth/src/node/mod.rs)
|
||||
[File: bin/reth/src/node/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/bin/reth/src/node/mod.rs)
|
||||
```rust,ignore
|
||||
async fn start_network<C>(config: NetworkConfig<C>) -> Result<NetworkHandle, NetworkError>
|
||||
where
|
||||
@ -107,7 +107,7 @@ It gets the handles for the network management, transactions, and ETH requests t
|
||||
|
||||
The `NetworkManager::builder` constructor requires a `NetworkConfig` struct to be passed in as a parameter, which can be used as the main entrypoint for setting up the entire network layer:
|
||||
|
||||
[File: crates/net/network/src/config.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/config.rs)
|
||||
[File: crates/net/network/src/config.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/config.rs)
|
||||
```rust,ignore
|
||||
pub struct NetworkConfig<C> {
|
||||
/// The client type that can interact with the chain.
|
||||
@ -152,7 +152,7 @@ pub struct NetworkConfig<C> {
|
||||
|
||||
The discovery task progresses as the network management task is polled, handling events regarding peer management through the `Swarm` struct which is stored as a field on the `NetworkManager`:
|
||||
|
||||
[File: crates/net/network/src/swarm.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/swarm.rs)
|
||||
[File: crates/net/network/src/swarm.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/swarm.rs)
|
||||
```rust,ignore
|
||||
pub(crate) struct Swarm<C> {
|
||||
/// Listens for new incoming connections.
|
||||
@ -180,7 +180,7 @@ Let's walk through how each is implemented, and then apply that knowledge to und
|
||||
|
||||
The `NetworkHandle` struct is a client for the network management task that can be shared across threads. It wraps an `Arc` around the `NetworkInner` struct, defined as follows:
|
||||
|
||||
[File: crates/net/network/src/network.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/network.rs)
|
||||
[File: crates/net/network/src/network.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/network.rs)
|
||||
```rust,ignore
|
||||
struct NetworkInner {
|
||||
/// Number of active peer sessions the node's currently handling.
|
||||
@ -200,7 +200,7 @@ struct NetworkInner {
|
||||
|
||||
The field of note here is `to_manager_tx`, which is a handle that can be used to send messages in a channel to an instance of the `NetworkManager` struct.
|
||||
|
||||
[File: crates/net/network/src/manager.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/manager.rs)
|
||||
[File: crates/net/network/src/manager.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/manager.rs)
|
||||
```rust,ignore
|
||||
pub struct NetworkManager<C> {
|
||||
/// The type that manages the actual network part, which includes connections.
|
||||
@ -235,7 +235,7 @@ While the `NetworkManager` is meant to be spawned as a standalone [`tokio::task`
|
||||
|
||||
In the pipeline, the `NetworkHandle` is used to instantiate the `FetchClient` - which we'll get into next - and is used in the `HeaderStage` to update the node's ["status"](https://github.com/ethereum/devp2p/blob/master/caps/eth.md#status-0x00) (record the total difficulty, hash, and height of the last processed block).
|
||||
|
||||
[File: crates/stages/src/stages/headers.rs](https://github.com/paradigmxyz/reth/blob/main/crates/stages/src/stages/headers.rs)
|
||||
[File: crates/stages/src/stages/headers.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/stages/src/stages/headers.rs)
|
||||
```rust,ignore
|
||||
async fn update_head<DB: Database>(
|
||||
&self,
|
||||
@ -255,7 +255,7 @@ Now that we have some understanding about the internals of the network managemen
|
||||
|
||||
The `FetchClient` struct, similar to `NetworkHandle`, can be shared across threads, and is a client for fetching data from the network. It's a fairly lightweight struct:
|
||||
|
||||
[File: crates/net/network/src/fetch/client.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/fetch/client.rs)
|
||||
[File: crates/net/network/src/fetch/client.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/fetch/client.rs)
|
||||
```rust,ignore
|
||||
pub struct FetchClient {
|
||||
/// Sender half of the request channel.
|
||||
@ -271,7 +271,7 @@ The `request_tx` field is a handle to a channel that can be used to send request
|
||||
|
||||
The fields `request_tx` and `peers_handle` are cloned off of the `StateFetcher` struct when instantiating the `FetchClient`, which is the lower-level struct responsible for managing data fetching operations over the network:
|
||||
|
||||
[File: crates/net/network/src/fetch/mod.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/fetch/mod.rs)
|
||||
[File: crates/net/network/src/fetch/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/fetch/mod.rs)
|
||||
```rust,ignore
|
||||
pub struct StateFetcher {
|
||||
/// Currently active [`GetBlockHeaders`] requests
|
||||
@ -295,7 +295,7 @@ pub struct StateFetcher {
|
||||
|
||||
This struct itself is nested deeply within the `NetworkManager`: its `Swarm` struct (shown earlier in the chapter) contains a `NetworkState` struct that has the `StateFetcher` as a field:
|
||||
|
||||
[File: crates/net/network/src/state.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/state.rs)
|
||||
[File: crates/net/network/src/state.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/state.rs)
|
||||
```rust,ignore
|
||||
pub struct NetworkState<C> {
|
||||
/// All active peers and their state.
|
||||
@ -322,7 +322,7 @@ pub struct NetworkState<C> {
|
||||
|
||||
The `FetchClient` implements the `HeadersClient` and `BodiesClient` traits, defining the functionality to get headers and block bodies from available peers.
|
||||
|
||||
[File: crates/net/network/src/fetch/client.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/fetch/client.rs)
|
||||
[File: crates/net/network/src/fetch/client.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/fetch/client.rs)
|
||||
```rust,ignore
|
||||
impl HeadersClient for FetchClient {
|
||||
/// Sends a `GetBlockHeaders` request to an available peer.
|
||||
@ -346,7 +346,7 @@ This functionality is used in the `HeaderStage` and `BodyStage`, respectively.
|
||||
|
||||
In the pipeline used by the main Reth binary, the `HeaderStage` uses a `ReverseHeadersDownloader` to stream headers from the network:
|
||||
|
||||
[File: crates/net/downloaders/src/headers/linear.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/downloaders/src/headers/linear.rs)
|
||||
[File: crates/net/downloaders/src/headers/reverse_headers.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/downloaders/src/headers/reverse_headers.rs)
|
||||
```rust,ignore
|
||||
pub struct ReverseHeadersDownloader<C, H> {
|
||||
/// The consensus client
|
||||
@ -362,7 +362,7 @@ pub struct ReverseHeadersDownloader<C, H> {
|
||||
|
||||
A `FetchClient` is passed in to the `client` field, and the `get_headers` method it implements gets used when polling the stream created by the `ReverseHeadersDownloader` in the `execute` method of the `HeaderStage`.
|
||||
|
||||
[File: crates/net/downloaders/src/headers/linear.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/downloaders/src/headers/linear.rs)
|
||||
[File: crates/net/downloaders/src/headers/reverse_headers.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/downloaders/src/headers/reverse_headers.rs)
|
||||
```rust,ignore
|
||||
fn get_or_init_fut(&mut self) -> HeadersRequestFuture {
|
||||
match self.request.take() {
|
||||
@ -388,7 +388,7 @@ fn get_or_init_fut(&mut self) -> HeadersRequestFuture {
|
||||
|
||||
In the `BodyStage` configured by the main binary, a `BodiesDownloader` is used:
|
||||
|
||||
[File: crates/net/downloaders/src/bodies/concurrent.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/downloaders/src/bodies/concurrent.rs)
|
||||
[File: crates/net/downloaders/src/bodies/bodies.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/downloaders/src/bodies/bodies.rs)
|
||||
```rust,ignore
|
||||
pub struct BodiesDownloader<Client, Consensus> {
|
||||
/// The bodies client
|
||||
@ -406,7 +406,7 @@ pub struct BodiesDownloader<Client, Consensus> {
|
||||
|
||||
Here, similarly, a `FetchClient` is passed in to the `client` field, and the `get_block_bodies` method it implements is used when constructing the stream created by the `BodiesDownloader` in the `execute` method of the `BodyStage`.
|
||||
|
||||
[File: crates/net/downloaders/src/bodies/concurrent.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/downloaders/src/bodies/concurrent.rs)
|
||||
[File: crates/net/downloaders/src/bodies/bodies.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/downloaders/src/bodies/bodies.rs)
|
||||
```rust,ignore
|
||||
async fn fetch_bodies(
|
||||
&self,
|
||||
@ -425,7 +425,7 @@ When `FetchClient.get_headers` or `FetchClient.get_block_bodies` is called, thos
|
||||
|
||||
Every time the `StateFetcher` is polled, it finds the next idle peer available to service the current request (for either a block header, or a block body). In this context, "idle" means any peer that is not currently handling a request from the node:
|
||||
|
||||
[File: crates/net/network/src/fetch/mod.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/fetch/mod.rs)
|
||||
[File: crates/net/network/src/fetch/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/fetch/mod.rs)
|
||||
```rust,ignore
|
||||
/// Returns the next action to return
|
||||
fn poll_action(&mut self) -> PollAction {
|
||||
@ -455,7 +455,7 @@ The ETH requests task serves _incoming_ requests related to blocks in the [`eth`
|
||||
|
||||
Similar to the network management task, it's implemented as an endless future, but it is meant to run as a background task (on a standalone `tokio::task`) and not to be interacted with directly from the pipeline. It's represented by the following `EthRequestHandler` struct:
|
||||
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/eth_requests.rs)
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/eth_requests.rs)
|
||||
```rust,ignore
|
||||
pub struct EthRequestHandler<C> {
|
||||
/// The client type that can interact with the chain.
|
||||
@ -480,7 +480,7 @@ As the `NetworkManager` is polled and listens for events from peers passed throu
|
||||
|
||||
Being an endless future, the core of the ETH requests task's functionality is in its `poll` method implementation. As the `EthRequestHandler` is polled, it listens for any ETH requests coming through the channel, and handles them accordingly. At the time of writing, the ETH requests task can handle the [`GetBlockHeaders`](https://github.com/ethereum/devp2p/blob/master/caps/eth.md#getblockheaders-0x03) and [`GetBlockBodies`](https://github.com/ethereum/devp2p/blob/master/caps/eth.md#getblockbodies-0x05) requests.
|
||||
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/eth_requests.rs)
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/eth_requests.rs)
|
||||
```rust,ignore
|
||||
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
let this = self.get_mut();
|
||||
@ -506,7 +506,7 @@ fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
|
||||
The handling of these requests is fairly straightforward. The `GetBlockHeaders` payload is the following:
|
||||
|
||||
[File: crates/net/eth-wire/src/types/blocks.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/types/blocks.rs)
|
||||
[File: crates/net/eth-wire/src/types/blocks.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/types/blocks.rs)
|
||||
```rust,ignore
|
||||
pub struct GetBlockHeaders {
|
||||
/// The block number or hash that the peer should start returning headers from.
|
||||
@ -528,7 +528,7 @@ pub struct GetBlockHeaders {
|
||||
|
||||
In handling this request, the ETH requests task attempts, starting with `start_block`, to fetch the associated header from the database, increment/decrement the block number to fetch by `skip` depending on the `direction` while checking for overflow/underflow, and checks that bounds specifying the maximum numbers of headers or bytes to send have not been breached.
|
||||
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/eth_requests.rs)
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/eth_requests.rs)
|
||||
```rust,ignore
|
||||
fn get_headers_response(&self, request: GetBlockHeaders) -> Vec<Header> {
|
||||
let GetBlockHeaders { start_block, limit, skip, direction } = request;
|
||||
@ -598,7 +598,7 @@ fn get_headers_response(&self, request: GetBlockHeaders) -> Vec<Header> {
|
||||
|
||||
The `GetBlockBodies` payload is simpler, it just contains a vector of requested block hashes:
|
||||
|
||||
[File: crates/net/eth-wire/src/types/blocks.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/eth-wire/src/types/blocks.rs)
|
||||
[File: crates/net/eth-wire/src/types/blocks.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/eth-wire/src/types/blocks.rs)
|
||||
```rust,ignore
|
||||
pub struct GetBlockBodies(
|
||||
/// The block hashes to request bodies for.
|
||||
@ -608,7 +608,7 @@ pub struct GetBlockBodies(
|
||||
|
||||
In handling this request, similarly, the ETH requests task attempts, for each hash in the requested order, to fetch the block body (transactions & ommers), while checking that bounds specifying the maximum numbers of bodies or bytes to send have not been breached.
|
||||
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/eth_requests.rs)
|
||||
[File: crates/net/network/src/eth_requests.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/eth_requests.rs)
|
||||
```rust,ignore
|
||||
fn on_bodies_request(
|
||||
&mut self,
|
||||
@ -653,7 +653,7 @@ in the [transaction-pool](../../../ethereum/transaction-pool/README.md) chapter.
|
||||
|
||||
Again, like the network management and ETH requests tasks, the transactions task is implemented as an endless future that runs as a background task on a standalone `tokio::task`. It's represented by the `TransactionsManager` struct:
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
pub struct TransactionsManager<Pool> {
|
||||
/// Access to the transaction pool.
|
||||
@ -688,7 +688,7 @@ pub struct TransactionsManager<Pool> {
|
||||
|
||||
Unlike the ETH requests task, but like the network management task's `NetworkHandle`, the transactions task can also be accessed via a shareable "handle" struct, the `TransactionsHandle`:
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
pub struct TransactionsHandle {
|
||||
/// Command channel to the [`TransactionsManager`]
|
||||
@ -711,7 +711,7 @@ Let's get a view into the transactions task's operation by walking through the `
|
||||
The `poll` method lays out an order of operations for the transactions task. It begins by draining the `TransactionsManager.network_events`, `TransactionsManager.command_rx`, and `TransactionsManager.transaction_events` streams, in this order.
|
||||
Then, it checks on all the current `TransactionsManager.inflight_requests`, which are requests sent by the node to its peers for full transaction objects. After this, it checks on the status of completed `TransactionsManager.pool_imports` events, which are transactions that are being imported into the node's transaction pool. Finally, it drains the new `TransactionsManager.pending_transactions` events from the transaction pool.
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
let this = self.get_mut();
|
||||
@ -786,7 +786,7 @@ The `TransactionsManager.network_events` stream is the first to have all of its
|
||||
|
||||
The events received in this channel are of type `NetworkEvent`:
|
||||
|
||||
[File: crates/net/network/src/manager.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/manager.rs)
|
||||
[File: crates/net/network/src/manager.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/manager.rs)
|
||||
```rust,ignore
|
||||
pub enum NetworkEvent {
|
||||
/// Closed the peer session.
|
||||
@ -822,7 +822,7 @@ Removes the peer given by `NetworkEvent::SessionClosed.peer_id` from the `Transa
|
||||
**`NetworkEvent::SessionEstablished`**
|
||||
Begins by inserting a `Peer` into `TransactionsManager.peers` by `peer_id`, which is a struct of the following form:
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
struct Peer {
|
||||
/// Keeps track of transactions that we know the peer has seen.
|
||||
@ -838,7 +838,7 @@ The `request_tx` field on the `Peer` is used at the sender end of a channel to s
|
||||
|
||||
After the `Peer` is added to `TransactionsManager.peers`, the hashes of all of the transactions in the node's transaction pool are sent to the peer in a [`NewPooledTransactionHashes` message](https://github.com/ethereum/devp2p/blob/master/caps/eth.md#newpooledtransactionhashes-0x08).
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
fn on_network_event(&mut self, event: NetworkEvent) {
|
||||
match event {
|
||||
@ -875,7 +875,7 @@ fn on_network_event(&mut self, event: NetworkEvent) {
|
||||
|
||||
Next in the `poll` method, `TransactionsCommand`s sent through the `TransactionsManager.command_rx` stream are handled. These are the next to be handled as they are those sent manually via the `TransactionsHandle`, giving them precedence over transactions-related requests picked up from the network. The `TransactionsCommand` enum has the following form:
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
enum TransactionsCommand {
|
||||
PropagateHash(H256),
|
||||
@ -886,7 +886,7 @@ enum TransactionsCommand {
|
||||
|
||||
`on_new_transactions` propagates the full transaction object, with the signer attached, to a small random sample of peers using the `propagate_transactions` method. Then, it notifies all other peers of the hash of the new transaction, so that they can request the full transaction object if they don't already have it.
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
fn on_new_transactions(&mut self, hashes: impl IntoIterator<Item = TxHash>) {
|
||||
trace!(target: "net::tx", "Start propagating transactions");
|
||||
@ -946,7 +946,7 @@ fn propagate_transactions(
|
||||
|
||||
After `TransactionsCommand`s, it's time to take care of transactions-related requests sent by peers in the network, so the `poll` method handles `NetworkTransactionEvent`s received through the `TransactionsManager.transaction_events` stream. `NetworkTransactionEvent` has the following form:
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
pub enum NetworkTransactionEvent {
|
||||
/// Received list of transactions from the given peer.
|
||||
@ -976,7 +976,7 @@ To understand this a bit better, let's double back and examine what `Transaction
|
||||
|
||||
`TransactionsManager.pool_imports` is a set of futures representing the transactions which are currently in the process of being imported to the node's transaction pool. This process is asynchronous due to the validation of the transaction that must occur, thus we need to keep a handle on the generated future.
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
fn import_transactions(&mut self, peer_id: PeerId, transactions: Vec<TransactionSigned>) {
|
||||
let mut has_bad_transactions = false;
|
||||
@ -1026,7 +1026,7 @@ This event is generated from the [`NewPooledTransactionHashes` protocol message]
|
||||
|
||||
Here, it begins by adding the transaction hashes included in the `NewPooledTransactionHashes` payload to the LRU cache for the `Peer` identified by `peer_id` in `TransactionsManager.peers`. Next, it filters the list of hashes to those that are not already present in the transaction pool, and for each such hash, requests its full transaction object from the peer by sending it a [`GetPooledTransactions` protocol message](https://github.com/ethereum/devp2p/blob/master/caps/eth.md#getpooledtransactions-0x09) through the `Peer.request_tx` channel. If the request was successfully sent, a `GetPooledTxRequest` gets added to `TransactionsManager.inflight_requests` vector:
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
struct GetPooledTxRequest {
|
||||
peer_id: PeerId,
|
||||
@ -1036,7 +1036,7 @@ struct GetPooledTxRequest {
|
||||
|
||||
As you can see, this struct also contains a `response` channel from which the peer's response can later be polled.
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
fn on_new_pooled_transactions(&mut self, peer_id: PeerId, msg: NewPooledTransactionHashes) {
|
||||
if let Some(peer) = self.peers.get_mut(&peer_id) {
|
||||
@ -1072,7 +1072,7 @@ This event is generated from the [`GetPooledTransactions` protocol message](http
|
||||
|
||||
Here, it collects _all_ the transactions in the node's transaction pool, recovers their signers, adds their hashes to the LRU cache of the requesting peer, and sends them to the peer in a [`PooledTransactions` protocol message](https://github.com/ethereum/devp2p/blob/master/caps/eth.md#pooledtransactions-0x0a). This is sent through the `response` channel that's stored as a field of the `NetworkTransaction::GetPooledTransactions` variant itself.
|
||||
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/main/crates/net/network/src/transactions.rs)
|
||||
[File: crates/net/network/src/transactions.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/net/network/src/transactions.rs)
|
||||
```rust,ignore
|
||||
fn on_get_pooled_transactions(
|
||||
&mut self,
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
The `stages` lib plays a central role in syncing the node, maintaining state, updating the database and more. The stages involved in the Reth pipeline are the `HeaderStage`, `BodyStage`, `SenderRecoveryStage`, and `ExecutionStage` (note that this list is non-exhaustive, and more pipeline stages will be added in the near future). Each of these stages are queued up and stored within the Reth pipeline.
|
||||
|
||||
[File: crates/stages/src/pipeline.rs](https://github.com/paradigmxyz/reth/blob/main/crates/stages/src/pipeline.rs)
|
||||
[File: crates/stages/src/pipeline/mod.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/stages/src/pipeline/mod.rs)
|
||||
```rust,ignore
|
||||
pub struct Pipeline<DB: Database, U: SyncStateUpdater> {
|
||||
stages: Vec<BoxedStage<DB>>,
|
||||
@ -19,7 +19,7 @@ When the node is first started, a new `Pipeline` is initialized and all of the s
|
||||
|
||||
Each stage within the pipeline implements the `Stage` trait which provides function interfaces to get the stage id, execute the stage and unwind the changes to the database if there was an issue during the stage execution.
|
||||
|
||||
[File: crates/stages/src/stage.rs](https://github.com/paradigmxyz/reth/blob/main/crates/stages/src/stage.rs)
|
||||
[File: crates/stages/src/stage.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/stages/src/stage.rs)
|
||||
```rust,ignore
|
||||
pub trait Stage<DB: Database>: Send + Sync {
|
||||
/// Get the ID of the stage.
|
||||
@ -53,7 +53,7 @@ To get a better idea of what is happening at each part of the pipeline, lets wal
|
||||
<!-- TODO: Cross-link to eth/65 chapter when it's written -->
|
||||
The `HeaderStage` is responsible for syncing the block headers, validating the header integrity and writing the headers to the database. When the `execute()` function is called, the local head of the chain is updated to the most recent block height previously executed by the stage. At this point, the node status is also updated with that block's height, hash and total difficulty. These values are used during any new eth/65 handshakes. After updating the head, a stream is established with other peers in the network to sync the missing chain headers between the most recent state stored in the database and the chain tip. The `HeaderStage` contains a `downloader` attribute, which is a type that implements the `HeaderDownloader` trait. A `HeaderDownloader` is a `Stream` that returns batches of headers.
|
||||
|
||||
[File: crates/interfaces/src/p2p/headers/downloader.rs](https://github.com/paradigmxyz/reth/blob/main/crates/interfaces/src/p2p/headers/downloader.rs)
|
||||
[File: crates/interfaces/src/p2p/headers/downloader.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/interfaces/src/p2p/headers/downloader.rs)
|
||||
```rust,ignore
|
||||
pub trait HeaderDownloader: Send + Sync + Stream<Item = Vec<SealedHeader>> + Unpin {
|
||||
/// Updates the gap to sync which ranges from local head to the sync target
|
||||
@ -77,7 +77,7 @@ pub trait HeaderDownloader: Send + Sync + Stream<Item = Vec<SealedHeader>> + Unp
|
||||
|
||||
The `HeaderStage` relies on the downloader stream to return the headers in descending order starting from the chain tip down to the latest block in the database. While other stages in the `Pipeline` start from the most recent block in the database up to the chain tip, the `HeaderStage` works in reverse to avoid [long-range attacks](https://messari.io/report/long-range-attack). When a node downloads headers in ascending order, it will not know if it is being subjected to a long-range attack until it reaches the most recent blocks. To combat this, the `HeaderStage` starts by getting the chain tip from the Consensus Layer, verifies the tip, and then walks backwards by the parent hash. Each value yielded from the stream is a `SealedHeader`.
|
||||
|
||||
[File: crates/primitives/src/header.rs](https://github.com/paradigmxyz/reth/blob/main/crates/primitives/src/header.rs)
|
||||
[File: crates/primitives/src/header.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/primitives/src/header.rs)
|
||||
```rust,ignore
|
||||
pub struct SealedHeader {
|
||||
/// Locked Header fields.
|
||||
@ -130,7 +130,7 @@ The new block is then pre-validated, checking that the ommers hash and transacti
|
||||
|
||||
Following a successful `BodyStage`, the `SenderRecoveryStage` starts to execute. The `SenderRecoveryStage` is responsible for recovering the transaction sender for each of the newly added transactions to the database. At the beginning of the execution function, all of the transactions are first retrieved from the database. Then the `SenderRecoveryStage` goes through each transaction and recovers the signer from the transaction signature and hash. The transaction hash is derived by taking the Keccak 256-bit hash of the RLP encoded transaction bytes. This hash is then passed into the `recover_signer` function.
|
||||
|
||||
[File: crates/primitives/src/transaction/signature.rs](https://github.com/paradigmxyz/reth/blob/main/crates/primitives/src/transaction/signature.rs)
|
||||
[File: crates/primitives/src/transaction/signature.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/primitives/src/transaction/signature.rs)
|
||||
```rust,ignore
|
||||
pub(crate) fn recover_signer(&self, hash: H256) -> Option<Address> {
|
||||
let mut sig: [u8; 65] = [0; 65];
|
||||
@ -157,7 +157,7 @@ Once the transaction signer has been recovered, the signer is then added to the
|
||||
|
||||
Finally, after all headers, bodies and senders are added to the database, the `ExecutionStage` starts to execute. This stage is responsible for executing all of the transactions and updating the state stored in the database. For every new block header added to the database, the corresponding transactions have their signers attached to them and `reth_blockchain_tree::executor::execute_and_verify_receipt()` is called, pushing the state changes resulting from the execution to a `Vec`.
|
||||
|
||||
[File: crates/stages/src/stages/execution.rs](https://github.com/paradigmxyz/reth/blob/main/crates/stages/src/stages/execution.rs)
|
||||
[File: crates/stages/src/stages/execution.rs](https://github.com/paradigmxyz/reth/blob/1563506aea09049a85e5cc72c2894f3f7a371581/crates/stages/src/stages/execution.rs)
|
||||
```rust,ignore
|
||||
pub fn execute_and_verify_receipt<DB: StateProvider>(
|
||||
block: &Block,
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
* We implemented that trait for the following encoding formats:
|
||||
* [Ethereum-specific Compact Encoding](https://github.com/paradigmxyz/reth/blob/0d9b9a392d4196793736522f3fc2ac804991b45d/crates/codecs/derive/src/compact/mod.rs): A lot of Ethereum datatypes have unnecessary zeros when serialized, or optional (e.g. on empty hashes) which would be nice not to pay in storage costs.
|
||||
* [Erigon](https://github.com/ledgerwatch/erigon/blob/12ee33a492f5d240458822d052820d9998653a63/docs/programmers_guide/db_walkthrough.MD) achieves that by having a `bitfield` set on Table "PlainState which adds a bitfield to Accounts.
|
||||
* Akula expanded it for other tables and datatypes manually. It also saved some more space by storing the length of certain types (U256, u64) using the modular_bitfield crate, which compacts this information.
|
||||
* Akula expanded it for other tables and datatypes manually. It also saved some more space by storing the length of certain types (U256, u64) using the [`modular_bitfield`](https://docs.rs/modular-bitfield/latest/modular_bitfield/) crate, which compacts this information.
|
||||
* We generalized it for all types, by writing a derive macro that autogenerates code for implementing the trait. It, also generates the interfaces required for fuzzing using ToB/test-fuzz:
|
||||
* [Scale Encoding](https://github.com/paritytech/parity-scale-codec)
|
||||
* [Postcard Encoding](https://github.com/jamesmunns/postcard)
|
||||
|
||||
Reference in New Issue
Block a user