Commit Graph

5487 Commits

Author SHA1 Message Date
ccdf2ffa92 hotfix(eth-wire): loosen lints temporarily 2022-11-03 12:43:19 -07:00
699db03187 feat(db): db error handling and Provider scaffolding (#154)
* feat(db): db error handling and Provider scaffolding
* remove one map_err
* fix after merge
2022-11-03 17:00:58 +01:00
bff27a4154 fix(rpc): type encoding (#155)
* fix(rpc): type encoding

* copy bytes from ethers
2022-11-03 03:04:09 -07:00
becceb2802 fix(db): add traits Encode/Decode for keys and Compress/Uncompress for values (#151)
* add UncompressedUint

* add more UncompressedUint keys

* add docs

* split key/value to encode/decode and compress/uncompress traits

* reveert into

* clippy

* rm whitespaces

* remove TODO

* Remove scale encode/decode traits

* decompress

* clippy

Co-authored-by: rakita <dragan0rakita@gmail.com>
2022-11-03 00:02:19 -07:00
3c72a12aff feat(eth-wire): Implement p2p stream (#114)
* wip: p2pstream

* add comment for handshake timeout

* temp allow some lint violations

* ignore unused_variables

* start of ping task

 * TODO: make it compile
 * TODO: test ping/pong/disconnect state machine
 * TODO: send subprotocol messages to stream
 * TODO: encode non-hello p2p messages as snappy encoding without using
   an encoder
 * TODO: create test comparing encoder to hand-written snappy encoding
   for ping, pong, disconnect messages

* implement message handling in stream poll method

 * restricts S to be Stream+Sink for P2PStream to implement Stream
 * start of a poll-based refactor

* impl Stream and Sink for P2PStream

 * add tests
 * TODO: make stream/sink types compatible
 * TODO: handshake message ids
 * TODO: inner poll fn
 * TODO: pinger interval
 * TODO: ethstream test
 * TODO: passthrough test

* create pingers and test

* impl working timeout interval pinger

 * it should be much easier to poll for pings and detect timeouts now

* use pinger in p2p stream

 * change item produced by stream so it's compatible with EthStream
   * add note on pros/cons
 * shorten message sends in stream

* improve errors and remove redundant methods

* fix handshake

* debugging printlns

* fix encoding and decoding

 * switch to snappy formatting for non-hello p2p messages

* cargo fmt

* perform handshake in ethstream over p2pstream test

 * remove check for `Hello` messages outside of the handshake because
   `P2PStream`s should assume messages sent in the sink are subprotocol
   messages, not `p2p` messages.

* impl From<EthVersion> for CapabilityMessage

* remove printlns

* add total_message method to EthVersion

* decode Hello in handshake

 * disallow protocol versions other than v5

* Integrate snappy and implement message size limits

* document constants, move stream definition

* fix missing hello message id

* implement shared capabilities

 * todo: test shared capabilities
 * todo: determine how / when / why to support multiple capabilities
 * removes obsolete authed and offset fields

* add sink api TODOs

* remove les

 * should add protocols when necessary rather than name unsupported
   protocols

* fix snappy compression length

 * add test for p2pstream over a passthrough codec which tests that
   peers agree on a single shared capability

* fix some clippy lints
2022-11-03 00:01:54 -07:00
c232a72338 test(stage): headers linear downloader (#143)
* headers stage linear test

* cleanup unwind
2022-11-02 17:28:44 -07:00
ac2f3fcd8a feat: Consensus crate and verification functions. (#152)
* wip executor

* wip

* Cleanup added some checks and structure to executor

* adding additional block/header checks

* add basefee calculation and check

* some cleanup

* Sanity check test

* Test for sanity check

* move verification to consensus crate

* cleanup

* Better Error handling
2022-11-02 12:59:51 +01:00
1ea98d40cf chore: remove mdbx c files from github lang stat (#153) 2022-11-01 14:24:03 +01:00
ee61be9d07 chore(db): add docs and doctest to tables (#150)
* add table docs

* add doctest to AccountHistory table

* add docs to placeholder types

* change from outer to docs

* no camelcase type
2022-10-28 08:53:57 -07:00
4a22936898 feat(rlp): add encode for &str (#147)
* feat(rlp): add encode for &str

* add test
2022-10-28 11:54:19 +02:00
30ef6bff21 feat(rlp): add smol-str feature (#146) 2022-10-28 11:54:01 +02:00
580c8eb1e4 chore(sync): db integrity error (#148)
* db integrity error

* rm unused import
2022-10-28 01:29:57 -07:00
ee99c29c8f feat(interfaces): add ShardedKey (#142)
* add ShardedKey

* Update crates/interfaces/src/db/models/sharded_key.rs

Co-authored-by: rakita <rakita@users.noreply.github.com>

Co-authored-by: rakita <rakita@users.noreply.github.com>
2022-10-27 20:46:22 -07:00
6c0e2753dd test: add mock discovery testing (#139) 2022-10-26 14:33:13 +02:00
61b8829bdf feat(sync): headers stage (#126)
* port headers stage

* clean up, utils & tests

* more tests

* unwind cleanup & tests

* remove unused rand crate
2022-10-25 16:50:01 -07:00
ce64fefd78 feat(net): add discv4 crate (#113)
* port kad

* feat: port kad bucket

* feat: add discv4

* chore: rustfmt

* cargo update

* just reuse discv5 table

* test: add rlp tests

* message encoding

* feat: impl codec roundtrip testing

* more work in message handling

* implement ping

* feat: impl commands

* cleanup

* more cleanup

* trim config

* more docs

* feat: implement recursive lookup

* docs

* cleanup config

* feat: implement update stream

* chore: config cleanup

* docs: add crate docs

* feat: more testing

* fix deny

* clarify ring

* docs: more docs

* use discv5 master

* docs: address review and add comments

* update readme

* rustmft

* chore(clippy): make clippy happy
2022-10-25 14:23:24 +02:00
2b67e75c05 fix: use correct hash function (#138) 2022-10-25 05:12:45 -07:00
ee41dfac35 feat(db): set-upChangeSet tables (#135)
* add AccountChangeSet

* add StorageChangeSet

* add tests to TxNumberAddress

* minor fixes
2022-10-25 03:47:45 -07:00
8ac5214fc6 chore(libmbx): fmt, clippy and deny list updated (#134)
* feat(db): Add mdbx-rs apache licenced code 55e234

* feat(db): replace mdbx with reth-mdbx, metadata changes

* chore(db): bump mdbx-sys to 0.12.1

* remove libmdbx from cargo deny

* cargo fmt

* cargo clippy

* one more clippy error
2022-10-25 03:18:51 -07:00
5c55c57c5e feat(db): switch to reth-libmdbx-rs, bump reth-mdbx-sys to 0.12.1 (#133)
* feat(db): Add mdbx-rs apache licenced code 55e234

* feat(db): replace mdbx with reth-mdbx, metadata changes

* chore(db): bump mdbx-sys to 0.12.1
2022-10-25 02:50:02 -07:00
4d2e67e9f5 feat(db): Add mdbx-rs apache licenced code 55e234 (#132) 2022-10-25 11:41:04 +02:00
1fe0affa4b chore(db): comment for cursor, cleanup unneeded fn (#131) 2022-10-25 10:34:30 +02:00
2a8189d7c9 chore(db): Add tests and fuzz target to BlockNumHash (#130)
* add tests and fuzz target for BlockNumHash

* use block 1 for test_block_num_hash

* new clippy stds

* fmt
2022-10-25 01:23:10 -07:00
0998fc3eb7 chore: bump tiny-keccak (#127) 2022-10-24 17:06:25 -07:00
77ea53bb6e chore: reexport hex (#128) 2022-10-24 17:06:12 -07:00
1bbf9d2d82 chore: clarify ring (#129) 2022-10-24 17:05:56 -07:00
3c5b235455 feat(db): Add DupSort PlainStorageState table (#121)
* add dupsort table and test

* use env.update instead

Co-authored-by: Georgios Konstantopoulos <me@gakonst.com>
2022-10-24 11:23:28 -07:00
4eca851429 feat(interface): implicit trait bound for DB cursors (#122)
* feat(interface): implicit trait bound for DB cursors

* test cursor

* walking fixed for RO

* impl for Walker for DupCursor
2022-10-24 11:20:14 -07:00
a3185f9997 docs: add workflow links in readme 2022-10-24 16:37:23 +02:00
e50eb65078 ci: add cargo deny action (#125)
* ci: add `cargo-deny` workflow

* ci: allow advisory failures in deny workflow

* ci: only run deny workflow if `Cargo.lock` changed

* chore: driveby formatting of readme
2022-10-24 16:34:37 +02:00
eb40bd9eaf feat: add cargo deny (#112)
* feat: add cargo deny

* cleanup deny
2022-10-24 06:52:22 -07:00
65247ef203 feat: metrics (#92)
* feat(stages): `stage.progress` metric

* docs: metrics
2022-10-24 15:16:45 +02:00
8bc8bd6820 feat: add IntegerList type Elias-Fano (#111)
* add elias

* docs: should not be empty

* change models doc

* add arbitrary feature

* add simple test

* add test-utils to primitives

* cfg arbitrary
2022-10-24 06:14:59 -07:00
6d3ebe5096 ci: run coverage in parallel (#124) 2022-10-24 06:14:38 -07:00
2ace922ef8 chore: make codecov informational for now (#123) 2022-10-24 06:12:20 -07:00
15bd88e637 headers(part2) - feat: add Downloader trait and test utils (#118)
* feat(interfaces): implement header client traits

* feat: add downloader trait implementer

* feat: use explicit error type instead of ok(false)

* feat: add constructor to HeaderLocked

* test: scaffold mock consensus, downloader and headersclient helpers

* test: implement test consensus

* test: implement test headers client

* refactor: cleanup download headers

* chore: fix lint

* s/test_utils/test_helpers

* headers(part 3) feat: implement Linear downloader (#119)

* feat: add headers downloaders crate

* feat: more scaffolding

* interfaces: generalize retryable erros

* feat: implement linear downloader

* fix linear downloader tests & add builder

* extend & reverse

* feat: linear downloader generics behind arc and reversed return order (#120)

* put client & consensus behind arc and return headers in rev

* cleanup

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>

* extract test_utils

* cargo fmt

Co-authored-by: Roman Krasiuk <rokrassyuk@gmail.com>
2022-10-24 04:56:43 -07:00
a4e505132c headers(part1) feat(interfaces): introduce implicit trait bounds (#117)
* feat(ifaces): modify DB GAT to have implicit bounds

https://sabrinajewson.org/blog/the-better-alternative-to-lifetime-gats\#the-better-gats

* test(ifaces): impl new trait for mock

* feat(ifaces): impl db container

we use that downstream to avoid consuming the db transactions
when committing a stage

* fix(test): explicitly define lifetime

* test: ensure can spawn

* test: pls review

* chore: lints/warnings

* feat(db): impl the new gats

* test(db): try to make real db work with gats like in stages

* test(db): make Stage trait take DBContainer

otherwise we cannot call db.commit()
gst

* feat(stages): impl new traits

* chore(db): cleanup tests

* chore: remove unused imports
2022-10-23 23:10:59 -07:00
a123a2b6fd chore: add link to codecov on badge (#116) 2022-10-21 20:11:05 +02:00
8af8c89018 feat: support finalized and safe variants (#115) 2022-10-21 20:02:06 +02:00
5cbbf2b4a3 chore: add coverage badge 2022-10-21 19:59:09 +02:00
2e6b18a154 wip: feat(txpool): add on block update (#69)
* feat(txpool): add on new block handler

* feat(txpool): work on update routine

* feat(txpool): start update process

* feat: implement update loop
2022-10-20 08:34:04 -07:00
483bcdf9ab feat(db): Database Abstraction (#101)
* database tx traits

* wip build passes

* Db and tx results

* nightly and db GAT

* Impl tx, wip cursor

* Move Decode to Table Key, working cursor trait

* wip dupsort

* build all Cursor abstraction

* cleanup

* wip cleanup

* old stages

* codecs moved o interface,stages wip

* resolve db ref, it builds

* Add tx commit after execution

* fmt

* Remove sync send restriction

* Add missing rw cursor functions

* Cleanup, added missing cursor fn. rust toolchain

* fmt

* add nightly to ci

* deny dead_code, remove unwrap

* rm printfn, stages fix, bench fix
2022-10-20 08:32:13 -07:00
e9945b565a chore: reexport sha3 from primitives (#109)
* chore: reexport sha3 from primitives

* feat: add keccak256

* replace sha3 with tinykeccak
2022-10-20 16:43:29 +02:00
4536e09c99 feat(primitives): use with_capacity for sighash (#108) 2022-10-20 10:26:08 +02:00
c277015f5c feat(eth): add remaining eth protocol message types (#82)
* chore: port over remaining types from ethp2p

https://github.com/Rjected/ethp2p/

* replace fastrlp with reth_rlp

* use correct type for tx messages

 * encoding / decoding still todo

* derive Default for AccessList

* export receipts

 * also add Hash to more types

* fix receipts tests

* remove unused receipts test imports

* add convenience methods on transaction

* add block body

* move blockbody to eth-wire, uncomment wire type

* uncomment rest of messages

 * TODO: refactor tests and make tests pass

* use U128 instead of Uint for td

* expose wire types

* use reth_eth_wire instead of ethp2p

* expose Signature

* refactor pooled transaction tests

* fix hash calculation

 * do not hash the entire buffer

* uncomment block test and make clippy happy

* module-level documentation for message types

 * apply a clippy fix

* cargo fmt

* actually make clippy happy

* use H256 instead of [u8; 32]

 * use partition in split_transaction_by_hashes instead of peekable

Co-authored-by: Dan Cline <6798349+Rjected@users.noreply.github.com>
2022-10-19 23:24:50 -04:00
e7851492b1 feat(txpool): enforce size limits (#98)
* feat(txpool): enforce size limits

* chore: rustfmt
2022-10-19 17:01:49 -07:00
0ff0ffa7d5 feat(primitives): transaction encoding tests (#103)
* feat(primitives): transaction encoding tests

* fix encoding and decoding

* remove printlns

* remove obsolete comment

* rename y_parity local variable

* add explicit non-legacy case for signature length
2022-10-19 19:52:26 -04:00
fe84bfb68c ci: coverage (#106)
* ci: update action versions

* ci: add coverage job

* ci: require `CODECOV_TOKEN` secret

* ci: collect coverage for entire workspace

* ci: typo

* ci: dedup

* ci: add concurrency control
2022-10-19 14:46:47 -07:00
4298e3aa87 feat(rpc): more rpc scaffolding (#99)
* feat(rpc): more rpc scaffolding

* use Into<String>

* rm unused error
2022-10-19 22:12:53 +02:00
630baf5d70 feat(db): fuzzing & benchmark (#86)
* fuzz with test-fuzz

* move fuzzing to db/codecs

* add criterion & iai

* print encoded size sum on criterion benchmark

* fix BlockNumHash encode

* add gh action for benchmarks

* don't sum results

* test ci values

* Revert "test ci values"

This reverts commit cc47fd80538b2c0073592f824c2693c927021f8f.

* specify criterion version

* add docs

* remove benchmark job
2022-10-19 13:08:07 -07:00