11 Commits

Author SHA1 Message Date
97a71926d7 feat: make RUST_LOG parametrized (#11)
## What 
Customize RUST_LOG variable

## Why 
There're lot of packages used for EN and each of them has their own
logging verbosity

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Documentation comments have been added / updated.
2024-03-20 15:51:04 +02:00
3fb4985ea4 feat: add parameter for set log verbosity and json format (#10)
## What 
Logs in json format and option for set verbosity level for rust 

## Why 
It can be useful for specify verbose level.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Documentation comments have been added / updated.
2024-03-20 14:55:06 +02:00
94db2c0a69 feat!: remove node exporter from monitoring stack (#9)
## What 
Remove node exporter from monitoring

## Why 
Redundant for this context 

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Documentation comments have been added / updated.
2024-03-12 17:06:09 +02:00
26319af6d7 Merge pull request #8 from matter-labs/docs-envs
chore(docs): Better explain chain settings, minor tweaks
2024-03-08 09:24:14 +01:00
37bbe33ad7 minor tweaks 2024-03-08 09:18:55 +01:00
e776080ba4 typo fix 2024-03-08 09:17:00 +01:00
991e7ba587 typo fix 2024-03-08 09:16:24 +01:00
9f7f0bb6cd prettify dump links 2024-03-08 09:15:58 +01:00
ea06fe4c67 chore(docs): Better explain env settings, minor tweaks 2024-03-08 09:13:20 +01:00
254085bea2 Merge pull request #7 from matter-labs/pgtune-docs-fix
chore(docs): Fix duplicated pgtune block in README
2024-03-07 09:51:48 +01:00
b5cb15078b chore(docs): Fix duplicated pgtune block in README 2024-03-07 09:40:57 +01:00
5 changed files with 20 additions and 30 deletions

View File

@ -1,6 +1,6 @@
# ansible-en-role
Ansible role to deploy and configure zkSync Era External Node, including DB instance setup on the same machine, Traefik as reverse proxy, and Prometheus monitoring (PostgreSQL exporter, Node exporter, cAdvisor, Traefik, External Node native metrics, and VictoriaMetrics vmagent to scrape all of them).
Ansible role to deploy and configure zkSync Era External Node, including DB instance setup on the same machine, Traefik as reverse proxy, and Prometheus monitoring (PostgreSQL exporter, cAdvisor, Traefik, External Node native metrics, and VictoriaMetrics vmagent to scrape all of them).
Make sure to configure Prometheus remote write endpoint to send metrics to centralized metrics storage.
@ -26,6 +26,8 @@ l1_chain_id: ""
l2_chain_id: ""
```
Please refer to [External Node docs](https://github.com/matter-labs/zksync-era/tree/main/docs/guides/external-node/prepared_configs) to find values for different zkSync Era chains.
If you want to use monitoring (which we highly recommend), you have to change these variables:
```yaml
@ -54,7 +56,7 @@ iptables_packages:
loadbalancer_ip: "1.2.3.4"
```
In most cases, you'd want to change PostgreSQL parameters (we recommend to use <https://pgtune.leopard.in.ua/> with "Online transaction processing system" preset as sane defaults), so you can do it using `postgres_arguments` variable, eg:
In most cases, you'd want to change PostgreSQL parameters, so you can do it using `postgres_arguments` variable, eg:
```yaml
postgres_arguments:
@ -66,15 +68,17 @@ postgres_arguments:
- -c
```
We recommend using pgtune [online]<https://pgtune.leopard.in.ua/> or [self-hosted](https://github.com/le0pard/pgtune) version with "Online transaction processing system" preset as a good starting point for generating optimal config for your hardware.
We recommend using pgtune [online](https://pgtune.leopard.in.ua/) or [self-hosted](https://github.com/le0pard/pgtune) version with "Online transaction processing system" preset as a good starting point for generating optimal config for your hardware.
If you want to use basic auth for inbound requests, you have to change next variables:
```yaml
# Enable basic auth for external node
enable_basic_auth: true
basic_auth_secret: "htpasswd-generated-secret"
```
Basic auth secret can be generated by `htpasswd` and `sed` for interpolation:
Basic auth secret can be generated by `htpasswd` and `sed` for interpolation:
```echo $(htpasswd -nb <username> <password>) | sed -e s/\\$/\\$\\$/g```
## Step-by-step guide
@ -82,12 +86,17 @@ Basic auth secret can be generated by `htpasswd` and `sed` for interpolation:
1. Install the ansible collection on your machine from where you will run ansible:
`ansible-galaxy collection install community.general`
2. Prepare the latest database backup on your host. you can download it from our [public GCS bucket](https://storage.googleapis.com/zksync-era-mainnet-external-node-backups/external_node_latest.pgdump).
you should place it to `{{ storage_directory }}/pg_backups` directory. By default, `{{ storage_directory }}` is `/usr/src/en`
2. Prepare the latest database backup on your host. you can download it from our public GCS buckets:
3. **OPTIONAL**: If you already have external-node, you can copy tree directory to new host. Copy external-node database tree to `{{ storage_directory }}/db`.
* [Era Mainnet latest dump](https://storage.googleapis.com/zksync-era-mainnet-external-node-backups/external_node_latest.pgdump)
* [Era Sepolia Testnet latest dump](https://storage.googleapis.com/zksync-era-boojnet-external-node-snapshots/external_node_latest.pgdump)
* [Era Goerli Testnet latest dump](https://storage.googleapis.com/zksync-era-testnet-external-node-backups/external_node_latest.pgdump)
**Keep in mind, external-node tree should be older than PostgreSQL database backup.**
Downloaded dump file should be placed into `{{ storage_directory }}/pg_backups` directory (`/usr/src/en/pg_backups` by default)
3. **OPTIONAL**: If you already have running node, you can copy its tree and state directory to a new host's `{{ storage_directory }}/db`. (`/usr/src/en/db` by default)
**Keep in mind that tree and state should be older than PostgreSQL database backup.**
4. Run ansible-playbook using this role. We recommend encrypting next variables with ansible-vault or some another way:

View File

@ -13,7 +13,6 @@ traefik_version: 2.11
postgres_version: 14
external_node_version: 21.0.2
vmagent_version: 1.95.1
node_exporter_version: 1.7.0
cadvisor_version: 0.47.2
postgres_exporter_version: 0.15.0
@ -81,6 +80,7 @@ rpc_http_port: 3060
rpc_ws_port: 3061
healthcheck_port: 3081
metrics_port: 3082
rust_log: zksync_external_node=info,zksync_core=info,zksync_core::sync_layer=info,zksync_server=info,zksync_prover=info,zksync_contract_verifier=info,zksync_dal=info,zksync_eth_client=info,zksync_storage=info,zksync_db_manager=info,zksync_merkle_tree=info,zksync_state=info,zksync_utils=info,zksync_types=info,loadnext=info,dev_ticker=info,vm=info,block_sizes_test=info,zksync_verification_key_generator_and_server=info,zksync_object_store=info,setup_key_generator_and_server=info,zksync_circuit_synthesizer=info,zksync_queued_job_processor=info,zksync_health_check=info
# Monitoring options section
enable_monitoring: false

View File

@ -90,7 +90,8 @@ services:
CHAIN_STATE_KEEPER_VALIDATION_COMPUTATIONAL_GAS_LIMIT: 2000000
DATABASE_POOL_SIZE: 200
EN_MAX_BLOCKS_PER_TREE_BATCH: 200
RUST_LOG: zksync_core=debug,zksync_dal=info,zksync_eth_client=info,zksync_merkle_tree=info,zksync_storage=info,zksync_state=debug,zksync_types=info,vm=info,zksync_external_node=info,zksync_utils=debug
MISC_LOG_FORMAT: json
RUST_LOG: {{ rust_log }}
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:{{ healthcheck_port }}/health" ]
interval: 1m

View File

@ -21,18 +21,6 @@ services:
- "--remoteWrite.vmProtoCompressLevel=2"
restart: always
node-exporter:
image: "prom/node-exporter:v{{ node_exporter_version }}"
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
restart: unless-stopped
command:
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
cadvisor:
image: "gcr.io/cadvisor/cadvisor:v{{ cadvisor_version }}"
volumes:

View File

@ -14,14 +14,6 @@ scrape_configs:
- source_labels: [instance]
target_label: instance
replacement: '{{ node_name | mandatory }}'
- job_name: node-exporter
static_configs:
- targets:
- "node-exporter:9100"
relabel_configs:
- source_labels: [instance]
target_label: instance
replacement: '{{ node_name | mandatory }}'
- job_name: cadvisor
static_configs:
- targets: