Operators
Operator Guides
Performance Monitoring & Testing
Prometheus & Grafana
ExploreNYM Scripts

ExploreNym Monitoring Scripts

⚠️

This setup and the scripts included were not written by Nym developers. As always do your own audit before installing any scripts on your machine and familiarize yourself with the security risks involved when opening ports or allowing http access.

Community Monitoring Tools

Individual operators, node families and squads are the foundation of distributed network. There has been a great number of tools coming out of this community some of which can be deployed for the node monitoring setup.

ExploreNYM Tools

Long term involved operator Pawnflake, an author of ExploreNYM (opens in a new tab) explorer, created a monitoring flow, which can be used by other operators called self-hosted-monitor (opens in a new tab). It utilises bash scripts to enable operators setup Prometheus (opens in a new tab) and Grafana (opens in a new tab) together with Node Exporter (opens in a new tab) and Nginx (opens in a new tab) to run their metrics monitoring stack locally.

In collaboration with ExploreNYM we published a step by step guide to set this up.

ExploreNYM also has a network measuring instance called enym-monitor. This setup is very simple for users, however it means that their data are all aggregated into one server such design always brings a risk of centralisation of distributed node's data into one computer.

⚠️

Make sure you understand and properly evaluate what degree of control you give permission to before granting access to your data to any tools running on someone else's servers.

Setup

Minimum requirements of the monitor stack

  • 2 CPU
  • 4 GB RAM
  • 20 GB of free disk space.

SSH to your server as root or become one running sudo -i or su. If you prefer to administrate your VPS from a user environment, supply the commands with prefix sudo.

1. The monitoring part setup

This can be setup on another VPS than the node if desired. We recommend to try to set this up on the same VPS, as your node as we expect the machine to be strong enough to handle the node with enough capacity reserve for monitor.

  • Install git
apt install git
  • Clone the repository to ~/self-hosted-monitor
git clone https://github.com/ExploreNYM/self-hosted-monitor ~/self-hosted-monitor
chmod +x ~/self-hosted-monitor/prometheus.sh && ~/self-hosted-monitor/prometheus.sh
chmod +x ~/self-hosted-monitor/grafana.sh && ~/self-hosted-monitor/grafana.sh
  • Open port 3000 to allow access to Grafana
sudo ufw allow 3000

  • You can now access Grafana at http://<IP_ADDRESS>:3000.

  • Optional step: If you have a registered domain and prefer to use it with https, give permissions to nginx-certbot.sh (opens in a new tab) script and run it to setup Nginx and Certbot

chmod +x ~/self-hosted-monitor/nginx-certbot.sh && ~/self-hosted-monitor/nginx-certbot.sh
chmod +x ~/self-hosted-monitor/prometheus-target.sh && ~/self-hosted-monitor/prometheus-target.sh
2. The target server (the part to be monitored) setup
  • In case you run this part on another VPS: Install git
apt install git
  • In case you run this part on another VPS: Clone the repository to ~/self-hosted-monitor
git clone https://github.com/ExploreNYM/self-hosted-monitor ~/self-hosted-monitor
chmod +x ~/self-hosted-monitor/node-exporter.sh && ~/self-hosted-monitor/node-exporter.sh
3. Grafana dashboard setup

Finally we need to access Grafana dashboards.

  • Open a browser at http://<IP_ADDRESS>:3000 or https://<HOSTNAME> (depends on your setup), enter username admin and password admin and setup new credentials on prompt

  • Setup Data source by opening menu -> Connections -> Data sources -> + Add new data source -> Prometheus

  • In the field Connection next to Prometheus server URL enter http://localhost:9090 (regardless if you accessing Grafana via http or https as this is for internal connection on the server). When you are done in the bottom confirm by Save & Test

  • In the menu open: Dashboards -> + Create dashboard -> Import dashboard

  • ID field: enter 1860 -> Load

  • In Import dashboard page select Prometheus in the bottom and finally Import

Now you have your Prometheus panels displayed via Grafana dashboard for a simple monitoring of your node.

Verification and Troubleshooting

To ensure that your services are running correctly, you can verify that by running systemctl status <SERVICE> or run a journalctl -f -u <SERVICE> to print service logs. It shall return status Active: active (running). For example:

  • To check if Prometheus service is active:
systemctl status prometheus
  • To check if Grafana service is active:
systemctl status grafana-server
  • To check if node-exporter service is active:
systemctl status node_exporter
  • To run journal log:
journalctl -f -u prometheus # or any other service you want to see
Prometheus & GrafanaTroubleshooting VPS Setup