Introduction

This is Nym’s Operators guide, containing information and setup guides for the various pieces of Nym Mixnet infrastructure (Mix Node, Gateway and Network Requester) and Nyx blockchain validators.

If you are new to Nym and want to learn about the mixnet, explore kickstart options and demos, learn how to integrate with the network, and follow developer tutorials check out the Developer Portal.

If you want to dive deeper into Nym’s architecture, clients, nodes, and SDK examples visit the technical docs.

Binary Information

Node setup and usage guides:

Maintenance, troubleshooting and FAQ

Last change: 2024-04-11, commit: f978552

Pre-built Binaries

The Github releases page has pre-built binaries which should work on Ubuntu 20.04 and other Debian-based systems, but at this stage cannot be guaranteed to work everywhere.

If the pre-built binaries don’t work or are unavailable for your system, you will need to build the platform yourself.

Setup Binaries

Any syntax in <> brackets is a user’s unique variable. Exchange with a corresponding name without the <> brackets.

Download Binary

  1. Open Github releases page and right click on the binary you want
  2. Select Copy Link
  3. Open your VPS terminal in a directory where you want to download Nym binaries.
  4. Download binary by running wget <BINARY_LINK> where <BINARY_LINK> shall be in your clipboard from point # 2.

Make Executable

  1. Run command:
chmod +x <BINARY>
# for example: chmod +x nym-mixnode

Run Binary

Now you can use your binary, initialise and run your Nym Node. Follow the guide according to the type of your binary.

Node setup and usage guides:

Last change: 2024-04-11, commit: f978552

Building from Source

Nym runs on Mac OS X, Linux, and Windows. All nodes except the Desktop Wallet and NymConnect on Windows should be considered experimental - it works fine if you’re an app developer but isn’t recommended for running nodes.

Building Nym

Nym has two main codebases:

This page details how to build the main Nym platform code. If you want to build and run a validator, go here instead.

Prerequisites

  • Debian/Ubuntu: pkg-config, build-essential, libssl-dev, curl, jq, git
apt install pkg-config build-essential libssl-dev curl jq git
  • Arch/Manjaro: base-devel
pacman -S base-devel
  • Mac OS X: pkg-config , brew, openss1, protobuf, curl, git Running the following the script installs Homebrew and the above dependencies:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
  • Rust & cargo >= 1.66

We recommend using the Rust shell script installer. Installing cargo from your package manager (e.g. apt) is not recommended as the packaged versions are usually too old.

If you really don’t want to use the shell script installer, the Rust installation docs contain instructions for many platforms.

Download and build Nym binaries

The following commands will compile binaries into the nym/target/release directory:

rustup update
git clone https://github.com/nymtech/nym.git
cd nym

git reset --hard # in case you made any changes on your branch
git pull # in case you've checked it out before

git checkout master # master branch has the latest release version: `develop` will most likely be incompatible with deployed public networks

cargo build --release # build your binaries with **mainnet** configuration

Quite a bit of stuff gets built. The key working parts are:

The repository also contains Typescript applications which aren’t built in this process. These can be built by following the instructions on their respective docs pages.

You cannot build from GitHub’s .zip or .tar.gz archive files on the releases page - the Nym build scripts automatically include the current git commit hash in the built binary during compilation, so the build will fail if you use the archive code (which isn’t a Git repository). Check the code out from github using git clone instead.

Last change: 2024-04-11, commit: f978552

Node Setup Guides

To setup any type of Nym’s node, start with building Nym’s platform on the machine (VPS) where you want to run the node. Nodes will need to be bond to Nym’s wallet, setup one here.

This section contains setup guides for the following node types:

Last change: 2024-04-11, commit: f978552

Preliminary Steps

The Nym mixnode, gateway and network-requester binaries were built in the building nym section. If you haven’t yet built Nym and want to run the code, go there first.

There are a couple of steps that need completing before starting to set up your mix node, gateway or a network requester:

Wallet preparation

Mainnet

Before you initialise and run your mix node, head to our website and download the Nym wallet for your operating system. If pre-compiled binaries for your operating system aren’t available, you can build the wallet yourself with instructions here.

If you don’t already have one, please create a Nym address using the wallet, and fund it with tokens. The minimum amount required to bond a mix node is 100 NYM, but make sure you have a bit more to account for gas costs.

NYM can be purchased via Bity from the wallet itself with BTC or fiat, and is currently present on several exchanges.

Remember that you can only use Cosmos NYM tokens to bond your mix node. You cannot use ERC20 representations of NYM to run a node.

Sandbox testnet

Make sure to download a wallet and create an account as outlined above. Then head to our token faucet and get some tokens to use to bond it.

VPS Hardware Specs

You will need to rent a VPS to run your node on. One key reason for this is that your node must be able to send TCP data using both IPv4 and IPv6 (as other nodes you talk to may use either protocol).

For the moment, we haven’t put a great amount of effort into optimizing concurrency to increase throughput, so don’t bother provisioning a beastly server with multiple cores. This will change when we get a chance to start doing performance optimizations in a more serious way. Sphinx packet decryption is CPU-bound, so once we optimize, more fast cores will be better.

For now, see the below rough specs:

  • Processors: 2 cores are fine. Get the fastest CPUs you can afford.

For mix node

  • RAM: Memory requirements are very low - typically a mix node may use only a few hundred MB of RAM.
  • Disks: The mixnodes require no disk space beyond a few bytes for the configuration files.

For Gateway

  • RAM: Memory requirements depend on the amount of users your Gateway will be serving at any one time. If you’re just going to be using it yourself, then minimal RAM is fine. If you’re running your Gateway as part of a Service Grant, get something with at least 4GB RAM.
  • Disks: much like the amount of RAM your Gateway could use, the amount of disk space required will vary with the amount of users your Gateway is serving. If you’re running your Gateway as part of a Service Grant, get something with at least 40GB storage.
Last change: 2024-04-11, commit: f978552

Mix Nodes

The Nym Mix Node binary was built in the building nym section. If you haven’t yet built Nym and want to run the code, go there first.

Any syntax in <> brackets is a user’s unique variable. Exchange with a corresponding name without the <> brackets.

Current version

1.1.35

The nym-mix node binary is currently one point version ahead of the rest of the platform binaries due to a patch applied between releases.

Preliminary steps

Make sure you do the preparation listed in the preliminary steps page before setting up your Mix Node.

Mix node setup

Now that you have built the codebase, set up your wallet, and have a VPS with the nym-mix node binary, you can set up your Mix Node with the instructions below.

To begin, move to /target/release directory from which you run the node commands:

cd target/release

Viewing command help

You can check that your binaries are properly compiled with:

./nym-mixnode --help

Which should return a list of all available commands.

Console output

Implementation of a Loopix-based Mixnode

Usage: nym-mixnode [OPTIONS] <COMMAND>

Commands:
  describe           Describe your mixnode and tell people why they should delegate state to you
  init               Initialise the mixnode
  run                Starts the mixnode
  sign               Sign text to prove ownership of this mixnode
  node-details       Show details of this mixnode
  build-info         Show build information of this binary
  completions        Generate shell completions
  generate-fig-spec  Generate Fig specification
  help               Print this message or the help of the given subcommand(s)

Options:
  -c, --config-env-file <CONFIG_ENV_FILE>  Path pointing to an env file that configures the mixnode
      --no-banner                          Flag used for disabling the printed banner in tty
  -h, --help                               Print help
  -V, --version                            Print version

You can also check the various arguments required for individual commands with:

./nym-mixnode <COMMAND> --help

Adding --no-banner startup flag will prevent Nym banner being printed even if run in tty environment.

Initialising your Mix Node

To check available configuration options for initializing your node use:

./nym-mixnode init --help

Console output

Initialise the mixnode

Usage: nym-mixnode init [OPTIONS]

Options:
      --id <ID>  Id of the mixnode we want to create config for
  -h, --help     Print help

Initialise your Mix Node with the following command, replacing the value of --id with the moniker you wish to give your Mix Node. Your --host must be publicly routable on the internet in order to mix packets, and can be either an Ipv4 or IPv6 address. The $(curl -4 https://ifconfig.me) command returns your IP automatically using an external service. If you enter your IP address manually, enter it without any port information.

./nym-mixnode init --id <YOUR_ID> --host $(curl -4 https://ifconfig.me) 

If <YOUR_ID> was my-node, the output will look like this:

Console output

The init command will refuse to destroy existing Mix Node keys.

During the init process you will have the option to change the http_api, verloc and mixnode ports from their default settings. If you wish to change these in the future you can edit their values in the config.toml file created by the initialization process, which is located at ~/.nym/mixnodes/<YOUR_ID>/.

Node Description (optional)

In order to easily identify your node via human-readable information later on, you can describe your Mix Node with the following command:

./nym-mixnode describe --id <YOUR_ID>

Node description is a short text that describes your node. It is displayed in the ./nym-mixnode list command and in the ./nym-mixnode node-details --id <YOUR_ID> command. It also shows up in the node explorer to let people know what your node is about and link to your website.

You can set your node description, by creating a file called description.toml and put it in the same directory as your config.toml file (~/.nym/mixnodes/<YOUR_ID>/config/description.toml). The file should look like this example:

name = "Winston Smith"
description = "I am the Sphinx"
link = "https://nymtech.net"
location = "Giza, Egypt"

Remember to restart your nym-mixnode process in order for the new description to be propagated.

Running your Mix Node

Run your Mix Node with:

./nym-mixnode run --id <YOUR_ID>

Have a look at the saved configuration files in $HOME/.nym/mixnodes/ to see more configuration options.

Bonding your Mix Node

Caution

From v1.1.3, if you unbond your Mix Node that means you are leaving the mixnet and you will lose all your delegations (permanently). You can join again with the same identity key, however, you will start with no delegations.

To initialise, run and bond your Mix Node are the minimum steps to do in order for your Mix Node to work. However we recommend to do a few more steps before bonding. These steps will make it easier for you as a node operator on a long run as well as for others to possibly delegate Nym tokens to your Mix Node. These steps are:

You can bond your Mix Node via the Desktop wallet.

  • Open your wallet, and head to the Bond page, then select the node type Mixnode and input your node details. Press Next.

  • Enter the Amount, Operating cost and Profit margin and press Next.

  • You will be asked to run a the sign command with your mixnode - copy and paste the long signature as the value of --contract-msg and run it.

./nym-mixnode sign --id <YOUR_ID> --contract-msg <PAYLOAD_GENERATED_BY_THE_WALLET>

It will look something like this:

Console output

  • Copy the resulting signature:
# >>> The base58-encoded signature is:
2bbDJSmSo9r9qdamTNygY297nQTVRyQaxXURuomVcRd7EvG9oEC8uW8fvZZYnDeeC9iWyG9mAbX2K8rWEAxZBro1
  • And paste it into the wallet nodal, press Next and confirm the transaction.

Paste Signature
This image is just an example, copy-paste your own base58-encoded signature

  • Your node will now be bonded and ready to mix at the beginning of the next epoch (at most 1 hour).

You are asked to sign a transaction on bonding so that the Mixnet smart contract is able to map your nym address to your node. This allows us to create a nonce for each account and defend against replay attacks.

If everything worked, you’ll see your node running on the either the Sandbox testnet network explorer or the mainnet network explorer, depending on which network you’re running.

Note that your node’s public identity key is displayed during startup, you can use it to identify your node in the list.

Bond via the CLI (power users)

If you want to bond your Mix Node via the CLI, then check out the relevant section in the Nym CLI docs.

Node Families

Node family involves setting up a group of Mix Nodes that work together to provide greater privacy and security for network communications. This is achieved by having the nodes in the family share information and routes, creating a decentralized network that makes it difficult for third parties to monitor or track communication traffic.

Create a Node Family

To create a Node family, you will need to install and configure multiple Mix Nodes, and then use the CLI to link them together into a family. Once your Node family is up and running, you can use it to route your network traffic through a series of nodes, obscuring the original source and destination of the communication.

You can use either nym-cli which can be downloaded from the release page or compiling nyxd.

Change directory by cd <PATH>/<TO>/<THE>/<RELEASE> and run the following on the family head to obtain the signature for the member:

./nym-mixnode sign --id <YOUR_ID> --text <TEXT>

Console output

Using nym-cli:

--mnemonic is the mnemonic of the member wanting to be the head of family.

/nym-cli cosmwasm execute <WALLET_ADDRESS> '{"create_family": {"signature": "<base58-encoded-signature>","family_head": "<TEXT>","owner_signature":"<NODE_OWNER_SIGNATURE>","label": "<NODE_LABEL>"}}' --mnemonic <MNEMONIC_FROM_THE_NODE_TO_THE_HEAD>

Using nyxd:

--from is mnemonic of the member wanting to join the family.

./nyxd tx wasm execute ${MIXNET-CONTRACT} '{"join_family": {"signature": "<base58-encoded-signature>","family_head": "<TEXT>"}}' --node ${VALIDATOR-ENDPOINT} --from mix1 --chain-id nyx --gas-prices 0.025unym --gas auto --gas-adjustment 1.3 -y -b block

To get the node owner signature, use:

./nym-mixnode node-details --id <NODE_ID>

Joining a Node Family

Change directory by cd <PATH>/<TO>/<THE>/<RELEASE> and run the following on the family head to obtain the signature for the member:

./nym-mixnode sign --id <YOUR_ID> --text <TEXT>

Console output

Using nym-cli:

./nym-cli cosmwasm execute <WALLET_ADDRESS> '{"join_family": {"signature": "<base58-encoded-signature>","family_head": "<TEXT>","owner_signautre": "<OWNER_SIGNATURE_FROM_NODE_TO_JOIN>", "label":"<NODE_TO_JOIN_LABEL>"}}'  --mnemonic <MNEMONIC_FROM_NODE_TO_JOIN>

Using nyxd:

./nyxd tx wasm execute ${MIXNET-CONTRACT} '{"join_family": {"signature": "<base58-encoded-signature>","family_head": "<TEXT>"}}' --node ${VALIDATOR-ENDPOINT} --from mix1 --chain-id nyx --gas-prices 0.025unym --gas auto --gas-adjustment 1.3 -y -b block

To get the node owner signature, use:

./nym-mixnode node-details --id <NODE_ID>

Leaving a family

If wanting to leave, run the same initial command as above, followed by:

Using nym-cli:

./nym-cli cosmwasm execute <WALLET_ADDRESS> '{"leave_family": {"signature": "<base58-encoded-signature>","family_head": "<TEXT>","owner_signautre": "<OWNER_IGNATURE_FROM_NODE_TO_LEAVE>"}}'  --mnemonic <MNEMONIC_FROM_NODE_TO_LEAVE>

Using nyxd:

./nyxd tx wasm execute ${MIXNET-CONTRACT} '{"join_family": {"signature": "<base58-encoded-signature>","family_head": "<TEXT>"}}' --node ${VALIDATOR-ENDPOINT} --from mix1 --chain-id nyx --gas-prices 0.025unym --gas auto --gas-adjustment 1.3 -y -b block

Checking that your node is mixing correctly

Network explorers

Once you’ve started your Mix Node and it connects to the validator, your node will automatically show up in the ‘Mix Nodes’ section of either the Nym Network Explorers:

Enter your identity key to find your node. There are numerous statistics about your node on that page that are useful for checking your up-time history, packets mixed, and any delegations your node may have.

There are also 2 community explorers which have been created by Nodes Guru:

For more details see Troubleshooting FAQ

Maintenance

For Mix Node upgrade, firewall setup, port configuration, API endpoints, VPS suggestions, automation and more, see the maintenance page

Last change: 2024-04-11, commit: f978552

Gateways

The Nym gateway was built in the building nym section. If you haven’t yet built Nym and want to run the code, go there first.

Info

As a result of Project Smoosh, the current version of nym-gateway binary also contains nym-network-requester functionality which can be enabled by the operator. This combination is a basis of Nym Exit Gateway node - an essential piece in our new setup. Please read more in our Project Smoosh FAQ and Exit Gateway pages. We recommend operators begin to shift their setups to this new combined node, instead of operating two separate binaries.

Any syntax in <> brackets is a user’s unique variable. Exchange with a corresponding name without the <> brackets.

Current version

1.1.33

Preliminary steps

Make sure you do the preparation listed in the preliminary steps page before setting up your Gateway.

Gateway setup

Now that you have built the codebase, set up your wallet, and have a VPS with the nym-gateway binary, you can set up your gateway with the instructions below.

To begin, move to /target/release directory from which you run the node commands:

cd target/release

Viewing command help

You can check that your binaries are properly compiled with:

./nym-gateway --help

Console output

Implementation of the Nym Mixnet Gateway

Usage: nym-gateway [OPTIONS] <COMMAND>

Commands:
  init                     Initialise the gateway
  node-details             Show details of this gateway
  run                      Starts the gateway
  setup-network-requester  Add network requester support to this gateway
  sign                     Sign text to prove ownership of this mixnode
  build-info               Show build information of this binary
  completions              Generate shell completions
  generate-fig-spec        Generate Fig specification
  help                     Print this message or the help of the given subcommand(s)

Options:
  -c, --config-env-file <CONFIG_ENV_FILE>  Path pointing to an env file that configures the gateway
      --no-banner                          Flag used for disabling the printed banner in tty
  -h, --help                               Print help
  -V, --version                            Print version

You can also check the various arguments required for individual commands with:

./nym-gateway <COMMAND> --help

Adding --no-banner startup flag will prevent Nym banner being printed even if run in tty environment.

Initialising your Gateway

As Nym developers build towards Exit Gateway functionality, operators can now run their nym-gateway binary with inbuilt Network Requester and include the our new exit policy. Considering the plan to smoosh all the nodes into one binary and have wide opened Exit Gateways, we recommend this setup, instead of operating two separate binaries.

Warning

Before you start an Exit Gateway, read our Operators Legal Forum page and Project Smoosh FAQ.

Info

There has been an ongoing development with dynamic upgrades. Follow the status of the Project Smoosh changes and the progression state of exit policy implementation to be up to date with the current design.

Note: Due to the development towards Exit Gateway functionality the --host flag has been replaced with --listening-address, this is the IP address which is used for receiving sphinx packets and listening to client data. Another flag --public-ips is required; it’s a comma separated list of IP’s that are announced to the nym-api, it is usually the address which is used for bonding.

Initialising Exit Gateway

An operator can initialise the Exit Gateway functionality by adding Network Requester with the new exit policy option:

./nym-gateway init --id <ID> --listening-address 0.0.0.0 --public-ips "$(curl -4 https://ifconfig.me)" --with-network-requester --with-exit-policy true

If we follow the previous example with <ID> chosen superexitgateway, adding the --with-network-requester and --with-exit-policy flags, the outcome will be:

Console output

You can see that the printed information besides identity and sphinx keys also includes a long string called address. This is the address to be provided to your local socks5 client as a --provider if you wish to connect to your own Exit Gateway.

Additionally

Add Network Requester to an existing Gateway

If you already upgraded your Gateway to the latest version and initialised without a Network Requester, you can easily change its functionality to Exit Gateway with a command setup-network-requester.

See the options:

./nym-gateway setup-network-requester --help

Console output

Add network requester support to this gateway

Usage: nym-gateway setup-network-requester [OPTIONS] --id <ID>

Options:
      --id <ID>
          The id of the gateway you want to initialise local network requester for
      --custom-config-path <CUSTOM_CONFIG_PATH>
          Path to custom location for network requester's config
      --enabled <ENABLED>
          Specify whether the network requester should be enabled [possible values: true, false]
      --open-proxy <OPEN_PROXY>
          Specifies whether this network requester should run in 'open-proxy' mode [possible values: true, false]
      --enable-statistics <ENABLE_STATISTICS>
          Enable service anonymized statistics that get sent to a statistics aggregator server [possible values: true, false]
      --statistics-recipient <STATISTICS_RECIPIENT>
          Mixnet client address where a statistics aggregator is running. The default value is a Nym aggregator client
  -o, --output <OUTPUT>
          [default: text] [possible values: text, json]
  -h, --help
          Print help

To setup Exit Gateway functionality with our new exit policy add a flag --with-exit-policy true.

./nym-gateway setup-network-requester --enabled true --with-exit-policy true --id <ID> 

Say we have a Gateway with <ID> as new-gateway, originally initialised and ran without the Exit Gateway functionality. To change the setup, run:

./nym-gateway setup-network-requester --enabled true --with-exit-policy true --id new-gateway

Console output

In case there are any unexpected problems, you can also change it manually by editing the Gateway config file stored in /home/user/.nym/gateways/<ID>/config/config.toml where the line under [network_requester] needs to be edited from false to true.

[network_requester]
# Specifies whether Network Requester service is enabled in this process.
enabled = true

Save, exit and restart your Gateway. Now you are an operator of post-smooshed Exit Gateway.

Enable Nym exit policy to an existing Gateway with Network Requester functionality

In case you already added Network Requester functionality to your Gateway as described above but haven’t enabled the exit policy there is an easy tweak to do so and turn your node into Nym Exit Gateway.

Open the config file stored at .nym/gateways/<ID>/config/network_requester_config.toml and set:

use_deprecated_allow_list = false

Save, exit and restart your Gateway. Now you are an operator of post-smooshed Exit gateway.

Info

All information about Network Requester part of your Exit Gateway is in /home/user/.nym/gateways/<ID>/config/network_requester_config.toml.

For now you can run Gateway without Network Requester or with and without the new exit policy. This will soon change as we inform in our Project Smoosh FAQ.

To read more about the configuration like whitelisted outbound requesters in allowed.list and other useful information, see the page Network Requester whitelist.

Initialising Gateway without Network Requester

In case you don’t want to run your Gateway with the Exit Gateway functionality, you still can run a simple Gateway.

To check available configuration options use:

 ./nym-gateway init --help

Console output

Initialise the gateway

Usage: nym-gateway init [OPTIONS]

Options:
      --id <ID>  Id of the gateway we want to create config for
  -h, --help     Print help

The following command returns a Gateway on your current IP with the <ID> of simple-gateway:

./nym-gateway init --id simple-gateway --listening-address 0.0.0.0 --public-ips "$(curl -4 https://ifconfig.me)"

Console output

The $(curl -4 https://ifconfig.me) command above returns your IP automatically using an external service. Alternatively, you can enter your IP manually if you wish. If you do this, remember to enter your IP without any port information.

Running your Gateway

The run command starts the Gateway:

./nym-gateway run --id <ID>

Bonding your Gateway

Info

Before you bond your Gateway, please make sure the firewall configuration is setup so your Gateway can be reached from the outside. You can also setup WSS on your Gateway and automate your Gateway to simplify the operation overhead. We highly recommend to run any of these steps before bonding to prevent disruption of your Gateway’s routing score later on.

You can bond your Gateway via the Desktop wallet. Make sure your Gateway is running first, then follow the steps below:

  1. Open your wallet, and head to the Bonding page, then select the node type Gateway and input your node details. Press Next.

  2. Enter the Amount, Operating cost and press Next.

  3. You will be asked to run a the sign command with your gateway - copy and paste the long signature <PAYLOAD_GENERATED_BY_THE_WALLET> and paste it as a value of --contract-msg in the following command:

./nym-gateway sign --id <YOUR_ID> --contract-msg <PAYLOAD_GENERATED_BY_THE_WALLET>

It will look something like this (as <YOUR_ID> we used supergateway):

Console output

./nym-gateway sign --id supergateway --contract-msg 2Mf8xYytgEeyJke9LA7TjhHoGQWNBEfgHZtTyy2krFJfGHSiqy7FLgTnauSkQepCZTqKN5Yfi34JQCuog9k6FGA2EjsdpNGAWHZiuUGDipyJ6UksNKRxnFKhYW7ri4MRduyZwbR98y5fQMLAwHne1Tjm9cXYCn8McfigNt77WAYwBk5bRRKmC34BJMmWcAxphcLES2v9RdSR68tkHSpy2C8STfdmAQs3tZg8bJS5Qa8pQdqx14TnfQAPLk3QYCynfUJvgcQTrg29aqCasceGRpKdQ3Tbn81MLXAGAs7JLBbiMEAhCezAr2kEN8kET1q54zXtKz6znTPgeTZoSbP8rzf4k2JKHZYWrHYF9JriXepuZTnyxAKAxvGFPBk8Z6KAQi33NRQkwd7MPyttatHna6kG9x7knffV6ebGzgRBf7NV27LurH8x4L1uUXwm1v1UYCA1WSBQ9Pp2JW69k5v5v7G9gBy8RUcZnMbeL26Qqb8WkuGcmuHhaFfoqSfV7PRHPpPT4M8uRqUyR4bjUtSJJM1yh6QSeZk9BEazzoJqPeYeGoiFDZ3LMj2jesbJweQR4caaYuRczK92UGSSqu9zBKmE45a


      _ __  _   _ _ __ ___
     | '_ \| | | | '_ \ _ \
     | | | | |_| | | | | | |
     |_| |_|\__, |_| |_| |_|
            |___/

             (nym-gateway - version v1.1.<XX>)  


>>> attempting to sign 2Mf8xYytgEeyJke9LA7TjhHoGQWNBEfgHZtTyy2krFJfGHSiqy7FLgTnauSkQepCZTqKN5Yfi34JQCuog9k6FGA2EjsdpNGAWHZiuUGDipyJ6UksNKRxnFKhYW7ri4MRduyZwbR98y5fQMLAwHne1Tjm9cXYCn8McfigNt77WAYwBk5bRRKmC34BJMmWcAxphcLES2v9RdSR68tkHSpy2C8STfdmAQs3tZg8bJS5Qa8pQdqx14TnfQAPLk3QYCynfUJvgcQTrg29aqCasceGRpKdQ3Tbn81MLXAGAs7JLBbiMEAhCezAr2kEN8kET1q54zXtKz6znTPgeTZoSbP8rzf4k2JKHZYWrHYF9JriXepuZTnyxAKAxvGFPBk8Z6KAQi33NRQkwd7MPyttatHna6kG9x7knffV6ebGzgRBf7NV27LurH8x4L1uUXwm1v1UYCA1WSBQ9Pp2JW69k5v5v7G9gBy8RUcZnMbeL26Qqb8WkuGcmuHhaFfoqSfV7PRHPpPT4M8uRqUyR4bjUtSJJM1yh6QSeZk9BEazzoJqPeYeGoiFDZ3LMj2jesbJweQR4caaYuRczK92UGSSqu9zBKmE45a
>>> decoding the message...
>>> message to sign: {"nonce":0,"algorithm":"ed25519","message_type":"gateway-bonding","content":{"sender":"n1ewmme88q22l8syvgshqma02jv0vqrug9zq9dy8","proxy":null,"funds":[{"denom":"unym","amount":"100000000"}],"data":{"gateway":{"host":"62.240.134.189","mix_port":1789,"clients_port":9000,"location":"62.240.134.189","sphinx_key":"FKbuN7mPdoCG9jA3CkAfXxC5X4rHhqeMVtmfRtJ3cFZd","identity_key":"3RoAhR8gEdfBETMjm2vbMFzKddxXDdE9ygBAnJHWqSzD","version":"1.1.13"}}}}
>>> The base58-encoded signature is:
2SPDjLjX4b6XEtkgG7yD8Znsb1xycL1edFvRK4JcVnPsM9k6HXEUUeVS6rswRiYxoj1bMgiRKyPDwiksiuyxu8Xi
  • Copy the resulting signature:
# >>> The base58-encoded signature is:
2SPDjLjX4b6XEtkgG7yD8Znsb1xycL1edFvRK4JcVnPsM9k6HXEUUeVS6rswRiYxoj1bMgiRKyPDwiksiuyxu8Xi
  • And paste it into the wallet nodal, press Next and confirm the transaction.

Paste Signature
This image is just an example, copy-paste your own base58-encoded signature.

  • Your Gateway is now bonded.

You are asked to sign a transaction on bonding so that the Mixnet smart contract is able to map your Nym address to your node. This allows us to create a nonce for each account and defend against replay attacks.

Via the CLI (power users)

If you want to bond your Gateway via the CLI, then check out the relevant section in the Nym CLI docs.

Maintenance

For Gateway upgrade, firewall setup, port configuration, API endpoints, VPS suggestions, automation, WSS setup and more, see the maintenance page

Last change: 2024-04-11, commit: f978552

Network Requesters

Nym Network Requester was built in the building nym section. If you haven’t yet built Nym and want to run the code, go there first.

Info

As a result of Project Smoosh, the current version of nym-gateway binary also contains nym-network-requester functionality which can be enabled by the operator. This combination is a basis of Nym Exit Gateway node - an essential piece in our new setup. Please read more in our Project Smoosh FAQ and Exit Gateways Page. We recommend operators begin to shift their setups to this new combined node, instead of operating two separate binaries.

Any syntax in <> brackets is a user’s unique variable. Exchange with a corresponding name without the <> brackets.

Current version

1.1.33

Preliminary steps

Make sure you do the preparation listed in the preliminary steps page before setting up your Network Requester.

Network Requester Whitelist

If you have access to a server, you can run the Network Requester, which allows Nym users to send outbound requests from their local machine through the Mixnet to a server, which then makes the request on their behalf, shielding them (and their metadata) from clearnet, untrusted and unknown infrastructure, such as email or message client servers.

By default the Network Requester is not an open proxy (although it can be used as one). It uses a file called allowed.list (located in ~/.nym/service-providers/network-requester/<NETWORK-REQUESTER-ID>/) as a whitelist for outbound requests.

Note: If you run Network Requester as a part of the Exit Gateway (suggested setup) the allowed.list will be stored in ~/.nym/gateways/<ID>/data/network-requester-data/allowed.list.

Any request to a URL which is not on this list will be blocked.

On startup, if this file is not present, the requester will grab the default whitelist from Nym’s default list automatically.

This default whitelist is useful for knowing that the majority of Network Requesters are able to support certain apps ‘out of the box’.

Operators of a Network Requester are of course free to edit this file and add the URLs of services they wish to support to it! You can find instructions below on adding your own URLs or IPs to this list.

The domains and IPs on the default whitelist can be broken down by application as follows:

# Keybase
keybaseapi.com
s3.amazonaws.com
amazonaws.com
twitter.com
keybase.io
gist.githubusercontent.com

# Used to for uptime healthcheck (see the section on testing your requester below for more)
nymtech.net

# Blockstream Green Bitcoin Wallet
blockstream.info
blockstream.com
greenaddress.it

# Electrum Bitcoin Wallet
electrum.org

# Helios Ethereum Client
alchemy.com
lightclientdata.org
p2pify.com

# Telegram - these IPs have been copied from https://core.telegram.org/resources/cidr.txt as Telegram does
# not seem to route by domain as the other apps on this list do
91.108.56.0/22
91.108.4.0/22
91.108.8.0/22
91.108.16.0/22
91.108.12.0/22
149.154.160.0/20
91.105.192.0/23
91.108.20.0/22
185.76.151.0/24
2001:b28:f23d::/48
2001:b28:f23f::/48
2001:67c:4e8::/48
2001:b28:f23c::/48
2a0a:f280::/32

# nym matrix server
nymtech.chat

# generic matrix server backends
vector.im
matrix.org

# monero desktop - mainnet
212.83.175.67
212.83.172.165
176.9.0.187
88.198.163.90
95.217.25.101
136.244.105.131
104.238.221.81
66.85.74.134
88.99.173.38
51.79.173.165

# monero desktop - stagenet
162.210.173.150
176.9.0.187
88.99.173.38
51.79.173.165

# alephium 
alephium.org

Network Requester Directory

You can find a list of Network Requesters running the default whitelist in the explorer. This list comprises of the NRs running as infrastructure for NymConnect.

We are currently working on a smart-contract based solution more in line with how Mix Nodes and Gateways announce themselves to the network.

Viewing command help

Info

If you run your Network Requester as a part of your Exit Gateway according to the suggested setup, please skip this part of the page and read about Exit Gateway setup instead.

To begin, move to /target/release directory from which you run the node commands:

cd target/release

The ./nym-network-requester --help command can be used to show a list of available parameters.

Console output

Usage: nym-network-requester [OPTIONS] <COMMAND>

Commands:
  init               Initialize a network-requester. Do this first!
  run                Run the network requester with the provided configuration and optionally override parameters
  sign               Sign to prove ownership of this network requester
  import-credential  Import a pre-generated credential
  list-gateways      List all registered with gateways
  add-gateway        Add new gateway to this client
  switch-gateway     Change the currently active gateway. Note that you must have already registered with the new gateway!
  build-info         Show build information of this binary
  completions        Generate shell completions
  generate-fig-spec  Generate Fig specification
  help               Print this message or the help of the given subcommand(s)

Options:
  -c, --config-env-file <CONFIG_ENV_FILE>  Path pointing to an env file that configures the client
      --no-banner                          Flag used for disabling the printed banner in tty
  -h, --help                               Print help
  -V, --version                            Print version

You can check the required parameters for available commands by running:

./nym-network-requester <COMMAND> --help

Adding --no-banner startup flag will prevent Nym banner being printed even if run in tty environment.

Initializing and running your Network Requester

The Network Requester needs to be initialized before it can be run. This is required for the embedded nym-client to connect successfully to the Mixnet. We want to specify an <ID> using the --id command and give it a value of your choice. The following command will achieve that:

 ./nym-network-requester init --id <YOUR_ID>

In the following we used example.

Console output

Version: 1.1.33
ID: example
Identity key: 9494w4U9mddZY8FMpkZhT37fj6US3zHvD3rqgPzNWmHw
Encryption: HNgEuwHmbsr5NLjCh8TBX6qAuKM3oQZnxcgWSz3ESaqE
Gateway ID: 3ADenfD7dibFMYrpZGRbrnoSxSzpJXtL2U76e9oYT1Lw
Gateway: ws://138.124.183.8:9000/
Registered at: 2024-04-11 12:17:13.221349842 +00:00:00
Address of this network-requester: 9494w4U9mddZY8FMpkZhT37fj6US3zHvD3rqgPzNWmHw.HNgEuwHmbsr5NLjCh8TBX6qAuKM3oQZnxcgWSz3ESaqE@3ADenfD7dibFMYrpZGRbrnoSxSzpJXtL2U76e9oYT1Lw

Now that we have initialized our network-requester, we can start it with the following command:

 ./nym-network-requester run --id <YOUR_ID>

Using your Network Requester

The next thing to do is use your requester, share its address with friends (or whoever you want to help privacy-enhance their app traffic). Is this safe to do? If it was an open proxy, this would be unsafe, because any Nym user could make network requests to any system on the internet.

To make things a bit less stressful for administrators, the Network Requester drops all incoming requests by default. In order for it to make requests, you need to add specific domains to the allowed.list file at $HOME/.nym/service-providers/network-requester/allowed.list or if Network Requester is ran as a part of Exit Gateway, the allowed.list will be stored in ~/.nym/gateways/<ID>/data/network-requester-data/allowed.list

Global vs local allow lists

Your Network Requester will check for a domain against 2 lists before allowing traffic through for a particular domain or IP.

  • The first list is the default list on the nymtech.net server. Your Requester will not check against this list every time, but instead will keep a record of accepted domains in memory.

  • The second is the local allowed.list file.

Supporting custom domains with your Network Requester

It is easy to add new domains and services to your Network Requester - simply find out which endpoints (both URLs and raw IP addresses are supported) you need to whitelist, and then add these endpoints to your allowed.list.

In order to keep things more organised, you can now use comments in the allow.list like the example at the top of this page.

How to go about this? Have a look in your nym-network-requester config directory:

# nym-network-requester binary
ls -lt $HOME/.nym/service-providers/network-requester/*/data | grep "list"

# nym-gateway binary
ls -lt $HOME/.nym/gateways/*/data/network-requester-data | grep "list"

# returns: allowed.list  unknown.list

We already know that allowed.list is what lets requests go through. All unknown requests are logged to unknown.list. If you want to try using a new client type, just start the new application, point it at your local socks client (configured to use your remote nym-network-requester), and keep copying URLs from unknown.list into allowed.list (it may take multiple tries until you get all of them, depending on the complexity of the application). Make sure to delete the copied ones in unknown.list and restart your Exit Gateway or standalone Network Requester.

If you are adding custom domains, please note that whilst they may appear in the logs of your network-requester as something like api-0.core.keybaseapi.com:443, you only need to include the main domain name, in this instance keybaseapi.com

Running an open proxy

If you really want to run an open proxy, perhaps for testing purposes for your own use or among a small group of trusted friends, it is possible to do so. You can disable Network checks by passing the flag --open-proxy flag when you run it. If you run in this configuration, you do so at your own risk.

Testing your Network Requester

  1. Make sure nymtech.net is in your allowed.list (remember to restart your Network Requester).

  2. Ensure that your nym-network-requester is initialized and running.

  3. In another terminal window, run the following:

curl -x socks5h://localhost:1080 https://nymtech.net/.wellknown/connect/healthcheck.json

This command should return the following:

{ "status": "ok" }

Maintenance

For Network Requester upgrade (including an upgrade from <v1.1.9 to >= v1.1.10), firewall setup, port configuration, API endpoints, VPS suggestions, automation and more, see the maintenance page.

Last change: 2024-04-11, commit: f978552

Validators

Nym has two main codebases:

  • the Nym platform, written in Rust. This contains all of our code except for the validators.
  • the Nym validators, written in Go & maintained as a no-modification fork of wasmd

The validator is a Go application which implements it’s functionalities using Cosmos SDK. The underlying state-replication engine is powered by CometBFT, where the consensus mechanism is based on the Tendermint Consensus Algorithm. Finally, a CosmWasm smart contract module controls crucial mixnet functionalities like decentralised directory service, node bonding, and delegated mixnet staking.

We are currently running mainnet with a closed set of reputable validators. To ensure decentralisation of Nyx chain, we are working on a mechanism to onboard new validators to the network. To join the waitlist, please drop an email to validators [at] nymtech.net with details of your setup, experience and any other relevent information

Building your validator

Any syntax in <> brackets is a user’s unique variable. Exchange it with a corresponding name without the <> brackets.

Prerequisites

git, gcc, jq

  • Debian-based systems:
apt install git build-essential jq

# optional additional manual pages can be installed with:
apt-get install manpages-dev
  • Arch-based systems: Install git, gcc and jq with the following:
pacman -S git gcc jq

Go

Go can be installed via the following commands (taken from the Go Download and install page):

# First remove any existing old Go installation and extract the archive you just downloaded into /usr/local:
# You may need to run the command as root or through sudo
rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.10.linux-amd64.tar.gz

# Add /usr/local/go/bin to the PATH environment variable
export PATH=$PATH:/usr/local/go/bin
source $HOME/.profile

Verify Go is installed with:

go version
# Should return something like:
go version go1.20.10 linux/amd64

Download a precompiled validator binary

You can find pre-compiled binaries for Ubuntu 22.04 and 20.04 here.

Manually compiling your validator binary

The codebase for the Nyx validators can be found here.

The validator binary can be compiled by running the following commands:

git clone https://github.com/nymtech/nyxd.git
cd nyxd

# Make sure to check releases for the latest version information
git checkout release/<NYXD_VERSION>

# Build the binaries
make build

At this point, you will have a copy of the nyxd binary in your build/ directory. Test that it’s compiled properly by running:

./build/nyxd

You should see a similar help menu printed to you:

Console output

Nyx Daemon (server)

Usage:
  nyxd [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  config      Create or query an application CLI configuration file
  debug       Tool for helping with debugging your application
  export      Export state to JSON
  genesis     Application's genesis-related subcommands
  help        Help about any command
  init        Initialize private validator, p2p, genesis, and application configuration files
  keys        Manage your application's keys
  prune       Prune app history states by keeping the recent heights and deleting old heights
  query       Querying subcommands
  rollback    rollback cosmos-sdk and tendermint state by one height
  rosetta     spin up a rosetta server
  start       Run the full node
  status      Query remote node for status
  tendermint  Tendermint subcommands
  testnet     subcommands for starting or configuring local testnets
  tx          Transactions subcommands
  version     Print the application binary version information

Flags:
  -h, --help                help for nyxd
      --home string         directory for config and data
      --log_format string   The logging format (json|plain) (default "plain")
      --log_level string    The logging level (trace|debug|info|warn|error|fatal|panic) (default "info")
      --trace               print out full stack trace on errors

Use "nyxd [command] --help" for more information about a command.

Linking nyxd to libwasmvm.so

libwasmvm.so is the wasm virtual machine which is needed to execute smart contracts in v0.26.1. This file is renamed in libwasmvm.x86_64.so in v0.31.1 and above.

If you downloaded your nyxd binary from Github, you will have seen this file when un-tar-ing the .tar.gz file from the releases page.

If you are seeing an error concerning this file when trying to run nyxd, then you need to move the libwasmvm.x86_64.so file to correct location.

Simply cp or mv that file to /lib/x86_64-linux-gnu/ and re-run nyxd.

Adding nyxd to your $PATH

You’ll need to set LD_LIBRARY_PATH in your user’s ~/.bashrc or ~/.zshrc file (depends on the terminal you use), and add that to our path. Replace /home/<USER>/<PATH-TO-NYM>/binaries in the command below to the locations of nyxd and libwasmvm.so and run it. If you have compiled these on the server, they will be in the build/ folder:

NYX_BINARIES=/home/<USER>/<PATH-TO-VALIDATOR>/<BINARY>

# if you are using another shell like zsh replace '.bashrc' with the relevant config file
echo 'export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:'NYX_BINARIES >> ~/.bashrc
echo 'export PATH=$PATH:'${NYX_BINARIES} >> ~/.bashrc
source ~/.bashrc

Test everything worked:

nyxd

This should return the regular help menu:

Console output

Nyx Daemon (server)

Usage:
  nyxd [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  config      Create or query an application CLI configuration file
  debug       Tool for helping with debugging your application
  export      Export state to JSON
  genesis     Application's genesis-related subcommands
  help        Help about any command
  init        Initialize private validator, p2p, genesis, and application configuration files
  keys        Manage your application's keys
  prune       Prune app history states by keeping the recent heights and deleting old heights
  query       Querying subcommands
  rollback    rollback cosmos-sdk and tendermint state by one height
  rosetta     spin up a rosetta server
  start       Run the full node
  status      Query remote node for status
  tendermint  Tendermint subcommands
  testnet     subcommands for starting or configuring local testnets
  tx          Transactions subcommands
  version     Print the application binary version information

Flags:
  -h, --help                help for nyxd
      --home string         directory for config and data
      --log_format string   The logging format (json|plain) (default "plain")
      --log_level string    The logging level (trace|debug|info|warn|error|fatal|panic) (default "info")
      --trace               print out full stack trace on errors

Use "nyxd [command] --help" for more information about a command.

Initialising your validator

Prerequisites:

  • FQDN Domain name
  • IPv4 and IPv6 connectivity

Choose a name for your validator and use it in place of <ID> in the following command:

# Mainnet
nyxd init <ID> --chain-id=nyx

# Sandbox testnet
nyxd init <ID> --chain-id=sandbox

init generates priv_validator_key.json and node_key.json.

If you have already set up a validator on a network, make sure to back up the key located at ~/.nyxd/config/priv_validator_key.json.

If you don’t save the validator key, then it can’t sign blocks and will be jailed all the time, and there is no way to deterministically (re)generate this key.

At this point, you have a new validator, with its own genesis file located at $HOME/.nyxd/config/genesis.json. You will need to replace the contents of that file that with either the Nyx Mainnet or Sandbox Testnet genesis file.

You can use the following command to download them for the correct network:

# Mainnet
wget  -O $HOME/.nyxd/config/genesis.json https://nymtech.net/genesis/genesis.json

# Sandbox testnet
curl https://rpc.sandbox.nymtech.net/snapshots/genesis.json | jq '.result.genesis' > $HOME/.nyxd/config/genesis.json

config.toml configuration

Edit the following config options in $HOME/.nyxd/config/config.toml to match the information below for your network:

# Mainnet
persistent_peers = "ee03a6777fb76a2efd0106c3769daaa064a3fcb5@51.79.21.187:26656"
laddr = "tcp://0.0.0.0:26656"
# Sandbox testnet
cors_allowed_origins = ["*"]
persistent_peers = "26f7782aff699457c8e6dd9a845e5054c9b0707e@:3.72.19.120:26656"
laddr = "tcp://0.0.0.0:26656"

These affect the following:

  • persistent_peers = "<PEER_ADDRESS>@<DOMAIN>.nymtech.net:26666" allows your validator to start pulling blocks from other validators. The main sandbox validator listens on 26666 instead of the default 26656 for debugging. It is recommended you do not change your port from 26656.
  • laddr = "tcp://0.0.0.0:26656" is in your p2p configuration options

Optionally, if you want to enable Prometheus metrics then the following must also match in the config.toml:

  • prometheus = true
  • prometheus_listen_addr = ":26660"

Remember to enable metrics in the ‘Configuring Prometheus metrics’ section below as well.

And if you wish to add a human-readable moniker to your node:

  • moniker = "<YOUR_VALIDATOR_NAME>"

Finally, if you plan on using Cockpit on your server, change the grpc port from 9090 as this is the port used by Cockpit.

app.toml configuration

In the file $HOME/.nyxd/config/app.toml, set the following values:

# Mainnet
minimum-gas-prices = "0.025unym,0.025unyx"
# Sandbox Testnet
minimum-gas-prices = "0.025unym,0.025unyx"

Setting up your validator’s admin user

You’ll need an admin account to be in charge of your validator. Set that up with:

nyxd keys add nyxd-admin

Key Backends

Cosmos SDK offers multiple backends for securing your keys. Please refer to the Cosmos SDK docs on available keyring backends to learn more

While using the default settings, this will add keys for your account to your system’s keychain and log your name, address, public key, and mnemonic. As the instructions say, remember to write down your mnemonic.

You can get the current account address with:

nyxd keys show nyxd-admin -a

Type in your keychain password, not the mnemonic, when asked.

Starting your validator

Everything should now be ready to go. You’ve got the validator set up, all changes made in config.toml and app.toml, the Nym genesis file copied into place (replacing the initial auto-generated one). Now let’s validate the whole setup:

nyxd validate-genesis

If this check passes, you should receive the following output:

File at /path/to/genesis.json is a valid genesis file

If this test did not pass, check that you have replaced the contents of /<PATH-TO>/.nyxd/config/genesis.json with that of the correct genesis file.

Setting up nyxd as full node (non-signing)

Skip this section if you’re planning to run a validator node to join network consensus. To ensure security & maximum availability of validators, do not expose additional services to the Internet

Unlike signing validators, full nodes do not propose / sign blocks. A full node is typically used for indexing blocks produced on the chain and for exposing web interfaces such as RPC, API and gRPC endpoints required for external applications/services to interact with the blockchain.

By default, API server is disabled and RPC/gRPC servers listen to the loopback address only. In a production setup, it is recommended to use a webserver such as Nginx or caddy to proxy requests to the endpoints as required.

To enable Cosmos REST API, you can enable it in $HOME/.nyxd/config/app.toml like :

[api]

# Enable defines if the API server should be enabled. Toggle this to `true`
enable = true

# Swagger defines if swagger documentation should automatically be registered.
# You can also expose swagger documentation by toggling the below configuration to true
swagger = true

For more information on enabling access to various endpoints via Nginx, refer to the example configuration here

Open firewall ports

Before starting the validator, we will need to open the firewall ports:

# if ufw is not already installed:
sudo apt install ufw
sudo ufw enable

# Customise according to your port bindings. This is only for reference
# 26656 : p2p gossip port
# 26660: If prometheus is enabled
# 22 : Default SSH port
sudo ufw allow 26656,26660,22

## !! FOR FULL NODES ONLY !! - exposing Nginx for serving web requests
sudo ufw allow 80,443

# to check everything worked
sudo ufw status

For more information about your validator’s port configuration, check the validator port reference table below. These can be customised in app.toml and config.toml files.

If you are planning to use Cockpit on your validator server then you will have defined a different grpc port in your config.toml above: remember to open this port as well.

Start the validator:

nyxd start

Once your validator starts, it will start requesting blocks from other validators. This may take several hours. Once it’s up to date, you can issue a request to join the validator set with the command below.

Syncing from a snapshot

If you wish to sync from a snapshot on mainnet use Polkachu’s mainnet resources.

If you wish to sync from a snapshot on Sandbox testnet use the below commands, which are a modified version of Polkachu’s excellent resources. These commands assume you are running an OS with apt as the package manager:

# install lz4 if necessary
sudo apt install snapd -y
sudo snap install lz4

# download the snapshot
wget -O nyxd-sandbox-snapshot-data.tar.lz4 https://rpc.sandbox.nymtech.net/snapshots/nyxd-sandbox-snapshot-data.tar.lz4

# reset your validator state
nyxd tendermint unsafe-reset-all

# unpack the snapshot
lz4 -c -d nyxd-sandbox-snapshot-data.tar.lz4 | tar -x -C $HOME/.nyxd

You can then restart nyxd - it should start syncing from a block > 2000000.

Joining Consensus

You can skip this section if you are planning to run a full-node. This step will make your node a signing validator which joins network consensus

Once your validator has synced and you have received tokens, you can join consensus and produce blocks.

# Mainnet
nyxd tx staking create-validator
  --amount=<10000000unyx>
  --pubkey=$(/home/<USER>/<PATH-TO>/nyxd/binaries/nyxd tendermint show-validator)
  --moniker="<YOUR_VALIDATOR_NAME>"
  --chain-id=nyx
  --commission-rate="0.10"
  --commission-max-rate="0.20"
  --commission-max-change-rate="0.01"
  --min-self-delegation="1"
  --gas="auto"
  --gas-adjustment=1.15
  --gas-prices=0.025unyx
  --from=<"KEYRING_NAME">
  --node=https://rpc.nymtech.net:443
# Sandbox Testnet
nyxd tx staking create-validator
  --amount=<10000000unyx>
  --pubkey=$(/home/<USER>/<PATH-TO>/nym/binaries/nyxd tendermint show-validator)
  --moniker="<YOUR_VALIDATOR_NAME>"
  --chain-id=sandbox
  --commission-rate="0.10"
  --commission-max-rate="0.20"
  --commission-max-change-rate="0.01"
  --min-self-delegation="1"
  --gas="auto"
  --gas-adjustment=1.15
  --gas-prices=0.025unyx
  --from=<"KEYRING_NAME">
  --node https://rpc.sandbox.nymtech.net:443

You’ll need Nyx tokens on mainnet / sandbox to perform the above tasks.

If you want to edit some details for your node you will use a command like this:

# Mainnet
nyxd tx staking edit-validator
  --chain-id=nyx
  --moniker="<YOUR_VALIDATOR_NAME>"
  --details="Nyx validator"
  --security-contact="<YOUR_EMAIL>"
  --identity="<YOUR_IDENTITY>"
  --gas="auto"
  --gas-adjustment=1.15
  --gas-prices=0.025unyx
  --from="KEYRING_NAME"
# Sandbox testnet
nyxd tx staking edit-validator
  --chain-id=sandbox
  --moniker="<YOUR_VALIDATOR_NAME>"
  --details="Sandbox testnet validator"
  --security-contact="your email"
  --identity="<YOUR_IDENTITY>"
  --gas="auto"
  --gas-adjustment=1.15
  --gas-prices=0.025unyx
  --from="KEYRING_NAME"

With above command you can specify the gpg key last numbers (as used in keybase) as well as validator details and your email for security contact.

Automating your validator with systemd

You will most likely want to automate your validator restarting if your server reboots. Checkout the maintenance page with a quick tutorial.

Setting the ulimit

Linux machines limit how many open files a user is allowed to have. This is called a ulimit. We need to set it to a higher value than the default 1024. Follow the instructions in the maintenance page to change the ulimit value for validators.

Using your validator

Un-jailing your validator

If your validator gets jailed, you can fix it with the following command:

# Mainnet
nyxd tx slashing unjail
  --broadcast-mode=block
  --from="KEYRING_NAME"
  --chain-id=nyx
  --gas=auto
  --gas-adjustment=1.4
  --gas-prices=0.025unyx
# Sandbox Testnet
nyxd tx slashing unjail
  --broadcast-mode=block
  --from="KEYRING_NAME"
  --chain-id=sandbox
  --gas=auto
  --gas-adjustment=1.4
  --gas-prices=0.025unyx

Upgrading your validator

To upgrade your validator, follow the steps on the maintenance page.

Common reasons for your validator being jailed

Your validator will be jailed if your node:

  • misses x amount of blocks in y interval, where x and y are parameters set by chain governance
  • performs double signing (two conflicting signatures on the same block using the same key)

Double signing is a serious infraction. If a node double signs, all the delegators to the node (including self-delegation) will be slashed by 5%. Additionally, the node will be permanently jailed and removed from consensus (called tombstoning)

One of the most common reason for your validator being jailed is that your validator is out of memory because of bloated syslogs.

Running the command df -H will return the size of the various partitions of your VPS. If the partition with blockchain data is almost full, try pruning the blockchain data or expanding the storage size.

Day 2 operations with your validator

You can check your current balances with:

nyxd query bank balances ${ADDRESS}

For example, on the Sandbox testnet this would return:

balances:
- amount: "919376"
denom: unym
pagination:
next_key: null
total: "0"

You can, of course, stake back the available balance to your validator with the following command.

Remember to save some tokens for gas costs!

# Mainnet
nyxd tx staking delegate VALOPERADDRESS AMOUNTunyx
  --from="KEYRING_NAME"
  --keyring-backend=os
  --chain-id=nyx
  --gas="auto"
  --gas-adjustment=1.15
  --gas-prices=0.025unyx

# Sandbox Testnet
nyxd tx staking delegate VALOPERADDRESS AMOUNTunyx
  --from="KEYRING_NAME"
  --keyring-backend=os
  --chain-id=sandbox
  --gas="auto"
  --gas-adjustment=1.15
  --gas-prices=0.025unyx
Last change: 2024-04-11, commit: f978552

Nym API Setup

Any syntax in <> brackets is a user’s unique variable. Exchange with a corresponding name without the <> brackets.

What is the Nym API?

The Nym API is a binary that will be operated by the Nyx validator set. This binary can be run in several different modes, and has two main bits of functionality:

  • network monitoring (calculating the routing score of Mixnet nodes)
  • generation and validation of zk-nyms, our implementation of the Coconut Selective Disclosure Credential Scheme.

This is important for both the proper decentralisation of the network uptime calculation and, more pressingly, enabling the NymVPN to utilise privacy preserving payments.

The process of enabling these different aspects of the system will take time. At the moment, Nym API operators will only have to run the binary in a minimal ‘caching’ mode in order to get used to maintaining an additional process running alongside a full node.

Warning

It is highly recommended to run nym-api alongside a full node and NOT a validator node, since you will be exposing HTTP port(s) to the Internet. We also observed degradation in p2p and block signing operations when nym-api was run alongside a signing validator.

Rewards

Operators of Nym API will be rewarded for performing the extra work of taking part in credential generation. These rewards will be calculated separately from rewards for block production.

Rewards for credential signing will be calculated hourly, with API operators receiving a proportional amount of the reward pool (333NYM per hour / 237,600 NYM per month), proportional to the percentage of credentials they have signed.

Hardware Requirements

The specification mentioned below is for running a full node alongside the nym-api. It is recommended to run nym-api and a full Nyx node on the same machine for optimum performance.

Bear in mind that credential signing is primarily CPU-bound, so choose the fastest CPU available to you.

Minimum Requirements

HardwareMinimum Specification
CPU8-cores, 2.8GHz base clock speed or higher
RAM16GB DDR4+
Disk500 GiB+ NVMe SSD
HardwareMinimum Specification
CPU16-cores, 2.8GHz base clock speed or higher
RAM32GB DDR4+
Disk1 TiB+ NVMe SSD

Full node configuration

To install a full node from scratch, refer to the validator setup guide and follow the steps outlined there. Additionally, to ensure nym-api works as expected, ensure the configuration is as below:

Ensure transaction index is turned on in your config.toml:

[tx_index]

# Ensure that this is not set to "null". You're free to use any indexer

indexer = "kv"

Ensure pruning settings are manually configured

nym-api needs to check validity of user-submitted transactions (in the past) while issuing credentials and as part of double-spend check. Hence, aggressively pruning data will lead to errors with your nym-api

Make sure your pruning settings are configured as below in app.toml:

pruning = "custom"

# This number is likely to be updated once zk-nym signing goes live
pruning-keep-recent = "750000"
pruning-interval = "100"

The example value of 100 for pruning-interval can be customised as per your requirement.

Credential Generation

Validators that take part in the DKG ceremony will become part of the ‘quorum’ generating and verifying zk-nym credentials. These will initially be used for private proof of payment for NymVPN (see our blogposts here and here for more on this), and in the future will be expanded into more general usecases such as offline ecash.

The DKG ceremony will be used to create a subset of existing validators who run nym-api alongside a Nyx full-node. As outlined above, they will be the ones taking part in the generation and verification of zk-nym credentials. The size of the ‘minimum viable quorum’ is 10 - we are aiming for a larger number than this for the initial quorum in order to have some redundancy in the case of a validator dropping or going offline.

We will be releasing more detailed step-by-step documentation for involved validators nearer to the ceremony itself, but at a high level it will involve:

  • the deployment and initialisation of group and multisig contracts by Nym. Validators that are members of the group contract are the only ones that will be able to take part in the ceremony.
  • the deployment and initialisation of an instance of the DKG contract by Nym.
  • Validators will update their nym-api configs with the address of the deployed contracts. They will also stop running their API instance in caching only mode, instead switching over run with the --enabled-credentials-mode.
  • From the perspective of operators, this is all they have to do. Under the hood, each nym-api instance will then take part in several rounds of key submission, verification, and derivation. This will continue until quorum is acheived.

We will be communicating individually with members of the existing validator set who have expressed interest in joining the quorum concerning the timing and specifics of the ceremony.

Current version

1.1.35

Setup and Usage

Viewing command help

You can check that your binary is properly compiled with:

./nym-api --help

Which should return a list of all available commands.

Console output

Starting nym api...
Usage: nym-api [OPTIONS] <COMMAND>

Commands:
  init        Initialise a Nym Api instance with persistent config.toml file
  run         Run the Nym Api with provided configuration optionally overriding set parameters
  build-info  Show build information of this binary
  help        Print this message or the help of the given subcommand(s)

Options:
  -c, --config-env-file <CONFIG_ENV_FILE>  Path pointing to an env file that configures the Nym API
  -h, --help                               Print help
  -V, --version                            Print version

You can also check the various arguments required for individual commands with:

./nym-api <COMMAND> --help

Initialising your Nym API Instance in caching mode

Initialise your API instance with:

./nym-api init

You can optionally pass a local identifier for this instance with the --id flag. Otherwise the ID of your instance defaults to default.

Enabling credential signing on your Nym API instance

To engage in the Distributed Key Generation (DKG) ceremony, it’s essential to transition your nym-api instance from its default caching mode to the active credential signing mode. This section guides you through the process of enabling credential signing

Generate a new wallet

Begin by generating a new wallet address specifically for your instance to use in credential signing mode. Utilize the nyxd command-line tool with the following command:

nyxd keys add signer

Backup your keys!

It’s critical to securely back up the mnemonic phrase generated during this process. This mnemonic is your key to recovering the wallet in the future, so store it in a secure, offline location.

Fund the address

Next, deposit NYM tokens into the newly created wallet address to ensure it can cover transaction fees incurred during the credential signing process. nym-api will not operate if the wallet’s balance falls below 10 NYM tokens, displaying an error message upon startup.

We recommand beginning with an initial deposit of 100 NYM tokens and monitoring the balance regularly, topping it up as necessary to maintain operational readiness.

Update API configuration

With your new wallet ready and funded, proceed to update your nym-api configuration to enable credential signing:

Update your config.toml located in <HOME_DIR>/.nym/nym-api/foo/config/config.toml as below:

Enable the coconut signer:

[coconut_signer]
# Specifies whether coconut signing protocol is enabled in this process.
enabled = true  # This was previously false

Set your announce address if it is empty. This is the URL you previously configured for your nym-api instance

# This is the address you previously configured for the nym-api
# Not to be confused with the Cosmos REST API URL
announce_address = 'https://nym-api.your.tld/'

Finally, input the mnemonic phrase generated during the wallet creation step into the mnemonic field

mnemonic = '<YOUR_MNEMONIC_HERE>'

After completing these steps, your nym-api instance is configured to participate in credential signing and the DKG ceremony.

Running your Nym API Instance

The API binary currently defaults to running in caching mode. You can run your API with:

./nym-api run --id <YOUR_ID>

By default the API will be trying to query your full node running locally on localhost:26657. If your node is hosted elsewhere, you can specify the RPC location by using the --nyxd-validator flag on run:

./nym-api run --id <YOUR_ID> --nyxd-validator https://rpc-nym.yourcorp.tld:443

You can also change the value of local_validator in the config file found by default in $HOME/.nym/nym-api/<ID>/config/config.toml.

This process is quite noisy, but informative:

Console output

Starting nym api...
 2023-12-12T14:29:55.800Z INFO  rocket::launch > 🔧 Configured for release.
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > address: 127.0.0.1
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > port: 8000
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > workers: 4
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > max blocking threads: 512
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > ident: Rocket
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > IP header: X-Real-IP
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > limits: bytes = 8KiB, data-form = 2MiB, file = 1MiB, form = 32KiB, json = 1MiB, msgpack = 1MiB, string = 8KiB
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > temp dir: /tmp
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > http/2: true
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > keep-alive: 5s
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > tls: disabled
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > shutdown: ctrlc = true, force = true, signals = [SIGTERM], grace = 2s, mercy = 3s
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > log level: critical
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > cli colors: true
 2023-12-12T14:29:55.800Z INFO  rocket::launch    > 📬 Routes:
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_registered_names) GET /v1/names
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnodes) GET /v1/mixnodes
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_gateways) GET /v1/gateways
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_services) GET /v1/services
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /v1/openapi.json
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_full_circulating_supply) GET /v1/circulating-supply
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_current_epoch) GET /v1/epoch/current
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_active_set) GET /v1/mixnodes/active
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnodes_detailed) GET /v1/mixnodes/detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_rewarded_set) GET /v1/mixnodes/rewarded
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_gateways_described) GET /v1/gateways/described
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_interval_reward_params) GET /v1/epoch/reward_params
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_blacklisted_mixnodes) GET /v1/mixnodes/blacklisted
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_blacklisted_gateways) GET /v1/gateways/blacklisted
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_total_supply) GET /v1/circulating-supply/total-supply-value
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_circulating_supply) GET /v1/circulating-supply/circulating-supply-value
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_active_set_detailed) GET /v1/mixnodes/active/detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_rewarded_set_detailed) GET /v1/mixnodes/rewarded/detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /cors/<status>
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/index.css
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/index.html
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/swagger-ui.css
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/oauth2-redirect.html
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/swagger-ui-bundle.js
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/swagger-ui-config.json
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/swagger-initializer.js
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > GET /swagger/swagger-ui-standalone-preset.js
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnodes_detailed) GET /v1/status/mixnodes/detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnode_inclusion_probabilities) GET /v1/status/mixnodes/inclusion_probability
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnode_status) GET /v1/status/mixnode/<mix_id>/status
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_active_set_detailed) GET /v1/status/mixnodes/active/detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_rewarded_set_detailed) GET /v1/status/mixnodes/rewarded/detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnode_stake_saturation) GET /v1/status/mixnode/<mix_id>/stake-saturation
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (get_mixnode_inclusion_probability) GET /v1/status/mixnode/<mix_id>/inclusion-probability
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (network_details) GET /v1/network/details
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (nym_contracts) GET /v1/network/nym-contracts
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > (nym_contracts_detailed) GET /v1/network/nym-contracts-detailed
 2023-12-12T14:29:55.800Z INFO  rocket::launch    > 📡 Fairings:
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > Validator Cache Stage (ignite)
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > Circulating Supply Cache Stage (ignite)
 2023-12-12T14:29:55.800Z INFO  rocket::launch::_ > Shield (liftoff, response, singleton)
 2023-12-12T14:29:55.801Z INFO  rocket::launch::_ > CORS (ignite, request, response)
 2023-12-12T14:29:55.801Z INFO  rocket::launch::_ > Node Status Cache (ignite)
 2023-12-12T14:29:55.801Z INFO  rocket::shield::shield > 🛡️ Shield:
 2023-12-12T14:29:55.801Z INFO  rocket::shield::shield::_ > X-Content-Type-Options: nosniff
 2023-12-12T14:29:55.801Z INFO  rocket::shield::shield::_ > X-Frame-Options: SAMEORIGIN
 2023-12-12T14:29:55.801Z INFO  rocket::shield::shield::_ > Permissions-Policy: interest-cohort=()
 2023-12-12T14:29:55.801Z WARN  rocket::launch            > 🚀 Rocket has launched from http://127.0.0.1:8000
 2023-12-12T14:29:56.375Z INFO  nym_api::nym_contract_cache::cache::refresher > Updating validator cache. There are 888 mixnodes and 105 gateways
 2023-12-12T14:29:56.375Z INFO  nym_api::node_status_api::cache::refresher    > Updating node status cache
 2023-12-12T14:29:57.359Z INFO  nym_api::circulating_supply_api::cache        > Updating circulating supply cache
 2023-12-12T14:29:57.359Z INFO  nym_api::circulating_supply_api::cache        > the mixmining reserve is now 220198535489690unym
 2023-12-12T14:29:57.359Z INFO  nym_api::circulating_supply_api::cache        > the number of tokens still vesting is now 145054386857730unym
 2023-12-12T14:29:57.359Z INFO  nym_api::circulating_supply_api::cache        > the circulating supply is now 634747077652580unym
 2023-12-12T14:30:00.803Z INFO  nym_api::support::caching::refresher          > node-self-described-data-refresher: refreshing cache state
 2023-12-12T14:31:56.290Z INFO  nym_api::nym_contract_cache::cache::refresher > Updating validator cache. There are 888 mixnodes and 105 gateways
 2023-12-12T14:31:56.291Z INFO  nym_api::node_status_api::cache::refresher    > Updating node status cache

Automation

You will most likely want to automate your validator restarting if your server reboots. Checkout the maintenance page for an example service file.

You can also use nymvisor to automatically update the nym-api node. The steps to install Nymvisor can be found here.

Exposing web endpoint using HTTPS

It is recommended to expose the webserver over HTTPS by using a webserver like Nginx. An example configuration for configuring Nginx is listed on the maintenance page. If you’re using a custom solution, ensure to allow requests from anywhere by setting a permissive CORS policy.

For example, it is configured in Nginx using: add_header 'Access-Control-Allow-Origin' '*';

Last change: 2024-04-11, commit: f978552

Maintenance

Useful commands

Adding --no-banner startup flag will prevent Nym banner being printed even if run in tty environment.

build-info

A build-info command prints the build information like commit hash, rust version, binary version just like what command --version does. However, you can also specify an --output=json flag that will format the whole output as a json, making it an order of magnitude easier to parse.

For example ./target/debug/nym-network-requester --no-banner build-info --output json will return:

{"binary_name":"nym-network-requester","build_timestamp":"2023-07-24T15:38:37.00657Z","build_version":"1.1.23","commit_sha":"c70149400206dce24cf20babb1e64f22202672dd","commit_timestamp":"2023-07-24T14:45:45Z","commit_branch":"feature/simplify-cli-parsing","rustc_version":"1.71.0","rustc_channel":"stable","cargo_profile":"debug"}

Run Web Secure Socket (WSS) on Gateway

Now you can run WSS on your Gateway.

WSS on a new Gateway

These steps are for an operator who is setting up a Gateway for the first time and wants to run it with WSS.

  1. New flags will need to be added to the init and run command. The --host option should be replaced with these flags:
  • --listening-address: The IP address which is used for receiving sphinx packets and listening to client data.
  • --public-ips: A comma separated list of IP’s that are announced to the nym-api. In the most cases --public-ips is the address used for bonding.
--listening-address 0.0.0.0 --public-ips "$(curl -4 https://ifconfig.me)"
  • --hostname (optional): This flag is required if the operator wishes to run WSS. It can be something like mainnet-gateway2.nymtech.net.
  1. Make sure to enable all necessary ports on the Gateway:
sudo ufw allow 1789,1790,8000,9000,9001,22/tcp, 9001/tcp

The Gateway will then be accessible on something like: http://85.159.211.99:8080/api/v1/swagger/index.html

Are you seeing something like: this node attempted to announce an invalid public address: 0.0.0.0.?

Please modify [host.public_ips] section of your config file stored as ~/.nym/gateways/<ID>/config/config.toml.

WSS on an existing Gateway

In case you already run a working Gateway and want to add WSS on it, here are the pre-requisites to running WSS on Gateways:

  • You need to use the latest nym-gateway binary version and restart it.
  • That will add the relevant fields to update your config.
  • These two values will be added and need to be amended in your config.toml:
clients_wss_port = 0
hostname = ""

Then you can run this:

port=$1 // in the example below we will use 9001
host=$2 = // this would be a domain name registered for your Gateway for example: mainnet-gateway2.nymtech.net


sed -i "s/clients_wss_port = 0/clients_wss_port = ${port}/" ${HOME}/.nym/gateways/*/config/config.toml
sed -i "s|hostname = ''|hostname = '${host}'|" ${HOME}/.nym/gateways/*/config/config.toml

The following shell script can be run:

#!/bin/bash

if [ "$#" -ne 2 ]; then
    echo "Usage: sudo ./install_run_caddy.sh <host_name> <port_to_run_wss>"
    exit 1
fi

host=$1
port_value=$2

apt install -y debian-keyring debian-archive-keyring apt-transport-https
apt --fix-broken install

curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg

curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list

apt update
apt install caddy

systemctl enable caddy.service

cd /etc/caddy

# check if Caddyfile exists, if it does, remove and insert a new one
if [ -f Caddyfile ]; then
    echo "removing caddyfile inserting a new one"
    rm -f Caddyfile
fi

cat  <<EOF >> Caddyfile
${host}:${port_value} {
	@websockets {
		header Connection *Upgrade*
		header Upgrade websocket
	}
	reverse_proxy @websockets localhost:9000
}
EOF

cat Caddyfile

echo "script completed successfully!"

systemctl restart caddy.service
echo "have a nice day!"
exit 0

Although your Gateway is Now ready to use its wss_port, your server may not be ready - the following commands will allow you to set up a properly configured firewall using ufw:

ufw allow 9001/tcp

Lastly don’t forget to restart your Gateway, now the API will render the WSS details for this Gateway:

Configure your firewall

Although your <NODE> is now ready to receive traffic, your server may not be. The following commands will allow you to set up a firewall using ufw.

# check if you have ufw installed
ufw version

# if it is not installed, install with
sudo apt install ufw -y

# enable ufw
sudo ufw enable

# check the status of the firewall
sudo ufw status

Finally open your <NODE> p2p port, as well as ports for ssh and ports for verloc and measurement pings:

# for Mix Node, Gateway and Network Requester
sudo ufw allow 1789,1790,8000,9000,9001,22/tcp

# in case of setting up WSS on Gateway add:
sudo ufw allow 9001/tcp

# In case of reverse proxy for the Gateway swagger page add:
sudo ufw allow 8080,80/443

# for validator
sudo ufw allow 1317,26656,26660,22,80,443/tcp

Check the status of the firewall:

sudo ufw status

For more information about your node’s port configuration, check the port reference table below.

VPS Setup and Automation

Replace <NODE> variable with nym-mixnode, nym-gateway or nym-network-requester according the node you running on your machine.

Automating your node with nohup, tmux and systemd

Although it’s not totally necessary, it’s useful to have the Mix Node automatically start at system boot time. We recommend to run your remote operation via tmux for easier management and a handy return to your previous session. For full automation, including a failed node auto-restart and ulimit setup, systemd is a good choice.

Do any of these steps and run your automated node before you start bonding process!

nohup

nohup is a command with which your terminal is told to ignore the HUP or ‘hangup’ signal. This will stop the node process ending if you kill your session.

nohup ./<NODE> run --id <YOUR_ID> # where `<YOUR_ID>` is the id you set during the `init` command and <NODE> depends on which node you starting

tmux

One way is to use tmux shell on top of your current VPS terminal. Tmux is a terminal multiplexer, it allows you to create several terminal windows and panes from a single terminal. Processes started in tmux keep running after closing the terminal as long as the given tmux window was not terminated.

Use the following command to get tmux.

PlatformInstall Command
Arch Linuxpacman -S tmux
Debian or Ubuntuapt install tmux
Fedoradnf install tmux
RHEL or CentOSyum install tmux
macOS (using Homebrewbrew install tmux
macOS (using MacPorts)port install tmux
openSUSEzypper install tmux

In case it didn’t work for your distribution, see how to build tmux from version control.

Running tmux

Now you have installed tmux on your VPS, let’s run a Mix Node on tmux, which allows you to detach your terminal and let your <NODE> run on its own on the VPS.

  • Pause your <NODE>
  • Start tmux with the command
tmux
  • The tmux terminal should open in the same working directory, just the layout changed into tmux default layout.
  • Start the <NODE> again with a command:
./<NODE> run --id <YOUR_ID>
  • Now, without closing the tmux window, you can close the whole terminal and the <NODE> (and any other process running in tmux) will stay active.
  • Next time just start your teminal, ssh into the VPS and run the following command to attach back to your previous session:
tmux attach-session
  • To see keybinding options of tmux press ctrl+b and after 1 second ?

systemd

To automate with systemd use this init service file and follow the steps below.

For Mix Node
[Unit]
Description=Nym Mixnode <VERSION>
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=<USER>
LimitNOFILE=65536
ExecStart=/home/<USER>/<PATH>/nym-mixnode run --id <YOUR_ID>
KillSignal=SIGINT
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
  • Put the above file onto your system at /etc/systemd/system/nym-mixnode.service and follow the next steps.
For Gateway
[Unit]
Description=Nym Gateway <VERSION>
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=nym
LimitNOFILE=65536
ExecStart=/home/<USER>/<PATH>/nym-gateway run --id <YOUR_ID>
KillSignal=SIGINT
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
  • Put the above file onto your system at /etc/systemd/system/nym-gateway.service and follow the next steps.
For Network Requester
[Unit]
Description=Nym Network Requester <VERSION>
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=nym # replace this with whatever user you wish
LimitNOFILE=65536
# remember to add the `--enable-statistics` flag if running as part of a service grant and check the path to your nym-network-requester binary
ExecStart=/home/nym/nym-network-requester run --id <YOUR_ID>
KillSignal=SIGINT
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
  • Put the above file onto your system at /etc/systemd/system/nym-network-requester.service and follow the next steps.
For Nymvisor

Since you’re running your node via a Nymvisor instance, as well as creating a Nymvisor .service file, you will also want to stop any previous node automation process you already have running.

[Unit]
Description=Nymvisor <VERSION>
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=nym # replace this with whatever user you wish
LimitNOFILE=65536
ExecStart=/home/<USER>/<PATH>/nymvisor run run --id <NODE_ID>
KillSignal=SIGINT
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target
  • Put the above file onto your system at /etc/systemd/system/nymvisor.service and follow the next steps.

Following steps for Nym nodes running as systemd service

Change the <PATH> in ExecStart to point at your <NODE> binary (nym-mixnode, nym-gateway or nym-network-requester), and the <USER> so it is the user you are running as.

Example: If you have built nym in the $HOME directory on your server, your username is jetpanther, and node <ID> is puma, then the ExecStart line (command) in the script located in /etc/systemd/system/nym-mixnode.service for Nym Mixnode might look like this:

ExecStart=/home/jetpanther/nym/target/release/nym-mixnode run --id puma.

Basically, you want the full /<PATH>/<TO>/nym-mixnode run --id <WHATEVER-YOUR-NODE-ID-IS>. If you are unsure about your /<PATH>/<TO>/<NODE>, then cd to your directory where you run your <NODE> from and run pwd command which returns the full path for you.

Once done, save the script and follow these steps:

systemctl daemon-reload
# to pickup the new unit file

Enable the newly created service:

# for Mix Node
systemctl enable nym-mixnode.service

# for Gateway
systemctl enable nym-gateway.service

# for Network Requester
systemctl enable nym-network-requester.service

# for Nymvisor
systemctl enable nymvisor.service

Start your <NODE> as a systemd service:

# for Mix Node
service nym-mixnode start

# for Gateway
service nym-gateway start

# for Network Requester
service nym-network-requester.service

# for Nymvisor
service nymvisor.service start

This will cause your <NODE> to start at system boot time. If you restart your machine, your <NODE> will come back up automatically.

You can monitor system logs of your node by running:

journalctl -f -u <NODE>.service
# for example journalctl -f -u nym-mixnode.service

Or check a status by running:

systemctl status <NODE>.service
# for example systemctl status nym-mixnode.service

You can also do service <NODE> stop or service <NODE> restart.

Note: if you make any changes to your systemd script after you’ve enabled it, you will need to run:

systemctl daemon-reload

This lets your operating system know it’s ok to reload the service configuration. Then restart your <NODE>.

For Validator

Below is a systemd unit file to place at /etc/systemd/system/nymd.service to automate your validator:

[Unit]
Description=Nyxd
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=<USER>                                                       # change to your user
Type=simple
Environment="LD_LIBRARY_PATH=/home/<USER>/<PATH_TO_NYX_BINARIES>" # change to correct path
ExecStart=/home/<USER>/<PATH_TO_NYX_BINARIES>/nymd start          # change to correct path
Restart=on-failure
RestartSec=30
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target

Proceed to start it with:

systemctl daemon-reload # to pickup the new unit file
systemctl enable nymd   # to enable the service
systemctl start nymd    # to actually start the service
journalctl -f -u nymd # to monitor system logs showing the service start
For Nym API

Below is a systemd unit file to place at /etc/systemd/system/nym-api.service to automate your API instance:

[Unit]
Description=NymAPI
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=<USER>                                                       # change to your user
Type=simple
ExecStart=/home/<USER>/<PATH_TO_BINARY>/nym-api start             # change to correct path
Restart=on-failure
RestartSec=30
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target

Proceed to start it with:

systemctl daemon-reload  # to pickup the new unit file
systemctl enable nym-api # to enable the service
systemctl start nym-api  # to actually start the service
journalctl -f -u nym-api # to monitor system logs showing the service start

Setting the ulimit

Linux machines limit how many open files a user is allowed to have. This is called a ulimit.

ulimit is 1024 by default on most systems. It needs to be set higher, because Mix Nodes make and receive a lot of connections to other nodes.

If you see errors such as:

Failed to accept incoming connection - Os { code: 24, kind: Other, message: "Too many open files" }

This means that the operating system is preventing network connections from being made.

Set the ulimit via systemd service file

Replace <NODE> variable with nym-mixnode, nym-gateway or nym-network-requester according the node you running on your machine.

The ulimit setup is relevant for maintenance of Nym Mix Node only.

Query the ulimit of your <NODE> with:

# for nym-mixnode, nym-gateway and nym-network-requester:
grep -i "open files" /proc/$(ps -A -o pid,cmd|grep <NODE> | grep -v grep |head -n 1 | awk '{print $1}')/limits

# for nyx validator:
grep -i "open files" /proc/$(ps -A -o pid,cmd|grep nymd | grep -v grep |head -n 1 | awk '{print $1}')/limits

You’ll get back the hard and soft limits, which looks something like this:

Max open files            65536                65536                files

If your output is the same as above, your node will not encounter any ulimit related issues.

However if either value is 1024, you must raise the limit via the systemd service file. Add the line:

LimitNOFILE=65536

Reload the daemon:

systemctl daemon-reload

or execute this as root for system-wide setting of ulimit:

echo "DefaultLimitNOFILE=65535" >> /etc/systemd/system.conf

Reboot your machine and restart your node. When it comes back, use:

# for nym-mixnode, nym-gateway and nym-network-requester:
cat /proc/$(pidof <NODE>)/limits | grep "Max open files"

# for validator
cat /proc/$(pidof nym-validator)/limits | grep "Max open files"

Make sure the limit has changed to 65535.

Set the ulimit on non-systemd based distributions

In case you chose tmux option for Mix Node automation, see your ulimit list by running:

ulimit -a

# watch for the output line -n
-n: file descriptors          1024

You can change it either by running a command:

ulimit -u -n 4096

or editing etc/security/conf and add the following lines:

# Example hard limit for max opened files
username        hard nofile 4096

# Example soft limit for max opened files
username        soft nofile 4096

Then reboot your server and restart your Mix Node.

Moving a node

In case of a need to move a node from one machine to another and avoiding to lose the delegation, here are few steps how to do it.

The following examples transfers a Mix Node (in case of other nodes, change the mixnodes in the command for the <NODE> of your desire.

  • Pause your node process.

Assuming both machines are remote VPS.

  • Make sure your ~/.ssh/<YOUR_KEY>.pub is in both of the machines ~/.ssh/authorized_keys file
  • Create a mixnodes folder in the target VPS. Ssh in from your terminal and run:
# in case none of the nym configs was created previously
mkdir ~/.nym

#in case no nym Mix Node was initialized previously
mkdir ~/.nym/mixnodes
  • Move the node data (keys) and config file to the new machine by opening a local terminal (as that one’s ssh key is authorized in both of the machines) and running:
scp -r -3 <SOURCE_USER_NAME>@<SOURCE_HOST_ADDRESS>:~/.nym/mixnodes/<YOUR_ID> <TARGET_USER_NAME>@<TARGET_HOST_ADDRESS>:~/.nym/mixnodes/
  • Re-run init (remember that init doesn’t overwrite existing keys) to generate a config with the new listening address etc.
  • Change the node smart contract info via the wallet interface. Otherwise the keys will point to the old IP address in the smart contract, and the node will not be able to be connected, and it will fail up-time checks.
  • Re-run the node from the new location.

Virtual IPs and hosting via Google & AWS

For true internet decentralization we encourage operators to use diverse VPS providers instead of the largest companies offering such services. If for some reasons you have already running AWS or Google and want to setup a <NODE> there, please read the following.

On some services (AWS, Google, etc) the machine’s available bind address is not the same as the public IP address. In this case, bind --host to the local machine address returned by $(curl -4 https://ifconfig.me), but that may not the public IP address to bond your <NODE> in the wallet.

You can run ifconfig command. For example, on a Google machine, you may see the following output:

ens4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1460
        inet 10.126.5.7  netmask 255.255.255.255  broadcast 0.0.0.0
        ...

The ens4 interface has the IP 10.126.5.7. But this isn’t the public IP of the machine, it’s the IP of the machine on Google’s internal network. Google uses virtual routing, so the public IP of this machine is something else, maybe 36.68.243.18.

To find the right IP configuration, contact your VPS provider for support to find the right public IP and use it to bond your <NODE> with the nym-api via Nym wallet.

On self-hosted machine it’s a bit more tricky. In that case as an operator you must be sure that your ISP allows for public IPv4 and IPv6 and then it may be a bit of playing around to find the right configuration. One way may be to bind your binary with the --host flag to local address 127.0.0.1 and run echo "$(curl -4 https://ifconfig.me)" to get a public address which you use to bond your Mix Node to nym-api via Nym wallet.

It’s up to you as a node operator to ensure that your public and private IPs match up properly.

Nym API (previously ‘Validator API’) endpoints

Numerous API endpoints are documented on the Nym API (previously ‘Validator API’)’s Swagger Documentation. There you can also try out various requests from your browser, and download the response from the API. Swagger will also show you what commands it is running, so that you can run these from an app or from your CLI if you prefer.

Mix Node Reward Estimation API endpoint

The Reward Estimation API endpoint allows Mix Node operators to estimate the rewards they could earn for running a Nym Mix Node with a specific MIX_ID.

The <MIX_ID> can be found in the “Mix ID” column of the Network Explorer.

The endpoint is a particularly common for Mix Node operators as it can provide an estimate of potential earnings based on factors such as the amount of traffic routed through the Mix Node, the quality of the Mix Node’s performance, and the overall demand for Mix Nodes in the network. This information can be useful for Mix Node operators in deciding whether or not to run a Mix Node and in optimizing its operations for maximum profitability.

Using this API endpoint returns information about the Reward Estimation:

/status/mixnode/<MIX_ID>/reward-estimation

Query Response:

    "estimation": {
        "total_node_reward": "942035.916721770541325331",
        "operator": "161666.263307386408152071",
        "delegates": "780369.65341438413317326",
        "operating_cost": "54444.444444444444444443"
    },

The unit of value is measured in uNYM.

  • estimated_total_node_reward - An estimate of the total amount of rewards that a particular Mix Node can expect to receive during the current epoch. This value is calculated by the Nym Validator based on a number of factors, including the current state of the network, the number of Mix Nodes currently active in the network, and the amount of network traffic being processed by the Mix Node.

  • estimated_operator_reward - An estimate of the amount of rewards that a particular Mix Node operator can expect to receive. This value is calculated by the Nym Validator based on a number of factors, including the amount of traffic being processed by the Mix Node, the quality of service provided by the Mix Node, and the operator’s stake in the network.

  • estimated_delegators_reward - An estimate of the amount of rewards that Mix Node delegators can expect to receive individually. This value is calculated by the Nym Validator based on a number of factors, including the amount of traffic being processed by the Mix Node, the quality of service provided by the Mix Node, and the delegator’s stake in the network.

  • estimated_node_profit - An estimate of the profit that a particular Mix node operator can expect to earn. This value is calculated by subtracting the Mix Node operator’s operating_costs from their estimated_operator_reward for the current epoch.

  • estimated_operator_cost - An estimate of the total cost that a particular Mix Node operator can expect to incur for their participation. This value is calculated by the Nym Validator based on a number of factors, including the cost of running a Mix Node, such as server hosting fees, and other expenses associated with operating the Mix Node.

Validator: Installing and configuring nginx for HTTPS

Setup

Nginx is an open source software used for operating high-performance web servers. It allows us to set up reverse proxying on our validator server to improve performance and security.

Install nginx and allow the ‘Nginx Full’ rule in your firewall:

sudo ufw allow 'Nginx Full'

Check nginx is running via systemctl:

systemctl status nginx

Which should return:

● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-04-20 16:08:19 UTC; 3 days ago
     Docs: man:nginx(8)
 Main PID: 2369 (nginx)
    Tasks: 2 (limit: 1153)
   CGroup: /system.slice/nginx.service
           ├─2369 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           └─2380 nginx: worker process

Full Node Configuration

Proxying various full node services through port 80 can then be done by creating a file with the following at /etc/nginx/sites-enabled/nyxd-webrequests.conf:

Setting up a reverse proxy using a webserver such as Nginx allows you to easily configure SSL certificates for the endpoints. When running on mainnet, it is recommended to encrypt all web traffic to your node.

### To expose RPC server
server {
  listen 80;
  listen [::]:80;
  server_name "<rpc.nyx.yourdomain.tld>";

  location / {
    proxy_pass http://127.0.0.1:26657;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }

  location /websocket {
      proxy_pass http://127.0.0.1:26657;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "Upgrade";
      proxy_set_header Host $host;
    }
}

### To expose Cosmos API server
server {
  server_name "<api.nyx.yourdomain.tld>";
  location / {
    proxy_pass http://127.0.0.1:1317;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header Host $http_host;
    proxy_set_header Upgrade websocket;
    proxy_set_header Connection Upgrade;
  }
}

### To expose GRPC endpoint
server {
  server_name "<grpc.nyx.yourdomain.tld>";
  location / {
    grpc_pass 127.0.0.1:9090;
  }
}

nym-api Configuration

### To expose nym-api webserver
server {
  listen 80;
  listen [::]:80;
  server_name "<nym-api.nyx.yourdomain.tld>";
  add_header 'Access-Control-Allow-Origin' '*';

  location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
}

Followed by:

sudo apt install certbot nginx python3
certbot --nginx -m <you@yourdomain.com> --agree-tos

If using a VPS running Ubuntu 20: replace certbot nginx python3 with python3-certbot-nginx

These commands will get you an https encrypted nginx proxy in front of the various endpoints.

Configuring Prometheus metrics (optional)

Configure Prometheus with the following commands (adapted from NodesGuru’s Agoric setup guide):

echo 'export OTEL_EXPORTER_PROMETHEUS_PORT=9464' >> $HOME/.bashrc
source ~/.bashrc
sed -i '/\[telemetry\]/{:a;n;/enabled/s/false/true/;Ta}' $HOME/.nymd/config/app.toml
sed -i "s/prometheus-retention-time = 0/prometheus-retention-time = 60/g" $HOME/.nymd/config/app.toml
sudo ufw allow 9464
echo 'Metrics URL: http://'$(curl -s ifconfig.me)':26660/metrics'

Your validator’s metrics will be available to you at the returned ‘Metrics URL’.

Console output

# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 6.7969e-05
go_gc_duration_seconds{quantile="0.25"} 7.864e-05
go_gc_duration_seconds{quantile="0.5"} 8.4591e-05
go_gc_duration_seconds{quantile="0.75"} 0.000115919
go_gc_duration_seconds{quantile="1"} 0.001137591
go_gc_duration_seconds_sum 0.356555301
go_gc_duration_seconds_count 2448
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 668
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.15.7"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 1.62622216e+08
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 2.09341707264e+11
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 5.612319e+06
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 2.828263344e+09
# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
# TYPE go_memstats_gc_cpu_fraction gauge
go_memstats_gc_cpu_fraction 0.03357798610671518
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 1.3884192e+07

Ports

All <NODE>-specific port configuration can be found in $HOME/.nym/<NODE>/<YOUR_ID>/config/config.toml. If you do edit any port configs, remember to restart your client and node processes.

Mix Node port reference

Default portUse
1789Listen for Mixnet traffic
1790Listen for VerLoc traffic
8000Metrics http API endpoint

Gateway port reference

Default portUse
1789Listen for Mixnet traffic
9000Listen for Client traffic
9001WSS

Network Requester port reference

Default portUse
9000Listen for Client traffic

Validator port reference

All validator-specific port configuration can be found in $HOME/.nymd/config/config.toml. If you do edit any port configs, remember to restart your validator.

Default portUse
1317REST API server endpoint
26656Listen for incoming peer connections
26660Listen for Prometheus connections
Last change: 2024-04-11, commit: f978552

Manual Node Upgrade

The process here is similar for the Mix Node, Gateway and Network Requester binaries. In the following steps we use a placeholder <NODE> in the commands, please change it for the binary name you want to upgrade (e.g.nym-mixnode). Any particularities for the given type of node are included.

Upgrading your node is a two-step process:

  1. Updating the binary and ~/.nym/<NODE>/<YOUR_ID>/config/config.toml on your VPS
  2. Updating the node information in the mixnet smart contract. This is the information that is present on the mixnet explorer.

Step 1: Upgrading your binary

Follow these steps to upgrade your Node binary and update its config file:

  • Pause your node process.
    • if you see the terminal window with your node, press ctrl + c
    • if you run it as systemd service, run: systemctl stop <NODE>.service
  • Replace the existing <NODE> binary with the newest binary (which you can either compile yourself or grab from our releases page).
  • Re-run init with the same values as you used initially for your <NODE> (Mix Node, Gateway) . This will just update the config file, it will not overwrite existing keys.
  • Restart your node process with the new binary:
    • if your node is not automated, just run your <NODE> with ./<NODE> run --id <ID>. Here are exact guidelines for Mix Node and Gateway.
    • if you automated your node with systemd (recommended) run:
systemctl daemon-reload # to pickup the new unit file
systemctl start <NODE>.service
journalctl -f -u <NODE>.service # to monitor log of you node

If these steps are too difficult and you prefer to automate the process, try to setup your flow with Nymvisor.

In case of a Network Requester this is all, the following step is only for Mix Nodes and Gateways.

Step 2: Updating your node information in the smart contract

Follow these steps to update the information about your <NODE> which is publicly available from the nym-api and information displayed on the Mixnet explorer.

You can either do this graphically via the Desktop Wallet, or the CLI.

  • Navigate to the Bonding page and click the Node Settings link in the top right corner:

Bonding page

  • Update the fields in the Node Settings page (usually the field Version is the only one to change) and click Submit changes to the blockchain.

Node Settings Page

Updating node information via the CLI

If you want to bond your <NODE> via the CLI, then check out the relevant section in the Nym CLI docs.

Upgrading Network Requester to >= v1.1.10 from <v1.1.9

In the previous version of the network-requester, users were required to run a nym-client along side it to function. As of v1.1.10, the network-requester now has a nym client embedded into the binary, so it can run standalone.

If you are running an existing Network Requester registered with nym-connect, upgrading requires you move your old keys over to the new Network Requester configuration. We suggest following these instructions carefully to ensure a smooth transition.

Initiate the new Network Requester:

nym-network-requester init --id <YOUR_ID>

Copy the old keys from your client to the network-requester configuration that was created above:

cp -vr ~/.nym/clients/myoldclient/data/* ~/.nym/service-providers/network-requester/<YOUR_ID>/data

Edit the configuration to match what you used on your client. Specifically, edit the configuration file at:

~/.nym/service-providers/network-requester/<YOUR_ID>/config/config.toml

Ensure that the fields gateway_id, gateway_owner, gateway_listener in the new config match those in the old client config at:

~/.nym/clients/myoldclient/config/config.toml

Upgrading your validator

Upgrading from v0.31.1 -> v0.32.0 process is fairly simple. Grab the v0.32.0 release tarball from the nyxd releases page, and untar it. Inside are two files:

  • the new validator (nyxd) v0.32.0
  • the new wasmvm (it depends on your platform, but most common filename is libwasmvm.x86_64.so)

Wait for the upgrade height to be reached and the chain to halt awaiting upgrade, then:

  • copy libwasmvm.x86_64.so to the default LD_LIBRARY_PATH on your system (on Ubuntu 20.04 this is /lib/x86_64-linux-gnu/) replacing your existing file with the same name.
  • swap in your new nyxd binary and restart.

You can also use something like Cosmovisor - grab the relevant information from the current upgrade proposal here.

Note: Cosmovisor will swap the nyxd binary, but you’ll need to already have the libwasmvm.x86_64.so in place.

Common reasons for your validator being jailed

The most common reason for your validator being jailed is that your validator is out of memory because of bloated syslogs.

Running the command df -H will return the size of the various partitions of your VPS.

If the /dev/sda partition is almost full, try pruning some of the .gz syslog archives and restart your validator process.

Last change: 2024-04-11, commit: f978552

Automatic Node Upgrade: Nymvisor Setup and Usage

The Nymvisor binary was built in the building nym section. If you haven’t yet built Nym and want to run the code, go there first. You can build just Nymvisor with cargo build --release --bin nymvisor.

Any syntax in <> brackets is a user’s unique variable. Exchange with a corresponding name without the <> brackets.

What is Nymvisor?

Nymvisor is a process manager for Nym binaries that monitors the Nym release information for any newly released binaries. If it detects any changes, Nymvisor can automatically download the binary, stop the current binary, switch from the old binary to the new one, and finally restart the underlying process with the new binary.

In essence, it tries to mirror the behaviour of Cosmovisor, a tool used by Cosmos blockchain operators for managing/automating chain upgrades. Nymvisor, however, introduces some Nym-specific changes since, for example, upgrade information is obtained from our GitHub releases page instead of (in the case of Cosmos blockchains) governance proposals.

You can use Nymvisor to automate the upgrades of the following binaries:

  • nym-api
  • nym-mixnode
  • nym-gateway
  • nym-network-requester
  • nym-client
  • nym-socks5-client

Warning

Nymvisor is an early and experimental software. Users should use it at their own risk.

Current version

0.1.0

Preliminary steps

You need to have at least one Mixnet node / client / Nym API instance already set up on the same VPS that you wish to run Nymvisor on.

Using Nymvisor presumes your VPS is running an operating system that is compatible with the pre-compiled binaries avaliable on the Github releases page. If you’re not, then until we’re packaging for a greater variety of operating systems, you’re stuck with manually upgrading your node.

Setup and Usage

Viewing command help

You can check that your binaries are properly compiled with:

./nymvisor --help

Which should return a list of all available commands.

Console output

Usage: nymvisor [OPTIONS] <COMMAND>

Commands:
  init               Initialise a nymvisor instance with persistent Config.toml file
  run                Run the associated daemon with the preconfigured settings
  build-info         Show build information of this binary
  daemon-build-info  Show build information of the associated daemon
  add-upgrade        Queues up another upgrade for the associated daemon
  config             Show configuration options being used by this instance of nymvisor
  help               Print this message or the help of the given subcommand(s)

Options:
  -c, --config-env-file <CONFIG_ENV_FILE>
          Path pointing to an env file that configures the nymvisor and overrides any preconfigured values
  -h, --help
          Print help
  -V, --version
          Print version

You can also check the various arguments required for individual commands with:

./nymvisor <COMMAND> --help

Initialising your Nymvisor Instance

This example will use the Mix Node binary as an example - however replacing nym-mixnode with any other supported binary will work the same.

Initialise your Nymvisor instance with the following command. You must initialise Nymvisor with the binary you wish to add upgrades for:

./nymvisor init --daemon-home ~/.nym/<NODE_TYPE>/<NODE_ID> <PATH_TO_NODE_BINARY>

Where the value of --daemon-home might be ~/.nym/mixnodes/my-node and <PATH_TO_NODE_BINARY> might be /home/my_user/nym/target/release/nym-mixnode, or wherever your node binary is located.

Console output

2024-04-11T12:17:13.258984Z  INFO common/config/src/lib.rs:76: saving config file to /home/runner/.nym/nymvisors/instances/nym-mixnode-default/config/config.toml    

id:                                nym-mixnode-default
daemon name:                       nym-mixnode
daemon home:                       /home/runner/.nym/mixnodes/my-node
upstream base upgrade url:         https://nymtech.net/.wellknown/
disable nymvisor logs:             false
CUSTOM upgrade data directory      ""
upstream absolute upgrade url:     ""
allow binaries download:           true
enforce download checksum:         true
restart after upgrade:             true
restart on failure:                false
on failure restart delay:          10s
max startup failures:              10
startup period duration:           2m
shutdown grace period:             10s
CUSTOM backup data directory       ""
UNSAFE skip backups                false

By default this will create config files at ~/.nym/nymvisors/instances/<NODE_TYPE>-default/config/config.toml as shown in the console output above. For config options look at the different --flags available, or the environment variables section below.

Running your Nymvisor Instance

Nymvisor acts as a wrapper around the specified node process - it has to do this in order to be able to pause and restart this process. As such, you need to run your node via Nymvisor!

The interface to the nymvisor run <ARGS> command is quite simple. Any argument passed after the run command will be passed directly to the underlying daemon, for example: nymvisor run run --id my-mixnode will run the $DAEMON_NAME run --id my-mixnode command (where DAEMON_NAME is the name of the binary itself (e.g. nym-api, nym-mixnode, etc.)).

run Nymvisor and start your node via the following command. Make sure to stop any existing node before running this command.

./nymvisor run run --id <NODE_ID>

Console output

2024-04-11T12:17:13.262203Z  INFO tools/nymvisor/src/cli/run.rs:31: starting nymvisor for nym-mixnode
2024-04-11T12:17:13.262748Z  INFO get_build_information: tools/nymvisor/src/daemon/mod.rs:63: attempting to obtain daemon build information self.executable_path="/home/runner/.nym/nymvisors/data/nym-mixnode/current/bin/nym-mixnode"
2024-04-11T12:17:13.265290Z  INFO tools/nymvisor/src/tasks/launcher/mod.rs:115: checking changes in the upgrade plan file...
2024-04-11T12:17:13.267630Z  WARN tools/nymvisor/src/tasks/launcher/mod.rs:174: the daemon has terminated by itself - was it a short lived command?
2024-04-11T12:17:13.267640Z  INFO tools/nymvisor/src/tasks/launcher/mod.rs:176: it finished with the following exit status: exit status: 1
2024-04-11T12:17:13.267645Z  INFO tools/nymvisor/src/tasks/launcher/mod.rs:42: the daemon has run for 5ms 316us 564ns
2024-04-11T12:17:13.267650Z  INFO tools/nymvisor/src/cli/run.rs:57: the daemon has finished execution
2024-04-11T12:17:13.267683Z  INFO tools/nymvisor/src/cli/run.rs:77: Upgrade plan watcher has finished execution
2024-04-11T12:17:13.267688Z  INFO tools/nymvisor/src/cli/run.rs:77: Upstream poller has finished execution

Nymvisor will now manage your node process (for an in-depth overview of this command check the in-depth command information below). It will periodically poll this endpoint (replace nym-mixnode with whatever node you may actually be running via Nymvisor) and check for a new version of the binary it is watching. If this exists, it will then, using the information there:

  • pause your node process
  • grab the new binary (version)
  • verify it against the provided checksum
  • perform a data backup of the existing node
  • replace the old binary with the new one
  • restart the process

And that’s it! Check the maintenance page for information on Nymvisor process maintenance and automation, and you can find more in-depth information about the various aspects of Nymvisor such as what happens under the hood for various commands.

Creating an Adhoc Upgrade

nymvisor add-upgrade <PATH_TO_EXECUTABLE> --upgrade-name=<NAME> --arg1=value1 --arg2=value2 ... can be used to amend an existing upgrade-plan.json by creating new entries or to add an executable to an existing scheduled upgrade so that it would not have to be downloaded.

Generally users will not have to use this command. Situations in which this command might be used are:

  • an adhoc upgrade if e.g. a patched version of a binary was required
  • if a user doesn’t trust the verification process of Nymvisor’s pipeline and wishes to build/verify the binary themselves before using Nymvisor to perform the upgrade
  • if a user isn’t using a currently supported operating system and needs to manually specify a binary they have built themselves

Similarly to init, add-upgrade requires a positional argument specifying a valid path to the upgrade binary. Furthermore, the --upgrade-name keyword argument must be set in order to declare the upgrade name. The remaining arguments are optional. They include:

  • --force - if specified, will allow Nymvisor to overwrite existing upgrade binary / upgrade-info.json files if they already exist
  • --add-binary - indicate that this command should only add binary to an existing scheduled upgrade
  • --now - if specified will force the upgrade to be performed immediately (technically not ‘immediately’ within few seconds). It can’t be specified alongside either --upgrade-time or --upgrade-delay arguments
  • --publish-date - if a new upgrade-info.json file is going to be created, this argument will specify the publish_date metadata field. Otherwise, the current time will be used. The RFC3339 datetime format is expected
  • --upgrade-time - specifies the time at which the provided upgrade will be performed (RFC3339 formatted). If left unset, the upgrade will be performed in 15 minutes. It can’t be specified alongside either --now or --upgrade-delay arguments.
  • --upgrade-delay - specifies delay until the provided upgrade is going to get performed. If let unset, the upgrade will be performed in 15 minutes. It can’t be specified alongside either --upgrade_time or --now arguments.

Config

The output format of nymvisor config can be further configured with --output argument. By default a human-readable text representation is used:

id:                                nym-mixnode-default
daemon name:                       nym-mixnode
daemon home:                       /home/nym/.nym/mixnodes/my-mixnode
upstream base upgrade url:         https://nymtech.net/.wellknown/
disable nymvisor logs:             false
CUSTOM upgrade data directory      ""
upstream absolute upgrade url:     ""
allow binaries download:           true
enforce download checksum:         true
restart after upgrade:             true
restart on failure:                false
on failure restart delay:          10s
max startup failures:              10
startup period duration:           2m
shutdown grace period:             10s
CUSTOM backup data directory       ""
UNSAFE skip backups                false

Adding --output=json would format the same data into JSON which can be more easily parsed programmatically to e.g. pipe the output into jq for further processing.

nymvisor config --output=json

outputs:

{"nymvisor":{"id":"nym-mixnode-default","upstream_base_upgrade_url":"https://nymtech.net/.wellknown/","upstream_polling_rate":"1h","disable_logs":false,"upgrade_data_directory":null},"daemon":{"name":"nym-mixnode","home":"/home/nym/.nym/mixnodes/my-mixnode","absolute_upstream_upgrade_url":null,"allow_binaries_download":true,"enforce_download_checksum":true,"restart_after_upgrade":true,"restart_on_failure":false,"failure_restart_delay":"10s","max_startup_failures":10,"startup_period_duration":"2m","shutdown_grace_period":"10s","backup_data_directory":null,"unsafe_skip_backup":false}}

CLI Overview

Command options are:

  • help, --help, or -h - Output Nymvisor help information and display the available commands.
  • config - Display the current Nymvisor configuration, that means displaying the current configuration file that might have been overridden with environment variables value that Nymvisor is using.
  • init - Generate a config.toml file for this instance of Nymvisor that will use the provided arguments alongside any environmental variables that are set.
  • add-upgrade - Add an upgrade manually to Nymvisor. This command allows you to easily add the binary corresponding to an upgrade or amend the existing upgrade-plan.json whilst creating new upgrade-info.json file.
  • build-info - Output the build information.
  • daemon-build-info - Output the build information of the current binary used by the associated daemon.
  • run - Run the configured binary using the rest of the provided arguments.
  • -V or --version - Output the Nymvisor version

Similarly to other Nym binaries, Nymvisor supports a global --config-env-file or -c flag that allows specifying path to a .env file defining any relevant environmental variables that are going to be applied to any of the Nymvisor commands as described in the Environment section.

For commands that depend on Nymvisor config file (i.e. all but init), the configuration file is loaded as follows:

  • if available, reading $NYMVISOR_CONFIG_PATH
  • otherwise, if $NYMVISOR_ID is set, a default path will be used, i.e. $HOME/.nym/nymvisors/instances/$NYMVISOR_ID/config/config.toml
  • finally, if there’s only a single nymvisor instance instantiated (as defined by directories in $HOME/.nym/nymvisors/instances), that one will be loaded
  • if all of the above fails, an error is returned

Nymvisor attempts to load the file from the derived path. If it fails, it attempts to use one of the older schemas to and upgrade it as it goes, the loaded configuration is then overridden with any value that might have been set in the environment.

Environment Variables

Please note environmental variables take precedence over any arguments passed, i.e. if one were to specify --daemon_home="/foo" and set DAEMON_HOME="bar", the value of "bar" would end up being used.

For any of its commands as described in CLI Overview section, Nymvisor reads its configuration from the following environment variables:

  • NYMVISOR_ID is the human-readable identifier of the particular Nymvisor instance.
  • NYMVISOR_CONFIG_PATH is used to manually override path to the configuration file of the Nymvisor instance.
  • NYMVISOR_UPSTREAM_BASE_UPGRADE_URL (defaults to https://nymtech.net/.wellknown/) is the base url of the upstream source for obtaining upgrade information for the daemon. It will be used fo constructing the full url, i.e. $NYMVISOR_UPSTREAM_BASE_UPGRADE_URL/$DAEMON_NAME/upgrade-info.json.
  • NYMVISOR_UPSTREAM_POLLING_RATE (defaults to 1h) is polling rate the upstream url for upgrade information.
  • NYMVISOR_DISABLE_LOGS (defaults to false). If set to true, this will disable Nymvisor logs (but not the underlying process) completely.
  • NYMVISOR_UPGRADE_DATA_DIRECTORY is the custom directory for upgrade data - binaries and upgrade plans. If not set, the global Nymvisors’ data directory will be used instead.
  • DAEMON_NAME is the name of the binary itself (e.g. nym-api, nym-mixnode, etc.).
  • DAEMON_HOME is the location where the nymvisor/ directory is kept that contains the auxiliary files associated with the underlying daemon instance, such as any backups or current version information, e.g. $HOME/.nym/nym-api/my-nym-api, $HOME/.nym/mixnodes/my-mixnode, etc.
  • DAEMON_ABSOLUTE_UPSTREAM_UPGRADE_URL is the absolute (i.e. the full url) upstream source for upgrade plans for this daemon. The url has to point to an endpoint containing a valid UpgradeInfo json file. If set it takes precedence over NYMVISOR_UPSTREAM_BASE_UPGRADE_URL.
  • DAEMON_ALLOW_BINARIES_DOWNLOAD (defaults to true), if set to true, it will enable auto-downloading of new binaries (as declared by urls in corresponding upgrade-info.json files). For security reasons one might wish to disable it and instead manually provide binaries by either placing them in the appropriate directory or by invoking add-upgrade command.
  • DAEMON_ENFORCE_DOWNLOAD_CHECKSUM (defaults to true), if set to true Nymvisor will require that a checksum is provided in the upgrade plan for the upgrade binary to be downloaded. If disabled, Nymvisor will not require a checksum to be provided, but still check the checksum if one is provided.
  • DAEMON_RESTART_AFTER_UPGRADE (defaults to true), if set to true Nymvisor will restart the subprocess with the same command-line arguments and flags (but with the new binary) after a successful upgrade. Otherwise (false), Nymvisor stops running after an upgrade and requires the system administrator to manually restart it. Note restart is only after the upgrade and does not auto-restart the subprocess after an error occurs. That is controlled via DAEMON_RESTART_ON_FAILURE.
  • DAEMON_RESTART_ON_FAILURE (defaults to true), if set to true, Nymvisor will restart the subprocess with the same command-line arguments and flags if it has terminated with a non-zero exit code.
  • DAEMON_FAILURE_RESTART_DELAY (defaults to 10s), if DAEMON_RESTART_ON_FAILURE is set to true, this will specify a delay between the process shutdown (with a non-zero exit code) and it being restarted.
  • DAEMON_MAX_STARTUP_FAILURES (defaults to 10) if DAEMON_RESTART_ON_FAILURE is set to true, this defines the maximum number of startup failures the subprocess can experience in a quick succession before no further restarts will be attempted and Nymvisor will terminate.
  • DAEMON_STARTUP_PERIOD_DURATION (defaults to 120s) if DAEMON_RESTART_ON_FAILURE is set to true, this defines the length of time during which the subprocess is still considered to be in the startup phase when its failures are going to be counted towards the limit defined in DAEMON_MAX_STARTUP_FAILURES.
  • DAEMON_SHUTDOWN_GRACE_PERIOD (defaults to 10s), specifies the amount of time Nymvisor is willing to wait for the subprocess to undergo graceful shutdown after receiving an interrupt before it sends a kill signal.
  • DAEMON_BACKUP_DATA_DIRECTORY specifies custom backup directory for daemon data. If not set, DAEMON_HOME/nymvisor/backups is used instead.
  • DAEMON_UNSAFE_SKIP_BACKUP (defaults to false), if set to true, all upgrades will be performed directly without performing any backups. Otherwise (false), Nymvisor will back up the contents of DAEMON_HOME before trying the upgrade.

Dir structure

The folder structure of Nymvisor is heavily inspired by Cosmovisor, but with some notable changes to accommodate our binaries having possibly multiple instances due to their different --id flags. The data is spread through three main directories:

  • in a global nymvisors data directory shared by all Nymvisor instances (default: $HOME/.nym/nymvisors/data) that contains daemon upgrade plans, binaries and upgrades histories. It includes a subdirectory for each version of the application (i.e. genesis or upgrades<name>). Within each subdirectory is the application binary (i.e. bin/$DAEMON_NAME), the associated upgrade-info.json and any additional auxiliary files associated with each binary. current is a symbolic link to the currently active directory (i.e. genesis or upgrades/<NAME>)
  • in a home directory of a particular nymvisor instance (e.g. $HOME/.nym/nymvisors/instances/<nymvisor-instance-id>/). It includes subdirectories for its configuration file (i.e. config/config.toml), that preconfigures the instance, and for any additional persistent data that might be added in the future (i.e. data)
  • in a nymvisor data directory inside the home directory of the managed daemon instance (default: $HOME/.nym/$DAEMON_NAME/nymvisor) that contains subdirectories for data backups (i.e. backups/<name>) and current version information (current-version-info.json)

A sample full structure looks as follows:

~/.nym
├── nymvisors
│   ├── instances
│   │   ├── <id1>
│   │   │   ├── config
│   │   │   │   └── config.toml
│   │   │   └── data
│   │   │       └── ...
│   │   └── <id2>
│   │       └── ...
│   └── data
│       ├── nym-api
│       │   ├── current -> genesis or upgrades/<name>
│       │   ├── genesis
│       │   │   ├── bin
│       │   │   │   └── nym-api
│       │   │   └── upgrade-info.json
│       │   ├── upgrades
│       │   │   └── <upgrade-name>
│       │   │       ├── bin
│       │   │       │   └── nym-api
│       │   │       └── upgrade-info.json
│       │   ├── upgrade-history.json
│       │   └── upgrade-plan.json
│       ├── nym-mixnode
│       │   └── ...
│       └── $DAEMON_NAME
│           └── ...
└── nym-api
├── <id1>
│   ├── config
│   │   └── <nym-api-config-data>
│   ├── data
│   │   └── <nym-api-data>
│   └── nymvisor
│       ├── backups
│       │   └── <upgrade-name>
│       │       └── ....
│       └── current-version-info.json
└── <id2>
└── ...

Commands In-Depth

This section outlines what happens under the hood with the following commands:

Init

init does the following:

  • executes the $DAEMON_NAME build-info command on the daemon executable to check its validity and obtain its name
  • creates the required directory structure:
    • $DAEMON_HOME/nymvisor folder if it doesn’t yet exist
    • $DAEMON_BACKUP_DATA_DIRECTORY folder if it doesn’t yet exist
    • $NYMVISOR_UPGRADE_DATA_DIRECTORY folder if it doesn’t yet exist
    • $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/genesis/bin folder if it doesn’t yet exist
    • $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/upgrades folder if it doesn’t yet exist
  • copies the provided executable to $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/genesis/bin/$DAEMON_NAME
  • generates initial $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/genesis/upgrade-info.json file based on the provided binary
  • generates initial $DAEMON_HOME/nymvisor/current-version-info.json file based on the provided binary
  • creates a $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/current symlink pointing to $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/genesis
  • saves the Nymvisor instance’s config file to $NYMVISOR_CONFIG_PATH and creates the full directory structure for the file
  • outputs (to stdout) the full configuration used

nymvisor init is specifically for initializing Nymvisor, and should not be confused with a daemon’s init command - such as nym-mixnode init (e.g. cosmovisor run init).

Run

nymvisor run is a lightweight wrapper around the underlying daemon. It uses only a single thread and spawns three simple tasks:

  • an upstream poller that checks the upstream source (as defined by DAEMON_ABSOLUTE_UPSTREAM_UPGRADE_URL or derived from NYMVISOR_UPSTREAM_BASE_UPGRADE_URL) for any recently released upgrades. It then creates appropriate upgrade-info.json file and updates the upgrade-plan.json
  • a file watcher for the upgrade-plan.json file that can notify the main run loop of upgrades that were added by either the above upstream poller task or via the add-upgrade command,
  • the daemon run loop that:
    • runs the DAEMON_NAME with the provided arguments until:
      • it completes the execution (with any exit code),
      • Nymvisor receives a SIGINT, SIGTERM or SIGQUIT,
      • a new upgrade is scheduled to be performed (by other task watching for changes in upgrade-plan.json and waiting until the upgrade_time,
    • if DAEMON_UNSAFE_SKIP_BACKUP is not set to true, it backups the content of DAEMON_HOME directory,
    • performs the binary upgrade by:
      • creating a temporary, exclusive and non-blocking, upgrade.lock file for the DAEMON_NAME. flock with LOCK_EX | LOCK_NB is used for that purpose. The file is created in case users didn’t read any warnings and attempted to run multiple instances of nymvisor managing the same DAEMON_NAME,
      • downloading the upgrade binary for the runners architecture using one of the urls defined in upgrade-info.json. Note, however, that this is only done if the binary associated with the <UPGRADE-NAME> does not already exist and DAEMON_ALLOW_DOWNLOAD_BINARIES is set to true,
        • if the binary has been downloaded and DAEMON_ENFORCE_DOWNLOAD_CHECKSUM is set to true, the file checksum is verified using the specified algorithm,
      • verifying the upgrade binary - checking if it’s a valid executable with expected build-info. Note that this will also set a+x bits on the file if those permissions have not already been set,
        • removing the queued upgrade from upgrade-plan.json,
        • inserting new upgrade into the upgrade-history.json,
        • updating the current-version-info.json,
        • updating the $NYMVISOR_UPGRADE_DATA_DIRECTORY/$DAEMON_NAME/current symlink to the upgrade directory,
        • removing the upgrade.lock file.
    • the above loop is repeated if either:
      • the daemon has crashed and DAEMON_MAX_STARTUP_FAILURES has not been reached yet,
      • the daemon has successfully been upgraded, DAEMON_RESTART_AFTER_UPGRADE has been set to true and the manual flag on the performed upgrade has been set to false.

Add-Upgrade

nymvisor add-upgrade does the following:

  • executes the $DAEMON_NAME build-info command on the daemon executable to check its validity and obtain its name
  • attempts to load existing upgrade-info.json for the provided <upgrade-name>. If it already exists and neither --force nor --add-binary was specified, Nymvisor will terminate
  • checks if upgrades/<upgrade-name>/$DAEMON_NAME binary already exists. If it does and --force flag was not specified, Nymvisor will terminate the provided upgrade binary is copied to its appropriate location
  • if applicable, new upgrade-info.json is created and written to its appropriate location
  • upgrade-plan.json is updated with the new upgrade details. If there’s an active Nymvisor instance running, this change will be detected to initialise upgrade process
Last change: 2024-04-11, commit: f978552

Performance Testing

To configure your node for a testing event, visit node setup page.

Nym Mixnet has been running on mainnet for quite some time. There is still work to be done in order for the network to meet its full potential - mass adoption of privacy through fully distributed Mixnet.

Nym asks its decentralised community of operators to join a series of performance testing events in order to increase the overall quality of the Mixnet. The main takeaways of such event are:

  1. Understanding of the network behavior under full load
    • How many mixnet client users can all active set entry gateways handle simultaneously?
    • How much sustained IP traffic can a subset of mainnet nodes sustain?
  2. Needed improvements of Nym Node binaries to improve the throughput on mainnet
  3. Measurement of required machine specs
  4. Raw data record
  5. Increase quality of Nym Nodes
  6. Show each operator a way to monitor their nodes in a distributed fashion

Performance Testing Work Flow

  • Nym runs a paralel network environment validator.performance.nymte.ch with a chain ID perf
  • Operators of Nym Nodes (currently nym-mixnode and nym-gateway) join by following easy steps on performance testing web page, including simplified node authentication signature (while keep running their nodes on the mainnet)
  • Once signed in, operators will be asked to swap their binary for the modified version with metrics endpoint to be able to connect their own monitoring system
  • Core node data will be fed to a unique mixnet contract for the perf side chain
  • Nym starts a new API and start packet transition in high load through these nodes in both settings
  • Nym tracks packet flow using Prometheus and Grafana
  • Nym creates a large number of clients to the performance validator network, intensifying the packet traffic
  • Observe and record the metrics - this will probably put more strain on mainnet as well as many nodes spec is minimal

More Information

Last change: 2024-04-11, commit: f978552

Node Setup for Performance Testing Event

To join the Performance testing event node operators need to do proceed with the following tasks:

  1. Sign their node into the testing environment
  2. Configure their node for the test
  3. (Not mandatory) Setup metric monitoring system to observe node performance at any time

Node Configuration

Any syntax in <> brackets is a user’s unique variable/version. Exchange with a corresponding name without the <> brackets.

After you signed your node (or several) into the Performance testing environment, open the machine with (each of) your nodes and follow the steps below to configure your node for the event.

1. Download and upgrade to v2024.2-fast-and-furious-v2 binary

  • Download the binary from Nym release page (use wget or curl and binary download URL, don’t compile from master)
  • Follow the steps to upgrade node on the maintenance page
  • Before you re-start your node, follow the steps below

2. If you run nym-gateway proceed with these steps. If not, go to the next point

  • Make sure to have your nym-gateway setup as Nym Exit Gateway following the commands here
  • Enable [ip_packet_router] (IPR) in your ~/.nym/gateways/*/config/config.toml and IPv4 and IPv6 with this script by running the two command below
# command to enable IPR
./nym-gateway setup-ip-packet-router --id <GATEWAY_ID> --enabled true

# script to enable IPv4 and IPv6
curl -o enable_networking_for_nym_nodes.sh https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/0840e1d2ee9949716c45655457d198607dfd3107/enable_networking_for_nym_nodes.sh -L && chmod u+x enable_networking_for_nym_nodes.sh && sudo ./enable_networking_for_nym_nodes.sh

3. Restart your node with root privileges

  • Either in a root shell or with sudo -E command
  • In case you run your node as a systemd service make sure to run systemctl daemon-reload before the service restart

Troubleshooting

If you come to any errors during the setup see troubleshooting page related to Mix Nodes and Gateways. In case your issue isn’t documented ask in our Element Node Operators channel or raise an issue on Github.

Last change: 2024-04-11, commit: f978552

Metrics of Performance Testing

At Nym as well as several core community operators had setup metrics monitors for a clear overview of node performance.

It is beneficial for a one time event or a limited period to connect nodes to the existing monitoring infrastructure system that Nym developers built to collect useful metrics. For the purpose of maximal privacy and decentralisation of the data - preventing Nym Mixnet from any global adversary takeover - we created these pages as a source of mutual empowerment, a place where operators can share and learn new skills to setup metrics monitors on their own infrastructure.

Collecting Testing Metrics

For the purpose of the performance testing Nym core developers plan to run instances of Prometheus and Grafana connected to Node explorer in the house. The network overall key insights we seek from these tests are primarily internal. We’re focused on pinpointing bottlenecks, capacity loads, and monitoring cpu usage on the nodes’ machines.

Guides to Setup Own Metrics

A list of different scripts, templates and guides for easier navigation:

Last change: 2024-04-11, commit: f978552

Prometheus & Grafana

The combination of Prometheus and Grafana is a common stack used by Nym team for internal monitoring as well as by core community operators like ExploreNym or No Trust Verify.

Prometheus

Prometheus is a free and open source monitoring systems. It allows operators to have their metrics, events, and alerts under full control. This ecosystem offers multiple advantages:

  • collects and records metrics from servers, containers, and applications
  • provides a flexible query language (PromQL)
  • multiple modes visualization tools
  • an alerting mechanism that sends notifications

Prometheus collects and stores its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.

Grafana

Grafana is an open-source analytics and interactive front end. It is widely used for its easy to manage dashboards with visualizations like graphs, charts and alerts, all connected to live data sources.

Setup Guides

There are various ways how to setup this stack. You can chose based on your preferences to do your own flow or follow some of the documented ones:

  • ExploreNYM scripts for self-hosted monitoring
  • Setup monitoring in a Docker container (detailed guide will be published soon)

References and further reading

Last change: 2024-04-11, commit: f978552

ExploreNym Monitoring Scripts

Community Monitoring Tools

Individual operators, node families and squads are the foundation of distributed network. There has been a great number of tools coming out of this community some of which can be deployed for the node monitoring setup.

Warning

Make sure you understand and properly evaluate what degree of control you give permission to before granting access to your data to any tools running on someone else’s servers.

ExploreNYM Tools

Long term involved operator Pawnflake, an author of ExploreNYM explorer, created a monitoring flow, which can be used by other operators called self-hosted-monitor. It utilises bash scripts to enable operators setup Prometheus and Grafana together with Node Exporter and Nginx to run their metrics monitoring stack locally.

In collaboration with ExploreNYM we published a step by step guide to set this up.

ExploreNYM also has a network measuring instance called enym-monitor. This setup is very simple for users, however it means that their data are all aggregated into one server and that is a design we would like to discourage from using as a primary one because such setup always brings a risk of centralisation of distributed node’s data into one computer.

Setup

Warning

This setup and the scripts included were not written by Nym developers. As always do your own audit before installing any scripts on your machine and familiarize yourself with the security risks involved when opening ports or allowing http access.

According to ExploreNYM the system requirements of the monitor stack are:

  • 2 CPU
  • 4 GB RAM
  • 20 GB of free disk space.

The monitoring part setup

This can be setup on another VPS than the node if desired. We recommend to try to set this up on the same VPS, as your node as we expect the machine to be strong enough to handle the node with enough capacity reserve for monitor.

  1. Install git
apt install git
  1. Clone the repository to ~/self-hosted-monitor
git clone https://github.com/ExploreNYM/self-hosted-monitor ~/self-hosted-monitor
  1. Give permissions to prometheus.sh script and run it to setup Prometheus
chmod +x ~/self-hosted-monitor/prometheus.sh && ~/self-hosted-monitor/prometheus.sh
  1. Give permissions to grafana.sh script and run it to setup Grafana
chmod +x ~/self-hosted-monitor/grafana.sh && ~/self-hosted-monitor/grafana.sh
  1. Open port 3000 to allow access to Grafana
sudo ufw allow 3000
  1. You can now access Grafana at http://<YOUR_IP_ADDRESS>:3000.

  2. (Optional step) If you have a registered domain and prefer to use it with https, give permissions to nginx-certbot.sh script and run it to setup Nginx and Certbot

chmod +x ~/self-hosted-monitor/nginx-certbot.sh && ~/self-hosted-monitor/nginx-certbot.sh
  1. Give permissions to prometheus-target.sh script and run it to add a scrape target. This can be run multiple times to add a new server to be monitored via Prometheus/
chmod +x ~/self-hosted-monitor/prometheus-target.sh && ~/self-hosted-monitor/prometheus-target.sh

The target server (the part to be monitored) setup

If you run a monitoring stack and your node on two different VPS, the steps 9 and 10 need to be done on the VPS with the running node. In case you do it on the same VPS, skip directly to step 11 and continue.

  1. Install git
apt install git
  1. Clone the repository to ~/self-hosted-monitor
git clone https://github.com/ExploreNYM/self-hosted-monitor ~/self-hosted-monitor
  1. Give permissions to node-exporter.sh script and run it to setup Node exporter.
chmod +x ~/self-hosted-monitor/node-exporter.sh && ~/self-hosted-monitor/node-exporter.sh

Grafana dashboard setup

Finally we need to access Grafana dashboards.

  1. Open a browser at http://<YOUR_IP_ADDRESS>:3000 or https://<YOUR_DOMAIN> (depends on your setup), enter username admin and password admin and setup new credentials on prompt

  2. Setup Data source by opening menu -> Connections -> Data sources -> + Add new data source -> Prometheus

  1. In the field Connection next to Prometheus server URL enter http://localhost:9090 (regardless if you accessing Grafana via http or https as this is for internal connection on the server). When you are done in the bottom confirm by Save & Test

  2. In the menu open: Dashboards -> + Create dashboard -> Import dashboard

  1. ID field: enter 1860 -> Load

  1. In Import dashboard page select Prometheus in the bottom and finally Import

Now you have your Prometheus panels displayed via Grafana dashboard for a simple monitoring of your node.

Verification and Troubleshooting

To ensure that your services are running correctly, you can verify that by running systemctl status <SERVICE> or run a journalctl -f -u <SERVICE> to print service logs. It shall return status Active: active (running). For example:

# to check if Prometheus service is active
systemctl status prometheus

# to check if Grafana service is active
systemctl status grafana-server

# to check if node-exporter service is active
systemctl status node_exporter

# to run journal log
journalctl -f -u prometheus # or any other service you want to see
Last change: 2024-04-11, commit: f978552

Troubleshooting

Binary Build Problems

I am trying to build from the GitHub archive files and the build fails

GitHub automatically includes .zip and tar.gz files of the Nym repository in its release. You cannot extract these and build - you’ll see something like this:

  process didn't exit successfully: `/build/nym/src/nym-0.12.1/target/release/build/nym-socks5-client-c1d0f76a8c7d7e9a/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at 'failed to extract build metadata: could not find repository from '/build/nym/src/nym-0.12.1/clients/socks5'; class=Repository (6); code=NotFound (-3)', clients/socks5/build.rs:7:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

Why does this happen?

We have scripts which automatically include the Git commit hash and Git tag in the binary for easier debugging later. If you download a .zip and try building from that, it’s not a Git repository and build will fail as above.

What to do?

  • Open terminal in the directory where you want to have a git repository
  • To get Nym repository for the first time, run:
git clone https://github.com/nymtech/nym.git
  • Follow the instructions to build the platform
  • To upgrade, pause your nodes, in the same terminal window run git pull, follow the upgrade instructions and re-start your nodes.

General Node Config

Where can I find my private and public keys and config?

All config and keys files are stored in a directory named after your id which you chose during the init process, and can be found at the following PATH: $HOME/.nym/<NODE_TYPE>/<NODE_ID> where $HOME is a home directory of the user (your current user in this case) that launched the node or client.

The directory structure for each node will be roughly as follows:

bob@nym:~$ tree /home/nym/.nym/mixnodes/
/home/nym/.nym/mixnodes/
|-- nym010
|   |-- config
|   |   `-- config.toml
|   `-- data
|       |-- private_identity.pem
|       |-- private_sphinx.pem
|       |-- public_identity.pem
|       `-- public_sphinx.pem

If you cat the public_sphinx.pem key, the output will be different from the public key you will see on Nym dashboard. The reason for this is that .pem files are encoded in base64, however on the web they are in base58. Don’t be confused if your keys look different. They are the same keys, just with different encoding :)

Mix Nodes

How can I tell my node is up and running and mixing traffic?

First of all check the ‘Mixnodes’ section of either of the Nym Network Explorers:

Enter your identity key to find your node. Check the contents of the Mixnode stats and Routing score sections.

There are 2 community explorers currently, which have been created by Nodes Guru:

Here is a dictionary with Nyx chain registry entry regarding all explorers.

If you want more information, or if your node isn’t showing up on the explorer of your choice and you want to double-check, here are some examples on how to check if the node is configured properly.

Check from your VPS

Additional details can be obtained via various methods after you connect to your VPS:

Socket statistics with ss
sudo ss -s -t | grep 1789 # if you have specified a different port in your Mix Node config, change accordingly

This command should return a lot of data containing ESTAB. This command should work on every unix based system.

List open files and reliant processes with lsof
# check if lsof is installed:
lsof -v
# install if not installed
sudo apt install lsof
# run against nym-mix-node node port
sudo lsof -i TCP:1789 # if you have specified a different port in your mixnode config, change accordingly

This command should return something like this:

nym-mixno 103349 root   53u  IPv6 1333229972      0t0  TCP [2a03:b0c0:3:d0::ff3:f001]:57844->[2a01:4f9:c011:38ae::5]:1789 (ESTABLISHED)
nym-mixno 103349 root   54u  IPv4 1333229973      0t0  TCP nym:57104->194.5.78.73:1789 (ESTABLISHED)
nym-mixno 103349 root   55u  IPv4 1333229974      0t0  TCP nym:48130->static.236.109.119.168.clients.your-server.de:1789 (ESTABLISHED)
nym-mixno 103349 root   56u  IPv4 1333229975      0t0  TCP nym:52548->vmi572614.contaboserver.net:1789 (ESTABLISHED)
nym-mixno 103349 root   57u  IPv6 1333229976      0t0  TCP [2a03:b0c0:3:d0::ff3:f001]:43244->[2600:1f18:1031:2401:c04b:2f25:ca79:fef3]:1789 (ESTABLISHED)
Query systemd journal with journalctl
sudo journalctl -u nym-mixnode -o cat | grep "Since startup mixed"

If you have created nym-mixnode.service file (i.e. you are running your Mix Node via systemd) then this command shows you how many packets have you mixed so far, and should return a list of messages like this:

2021-05-18T12:35:24.057Z INFO  nym_mixnode::node::metrics                      > Since startup mixed 233639 packets!
2021-05-18T12:38:02.178Z INFO  nym_mixnode::node::metrics                      > Since startup mixed 233739 packets!
2021-05-18T12:40:32.344Z INFO  nym_mixnode::node::metrics                      > Since startup mixed 233837 packets!
2021-05-18T12:46:08.549Z INFO  nym_mixnode::node::metrics                      > Since startup mixed 234081 packets!
2021-05-18T12:56:57.129Z INFO  nym_mixnode::node::metrics                      > Since startup mixed 234491 packets!

You can add | tail to the end of the command to watch for new entries in real time if needed.

build-info

A build-info command prints the build information like commit hash, rust version, binary version just like what command --version does. However, you can also specify an --output=json flag that will format the whole output as a json, making it an order of magnitude easier to parse.

For example ./target/debug/nym-network-requester --no-banner build-info --output json will return:

{"binary_name":"nym-network-requester","build_timestamp":"2023-07-24T15:38:37.00657Z","build_version":"1.1.23","commit_sha":"c70149400206dce24cf20babb1e64f22202672dd","commit_timestamp":"2023-07-24T14:45:45Z","commit_branch":"feature/simplify-cli-parsing","rustc_version":"1.71.0","rustc_channel":"stable","cargo_profile":"debug"}

Check from your local machine

Scan ports with nmap:
nmap -p 1789 <IP ADDRESS> -Pn

If your Mix Node is configured properly it should output something like this:

bob@desktop:~$ nmap -p 1789 95.296.134.220 -Pn

Host is up (0.053s latency).

PORT     STATE SERVICE
1789/tcp open  hello
Query online nodes:
curl --location --request GET 'https://validator.nymtech.net/api/v1/mixnodes/'

Will return a list all nodes currently online.

You can query Gateways by replacing nym-mixnodes with nym-gateways in the above command, and can query for the Mix Nodes and Gateways on the Sandbox testnet by replacing validator with sandbox-validator.

Check with Network API

We currently have an API set up returning our metrics tests of the network. There are two endpoints to ping for information about your Mix Node, report and history. Find more information about this in the Mixnodes metrics documentation.

Why is my node not mixing any packets?

If you are still unable to see your node on the dashboard, or your node is declaring it has not mixed any packets, there are several potential issues:

  • The firewall on your host machine is not configured properly. Checkout the instructions.
  • You provided incorrect information when bonding your node.
  • You are running your Mix Node from a VPS without IPv6 support.
  • You did not use the --announce-host flag while running the Mix Node from your local machine behind NAT.
  • You did not configure your router firewall while running the Mix Node from your local machine behind NAT, or you are lacking IPv6 support.
  • Your Mix Node is not running at all, it has either exited / panicked or you closed the session without making the node persistent. Check out the instructions.

Caution

Your Mix Node must speak both IPv4 and IPv6 in order to cooperate with other nodes and route traffic. This is a common reason behind many errors we are seeing among node operators, so check with your provider that your VPS is able to do this!

Incorrect bonding information

Check that you have provided the correct information when bonding your Mix Node in the web wallet interface. When in doubt, un-bond and then re-bond your node!

All delegated stake will be lost when un-bonding! However the Mix Node must be operational in the first place for the delegation to have any effect.

Missing announce-host flag

On certain cloud providers such as AWS and Google Cloud, you need to do some additional configuration of your firewall and use --host with your local ip and --announce-host with the public ip of your Mix Node host.

If the difference between the two is unclear, contact the help desk of your VPS provider.

No IPv6 connectivity

Make sure that your VPS has IPv6 connectivity available with whatever provider you are using.

To get all ip addresses of your host, try following commands:

hostname -i

Will return your local ip address.

hostname -I

Will return all of the ip addresses of your host. This output should look something like this:

bob@nym:~$ hostname -I
88.36.11.23 172.18.0.1 2a01:28:ca:102::1:641
  • The first ipv4 is the public ip you need to use for the --announce-host flag.
  • The second ipv4 is the local ip you need to use for the --host flag.
  • The 3rd output should confirm if your machine has ipv6 available.

Running on a local machine behind NAT with no fixed IP address

Your ISP has to be IPv6 ready if you want to run a Mix Node on your local machine. Sadly, in 2020, most of them are not and you won’t get an IPv6 address by default from your ISP. Usually it is an extra paid service or they simply don’t offer it.

Before you begin, check if you have IPv6 here or by running command explained in the section above. If not, then don’t waste your time to run a node which won’t ever be able to mix any packet due to this limitation. Call your ISP and ask for IPv6, there is a plenty of it for everyone!

If all goes well and you have IPv6 available, then you will need to init the Mix Node with an extra flag, --announce-host. You will also need to edit your config.toml file each time your IPv4 address changes, that could be a few days or a few weeks. Check the your IPv4 in the section above.

Additional configuration on your router might also be needed to allow traffic in and out to port 1789 and IPv6 support.

Here is a sample of the init command example to create the Mix Node config.

./nym-mixnode init --id <YOUR_ID> --host 0.0.0.0 --announce-host 85.160.12.13
  • --host 0.0.0.0 should work every time even if your local machine IPv4 address changes. For example on Monday your router gives your machine an address 192.168.0.13 and on Wednesday, the DHCP lease will end and you will be assigned 192.168.0.14. Using 0.0.0.0 should avoid this without having to set any static IP in your router’s configuration.

  • you can get your current IPv4 address by either using curl ipinfo.io if you’re on MacOS or Linux or visiting whatsmyip site. Simply copy it and use it as --anounce-host address.

Make sure you check if your node is really mixing. We are aiming to improve the setup for operators running locally, however you may need a bit of patience to set this up from your home behind NAT.

Accidentally killing your node process on exiting session

When you close your current terminal session, you need to make sure you don’t kill the Mix Node process! There are multiple ways on how to make it persistent even after exiting your ssh session, the easiest solution is to use tmux or nohup, and the more elegant solution is to run the node with systemd. Read the automation manual here.

Common errors and warnings

Most of the ERROR and WARN messages in your node logs are benign - as long as your node outputs since startup mixed X packets! (X bust be > 0) in your logs (and this number increases over time), your node is mixing packets. If you want to be sure, check the Nym dashboard or see other ways on how to check if your node is mixing properly as outlined in the section How can I tell my node is up and running and mixing traffic? above.

More specific errors and warnings are covered below.

tokio runtime worker error

If you are running into issues with an error including the following:

thread 'tokio-runtime-worker' panicked at 'Failed to create TCP listener: Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" }'

Then you need to --announce-host <PUBLIC_IP> and --host <LOCAL_IP> on startup. This issue is addressed above

Can I use a port other than 1789?

Yes! Here is what you will need to do:

Assuming you would like to use port 1337 for your Mix Node, you need to open the new port (and close the old one):

sudo ufw allow 1337
sudo ufw deny 1789

And then edit the Mix Node’s config.

If you want to change the port for an already running node, you need to stop the process before editing your config file.

The config file is located at ~/.nym/mixnodes/<YOUR_ID>/config/config.toml.

For example, assuming <YOUR_ID> was chosen to be alice-node:

nano ~/.nym/mixnodes/alice-node/config/config.toml

You will need to edit two parts of the file. announce_address and listening_address in the config.toml file. Simply replace :1789 (the default port) with :1337 (your new port) after your IP address.

Finally, restart your node. You should see if the Mix Node is using the port you have changed in the config.toml file right after you run the node.

What is verloc and do I have to configure my Mix Node to implement it?

verloc is short for verifiable location. Mix Nodes and Gateways now measure speed-of-light distances to each other, in an attempt to verify how far apart they are. In later releases, this will allow us to algorithmically verify node locations in a non-fake-able and trustworthy manner.

You don’t have to do any additional configuration for your node to implement this, it is a passive process that runs in the background of the mixnet from version 0.10.1 onward.

Gateways & Network Requesters

My Gateway is running but appears offline in the explorer

Let your Gateway run and follow these steps:

  1. Check if your firewall configuration is active and if the necessary ports are open / allowed, including the ones for Swagger page and Reversed proxy/WSS if this is your case.
  2. See if the Gateway is not on the list of blacklisted Gateways
  3. If it’s blacklisted, check out the point below

My Gateway is blacklisted

Nym API measures performance by routing traffic through the Mixnet. If the average of a Gateway’s routing score in past 24h is less than 50%, the Gateway gets blacklisted and remains so until this number is higher than 50%.

In case your Gateway appeared on the blacklist, it’s because there is some flaw in the configuration. The most common sources of problems are:

  • Bonding before starting the node/service
  • Bonding before opening the needed ports
  • VPS restarted without operator having a systemd automation or some alert notification flow setup

What to do:

  • Make sure your node is running and do not stop it if there is no need
  • Open all needed ports
  • Wait until your node gets above 50% of performance (average of last 24h) - this will likely take several hours, up to a day. During this time your node is tested by nym-api and every positive response picks up your Gateway’s routing score.

Do not restart your Gateway without reason, your routing score will only get worse!

My exit Gateway “is still not online…”

The Nyx chain epoch takes up to 60 min. To prevent the Gateway getting blacklisted, it’s essential to start it before the bonding process and let it running. In case it already got blacklisted check the steps above.

When enabling ip_packet_router (IPR) I get a client-core error

This error tells you that you already have IPR keys in your data storage, to activate them you have two options:

  1. Open ~/.nym/gateways/<ID>/config/config.toml and set the correct values
[ip_packer_router_enabled]
enabled = true

# UNDER [storage_paths] CHANGE
ip_packet_router_config = '~/.nym/gateways/<ID>/config/ip_packet_router_config.toml'
  1. Or remove the IPR data storage and initialise a new one with these commands
rm -rf ~/.nym/gateways/<ID>/data/ip-packet-router-data

./nym-gateway setup-ip-packet-router --id <ID>

My ip_packet_router (IPR) seems to not work

There are a few steps to mitigate problems with IPR:

  1. Check out the issue right above regarding the Exit Gateway config
  2. Open your browser and checkout the Swagger UI page and see if all the roles are enabled:
# in case of IP
http://<YOUR_LISTENING_IP_ADDRESS>:8080/api/v1/roles

# in case of hostname domain
https://<YOUR_DOMAIN>/api/v1/roles
  1. Make sure all your ports are open properly
  2. Make sure to run your Gateway with embedded IPR as root. Either in a root shell with your configs in /root/.nym/ or with a command sudo -E which gives root privileges but looks for user config folder
  3. If it’s all good in the API but you don’t see the right tick/badge in the Performance testing list, just wait some time and then try to refresh the page

Validators

Common reasons for your validator being jailed

The most common reason for your validator being jailed is that your validator is out of memory because of bloated syslogs.

Running the command df -H will return the size of the various partitions of your VPS.

If the /dev/sda partition is almost full, try pruning some of the .gz syslog archives and restart your validator process.

Where can I get more help?

The fastest way to reach one of us or get a help from the community, visit our Telegram Node Setup Help Chat or head to our Discord.

For more tech heavy question join our Matrix core community channel, where you can meet other builders and Nym core team members.

Last change: 2024-04-11, commit: f978552

Nyx Validator Rewards

Summary

  • Nyx Validators are rewarded in NYM tokens from the Nym mixmining pool and increasingly from apps that run on the Nym mixnet, the first of which is the NymVPN.
  • Validators are rewarded for two different types of work: signing blocks in the Nyx chain and running the NymAPI to monitor mixnet routing and sign zk-nym credentials.
  • New validators can join via a NYM-to-NYX swap contract. The contract will not allow more than 1% of total stake increase per month to prevent sudden hostile takeovers. Current stake is 46.265 million Nyx. Rate: 1:4.8 ~ 288k NYX for 60k NYM => 0.62% voting power
  • NYX tokens serve no other purpose than self-delegation for voting power and governance. All rewards are in NYM and distributed directly to the validators self-delegation address and are not distributed to stakers.
  • The contract will only allow swapping NYM to NYX and will not allow exchanging NYX back to NYM. A NYX holder who wishes to sell their NYX stake will have to do so via OTC trades.

Validator Rewards

Nyx Validators perform two types of work for which they will be rewarded:

1. Signing blocks in the Nyx chain

A “block signing monitor“ monitors blocks being produced on the Nyx chain and gathers the signatures present on every block. After an epoch ends, the monitor will assess performance of a validator and distribute tokens (to the self-delegation wallet) proportional to the voting period and uptime of the validator.

2. Running the NymAPI to monitor Mixnet routing and sign zk-nyms (Nym’s anonymous credentials)

Validator rewards initially come from the Nym mixmining pool with additional rewards increasingly coming from paid applications running on the Nym mixnet. The first paid application is the NymVPN. Nyx validators will be rewarded for their work directly in NYM tokens to their validator self-delegation address.

  1. From mixmining pool - at a rate of 1000 NYM per hour, of which 2/3 are distributed for signing blocks and 1/3 for zk-nyms. These are stable in NYM, and therefore will fluctuate in their fiat value depending on exchange rate.

  2. From vpn user subscriptions - the rate is tied to the growth of NymVPN subscriptions and will be stable in fiat, fluctuating in NYM depending on exchange rate. 1/3 will be distributed for signing blocks and 2/3 for zk-nyms.

SourceSigning blocksRunning NymAPICurrency
Mixmining pool2/31/3NYM
NymVPN1/32/3fiat

zk-nyms

The zk-nyms enable people to anonymously prove access rights to the upcoming NymVPN client without having to reveal payment details that might compromise their privacy. This is the first of what we imagine to be many possible use-cases for the zk-nym scheme.

Allocation of Rewards from Nym mixmining pool

Rewards for validators will be distributed at an hourly rate from the mixmining pool. The amount is 1000 NYM per hour to be distributed among all validators. The fraction of mixmining rewards received by each individual validator is proportional to its contributions to the network.

Two thirds of the available rewards (670 NYM per hour) are distributed proportionally to each validator’s share when signing blocks in the chain, for which tx fees are also received in the same proportion; while the last third (330 NYM per hour) is allocated to validators running the NymAPI, proportionally to their contribution to signing zk-nym credentials.

The rewards are stable in NYM and fluctuate in their fiat value depending on the exchange rate of NYM tokens.

Permissionless Nyx Chain

To allow new validators to join Nyx chain a new smart contract will be set up to release NYX in exchange for NYM. This contract allows a limited amount of NYM tokens to be deposited per month. The deposited NYM tokens are added to the mixmining pool and thus contribute to future rewards of all nodes and validators.

The smart contract will have two parameters:

  1. the maximum amount of NYX available for purchase per month
  2. the NYM-to-NYX exchange rate offered by the contract

Maximum Amount of NYX Available for Purchase per Month

The contract will not allow more than 1% of total stake increase per month to prevent sudden hostile takeovers. Current stake level is 46.265 million Nyx.

Last change: 2024-04-11, commit: f978552

Frequently Asked Questions

Nym Mixnet

To see different stats about Nym Mixnet live, we recommend you to visit status.notrustverify.ch built by No Trust Verify crew, one of the squads within Nym core community.

Is there an explorer for Nym Mixnet?

Yes, there are several places, some are built by Nym core community:

What determines the rewards when running a Mix Node?

The stake required for a Mix Node to achieve maximum rewards is called Mix Node saturation point. This is calculated from the staking supply (all circulating supply + part of unlocked tokens). The target level of staking is to have 40% of the staking supply locked in Mix Nodes.

The node stake saturation point, which we denote by Nsat, is given by the stake supply, target level of staking divided between the rewarded nodes.

This design ensures the nodes aim to have a same size of stake (reputation) which can be done by delegation staking, as well as it secures a whale prevention and decentralization of staking, as any higher level of delegated $NYM than Nsat per node results in worsening reward ratio. On the contrary, the more Mix Nodes are active, the lower is Nsat. The equilibrium is reached when the staked tokens are delegated equally across the active Mix nodes and that’s our basis for this incentive system.

The rewarded nodes are the nodes which will receive some rewards by the end of the given epoch. These can be separated further separated into:

  1. Active: Top N nodes of the rewarded set (currently all of them but this can change), these are nodes which are used by the clients and mix packets.

  2. Standby: Bottom N nodes of the rewarded set, they don’t mix data from the clients but are used for testing. Their reward is smaller.

For more detailed calculation, read our blog post Nym Token Economics update. More info on staking can be found here. And here is more info on how to choose a Mix Node for delegation. And finally an update on token economics from July 2023.

More graphs and stats at stats.notrustverify.ch.

Which VPS providers would you recommend?

Consider in which jurisdiction you reside and where do you want to run a Mix Node. Do you want to pay by crypto or not and what are the other important particularities for your case? We always recommend operators to try to choose smaller and decentralised VPS providers over the most known ones controlling a majority of the internet. We receive some good feedback on these: Linode, Ghandi, Flokinet and Exoscale. Do your own research and share with the community.

Why is a mix node setup on a self-hosted machine so tricky?

We don’t recommend this setup because it’s really difficult to get a static IP and route IPv6 traffic.

What’s the Sphinx packet size?

The sizes are shown in the configs here (default is the one clients use, the others are for research purposes, not to be used in production as this would fragment the anonymity set). More info can be found here.

Why a Mix Node and a Gateway cannot be bonded with the same wallet?

Because of the way the smart contract works we keep it one-node one-address at the moment.

Which nodes are the most needed to be setup to strengthen Nym infrastructure and which ones bring rewards?

Ath this point the most crutial component needed are Exit Gateways.

Are Mix Nodes whitelisted?

Nope, anyone can run a Mix Node. Purely reliant on the node’s reputation (self stake + delegations) & routing score.

Validators and tokens

What’s the difference between NYM and uNYM?

1 NYM = 1 000 000 uNYM

Why some Nyx blockchain operations take one hour and others are instant?

This is based on the definition in Nym’s CosmWasm smart contracts code.

Whatever is defined as a pending epoch event will get resolved at the end of the current epoch.

And whatever is defined as a pending interval event will get resolved at the end of the current interval.

Can I run a validator?

We are currently working towards building up a closed set of reputable validators. You can ask us for coins to get in, but please don’t be offended if we say no - validators are part of our system’s core security and we are starting out with people we already know or who have a solid reputation.

Why is validator set entry whitelisted?

We understand that the early days of the Nyx blockchain will face possible vulnerabilities in terms of size - easy to disrupt or halt the chain if a malicious party entered with a large portion of stake. Besides that, there are some legal issues we need to address before we can distribute the validator set in a fully permissions fashion.

Why does Nym do airdrops?

It is part of ensuring decentralisation - we need to avoid a handful of people having too much control over the token and market. Of course ideally people will stake the tokens and contribute to the project at this stage. We run surveys to better understand what people are doing with their tokens and what usability issues there are for staking. Any feedback is appreciated as it helps us improve all aspects of using the token and participating in the ecosystem.

Last change: 2024-04-11, commit: f978552

Project Smoosh - FAQ

We aim on purpose to make minimal changes to reward scheme and software. We’re just ‘smooshing’ together stuff we already debugged and know works.
– Harry Halpin, Nym CEO


This page refer to the changes which are planned to take place over Q3 and Q4 2023. As this is a transition period in the beginning (Q3 2023) the Mix Nodes FAQ page holds more answers to the current setup as project Smoosh refers to the eventual setup. As project Smoosh gets progressively implemented the answers on this page will become to be more relevant to the current state and eventually this FAQ page will be merged with the still relevant parts of the main Mix Nodes FAQ page.

If any questions are not answered or it’s not clear for you in which stage project Smoosh is right now, please reach out in Node Operators Matrix room.

Overview

What is project Smoosh?

As we shared in our blog post article What does it take to build the wolds most powerful VPN, project Smoosh is:

A nick-name by CTO Dave Hrycyszyn and Chief Scientist Claudia Diaz for the work they are currently doing to “smoosh” Nym Nodes so that the same operator can serve alternately as Mix Node, Gateway or VPN node. This requires careful calibration of the Nym token economics, for example, only nodes with the highest reputation for good quality service will be in the VPN set and have the chance to earn higher rewards. By simplifying the components, adding VPN features and supporting new node operators, the aim is to widen the geographical coverage of nodes and have significant redundancy, meaning plenty of operators to be able to meet demand. This requires strong token economic incentives as well as training and support for new node operators.

Technical Questions

What are the changes?

Project Smoosh will have four steps, please follow the table below to track the dynamic progress:

StepStatus
1. Combine the nym-gateway and nym-network-requester into one binary✅ done
2. Create Exit Gateway: Take the nym-gateway binary including nym-network-requester combined in #1 and switch from allowed.list to a new exit policy✅ done
3. Combine all the nodes in the Nym Mixnet into one binary, that is nym-mixnode, nym-gateway (entry and exit) and nym-network-requester.🛠️ in progress
4. Adjust reward scheme to incentivise and reward Exit Gateways as a part of nym-node binary, implementing zkNym credentials.🛠️ in progress

These steps will be staggered over time - period of several months, and will be implemented one by one with enough time to take in feedback and fix bugs in between.
Generally, the software will be the same, just instead of multiple binaries, there will be one Nym Node (nym-node) binary. Delegations will remain on as they are now, per our token economics (staking, saturation etc)

What does it mean for Nym nodes operators?

We are exploring two potential methods for implementing binary functionality in practice and will provide information in advance. The options are:

  1. Make a selection button (command/argument/flag) for operators to choose whether they want their node to provide all or just some of the functions nodes have in the Nym Mixnet. Nodes functioning as Exit Gateways (in that epoch) will then have bigger rewards due to their larger risk exposure and overhead work with the setup.

  2. All nodes will be required to have the Exit Gateway functionality. All nodes are rewarded the same as now, and the difference is that a node sometimes (some epochs) may be performing as Exit Gateway sometimes as Mix node or Entry Gateway adjusted according the network demand by an algorithm.

Where can I read more about the Exit Gateway setup?

We created an entire page about the technical and legal questions around Exit Gateway.

What is the change from allow list to deny list?

The operators running Gateways would have to “open” their nodes to a wider range of online services, in a similar fashion to Tor exit relays. The main change will be to expand the original short allowed.list to a more permissive setup. An exit policy will constrain the hosts that the users of the Nym VPN and Mixnet can connect to. This will be done in an effort to protect the operators, as Gateways will act both as SOCKS5 Network Requesters, and exit nodes for IP traffic from Nym VPN and Mixnet clients.

How will the Exit policy be implemented?

Follow the dynamic progress of exit policy implementation on Gateways below:

StepStatus
1. By default the exit policy filtering is disabled and the allowed.list filtering is going to continue be used. This is to prevent operators getting surprised by upgrading their Gateways (or Network Requesters) and suddenly be widely open to the internet. To enable the new exit policy, operators must use --with-exit-policy flag or modify the config.toml file.✅ done
2. The exit policy is part of the Gateway setup by default. To disable this exit policy, operators must use --disable-exit-policy flag.🛠️ in progress
3. The exit policy is the only option. The allowed.list is completely removed.🛠️ in progress

Keep in mind the table above only relates to changes happening on Gateways. For the Project Smoosh progress refer to the table above. Whether Exit Gateway functionality will be optional or mandatory part of every active Nym Node depends on the chosen design.

Can I run a Mix Node only?

It depends which design will ultimately be used. In case of the first - yes. In case of the second option, all the nodes will be setup with Exit Gateway functionality turned on.

Token Economics & Rewards

Info

For any specifics on Nym token economics and Nym Mixnet reward system, please read the Nym token economics paper.

What are the incentives for the node operator?

In the original setup there were no incentives to run a nym-network-requester binary. After the transition all the users will buy multiple tickets of zkNyms credentials and use those as anonymous e-cash to pay for their data traffic (Nym API will do the do cryptographical checks to prevent double-spending). All collected fees get distributed to all active nodes proportionally to their work by the end of each epoch.

How does this change the token economics?

The token economics will stay the same as they are, same goes for the reward algorithm.

How are the rewards distributed?

This depends on design chosen. In case of #1, it will look like this:

As each operator can choose what roles their nodes provide, the nodes which work as open Gateways will have higher rewards because they are the most important to keep up and stable. Besides that the operators of Gateways may be exposed to more complication and possible legal risks.

The nodes which are initialized to run as Mix Nodes and Gateways will be chosen to be on top of the active set before the ones working only as a Mix Node.

I case we go with #2, all nodes active in the epoch will be rewarded proportionally according their work.

In either way, Nym will share all the specifics beforehand.

How will be the staking and inflation after project Smoosh?

Nym will run tests to count how much payment comes from the users of the Mixnet and if that covers the reward payments. If not, we may need to keep inflation on to secure incentives for high quality Gateways in the early stage of the transition.

When project smooth will be launched, it would be the mixmining pool that will pay for the Gateway rewards based on amount of traffic routed ?

Yes, the same pool. Nym’s aim is to do minimal modifications. The only real modification on the smart contract side will be to get into top X of ‘active set’ operators will need to have open Gateway function enabled.

What does this mean for the current delegators?

From an operator standpoint, it shall just be a standard Nym upgrade, a new option to run the Gateway software on your node. Delegators should not have to re-delegate.

So far the general line is that running a Gateway is not illegal (unless you are in Iran, China, and a few other places) and due to encryption/mixing less risky than running a normal VPN node. For Mix Nodes, it’s very safe as they have “no idea” what packets they are mixing.

There are several legal questions and analysis to be made for different jurisdictions. To be able to share resources and findings between the operators themselves we created a Community Legal Forum.

Last change: 2024-04-11, commit: f978552

Nym Operators Legal Forum: Running Exit Gateway

Info

The entire content of this page is under Creative Commons Attribution 4.0 International Public License.

This page is a part of Nym Community Legal Forum and its content is composed by shared advices in Node Operators Legal Forum (Matrix chat) as well as though pull requests done by the node operators directly to our repository, reviewed by Nym DevRels.

This document presents an initiative to further support Nym’s mission of allowing privacy for everyone everywhere. This would be achieved with the support of Nym node operators operating Gateways and opening these to any online service. Such setup needs a clear policy, one which will remain the same for all operators running Nym nodes. The proposed Exit policy is a combination of two existing safeguards: Tor Null ‘deny’ list and Tor reduced policy.

All the technical changes on the side of Nym nodes - Project Smoosh - are described in the FAQ section.

Warning

Nym core team cannot provide comprehensive legal advice across all jurisdictions. Knowledge and experience with the legalities are being built up with the help of our counsel and with you, the community of Nym node operators. We encourage Nym node operators to join the operator channels (Element, Discord, Telegram) to share best practices and experiences.

Summary

  • This document outlines a plan to change Nym Gateways from operating with an ‘allow’ to a ‘deny’ list to enable broader uptake and usage of the Nym Mixnet. It provides operators with an overview of the plan, pros and cons, legal as well as technical advice.

  • Nym is committed to ensuring privacy for all users, regardless of their location and for the broadest possible range of online services. In order to achieve this aim, the Nym Mixnet needs to increase its usability across a broad range of apps and services.

  • Currently, Nym Gateway nodes only enable access to apps and services that are on an ‘allow’ list that is maintained by the core team.

  • To decentralise and enable privacy for a broader range of services, this initiative will have to transition from the current ‘allow’ list to a ‘deny’ list - exit policy. In accordance with the Tor Null ‘deny’ list and Tor reduced policy, which are two established safeguards.

  • This will enhance the usage and appeal of Nym products for end users. As a result, increased usage will ultimately lead to higher revenues for Nym operators.

  • Nym core team cannot provide operators with definitive answers regarding the potential risks of operating open Gateways. However, there is online evidence of operating Tor exit relays:

    • From a technical perspective, Nym node operators may need to implement additional controls, such as dedicated hardware and IP usage, or setting up an HTML exit notice on port 80.
    • From an operational standpoint, node operators may be expected to actively manage their relationship with their ISP or VPS provider and respond to abuse requests using the proposed templates.
    • Legally, exit relays are typically considered “telecommunication networks” and are subject to intermediary liability protection. However, there may be exceptions, particularly in cases involving criminal law and copyright claims. Operators could seek advice from local privacy associations and may consider running nodes under an entity rather than as individuals.
  • This document serves as the basis for a consultation with Nym node operators on any concerns or additional support and information you need for this change to be successful and ensure maximum availability, usability and adoption.

Goal of the initiative

Nym supports privacy for everyone, everywhere.

To offer a better and more private everyday experience for its users, Nym would like them to use any online services they please, without limiting its access to a few messaging apps or crypto wallets.

To achieve this, operators running Exit Gateways would have to “open” their nodes to a wider range of online services, in a similar fashion to Tor exit relays following this exit policy.

Pros and cons of the initiative

Previous setup: Running nodes supporting strict SOCKS5 app-based traffic

DimensionProsCons
Aspirational- Very limited use cases, not supportive of the “Privacy for everyone everywhere” aspiration
- Limited appeal to users, low competitiveness in the market, thus low usage
Technical- No changes required in technical setup
Operational- No impact on operators operations (e.g., relationships with VPS providers)
- Low overhead
- Can be run as an individual
Legal- Limited legal risks for operators
Financial- Low revenues for operators due to limited product traction

The new setup: Running nodes supporting traffic of any online service (with safeguards in the form of a denylist)

DimensionProsCons
Aspirational- Higher market appeal of a fully-fledged product able to answer all users’ use cases
- Relevance in the market, driving higher usage
Technical- Very limited changes required in the technical setup (changes in the allow -> denylist)- Increased monitoring required to detect and prevent abuse (e.g. spam)
Operational- Higher operational overhead, such as dealing with DMCA / abuse complaints, managing the VPS provider questions, or helping the community to maintain the denylist
- Administrative overhead if running nodes as a company or an entity
Legal- Ideally requires to check legal environment with local privacy association or lawyer

Exit Gateways: New setup

In our previous technical setup, Network Requesters acted as a proxy, and only made requests that match an allow list. That was a default IP based list of allowed domains stored at Nym page in a centralised fashion possibly re-defined by any Network Requester operator.

This restricts the hosts that the NymConnect app can connect to and has the effect of selectively supporting messaging services (e.g. Telegram, Matrix) or crypto wallets (e.g. Electrum or Monero). Operators of Network Requesters can have confidence that the infrastructure they run only connects to a limited set of public internet hosts.

The principal change in the new configuration is to make this short allow list more permissive. Nym’s exit policy will restrict the hosts to which Nym Mixnet and Nym VPN users are permitted to connect. This will be done in an effort to protect the operators, as Gateways will act both as SOCKS5 Network Requesters, and exit nodes for IP traffic from Nym Mixnet VPN and VPN clients (both wrapped in the same app).

As of now we the Gateways will be defaulted to a combination of Tor Null ‘deny’ list (note: Not affiliated with Tor) - reproduction permitted under Creative Commons Attribution 3.0 United States License which is IP-based, e.g., ExitPolicy reject 5.188.10.0/23:* and Tor reduced policy. Whether we will stick with this list, do modifications or compile another one is still a subject of discussion. In all cases, this policy will remain the same for all the nodes, without any option to modify it by Nym node operators to secure stable and reliable service for the end users.

For exit relays on ports 80 and 443, the Gateways will exhibit an HTML page resembling the one proposed by Tor. By doing so, the operator will be able to disclose details regarding their Gateway, including the currently configured exit policy, all without the need for direct correspondence with regulatory or law enforcement agencies. It also makes the behavior of Exit Gateways transparent and even computable (a possible feature would be to offer a machine readable form of the notice in JSON or YAML).

We also recommend operators to check the technical advice from Tor.

Giving the legal similarity between Nym Exit Gateways and Tor Exit Relays, it is helpful to have a look in Tor community Exit Guidelines. This chapter is an exert of tor page.

Note that Tor states:

This FAQ is for informational purposes only and does not constitute legal advice.

Check legal advice prior to running an exit relay

Run an exit relay within an entity

As an organisation - it might help from a liability perspective

  • Within your university
  • With a node operators’ association (e.g., a Torservers.net partner)
  • Within a company

Be ready to respond to abuse complaints

  • Make your contact details (email, phone, or even fax) available, instead of those of the ISP
  • Reply in a timely manner (e.g., 24 hours) using the provided templates
  • Note that Tor states: “We are not aware of any case that made it near a court, and we will do everything in our power to support you if it does.”
  • Document experience with ISPs at community.torproject.org/relay/community-resources/good-bad-isps

Useful links:

The Node Operators Legal Forum pages are divided according jurisdictions. Read below to learn how to add findings about your jurisdiction or edit existing info.

Community Counsel

Nym Node operators are invited to add their legal findings or helpful suggestions directly through pull requests. This can be done as a new legal information (or entire new country) to the list of jurisdictions or in a form of an advice to Community counsel pages, sharing examples of Exit Gateway landing pages, templates etcetra.

How to add content

Our aim is to establish a strong community network, sharing legal findings and other suggestions with each other. We would like to encourage all of the current and future operators to do research about the situation in the jurisdiction they operate in as well as solutions to any challenges when running an Exit Gateway and add those through a pull request (PR). Please check out the steps to make a pull request.

Last change: 2024-04-11, commit: f978552

Community Counsel

Info

The entire content of this page is under Creative Commons Attribution 4.0 International Public License.

Running an Exit Gateway is a commitment and as such is exposed to various challenges. Besides different legal regulations typical difficulties may be dealing with VPS or ISP providers. Our strength lies in decentralised community of squads and individuals supporting each other. Sharing examples of landing pages, templates for communication and FAQs is a great way to empower other operators sharing the mission of liberating internet. Below is a simple way how to create a pull request directly to Nym Operator Guide book.

How to add content

Our aim is to establish a strong community network, sharing legal findings and other suggestions with each other. We would like to encourage all of the current and future operators to do research about the situation in the jurisdiction they operate in as well as solutions to any challenges when running an Exit Gateway and add those through a pull request (PR). Please check out the steps to make a pull request.

Last change: 2024-04-11, commit: f978552

Exit Gateway - Jurisdictions

Info

The entire content of this page is under Creative Commons Attribution 4.0 International Public License.

Warning

The following part is for informational purposes only. Nym core team cannot provide comprehensive legal advice across all jurisdictions. Knowledge and experience with the legalities are being built up with the help of our counsel and with you, the community of Nym node operators. We encourage Nym node operators to join the Node Operator and Operators Legal Forum channels on Element to share best practices and experiences.

There is no one silver bullet covering the entire globe. Every jurisdiction has different set of laws and practices regarding internet traffic routing. Therefore the Node Operators Legal Forum pages are divided per region.

These are some findings from our legal team as an example:

Nym Node operators are invited to add their legal findings directly through pull requests.

Last change: 2024-04-11, commit: f978552

Legal environment: Switzerland

Info

The entire content of this page is under Creative Commons Attribution 4.0 International Public License.

Warning

The following part is for informational purposes only. Nym core team cannot provide comprehensive legal advice across all jurisdictions. Knowledge and experience with the legalities are being built up with the help of our counsel and with you, the community of Nym node operators. We encourage Nym node operators to join the Node Operator and Operators Legal Forum channels on Element to share best practices and experiences.

Note: The information shared below is in the stage of conclusions upon final confirmation. The text is a not edited exert from a legal counsel. Nym core team is asking for more clarifications.

Operators of Exit Nodes

Telecoms Law

As well as operators of normal mixnet nodes, operators of exit nodes might be considered telecommunications providers according to the broad term of the telecommunications act (TCA). The regulatory consequences have already been laid out in section 5.1.2.2.1 above.

Telecoms Surveillance Law

Unlike normal mixnet nodes, exit nodes might have information about the communication party which uses the respective exit node (in particular its IP address). They might therefore be a target for surveillance authorities, at least at first glance.

However, as the IP address of the communications party is disguised on the other side of the communications through the Nym encryption infrastructure, the usual situation, where an IP address or another trace of an Internet user is found in the connection with a criminal activity (e.g., in a web server protocol), and then used in cooperation with the user’s provider to identify the user, is not going to take place.

The same is true for the opposite side: The node operator does not see the communication party of his user.

Experience has shown that Swiss investigative authorities are aware of these limitations and do not conduct investigations against individuals who operate TOR nodes, for example. In one specific case that I know of, the investigation was stopped by the police as soon as it was clear that a TOR node was being operated.

I therefore consider the risk for an exit node operator to become involved in a SPTA proceeding as low.

Nevertheless, in such a situation, exit node operators providers would have to provide the authorities with the information already available to them (Art. 22 Para. 3 SPTA), and they would have to tolerate monitoring by the authorities or by the persons commissioned by the service of the data which the monitored person transmits or stores using derived communications services (Art. 27 Para. 1 SPTA; see above, 5.1.1.2). There is no duty of data retention for providers of derived communication services, though.

The the risk for exit node operators of being upgraded according to Art. 22 Para. 4 SPTA is low to non existent for the reasons mentioned above.

Intelligence Service Law

Operators of exit nodes do not provide wire-based telecommunications services either and therefore do not fall under the IntelSA.

Nym as VPN provider

Telecoms Law

Nym as a VPN operator might be considered a telecommunication provider under the newly revised TCA, as the term now also covers operators of Over-the-Top services which are carried out over the internet.

However I consider possible administrative burdens arising from this qualification as negligible (see above, 5.1.2.1).

Telecoms Surveillance Law

VPN providers have information about the communication party which uses the respective exit node (in particular its IP address). They might therefore be a target for surveillance authorities, at least at first glance.

However, for the same reason I see a risk low for exit node operators to become involved in a SPTA proceeding (the IP address is not visible to the communication partner, which is exactly the reason the Nym VPN is being used at all), I also see a low risk for Nym itself to become involved in such a proceeding (see above, 5.1.3.2).

Intelligence Service Law

VPN operators do not provide wire-based telecommunications services and therefore do not fall under the IntelSA.

EU chat control regulation in particular

According to a EU commission proposal for a regulation laying down rules to prevent and combat child sexual abuse (https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX: 52022PC0209) hosting providers and providers of so-called interpersonal communication services should be obliged to perform an assessment of risks of online child sexual abuse. Additionally an obligation for certain providers should be established to detect such abuse, to report it via the EU Centre, to remove or disable access to, or to block online child sexual abuse material when so ordered.

’Interpersonal communications service’ means a service normally provided for remuneration that enables direct interpersonal and interactive exchange of information via electronic communications networks between a finite number of persons, whereby the persons initiating or participating in the communication determine its recipient(s) and does not include services which enable interpersonal and interactive communication merely as a minor ancillary feature that is intrinsically linked to another service (Art. 2 Point 5 Directive (EU) 2018/1972, which is also relevant for the mentioned proposal).

Interpersonal communications services are services that enable interpersonal and interactive exchange of information. Interactive communication entails that the service allows the recipient of the information to respond. The proposal therefore only covers services like traditional voice calls between two individuals but also all types of emails, messaging services, or group chats. Examples for services which do not meet those requirements are linear broadcasting, video on demand, websites, social networks, blogs, or exchange of information between machines (Directive (EU) 2018/1972, Consideration 17).

Neither the Nym encryption infrastructure nor the NYM VPN are used as means for an interactive exchange of information in the aforementioned sense (of e-mail, messaging, chats or similar).

I therefore consider the risk arising from the mentioned proposal for Nym as low, be it as software developer or VPN operator.

However, an application provider which uses the Nym encryption infrastructure to provide encrypted chat services or similar could still fall under the proposal. This might pose a commercial risk for Nym as the provider of the basic infrastructure for such services, because such services might lose their commercial value for end customers.

Currently the EU decision on chat control has been postponed because there is a blocking minority which can prevent the adoption of the respective parts of the law. In addition, even EU internal lawyers held that the proposal was clearly in violation of the EU charter of fundamental rights and would therefore be nullified by the EU courts in case it would still be enacted by the parliament.

I therefore consider the risk that the mentioned proposal is enacted by the EU authorities and finally upheld by the courts in its planned form as low.

Last change: 2024-04-11, commit: f978552

Legal environment: United States

Info

The entire content of this page is under Creative Commons Attribution 4.0 International Public License.

Warning

The following part is for informational purposes only. Nym core team cannot provide comprehensive legal advice across all jurisdictions. Knowledge and experience with the legalities are being built up with the help of our counsel and with you, the community of Nym node operators. We encourage Nym node operators to join the Node Operator and Operators Legal Forum channels on Element to share best practices and experiences.

Note: The information shared below is in the stage of conclusions upon final confirmation. The text is a not edited exert from a legal counsel. Nym core team is asking for more clarifications.

The US legal counsel have so far provided the following advice:

The legal risk faced by VPN operators subject to United States jurisdiction depends on various statutes and regulations related to privacy, anonymity, and electronic communications. The key areas to consider are: intermediary liability and exceptions, data protection, copyright infringement, export controls, criminal law, government requests for data and assistance, and third party liability.

As outlined in Part A, the United States treats VPNs as telecommunications networks subject to intermediary liability protection from wrongful conduct that occurs on its network. However, such protections do have exceptions including criminal law and copyright claims that are worth considering. In the United States, I am not aware of an individual ever being prosecuted or convicted for running a node for a dVPN or a Privacy Enhancing Network.

However, as discussed in Part B-C, VPN operators are subject to law enforcement requests for access or assistance in obtaining access to data relevant to an investigation into allegedly unlawful conduct that was facilitated by the network as an intermediary. As shown in Part C, governments may also request assistance from node operators for certain high-level and national security targets.

Finally, as outlined in Parts D-G, VPN operators may also be subject to non-criminal liability including (Part D) failing to respond to notices under the DMCA, (Part E) privacy and data protection law, (Part F) third party lawsuits stemming from wrongful acts committed using the network, and (G) export control violations.

Last change: 2024-04-11, commit: f978552

Landing Pages

Info

The entire content of this page is under Creative Commons Attribution 4.0 International Public License.

Nym Node operators are invited to add their legal findings and suggestions directly through pull requests.

Exit Gateway landing page is a great and transparent way to prevent possible troubles by providing people with basic facts, links to laws and regulations on a given topic as well as operator’s contact info. To inspire each other we share some examples how Nym community handled this issue below.

Avril 14th Exit Gateways

The following snippet needs be modified as described below according to the public identity that you may want to show on this public notice, i.e. your graphics and your email. It would allow you to serve it as a landing page resembling the one proposed by Tor but with all the changes needed to adhere to the Nym’s operators case.

Avril 14th is a team of professionals and optimistic realists eager to contribute to a future of privacy, decentralization and connectivity.

When done with the customization needs as described below, you’ll need to upload the file and reference it as the landing_page_assets_path on the [http] section of the config.toml file:

landing_page_assets_path = '<PATH_TO_YOUR_CUSTOM_FILE>'

HTML file customization

  • add your favicon logo on the line:
<link rel="icon" type="image/png" href="">

and your header logo on the line:

<img class="logo" src="<FIXME>">

by either setting the URl to the image (if you’re hosting it publicly, i.e. on your web server)

href="<PATH_TO_YOUR_PUBLIC_URL>"

and

src="<PATH_TO_YOUR_PUBLIC_URL>"

or by adding the image inline as base64 encoded image

href="href="data:image/x-icon;base64,AAABAAMA....""

and

src="href="data:image/x-icon;base64,AAABAAMA....""
  • add the email address you’re willing to use for being contacted.
<a href="mailto:>FIXME>">maintainer</a>
  • If you’re running the node within the US check the sections marked as FIXME, add your DNS name and uncomment those.

<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<title>This is a NYM Exit Gateway</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/png" href="">
<style>
:root {
  font-family: Consolas, "Ubuntu Mono", Menlo, "DejaVu Sans Mono", monospace;
}
:root{
--background-color: #121726;
--text-color: #f2f2f2;
--link-color: #fb6e4e;
}
html{
background: var(--background-color);
}
body{
margin-left: auto;
margin-right: auto;
padding-left: 5vw;
padding-right: 5vw;
max-width: 1000px;
}
h1{
font-size: 55px;
text-align: center;
color: var(--title-color)
}
p{
color: var(--text-color);
}
p, a{
font-size: 20px;
}
a{
color: var(--link-color);
text-decoration: none;
}
a:hover{
filter: brightness(.8);
text-decoration: underline;
}
.links{
display: flex;
flex-wrap: wrap;
justify-content: space-evenly;
}
.links > a{
margin: 10px;
white-space: nowrap;
}
</style>

</head>
<body>
<main>
<h1>This is a NYM Exit Gateway</h1>
<p style="text-align:center">
<img class="logo" src="<FIXME>">
</p>

<p>
You are most likely accessing this website because you've had some issue with
the traffic coming from this IP. This router is part of the <a
href="https://nymtech.net/">NYM project</a>, which is
dedicated to <a href="https://nymtech.net/about/mission">create</a> outstanding
privacy software that is legally compliant without sacrificing integrity or
having any backdoors.
This router IP should be generating no other traffic, unless it has been
compromised.</p>

<p>
The Nym mixnet is operated by a decentralised community of node operators
and stakers. The Nym mixnet is trustless, meaning that no parts of the system
nor its operators have access to information that might compromise the privacy
of users. Nym software enacts a strict principle of data minimisation and has
no back doors. The Nym mixnet works by encrypting packets in several layers
and relaying those through a multi-layered network called a mixnet,  eventually
letting the traffic exit the Nym mixnet through an exit gateway like this one.
This design makes it very hard for a service to know which user is connecting to it,
since it can only see the IP-address of the Nym exit gateway:</p>

<p style="text-align:center;margin:40px 0">
<svg xmlns="http://www.w3.org/2000/svg" width="500" viewBox="0 0 490.28 293.73" style="width:100%;max-width:600px">
<desc>Illustration showing how a user might connect to a service through the Nym network. The user first sends their data through three daisy-chained encrypted Nym nodes that exist on three different continents. Then the last Nym node in the chain connects to the target service over the normal internet.</desc>
<defs>
<style>
.t{
fill: var(--text-color);
stroke: var(--text-color);
}
</style>
</defs>
<path fill="#6fc8b7" d="M257.89 69.4c-6.61-6.36-10.62-7.73-18.36-8.62-7.97-1.83-20.06-7.99-24.17-.67-3.29 5.85-18.2 12.3-16.87 2.08.92-7.03 11.06-13.28 17-17.37 8.69-5.99 24.97-2.87 26.1-10.28 1.04-6.86-8.33-13.22-8.55-2.3-.38 12.84-19.62 2.24-8.73-6.2 8.92-6.9 16.05-9.02 25.61-6.15 12.37 4.83 25.58-2.05 33.73-.71 12.37-2.01 24.69-5.25 37.39-3.96 13 .43 24.08-.14 37.06.63 9.8 1.58 16.5 2.87 26.37 3.6 6.6.48 17.68-.82 24.3 1.9 8.3 4.24.44 10.94-6.89 11.8-8.79 1.05-23.59-1.19-26.6 1.86-5.8 7.41 10.75 5.68 11.27 14.54.57 9.45-5.42 9.38-8.72 16-2.7 4.2.3 13.93-1.18 18.45-1.85 5.64-19.64 4.47-14.7 14.4 4.16 8.34 1.17 19.14-10.33 12.02-5.88-3.65-9.85-22.04-15.66-21.9-11.06.27-11.37 13.18-12.7 17.52-1.3 4.27-3.79 2.33-6-.63-3.54-4.76-7.75-14.22-12.01-17.32-6.12-4.46-10.75-1.17-15.55 2.83-5.63 4.69-8.78 7.82-7.46 16.5.78 9.1-12.9 15.84-14.98 24.09-2.61 10.32-2.57 22.12-8.81 31.47-4 5.98-14.03 20.12-21.27 14.97-7.5-5.34-7.22-14.6-9.56-23.08-2.5-9.02.6-17.35-2.57-26.2-2.45-6.82-6.23-14.54-13.01-13.24-6.5.92-15.08 1.38-19.23-2.97-5.65-5.93-6-10.1-6.61-18.56 1.65-6.94 5.79-12.64 10.38-18.63 3.4-4.42 17.45-10.39 25.26-7.83 10.35 3.38 17.43 10.5 28.95 8.57 3.12-.53 9.14-4.65 7.1-6.62zm-145.6 37.27c-4.96-1.27-11.57 1.13-11.8 6.94-1.48 5.59-4.82 10.62-5.8 16.32.56 6.42 4.34 12.02 8.18 16.97 3.72 3.85 8.58 7.37 9.3 13.1 1.24 5.88 1.6 11.92 2.28 17.87.34 9.37.95 19.67 7.29 27.16 4.26 3.83 8.4-2.15 6.52-6.3-.54-4.54-.6-9.11 1.01-13.27 4.2-6.7 7.32-10.57 12.44-16.64 5.6-7.16 12.74-11.75 14-20.9.56-4.26 5.72-13.86 1.7-16.72-3.14-2.3-15.83-4-18.86-6.49-2.36-1.71-3.86-9.2-9.86-12.07-4.91-3.1-10.28-6.73-16.4-5.97zm11.16-49.42c6.13-2.93 10.58-4.77 14.61-10.25 3.5-4.28 2.46-12.62-2.59-15.45-7.27-3.22-13.08 5.78-18.81 8.71-5.96 4.2-12.07-5.48-6.44-10.6 5.53-4.13.38-9.2-5.66-8.48-6.12.8-12.48-1.45-18.6-1.73-5.3-.7-10.13-1-15.45-1.37-5.37-.05-16.51-2.23-25.13.87-5.42 1.79-12.5 5.3-16.73 9.06-4.85 4.2.2 7.56 5.54 7.45 5.3-.22 16.8-5.36 20.16.98 3.68 8.13-5.82 18.29-5.2 26.69.1 6.2 3.37 11 4.74 16.98 1.62 5.94 6.17 10.45 10 15.14 4.7 5.06 13.06 6.3 19.53 8.23 7.46.14 3.34-9.23 3.01-14.11 1.77-7.15 8.49-7.82 12.68-13.5 7.14-7.72 16.41-13.4 24.34-18.62zM190.88 3.1c-4.69 0-13.33.04-18.17-.34-7.65.12-13.1-.62-19.48-1.09-3.67.39-9.09 3.34-5.28 7.04 3.8.94 7.32 4.92 7.1 9.31 1.32 4.68 1.2 11.96 6.53 13.88 4.76-.2 7.12-7.6 11.93-8.25 6.85-2.05 12.5-4.58 17.87-9.09 2.48-2.76 7.94-6.38 5.26-10.33-1.55-1.31-2.18-.64-5.76-1.13zm178.81 157.37c-2.66 10.08-5.88 24.97 9.4 15.43 7.97-5.72 12.58-2.02 17.47 1.15.5.43 2.65 9.2 7.19 8.53 5.43-2.1 11.55-5.1 14.96-11.2 2.6-4.62 3.6-12.39 2.76-13.22-3.18-3.43-6.24-11.03-7.7-15.1-.76-2.14-2.24-2.6-2.74-.4-2.82 12.85-6.04 1.22-10.12-.05-8.2-1.67-29.62 7.17-31.22 14.86z"/>
<g fill="none">
<path stroke="#cf63a6" stroke-linecap="round" stroke-width="2.76" d="M135.2 140.58c61.4-3.82 115.95-118.83 151.45-103.33"/>
<path stroke="#cf63a6" stroke-linecap="round" stroke-width="2.76" d="M74.43 46.66c38.15 8.21 64.05 42.26 60.78 93.92M286.65 37.25c-9.6 39.44-3.57 57.12-35.64 91.98"/>
<path stroke="#e4c101" stroke-dasharray="9.06,2.265" stroke-width="2.27" d="M397.92 162.52c-31.38 1.26-90.89-53.54-148.3-36.17"/>
<path stroke="#cf63a6" stroke-linecap="round" stroke-width="2.77" d="M17.6 245.88c14.35 0 14.4.05 28-.03"/>
<path stroke="#e3bf01" stroke-dasharray="9.06,2.265" stroke-width="2.27" d="M46.26 274.14c-17.52-.12-16.68.08-30.34.07"/>
</g>
<g transform="translate(120.8 -35.81)">
<circle cx="509.78" cy="68.74" r="18.12" fill="#240a3b" transform="translate(-93.3 38.03) scale(.50637)"/>
<circle cx="440.95" cy="251.87" r="18.12" fill="#240a3b" transform="translate(-93.3 38.03) scale(.50637)"/>
<circle cx="212.62" cy="272.19" r="18.12" fill="#240a3b" transform="translate(-93.3 38.03) scale(.50637)"/>
<circle cx="92.12" cy="87.56" r="18.12" fill="#240a3b" transform="translate(-93.3 38.03) scale(.50637)"/>
<circle cx="730.88" cy="315.83" r="18.12" fill="#67727b" transform="translate(-93.3 38.03) scale(.50637)"/>
<circle cx="-102.85" cy="282.18" r="9.18" fill="#240a3b"/>
<circle cx="-102.85" cy="309.94" r="9.18" fill="#67727b"/>
</g>
<g class="t">
<text xml:space="preserve" x="-24.76" y="10.37" stroke-width=".26" font-size="16.93" font-weight="700" style="line-height:1.25" transform="translate(27.79 2.5)" word-spacing="0"><tspan x="-24.76" y="10.37">The user</tspan></text>
<text xml:space="preserve" x="150.63" y="196.62" stroke-width=".26" font-size="16.93" font-weight="700" style="line-height:1.25" transform="translate(27.79 2.5)" word-spacing="0"><tspan x="150.63" y="196.62">This server</tspan></text>
<text xml:space="preserve" x="346.39" y="202.63" stroke-width=".26" font-size="16.93" font-weight="700" style="line-height:1.25" transform="translate(27.79 2.5)" word-spacing="0"><tspan x="346.39" y="202.63">Your service</tspan></text>
<text xml:space="preserve" x="34.52" y="249.07" stroke-width=".26" font-size="16.93" font-weight="700" style="line-height:1.25" transform="translate(27.79 2.5)" word-spacing="0"><tspan x="34.52" y="249.07">Nym network link</tspan></text>
<text xml:space="preserve" x="34.13" y="276.05" stroke-width=".26" font-size="16.93" font-weight="700" style="line-height:1.25" transform="translate(27.79 2.5)" word-spacing="0"><tspan x="34.13" y="276.05">Unencrypted link</tspan></text>
<path fill="none" stroke-linecap="round" stroke-width="1.67" d="M222.6 184.1c-2.6-15.27 8.95-23.6 18.43-38.86m186.75 45.61c-.68-10.17-9.4-17.68-18.08-23.49"/>
<path fill="none" stroke-linecap="round" stroke-width="1.67" d="M240.99 153.41c.35-3.41 1.19-6.17.04-8.17m-7.15 5.48c1.83-2.8 4.58-4.45 7.15-5.48"/>
<path fill="none" stroke-linecap="round" stroke-width="1.67" d="M412.43 173.21c-2.2-3.15-2.54-3.85-2.73-5.85m0 0c2.46-.65 3.85.01 6.67 1.24M61.62 40.8C48.89 36.98 36.45 27.54 36.9 18.96M61.62 40.8c.05-2.58-3.58-4.8-5.25-5.26m-2.65 6.04c1.8.54 6.8 1.31 7.9-.78"/>
<path fill="none" stroke-linecap="round" stroke-linejoin="round" stroke-width="2.44" d="M1.22 229.4h247.74v63.1H1.22z"/>
</g>
</svg>
</p>

<p>
<a href="https://nymtech.net/about/mixnet">Read more about how Nym works.</a></p>

<p>
Nym relies on a growing ecosystem of users, developers and researcher partners
aligned with the mission to make sure Nym software is running, remains usable
and solves real problems. While Nym is not designed for malicious computer
users, it is true that they can use the network for malicious ends. This
is largely because criminals and hackers have significantly better access to
privacy and anonymity than do the regular users whom they prey upon. Criminals
can and do build, sell, and trade far larger and more powerful networks than
Nym on a daily basis. Thus, in the mind of this operator, the social need for
easily accessible censorship-resistant private, anonymous communication trumps
the risk of unskilled bad actors, who are almost always more easily uncovered
by traditional police work than by extensive monitoring and surveillance anyway.</p>

<p>
In terms of applicable law, the best way to understand Nym is to consider it a
network of routers operating as common carriers, much like the Internet
backbone. However, unlike the Internet backbone routers, Nym mixnodes do not
contain identifiable routing information about the source of a packet and do
mix the user internet traffic with that of other users, making communications
private and protecting not just the user content but the metadata
(user's IP address, who the user talks to, when, where, from what device and
more) and no single Nym node can determine both the origin and destination
of a given transmission.</p>

<p>
As such, there is little the operator of this Exit Gateway can do to help you
track the connection further. This Exit Gateway maintains no logs of any of the
Nym mixnet traffic, so there is little that can be done to trace either legitimate or
illegitimate traffic (or to filter one from the other).  Attempts to
seize this router will accomplish nothing.</p>

<!-- FIXME: US-Only section. Remove if you are a non-US operator -->
<!--
<p>
Furthermore, this machine also serves as a carrier of email, which means that
its contents are further protected under the ECPA. <a
href="https://www.law.cornell.edu/uscode/text/18/2707">18
USC 2707</a> explicitly allows for civil remedies ($1000/account
<i>plus</i>  legal fees)
in the event of a seizure executed without good faith or probable cause (it
should be clear at this point that traffic with an originating IP address of
FIXME_DNS_NAME should not constitute probable cause to seize the
machine). Similar considerations exist for 1st amendment content on this
machine.</p>
-->
<!-- FIXME: May or may not be US-only. Some non-US tor nodes have in
     fact reported DMCA harassment... -->
<!--
<p>
If you are a representative of a company who feels that this router is being
used to violate the DMCA, please be aware that this machine does not host or
contain any illegal content. Also be aware that network infrastructure
maintainers are not liable for the type of content that passes over their
equipment, in accordance with <a
href="https://www.law.cornell.edu/uscode/text/17/512">DMCA
"safe harbor" provisions</a>. In other words, you will have just as much luck
sending a takedown notice to the Internet backbone providers.
</p>
-->

<p>To decentralise and enable privacy for a broad range of services, this
Exit Gateway adopts an <a href="https://nymtech.net/.wellknown/network-requester/exit-policy.txt">Exit Policy</a>
in accordance with the <a href="https://tornull.org/">Tor Null ‘deny’ list</a>
and the <a href="https://tornull.org/tor-reduced-reduced-exit-policy.php">Tor reduced policy</a>,
which are two established safeguards.
</p>

<p>
That being said, if you still have a complaint about the router, you may email the
 <a href="mailto:>FIXME>">maintainer</a>. If complaints are related
 to a particular service that is being abused, the maintainer will submit that to the
 NYM Operators Community in order to add it to the Exit Policy cited above.
If approved, that would prevent this router from allowing that traffic to exit through it.
That can be done only on an IP+destination port basis, however. Common P2P ports are already blocked.</p>

<p>
You also have the option of blocking this IP address and others on the Nym network if you so desire.
 The Nym project provides a <a href="https://explorer.nymtech.net/network-components/gateways">
 web service</a> to fetch a list of all IP addresses of Nym Gateway Exit nodes that allow exiting to a
specified IP:port combination. Please be considerate when using these options.</p>

</main>
</body>
</html>

Reverse proxy

You may set up a reverse proxy in order to serve this landing page with proper SSL and DNS management, i.e. to resolve it to https://nym-exit.<YOUR_DOMAIN> or https://nym-readme.<YOUR_DOMAIN>.

The following assumes that you’re owning a domain and that you’ve already set the Let’s Encrypt certificates on your hosting, and you’ve copied those on your Gateway, i.e. copy the two Let’s Encript pem files on your Gateway’s home folder. Else you may obtain a Let’s Encrypt certificate using a --certonly procedure.

sudo apt install nginx

If you’re running your Gateway (you should…) exposing only the needed ports as listed here, you need to add the Nginx service to your ufw configuration:

sudo ufw app list
sudo ufw allow 'Nginx Full'
sudo ufw reload

Disable the default Nginx landing page

sudo systemctl status nginx
sudo unlink /etc/nginx/sites-enabled/default
sudo systemctl restart nginx

Add your endpoint configuration to Nginx changing <YOUR_DOMAIN> occurrencies below with your domain name, including the top-level domain, i.e. avril4th.org

sudo nano /etc/nginx/sites-available/nym-exit.<YOUR_DOMAIN>
server {
  listen 443 ssl http2;
  listen [::]:443 ssl http2;

  server_name nym-exit.<YOUR_DOMAIN>;

  ssl_certificate <PATH_TO>/fullchain.pem;
  ssl_certificate_key <PATH_TO>/privkey.pem;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  location / {
    proxy_pass http://127.0.0.1:8080;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
}

server {
  listen 80;
  listen [::]:80;

  if ($host = nym-exit.<YOUR_DOMAIN>) {
    return 301 https://$host$request_uri;
  }

  server_name <YOUR_DOMAIN> www.<YOUR_DOMAIN>;

  return 301 https://<YOUR_DOMAIN>$request_uri;
}
sudo ln -s /etc/nginx/sites-available/nym-exit.<YOUR_DOMAIN> /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl restart nginx

When done with the customization needs as described below, you’ll need to upload the file and reference it as the landing_page_assets_path on the [http] section of the config.toml file that you’ll find in the ~/.nym/gateways/<YOUR_GATEWAY_ID>/config/config.toml file

nano ${HOME}/.nym/gateways/<YOUR_GATEWAY_ID>/config/config.toml

change

landing_page_assets_path = '<PATH_TO_YOUR_CUSTOM_FILE>'

If you’re running the Gateway as a service as described here and here you may just restart your service

service nym-gateway stop
service nym-gateway start

and check for the page being served either checking the service logs

sudo journalctl -u  nym-gateway.service | grep 8080

where you should read out

... Started NymNodeHTTPServer on 0.0.0.0:8080

or just point your browser to the URI which you set above, such as https://nym-exit.<YOUR_DOMAIN>

Last change: 2024-04-11, commit: f978552

Adding Content to Legal Forum

Our aim is to establish a strong community network, sharing legal findings and other suggestions with each other. We would like to encourage all of the current and future operators to do research about the situation in the jurisdiction they operate in, to share solutions to the challenges they encountered when running Exit Gateway, and create a pull request (PR).

First of all, please join our Node Operators Legal Forum (Matrix chat) and post any information or questions there.

To add your information to this book, you can create a PR directly to our repository.

Steps to make a pull request:

  1. Write down your legal findings, suggestions, communication templates, FAQ in a text editor

  2. Clone nymtech/nym repository or pull in case you already have it and switch to develop branch

# clone the repository
git clone https://github.com/nymtech/nym

# go to the directory nym
cd nym

# switch to branch develop
git checkout develop

# update the repository
git pull origin develop
  1. Make your own branch based off develop and switch to it
# choose a descriptive and consise name without using <>
git checkout -b operators/legal-forum/<MY_BRANCH_NAME>

# example: git checkout -b operators/legal-forum/alice-vps-faq-template

# you can double check that you are on the right branch
git branch
  1. Save your legal findings as <JURISDICTION_NAME>.md to /nym/documentation/operators/src/legal or add info to an existing page

  2. Don’t change anything in SUMMARY.md, the admins will do it when merging

  3. Add, commit and push your changes

cd documentation/operators/src/legal
git add <FILE_NAME>.md
git commit -am "<DESCRIBE YOUR CHANGES>"
git push origin operators/legal-forum/<MY_BRANCH_NAME>
  1. Open the git generated link in your browser, fill the description and click on Create a Pull Request button
# the url will look like this
 https://github.com/nymtech/nym/pull/new/operators/legal-forum/<MY_BRANCH_NAME>
  1. Notify others in the Node Operators Legal Forum (Matrix chat) about the PR.
Last change: 2024-04-11, commit: f978552

Code of Conduct

We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.

Please avoid using overtly sexual aliases or other nicknames that might detract from a friendly, safe and welcoming environment for all.

Please be kind and courteous. There’s no need to be mean or rude.

Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.

Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.

We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term “harassment” as including the definition in the Citizen Code of Conduct; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don’t tolerate behaviour that excludes people in socially marginalized groups.

Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the channel ops or any of the Rust moderation team immediately. Whether you’re a regular contributor or a newcomer, we care about making this community a safe place for you and we’ve got your back.

Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.

Last change: 2024-04-11, commit: f978552

Licensing

As a general approach, licensing is as follows this pattern:

For accurate information, please check individual files.

Last change: 2024-04-11, commit: f978552