Skip to main content
Version: v1.0.1 (stable)

Mix nodes

note

The Nym mix node binary was built in the building nym section. If you haven't yet built Nym and want to run the code, go there first.

After your build is finished, the nym-mixnode binary will be located in /path/to/nym/target/release/ directory. You may move or copy it to wherever you wish (for example, you may wish to compile your binaries once locally and then move them to different machines).

Alternatively, you can fetch the binaries from our releases page.

Wallet preparation​

Mainnet​

Before you initialise and run your mixnode, head to our website and download the Nym wallet for your operating system. If pre-compiled binaries for your operating system aren't availiable, you can build the wallet yourself with instructions here.

If you don't already have one, please create a Nym address using the wallet, and fund it with tokens. The minimum amount required to bond a mixnode is 100 NYM, but make sure you have a bit more to account for gas costs.

NYM is currently present on several exchanges. Head to our telegram channels to find out where to get NYM tokens.

note

Remember that you can only use Cosmos NYM tokens to bond your mixnode. You cannot use ERC20 representations of NYM to run a node.

Sandbox testnet​

Make sure to download a wallet and create an account as outlined above. Then head to our token faucet and get some tokens to use to bond it.

Viewing command help​

You can check that your binaries are properly compiled with:

./nym-mixnode --help

Which should return a list of all avaliable commands.

console output
nym-mixnode 1.0.1
Nymtech

Build Timestamp:    2022-05-06T13:07:45.000871255+00:00
Build Version:      1.0.1
Commit SHA:         945dda0c24f2f964f27066af320441446973e383
Commit Date:        2022-05-04T15:57:36+00:00
Commit Branch:      detached HEAD
rustc Version:      1.60.0
rustc Channel:      stable
cargo Profile:      release


USAGE:
    nym-mixnode <SUBCOMMAND>

OPTIONS:
    -h, --help
            Print help information

    -V, --version
            Print version information

SUBCOMMANDS:
    describe
            Describe your mixnode and tell people why they should delegate state to you
    help
            Print this message or the help of the given subcommand(s)
    init
            Initialise the mixnode
    node-details
            Show details of this mixnode
    run
            Starts the mixnode
    sign
            Sign text to prove ownership of this mixnode
    upgrade
            Try to upgrade the mixnode

You can also check the various arguments required for individual commands with: 

./nym-mixnode <command> --help

Initialising your mixnode​

To check available configuration options for initializing your node use:

./nym-mixnode init --help
console output
nym-mixnode-init 
Initialise the mixnode

USAGE:
    nym-mixnode init [OPTIONS] --id <ID> --host <HOST> --wallet-address <WALLET_ADDRESS>

OPTIONS:
        --announce-host <ANNOUNCE_HOST>
            The custom host that will be reported to the directory server

    -h, --help
            Print help information

        --host <HOST>
            The host on which the mixnode will be running

        --http-api-port <HTTP_API_PORT>
            The port on which the mixnode will be listening for http requests

        --id <ID>
            Id of the mixnode we want to create config for

        --mix-port <MIX_PORT>
            The port on which the mixnode will be listening for mix packets

        --validators <VALIDATORS>
            Comma separated list of rest endpoints of the validators

        --verloc-port <VERLOC_PORT>
            The port on which the mixnode will be listening for verloc packets

        --wallet-address <WALLET_ADDRESS>
            The wallet address you will use to bond this mixnode, e.g.
            nymt1z9egw0knv47nmur0p8vk4rcx59h9gg4zuxrrr9

Initalise your mixnode with the following command, replacing the value of --id with the moniker you wish to give your mixnode, and the --wallet-address with the Nym address you created earlier:

./nym-mixnode init --id winston-smithnode --host $(curl ifconfig.me) --wallet-address <wallet-address>
caution

Please make sure you have access to the --wallet-address account, and download the Desktop Wallet here if you have not already done so, in order to be able to interact with your node!

Your --host must be publicly routable on the internet in order to mix packets, and can be either an Ipv4 or IPv6 address. Your node must be able to send TCP data using both IPv4 and IPv6 (as other nodes you talk to may use either protocol). The $(curl ifconfig.me) command above returns your IP automatically using an external service. Alternatively, you can enter your IP manually wish. If you do this, remember to enter your IP without any port information.

caution

Please note that the init command will refuse to destroy existing mixnode keys.

During the init process you will have the option to change the http_api, verloc and mixnode ports from their default settings. If you wish to change these in the future you can edit their values in the config.toml file created by the initialization process, which is located at ~/.nym/mixnodes/<your-id>/.

Bonding your mix node​

You can bond your mix node via the Desktop wallet.

Open your wallet, and head to the Bond page, then select the node type and input your node details.

Via the CLI (power users)​

Power users might wish to interact directly with the Mixnet smart contract itself.

You can do this via a call that looks like this via the validator binary. Below is an example command to execute this command on the mainnet: 

nyxd tx wasm execute n14hj2tavq8fpesdwxxcu44rty3hh90vhujrvcmstl4zr3txmfvw9sjyvg3g 
'{"bond_mixnode":{"mix_node":{"host":"HOST", "mix_port":1789, "verloc_port":1790, 
"http_api_port":8000, "sphinx_key":"SPHINX_KEY", "identity_key":"IDENTITY_KEY", 
profit_margin_percent":PROFIT_MARGIN, "version":"1.0.1"}, "owner_signature":"OWNER_SIG"}}' 
--from YOUR_ADDRESS --chain-id nyx --amount 100000000unym

Running your mixnode​

Now you've bonded your mixnode, run it with: 

./nym-mixnode run --id winston-smithnode
console output
Starting mixnode winston-smithnode...

To bond your mixnode you will need to install the Nym wallet, go to https://nymtech.net/get-involved and select the Download button.
Select the correct version and install it to your machine. You will need to provide the following: 

Identity Key: GWrymUuLaxVHSs8iE7YW48MB81npnKjrVuJzJsGkeji6
Sphinx Key: FU89ULkS4YYDXcm5jShhJvoit7H4jG4EXHxRKbS9cXSJ
Owner Signature: Kd5StZtg5PsjLtWRJ5eQejuLHz3JUNzZrk6Jd4WVS5u9Q5bFt6DvuVzN7NbiX9WMZYpsYMJoegH3Bz94o6gsY6b
Host: 62.240.134.46 (bind address: 62.240.134.46)
Version: 1.0.1
Mix Port: 1789, Verloc port: 1790, Http Port: 8000

You are bonding to wallet address: n1x42mm3gsdg808qu2n3ah4l4r9y7vfdvwkw8az6


2022-04-27T16:08:01.159Z INFO  nym_mixnode::node > Starting nym mixnode
2022-04-27T16:08:01.490Z INFO  nym_mixnode::node > Starting node stats controller...
2022-04-27T16:08:01.490Z INFO  nym_mixnode::node > Starting packet delay-forwarder...
2022-04-27T16:08:01.490Z INFO  nym_mixnode::node > Starting socket listener...
2022-04-27T16:08:01.490Z INFO  nym_mixnode::node::listener > Running mix listener on "62.240.134.46:1789"
2022-04-27T16:08:01.490Z INFO  nym_mixnode::node           > Starting the round-trip-time measurer...

If everything worked, you'll see your node running on the either the Sandbox testnet network explorer or the mainnet network explorer, depending on which network you're running.

Note that your node's public identity key is displayed during startup, you can use it to identify your node in the list.

Keep reading to find out more about configuration options or troubleshooting if you're having issues. There are also some tips for running on AWS and other cloud providers, some of which require minor additional setup.

Also have a look at the saved configuration files in $HOME/.nym/mixnodes/ to see more configuration options.

Describe your mixnode (optional)​

In order to easily identify your node via human-readable information later on in the development of the testnet when delegated staking is implemented, you can describe your mixnode with the following command:

./nym-mixnode describe --id winston-smithnode
console output
name: winston-smithnode
description: nym-mixnode hosted on Linode VPS in <location> with the following specs: <specs>.                                
link, e.g. https://mixnode.yourdomain.com: mixnode.mydomain.net
location, e.g. City: London, Country: UK: <your_location>

This information will be shown in the mixnode's page in the Network Explorer, and help people make delegated staking decisions.
caution

Remember to restart your mix node process in order for the new description to be propogated

Displaying mixnode information​

You can always check the details of your mixnode with the node-details command: 

./nym-mixnode node-details --id winston-smithnode
console output
Identity Key: GWrymUuLaxVHSs8iE7YW48MB81npnKjrVuJzJsGkeji6
Sphinx Key: FU89ULkS4YYDXcm5jShhJvoit7H4jG4EXHxRKbS9cXSJ
Owner Signature: Kd5StZtg5PsjLtWRJ5eQejuLHz3JUNzZrk6Jd4WVS5u9Q5bFt6DvuVzN7NbiX9WMZYpsYMJoegH3Bz94o6gsY6b
Host: 62.240.134.46 (bind address: 62.240.134.46)
Version: 1.0.1
Mix Port: 1789, Verloc port: 1790, Http Port: 8000

You are bonding to wallet address: n1x42mm3gsdg808qu2n3ah4l4r9y7vfdvwkw8az6

Configure your firewall​

The following commands will allow you to set up a firewall using ufw.

# check if you have ufw installed
ufw version
# if it is not installed, install with
sudo apt install ufw -y
# enable ufw
sudo ufw enable
# check the status of the firewall
sudo ufw status

Finally open your mixnode's p2p port, as well as ports for ssh, http, and https connections, and ports 8000 and 1790 for verloc and measurement pings:

sudo ufw allow 1789,1790,8000,22,80,443/tcp
# check the status of the firewall
sudo ufw status

For more information about your mixnode's port configuration, check the mixnode port reference table below.

Automating your mixnode with systemd​

It's useful to have the mixnode automatically start at system boot time. Here's a systemd service file to do that:

[Unit]
Description=Nym Mixnode (1.0.1)
StartLimitInterval=350
StartLimitBurst=10

[Service]
User=nym
LimitNOFILE=65536
ExecStart=/home/nym/nym-mixnode run --id mix0100
KillSignal=SIGINT
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

Put the above file onto your system at /etc/systemd/system/nym-mixnode.service.

Change the path in ExecStart to point at your mixnode binary (nym-mixnode), and the User so it is the user you are running as.

If you have built nym on your server, and your username is jetpanther, then the start command might look like this:

ExecStart=/home/jetpanther/nym/target/release/nym-mixnode run --id your-id. Basically, you want the full /path/to/nym-mixnode run --id whatever-your-node-id-is

Then run:

systemctl enable nym-mixnode.service

Start your node:

service nym-mixnode start

This will cause your node to start at system boot time. If you restart your machine, the node will come back up automatically.

You can also do service nym-mixnode stop or service nym-mixnode restart.

Note: if you make any changes to your systemd script after you've enabled it, you will need to run:

systemctl daemon-reload

This lets your operating system know it's ok to reload the service configuration.

Setting the ulimit​

Linux machines limit how many open files a user is allowed to have. This is called a ulimit.

ulimit is 1024 by default on most systems. It needs to be set higher, because mixnodes make and receive a lot of connections to other nodes.

If you see errors such as:

Failed to accept incoming connection - Os { code: 24, kind: Other, message: "Too many open files" }

This means that the operating system is preventing network connections from being made.

Set the ulimit via systemd service file​

Query the ulimit of your mixnode with:

grep -i "open files" /proc/$(ps -A -o pid,cmd|grep nym-mixnode | grep -v grep |head -n 1 | awk '{print $1}')/limits

You'll get back the hard and soft limits, which looks something like this:

Max open files            65536                65536                files

If your output is the same as above, your node will not encounter any ulimit related issues.

However if either value is 1024, you must raise the limit via the systemd service file. Add the line:

LimitNOFILE=65536

Reload the daemon:

systemctl daemon-reload

or execute this as root for system-wide setting of ulimit:

echo "DefaultLimitNOFILE=65535" >> /etc/systemd/system.conf

Reboot your machine and restart your node. When it comes back, use cat /proc/$(pidof nym-mixnode)/limits | grep "Max open files" to make sure the limit has changed to 65535.

Set the ulimit on non-systemd based distributions​

Edit etc/security/conf and add the following lines:

# Example hard limit for max opened files
username        hard nofile 4096
# Example soft limit for max opened files
username        soft nofile 4096

Then reboot your server and restart your mixnode.

Checking that your node is mixing correctly​

Once you've started your mixnode and it connects to the testnet validator, your node will automatically show up in the 'Mixnodes' section of either the Nym Network Explorers:

Enter your identity key to find your node. There are numerous statistics about your node on that page that are useful for checking your uptime history, packets mixed, and any delegations your node may have.

There are also 2 community explorers which have been created by Nodes Guru:

For more details see Troubleshooting FAQ

Virtual IPs and hosting via Google & AWS​

On some services (AWS, Google, etc), the machine's available bind address is not the same as the public IP address. In this case, bind --host to the local machine address returned by ifconfig, but also specify --announce-host with the public IP. Please make sure that you pass the correct, routable --announce-host.

For example, on a Google machine, you may see the following output from the ifconfig command:

ens4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1460
        inet 10.126.5.7  netmask 255.255.255.255  broadcast 0.0.0.0
        ...

The ens4 interface has the IP 10.126.5.7. But this isn't the public IP of the machine, it's the IP of the machine on Google's internal network. Google uses virtual routing, so the public IP of this machine is something else, maybe 36.68.243.18.

nym-mixnode init --host 10.126.5.7, initalises the mixnode, but no packets will be routed because 10.126.5.7 is not on the public internet.

Trying nym-mixnode init --host 36.68.243.18, you'll get back a startup error saying AddrNotAvailable. This is because the mixnode doesn't know how to bind to a host that's not in the output of ifconfig.

The right thing to do in this situation is nym-mixnode init --host 10.126.5.7 --announce-host 36.68.243.18.

This will bind the mixnode to the available host 10.126.5.7, but announce the mixnode's public IP to the directory server as 36.68.243.18. It's up to you as a node operator to ensure that your public and private IPs match up properly.

Mixnode Hardware Specs​

For the moment, we haven't put a great amount of effort into optimizing concurrency to increase throughput. So don't bother provisioning a beastly server with multiple cores.

  • Processors: 2 cores are fine. Get the fastest CPUs you can afford.
  • RAM: Memory requirements are very low - typically a mixnode may use only a few hundred MB of RAM.
  • Disks: The mixnodes require no disk space beyond a few bytes for the configuration files

This will change when we get a chance to start doing performance optimizations in a more serious way. Sphinx packet decryption is CPU-bound, so once we optimise, more fast cores will be better.

Metrics​

Here is an overview of the commands for getting information about a particular node via curl:

EndpointDescriptionCommand
/descriptionReturns the description of the node set with the describe commandcurl <NODE_IP_ADDRESS>:8000/description
/verlocReturns the verloc information of the node, updated every 12 hourscurl <NODE_IP_ADDRESS>:8000/verloc
/reportReturns the most recent node status test reportcurl https://validator.nymtech.net/api/v1/status/mixnode/<NODE_ID>/report
/historyReturns all previous test reportscurl https://validator.nymtech.net/api/v1/status/mixnode/<NODE_ID>/history
/statusReturns the status of the nodecurl https://validator.nymtech.net/api/v1/status/mixnode/<NODE_ID>/status
/reward-estimationReturns various reward estimation statisticscurl https://validator.nymtech.net/api/v1/status/mixnode/<NODE_ID>/reward-estimation
/stake-saturationReturns the stake saturation of the node as a decimal, with 1 being fully saturatedcurl https://validator.nymtech.net/api/v1/status/mixnode/<NODE_ID>/stake-saturation
/core-status-countReturns the amount of times the node has been selected for use in network testscurl https://validator.nymtech.net/api/v1/status/mixnode/<NODE_ID>/core-status-count
note

Replace validator with sandbox-validator in your commands if you're wanting to query the testnet

There are also several endpoints which return information about all mixnodes in the network:

EndpointDescriptionCommand
/blacklistReturns a list of mixnodes that failed connectivity checks in the previous roundcurl https://validator.nymtech.net/api/v1/mixnodes/blacklisted
/activeReturns the active setcurl https://validator.nymtech.net/api/v1/mixnodes/active

Metrics of interest​

Although some of the endpoints return information that is fairly self-explanatory, there are some which are more complex, which are explained in more detail here.

/report​

This endpoint returns different metrics returned regarding your mixnode's uptime and package-mixing capabilities:

  • identity: the identity key of the mixnode.
  • owner: the address of the owner of the mixnode.
  • last_hour: uptime over the last hour as a percentage.
  • last_day: uptime over the last 24 hours as a percentage.
/reward-estimation​

This endpoint returns different metrics returned regarding your mixnode's currently estimated rewards:

  • estimated_total_node_reward: the estimated total reward in uNYM that the mixnode will recieve for this epoch to be split between the operator and the delegator(s), if any.
  • estimated_operator_reward: the estimated total reward in uNYM that the operator will recieve for this epoch.
  • estimated_delegators_reward: the estimated reward in uNYM that will be split between all of the mixnode delegator(s), if any.
  • current_epoch_start: the UNIX timestamp of the beginning of the current epoch.
  • current_epoch_end: the UNIX timestamp of the end of the current epoch.
  • current_epoch_uptime: the uptime of the mixnode for the current epoch, represented as a percentage.
  • as_at: the UNIX timestamp when the metrics information cache was last refreshed.
/core-status-count​

This endpoint returns the number of times that the node has been selected from the rewarded set and had 1000 packets sent to it, before being used by the network monitor to test the rest of the network.

  • identity: the identity key of the mixnode.
  • count: the number of times it has been used for network testing.

Mixnode port reference​

All mixnode-specific port configuration can be found in $HOME/.nym/mixnodes/<your-id>/config/config.toml. If you do edit any port configs, remember to restart your mixnode.

Default portUse
1789Listen for mixnet traffic
1790Listen for VerLoc traffic
8000Metrics http API endpoint
Last updated on