Changelog

This page displays a full list of all the changes during our release cycle from v2024.3-eclipse onwards. Operators can find here the newest updates together with links to relevant documentation. The list is sorted so that the newest changes appear first.

v2024.10-caramello

CHANGELOG.md

  • Backport 4844 and 4845 (#4857)
  • Bugfix/client registration vol2 (#4856)
  • Remove wireguard feature flag and pass runtime enabled flag (#4839)
  • Eliminate cancel unsafe sig awaiting (#4834)
  • added explicit updateable admin to the mixnet contract (#4822)
  • using legacy signing payload in CLI and verifying both variants in contract (#4821)
  • adding ecash contract address (#4819)
  • Check profit margin of node before defaulting to hardcoded value (#4802)
  • Sync last_seen_bandwidth immediately (#4774)
  • Feature/additional ecash nym cli utils (#4773)
  • Better storage error logging (#4772)
  • bugfix: make sure DKG parses data out of events if logs are empty (#4764)
  • Fix clippy on rustc beta toolchain (#4746)
  • Fix clippy for beta toolchain (#4742)
  • Disable testnet-manager on non-unix (#4741)
  • Don’t set NYM_VPN_API to default (#4740)
  • Update publish-nym-binaries.yml (#4739)
  • Update ci-build-upload-binaries.yml (#4738)
  • Add NYM_VPN_API to network config (#4736)
  • Re-export RecipientFormattingError in nym sdk (#4735)
  • Persist wireguard peers (#4732)
  • Fix tokio error in 1.39 (#4730)
  • Feature/vesting purge plus ranged cost params (#4716)
  • Fix (some) feature unification build failures (#4681)
  • Feature Compact Ecash : The One PR (#4623)

Features

Testing steps performed

Scenario 1: Bandwidth Decreasing Continuously

  1. Started the client and noted the initial bandwidth (e.g., 1GB).
  2. Used the client and tracked bandwidth usage over time (e.g., decrease by 100MB every hour).
  3. Restarted the client after some usage.
  4. Verified the bandwidth continued from the last recorded value, not reset.

The bandwidth continued decreasing without resetting upon restart. Logs and reports correctly reflected the decreasing bandwidth.

Scenario 2: Bandwidth Reset Next Day

  1. Used the client normally until the end of the day.
  2. Suspended some clients and kept others active.
  3. Checked bandwidth at midnight.
  4. Verified that bandwidth reset to 1GB for both suspended and active clients.

Bandwidth reset to 1GB for all clients at midnight. Logs and reports correctly showed the reset.

Scenario 3: Bandwidth Reset at a Different Time (e.g., Midday)

  1. Configured the system to reset bandwidth at midday.
  2. Used the client and monitored bandwidth until midday.
  3. Kept the client connected during the reset time.
  4. Verified that bandwidth reset to 1GB live at midday.

Bandwidth reset to 1GB at midday while the client was connected. Logs and reports correctly reflected the reset.

Scenario 4: Stale Check for 3 Days

  1. Kept a client inactive for 3 days.
  2. Verified removal from the peer list after 3 days.
  3. Reconnected the client after 3 days and checked for a new private IP.
  4. Restarted a client within 3 days and verified it retained the same private IP.

The client was removed from the peer list after 3 days of inactivity. Upon re-connection after 3 days, the client received a new private IP. The client retained the same private IP when restarted within 3 days.

Testing steps performed

  • Verify that the nym-gateway binary and nym-mixnode binary commands return the error message stating to update to nym-node
  • Check that when adding the --force-run flag, it still allows the command to be run (aside from init which has been removed) and the message stating to update to nym-node is a warning now
  • Check nym-node is not affected
  • Reviewed the changes in the PR
  • Handle clients with different versions in IPR: Allow the IPR to handle clients connecting both using v6 and v7, independently. The motivation is that we want to be able to roll out a API version change gradually for VPN clients without breaking backwards compatibility. The main feature on the new v7 format that is not yet used, is that it adds signatures for connect/disconnect.

Testing steps performed

Run the same command (using same gateways deployed from this PR) on different versions of the nym-vpn-cli.

Example:

~/nym-vpn-core-v0.1.10_macos_universal ❯ sudo -E ./nym-vpn-cli -c ../qa.env run --entry-gateway-id $entry_gateway --exit-gateway-id $exit_gateway --enable-two-hop
 
~/nym-vpn-core-v0.1.11-dev_macos_universal ❯ sudo -E ./nym-vpn-cli -c ../qa.env run --entry-gateway-id $entry_gateway --exit-gateway-id $exit_gateway --enable-two-hop

Bugfix

Testing steps performed

  • Building all binaries is ok
  • Running cargo fmt returns no issues

Testing steps performed

Tested updating an old nym-node version and ensuring it did not throw any errors.

The script

#!/bin/bash

packages=$(cargo metadata --format-version 1 --no-deps | jq -r '.packages[].name')

# Loop through each package and build
for package in $packages; do
    echo "Building $package"
    cargo clean
    cargo check -p "$package"
    if [ $? -ne 0 ]; then
        echo "Build failed for $package. Stopping."
        exit 1
    fi
done

Crypto

Operators Guide, Tooling & Updates

  • Minimum profit margin = 20%
  • Maximum profit margin = 50%
  • Minimum operating cost = 0 NYM
  • Maximum operating cost = 1000 NYM
  • Nym Harbourmater has several new functionalities:

    • Version counting graph for Gateways and Mixnodes
    • Several new columns with larger nodes performance and settings overview.
    • Top routing score now includes:
  • New Nym Wallet is out!

    • Vesting contract functionalities have been purged, users can only remove tokens from vesting
    • Migrating from mixnode or gateway smart contracts to a new unifying nym-node smart contract will be available soon using Nym desktop wallet, just like you are used to for bonding and node settings. After this migration all nym-nodes will be able to receive delegation and rewards. We will share a step by step guide once this migration will be deployed. No action needed now.
  • Nym API Check CLI is upgraded according to the latest API endpoints, output is cleaner and more concise.

Operators Tasks

The steps below are highly recommended for all operators and mandatory for everyone who is a part of Nym Delegation or Grant program. Deadline is Friday, September 20th, 2024.

Every nym-node should be upgraded to the latest version! Operators can test using Sandbox env during the pre-release period, then upgrade on mainnet. During the upgrade, please follow the points below before you restart the node:

nym-node

  • Make sure to fill in basic description info, into the file located at .nym/nym-nodes/<ID>/data/description.toml (all nodes)
  • Configure wireguard routing with new network_tunnel_manager.sh following these steps (Gateways only for the time being)
  • Enable Wireguard with --wireguard-enabled true flag included in your run command (Gateways only for the time being)
    • Note: On some VPS this setup may not be enough to get the correct results as some ISPs have their own security groups setup below the individual VPS. In that case a ticket to ISP will have to be issued to open the needed settings. We are working on a template for such ticket.
  • Setup reverse proxy and WSS on nym-node (Gateways only for the time being)
  • Don’t forget to restart your node - or (preferably using systemd automation) reload daemon and restart the service
  • Optional: Use nym-gateway-probe and NymVPN CLI to test your own Gateway
  • Optional: Run the script below to measure ping speed of your Gateway and share your results in Nym Operators channel

The script to measure Gateway ping results

We made a script for pinging nymtech.net from your GWs. Can you please install it and then share the result together with your Gateway ID:

  1. Get the script onto your machine (soon on github for curl or wget):
# paste all this block as one command
cat <<'EOL' > ping_with_curl_average_for_wg_check.sh
#!/bin/bash

ping_with_curl_average_for_wg_check() {
    total_connect_time=0
    total_total_time=0
    iterations=5
    timeout=2

    for ((i=1; i<=iterations; i++)); do
        echo "ping attempt $i..."

        echo "curling nymtech.net to check ping response times"
        times=$(curl -I https://nymtech.net --max-time $timeout \
        -w "time_connect=%{time_connect}\ntime_total=%{time_total}" -o /dev/null -s)

        time_connect=$(echo "$times" | grep "time_connect" | cut -d"=" -f2)
        time_total=$(echo "$times" | grep "time_total" | cut -d"=" -f2)

        total_connect_time=$(echo "$total_connect_time + $time_connect" | bc)
        total_total_time=$(echo "$total_total_time + $time_total" | bc)

        echo "time to connect: $time_connect s"
        echo "total time: $time_total s"
    done

    average_connect_time=$(echo "scale=3; $total_connect_time / $iterations" | bc)
    average_total_time=$(echo "scale=3; $total_total_time / $iterations" | bc)

    echo "-----------------------------------"
    echo "average time to connect: $average_connect_time s"
    echo "average total time: $average_total_time s"
}

ping_with_curl_average_for_wg_check
EOL
  1. Make executable:
chmod +x ping_with_curl_average_for_wg_check.sh
  1. In case you don’t have bc, install it:
sudo apt install bc
  1. Run:
./ping_with_curl_average_for_wg_check.sh
  1. Share results and ID key in Nym Operators channel

THANK YOU!

validators

  • Validators need to update and prepare for ecash implementation.

Known Bugs & Undone features

  • New nym-nodes without a performance 24h history above 50% don’t show routing properly on nym-gateway-probe, on Nym Harbourmaster the page may appear blank - we are working on a fix.
  • Wireguard works on IPv4 only for the time being, we are working on IPv6 implementation.
  • Harbourmaster Role column shows nym-node --mode exit-gateway as EntryGateway, we are working to fix it.
  • In rare occassions Harbourmaster shows only “panda” without the “smiley” badge even for nodes, which have T&C’s accepted. We are working to fix it.
  • Sometimes nym-node running with --wireguard-enabled true gives this error on restart: Serialized netlink packet .. larger than maximum size ..
/home/ubuntu/.cargo/registry/src/index.crates.io-6f17d22bba15001f/defguard_wireguard_rs-0.4.2/src/netlink.rs:155: Serialized netlink packet (23240 bytes) larger than maximum size 12288: NetlinkMessage.

From what we found out it seems that one of our dependencies - DefGuard - is failing. Based on the reading on their fix, it seems that when node operators try to re-create a wireguard interface with too many previous peers (like on Gateway restart, with restoring from storage), there’s an overflow. So their fix is to just add them one by one. To be sure that bumping the dependency version fixes the problem there’s still two things we’d need to check - and your feedback would help us a lot:

  1. Did operators only encounter this error after a nym-node (Gateway) restart?
  2. Reprouce this error ourselves and see if it actually fixes our problem.

Please share your experience with us to help faster fix of this issue.


v2024.9-topdeck

CHANGELOG.md

  • chore: fix 1.80 lint issues (#4731)
  • Handle clients with different versions in IPR (#4723)
  • Add 1GB/day/user bandwidth cap (#4717)
  • Feature/merge back (#4710)
  • removed mixnode/gateway config migration code and disabled cli without explicit flag (#4706)

Features

Testing steps performed

  • Verify that the nym-gateway binary and nym-mixnode binary commands return the _error message_ stating to update to nym-node
  • Check that when adding the --force-run flag, it still allows the command to be run (aside from init which has been removed) and the message stating to update to nym-node is a _warning_ now
  • Check nym-node is not affected
  • Review the changes in the PR

Testing steps performed - Scenario 1: Bandwidth Decreasing Continuously

  1. Start the client and noted the initial bandwidth (e.g., 1GB).
  2. Us the client and track bandwidth usage over time (e.g., decrease by 100MB every hour).
  3. Restart the client after some usage.
  4. Verify the bandwidth continued from the last recorded value, not reset.

Notes: The bandwidth continued decreasing without resetting upon restart. Logs and reports correctly reflected the decreasing bandwidth.

Testing steps performed - Scenario 2: Bandwidth Reset Next Day

  1. Use the client normally until the end of the day.
  2. Suspend some clients and kept others active.
  3. Check bandwidth at midnight.
  4. Verify that bandwidth reset to 1GB for both suspended and active clients.

Notes: Bandwidth reset to 1GB for all clients at midnight. Logs and reports correctly showed the reset.

Testing steps performed - Scenario 3: Bandwidth Reset at a Different Time (e.g., Midday)

  1. Configure the system to reset bandwidth at midday.
  2. Use the client and monitored bandwidth until midday.
  3. Keep the client connected during the reset time.
  4. Verify that bandwidth reset to 1GB live at midday.

Notes: Bandwidth reset to 1GB at midday while the client was connected. Logs and reports correctly reflected the reset.

  • Handle clients with different versions in IPR: Allow the IPR to handle clients connecting both using v6 and v7, independently. The motivation is that we want to be able to roll out an API version change gradually for NymVPN clients without breaking backwards compatibility. The main feature on the new v7 format that is not yet used, is that it adds signatures for connect/disconnect.

Testing steps performed

Run the same command (using same gateways deployed from this PR) on different versions of the nym-vpn-cli.

Example:

sudo -E ./nym-vpn-cli -c ../qa.env run --entry-gateway-id $entry_gateway --exit-gateway-id $exit_gateway --enable-two-hop
 
sudo -E ./nym-vpn-cli -c ../qa.env run --entry-gateway-id $entry_gateway --exit-gateway-id $exit_gateway --enable-two-hop

Bugfix

Testing steps performed

  • Tested updating an old nym-node version and ensuring it did not throw any errors.

Testing steps performed

  • Building all binaries is ok
  • Running cargo fmt returns no issues

Operators Guide updates

  • WireGuard tunnel configuration guide for nym-node (currently Gateways functionalities). For simplicity we made a detailed step by step guide to upgrade an existing nym-node to the latest version and configure your VPS routing for WireGuard. Open by clicking on the example block below.

Upgrading nym-node with WG

Prerequisites

  • Nym Node Version: You must be running the 2024.9-topdeck release branch, which operates as nym-node version 1.1.6. You can find the release here: Nym 2024.9-topdeck Release.

  • Important: Before proceeding, make sure to back up your current nym-node configuration to avoid any potential data loss or issues.

  • Download Nym Node:

    • You can download the nym-node binary directly using the following command:
curl -L https://github.com/nymtech/nym/releases/download/nym-binaries-v2024.9-topdeck/nym-node -o nym-node && chmod u+x nym-node

Step 1: Update UFW Firewall Rules

  • Warning: Enabling the firewall with UFW without allowing SSH port 22 first will lead to losing access over SSH. Make sure port 22 is allowed before proceeding with any UFW configurations.

Run the following as root or with sudo prefix:

  1. Check the current status of UFW (Uncomplicated Firewall):
ufw status
  1. Ensure that the following ports are allowed on your machine before adding the WireGuard port:
ufw allow 22/tcp    # SSH - you're in control of these ports
ufw allow 80/tcp    # HTTP
ufw allow 443/tcp   # HTTPS
ufw allow 1789/tcp  # Nym specific
ufw allow 1790/tcp  # Nym specific
ufw allow 8080/tcp  # Nym specific - nym-node-api
ufw allow 9000/tcp  # Nym Specific - clients port
ufw allow 9001/tcp  # Nym specific - wss port 
ufw allow 51822/udp # WireGuard
  1. Confirm that the UFW rules have been updated:
ufw status

Step 2: Download and Prepare the Network Tunnel Manager Script

  1. Download the network_tunnel_manager.sh script:
curl -L -o network_tunnel_manager.sh https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/3c0a38c1416f8fdf22906c013299dd08d1497183/network_tunnel_manager.sh
  1. Make the script executable:
chmod u+x network_tunnel_manager.sh
  1. Apply the WireGuard IPTables rules:
./network_tunnel_manager.sh apply_iptables_rules_wg

Step 3: Update the Nym Node Service File

  1. Modify your nym-node service file to enable WireGuard. Open the file (usually located at /etc/systemd/system/nym-node.service) and update the [Service] section as follows:
[Service]
User=<YOUR_USER_NAME>
Type=simple
#Environment=RUST_LOG=debug
# CAHNGE PATH IF YOU DON'T RUN IT FROM ROOT HOME DIRECTORY
ExecStart=/root/nym-node run --mode exit-gateway --id <YOUR_NODE_LOCAL_ID> --accept-operator-terms-and-conditions --wireguard-enabled true
Restart=on-failure
RestartSec=30
StartLimitInterval=350
StartLimitBurst=10
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

# ADD OR TWEAK ANY CUSTOM SETTINGS
  1. Reload the systemd daemon to apply the changes:
systemctl daemon-reload
  1. Restart the nym-node service:
systemctl restart nym-node.service
  1. Optionally, you can check if the node is running correctly by monitoring the service logs:
journalctl -u nym-node.service -f -n 100

Step 4: Run the Network Tunnel Manager Script

Finally, run the following command to initiate our favorite routing test - run the joke through the WireGuard tunnel:

./network_tunnel_manager.sh joke_through_wg_tunnel
  • Note: Wireguard will return only IPv4 joke, not IPv6. WG IPv6 is under development. Running IPR joke through the mixnet with ./network_tunnel_manager.sh joke_through_the_mixnet should work with both IPv4 and IPv6!

Syntax to run nym-node with WG enabled

For Exit Gateway:

./nym-node run --id <ID> --mode exit-gateway --public-ips "$(curl -4 https://ifconfig.me)" --hostname "<YOUR_DOMAIN>" --http-bind-address 0.0.0.0:8080 --mixnet-bind-address 0.0.0.0:1789 --location <COUNTRY_FULL_NAME> --accept-operator-terms-and-conditions --wireguard-enabled true

# <YOUR_DOMAIN> is in format without 'https://' prefix
# <COUNTRY_FULL_NAME> is format like 'Jamaica',  or two-letter alpha2 (e.g. 'JM'), three-letter alpha3 (e.g. 'JAM') or three-digit numeric-3 (e.g. '388') can be provided.
# wireguard can be enabled from version 1.1.6 onwards

For Entry Gateway:

./nym-node run --id <ID> --mode entry-gateway --public-ips "$(curl -4 https://ifconfig.me)" --hostname "<YOUR_DOMAIN>" --http-bind-address 0.0.0.0:8080 --mixnet-bind-address 0.0.0.0:1789 --accept-operator-terms-and-conditions --wireguard-enabled true

# <YOUR_DOMAIN> is in format without 'https://' prefix
# <COUNTRY_FULL_NAME> is format like 'Jamaica',  or two-letter alpha2 (e.g. 'JM'), three-letter alpha3 (e.g. 'JAM') or three-digit numeric-3 (e.g. '388') can be provided.
# wireguard can be enabled from version 1.1.6 onwards
  • Update Nym exit policy: Based on the survey, AMA and following discussions we added several ports to Nym exit policy. The ports voted upon in the forum governance have not been added yet due to the concerns raised. These ports were unrestricted:

Newly opened ports in Nym exit policy

22 # SSH
123 # NTP
445 # SMB file share Windows
465 # URD for SSM
587 # SMTP
853 # DNS over TLS
1433 # databases
1521 # databases
2049 # NFS
3074 # Xbox Live
3306 # databases
5000-5005 # RTP / VoIP
5432 # databases
6543 # databases
8080 # HTTP Proxies
8767 # TeamSpeak
8883 # Secure MQ Telemetry Transport - MQTT over SSL
9053 # Tari
9339 # gaming
9443 # alternative HTTPS
9735 # Lightning
25565 # Minecraft
27000-27050 # Steam and game servers
60000-61000 # MOSH

Testing steps performed

  • Use deadlinkchecker.com to go over nymtech.net and correct all docs URLs
  • Go over search engines and old medium articles and check that all dead URLs re-directing correctly

snap of binding addresses and ports in config.toml

[host]
public_ips = [
'<YOUR_PUBLIC_IPv4>'
]

[mixnet]
bind_address = '0.0.0.0:1789'

[http]
bind_address = '0.0.0.0:8080'

[mixnode]
[mixnode.verloc]
bind_address = '0.0.0.0:1790'

[entry_gateway]
bind_address = '0.0.0.0:9000'

Tooling

  • Nym Harbourmaster has now several new functionalities:

    • Tab for Mixnodes
    • Tab with Charts
    • New columns with: Moniker (node description), DP delegatee, Accepted T&Cs - also part of a new category 🐼😀
  • Nym has a new Token page


v2024.8-wispa

CHANGELOG.md

  • add event parsing to support cosmos_sdk > 0.50 (#4697)
  • Fix NR config compatibility (#4690)
  • Remove UserAgent constructor since it’s weakly typed (#4689)
  • [bugfix]: Node_api_check CLI looked over roles on blacklisted nodes (#4687)
  • Add mixnodes to self describing api cache (#4684)
  • Move and whole bump of crates to workspace and upgrade some (#4680)
  • Remove code that refers to removed nym-network-statistics (#4679)
  • Remove nym-network-statistics (#4678)
  • Create UserAgent that can be passed from the binary to the nym api client (#4677)
  • Add authenticator (#4667)

Features

Testing steps performed

  1. Reviewed the changes in the nym-api-requests/src/models.rs file.
  2. Verified that the NymNodeDescription struct includes the new role field with a default value set by default_node_role.
  3. Checked the implementation of the default_node_role function to ensure it returns NodeRole::Inactive.
  4. Ran the updated code in the sandbox environment.
  5. Monitored the sandbox environment for any issues or errors related to the changes.

Notes (if any): The test was successful. No issues were flagged during the testing in the sandbox environment. The new default value for NodeRole ensures backward compatibility without causing disruptions.

Testing steps performed

  1. Reviewed the changes in the gateway/src/error.rs and gateway/src/node/mod.rs files.
  2. Verified the new error enum AuthenticatorStartupFailure was added to GatewayError.
  3. Confirmed the implementation of the StartedAuthenticator struct and its usage in the start_authenticator function.
  4. Ran the updated code in the canary environment.
  5. Monitored the canary environment for any issues or errors related to the changes.

Testing steps performed

  1. Reviewed the changes in common/client-libs/validator-client/src/nyxd/cosmwasm_client/client_traits/signing_client.rs, logs.rs, types.rs, and nym-api/src/coconut/tests/mod.rs files.
  2. Verified the addition of event parsing in the relevant functions and structs.
  3. Ensured that the find_attribute function correctly parses event attributes.
  4. Ran the updated code in the sandbox environment.
  5. Broadcasted transactions on the sandbox network to test the changes.
  6. Monitored the sandbox network for any malformed responses or errors after the test chain upgrade.

Testing steps performed

  1. Reviewed the changes in common/bandwidth-controller/src/event.rs, common/bandwidth-controller/src/lib.rs, and common/client-libs/gateway-client/src/client.rs files.
  2. Verified the implementation of BandwidthStatusMessage enum for emitting status messages.
  3. Ensured GatewayClient is updated to send bandwidth status messages when connecting.
  4. Deployed the updated code on the canary environment.
  5. Connected to the gateway and checked for the emission of bandwidth status messages.
  6. Verified that the messages were correctly parsed and consumed by the application layer.
  7. Ran the VPN client to observe the parsed events.
  • Fix NR config compatibility: Recently we deleted the old statistics service provider. This fixes some issues where old configs didn’t work with the latest changes.
    • Make NR able to read config with old keys in
    • Remove deleted config keys from NR template

Testing steps performed

  1. Reviewed the changes in the service-providers/network-requester/src/config/mod.rs and service-providers/network-requester/src/config/template.rs files.
  2. Ensured NetworkRequester config is able to read old keys for compatibility.
  3. Removed old and deleted config keys from the NetworkRequester template.
  4. Compiled the project to verify no issues or warnings appeared.
  5. Ran all tests to ensure that the changes did not affect the functionality.
  6. Validated that no leftover code from the old statistics service provider caused any issues.

Testing steps performed

  1. Reviewed the changes in common/http-api-client/src/user_agent.rs file.
  2. Verified the removal of the UserAgent constructor and ensured that all instances of UserAgent::new are updated accordingly.
  3. Checked the implementation of UserAgent struct using BinaryBuildInformation and BinaryBuildInformationOwned.
  4. Deployed the updated code across different environments (QA, sandbox, and canary).
  5. Ran tests to ensure that the UserAgent struct functions correctly without the constructor.
  • Add mixnodes to self describing api cache:
    • Abstracts getting the self describing info a bit
    • Adds mixnodes to the cache refresher as well
    • Adds role field to the NodeDescription struct, to be able to distinguish between mixnodes and gateways
    • Switched to using NodeStatusCache instead of ContractCache

Testing steps performed

Called the new /mixnodes/described endpoint as well as the existing /gateways/described endpoint and verified that the data returned for each was correct based on the settings that different nodes have when they are setup.

For gateway endpoint, the “role” for now does not differentiate between entry and exit gateways, this will be implemented in the future.

  • Move and whole bump of crates to workspace and upgrade some:
    • Fix cargo warning for default_features
    • Move dirs 4.0 to workspace
    • Use workspace base64 dep
    • Move rand_chacha and x25519-dalek to workspace
    • Use workspace ed25519-dalek dep
    • Move itertools to workspace deps and upgrade
    • Move a few partial deps to workspace while preserving versions

Testing steps performed

  1. Reviewed the changes to move and upgrade crates to the workspace.
  2. Verified the updated dependencies:
    • Moved dirs to version 4.0 in the workspace.
    • Updated the base64 dependency to use the workspace version.
    • Moved rand_chacha and x25519-dalek to the workspace.
    • Updated ed25519-dalek to use the workspace version.
    • Moved and upgraded itertools in the workspace.
    • Moved other partial dependencies to the workspace while preserving their versions.
  3. Ensured the Cargo.toml files across the project reflect these changes correctly.
  4. Compiled the entire project to check for any issues or warnings.
  5. Verified that all tests pass successfully after the changes.

Testing steps performed

  1. Reviewed the project to identify all references to nym-network-statistics.
  2. Removed all code and dependencies associated with nym-network-statistics.
  3. Ensured that no references to nym-network-statistics remain in the codebase, including comments, imports, and configuration files.
  4. Compiled the project to check for any issues or warnings.
  5. Ran all tests to ensure the removal did not affect the functionality of the project.

Testing steps performed

  1. Reviewed the project to identify all references to nym-network-statistics.
  2. Removed all code and dependencies associated with nym-network-statistics.
  3. Ensured that no references to nym-network-statistics remain in the codebase, including comments, imports, and configuration files.
  4. Compiled the project to check for any issues or warnings.
  5. Ran all tests to ensure the removal did not affect the functionality of the project.

Testing steps performed

Used the nym-vpn-cli to test this, and we can visibly see the UserAgent, no issues with the comments mentioned above.

Example of the user agent sent: nym-client/1.1.36/x86_64-unknown-linux-gnu/e18bb70

image

Connected with no problems

Bugfix

  • Node_api_check.py CLI looked over roles on blacklisted nodes: Removing/correcting this redundant function which results in unwanted error print, will resolve in the program not looking up the roles endpoint for blacklisted GWs, instead just ignores the role description and still return all other endpoints.

Operators Guide updates

snap of binding addresses and ports in config.toml

[host]
public_ips = [
'<YOUR_PUBLIC_IPv4>'
]

[mixnet]
bind_address = '0.0.0.0:1789'

[http]
bind_address = '0.0.0.0:8080'

[mixnode]
[mixnode.verloc]
bind_address = '0.0.0.0:1790'

[entry_gateway]
bind_address = '0.0.0.0:9000'

v2024.7-doubledecker

CHANGELOG.md

  • Add an early return in parse_raw_str_logs for empty raw log strings. (#4686)
  • Bump braces from 3.0.2 to 3.0.3 in /wasm/mix-fetch/internal-dev (#4672)
  • add expiry returned on import (#4670)
  • [bugfix] missing rustls feature (#4666)
  • Bump ws from 8.13.0 to 8.17.1 in /wasm/client/internal-dev-node (#4665)
  • Bump braces from 3.0.2 to 3.0.3 in /clients/native/examples/js-examples/websocket (#4663)
  • Bump ws from 8.14.2 to 8.17.1 in /sdk/typescript/packages/nodejs-client (#4662)
  • Update setup.md (#4661)
  • New clippy lints (#4660)
  • Bump braces from 3.0.2 to 3.0.3 in /nym-api/tests (#4659)
  • Bump braces from 3.0.2 to 3.0.3 in /docker/typescript_client/upload_contract (#4658)
  • Update vps-setup.md (#4656)
  • Update configuration.md (#4655)
  • Remove old PR template (#4639)

Features

Bugfix

  • missing rustls feature: It just happens to work due to feature-unification. It should probably have this feature inbuild.

Operators Guide updates


v2024.6-chomp

CHANGELOG.md

  • Remove additional code as part of Ephemera Purge and SP and contracts (#4650)
  • bugfix: make sure nym-api can handle non-cw2 (or without detailed build info) compliant contracts (#4648)
  • introduced a flag to accept toc and exposed it via self-described API (#4647)
  • bugfix: make sure to return an error on invalid public ip (#4646)
  • Add ci check for PR having an assigned milestone (#4644)
  • Removed ephemera code (#4642)
  • Remove stale peers (#4640)
  • Add generic wg private network routing (#4636)
  • Feature/new node endpoints (#4635)
  • standarised ContractBuildInformation and added it to all contracts (#4631)
  • validate nym-node public ips on startup (#4630)
  • Bump defguard wg (#4625)
  • Fix cargo warnings (#4624)
  • Update kernel peers on peer modification (#4622)
  • Handle v6 and v7 requests in the IPR, but reply with v6 (#4620)
  • fix typo (#4619)
  • Update crypto and rand crates (#4607)
  • Purge name service and service provider directory contracts (#4603)

Features

Testing steps performed

  • Verify that the execute function correctly checks if the accept_operator_terms flag is set.
  • Test that a warning is displayed when the accept_operator_terms flag is not set.
  • Confirm that the NymNode instance is initialized with with_accepted_toc(accepted_toc) when the flag is set.
  • Apply the --accept-toc flag in the service and confirmed the change by running:
curl -s -X 'GET' 'http://18.171.251.41:8080/api/v1/auxiliary-details?output=json' -H 'accept: application/json' | jq .accepted_toc
  • Verify that the output is true.

Testing steps performed

  • Use the latest release/chomp binary with nym-node and input a dodgy ip image

  • Validation: image When restarting the node it complains within the service launch file

  • New node endpoints: introduces new endpoints on nym-api (and creates scaffolding for additional ones) for providing unfiltered network topology alongside performance score of all nodes.
    • NymApiTopologyProvider got modified to use those endpoints alongside (configurable) filtering of nodes with score < 50% (like our current blacklist)
    • Old clients should work as before as no existing endpoint got removed

Testing steps performed

  • Validate that the skimmed endpoints are working, keeping in mind that they are unstable. The full-fat and semi-skimmed have not yet been implemented.

Testing steps performed

  • Check references to everything named SP and Ephemera and removed any additional references

Testing steps performed

  • Check references to everything named SP and Ephemera and removed any additional references

Testing steps performed

  • CI complains if no milestone is associated with the an issue.
  • Bump defguard wireguard
  • Add generic wireguard private network routing: as defguard wireguard only allows for peer routing modifications, we will configure the entire wireguard private network to be routed to the wg device. Configuring per peer is also not desirable, as the interface doesn’t allow removing routes, so unused ip routing won’t be cleaned until gateway restart (and it would also pollute to routing table with a lot of rules when many peers are added).

Testing steps performed

  • This is a part of a bigger ticket, but initial testing has proven to shown that launching nym-nodes (entry and exit gateways) in WG enable mode to be working

QA will use this template for the other related WG tickets in this release milestone.

- [Standarise `ContractBuildInformation` and add it to all contracts](https://github.com/nymtech/nym/pull/4631): Similarly to `cw2`, we're now saving `ContractBuildInformation` under a constant storage key, i.e. `b"contract_build_info"` that standarises the retrieval by nym-api. - Also each of our contracts now saves and updates that information upon init and migration.

Testing steps performed

  • Use the latest release/chomp contracts and deploy these to QA
  • Use the nym-api to query for the results of these new contracts
 curl -X 'GET' \
   'https://qa-nym-api.qa.nymte.ch/api/v1/network/nym-contracts-detailed' \
   -H 'accept: application/json'
  • It returns a detailed view of the contracts and which branch they were built from, alongside rust versions and so forth. image

Testing steps performed

  • This is a part of a bigger ticket, but initial testing has proven to shown that launching nym-nodes (entry and exit gateways) in WG enable mode to be working. QA will use this template for the other related WG tickets in this release milestone.
  • Handle v6 and v7 requests in the IPR, but reply with v6: teach the IPR to read both v6 and v7 requests, but always reply with v6. This is to prepare for bumping to v7 and signed connect/disconnect messages. Follow up PRs will add
    • Verify signature
    • Send v7 in client with signatures included
  • Purge name service and service provider directory contracts: this is a compiler assisted purge of the nym-name-service and nym-service-provider-directory contracts that were never deployed on mainnet, and will anyhow be superseded by the new mixnode directory that is being worked on.

Testing steps performed

It works insofar that it compiles, we need to deploy and test this on non-mainnet before merging in

  • Purge nym-name-service contract
  • Purge nym-name-service-common
  • Purge nym-service-provider-directory contract
  • Purge nym-service-provider-directory-common
  • Remove everywhere name-service contract is used
  • Remove everywhere sp contract is used

Performed:

  • Check references to everything named SP and Ephemera and removed any additional references

Crypto

Comments

This PR contains a test failure due to the update here

  • This is due a change in x25519-dalek from 1.1.1 to 2.
  • Crypto operations should be identical, but the byte representation has changed (sphinx clamps at creation, x25519 clamps at use). This cannot be changed in the sphinx crate without breaking changes.
  • There is a good chance that this failure doesn’t impact anything else, but it has to be tested to see.
  • A mix of old and new clients with a mix of old and new mixnodes should do

Bugfix

Testing steps performed

  • Use the latest release/chomp contracts and deploy these to QA
  • If the contract was not found, the API would complain of invalid contracts, thus not starting the rest of the operations of the API (network monitor / rewarding etc)

Jun 11 16:27:34 qa-v2-nym-api bash[1352642]: 2024-06-11T16:27:34.551Z ERROR nym_api::nym_contract_cache::cache::refresher > Failed to refresh validator cache - Abci query failed with code 6 - address n14y2x8a60knc5jjfeztt84kw8x8l5pwdgnqg256v0p9v4p7t2q6eswxyusw: no such contract: unknown request

Testing steps performed

  • Use the latest release/chomp binary with nym-node and input a dodgy ip image

  • Validation: image

Testing steps performed

  • Verify that the establish_connection function correctly attempts to establish a connection to the gateway.
  • Test error handling for NetworkConnectionFailed by simulating a failed connection.
  • Ensure that the NetworkConnectionFailed error includes the address and source details as expected.
  • Checked that SocketState::Available is set correctly when a connection is successfully established.

Cargo warnings

warning: /home/alice/src/nym/nym/common/dkg/Cargo.toml: default-features is ignored for bls12_381, since default-features was not specified for workspace.dependencies.bls12_381, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/dkg/Cargo.toml: default-features is ignored for ff, since default-features was not specified for workspace.dependencies.ff, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/dkg/Cargo.toml: default-features is ignored for group, since default-features was not specified for workspace.dependencies.group, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/client-libs/validator-client/Cargo.toml: default-features is ignored for bip32, since default-features was not specified for workspace.dependencies.bip32, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/client-libs/validator-client/Cargo.toml: default-features is ignored for prost, since default-features was not specified for workspace.dependencies.prost, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/credentials-interface/Cargo.toml: default-features is ignored for bls12_381, since default-features was not specified for workspace.dependencies.bls12_381, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/credentials/Cargo.toml: default-features is ignored for bls12_381, since default-features was not specified for workspace.dependencies.bls12_381, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/nymcoconut/Cargo.toml: default-features is ignored for bls12_381, since default-features was not specified for workspace.dependencies.bls12_381, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/nymcoconut/Cargo.toml: default-features is ignored for ff, since default-features was not specified for workspace.dependencies.ff, this could become a hard error in the future warning: /home/alice/src/nym/nym/common/nymcoconut/Cargo.toml: default-features is ignored for group, since default-features was not specified for workspace.dependencies.group, this could become a hard error in the future.

- This PR adds `default-features = false` to the workspace dependencies to fix these. An alternative way would be to remove `default-features = false` in the crates, but we assume these were put there for a good reason. Also we might have other crates outside of the main workspace that depends on these crates having default features disabled. - We also have the warning `warning: profile package spec nym-wasm-sdk in profile release did not match any packages` which we fix by commenting out the profile settings, since the crate is currently commented out in the workspace crate list.

Testing steps performed

  • All binaries have been built and deployed from this branch and no issues have surfaced.

Operators Guide updates


v2024.5-ragusa

CHANGELOG.md

  • Feature/nym node api location (#4605)
  • Add optional signature to IPR request/response (#4604)
  • Feature/unstable tested nodes endpoint (#4601)
  • nym-api: make report/avg_uptime endpoints ignore blacklist (#4599)
  • removed blocking for coconut in the final epoch state (#4598)
  • allow using explicit admin address for issuing freepasses (#4595)
  • Use rfc3339 for last_polled in described nym-api endpoint (#4591)
  • Explicitly handle constraint unique violation when importing credential (#4588)
  • [bugfix] noop flag for nym-api for nymvisor compatibility (#4586)
  • Chore/additional helpers (#4585)
  • Feature/wasm coconut (#4584)
  • upgraded axum and related deps to the most recent version (#4573)
  • Feature/nyxd scraper pruning (#4564)
  • Run cargo autoinherit on the main workspace (#4553)
  • Add rustls-tls to reqwest in validator-client (#4552)
  • Feature/rewarder voucher issuance (#4548)

Features

  • New nym-node API endpoint /api/v1/auxiliary-details: to expose any additional information. Currently it’s just the location. nym-api will then query all nodes for that information and put it in the self-described endpoint.
  • New nym-node location available - use one of the three options to add this to your node config:
    1. Update the location field under [host] section of config.toml
    2. For new nodes: Initialise the node with --location flag, where they have to provide the country info. Either full country name (e.g. ‘Jamaica’), two-letter alpha2 (e.g. ‘JM’), three-letter alpha3 (e.g. ‘JAM’) or three-digit numeric-3 (e.g. ‘388’) can be provided.
    3. For existing nodes: It’s also possible to use exactly the same --location argument as above, but make sure to also provide --write-changes (or -w) flag to persist those changes!
  • Feature/unstable tested nodes endpoint: Adds new data structures (TestNode, TestRoute, PartialTestResult) to handle test results for Mixnodes and Gateways. With the inclusion of pagination to handle large API responses efficiently. Lastly, introducing a new route with the tag unstable thus meaning not to be consumed without a user risk, prefixes in endpoints with unstable, are what it says on the tin.

Testing steps performed

  • Deploy new api changes to sandbox environment
  • Ensure current operations are transactional and standed operations are working
  • Run a script to ensure that the new endpoints are working as expected with pagination image

Testing steps performed

  • Build the project and deployed it in a test environment.
  • Manually test API endpoints for mixnode and gateway data.
  • Verify that the endpoints return the expected data and handle blacklists correctly.
  • API performance improved due to the efficient HashMap lookups
  • Data in mainnet will differ from test nets due to the increased amount of gateways and mixnodes in that environment
  • Test standard uptime routes:
curl -X 'GET' 'https://validator.nymtech.net/api/v1/status/gateway/Fo4f4SQLdoyoGkFae5TpVhRVoXCF8UiypLVGtGjujVPf/avg_uptime' -H 'accept: application/json'
  • Use rfc3339 for last_polled in described nym-api endpoint: Fix issue where the validator-client can’t parse the nym-api response for the described endpoint, in particular the latest_polled field that was recently added, by making the field use rfc3339
    • Note: This will require upgrading nym-api and everything that depends on the described endpoint.

Testing steps performed

  • Update a nym-api to the binary built from this branch, then restart the api
  • Check the journalctl for error messages
  • Connected via client and could not see the error messages, this is backwards compatible
  • Local testing using sdk examples:
cd <PATH_TO>/nym/sdk/rust/nym-sdk
cargo run --example simple

# outcome
thread 'main' panicked at sdk/rust/nym-sdk/examples/simple.rs:9:64:
called Result::unwrap() on an Err value: ClientCoreError(ValidatorClientError(NymAPIError { source: ReqwestClientError { source: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None,

Testing steps performed

  • Run cargo autoinherit in the root directory to move dependencies to the workspace level
  • Merge the new workspace dependencies into the main list
  • Ensure no other changes were mixed in during the process
  • Verify that all feature flags for all crates remained the same as before
  • Build all the binaries from this branch to confirm successful compilation
  • Deploy the built binaries across different environments to ensure there were no issues
  • Add rustls-tls to reqwest in validator-client: An attempt to make possible to end up in a situation where use use the validator-client but without functioning TLS support. For the monorepo this is masked by cargo feature unification, but becomes a problem for outside consumers, as as been noticed in many of the vpn client implementations.
    • In validator-client: reqwest, enable rustls-tls for non-wasm32
    • In client-core: Use default features enabled for non-wasm32 and switch to webpki roots, since that’s what we’re using with reqwest anyway
    • In gateway-client: Switch to webpki roots, since that’s what we’re using with reqwest anyway

Crypto

Testing steps performed

  • Build the project to ensure no compilation errors
  • Run tests to verify the functionality of the issue_credential function
  • Execute integration tests to check the behaviour during an epoch transition.

Testing steps performed

  • Add a main.rs file in the nyxd scraper dir, underneath lib.rs, amend config.pruning_options.validate()?; to be let _ = config.pruning_options.validate(); in the mod.rs file
  • Test the different variations of pruning_options:
    • Check the default option: pruning_options: PruningOptions::default()
    • Check the nothing option: pruning_options: PruningOptions::nothing()
    • Check the custom option, example: pruning_options: PruningOptions { keep_recent: (500), interval: (10), strategy: (PruningStrategy::Custom) }
    • Check the pruning in real life for the validator rewarder
  • Validate that the database table blocks was being updated accordingly

Bugfix

Testing steps performed

  • Build the project to ensure no compilation errors
  • Run the binary with different command-line arguments to verify the CLI functionality
  • Test with and without the --no_banner flag to ensure compatibility and expected behavior
  • Verify logging setup and configuration file parsing

Operators Guide updates

  • nym-gateway-probe: A CLI tool to check in-real-time networking status of any Gateway locally.
  • Where to host your nym-node?: A list of Internet Service Providers (ISPs) by Nym Operators community. We invite all operators to add their experiences with different ISPs to strengthen the community knowledge and Nym mixnet performance.
  • Make sure you run nym-node with --wireguard-enabled false and add a location description to your config.toml, both documented in nym-node setup manual.

v2024.4-nutella

  • Merged PRs
  • nym-node version 1.1.1
  • This release also contains: nym-gateway and nym-network-requester binaries
  • core improvements on nym-node configuration
  • Nym wallet changes:
    • Adding nym-node command to bonding screens
    • Fixed the delegation issues with fixing RPC
  • Network configuration section updates, in particular for --mode mixnode operators
  • VPS IPv6 troubleshooting updates

v2024.3-eclipse

  • Release Changelog.md
  • nym-node initial release
  • New tool for monitoring Gateways performance harbourmaster.nymtech.net
  • New versioning 1.1.0+nymnode mainly for internal migration testing, not essential for operational use. We aim to correct this in a future release to ensure mixnodes feature correctly in the main API
  • New VPS specs & configuration page
  • New configuration page with connectivity setup guide - a new requirement for exit-gateway
  • API endpoints redirection: Nym-mixnode and nym-gateway endpoints will eventually be deprecated; due to this, their endpoints will be redirected to new routes once the nym-node has been migrated and is running

API endpoints redirection

Previous endpointNew endpoint
http://<IP>:8000/statshttp://<IP>:8000/api/v1/metrics/mixing
http://<IP>:8000/hardwarehttp://<IP>:8000/api/v1/system-info
http://<IP>:8000/descriptionhttp://<IP>:8000/api/v1/description