Setup Netdata on Coolify as Netdata Parent with multiple Netdata Children

Install Netdata through wget Configure Dynamic Configuration through Traefik Setup server as Netdata Parent to centralize metrics collection Edit stream.conf Configuring a Netdata Parent Configure Alerts to be sent to Slack Setup Netdata Children Edit stream.conf Configuring Netdata Children Observability. At your fingertips.

Netdata is an amazing free tool that allows you to observe your server metrics. If you self-host a lot of things, it might be the perfect tool to keep track of usage metrics, uptime monitoring and alerting in case something goes wrong.

However, if you go through Netdata Cloud, you can only observe 5 nodes at a time for free. That’s what I had been doing. Plus you don’t get the premium alerting integrations like Slack.

After some research, I found that you could easily setup parent and child nodes independently from Netdata Cloud. A parent node is the main Netdata Agent which stores all the metrics while also serving the web dashboard to explore and discover metrics. The child nodes connect to the parent node and send metrics.

With this setup, you can configure everything in one place while also benefiting from using premium alerting.

I’ve put together a small guide on how you can achieve this.

Install Netdata through wget


wget -O /tmp/netdata-kickstart.sh https://get.netdata.cloud/kickstart.sh && sh /tmp/netdata-kickstart.sh --disable-telemetry

Reference:

Install Netdata with kickstart.sh | Learn Netdata

The kickstart.sh script installs Netdata from source, including all dependencies required to connect to Netdata Cloud, with a single command.

https://learn.netdata.cloud/docs/netdata-agent/installation/linux/

Configure Dynamic Configuration through Traefik

Point your domain to the server where you have Netdata installed.

Lets call it netdata.your-domain.com

Navigate to your Coolify instance

Servers > localhost > Proxy > Dynamic Configurations > + Add

Filename: netdata.yaml

Configuration:


http:
  middlewares:
    redirect-to-https:
      redirectscheme:
        scheme: https
    gzip:
      compress: true
# Set this if you need basic auth. You can use a
# tool to generate a bcrypt hash of your password
#    netdata-auth:
#      basicAuth:
#        users:
#          - 'username:hashed_password'
  routers:
    netdata-http:
      middlewares:
        - redirect-to-https
      entryPoints:
        - http
      service: netdata
      rule: Host(`netdata.your-domain.com`)
    netdata-https:
      entryPoints:
        - https
      service: netdata
      rule: Host(`netdata.your-domain.com`)
      tls:
        certresolver: letsencrypt
      middlewares:
        - netdata-auth
  services:
    netdata:
      loadBalancer:
        servers:
          -
            url: 'http://host.docker.internal:19999'

After a few moments, you should be able to access your Netdata Dashboard at netdata.your-domain.com.

Setup server as Netdata Parent to centralize metrics collection

Edit `stream.conf`

To edit stream.conf, run this on your terminal:


cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config stream.conf

Your editor will open, with defaults and commented stream.conf options.

Configuring a Netdata Parent

To enable the reception of metrics from Netdata Children, generate a random API key with this command:


uuidgen

Then, copy the UUID generated, edit stream.conf, find the section that reads like the following and replace API_KEY with the UUID you generated:


[API_KEY]
    # Accept metrics streaming from other Agents with the specified API key
    enabled = yes

Save the file and restart Netdata.


sudo service netdata restart

Reference:

Configuring Metrics Centralization Points | Learn Netdata

Metrics streaming configuration for both Netdata Children and Parents is done via stream.conf.

https://learn.netdata.cloud/docs/observability-centralization-points/metrics-centralization-points/configuring-metrics-centralization-points

Configure Alerts to be sent to Slack

Create a Slack App.

Create a Webhook Trigger to a specific channel. Copy the webhook url. You’ll need it later.

SSH into the Netdata Parent server.

Change directory to /etc/netdata


cd /etc/netdata

Create a new file called health_alarm_notify.conf


nano health_alarm_notify

Copy the following and fill out the SLACK_WEBHOOK_URL and DEFAULT_RECIPIENT_SLACK


# Enable/disable sending notifications to Slack
ENABLE_SLACK="YES"

# Slack incoming webhook URL
SLACK_WEBHOOK_URL="https://hooks.slack.com/services/XXXXXXXXXXX/YYYYYYYYYYY/ZZZZZZZZZZZZZZZZZZZZ"

# The default channel to post messages
DEFAULT_RECIPIENT_SLACK="#netdata-notifications"

# Set a custom username for the bot (optional)
# SLACK_USERNAME="Netdata"

# Set a custom icon for the bot (optional)
# SLACK_ICON_EMOJI=":chart_with_upwards_trend:"

Setup Netdata Children

Edit `stream.conf`

To edit stream.conf, run this on your terminal:


cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config stream.conf

Your editor will open, with defaults and commented stream.conf options.

Configuring Netdata Children

To enable streaming metrics to a Netdata Parent, edit stream.conf, and at the [stream] section at the top, set:


[stream]
    # Stream metrics to another Netdata
    enabled = yes
    # The IP and PORT of the parent
    destination = PARENT_IP_ADDRESS:19999
    # The shared API key, generated by uuidgen
    api key = API_KEY

Save the file and restart Netdata.


sudo service netdata restart

If you don’t see the name show up correctly, you can set the hostname of the server to have it display correctly.


 sudo hostnamectl set-hostname cfy.memox.io

Reference:

Configuring Metrics Centralization Points | Learn Netdata

Metrics streaming configuration for both Netdata Children and Parents is done via stream.conf.

https://learn.netdata.cloud/docs/observability-centralization-points/metrics-centralization-points/configuring-metrics-centralization-points