feat: VictoriaMetrics и NetBird между панелью и ger-nod-02
ansible-core-net / deploy (push) Canceled after 0s
ansible-core-secrets / deploy (push) Canceled after 0s
ansible-ger-nod-02 / deploy (push) Canceled after 0s
ansible-opnsense / deploy (push) Canceled after 0s
ansible-core-mon / deploy (push) Successful in 58s
ansible-rus-central / deploy (push) Successful in 44s

Prometheus остаётся для LAN. Нода отдаёт метрики через vmagent, а панель ходит на неё по отдельному ключу NetBird, не через группу роутеров.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
karpenokandCursor committed 2026-11-12 16:50:44 +07:00
1 parent 5704272c18
commit 72a9c86e52
46 files changed
+13496 -25

No files matched your search

+2
View File
@@ -8,6 +8,8 @@ on:
- 'ansible/ansible.cfg' - 'ansible/ansible.cfg'
- 'ansible/ci/**' - 'ansible/ci/**'
- 'ansible/roles/core-net/**' - 'ansible/roles/core-net/**'
- 'ansible/roles/netbird_client/**'
- 'ansible/roles/vmagent/**'
- 'ansible/roles/host/**' - 'ansible/roles/host/**'
- 'ansible/playbooks/core-net.yml' - 'ansible/playbooks/core-net.yml'
- 'ansible/playbooks/host.yml' - 'ansible/playbooks/host.yml'
+2
View File
@@ -8,6 +8,8 @@ on:
- 'ansible/ansible.cfg' - 'ansible/ansible.cfg'
- 'ansible/ci/**' - 'ansible/ci/**'
- 'ansible/roles/ger-nod-02/**' - 'ansible/roles/ger-nod-02/**'
- 'ansible/roles/netbird_client/**'
- 'ansible/roles/vmagent/**'
- 'ansible/roles/host/**' - 'ansible/roles/host/**'
- 'ansible/playbooks/ger-nod-02.yml' - 'ansible/playbooks/ger-nod-02.yml'
- 'ansible/playbooks/host.yml' - 'ansible/playbooks/host.yml'
+10 -7
View File
@@ -6,11 +6,11 @@
| Машина | Плейбук | Роли | Каталоги | | Машина | Плейбук | Роли | Каталоги |
|---|---|---|---| |---|---|---|---|
| `core-net` | `playbooks/core-net.yml` | `roles/core-net/traefik`; `roles/core-net/remnawave`; `roles/core-net/authentik` *(опц., по умолчанию выкл.)* | `/opt/traefik`; `/opt/remnawave`; `/opt/authentik` *(если включён)* | | `core-net` | `playbooks/core-net.yml` | `roles/core-net/traefik`; `roles/core-net/netbird`; `roles/core-net/remnawave`; `roles/core-net/vmagent`; `roles/core-net/authentik` *(опц., по умолчанию выкл.)* | `/opt/traefik`; клиент NetBird; `/opt/remnawave`; `/opt/vmagent`; `/opt/authentik` *(если включён)* |
| `core-secrets` | `playbooks/core-secrets.yml` | `roles/core-secrets/infisical`, `roles/core-secrets/vaultwarden` | `/opt/infisical`, `/opt/vaultwarden` | | `core-secrets` | `playbooks/core-secrets.yml` | `roles/core-secrets/infisical`, `roles/core-secrets/vaultwarden` | `/opt/infisical`, `/opt/vaultwarden` |
| `core-mon` | `playbooks/core-mon.yml` | `roles/core-mon/monitoring` | `/opt/monitoring` | | `core-mon` | `playbooks/core-mon.yml` | `roles/core-mon/monitoring` | `/opt/monitoring` (Prometheus и VictoriaMetrics) |
| `rus-central` | `playbooks/rus-central.yml` | `roles/rus-central/netbird` | `/opt/netbird` | | `rus-central` | `playbooks/rus-central.yml` | `roles/rus-central/netbird` | `/opt/netbird` |
| `ger-nod-02` | `playbooks/ger-nod-02.yml` | `roles/ger-nod-02/remnanode` | `/opt/remnanode` | | `ger-nod-02` | `playbooks/ger-nod-02.yml` | `roles/ger-nod-02/netbird`; `roles/ger-nod-02/remnanode`; `roles/ger-nod-02/vmagent` | клиент NetBird; `/opt/remnanode`; `/opt/vmagent` |
| `opnsense` | `playbooks/opnsense.yml` | `roles/opnsense/netbird` | REST API (`192.168.1.1`) | | `opnsense` | `playbooks/opnsense.yml` | `roles/opnsense/netbird` | REST API (`192.168.1.1`) |
Полная сверка одной машины: `ansible-playbook playbooks/site.yml --limit <машина>`. Gitea в автоматизацию не входит: без неё не запустить пайплайны. Полная сверка одной машины: `ansible-playbook playbooks/site.yml --limit <машина>`. Gitea в автоматизацию не входит: без неё не запустить пайплайны.
@@ -34,7 +34,7 @@ Terraform в этом репозитории описывает только у
| **CT 109** | `pve` | `192.168.1.217` | `gitea` | Gitea (`:3000`), Postgres (`:5432`), Gitea Actions Runner (`dind`) | | **CT 109** | `pve` | `192.168.1.217` | `gitea` | Gitea (`:3000`), Postgres (`:5432`), Gitea Actions Runner (`dind`) |
| **CT 901** | `netcore` | `192.168.1.102` | `core-secrets` | Infisical (`:8080`), Postgres, Redis, Vaultwarden (`:8000`) | | **CT 901** | `netcore` | `192.168.1.102` | `core-secrets` | Infisical (`:8080`), Postgres, Redis, Vaultwarden (`:8000`) |
| **CT 902** | `netcore` | `192.168.1.104` | `core-net` | Traefik (`:80`, `:443`); Remnawave Panel (`127.0.0.1:3000`) + Subscription (`127.0.0.1:3010`); Authentik (`127.0.0.1:9000`) — опционально, по умолчанию выкл. | | **CT 902** | `netcore` | `192.168.1.104` | `core-net` | Traefik (`:80`, `:443`); Remnawave Panel (`127.0.0.1:3000`) + Subscription (`127.0.0.1:3010`); Authentik (`127.0.0.1:9000`) — опционально, по умолчанию выкл. |
| **CT 903** | `netcore` | `192.168.1.108` | `core-mon` | Prometheus (`:9090`), Alertmanager (`:9093`), Grafana (`:3000`), Ntfy (`:80`) | | **CT 903** | `netcore` | `192.168.1.108` | `core-mon` | Prometheus (`:9090`), VictoriaMetrics (`:8428`), Alertmanager (`:9093`), Grafana (`:3000`), Ntfy (`:80`) |
### 1.3. Внешние узлы ### 1.3. Внешние узлы
| Имя | IP | Роль | Стек | | Имя | IP | Роль | Стек |
@@ -80,7 +80,8 @@ API Endpoint (внутренний LAN): `http://192.168.1.102:8080/api`
- `NETBIRD_DATASTORE_KEY`: 32-байтный ключ SQLite/datastore. - `NETBIRD_DATASTORE_KEY`: 32-байтный ключ SQLite/datastore.
- `NETBIRD_SESSION_KEY`: 32-байтный ключ веб-сессий. - `NETBIRD_SESSION_KEY`: 32-байтный ключ веб-сессий.
- `NETBIRD_API_TOKEN`: сервисный API-токен управления (создаётся и сохраняется пайплайном автоматически). - `NETBIRD_API_TOKEN`: сервисный API-токен управления (создаётся и сохраняется пайплайном автоматически).
- `NETBIRD_SETUP_KEY`: многоразовый ключ регистрации узлов (создаётся и сохраняется пайплайном автоматически). - `NETBIRD_SETUP_KEY`: многоразовый ключ регистрации роутеров (`homelab-routers`). Панель и ноды им не пользуются.
- `NETBIRD_REMNAWAVE_SETUP_KEY`: многоразовый ключ `remnawave-peers` для core-net и ger-nod-02. В группу роутеров не входит.
- *(Параметры `NETBIRD_ADMIN_NAME`, `NETBIRD_ADMIN_PASSWORD`, `NETBIRD_ADMIN_EMAIL` наследуются из `/USER/ADMIN`)*. - *(Параметры `NETBIRD_ADMIN_NAME`, `NETBIRD_ADMIN_PASSWORD`, `NETBIRD_ADMIN_EMAIL` наследуются из `/USER/ADMIN`)*.
- `/services/opnsense`: - `/services/opnsense`:
- `OPNSENSE_API_KEY`: API-ключ администратора OPNsense. - `OPNSENSE_API_KEY`: API-ключ администратора OPNsense.
@@ -90,6 +91,7 @@ API Endpoint (внутренний LAN): `http://192.168.1.102:8080/api`
- `REMNAWAVE_APP_SECRET`, `REMNAWAVE_POSTGRES_PASSWORD`, `REMNAWAVE_METRICS_PASS`, `REMNAWAVE_WEBHOOK_SECRET_HEADER`: генерируются пайплайном при первом прогоне, если пусто. - `REMNAWAVE_APP_SECRET`, `REMNAWAVE_POSTGRES_PASSWORD`, `REMNAWAVE_METRICS_PASS`, `REMNAWAVE_WEBHOOK_SECRET_HEADER`: генерируются пайплайном при первом прогоне, если пусто.
- `REMNAWAVE_API_TOKEN`: сервисный API-токен панели (`infra-automation`, scopes `*`), создаётся и сохраняется пайплайном (при сбое — создать в UI и положить сюда вручную). - `REMNAWAVE_API_TOKEN`: сервисный API-токен панели (`infra-automation`, scopes `*`), создаётся и сохраняется пайплайном (при сбое — создать в UI и положить сюда вручную).
- `REMNANODE_SECRET_KEY`: SECRET_KEY ноды Ger-nod-02 (`GET /api/keygen/`), сохраняется пайплайном. - `REMNANODE_SECRET_KEY`: SECRET_KEY ноды Ger-nod-02 (`GET /api/keygen/`), сохраняется пайплайном.
- `XRAY_CHECKER_SUBSCRIPTION_URL`: ссылка подписки пользователя `homelab` для Xray Checker на core-mon. В лог не пишется.
- `/proxmox/netcore`: - `/proxmox/netcore`:
- `PROXMOX_ENDPOINT`: `https://192.168.1.100:8006/` - `PROXMOX_ENDPOINT`: `https://192.168.1.100:8006/`
- `PROXMOX_API_TOKEN`: токен `root@pam!terraform=UUID` - `PROXMOX_API_TOKEN`: токен `root@pam!terraform=UUID`
@@ -151,7 +153,7 @@ API Endpoint (внутренний LAN): `http://192.168.1.102:8080/api`
- `/ci-cd` → `CLOUDFLARE_API` - `/ci-cd` → `CLOUDFLARE_API`
- `/VPS` → `PASS_GER_NOD_02` (пароль нового VPS) - `/VPS` → `PASS_GER_NOD_02` (пароль нового VPS)
2. Запустить **`ansible-core-net`** (полный прогон или `--tags remnawave` после того, как Traefik уже с роутами `rw`/`sub`). 2. Запустить **`ansible-core-net`** (полный прогон или `--tags remnawave` после того, как Traefik уже с роутами `rw`/`sub`).
3. Роль поднимает `/opt/remnawave` (panel+db+redis) и `/opt/remnawave/subscription`, создаёт `REMNAWAVE_ADMIN_*`, каждый прогон **перевыпускает** API-токен `infra-automation` и кладёт его в Infisical (старый сразу недействителен). Config profile **`DE-Yali`**, сквад `Default-Squad` на inbound Steal-Yali, нода **`Ger-nod-02`**, host **`yali.gunis.top`**. Пользователи — файл `ansible/roles/core-net/remnawave/files/users.yml` (добавление/изменение/удаление = правка файла). 3. Роль поднимает `/opt/remnawave` (panel+db+redis) и `/opt/remnawave/subscription`, создаёт `REMNAWAVE_ADMIN_*`, каждый прогон **перевыпускает** API-токен `infra-automation` и кладёт его в Infisical (старый сразу недействителен). Config profile **`DE-Yali`**, сквад `Default-Squad` на inbound Steal-Yali, нода **`Ger-nod-02`**, host **`yali.gunis.top`**. Пользователи — файл `ansible/roles/core-net/remnawave/files/users.yml` (добавление/изменение/удаление = правка файла). Когда peer `ger-nod-02` уже есть в NetBird, адрес ноды в панели меняется на его NetBird IP. Пока peer нет, остаётся `150.241.64.209`. Ссылка подписки `homelab` пишется в `XRAY_CHECKER_SUBSCRIPTION_URL` и в лог не попадает.
4. Проверить `https://rw.gunihome.ru` и `https://sub.gunihome.ru`. 4. Проверить `https://rw.gunihome.ru` и `https://sub.gunihome.ru`.
> [!IMPORTANT] > [!IMPORTANT]
@@ -161,8 +163,9 @@ API Endpoint (внутренний LAN): `http://192.168.1.102:8080/api`
1. После успешного `ansible-core-net` (чтобы в Infisical был `REMNANODE_SECRET_KEY`) запустить **`ansible-ger-nod-02`**. 1. После успешного `ansible-core-net` (чтобы в Infisical был `REMNANODE_SECRET_KEY`) запустить **`ansible-ger-nod-02`**.
2. Первый прогон: как у rus-central — `ansible_password` из `/VPS` → `PASS_GER_NOD_02`; роль `host` кладёт ключ и отключает пароль. Роль `remnanode` держит A-запись `yali.gunis.top` в Cloudflare, выпускает LE по **HTTP-01** (порт 80, без TXT `_acme-challenge` в зоне), поднимает remnanode + nginx SelfSteal. 2. Первый прогон: как у rus-central — `ansible_password` из `/VPS` → `PASS_GER_NOD_02`; роль `host` кладёт ключ и отключает пароль. Роль `remnanode` держит A-запись `yali.gunis.top` в Cloudflare, выпускает LE по **HTTP-01** (порт 80, без TXT `_acme-challenge` в зоне), поднимает remnanode + nginx SelfSteal.
3. В панели нода должна стать зелёной; браузер на `https://yali.gunis.top` — маскировочный сайт. 3. В панели нода должна стать зелёной; браузер на `https://yali.gunis.top` — маскировочный сайт.
4. Клиент NetBird ходит на `https://central.gunihome.ru` ключом `remnawave-peers`. Пока peer панели не виден, `NODE_PORT` 2222 остаётся открыт для WAN egress панели. Когда peer `core-net` есть, 2222 разрешён только с его NetBird IP. Публичный `:9100` закрыт: метрики ноды уходят vmagent-ом в VictoriaMetrics на `192.168.1.108:8428` через маршрут `homelab-lan`. В Prometheus эта нода больше не входит.
### Шаг 5. NetBird (rus-central) и связка с OPNsense ### Шаг 5. NetBird (rus-central) и связка с OPNsense
1. Запустить пайплайн **`ansible-rus-central`** в Gitea Actions. Ручной запуск сверяет машину целиком. Push в `roles/rus-central/netbird` ставит только NetBird. 1. Запустить пайплайн **`ansible-rus-central`** в Gitea Actions. Ручной запуск сверяет машину целиком. Push в `roles/rus-central/netbird` ставит только NetBird.
2. Раннер считывает ключи из `/services/netbird`, деплоит стек в `/opt/netbird` на `45.154.185.208`, автоматически регистрирует первого суперпользователя (`POST /api/setup`), создаёт сервисный токен (`NETBIRD_API_TOKEN`), группу `homelab-routers`, маршрут к подсети `192.168.1.0/24` и многоразовый ключ установки (`NETBIRD_SETUP_KEY`), после чего сохраняет `NETBIRD_API_TOKEN` и `NETBIRD_SETUP_KEY` в Infisical. 2. Раннер считывает ключи из `/services/netbird`, деплоит стек в `/opt/netbird` на `45.154.185.208`, автоматически регистрирует первого суперпользователя (`POST /api/setup`), создаёт сервисный токен (`NETBIRD_API_TOKEN`), группу `homelab-routers`, маршрут к подсети `192.168.1.0/24` и многоразовый ключ установки (`NETBIRD_SETUP_KEY`). Отдельно создаётся группа `remnawave` и ключ `remnawave-peers` (`NETBIRD_REMNAWAVE_SETUP_KEY`): он не добавляет peer в `homelab-routers`. Оба ключа сохраняются в Infisical.
3. Запустить пайплайн **`ansible-opnsense`** в Gitea Actions. Раннер обращается по HTTPS REST API к `192.168.1.1`, проверяет/устанавливает плагин `os-netbird`, включает службу и регистрирует шлюз OPNsense на `https://central.gunihome.ru:443` по `NETBIRD_SETUP_KEY`. 3. Запустить пайплайн **`ansible-opnsense`** в Gitea Actions. Раннер обращается по HTTPS REST API к `192.168.1.1`, проверяет/устанавливает плагин `os-netbird`, включает службу и регистрирует шлюз OPNsense на `https://central.gunihome.ru:443` по `NETBIRD_SETUP_KEY`.
+10
View File
@@ -64,6 +64,16 @@ if [ "$full" -eq 0 ] && [ -n "$changed" ]; then
ansible/ansible.cfg | ansible/ci/* | ansible/roles/host/* | ansible/playbooks/host.yml | ansible/playbooks/site.yml | ansible/playbooks/"$machine".yml | ansible/inventories/prod/hosts.yml | ansible/inventories/prod/group_vars/* | ansible/inventories/prod/host_vars/"$machine".yml) ansible/ansible.cfg | ansible/ci/* | ansible/roles/host/* | ansible/playbooks/host.yml | ansible/playbooks/site.yml | ansible/playbooks/"$machine".yml | ansible/inventories/prod/hosts.yml | ansible/inventories/prod/group_vars/* | ansible/inventories/prod/host_vars/"$machine".yml)
full_from_files=1 full_from_files=1
;; ;;
ansible/roles/netbird_client/*)
case "$machine" in
core-net | ger-nod-02) add_tag netbird ;;
esac
;;
ansible/roles/vmagent/*)
case "$machine" in
core-net | ger-nod-02) add_tag vmagent ;;
esac
;;
ansible/roles/"$machine"/*/*) ansible/roles/"$machine"/*/*)
product=${file#ansible/roles/"$machine"/} product=${file#ansible/roles/"$machine"/}
product=${product%%/*} product=${product%%/*}
+9
View File
@@ -64,6 +64,9 @@ case "$machine" in
export REMNAWAVE_WEBHOOK_SECRET_HEADER="${REMNAWAVE_WEBHOOK_SECRET_HEADER:-}" export REMNAWAVE_WEBHOOK_SECRET_HEADER="${REMNAWAVE_WEBHOOK_SECRET_HEADER:-}"
export REMNAWAVE_API_TOKEN="${REMNAWAVE_API_TOKEN:-}" export REMNAWAVE_API_TOKEN="${REMNAWAVE_API_TOKEN:-}"
export REMNANODE_SECRET_KEY="${REMNANODE_SECRET_KEY:-}" export REMNANODE_SECRET_KEY="${REMNANODE_SECRET_KEY:-}"
eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/services/netbird" --format=dotenv-export --silent)"
export NETBIRD_API_TOKEN="${NETBIRD_API_TOKEN:-}"
export NETBIRD_REMNAWAVE_SETUP_KEY="${NETBIRD_REMNAWAVE_SETUP_KEY:-}"
;; ;;
core-secrets) core-secrets)
eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/services/infisical" --format=dotenv-export --silent)" eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/services/infisical" --format=dotenv-export --silent)"
@@ -77,6 +80,8 @@ case "$machine" in
if [ -n "${MAIL_ADMIN_USER:-}" ]; then if [ -n "${MAIL_ADMIN_USER:-}" ]; then
extra+=(-e "grafana_admin_email=${MAIL_ADMIN_USER}") extra+=(-e "grafana_admin_email=${MAIL_ADMIN_USER}")
fi fi
eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/services/remnawave" --format=dotenv-export --silent 2>/dev/null || true)"
export XRAY_CHECKER_SUBSCRIPTION_URL="${XRAY_CHECKER_SUBSCRIPTION_URL:-}"
;; ;;
rus-central) rus-central)
input_email="${INPUT_LETSENCRYPT_EMAIL:-}" input_email="${INPUT_LETSENCRYPT_EMAIL:-}"
@@ -89,6 +94,7 @@ case "$machine" in
export NETBIRD_ADMIN_PASSWORD="${NETBIRD_ADMIN_PASSWORD:-${ADMIN_PASSWORD:-}}" export NETBIRD_ADMIN_PASSWORD="${NETBIRD_ADMIN_PASSWORD:-${ADMIN_PASSWORD:-}}"
export NETBIRD_API_TOKEN="${NETBIRD_API_TOKEN:-}" export NETBIRD_API_TOKEN="${NETBIRD_API_TOKEN:-}"
export NETBIRD_SETUP_KEY="${NETBIRD_SETUP_KEY:-}" export NETBIRD_SETUP_KEY="${NETBIRD_SETUP_KEY:-}"
export NETBIRD_REMNAWAVE_SETUP_KEY="${NETBIRD_REMNAWAVE_SETUP_KEY:-}"
if [ -n "$input_email" ]; then if [ -n "$input_email" ]; then
NETBIRD_LETSENCRYPT_EMAIL="$input_email" NETBIRD_LETSENCRYPT_EMAIL="$input_email"
fi fi
@@ -110,6 +116,9 @@ case "$machine" in
eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/VPS" --format=dotenv-export --silent)" eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/VPS" --format=dotenv-export --silent)"
# CLOUDFLARE_API уже из /ci-cd выше (только A-запись, без DNS-01). # CLOUDFLARE_API уже из /ci-cd выше (только A-запись, без DNS-01).
export REMNANODE_SECRET_KEY="${REMNANODE_SECRET_KEY:-}" export REMNANODE_SECRET_KEY="${REMNANODE_SECRET_KEY:-}"
eval "$(infisical export --domain="${INFISICAL_DOMAIN}" --token="${TOKEN}" --env=prod --projectId="${INFISICAL_PROJECT_ID}" --path="/services/netbird" --format=dotenv-export --silent 2>/dev/null || true)"
export NETBIRD_API_TOKEN="${NETBIRD_API_TOKEN:-}"
export NETBIRD_REMNAWAVE_SETUP_KEY="${NETBIRD_REMNAWAVE_SETUP_KEY:-}"
export CLOUDFLARE_API="${CLOUDFLARE_API:-}" export CLOUDFLARE_API="${CLOUDFLARE_API:-}"
export PASS_GER_NOD_02="${PASS_GER_NOD_02:-}" export PASS_GER_NOD_02="${PASS_GER_NOD_02:-}"
if [ -z "${REMNANODE_SECRET_KEY}" ]; then if [ -z "${REMNANODE_SECRET_KEY}" ]; then
@@ -6,3 +6,6 @@ remnanode_selfsteal_domain: "yali.gunis.top"
remnanode_panel_egress_ips: remnanode_panel_egress_ips:
- "213.128.28.107" # провайдер 1 (МТС) - "213.128.28.107" # провайдер 1 (МТС)
- "185.160.137.140" # провайдер 2 - "185.160.137.140" # провайдер 2
monitoring_scrape_enabled: false
monitoring_node_exporter_public: false
monitoring_node_exporter_listen: "127.0.0.1:9100"
+2 -1
View File
@@ -1,5 +1,6 @@
# Машины. Состав каждой машины — в playbooks/<хост>.yml. # Машины. Состав каждой машины — в playbooks/<хост>.yml.
# Scrape Prometheus (node-exporter :9100): все записи ниже, кроме monitoring_scrape_exclude_hosts в group_vars/роли monitoring. # Scrape Prometheus (node-exporter :9100): все записи ниже, кроме monitoring_scrape_exclude_hosts
# и хостов с monitoring_scrape_enabled: false (ger-nod-02 отдаёт метрики через vmagent).
all: all:
hosts: hosts:
core-net: core-net:
+4
View File
@@ -5,7 +5,11 @@
roles: roles:
- role: core-net/traefik - role: core-net/traefik
tags: [traefik] tags: [traefik]
- role: core-net/netbird
tags: [netbird]
- role: core-net/remnawave - role: core-net/remnawave
tags: [remnawave] tags: [remnawave]
- role: core-net/authentik - role: core-net/authentik
tags: [authentik] tags: [authentik]
- role: core-net/vmagent
tags: [vmagent]
+4
View File
@@ -3,5 +3,9 @@
hosts: ger-nod-02 hosts: ger-nod-02
gather_facts: false gather_facts: false
roles: roles:
- role: ger-nod-02/netbird
tags: [netbird]
- role: ger-nod-02/remnanode - role: ger-nod-02/remnanode
tags: [remnanode] tags: [remnanode]
- role: ger-nod-02/vmagent
tags: [vmagent]
@@ -11,6 +11,13 @@ monitoring_node_exporter_port: 9100
monitoring_scrape_job_name: node-exporter monitoring_scrape_job_name: node-exporter
monitoring_env_label: prod monitoring_env_label: prod
victoriametrics_image: victoriametrics/victoria-metrics:v1.153.0
vmagent_image: victoriametrics/vmagent:v1.153.0
cadvisor_image: gcr.io/cadvisor/cadvisor:v0.49.2
xray_checker_image: kutovoys/xray-checker:v1.3.1
xray_checker_subscription_url: "{{ lookup('env', 'XRAY_CHECKER_SUBSCRIPTION_URL', default='') }}"
vmagent_cluster: homelab
grafana_admin_user: "{{ lookup('ansible.builtin.env', 'ADMIN_USER', default='karpenok') }}" grafana_admin_user: "{{ lookup('ansible.builtin.env', 'ADMIN_USER', default='karpenok') }}"
grafana_admin_password: "{{ lookup('ansible.builtin.env', 'ADMIN_PASSWORD', default='') }}" grafana_admin_password: "{{ lookup('ansible.builtin.env', 'ADMIN_PASSWORD', default='') }}"
grafana_admin_email: "{{ lookup('ansible.builtin.env', 'MAIL_ADMIN_USER', default=grafana_admin_user ~ '@gunihome.ru') }}" grafana_admin_email: "{{ lookup('ansible.builtin.env', 'MAIL_ADMIN_USER', default=grafana_admin_user ~ '@gunihome.ru') }}"
@@ -0,0 +1,817 @@
{
"__inputs": [
{
"name": "DS_PROMETHEUS",
"label": "Prometheus",
"description": "Prometheus as the datasource is obligatory",
"type": "datasource",
"pluginId": "prometheus",
"pluginName": "Prometheus"
}
],
"__requires": [
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "7.4.5"
},
{
"type": "panel",
"id": "graph",
"name": "Graph",
"version": ""
},
{
"type": "datasource",
"id": "prometheus",
"name": "Prometheus",
"version": "1.0.0"
},
{
"type": "panel",
"id": "table",
"name": "Table",
"version": ""
}
],
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": 14282,
"graphTooltip": 0,
"id": null,
"iteration": 1617715580880,
"links": [],
"panels": [
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 0
},
"id": 8,
"panels": [],
"title": "CPU",
"type": "row"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 7,
"w": 24,
"x": 0,
"y": 1
},
"hiddenSeries": false,
"id": 15,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null as zero",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(container_cpu_usage_seconds_total{instance=~\"$host\",name=~\"$container\",name=~\".+\"}[5m])) by (name) *100",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "CPU Usage",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:606",
"format": "percent",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:607",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 8
},
"id": 11,
"panels": [],
"title": "Memory",
"type": "row"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 9
},
"hiddenSeries": false,
"id": 9,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null as zero",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(container_memory_rss{instance=~\"$host\",name=~\"$container\",name=~\".+\"}) by (name)",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Memory Usage",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:606",
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:607",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 9
},
"hiddenSeries": false,
"id": 14,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null as zero",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum(container_memory_cache{instance=~\"$host\",name=~\"$container\",name=~\".+\"}) by (name)",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Memory Cached",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:606",
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:607",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 17
},
"id": 2,
"panels": [],
"title": "Network",
"type": "row"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 18
},
"hiddenSeries": false,
"id": 4,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"hideEmpty": false,
"hideZero": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"sideWidth": null,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(container_network_receive_bytes_total{instance=~\"$host\",name=~\"$container\",name=~\".+\"}[5m])) by (name)",
"hide": false,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Received Network Traffic",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:674",
"format": "Bps",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:675",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {}
},
"overrides": []
},
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 18
},
"hiddenSeries": false,
"id": 6,
"legend": {
"alignAsTable": true,
"avg": true,
"current": false,
"max": true,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": true
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"alertThreshold": true
},
"percentage": false,
"pluginVersion": "7.4.5",
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(container_network_transmit_bytes_total{instance=~\"$host\",name=~\"$container\",name=~\".+\"}[5m])) by (name)",
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "Sent Network Traffic",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"$$hashKey": "object:832",
"format": "Bps",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"$$hashKey": "object:833",
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"collapsed": false,
"datasource": "${DS_PROMETHEUS}",
"gridPos": {
"h": 1,
"w": 24,
"x": 0,
"y": 26
},
"id": 19,
"panels": [],
"title": "Misc",
"type": "row"
},
{
"datasource": "${DS_PROMETHEUS}",
"fieldConfig": {
"defaults": {
"custom": {
"align": null,
"filterable": false
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "id"
},
"properties": [
{
"id": "custom.width",
"value": 260
}
]
},
{
"matcher": {
"id": "byName",
"options": "Running"
},
"properties": [
{
"id": "unit",
"value": "d"
},
{
"id": "decimals",
"value": 1
},
{
"id": "custom.displayMode",
"value": "color-text"
},
{
"id": "color",
"value": {
"fixedColor": "dark-green",
"mode": "fixed"
}
}
]
}
]
},
"gridPos": {
"h": 10,
"w": 24,
"x": 0,
"y": 27
},
"id": 17,
"options": {
"showHeader": true,
"sortBy": []
},
"pluginVersion": "7.4.5",
"targets": [
{
"expr": "(time() - container_start_time_seconds{instance=~\"$host\",name=~\"$container\",name=~\".+\"})/86400",
"format": "table",
"instant": true,
"interval": "",
"legendFormat": "{{name}}",
"refId": "A"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Containers Info",
"transformations": [
{
"id": "filterFieldsByName",
"options": {
"include": {
"names": [
"container_label_com_docker_compose_project",
"container_label_com_docker_compose_project_working_dir",
"image",
"instance",
"name",
"Value",
"container_label_com_docker_compose_service"
]
}
}
},
{
"id": "organize",
"options": {
"excludeByName": {},
"indexByName": {},
"renameByName": {
"Value": "Running",
"container_label_com_docker_compose_project": "Label",
"container_label_com_docker_compose_project_working_dir": "Working dir",
"container_label_com_docker_compose_service": "Service",
"image": "Registry Image",
"instance": "Instance",
"name": "Name"
}
}
}
],
"type": "table"
}
],
"schemaVersion": 27,
"style": "dark",
"tags": [
"cadvisor",
"docker"
],
"templating": {
"list": [
{
"allValue": ".*",
"current": {},
"datasource": "${DS_PROMETHEUS}",
"definition": "label_values({__name__=~\"container.*\"},instance)",
"description": null,
"error": null,
"hide": 0,
"includeAll": true,
"label": "Host",
"multi": false,
"name": "host",
"options": [],
"query": {
"query": "label_values({__name__=~\"container.*\"},instance)",
"refId": "Prometheus-host-Variable-Query"
},
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 5,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
},
{
"allValue": ".*",
"current": {},
"datasource": "${DS_PROMETHEUS}",
"definition": "label_values({__name__=~\"container.*\", instance=~\"$host\"},name)",
"description": null,
"error": null,
"hide": 0,
"includeAll": true,
"label": "Container",
"multi": false,
"name": "container",
"options": [],
"query": {
"query": "label_values({__name__=~\"container.*\", instance=~\"$host\"},name)",
"refId": "Prometheus-container-Variable-Query"
},
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"sort": 0,
"tagValuesQuery": "",
"tags": [],
"tagsQuery": "",
"type": "query",
"useTags": false
}
]
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Cadvisor exporter",
"uid": "pMEd7m0Mz",
"version": 1,
"description": "Simple exporter for cadvisor only"
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -19,6 +19,9 @@
- { path: "{{ monitoring_dir }}/grafana/provisioning/dashboards", owner: "root", group: "root", mode: "0755" } - { path: "{{ monitoring_dir }}/grafana/provisioning/dashboards", owner: "root", group: "root", mode: "0755" }
- { path: "{{ monitoring_dir }}/ntfy_data", owner: "root", group: "root", mode: "0777" } - { path: "{{ monitoring_dir }}/ntfy_data", owner: "root", group: "root", mode: "0777" }
- { path: "{{ monitoring_dir }}/alertmanager-ntfy", owner: "root", group: "root", mode: "0755" } - { path: "{{ monitoring_dir }}/alertmanager-ntfy", owner: "root", group: "root", mode: "0755" }
- { path: "{{ monitoring_dir }}/victoria-metrics-data", owner: "65534", group: "65534", mode: "0775" }
- { path: "{{ monitoring_dir }}/vmagent", owner: "root", group: "root", mode: "0755" }
- { path: "{{ monitoring_dir }}/vmagent/conf.d", owner: "root", group: "root", mode: "0755" }
- name: Сбросить список целей scrape node-exporter - name: Сбросить список целей scrape node-exporter
ansible.builtin.set_fact: ansible.builtin.set_fact:
@@ -82,13 +85,14 @@
group: root group: root
mode: "0644" mode: "0644"
- name: Установить источник данных Prometheus в Grafana - name: Установить источники данных Grafana
ansible.builtin.template: ansible.builtin.template:
src: grafana-datasources.yml.j2 src: grafana-datasources.yml.j2
dest: "{{ monitoring_dir }}/grafana/provisioning/datasources/prometheus.yml" dest: "{{ monitoring_dir }}/grafana/provisioning/datasources/prometheus.yml"
owner: root owner: root
group: root group: root
mode: "0644" mode: "0644"
register: grafana_datasources
- name: Установить провайдер дашбордов Grafana - name: Установить провайдер дашбордов Grafana
ansible.builtin.template: ansible.builtin.template:
@@ -106,6 +110,51 @@
group: root group: root
mode: "0644" mode: "0644"
- name: Конфиг vmagent на core-mon
ansible.builtin.template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: root
group: root
mode: "0644"
loop:
- { src: vmagent-scrape.yml.j2, dest: "{{ monitoring_dir }}/vmagent/scrape.yml" }
- { src: vmagent-nodeexporter.yml.j2, dest: "{{ monitoring_dir }}/vmagent/conf.d/nodeexporter.yml" }
- { src: vmagent-cadvisor.yml.j2, dest: "{{ monitoring_dir }}/vmagent/conf.d/cadvisor.yml" }
- name: Убрать scrape Xray Checker без ссылки подписки
ansible.builtin.file:
path: "{{ item }}"
state: absent
loop:
- "{{ monitoring_dir }}/vmagent/conf.d/xraychecker.yml"
- "{{ monitoring_dir }}/xray-checker.env"
when: xray_checker_subscription_url | length == 0
- name: Ссылки подписки для Xray Checker нет
ansible.builtin.debug:
msg: "Ссылки Xray Checker нет, контейнер не запускается."
when: xray_checker_subscription_url | length == 0
- name: Файл окружения Xray Checker
ansible.builtin.template:
src: xray-checker.env.j2
dest: "{{ monitoring_dir }}/xray-checker.env"
owner: root
group: root
mode: "0600"
no_log: true
when: xray_checker_subscription_url | length > 0
- name: Scrape Xray Checker
ansible.builtin.template:
src: vmagent-xray.yml.j2
dest: "{{ monitoring_dir }}/vmagent/conf.d/xraychecker.yml"
owner: root
group: root
mode: "0644"
when: xray_checker_subscription_url | length > 0
- name: Установить compose.yaml для стека мониторинга - name: Установить compose.yaml для стека мониторинга
ansible.builtin.template: ansible.builtin.template:
src: compose.yaml.j2 src: compose.yaml.j2
@@ -151,6 +200,20 @@
failed_when: false failed_when: false
no_log: true no_log: true
- name: Дождаться VictoriaMetrics
ansible.builtin.uri:
url: "http://{{ ansible_host }}:8428/health"
status_code: 200
register: victoriametrics_health
retries: 20
delay: 2
until: victoriametrics_health.status == 200
- name: Перезапустить Grafana, чтобы подхватить datasource
ansible.builtin.command: docker compose -f {{ monitoring_dir }}/compose.yaml restart grafana
when: grafana_datasources is changed
changed_when: true
- name: Дождаться готовности Grafana - name: Дождаться готовности Grafana
ansible.builtin.uri: ansible.builtin.uri:
url: "http://127.0.0.1:3000/api/health" url: "http://127.0.0.1:3000/api/health"
@@ -78,4 +78,64 @@ services:
depends_on: depends_on:
- prometheus - prometheus
vmsingle:
image: {{ victoriametrics_image }}
container_name: monitoring-vmsingle
hostname: vmsingle
restart: unless-stopped
command:
- --storageDataPath=/victoria-metrics-data
volumes:
- ./victoria-metrics-data:/victoria-metrics-data
ports:
- "{{ ansible_host }}:8428:8428"
cadvisor:
image: {{ cadvisor_image }}
container_name: monitoring-cadvisor
restart: unless-stopped
privileged: true
command:
- --listen_ip=0.0.0.0
- --port=9101
- --docker_only=true
volumes:
- /:/rootfs:ro
- /var/run:/var/run:ro
- /sys:/sys:ro
- /var/lib/docker:/var/lib/docker:ro
- /dev/disk:/dev/disk:ro
ports:
- "127.0.0.1:9101:9101"
{% if xray_checker_subscription_url | length > 0 %}
xray-checker:
image: {{ xray_checker_image }}
container_name: monitoring-xray-checker
hostname: xraychecker
restart: unless-stopped
env_file:
- ./xray-checker.env
ports:
- "127.0.0.1:2112:2112"
{% endif %}
vmagent:
image: {{ vmagent_image }}
container_name: monitoring-vmagent
restart: unless-stopped
command:
- -httpListenAddr=0.0.0.0:8429
- -promscrape.config=/etc/vmagent/scrape.yml
- -promscrape.configCheckInterval=60s
- -remoteWrite.url=http://vmsingle:8428/api/v1/write
volumes:
- ./vmagent/scrape.yml:/etc/vmagent/scrape.yml:ro
- ./vmagent/conf.d:/etc/vmagent/conf.d:ro
ports:
- "127.0.0.1:8429:8429"
depends_on:
- vmsingle
- cadvisor
@@ -10,3 +10,12 @@ datasources:
jsonData: jsonData:
timeInterval: 15s timeInterval: 15s
- name: VictoriaMetrics
uid: victoriametrics
type: prometheus
access: proxy
url: http://vmsingle:8428
isDefault: false
jsonData:
timeInterval: 15s
@@ -0,0 +1,7 @@
- job_name: integrations/cAdvisor
scrape_interval: 15s
static_configs:
- targets: ["cadvisor:9101"]
labels:
instance: "{{ inventory_hostname }}"
cluster: "{{ vmagent_cluster }}"
@@ -0,0 +1,7 @@
- job_name: integrations/node_exporter
scrape_interval: 15s
static_configs:
- targets: ["{{ ansible_host }}:9100"]
labels:
instance: "{{ inventory_hostname }}"
cluster: "{{ vmagent_cluster }}"
@@ -0,0 +1,5 @@
scrape_config_files:
- "/etc/vmagent/conf.d/*.yml"
global:
scrape_interval: 15s
@@ -0,0 +1,7 @@
- job_name: xray_checker
scrape_interval: 15s
static_configs:
- targets: ["xray-checker:2112"]
labels:
instance: "{{ inventory_hostname }}"
cluster: "{{ vmagent_cluster }}"
@@ -0,0 +1,3 @@
SUBSCRIPTION_URL={{ xray_checker_subscription_url }}
PROXY_STATUS_CHECK_URL=http://google.com/generate_204
PROXY_CHECK_INTERVAL=60
@@ -0,0 +1,4 @@
---
- name: Клиент NetBird на панели
ansible.builtin.include_role:
name: netbird_client
@@ -29,6 +29,9 @@ remnawave_api_token: "{{ lookup('env', 'REMNAWAVE_API_TOKEN', default='') }}"
# Нода Ger-nod-02 / SelfSteal — создаётся через API панели # Нода Ger-nod-02 / SelfSteal — создаётся через API панели
remnawave_node_name: "Ger-nod-02" remnawave_node_name: "Ger-nod-02"
remnawave_node_address: "150.241.64.209" remnawave_node_address: "150.241.64.209"
remnawave_node_netbird_hostname: "ger-nod-02"
remnawave_netbird_peers_url: "https://central.gunihome.ru/api/peers"
remnawave_xray_checker_user: "homelab"
remnawave_node_port: 2222 remnawave_node_port: 2222
remnawave_node_country: "DE" remnawave_node_country: "DE"
remnawave_selfsteal_domain: "yali.gunis.top" remnawave_selfsteal_domain: "yali.gunis.top"
@@ -186,6 +186,37 @@
group: root group: root
mode: "0600" mode: "0600"
- name: Узнать NetBird IP ноды
ansible.builtin.shell: |
set -euo pipefail
if [ -z "${NB_API_TOKEN:-}" ]; then
exit 0
fi
curl -fsS --max-time 20 -H "Authorization: Token ${NB_API_TOKEN}" \
"{{ remnawave_netbird_peers_url }}" \
| jq -r --arg h "$NB_HOST" '
(if type == "array" then . elif type == "object" then (.items // .peers // []) else [] end)
| map(select((.hostname // "") == $h or (.name // "") == $h or (.dns_label // "") == $h))
| .[0].ip // empty'
args:
executable: /bin/bash
environment:
NB_API_TOKEN: "{{ lookup('env', 'NETBIRD_API_TOKEN', default='') }}"
NB_HOST: "{{ remnawave_node_netbird_hostname }}"
register: remnawave_node_nb
changed_when: false
failed_when: false
- name: Подставить NetBird-адрес ноды
ansible.builtin.set_fact:
remnawave_node_address: "{{ remnawave_node_nb.stdout | trim }}"
when: remnawave_node_nb.stdout | default('') | trim is match('^[0-9.]+$')
- name: Peer ноды ещё не в NetBird
ansible.builtin.debug:
msg: "Peer {{ remnawave_node_netbird_hostname }} ещё не в сети, адрес ноды остаётся {{ remnawave_node_address }}"
when: remnawave_node_nb.stdout | default('') | trim is not match('^[0-9.]+$')
- name: Инициализировать админа, API-токен, профиль DE-Yali, ноду и хост - name: Инициализировать админа, API-токен, профиль DE-Yali, ноду и хост
ansible.builtin.shell: | ansible.builtin.shell: |
set -euo pipefail set -euo pipefail
@@ -203,6 +234,8 @@
RW_NODE_NAME: "{{ remnawave_node_name }}" RW_NODE_NAME: "{{ remnawave_node_name }}"
RW_NODE_ADDR: "{{ remnawave_node_address }}" RW_NODE_ADDR: "{{ remnawave_node_address }}"
RW_NODE_PORT: "{{ remnawave_node_port }}" RW_NODE_PORT: "{{ remnawave_node_port }}"
RW_SUB_DOMAIN: "{{ remnawave_sub_domain }}"
RW_XRAY_USER: "{{ remnawave_xray_checker_user }}"
RW_NODE_CC: "{{ remnawave_node_country }}" RW_NODE_CC: "{{ remnawave_node_country }}"
RW_SELFSTEAL: "{{ remnawave_selfsteal_domain }}" RW_SELFSTEAL: "{{ remnawave_selfsteal_domain }}"
RW_XHTTP_PATH: "{{ remnawave_xhttp_path }}" RW_XHTTP_PATH: "{{ remnawave_xhttp_path }}"
@@ -231,6 +264,7 @@
ansible.builtin.set_fact: ansible.builtin.set_fact:
remnawave_api_token: "{{ (remnawave_bootstrap_secrets.content | b64decode | from_json).api_token }}" remnawave_api_token: "{{ (remnawave_bootstrap_secrets.content | b64decode | from_json).api_token }}"
remnawave_node_secret: "{{ (remnawave_bootstrap_secrets.content | b64decode | from_json).node_secret }}" remnawave_node_secret: "{{ (remnawave_bootstrap_secrets.content | b64decode | from_json).node_secret }}"
remnawave_xray_checker_url: "{{ (remnawave_bootstrap_secrets.content | b64decode | from_json).xray_checker_subscription_url | default('') }}"
no_log: true no_log: true
- name: Проверить, что SECRET_KEY ноды получен - name: Проверить, что SECRET_KEY ноды получен
@@ -250,14 +284,20 @@
- name: Сохранить API-токен и SECRET_KEY ноды в Infisical - name: Сохранить API-токен и SECRET_KEY ноды в Infisical
ansible.builtin.shell: | ansible.builtin.shell: |
set -euo pipefail set -euo pipefail
args=(
"REMNAWAVE_API_TOKEN=${RW_API_TOKEN}"
"REMNANODE_SECRET_KEY=${RW_NODE_SECRET}"
)
if [ -n "${RW_XRAY_URL:-}" ]; then
args+=("XRAY_CHECKER_SUBSCRIPTION_URL=${RW_XRAY_URL}")
fi
infisical secrets set \ infisical secrets set \
--domain="${INFISICAL_DOMAIN}" \ --domain="${INFISICAL_DOMAIN}" \
--token="${INFISICAL_TOKEN}" \ --token="${INFISICAL_TOKEN}" \
--env=prod \ --env=prod \
--projectId="${INFISICAL_PROJECT_ID}" \ --projectId="${INFISICAL_PROJECT_ID}" \
--path="/services/remnawave" \ --path="/services/remnawave" \
"REMNAWAVE_API_TOKEN=${RW_API_TOKEN}" \ "${args[@]}" \
"REMNANODE_SECRET_KEY=${RW_NODE_SECRET}" \
--silent >/dev/null --silent >/dev/null
args: args:
executable: /bin/bash executable: /bin/bash
@@ -269,6 +309,7 @@
INFISICAL_PROJECT_ID: "{{ lookup('env', 'INFISICAL_PROJECT_ID') }}" INFISICAL_PROJECT_ID: "{{ lookup('env', 'INFISICAL_PROJECT_ID') }}"
RW_API_TOKEN: "{{ remnawave_api_token }}" RW_API_TOKEN: "{{ remnawave_api_token }}"
RW_NODE_SECRET: "{{ remnawave_node_secret | default(lookup('env', 'REMNANODE_SECRET_KEY', default='')) }}" RW_NODE_SECRET: "{{ remnawave_node_secret | default(lookup('env', 'REMNANODE_SECRET_KEY', default='')) }}"
RW_XRAY_URL: "{{ remnawave_xray_checker_url | default('') }}"
when: remnawave_api_token | default('') | length > 0 when: remnawave_api_token | default('') | length > 0
changed_when: true changed_when: true
no_log: true no_log: true
@@ -321,5 +362,6 @@
- "Админ создан в этом прогоне: {{ remnawave_bootstrap.admin_created | default(false) }}" - "Админ создан в этом прогоне: {{ remnawave_bootstrap.admin_created | default(false) }}"
- "API-токен обновлён в Infisical: {{ remnawave_bootstrap.token_updated | default(false) }}" - "API-токен обновлён в Infisical: {{ remnawave_bootstrap.token_updated | default(false) }}"
- "Профиль {{ remnawave_profile_name }}: {{ remnawave_bootstrap.profile_uuid | default('n/a') }}" - "Профиль {{ remnawave_profile_name }}: {{ remnawave_bootstrap.profile_uuid | default('n/a') }}"
- "Нода {{ remnawave_node_name }}: {{ remnawave_bootstrap.node_uuid | default('n/a') }}" - "Нода {{ remnawave_node_name }}: {{ remnawave_bootstrap.node_uuid | default('n/a') }} адрес {{ remnawave_bootstrap.node_address | default(remnawave_node_address) }}"
- "Ссылка Xray Checker записана в Infisical: {{ remnawave_bootstrap.xray_checker_url_saved | default(false) }}"
- "Host {{ remnawave_selfsteal_domain }}: {{ remnawave_bootstrap.host_uuid | default('n/a') }}" - "Host {{ remnawave_selfsteal_domain }}: {{ remnawave_bootstrap.host_uuid | default('n/a') }}"
@@ -10,6 +10,7 @@ admin_created=0
token_updated=0 token_updated=0
profile_created=0 profile_created=0
node_created=0 node_created=0
node_address_updated=0
host_created=0 host_created=0
node_secret="" node_secret=""
active_token="${RW_API_TOKEN:-}" active_token="${RW_API_TOKEN:-}"
@@ -20,7 +21,7 @@ redact_json() {
jq 'walk( jq 'walk(
if type == "object" then if type == "object" then
with_entries( with_entries(
if (.key | test("(?i)privateKey|privKey|secretKey|SECRET_KEY|accessToken|apiToken|password|^token$|vlessUuid|trojanPassword|ssPassword|shortUuid|subscriptionUrl")) if (.key | test("(?i)privateKey|privKey|secretKey|SECRET_KEY|accessToken|apiToken|password|^token$|vlessUuid|trojanPassword|ssPassword|shortUuid|subscriptionUrl|setupKey|setup_key|METRICS_PASS"))
then .value = "[REDACTED]" then .value = "[REDACTED]"
else . else .
end end
@@ -223,20 +224,29 @@ if [ -z "${node_uuid}" ]; then
node_created=1 node_created=1
fi fi
current_addr=$(printf '%s' "${nodes_json}" | jq -r --arg n "$RW_NODE_NAME" '(.response // []) | map(select(.name == $n)) | .[0].address // empty')
if [ -n "${RW_NODE_ADDR}" ] && [ "${current_addr}" != "${RW_NODE_ADDR}" ]; then
node_address_updated=1
fi
node_update=$(jq -n \ node_update=$(jq -n \
--arg u "$node_uuid" \ --arg u "$node_uuid" \
--arg pu "$profile_uuid" \ --arg pu "$profile_uuid" \
--arg iu "$inbound_uuid" \ --arg iu "$inbound_uuid" \
--arg addr "$RW_NODE_ADDR" \
--argjson port "$RW_NODE_PORT" \
'{ '{
uuid: $u, uuid: $u,
address: $addr,
port: $port,
configProfile: { configProfile: {
activeConfigProfileUuid: $pu, activeConfigProfileUuid: $pu,
activeInbounds: [$iu] activeInbounds: [$iu]
} }
}') }')
api PATCH /nodes/ -d "${node_update}" >/dev/null 2>&1 \ if ! api PATCH /nodes/ -d "${node_update}" >/dev/null \
|| api PUT /nodes/ -d "${node_update}" >/dev/null 2>&1 \ && ! api PUT /nodes/ -d "${node_update}" >/dev/null; then
|| true echo "WARN: не удалось обновить адрес ноды ${RW_NODE_NAME}" >&2
fi
hosts_json=$(api GET /hosts/) hosts_json=$(api GET /hosts/)
host_uuid=$(printf '%s' "${hosts_json}" | jq -r --arg a "$RW_SELFSTEAL" ' host_uuid=$(printf '%s' "${hosts_json}" | jq -r --arg a "$RW_SELFSTEAL" '
@@ -420,11 +430,33 @@ if [ -n "${RW_USERS_FILE:-}" ] && [ -f "${RW_USERS_FILE}" ]; then
done < <(printf '%s' "${existing}" | jq -r '.response.users[]? | "\(.username)\t\(.uuid)"') done < <(printf '%s' "${existing}" | jq -r '.response.users[]? | "\(.username)\t\(.uuid)"')
fi fi
xray_url=""
xray_user="${RW_XRAY_USER:-homelab}"
if [ -n "${xray_user}" ]; then
users_now=$(api GET "/users/?size=500" || api GET /users/ || echo '{}')
xray_url=$(printf '%s' "${users_now}" | jq -r --arg u "${xray_user}" '
.response.users[]? | select(.username == $u) | .subscriptionUrl // empty' | head -1)
if [ -z "${xray_url}" ]; then
short=$(printf '%s' "${users_now}" | jq -r --arg u "${xray_user}" '
.response.users[]? | select(.username == $u) | .shortUuid // empty' | head -1)
if [ -n "${short}" ] && [ -n "${RW_SUB_DOMAIN:-}" ]; then
xray_url="https://${RW_SUB_DOMAIN}/${short}"
fi
fi
fi
xray_saved=0
if [ -n "${xray_url}" ]; then
xray_saved=1
fi
# Секреты только в файл 0600. В stdout — статус без значений ключей. # Секреты только в файл 0600. В stdout — статус без значений ключей.
secrets_file="${RW_SECRETS_FILE:-/opt/remnawave/.bootstrap-secrets.json}" secrets_file="${RW_SECRETS_FILE:-/opt/remnawave/.bootstrap-secrets.json}"
umask 077 umask 077
jq -n --arg api_token "${active_token}" --arg node_secret "${node_secret}" \ jq -n \
'{api_token: $api_token, node_secret: $node_secret}' > "${secrets_file}" --arg api_token "${active_token}" \
--arg node_secret "${node_secret}" \
--arg xray_url "${xray_url}" \
'{api_token: $api_token, node_secret: $node_secret, xray_checker_subscription_url: $xray_url}' > "${secrets_file}"
chmod 600 "${secrets_file}" chmod 600 "${secrets_file}"
jq -n \ jq -n \
@@ -432,22 +464,28 @@ jq -n \
--arg node_uuid "${node_uuid}" \ --arg node_uuid "${node_uuid}" \
--arg host_uuid "${host_uuid}" \ --arg host_uuid "${host_uuid}" \
--arg reality_pub "${pub}" \ --arg reality_pub "${pub}" \
--arg node_address "${RW_NODE_ADDR}" \
--argjson admin_created "${admin_created}" \ --argjson admin_created "${admin_created}" \
--argjson token_updated "${token_updated}" \ --argjson token_updated "${token_updated}" \
--argjson profile_created "${profile_created}" \ --argjson profile_created "${profile_created}" \
--argjson node_created "${node_created}" \ --argjson node_created "${node_created}" \
--argjson node_address_updated "${node_address_updated}" \
--argjson host_created "${host_created}" \ --argjson host_created "${host_created}" \
--argjson users_changed "${users_changed}" \ --argjson users_changed "${users_changed}" \
--argjson xray_saved "${xray_saved}" \
'{ '{
profile_uuid: $profile_uuid, profile_uuid: $profile_uuid,
node_uuid: $node_uuid, node_uuid: $node_uuid,
host_uuid: $host_uuid, host_uuid: $host_uuid,
node_address: $node_address,
reality_public_key: $reality_pub, reality_public_key: $reality_pub,
admin_created: ($admin_created == 1), admin_created: ($admin_created == 1),
token_updated: ($token_updated == 1), token_updated: ($token_updated == 1),
profile_created: ($profile_created == 1), profile_created: ($profile_created == 1),
node_created: ($node_created == 1), node_created: ($node_created == 1),
node_address_updated: ($node_address_updated == 1),
host_created: ($host_created == 1), host_created: ($host_created == 1),
users_changed: ($users_changed == 1), users_changed: ($users_changed == 1),
changed: (($admin_created + $token_updated + $profile_created + $node_created + $host_created + $users_changed) > 0) xray_checker_url_saved: ($xray_saved == 1),
changed: (($admin_created + $token_updated + $profile_created + $node_created + $node_address_updated + $host_created + $users_changed + $xray_saved) > 0)
}' }'
@@ -0,0 +1,6 @@
---
- name: vmagent панели
ansible.builtin.include_role:
name: vmagent
vars:
vmagent_scrape_remnawave: true
@@ -0,0 +1,4 @@
---
- name: Клиент NetBird на ноде
ansible.builtin.include_role:
name: netbird_client
@@ -178,12 +178,49 @@
register: ufw_https register: ufw_https
changed_when: "'added' in (ufw_https.stdout | lower)" changed_when: "'added' in (ufw_https.stdout | lower)"
- name: Узнать NetBird IP панели
ansible.builtin.shell: |
set -euo pipefail
if [ -z "${NB_API_TOKEN:-}" ]; then
exit 0
fi
curl -fsS --max-time 20 -H "Authorization: Token ${NB_API_TOKEN}" \
"https://central.gunihome.ru/api/peers" \
| jq -r --arg h "core-net" '
(if type == "array" then . elif type == "object" then (.items // .peers // []) else [] end)
| map(select((.hostname // "") == $h or (.name // "") == $h or (.dns_label // "") == $h))
| .[0].ip // empty'
args:
executable: /bin/bash
environment:
NB_API_TOKEN: "{{ lookup('env', 'NETBIRD_API_TOKEN', default='') }}"
register: remnanode_panel_nb
changed_when: false
failed_when: false
- name: Разрешить NODE_PORT с NetBird IP панели
ansible.builtin.command: >
ufw allow from {{ remnanode_panel_nb.stdout | trim }} to any port {{ remnanode_node_port }} proto tcp
register: ufw_node_nb
changed_when: "'added' in (ufw_node_nb.stdout | lower)"
when: remnanode_panel_nb.stdout | default('') | trim is match('^[0-9.]+$')
- name: Убрать WAN-доступ к NODE_PORT
ansible.builtin.command: >
ufw --force delete allow from {{ item }} to any port {{ remnanode_node_port }} proto tcp
loop: "{{ remnanode_panel_egress_ips }}"
register: ufw_node_wan_del
changed_when: "'deleted' in (stdout | default('') | lower)"
failed_when: false
when: remnanode_panel_nb.stdout | default('') | trim is match('^[0-9.]+$')
- name: Разрешить NODE_PORT только с egress панели (оба провайдера) - name: Разрешить NODE_PORT только с egress панели (оба провайдера)
ansible.builtin.command: > ansible.builtin.command: >
ufw allow from {{ item }} to any port {{ remnanode_node_port }} proto tcp ufw allow from {{ item }} to any port {{ remnanode_node_port }} proto tcp
loop: "{{ remnanode_panel_egress_ips }}" loop: "{{ remnanode_panel_egress_ips }}"
register: ufw_node register: ufw_node
changed_when: "'added' in (ufw_node.stdout | lower)" changed_when: "'added' in (stdout | default('') | lower)"
when: remnanode_panel_nb.stdout | default('') | trim is not match('^[0-9.]+$')
- name: Включить firewall - name: Включить firewall
ansible.builtin.command: ufw --force enable ansible.builtin.command: ufw --force enable
@@ -238,5 +275,10 @@
ansible.builtin.debug: ansible.builtin.debug:
msg: msg:
- "SelfSteal: https://{{ remnanode_selfsteal_domain }}" - "SelfSteal: https://{{ remnanode_selfsteal_domain }}"
- "NODE_PORT {{ remnanode_node_port }} только с {{ remnanode_panel_egress_ips | join(', ') }}" - >-
{{
'NODE_PORT ' ~ remnanode_node_port ~ ' только с NetBird панели ' ~ (remnanode_panel_nb.stdout | trim)
if (remnanode_panel_nb.stdout | default('') | trim is match('^[0-9.]+$'))
else 'Peer core-net ещё не в сети, NODE_PORT ' ~ remnanode_node_port ~ ' остаётся с WAN egress ' ~ (remnanode_panel_egress_ips | join(', '))
}}
- "Compose: {{ remnanode_dir }}/compose.yaml" - "Compose: {{ remnanode_dir }}/compose.yaml"
@@ -0,0 +1,7 @@
---
- name: vmagent ноды
ansible.builtin.include_role:
name: vmagent
vars:
monitoring_node_exporter_public: false
monitoring_node_exporter_listen: "127.0.0.1:9100"
+3
View File
@@ -1,4 +1,7 @@
--- ---
monitoring_node_exporter_public: true
monitoring_node_exporter_listen: "0.0.0.0:9100"
host_authorized_keys: host_authorized_keys:
- "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF/yxhAs022l/8FlPlhyi7NeF3CvVGsev3B1HtKICO7M root_from_windows" - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF/yxhAs022l/8FlPlhyi7NeF3CvVGsev3B1HtKICO7M root_from_windows"
- "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEpwqB40k8QQq3k8160gfMKDaCJCX9qzvmuijfe3mO4I gitea-runner" - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEpwqB40k8QQq3k8160gfMKDaCJCX9qzvmuijfe3mO4I gitea-runner"
+25 -2
View File
@@ -91,11 +91,22 @@
failed_when: false failed_when: false
changed_when: rm_legacy_exporter.rc == 0 changed_when: rm_legacy_exporter.rc == 0
- name: Слушать node-exporter только локально
ansible.builtin.copy:
dest: /etc/default/prometheus-node-exporter
content: |
ARGS="--web.listen-address={{ monitoring_node_exporter_listen }}"
owner: root
group: root
mode: "0644"
register: node_exporter_args
when: not (monitoring_node_exporter_public | bool)
- name: Включить prometheus-node-exporter - name: Включить prometheus-node-exporter
ansible.builtin.systemd: ansible.builtin.systemd:
name: prometheus-node-exporter name: prometheus-node-exporter
enabled: true enabled: true
state: started state: "{{ 'restarted' if (node_exporter_args is changed) else 'started' }}"
- name: Разрешить SSH - name: Разрешить SSH
ansible.builtin.command: ufw allow OpenSSH ansible.builtin.command: ufw allow OpenSSH
@@ -107,7 +118,19 @@
ansible.builtin.command: ufw allow 9100/tcp ansible.builtin.command: ufw allow 9100/tcp
register: ufw_metrics register: ufw_metrics
changed_when: "'added' in (ufw_metrics.stdout | lower)" changed_when: "'added' in (ufw_metrics.stdout | lower)"
when: ansible_virtualization_type | default('') != 'lxc' when:
- ansible_virtualization_type | default('') != 'lxc'
- monitoring_node_exporter_public | bool
- name: Закрыть порт метрик node-exporter снаружи
ansible.builtin.command: ufw --force delete allow 9100/tcp
register: ufw_metrics_del
changed_when: "'deleted' in (stdout | default('') | lower)"
failed_when: false
loop: [1, 2]
when:
- ansible_virtualization_type | default('') != 'lxc'
- not (monitoring_node_exporter_public | bool)
- name: Проверить состояние firewall - name: Проверить состояние firewall
ansible.builtin.command: ufw status ansible.builtin.command: ufw status
@@ -0,0 +1,4 @@
---
netbird_client_management_url: "https://central.gunihome.ru"
netbird_client_setup_key: "{{ lookup('env', 'NETBIRD_REMNAWAVE_SETUP_KEY', default='') }}"
netbird_client_hostname: "{{ inventory_hostname }}"
@@ -0,0 +1,86 @@
---
- name: Проверить ключ установки Remnawave
ansible.builtin.assert:
that:
- netbird_client_setup_key | length > 0
fail_msg: "Нет NETBIRD_REMNAWAVE_SETUP_KEY. Сначала прогоните ansible-rus-central."
- name: Проверить /dev/net/tun
ansible.builtin.stat:
path: /dev/net/tun
register: netbird_tun
- name: Нет TUN для NetBird
ansible.builtin.fail:
msg: "Нет /dev/net/tun. В LXC для NetBird нужны nesting и устройство /dev/net/tun."
when: not netbird_tun.stat.exists
- name: Поставить зависимости NetBird
ansible.builtin.apt:
name:
- ca-certificates
- curl
- gnupg
state: present
update_cache: true
- name: Ключ репозитория NetBird
ansible.builtin.shell: |
set -euo pipefail
install -d -m 0755 /usr/share/keyrings
curl -fsSL https://pkgs.netbird.io/debian/public.key \
| gpg --dearmor --yes -o /usr/share/keyrings/netbird-archive-keyring.gpg
chmod 0644 /usr/share/keyrings/netbird-archive-keyring.gpg
args:
executable: /bin/bash
creates: /usr/share/keyrings/netbird-archive-keyring.gpg
- name: Репозиторий NetBird
ansible.builtin.copy:
dest: /etc/apt/sources.list.d/netbird.list
content: |
deb [signed-by=/usr/share/keyrings/netbird-archive-keyring.gpg] https://pkgs.netbird.io/debian stable main
owner: root
group: root
mode: "0644"
- name: Поставить клиент NetBird
ansible.builtin.apt:
name: netbird
state: present
update_cache: true
- name: Включить службу NetBird
ansible.builtin.systemd:
name: netbird
enabled: true
state: started
- name: Узнать, подключён ли NetBird
ansible.builtin.command: netbird status
register: netbird_status_before
changed_when: false
failed_when: false
- name: Подключить NetBird к central.gunihome.ru
ansible.builtin.command: >
netbird up
--management-url {{ netbird_client_management_url }}
--setup-key {{ netbird_client_setup_key }}
--hostname {{ netbird_client_hostname }}
no_log: true
when: "'Management: Connected' not in (netbird_status_before.stdout | default(''))"
register: netbird_up
changed_when: true
- name: Дождаться Management Connected
ansible.builtin.command: netbird status
register: netbird_status
changed_when: false
retries: 10
delay: 3
until: "'Management: Connected' in netbird_status.stdout"
- name: Показать состояние NetBird
ansible.builtin.debug:
msg: "{{ netbird_status.stdout_lines | select('search', 'Management:|NetBird IP:|FQDN:|Peers:') | list }}"
@@ -19,5 +19,8 @@ netbird_setup_key: "{{ lookup('env', 'NETBIRD_SETUP_KEY') }}"
netbird_router_group_name: "homelab-routers" netbird_router_group_name: "homelab-routers"
netbird_setup_key_name: "homelab-routers-key" netbird_setup_key_name: "homelab-routers-key"
netbird_remnawave_group_name: "remnawave"
netbird_remnawave_setup_key_name: "remnawave-peers"
netbird_remnawave_setup_key: "{{ lookup('env', 'NETBIRD_REMNAWAVE_SETUP_KEY') }}"
netbird_lan_network_id: "homelab-lan" netbird_lan_network_id: "homelab-lan"
netbird_lan_subnet: "192.168.1.0/24" netbird_lan_subnet: "192.168.1.0/24"
@@ -228,6 +228,20 @@
router_group_id=$(printf '%s' "${grp_resp}" | jq -r '.id') router_group_id=$(printf '%s' "${grp_resp}" | jq -r '.id')
fi fi
remna_group_id=$(printf '%s' "${groups_json}" | jq -r --arg g "$NB_REMNAWAVE_GROUP" '.[] | select(.name == $g) | .id' | head -n 1)
if [ -z "${remna_group_id}" ]; then
grp_payload=$(jq -n --arg name "$NB_REMNAWAVE_GROUP" '{name: $name, peers: []}')
grp_resp=$(curl "${CURL_OPTS[@]}" -X POST "${API_BASE}/groups" \
-H "Authorization: Token ${active_token}" \
-H "Content-Type: application/json" \
-d "${grp_payload}")
remna_group_id=$(printf '%s' "${grp_resp}" | jq -r '.id // empty')
if [ -z "${remna_group_id}" ]; then
echo "Не удалось создать группу NetBird для Remnawave" >&2
exit 1
fi
fi
routes_json=$(curl "${CURL_OPTS[@]}" -H "Authorization: Token ${active_token}" "${API_BASE}/routes") routes_json=$(curl "${CURL_OPTS[@]}" -H "Authorization: Token ${active_token}" "${API_BASE}/routes")
existing_route_id=$(printf '%s' "${routes_json}" | jq -r --arg nid "$NB_LAN_NET_ID" --arg net "$NB_LAN_SUBNET" '.[] | select(.network_id == $nid or .network == $net) | .id' | head -n 1) existing_route_id=$(printf '%s' "${routes_json}" | jq -r --arg nid "$NB_LAN_NET_ID" --arg net "$NB_LAN_SUBNET" '.[] | select(.network_id == $nid or .network == $net) | .id' | head -n 1)
@@ -288,21 +302,57 @@
setup_key_updated=1 setup_key_updated=1
fi fi
active_remna_key="${NB_REMNAWAVE_SETUP_KEY:-}"
remna_key_updated=0
valid_remna_id=$(printf '%s' "${keys_json}" | jq -r --arg kname "$NB_REMNAWAVE_KEY_NAME" '.[] | select(.name == $kname and .valid == true and .revoked == false) | .id' | head -n 1)
if [ -z "${active_remna_key}" ] || [ -z "${valid_remna_id}" ]; then
fresh_keys=$(curl "${CURL_OPTS[@]}" -H "Authorization: Token ${active_token}" "${API_BASE}/setup-keys")
for old_kid in $(printf '%s' "${fresh_keys}" | jq -r --arg kname "$NB_REMNAWAVE_KEY_NAME" '.[] | select(.name == $kname) | .id'); do
curl "${CURL_OPTS[@]}" -X DELETE -H "Authorization: Token ${active_token}" "${API_BASE}/setup-keys/${old_kid}" >/dev/null || true
done
key_payload=$(jq -n \
--arg name "$NB_REMNAWAVE_KEY_NAME" \
--arg gid "$remna_group_id" \
'{
name: $name,
type: "reusable",
expires_in: 31536000,
revoked: false,
auto_groups: [$gid],
usage_limit: 0,
ephemeral: false
}')
key_resp=$(curl "${CURL_OPTS[@]}" -X POST "${API_BASE}/setup-keys" \
-H "Authorization: Token ${active_token}" \
-H "Content-Type: application/json" \
-d "${key_payload}")
active_remna_key=$(printf '%s' "${key_resp}" | jq -r '.key // empty')
if [ -z "${active_remna_key}" ]; then
echo "Не удалось создать ключ установки remnawave-peers" >&2
exit 1
fi
remna_key_updated=1
fi
jq -n \ jq -n \
--arg api_token "${active_token}" \ --arg api_token "${active_token}" \
--arg setup_key "${active_setup_key}" \ --arg setup_key "${active_setup_key}" \
--arg remna_key "${active_remna_key}" \
--argjson admin_created "${admin_created}" \ --argjson admin_created "${admin_created}" \
--argjson token_updated "${token_updated}" \ --argjson token_updated "${token_updated}" \
--argjson setup_key_updated "${setup_key_updated}" \ --argjson setup_key_updated "${setup_key_updated}" \
--argjson remna_key_updated "${remna_key_updated}" \
--argjson route_created "${route_created}" \ --argjson route_created "${route_created}" \
'{ '{
api_token: $api_token, api_token: $api_token,
setup_key: $setup_key, setup_key: $setup_key,
remnawave_setup_key: $remna_key,
admin_created: ($admin_created == 1), admin_created: ($admin_created == 1),
token_updated: ($token_updated == 1), token_updated: ($token_updated == 1),
setup_key_updated: ($setup_key_updated == 1), setup_key_updated: ($setup_key_updated == 1),
remnawave_setup_key_updated: ($remna_key_updated == 1),
route_created: ($route_created == 1), route_created: ($route_created == 1),
changed: (($admin_created + $token_updated + $setup_key_updated + $route_created) > 0) changed: (($admin_created + $token_updated + $setup_key_updated + $remna_key_updated + $route_created) > 0)
}' }'
args: args:
executable: /bin/bash executable: /bin/bash
@@ -314,6 +364,9 @@
NB_SETUP_KEY: "{{ netbird_setup_key }}" NB_SETUP_KEY: "{{ netbird_setup_key }}"
NB_ROUTER_GROUP: "{{ netbird_router_group_name }}" NB_ROUTER_GROUP: "{{ netbird_router_group_name }}"
NB_SETUP_KEY_NAME: "{{ netbird_setup_key_name }}" NB_SETUP_KEY_NAME: "{{ netbird_setup_key_name }}"
NB_REMNAWAVE_GROUP: "{{ netbird_remnawave_group_name }}"
NB_REMNAWAVE_KEY_NAME: "{{ netbird_remnawave_setup_key_name }}"
NB_REMNAWAVE_SETUP_KEY: "{{ netbird_remnawave_setup_key }}"
NB_LAN_NET_ID: "{{ netbird_lan_network_id }}" NB_LAN_NET_ID: "{{ netbird_lan_network_id }}"
NB_LAN_SUBNET: "{{ netbird_lan_subnet }}" NB_LAN_SUBNET: "{{ netbird_lan_subnet }}"
register: netbird_bootstrap_raw register: netbird_bootstrap_raw
@@ -331,6 +384,7 @@
--path="/services/netbird" \ --path="/services/netbird" \
"NETBIRD_API_TOKEN=${NEW_API_TOKEN}" \ "NETBIRD_API_TOKEN=${NEW_API_TOKEN}" \
"NETBIRD_SETUP_KEY=${NEW_SETUP_KEY}" \ "NETBIRD_SETUP_KEY=${NEW_SETUP_KEY}" \
"NETBIRD_REMNAWAVE_SETUP_KEY=${NEW_REMNAWAVE_SETUP_KEY}" \
--silent --silent
args: args:
executable: /bin/bash executable: /bin/bash
@@ -339,7 +393,11 @@
environment: environment:
NEW_API_TOKEN: "{{ (netbird_bootstrap_raw.stdout | from_json).api_token }}" NEW_API_TOKEN: "{{ (netbird_bootstrap_raw.stdout | from_json).api_token }}"
NEW_SETUP_KEY: "{{ (netbird_bootstrap_raw.stdout | from_json).setup_key }}" NEW_SETUP_KEY: "{{ (netbird_bootstrap_raw.stdout | from_json).setup_key }}"
when: (netbird_bootstrap_raw.stdout | from_json).token_updated or (netbird_bootstrap_raw.stdout | from_json).setup_key_updated NEW_REMNAWAVE_SETUP_KEY: "{{ (netbird_bootstrap_raw.stdout | from_json).remnawave_setup_key }}"
when: >-
(netbird_bootstrap_raw.stdout | from_json).token_updated or
(netbird_bootstrap_raw.stdout | from_json).setup_key_updated or
(netbird_bootstrap_raw.stdout | from_json).remnawave_setup_key_updated
changed_when: true changed_when: true
no_log: true no_log: true
@@ -352,4 +410,5 @@
- "Суперпользователь создан в этом прогоне: {{ nb_state.admin_created }}" - "Суперпользователь создан в этом прогоне: {{ nb_state.admin_created }}"
- "API-токен обновлен в Infisical: {{ nb_state.token_updated }}" - "API-токен обновлен в Infisical: {{ nb_state.token_updated }}"
- "Setup-ключ обновлен в Infisical: {{ nb_state.setup_key_updated }}" - "Setup-ключ обновлен в Infisical: {{ nb_state.setup_key_updated }}"
- "Ключ remnawave-peers обновлен в Infisical: {{ nb_state.remnawave_setup_key_updated }}"
- "Маршрут {{ netbird_lan_subnet }} создан в этом прогоне: {{ nb_state.route_created }}" - "Маршрут {{ netbird_lan_subnet }} создан в этом прогоне: {{ nb_state.route_created }}"
+16
View File
@@ -0,0 +1,16 @@
---
vmagent_dir: /opt/vmagent
vmagent_image: victoriametrics/vmagent:v1.153.0
# Публичный образ cAdvisor заканчивается на v0.49.2. Бинарника v0.60.6 из гайда в registry нет.
cadvisor_image: gcr.io/cadvisor/cadvisor:v0.49.2
vmagent_remote_host: "192.168.1.108"
vmagent_remote_write_url: "http://{{ vmagent_remote_host }}:8428/api/v1/write"
vmagent_remote_health_url: "http://{{ vmagent_remote_host }}:8428/health"
vmagent_instance: "{{ inventory_hostname }}"
vmagent_cluster: homelab
vmagent_scrape_remnawave: false
vmagent_metrics_user: admin
vmagent_metrics_pass: "{{ lookup('env', 'REMNAWAVE_METRICS_PASS', default='') }}"
vmagent_metrics_port: 3001
monitoring_node_exporter_public: true
monitoring_node_exporter_listen: "0.0.0.0:9100"
+106
View File
@@ -0,0 +1,106 @@
---
- name: Проверить пароль метрик панели
ansible.builtin.assert:
that:
- vmagent_metrics_pass | length > 0
fail_msg: "Нет REMNAWAVE_METRICS_PASS для scrape панели."
when: vmagent_scrape_remnawave | bool
- name: Слушать node-exporter локально, если порт не публичный
ansible.builtin.copy:
dest: /etc/default/prometheus-node-exporter
content: |
ARGS="--web.listen-address={{ monitoring_node_exporter_listen }}"
owner: root
group: root
mode: "0644"
register: vmagent_exporter_env
when: not (monitoring_node_exporter_public | bool)
- name: Перезапустить node-exporter после смены адреса
ansible.builtin.systemd:
name: prometheus-node-exporter
state: restarted
enabled: true
when: vmagent_exporter_env is changed
- name: Дождаться VictoriaMetrics
ansible.builtin.command: curl -fsS --max-time 5 {{ vmagent_remote_health_url }}
register: vmagent_vm_health
changed_when: false
retries: 12
delay: 5
until: vmagent_vm_health.rc == 0
failed_when: false
- name: VictoriaMetrics недоступна
ansible.builtin.fail:
msg: >-
{{ vmagent_remote_health_url }} не отвечает.
Сначала поднимите VictoriaMetrics на core-mon.
С ger-nod-02 этот адрес открывается маршрутом homelab-lan через OPNsense.
when: vmagent_vm_health.rc != 0
- name: Каталоги vmagent
ansible.builtin.file:
path: "{{ item }}"
state: directory
owner: root
group: root
mode: "0755"
loop:
- "{{ vmagent_dir }}"
- "{{ vmagent_dir }}/conf.d"
- name: Конфиг scrape vmagent
ansible.builtin.template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: root
group: root
mode: "{{ item.mode }}"
loop:
- { src: scrape.yml.j2, dest: "{{ vmagent_dir }}/scrape.yml", mode: "0644" }
- { src: nodeexporter.yml.j2, dest: "{{ vmagent_dir }}/conf.d/nodeexporter.yml", mode: "0644" }
- { src: cadvisor.yml.j2, dest: "{{ vmagent_dir }}/conf.d/cadvisor.yml", mode: "0644" }
- { src: compose.yaml.j2, dest: "{{ vmagent_dir }}/compose.yaml", mode: "0644" }
- name: Убрать scrape панели, если он не нужен
ansible.builtin.file:
path: "{{ vmagent_dir }}/conf.d/remnawave.yml"
state: absent
when: not (vmagent_scrape_remnawave | bool)
- name: Scrape метрик панели
ansible.builtin.template:
src: remnawave.yml.j2
dest: "{{ vmagent_dir }}/conf.d/remnawave.yml"
owner: root
group: root
mode: "0600"
no_log: true
when: vmagent_scrape_remnawave | bool
- name: Запустить vmagent и cAdvisor
ansible.builtin.command: docker compose -f {{ vmagent_dir }}/compose.yaml up -d --remove-orphans
register: vmagent_up
changed_when: >-
'Started' in vmagent_up.stderr or
'Created' in vmagent_up.stderr or
'Recreated' in vmagent_up.stderr or
'Pulled' in vmagent_up.stderr
- name: Дождаться локального vmagent
ansible.builtin.uri:
url: http://127.0.0.1:8429/health
status_code: 200
register: vmagent_local
retries: 15
delay: 2
until: vmagent_local.status == 200
- name: Итог vmagent
ansible.builtin.debug:
msg:
- "vmagent {{ inventory_hostname }} пишет в {{ vmagent_remote_write_url }}"
- "instance={{ vmagent_instance }} scrape панели={{ vmagent_scrape_remnawave | bool }}"
@@ -0,0 +1,7 @@
- job_name: integrations/cAdvisor
scrape_interval: 15s
static_configs:
- targets: ["127.0.0.1:9101"]
labels:
instance: "{{ vmagent_instance }}"
cluster: "{{ vmagent_cluster }}"
@@ -0,0 +1,33 @@
services:
cadvisor:
image: {{ cadvisor_image }}
container_name: vmagent-cadvisor
network_mode: host
privileged: true
restart: unless-stopped
command:
- --listen_ip=127.0.0.1
- --port=9101
- --docker_only=true
volumes:
- /:/rootfs:ro
- /var/run:/var/run:ro
- /sys:/sys:ro
- /var/lib/docker:/var/lib/docker:ro
- /dev/disk:/dev/disk:ro
vmagent:
image: {{ vmagent_image }}
container_name: vmagent
network_mode: host
restart: unless-stopped
command:
- -httpListenAddr=127.0.0.1:8429
- -promscrape.config=/etc/vmagent/scrape.yml
- -promscrape.configCheckInterval=60s
- -remoteWrite.url={{ vmagent_remote_write_url }}
volumes:
- ./scrape.yml:/etc/vmagent/scrape.yml:ro
- ./conf.d:/etc/vmagent/conf.d:ro
depends_on:
- cadvisor
@@ -0,0 +1,7 @@
- job_name: integrations/node_exporter
scrape_interval: 15s
static_configs:
- targets: ["127.0.0.1:9100"]
labels:
instance: "{{ vmagent_instance }}"
cluster: "{{ vmagent_cluster }}"
@@ -0,0 +1,10 @@
- job_name: remnawave
scrape_interval: 15s
static_configs:
- targets: ["127.0.0.1:{{ vmagent_metrics_port }}"]
labels:
instance: "{{ vmagent_instance }}"
cluster: "{{ vmagent_cluster }}"
basic_auth:
username: "{{ vmagent_metrics_user }}"
password: "{{ vmagent_metrics_pass }}"
@@ -0,0 +1,5 @@
scrape_config_files:
- "/etc/vmagent/conf.d/*.yml"
global:
scrape_interval: 15s