OpenSearch

If you like this project, consider supporting me on Buy Me a Coffee ☕️

OpenSearch is an open-source, community-driven search and analytics suite derived from Elasticsearch 7.10 and Kibana 7.10 (forked by AWS in 2021 after Elastic changed to SSPL license). It provides full-text search, log analytics, observability, and security analytics capabilities.

Common use cases / Типичные сценарии: Log aggregation and analysis (ELK/EFK replacement), full-text search engines, application performance monitoring (APM), security information and event management (SIEM), observability.

Status / Статус: Actively developed under Apache 2.0 license (truly open-source). Current stable: 2.x. Alternatives: Elasticsearch (original, SSPL/Elastic License), Zinc (lightweight Go alternative), Meilisearch (developer-friendly search), Typesense (typo-tolerant search). OpenSearch Dashboards replaces Kibana.

Default ports / Порты по умолчанию: 9200 (REST API), 9300 (Transport/inter-node)

📚 Table of Contents / Содержание

1. Installation & Configuration / Установка и Настройка
2. Cluster Management / Управление Кластером
3. Index Management / Управление Индексами
4. Document Management / Управление Документами
5. Backup & Restore / Бэкап и Восстановление
6. Security / Безопасность
7. Sysadmin Operations / Сисадминские Операции
8. Tools / Инструменты
9. Logrotate Configuration / Конфигурация Logrotate

Installation & Configuration

Install / Установка

env OPENSEARCH_INITIAL_ADMIN_PASSWORD=<PASSWORD> dpkg -i opensearch-2.19.2-linux-x64.deb # Install .deb with admin pass / Установка .deb с паролем
# or / или
tar -zxvf opensearch-1.3.0.tar.gz # Extract archive / Распаковка архива
./opensearch-tar-install.sh # Run installer / Запуск установщика
docker run opensearchproject/opensearch:3.7.0 # Run in Docker / Запуск в Docker

System Tuning / Настройка системы

# /etc/sysctl.d/99-opensearch.conf
vm.max_map_count = 262144
vm.swappiness = 1
fs.file-max = 262144

# Apply sysctl / Применить sysctl
sysctl --system

# /etc/security/limits.d/99-opensearch.conf
opensearch soft nofile 65536
opensearch hard nofile 65536
opensearch soft nproc  4096
opensearch hard nproc  4096
opensearch soft memlock unlimited
opensearch hard memlock unlimited

Systemd Override / Systemd Переопределение

systemctl edit opensearch

[Service]
LimitNOFILE=65536
LimitNPROC=4096
LimitMEMLOCK=infinity

Cluster Management

curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/                                   # Ping cluster (version) / Пинг кластера (версия)
curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/_cluster/health?pretty             # Cluster health / Здоровье кластера
curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/_cat/nodes?v                       # Nodes table / Таблица узлов
curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/_cluster/settings                  # View cluster settings / Настройки кластера

Drain Node / Исключение ноды

# Exclude node by name (drain) / Исключить ноду по имени
curl -XPUT localhost:9200/_cluster/settings -H 'Content-Type: application/json' -d '{
  "transient": { "cluster.routing.allocation.exclude._name": "os-data-1" }
}'

# Check shards on node / Проверить шарды на ноде
curl -s localhost:9200/_cat/shards?v | grep os-data-1 || echo "empty"

Index Management

Basics / Основы

curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/_cat/indices?v                     # Indices table / Таблица индексов
curl -sS -u 'admin:<PASSWORD>' -X DELETE http://localhost:9200/my-index                 # Delete index / Удалить индекс

# Create index with settings / Создать индекс с настройками
curl -sS -u 'admin:<PASSWORD>' -X PUT http://localhost:9200/my-index \
  -H 'Content-Type: application/json' \
  -d '{ "settings": {"number_of_shards":1,"number_of_replicas":1} }'

# Show mapping / Показать mapping
curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/my-index/_mapping?pretty

# Update mapping (add field) / Обновить mapping (добавить поле)
curl -sS -u 'admin:<PASSWORD>' -X PUT http://localhost:9200/my-index/_mapping \
  -H 'Content-Type: application/json' \
  -d '{ "properties": { "tag": { "type": "keyword" } } }'

# Force merge / Форс-мердж
curl -u 'admin:<PASSWORD>' -X POST "http://localhost:9200/my-index/_forcemerge?max_num_segments=1"

📘 Reindexing Guide / Руководство по реиндексации

1. Create Destination Index / Создание целевого индекса

curl -u 'admin:<PASSWORD>' -X PUT "http://localhost:9200/logs-v2" -H 'Content-Type: application/json' -d '{
  "settings": {
    "number_of_shards": 6,
    "number_of_replicas": 0,     // Disable replicas for speed / Без реплик для скорости
    "refresh_interval": "-1"     // Disable refresh for speed / Отключить refresh
  },
  "mappings": { "dynamic": "strict", "properties": { ... } }
}'

2. Run Reindex / Запуск

# Basic / Базовый
curl -u 'admin:<PASSWORD>' -X POST "http://localhost:9200/_reindex?refresh=true" -H 'Content-Type: application/json' -d '{
  "source": { "index": "logs-legacy" },
  "dest":   { "index": "logs-v2" }
}'

# Percentage/Async / Асинхронно
curl -u 'admin:<PASSWORD>' -X POST "http://localhost:9200/_reindex?wait_for_completion=false" ...

3. Aliases / Алиасы (Zero Downtime)

curl -u 'admin:<PASSWORD>' -X POST "http://localhost:9200/_aliases" -H 'Content-Type: application/json' -d '{
  "actions": [
    { "remove": { "alias": "logs", "index": "logs-legacy" } },
    { "add":    { "alias": "logs", "index": "logs-v2", "is_write_index": true } }
  ]
}'

Document Management

CRUD

# Index/Replace (ID=1) / Создать/Заменить (ID=1)
curl -sS -u 'admin:<PASSWORD>' -X POST http://localhost:9200/my-index/_doc/1 \
  -H 'Content-Type: application/json' \
  -d '{ "title":"hello","tag":"demo","ts":"2025-08-27T10:00:00Z" }'

# Get Document / Получить документ
curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/my-index/_doc/1

# Partial Update / Частичное обновление
curl -sS -u 'admin:<PASSWORD>' -X POST http://localhost:9200/my-index/_update/1 \
  -H 'Content-Type: application/json' \
  -d '{ "doc": { "tag":"updated" } }'

# Delete Document / Удалить документ
curl -sS -u 'admin:<PASSWORD>' -X DELETE http://localhost:9200/my-index/_doc/1

Search / Поиск

# Simple Match / Простой поиск
curl -sS -u 'admin:<PASSWORD>' -X POST http://localhost:9200/my-index/_search \
  -H 'Content-Type: application/json' \
  -d '{ "query": { "match": { "title": "hello" } }, "size": 5 }'

Bulk / Массовая загрузка

# bulk.json format / формат
# { "index": { "_index":"my-index","_id":"2" } }
# { "title":"bulk item 2","tag":"demo" }

curl -sS -u 'admin:<PASSWORD>' -X POST http://localhost:9200/_bulk \
  -H 'Content-Type: application/x-ndjson' --data-binary @bulk.json                 # Execute Bulk / Выполнить Bulk

Backup & Restore

Snapshot Repository / Репозиторий

# Register FS Repository / Регистрация FS репозитория
curl -X PUT "https://localhost:9200/_snapshot/my_backup_repository" \
     -H "Content-Type: application/json" \
     -u 'admin:<PASSWORD>' --insecure \
     -d '{ "type": "fs", "settings": { "location": "/var/backups/opensearch", "compress": true } }'

# Register S3 Repository / Регистрация S3 репозитория
# Use type "s3" and configure bucket/base_path / Используйте type "s3" и укажите bucket

Snapshots / Снапшоты

# Create Snapshot / Создать снапшот
curl -X PUT "https://localhost:9200/_snapshot/my_backup_repository/snapshot_$(date +%Y-%m-%d)" \
     -H "Content-Type: application/json" \
     -u 'admin:<PASSWORD>' --insecure \
     -d '{ "indices": "*", "ignore_unavailable": true, "include_global_state": false }'

# List Snapshots / Список снапшотов
curl -sS -u 'admin:<PASSWORD>' http://localhost:9200/_snapshot/my_backup_repository/_all?pretty

Restore / Восстановление

# Restore All / Восстановить всё
curl -X POST "https://localhost:9200/_snapshot/my_backup_repository/snapshot_DATE/_restore" \
     -u 'admin:<PASSWORD>' --insecure \
     -H 'Content-Type: application/json' \
     -d '{ "indices": "*", "ignore_unavailable": true, "include_global_state": false }'

# Restore with Rename / Восстановить с переименованием
curl -X POST "https://localhost:9200/_snapshot/my_backup_repository/snapshot_DATE/_restore" \
     -u 'admin:<PASSWORD>' --insecure \
     -H 'Content-Type: application/json' \
     -d '{
       "indices": "logs-*",
       "rename_pattern": "^(.*)$",
       "rename_replacement": "restored_$1"
     }'

🕒 Automation Script / Скрипт автоматизации

#!/bin/bash
# /usr/local/bin/os-backup.sh
SNAPSHOT_NAME="snapshot_$(date +%Y-%m-%d)"
curl -X PUT "https://localhost:9200/_snapshot/my_backup_repository/$SNAPSHOT_NAME" \
     -H "Content-Type: application/json" -u 'admin:<PASSWORD>' --insecure \
     -d '{ "indices": "*", "ignore_unavailable": true, "include_global_state": false }'
echo "✅ Snapshot $SNAPSHOT_NAME is created"

# Clean old snapshots (example > 7 days) / Очистка старых (> 7 дней)
OLD=$(date -d "-7 days" +%Y-%m-%d)
curl -s -u 'admin:<PASSWORD>' -X DELETE "https://localhost:9200/_snapshot/my_backup_repository/snapshot_${OLD}*"

Cron:

0 3 * * * /usr/local/bin/os-backup.sh >> /var/log/opensearch_backup.log 2>&1

📡 NFS Setup for Shared Repo / Настройка NFS

Goal/Цель: /mnt/backups shared across all nodes. / /mnt/backups доступен на всех нодах.

Server (NFS Host):

sudo dnf install -y nfs-utils                                       # Install NFS tools / Установка утилит
sudo mkdir -p /mnt/backups && sudo chown opensearch:opensearch /mnt/backups # Create & Chown / Создать и права
echo "/mnt/backups <SUBNET>/24(rw,sync,no_subtree_check)" | sudo tee -a /etc/exports # Export / Экспорт
sudo exportfs -ra && sudo systemctl enable --now nfs-server         # Apply & Start / Применить и запустить

Clients (OpenSearch Nodes):

sudo dnf install -y nfs-utils                                       # Install tools / Установка утилит
sudo mkdir -p /mnt/backups                                          # Create mountpoint / Точка монтирования
# Update /etc/fstab
echo "<NFS_SERVER_IP>:/mnt/backups /mnt/backups nfs rw,noatime,hard,intr,_netdev,nfsvers=4.2 0 0" | sudo tee -a /etc/fstab
sudo mount -a                                                       # Mount / Примонтировать

Security

User Management / Управление пользователями

# Create Read-Only Role / Создать роль (Read-Only)
curl -u 'admin:<PASSWORD>' -X PUT "http://localhost:9200/_plugins/_security/api/roles/ro-logs" \
  -H "Content-Type: application/json" -d '{
  "cluster_permissions": [],
  "index_permissions": [ { "index_patterns": ["logs-*"], "allowed_actions": ["read"] } ] }'

# Create User / Создать пользователя
curl -u 'admin:<PASSWORD>' -X PUT "http://localhost:9200/_plugins/_security/api/internalusers/alice" \
  -H "Content-Type: application/json" -d '{ "password": "<STRONG_PASSWORD>", "backend_roles": [], "description": "RO User" }'

# Map User to Role / Привязать пользователя к роли
curl -u 'admin:<PASSWORD>' -X PUT "http://localhost:9200/_plugins/_security/api/rolesmapping/ro-logs" \
  -H "Content-Type: application/json" -d '{ "users": ["alice"], "backend_roles": [], "hosts": [] }'

Action Groups / Группы действий

Change Admin Password / Смена пароля Admin

# Using securityadmin.sh (requires certificates) / Через скрипт (нужны сертификаты)
OPENSEARCH_JAVA_HOME=/usr/share/opensearch/jdk /usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh \
  -icl -cacert /etc/opensearch/root-ca.pem \
  -cert /etc/opensearch/kirk.pem -key /etc/opensearch/kirk-key.pem \
  -t internalusers -f current-config/internal_users.yml

Sysadmin Operations

Service & Logs / Сервис и Логи

systemctl start opensearch                                      # Start service / Запустить сервис
systemctl stop opensearch                                       # Stop service / Остановить сервис
systemctl status opensearch                                     # Status / Статус
journalctl -u opensearch -f                                     # Follow service logs / Следить за логами сервиса
tail -f /var/log/opensearch/opensearch.log                      # Follow main log / Следить за основным логом
grep "ERROR" /var/log/opensearch/opensearch.log                 # Find errors / Найти ошибки

Important Paths / Важные Пути

• Config: /etc/opensearch/opensearch.yml — Main config / Основной конфиг
• JVM: /etc/opensearch/jvm.options — Heap size & GC / Настройки Java
• Logs: /var/log/opensearch/ — Logs directory / Каталог логов
• Data: /var/lib/opensearch/ — Data directory (Indices) / Данные (Индексы)
• Home: /usr/share/opensearch/ — Binary home / Домашняя директория

JVM Tuning / Настройка JVM

# Edit /etc/opensearch/jvm.options
-Xms4g
-Xmx4g
# Rule: Set Xms and Xmx to 50% of RAM, but not more than 31GB
# Правило: Ставьте Xms и Xmx = 50% RAM, но не больше 31GB

Network & Firewall / Сеть и Файрвол

# Ports / Порты
# 9200: REST API (HTTP)
# 9300: Transport (Inter-node communication)

# Firewalld Rules / Правила Firewalld
firewall-cmd --permanent --add-port=9200/tcp
firewall-cmd --permanent --add-port=9300/tcp
firewall-cmd --reload

Keystore / Хранилище секретов

# List secrets / Список секретов
/usr/share/opensearch/bin/opensearch-keystore list

# Add secret (e.g. s3 keys) / Добавить секрет
/usr/share/opensearch/bin/opensearch-keystore add s3.client.default.access_key

Plugins / Плагины

# List plugins / Список плагинов
/usr/share/opensearch/bin/opensearch-plugin list

# Install plugin / Установить плагин
/usr/share/opensearch/bin/opensearch-plugin install analysis-icu

# Remove plugin / Удалить плагин
/usr/share/opensearch/bin/opensearch-plugin remove analysis-icu

Tools

cURL Toolbox

curl -L https://example.com                                       # Follow redirects / Следовать редиректам
curl -H 'Content-Type: application/json' -d @file.json URL        # POST from file / POST из файла
curl -s -o /dev/null -w '%{http_code}\n' URL                      # Print Status Code / Вывести HTTP код
curl -u 'user:pass' --insecure URL                                # Auth & Skip SSL / Авторизация и игнор SSL
curl -s -o /dev/null -w 'dns:%{time_namelookup} total:%{time_total}\n' URL  # Timing / Тайминги

Elasticdump

# Ignore SSL / Игнорировать SSL
export NODE_TLS_REJECT_UNAUTHORIZED=0

# Export Mapping / Экспорт Mapping
elasticdump --input=https://admin:<PASSWORD>@localhost:9200/my-index --output=mapping.json --type=mapping

# Export Data / Экспорт Данных
elasticdump --input=https://admin:<PASSWORD>@localhost:9200/my-index --output=data.json --type=data

# Import / Импорт
elasticdump --input=data.json --output=https://admin:<PASSWORD>@target:9200/my-index --type=data

Logrotate Configuration

/etc/logrotate.d/opensearch

/var/log/opensearch/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 640 opensearch opensearch
    sharedscripts
    postrotate
        systemctl reload opensearch > /dev/null 2>&1 || true
    endscript
}

[!NOTE] OpenSearch can also manage logs internally via logging.yml configuration. OpenSearch также может управлять логами через конфигурацию logging.yml.