AWX (Ansible Tower)

If you like this project, consider supporting me on Buy Me a Coffee ☕️

tags:

• infrastructure
• sysadmin

• devops

  AWX (Ansible Tower) — IT Automation Platform

Description / Описание: AWX is the open-source upstream project for Red Hat Ansible Automation Platform (formerly Ansible Tower). It provides a web-based UI, REST API, and task engine built on top of Ansible for centralized automation management. AWX allows teams to manage inventories, launch and schedule Ansible playbooks, track job results, and integrate with CI/CD pipelines via its REST API. It is widely used for infrastructure automation, configuration management, and orchestration.

[!NOTE] Current Status: AWX is actively developed and is the de facto standard for self-hosted Ansible automation UI. For enterprise support, Red Hat offers Ansible Automation Platform (commercial). Alternatives include Semaphore UI (lightweight, Go-based), Rundeck, and StackStorm. / Текущий статус: AWX активно развивается. Для промышленной поддержки Red Hat предлагает Ansible Automation Platform. Альтернативы: Semaphore UI, Rundeck, StackStorm.

Default Ports: Web UI: 8052 (HTTP), 8053 (HTTPS) | PostgreSQL: 5432

Installation & Configuration

[!IMPORTANT] AWX 18+ requires Kubernetes (K8s/K3s/Minikube) or Docker Compose for deployment. Standalone RPM/DEB installs are no longer supported. / AWX 18+ требует Kubernetes или Docker Compose для развёртывания.

Install via AWX Operator (Kubernetes)

# Install K3s (lightweight Kubernetes)
curl -sfL https://get.k3s.io | sh -

# Install AWX Operator via Helm
helm repo add awx-operator https://ansible.github.io/awx-operator/
helm repo update
helm install awx-operator awx-operator/awx-operator -n awx --create-namespace

AWX Instance CR

awx-instance.yaml

apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
  namespace: awx
spec:
  service_type: NodePort
  nodeport_port: 30080
  admin_user: admin
  postgres_storage_class: local-path

# Deploy AWX instance
kubectl apply -f awx-instance.yaml -n awx

# Check deployment status
kubectl get pods -n awx -w

# Get admin password
kubectl get secret awx-admin-password -n awx -o jsonpath='{.data.password}' | base64 -d; echo

Install via Docker Compose

# Clone AWX repository
git clone -b <VERSION_TAG> https://github.com/ansible/awx.git
cd awx

# Copy and edit inventory
cp tools/docker-compose/inventory.example tools/docker-compose/inventory

# Build and start
make docker-compose-build
make docker-compose

Deployment Methods Comparison

Essential Configuration

AWX settings are managed via the Web UI or REST API. Key settings:

# Configure AWX settings via CLI
awx-manage shell_plus  # Django management shell / Управляющая оболочка Django

# In Django shell
from awx.conf.models import Setting
Setting.objects.filter(key='AWX_TASK_ENV').update(value={'ANSIBLE_TIMEOUT': '60'})

Core Management

Web UI Access

# Web UI endpoints
http://<HOST>:8052/           # Web UI
https://<HOST>:8053/          # HTTPS (if configured / если настроен)
http://<HOST>:8052/api/v2/    # REST API

awx CLI (tower-cli replacement) / CLI awx

# Install awx CLI
pip install awxkit

# Configure connection
export TOWER_HOST=http://<HOST>:8052
export TOWER_USERNAME=<USER>
export TOWER_PASSWORD=<PASSWORD>

Common CLI Commands

# List organizations
awx organizations list

# List projects
awx projects list

# List inventories
awx inventories list

# List job templates
awx job_templates list

# Launch a job
awx job_templates launch <TEMPLATE_ID> --monitor

# List running jobs
awx jobs list --status running

# Cancel a job
awx jobs cancel <JOB_ID>

# List hosts in inventory
awx hosts list --inventory <INVENTORY_ID>

# List credentials
awx credentials list

# List workflow templates
awx workflow_job_templates list

AWX Object Hierarchy

Credential Types

Sysadmin Operations

Service Management (Kubernetes)

# Check AWX pods
kubectl get pods -n awx

# Check service status
kubectl get svc -n awx

# View AWX task pod logs
kubectl logs -f deployment/awx-task -n awx

# View AWX web pod logs
kubectl logs -f deployment/awx-web -n awx

# Restart AWX deployment
kubectl rollout restart deployment/awx-web -n awx
kubectl rollout restart deployment/awx-task -n awx

# Scale AWX workers
kubectl scale deployment awx-task --replicas=3 -n awx

Django Management Commands

# Enter AWX task container
kubectl exec -it deployment/awx-task -n awx -- bash

# Useful awx-manage commands inside container
awx-manage inventory_import --source=/path/to/hosts --inventory-name="<INVENTORY>"  # Import inventory / Импорт инвентаря
awx-manage createsuperuser                            # Create admin user / Создать суперпользователя
awx-manage changepassword <USER>                      # Change password / Сменить пароль
awx-manage cleanup_jobs --days=30                     # Clean old job data / Очистить старые данные задач
awx-manage cleanup_activitystream --days=90           # Clean activity stream / Очистить журнал активности
awx-manage list_instances                             # List cluster instances / Список экземпляров кластера
awx-manage update_password --username=admin --password=<PASSWORD>  # Reset admin password / Сбросить пароль admin

[!WARNING] cleanup_jobs permanently deletes job history. Make sure to configure appropriate retention. / cleanup_jobs безвозвратно удаляет историю задач. Настройте подходящий период хранения.

Important Paths (Container)

Log Locations

# AWX task logs
kubectl logs deployment/awx-task -n awx --tail=100

# AWX web logs
kubectl logs deployment/awx-web -n awx --tail=100

# Inside container
tail -f /var/log/tower/tower.log              # Main app log / Основной лог
tail -f /var/log/tower/callback_receiver.log  # Callback receiver / Приёмник callback
tail -f /var/log/tower/dispatcher.log         # Task dispatcher / Диспетчер задач
tail -f /var/log/tower/rsyslog.err            # Rsyslog errors / Ошибки rsyslog

Security

User & Team Management

# Create user via CLI
awx users create --username <USER> --password <PASSWORD> --email <EMAIL>

# List users
awx users list

# Add user to team
awx teams associate --user <USER_ID> <TEAM_ID>

Role-Based Access Control (RBAC)

LDAP/AD Integration

Configure via Web UI: Settings → Authentication → LDAP

Key parameters:

/etc/tower/conf.d/ldap.py (or via API)

AUTH_LDAP_SERVER_URI = "ldaps://<LDAP_HOST>:636"
AUTH_LDAP_BIND_DN = "cn=<BIND_USER>,dc=example,dc=com"
AUTH_LDAP_BIND_PASSWORD = "<PASSWORD>"
AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,dc=example,dc=com", ldap.SCOPE_SUBTREE, "(uid=%(user)s)")

Backup & Restore

Backup PostgreSQL

# From Kubernetes
kubectl exec -it deployment/awx-postgres-13 -n awx -- \
  pg_dump -U awx -d awx | gzip > /backup/awx_db_$(date +%F).sql.gz

# Backup AWX secrets
kubectl get secret -n awx -o yaml > /backup/awx_secrets_$(date +%F).yaml

Production Runbook: Full Backup

1. Backup the database / Сделать бэкап БД:

   kubectl exec -it deployment/awx-postgres-13 -n awx -- \
     pg_dump -U awx -d awx | gzip > /backup/awx_db_$(date +%F).sql.gz

2. Backup Kubernetes secrets / Бэкап секретов K8s:

   kubectl get secret -n awx -o yaml > /backup/awx_secrets_$(date +%F).yaml

3. Verify backup integrity / Проверить целостность бэкапа:

   ls -lh /backup/awx_db_$(date +%F).sql.gz
   gunzip -t /backup/awx_db_$(date +%F).sql.gz  # Test gzip integrity / Проверить целостность gzip

Restore Runbook

[!CAUTION] Restore will overwrite all current data. Perform only on a clean system or after understanding the full impact. / Восстановление перезапишет все данные. Выполняйте только на чистой системе.

1. Restore database / Восстановить БД:

   kubectl exec -i deployment/awx-postgres-13 -n awx -- \
     psql -U awx -d awx < <(gunzip -c /backup/awx_db_<DATE>.sql.gz)

2. Restore secrets / Восстановить секреты:

   kubectl apply -f /backup/awx_secrets_<DATE>.yaml

3. Restart AWX pods / Перезапустить поды AWX:

   kubectl rollout restart deployment/awx-web -n awx
   kubectl rollout restart deployment/awx-task -n awx

Troubleshooting & Tools

Common Issues

1. Jobs Stuck in "Pending"

# Check task pod logs
kubectl logs deployment/awx-task -n awx --tail=50 | grep -i error

# Check dispatcher
kubectl exec -it deployment/awx-task -n awx -- awx-manage list_instances

# Restart task pod
kubectl rollout restart deployment/awx-task -n awx

2. Project Sync Fails

# Check SCM credentials
awx credentials get <CREDENTIAL_ID>

# Test Git connectivity from container
kubectl exec -it deployment/awx-task -n awx -- git ls-remote <REPO_URL>

# Clear project cache
kubectl exec -it deployment/awx-task -n awx -- rm -rf /var/lib/awx/projects/<PROJECT_NAME>

[!WARNING] Clearing project cache forces a full re-clone from SCM on next sync. Ensure SCM connectivity before clearing. / Очистка кэша проекта приведёт к полному повторному клонированию из SCM.

3. High Memory Usage

# Check pod resource usage
kubectl top pods -n awx

# Cleanup old jobs
kubectl exec -it deployment/awx-task -n awx -- awx-manage cleanup_jobs --days=14
kubectl exec -it deployment/awx-task -n awx -- awx-manage cleanup_activitystream --days=30

Common Issues Quick Reference

API Examples

# Get API token
curl -s -X POST http://<HOST>:8052/api/v2/tokens/ \
  -H "Content-Type: application/json" \
  -u "<USER>:<PASSWORD>" | jq .token

# Launch job template via API
curl -s -X POST http://<HOST>:8052/api/v2/job_templates/<TEMPLATE_ID>/launch/ \
  -H "Authorization: Bearer <TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"extra_vars": {"env": "production"}}'

# Check job status
curl -s http://<HOST>:8052/api/v2/jobs/<JOB_ID>/ \
  -H "Authorization: Bearer <TOKEN>" | jq '.status, .failed'

Logrotate Configuration

[!NOTE] In Kubernetes deployments, logs are managed by the container runtime. This logrotate config applies to Docker Compose or local installs. / В Kubernetes-развёртываниях логи управляются средой выполнения контейнеров.

/etc/logrotate.d/awx

/var/log/tower/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 640 awx awx
    sharedscripts
    postrotate
        supervisorctl signal HUP tower-processes:* 2>/dev/null || true
    endscript
}

Documentation Links

• AWX Official Documentation: https://ansible.readthedocs.io/projects/awx/en/latest/
• AWX GitHub Repository: https://github.com/ansible/awx
• AWX Operator GitHub: https://github.com/ansible/awx-operator
• AWX REST API Reference: https://ansible.readthedocs.io/projects/awx/en/latest/rest_api/
• AWX CLI (awxkit) Documentation: https://docs.ansible.com/automation-controller/latest/html/controllercli/
• Red Hat Ansible Automation Platform (commercial): https://www.redhat.com/en/technologies/management/ansible