Title: πŸ•΅οΈ Process Diagnostics β€” Process State/Resources/Network Group: Diagnostics Icon: πŸ•΅οΈ Order: 2 ## Table of Contents - [Process Discovery & Identification](#process-discovery--identification) - [Process State & Activity](#process-state--activity) - [Resource Consumption (CPU, RAM, I/O)](#resource-consumption-cpu-ram-io) - [Network & Ports Monitoring](#network--ports-monitoring) - [Thread Analysis](#thread-analysis) - [File Handles & Descriptors](#file-handles--descriptors) - [System Integration & Logging](#system-integration--logging) - [Topic: MySQL Monitoring (Case Study)](#topic-mysql-monitoring-case-study) - [Advanced Tracing & Debugging](#advanced-tracing--debugging) - [Comparison Tables & Senior Tips](#comparison-tables--senior-tips) --- ## Process Discovery & Identification ### Search and List Processes / Поиск ΠΈ пСрСчислСниС процСссов ```bash ps aux | grep # List processes by name / Бписок процСссов ΠΏΠΎ ΠΈΠΌΠ΅Π½ΠΈ pgrep -fl # Find PID and full command line / Найти PID ΠΈ ΠΏΠΎΠ»Π½ΡƒΡŽ строку ΠΊΠΎΠΌΠ°Π½Π΄Ρ‹ pidof # Get PID only / ΠŸΠΎΠ»ΡƒΡ‡ΠΈΡ‚ΡŒ Ρ‚ΠΎΠ»ΡŒΠΊΠΎ PID pstree -p # Show process tree with PIDs / ΠŸΠΎΠΊΠ°Π·Π°Ρ‚ΡŒ Π΄Π΅Ρ€Π΅Π²ΠΎ процСссов с PID ``` ## Process State & Activity ### Monitor Process State / ΠœΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ состояния процСсса ```bash ps -o pid,state,wchan,cmd -p # View process state and wait channel / ΠŸΡ€ΠΎΡΠΌΠΎΡ‚Ρ€ состояния ΠΈ ΠΊΠ°Π½Π°Π»Π° оТидания cat /proc//status | grep State # Detailed state from procfs / ΠŸΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎΠ΅ состояниС ΠΈΠ· procfs cat /proc//wchan # Current wait channel (kernel function) / Π’Π΅ΠΊΡƒΡ‰ΠΈΠΉ ΠΊΠ°Π½Π°Π» оТидания (функция ядра) cat /proc//stack # Kernel call stack (if D/S state) / Π‘Ρ‚Π΅ΠΊ Π²Ρ‹Π·ΠΎΠ²ΠΎΠ² ядра (Ссли Π² состоянии D/S) ``` > [!NOTE] > **State Codes / ΠšΠΎΠ΄Ρ‹ состояний:** > - `R` (Running): Active on CPU / АктивСн Π½Π° CPU. > - `S` (Sleeping): Waiting for event / ΠžΠΆΠΈΠ΄Π°Π΅Ρ‚ события (ΠΏΡ€Π΅Ρ€Ρ‹Π²Π°Π΅ΠΌΡ‹ΠΉ). > - `D` (Uninterruptible Sleep): Waiting for I/O / ΠžΠΆΠΈΠ΄Π°Π΅Ρ‚ I/O (Π½Π΅ΠΏΡ€Π΅Ρ€Ρ‹Π²Π°Π΅ΠΌΡ‹ΠΉ). > - `Z` (Zombie): Finished but not reaped / Π—Π°Π²Π΅Ρ€ΡˆΠ΅Π½, Π½ΠΎ Π½Π΅ ΡƒΠ΄Π°Π»Π΅Π½ ΠΈΠ· Ρ‚Π°Π±Π»ΠΈΡ†Ρ‹. > - `T` (Stopped): Suspended by signal / ΠŸΡ€ΠΈΠΎΡΡ‚Π°Π½ΠΎΠ²Π»Π΅Π½ сигналом. ### Execution Context / ΠšΠΎΠ½Ρ‚Π΅ΠΊΡΡ‚ выполнСния ```bash readlink /proc//exe # Path to executable / ΠŸΡƒΡ‚ΡŒ ΠΊ исполняСмому Ρ„Π°ΠΉΠ»Ρƒ cat /proc//cmdline | xargs -0 # Full launch command / Полная ΠΊΠΎΠΌΠ°Π½Π΄Π° запуска cat /proc//environ | xargs -0 -n 1 # Environment variables / ΠŸΠ΅Ρ€Π΅ΠΌΠ΅Π½Π½Ρ‹Π΅ окруТСния cat /proc//limits # Resource limits (ulimit) / Π›ΠΈΠΌΠΈΡ‚Ρ‹ рСсурсов (ulimit) ``` ## Resource Consumption (CPU, RAM, I/O) ### Detailed CPU and RAM Usage / Π”Π΅Ρ‚Π°Π»ΡŒΠ½ΠΎΠ΅ использованиС CPU ΠΈ RAM ```bash top -p # Monitor specific PID / ΠœΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ ΠΊΠΎΠ½ΠΊΡ€Π΅Ρ‚Π½ΠΎΠ³ΠΎ PID htop -p # Interactive monitor for PID / Π˜Π½Ρ‚Π΅Ρ€Π°ΠΊΡ‚ΠΈΠ²Π½Ρ‹ΠΉ ΠΌΠΎΠ½ΠΈΡ‚ΠΎΡ€ для PID grep VmRSS /proc//status # Resident memory size (RAM) / ОбъСм Ρ€Π΅Π·ΠΈΠ΄Π΅Π½Ρ‚Π½ΠΎΠΉ памяти (ΠžΠ—Π£) pmap -x | tail -n 1 # Detailed memory map summary / Π˜Ρ‚ΠΎΠ³ΠΎΠ²Π°Ρ сводка ΠΊΠ°Ρ€Ρ‚Ρ‹ памяти ``` ### Interactive Filters (Heavy Hitters) / Π˜Π½Ρ‚Π΅Ρ€Π°ΠΊΡ‚ΠΈΠ²Π½Ρ‹Π΅ Ρ„ΠΈΠ»ΡŒΡ‚Ρ€Ρ‹ | Tool / Π˜Π½ΡΡ‚Ρ€ΡƒΠΌΠ΅Π½Ρ‚ | Key / Клавиша | Action (EN / RU) | | :--- | :--- | :--- | | **top** | `P` | Sort by CPU usage / Π‘ΠΎΡ€Ρ‚ΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ ΠΏΠΎ CPU | | **top** | `M` | Sort by Memory usage / Π‘ΠΎΡ€Ρ‚ΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ ΠΏΠΎ памяти | | **htop** | `F6` | Open sort menu / ΠžΡ‚ΠΊΡ€Ρ‹Ρ‚ΡŒ мСню сортировки | | **htop** | `F4` | Filter by name / Π€ΠΈΠ»ΡŒΡ‚Ρ€ ΠΏΠΎ ΠΈΠΌΠ΅Π½ΠΈ | | **htop** | `H` | Toggle threads visibility / Π’ΠΊΠ»/Π²Ρ‹ΠΊΠ» ΠΎΡ‚ΠΎΠ±Ρ€Π°ΠΆΠ΅Π½ΠΈΠ΅ ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² | ### I/O Usage / ИспользованиС Π²Π²ΠΎΠ΄Π°-Π²Ρ‹Π²ΠΎΠ΄Π° ```bash iotop -p # Live I/O monitoring / ΠœΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ I/O Π² Ρ€Π΅Π°Π»ΡŒΠ½ΠΎΠΌ Π²Ρ€Π΅ΠΌΠ΅Π½ΠΈ cat /proc//io # I/O statistics counters / Π‘Ρ‡Π΅Ρ‚Ρ‡ΠΈΠΊΠΈ статистики I/O ``` ## Network & Ports Monitoring ### Listeners and Connections / Π‘Π»ΡƒΡˆΠ°Ρ‚Π΅Π»ΠΈ ΠΈ соСдинСния ```bash ss -tunap | grep # Current sockets by PID / Π’Π΅ΠΊΡƒΡ‰ΠΈΠ΅ сокСты ΠΏΠΎ PID netstat -plntu | grep # Listening ports (classic) / ΠŸΡ€ΠΎΡΠ»ΡƒΡˆΠΈΠ²Π°Π΅ΠΌΡ‹Π΅ ΠΏΠΎΡ€Ρ‚Ρ‹ (классика) lsof -i -nP -p # Network files opened by process / Π‘Π΅Ρ‚Π΅Π²Ρ‹Π΅ Ρ„Π°ΠΉΠ»Ρ‹, ΠΎΡ‚ΠΊΡ€Ρ‹Ρ‚Ρ‹Π΅ процСссом ``` ### Process-Targeted Bandwidth / ΠŸΡ€ΠΎΠΏΡƒΡΠΊΠ½Π°Ρ ΡΠΏΠΎΡΠΎΠ±Π½ΠΎΡΡ‚ΡŒ процСсса ```bash nethogs # Monitor traffic per process / ΠœΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ Ρ‚Ρ€Π°Ρ„ΠΈΠΊΠ° ΠΏΠΎ процСссам iftop -P -i -f "port " # Traffic on specific port / Π’Ρ€Π°Ρ„ΠΈΠΊ Π½Π° ΠΊΠΎΠ½ΠΊΡ€Π΅Ρ‚Π½ΠΎΠΌ ΠΏΠΎΡ€Ρ‚Ρƒ # Press 'P' in iftop to show ports / НаТмитС 'P' Π² iftop для отобраТСния ΠΏΠΎΡ€Ρ‚ΠΎΠ² ``` ### Deep Packet Analysis / Π“Π»ΡƒΠ±ΠΎΠΊΠΈΠΉ Π°Π½Π°Π»ΠΈΠ· ΠΏΠ°ΠΊΠ΅Ρ‚ΠΎΠ² ```bash tcpdump -i port -n # Capture port traffic / Π—Π°Ρ…Π²Π°Ρ‚ Ρ‚Ρ€Π°Ρ„ΠΈΠΊΠ° ΠΏΠΎΡ€Ρ‚Π° tcpdump -i port -A # Show payload in ASCII / ΠŸΠΎΠΊΠ°Π·Π°Ρ‚ΡŒ содСрТимоС Π² ASCII tcpdump -i -w capture.pcap # Save to file for Wireshark / Π‘ΠΎΡ…Ρ€Π°Π½ΠΈΡ‚ΡŒ Π² Ρ„Π°ΠΉΠ» для Wireshark ``` ## Thread Analysis ### Thread Count and Details / ΠšΠΎΠ»ΠΈΡ‡Π΅ΡΡ‚Π²ΠΎ ΠΈ Π΄Π΅Ρ‚Π°Π»ΠΈ ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² ```bash ps -o nlwp,pid,cmd -p # Show thread count (NLWP) / ΠŸΠΎΠΊΠ°Π·Π°Ρ‚ΡŒ количСство ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² ps -eLf | grep # List every thread separately / Бписок ΠΊΠ°ΠΆΠ΄ΠΎΠ³ΠΎ ΠΏΠΎΡ‚ΠΎΠΊΠ° ΠΎΡ‚Π΄Π΅Π»ΡŒΠ½ΠΎ ls /proc//task | wc -l # Count threads via procfs / ΠŸΠΎΠ΄ΡΡ‡Π΅Ρ‚ ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² Ρ‡Π΅Ρ€Π΅Π· procfs top -H -p # Monitor individual threads / ΠœΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ ΠΎΡ‚Π΄Π΅Π»ΡŒΠ½Ρ‹Ρ… ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² ``` ## File Handles & Descriptors ### Open Files Tracking / ΠžΡ‚ΡΠ»Π΅ΠΆΠΈΠ²Π°Π½ΠΈΠ΅ ΠΎΡ‚ΠΊΡ€Ρ‹Ρ‚Ρ‹Ρ… Ρ„Π°ΠΉΠ»ΠΎΠ² ```bash lsof -p # List all open files / Бписок всСх ΠΎΡ‚ΠΊΡ€Ρ‹Ρ‚Ρ‹Ρ… Ρ„Π°ΠΉΠ»ΠΎΠ² ls -l /proc//fd # File descriptors count/paths / ΠŸΡƒΡ‚ΠΈ ΠΈ количСство дСскрипторов fuser -v # Find process using a file / Найти процСсс, ΠΈΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡŽΡ‰ΠΈΠΉ Ρ„Π°ΠΉΠ» ``` ## System Integration & Logging ### Service Control & Logs / Π£ΠΏΡ€Π°Π²Π»Π΅Π½ΠΈΠ΅ сСрвисом ΠΈ Π»ΠΎΠ³ΠΈ ```bash systemctl status # Check systemd status / ΠŸΡ€ΠΎΠ²Π΅Ρ€ΠΈΡ‚ΡŒ статус systemd journalctl -u -f # Follow service logs / Π‘Π»Π΅Π΄ΠΈΡ‚ΡŒ Π·Π° Π»ΠΎΠ³Π°ΠΌΠΈ сСрвиса journalctl -u --since "1 hour ago" # Logs for last hour / Π›ΠΎΠ³ΠΈ Π·Π° послСдний час ``` ## Topic: MySQL Monitoring (Case Study) ### Targeting MySQL specifically / БпСцифичСский ΠΌΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ MySQL ```bash # 1. Identify MySQL process / Найти процСсс MySQL pgrep -u mysql -fa # 2. Check MySQL threads / ΠŸΡ€ΠΎΠ²Π΅Ρ€ΠΈΡ‚ΡŒ ΠΏΠΎΡ‚ΠΎΠΊΠΈ MySQL ps -o nlwp,pid,cmd -p $(pgrep -u mysql -x mysqld) # 3. Monitor MySQL memory / ΠœΠΎΠ½ΠΈΡ‚ΠΎΡ€ΠΈΠ½Π³ памяти MySQL grep VmRSS /proc/$(pgrep -u mysql -x mysqld)/status # 4. MySQL network activity / БСтСвая Π°ΠΊΡ‚ΠΈΠ²Π½ΠΎΡΡ‚ΡŒ MySQL ss -tunap | grep mysqld iftop -P -i -f "port 3306" # Default MySQL port / Π‘Ρ‚Π°Π½Π΄Π°Ρ€Ρ‚Π½Ρ‹ΠΉ ΠΏΠΎΡ€Ρ‚ MySQL # 5. Internal MySQL Diagnostics / ВнутрСнняя диагностика MySQL # Run inside mysql client / Π’Ρ‹ΠΏΠΎΠ»Π½ΡΡ‚ΡŒ Π²Π½ΡƒΡ‚Ρ€ΠΈ mysql-ΠΊΠ»ΠΈΠ΅Π½Ρ‚Π° mysql -u -p -e "SHOW PROCESSLIST;" # List active threads / Бписок Π°ΠΊΡ‚ΠΈΠ²Π½Ρ‹Ρ… ΠΏΠΎΡ‚ΠΎΠΊΠΎΠ² mysql -u -p -e "SHOW ENGINE INNODB STATUS\G" # Detailed InnoDB state / Π”Π΅Ρ‚Π°Π»ΡŒΠ½ΠΎΠ΅ состояниС InnoDB ``` > [!TIP] > MySQL uses a **One-Process-Many-Threads** model. Resource visibility often combines all threads into the main process. Use `top -H` or Performance Schema inside MySQL for granular internal thread info. ## Advanced Tracing & Debugging ### Low-Level Activity / НизкоуровнСвая Π°ΠΊΡ‚ΠΈΠ²Π½ΠΎΡΡ‚ΡŒ ```bash strace -p -f -e trace=network,file # Trace syscalls / Врассировка систСмных Π²Ρ‹Π·ΠΎΠ²ΠΎΠ² perf top -p # CPU profiling / ΠŸΡ€ΠΎΡ„ΠΈΠ»ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠ΅ CPU gdb -p # Attach debugger (EXPERT ONLY) / ΠŸΠΎΠ΄ΠΊΠ»ΡŽΡ‡ΠΈΡ‚ΡŒ ΠΎΡ‚Π»Π°Π΄Ρ‡ΠΈΠΊ (Π’ΠžΠ›Π¬ΠšΠž Π­ΠšΠ‘ΠŸΠ•Π Π’Π«) ``` > [!WARNING] > Attaching `strace` or `gdb` to a high-load production process can cause significant performance degradation or temporarily "freeze" the app. > / ΠŸΠΎΠ΄ΠΊΠ»ΡŽΡ‡Π΅Π½ΠΈΠ΅ `strace` ΠΈΠ»ΠΈ `gdb` ΠΊ высоконагруТСнному процСссу ΠΌΠΎΠΆΠ΅Ρ‚ Π²Ρ‹Π·Π²Π°Ρ‚ΡŒ ΡΠ΅Ρ€ΡŒΠ΅Π·Π½ΠΎΠ΅ Π·Π°ΠΌΠ΅Π΄Π»Π΅Π½ΠΈΠ΅ ΠΈΠ»ΠΈ Π²Ρ€Π΅ΠΌΠ΅Π½Π½ΡƒΡŽ Β«Π·Π°ΠΌΠΎΡ€ΠΎΠ·ΠΊΡƒΒ» прилоТСния. ## Comparison Tables & Senior Tips ### Comparison: Process Memory Metrics / Π‘Ρ€Π°Π²Π½Π΅Π½ΠΈΠ΅: ΠœΠ΅Ρ‚Ρ€ΠΈΠΊΠΈ памяти процСссов | Metric / ΠœΠ΅Ρ‚Ρ€ΠΈΠΊΠ° | Name (EN/RU) | Description (EN / RU) | Use Case / Когда ΡΠΌΠΎΡ‚Ρ€Π΅Ρ‚ΡŒ | | :--- | :--- | :--- | :--- | | **VIRT** | Virtual Image / Π’ΠΈΡ€Ρ‚ΡƒΠ°Π»ΡŒΠ½Π°Ρ | Total address space shared + mapped / Π’Π΅ΡΡŒ адрСсный объСм | General limits / ΠžΠ±Ρ‰ΠΈΠ΅ Π»ΠΈΠΌΠΈΡ‚Ρ‹ | | **RSS** | Resident Set / РСзидСнтная | Non-swapped physical RAM / ЀизичСская ΠžΠ—Π£ Π±Π΅Π· swap | Actual usage / РСальноС ΠΏΠΎΡ‚Ρ€Π΅Π±Π»Π΅Π½ΠΈΠ΅ | | **SHR** | Shared Memory / РаздСляСмая | Memory shared with other processes / ΠŸΠ°ΠΌΡΡ‚ΡŒ, общая с Π΄Ρ€ΡƒΠ³ΠΈΠΌΠΈ | Library impact / ВлияниС Π±ΠΈΠ±Π»ΠΈΠΎΡ‚Π΅ΠΊ | | **SWAP** | Swap Size / Π‘Π²ΠΎΠΏ | Memory moved to disk / ВыгруТСнная Π½Π° диск ΠΏΠ°ΠΌΡΡ‚ΡŒ | Memory pressure / Π”Π΅Ρ„ΠΈΡ†ΠΈΡ‚ памяти | ### Senior Tips - **Soft vs Hard Limits:** Soft limits (`ulimit -Sn`) can be changed by the user; Hard limits (`ulimit -Hn`) are the absolute ceiling set by root. - **Zombie Processes:** A zombie process doesn't consume CPU/RAM but takes a slot in the process table. Cleaning them requires the parent to `wait()` or killing the parent. - **D-State (Uninterruptible):** Usually means wait for Hardware I/O (Disk/NFS). The process cannot be killed by `SIGKILL` until I/O returns.