Cheatsheet: Difference between revisions

2,545 bytes removed ,  4 years ago
Line 988:
ls > file.log 2>&1 OR ls &> file.log
ls > file.log 2> /dev/null
 
== Troubleshooting ==
 
Source: [https://scoutapm.com/blog/slow_server_flow_chart scoutapm.com]
 
1: Check I/O wait and CPU Idletime
Look for "wa" (I/O wait) and "id" (CPU idletime)
I/O Wait represents the amount of time the CPU waiting for disk or network I/O.
Anything above 10% I/O wait should be considered high.
CPU idle time is a metric you WANT to be high
If your idle time is consistently above 25%, consider it "high enough"
 
2: IO Wait is low and idle time is low: check CPU user time
Look for the %us column (first column), then look for a process or processes that is doing the damage.
If %usertime is high, see which program is monopolizing the CPU
Be default, top sorts the process list by %CPU, so you can just look at the top process or processes.
If situation seems anomalous: kill/restart the offending processes.
If situation seems typical given history: upgrade server or add more servers.
 
3: IO wait is low and idle time is high
Your slowness isn't due to CPU or IO problems, so it's likely an application-specific issue.
Slowness might be caused by another server in your cluster or by an external services like DB
If you suspect another server in your cluster use - Strace and Lsof
Strace will show you which file descriptors are being read or written to.
Lsof can give you a mapping of those file descriptors to network connections.
 
4: IO Wait is high: check your swap usage
Use top or free -m
Cache swaps will monopolize the disk
Processes with legitimate IO needs will be starved for disk access.
In other words, checking disk swap separates "real" IO wait problems from what are actually RAM problems that "look like" IO Wait problems.
 
5: Swap usage is high
High swap usage means that you are actually out of RAM.
 
6: Swap usage is low
Low swap means you have a "real" IO wait problem
iotop is an awesome tool for identifying io offenders.
 
7: Check memory usage
Once top is running, press the M key - this will sort applications by the memory used.
Important: don't look at the "free" memory -- it's misleading.
To get the actual memory available, subtract the "cached" memory from the "used" memory.
This is because Linux caches things liberally, and often the memory can be freed up when it's needed.
A memory leak can be satisfactorily addressed by a one-time or periodic restart of the process.
If memory usage seems anomalous: kill the offending processes.
If memory usage seems business-as-usual: add RAM to the server, or split high-memory using services to other servers.
 
= Sorting Algorithms =