High SWAP usage in Oracle Database Server

 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

2470184 mdatp     20   0 1190672 221212  80108 S  12.9   0.2   5:40.22 wdavdaemon

  14213 oracle    20   0   80.7g  27.4g  27.2g S  12.6  22.4   9365:40 ora_s000_ORCL

  14215 oracle    20   0   80.6g  27.7g  27.6g S  10.9  22.6   8742:56 ora_s001_ORCL

  14139 oracle    20   0   80.5g   1.3g   1.3g S   7.6   1.1 411:54.34 ora_d013_ORCL

2288719 root      20   0 1098456  36504  16628 S   4.6   0.0  20:14.67 wdavdaemon

2288664 root      20   0 1248312 123808  28312 S   2.3   0.1  17:59.23 wdavdaemon

  14057 oracle    20   0   80.5g 234028 205176 S   1.3   0.2 622:32.42 ora_dia0_ORCL

  14161 oracle    20   0   80.5g   1.3g   1.3g S   1.3   1.1 399:43.10 ora_d024_ORCL

  14207 oracle    20   0   80.5g   1.3g   1.3g S   1.3   1.1 401:37.92 ora_d047_ORCL

  14012 oracle    -2   0   80.5g   2224   1792 S   0.7   0.0 232:48.41 ora_vktm_ORCL

  14063 oracle    20   0   80.5g 269604 268372 S   0.7   0.2 382:58.31 ora_lgwr_ORCL

  14071 oracle    20   0   80.5g 221792 221228 S   0.7   0.2 424:40.60 ora_lg00_ORCL

  14131 oracle    20   0   80.5g   1.3g   1.3g S   0.7   1.1 371:15.74 ora_d009_ORCL

  14217 oracle    20   0   80.7g  25.2g  25.0g S   0.7  20.6   5266:11 ora_s002_ORCL

    135 root       0 -20       0      0      0 I   0.3   0.0  21:40.33 kworker/3:1H-kblockd

   4333 grid      20   0 2673388  18956   6936 S   0.3   0.0 133:00.72 ohasd.bin

  10088 grid      20   0 2218372  30340      0 S   0.3   0.0  84:58.80 ocssd.bin

  11066 grid      -2   0 2390276   4552   4376 S   0.3   0.0 236:37.57 asm_vktm_+asm

  11119 grid      20   0 2407516  11688  10864 S   0.3   0.0   8:28.19 asm_gmon_+asm

  14045 oracle    20   0   80.5g   3.4g   3.4g S   0.3   2.8 142:57.89 ora_dbrm_ORCL

  14111 oracle    20   0   80.5g 221588 219068 S   0.3   0.2 127:29.07 ora_mmnl_ORCL

  14133 oracle    20   0   80.5g   1.3g   1.3g S   0.3   1.1 394:59.40 ora_d010_ORCL

  14145 oracle    20   0   80.5g   1.3g   1.3g S   0.3   1.1 399:20.58 ora_d016_ORCL

  14177 oracle    20   0   80.5g   1.3g   1.3g S   0.3   1.0 363:20.75 ora_d032_ORCL

  14195 oracle    20   0   80.5g   1.3g   1.3g S   0.3   1.1 370:11.81 ora_d041_ORCL

  14199 oracle    20   0   80.5g   1.3g   1.3g S   0.3   1.1 355:24.68 ora_d043_ORCL

  15367 oracle    20   0   80.5g 420312 415096 S   0.3   0.3  32:22.48 ora_cjq0_ORCL

1682958 grid      20   0 2173264  21684  12880 S   0.3   0.0  24:07.97 oraagent.bin

2288782 root      20   0  291972  11884   7172 S   0.3   0.0   0:39.13 telemetryd_v2 

The system you are monitoring has high swap usage, as shown in the top output. Here's the key information and analysis:

Observations:

1. Memory Details:

   • Total Memory: 125252.5 MiB (approximately 122 GB).
   • Used Memory: 56199.0 MiB (approximately 55 GB).
   • Buffers/Cache: 68061.9 MiB (approximately 66 GB).
   • Free Memory: 991.6 MiB (less than 1 GB).

2. Swap Usage:

   • Total Swap: 20480.0 MiB (20 GB).
   • Used Swap: 20431.7 MiB (nearly the entire swap is used).
   • Available Memory: 2470.9 MiB (combination of free memory and part of cache).

3. Top Consumers:

• Oracle Processes (ora_* and grid):
• These processes are consuming a significant amount of virtual memory (e.g., ora_s000_ORCL, ora_s001_ORCL).
• Processes like ora_s002_ORCL are consuming 25.2 GB RES (resident memory).

he ora_s000,01,02 process in Oracle is a shared server process

• Wdavdaemon:
  • Several wdavdaemon processes from Microsoft Defender ATP (mdatp user) are also running, with memory footprints like 36504 KiB RES.

Likely Issues with Swap Usage:

• Insufficient Free Physical Memory:
  • With less than 1 GB of free RAM, the system starts relying heavily on swap.
• High Memory Buffers/Cache:
  • Buffers/cache is consuming 66 GB, which might be reclaimable, but the system is still using swap.
• Oracle Database Memory Settings:
  • The Oracle processes have massive memory allocations. Specifically:
    • ora_s000_ORCL: 27.4 GB.
    • ora_s001_ORCL: 27.7 GB.
    • The SGA (System Global Area) or PGA (Program Global Area) configurations might be too aggressive for the available RAM.

Recommendations:

1. Analyze Oracle Memory Settings:

   • Check the SGA and PGA settings for the database instance ORCL.
   • Reduce memory allocations if they are excessive for the workload or available hardware.

2. Reclaim Buffers/Cache:

   • Use sync; echo 3 > /proc/sys/vm/drop_caches (as root) to clear caches temporarily and free up memory. This won’t affect Oracle performance significantly as the database manages its own caching.

3. Investigate High Swap Users:

   • Run smem or ps aux --sort=-%mem to identify specific processes using swap excessively.

4. Optimize Application/Daemon Usage:

   • Evaluate the necessity of wdavdaemon processes (Defender ATP). They might be using more memory than needed.

5. Increase Physical Memory or Swap Space (if hardware allows):

   • Add more RAM to avoid excessive swapping in the long term.
   • Temporarily increase swap space (swapon utility) to handle immediate pressure.

By addressing Oracle memory configurations and evaluating the necessity of other running processes, you can alleviate memory pressure and reduce swap usage.