Efficient ways to clear CUDA memory usage without the need for machine restart

Is there a way to resolve the CUDA out of memory error in Ubuntu 20.04 without restarting the machine? The error was fixed by changing the tensor type to cuda.

Question:

Is there a way to eliminate the
CUDA out of memory
error in Ubuntu 20.04 without requiring a system reboot?

An attempt was made to assign 40.00 MiB on GPU 0, which has a total capacity of 7.80 GiB. Currently, 6.34 GiB has been allocated and there is 32.44 MiB available for allocation. Additionally, PyTorch has reserved 6.54 GiB in total.

Is there a method to clear up GPU memory without terminating the Jupyter notebook? While the approach I am presently using is effective, it causes
Jupyter notebook
to shut down.

(base) mona@mona:~/research/facial_landmark$ nvidia-smi
Tue Oct  6 20:28:05 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06    Driver Version: 450.51.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 2070    Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   47C    P8     9W /  N/A |   7883MiB /  7982MiB |      2%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1306      G   /usr/lib/xorg/Xorg                255MiB |
|    0   N/A  N/A      1743      G   /usr/bin/gnome-shell              151MiB |
|    0   N/A  N/A      3273      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      3359      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      3844      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      4222      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      4587      C   ...mona/anaconda3/bin/python     7459MiB |
+-----------------------------------------------------------------------------+
(base) mona@mona:~/research/facial_landmark$ kill -9  4587
(base) mona@mona:~/research/facial_landmark$ nvidia-smi
Tue Oct  6 20:28:24 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06    Driver Version: 450.51.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 2070    Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   47C    P8     9W /  N/A |    433MiB /  7982MiB |      4%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1306      G   /usr/lib/xorg/Xorg                255MiB |
|    0   N/A  N/A      1743      G   /usr/bin/gnome-shell              152MiB |
|    0   N/A  N/A      3273      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      3359      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      3844      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      4222      G   /usr/lib/firefox/firefox            2MiB |
+-----------------------------------------------------------------------------+
(base) mona@mona:~/research/facial_landmark$ 



Solution:

To free up
CUDA memory
, you may attempt utilizing the command torch.cuda.empty_cache() which clears the memory occupied by PyTorch.

Frequently Asked Questions

Posted in Uncategorized