We read every piece of feedback, and take your input very seriously.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1226.19 GiB. GPU has a total capacity of 63.98 GiB of which 36.29 GiB is free. Of the allocated memory 24.03 GiB is allocated by ...

近期,我们应用开始出现sunfire内存利用率的告警，规律是应用重启后，内存利用率缓慢增长，一段时间不重启后，就会出现告警，一开始看到内存利用率第一反应是堆内存利用率的问题，走了一些弯路，最终发现是堆外内存的影响，本文主要记录和总结该问题的排查过程。