Confluence runs out of memory 1-2 times per week. Is there a better option than nightly restart?
Asked by Bryan Stephenson:

Hello,  

I am running "Confluence 4.3.7 - Standalone distribution" on openSUSE Leap 42.2. It appears to leak memory. Looking at the VIRT column of the two outputs shown below from the "top" command, the memory has grown from 9.808g on Friday to 9.981g today, three days later. It has been running a total of four days since the last crash and restart. The java process is confluence in both cases. I have a cron job that restarts it once a week which previously was soon enough to avoid it running out of memory and crashing. But now this has happened twice in one week. I want to avoid it going down when I am asleep and failing to provide service to my coworkers in Europe until I wake up to restart it. Is my best option to restart it three times a week (or maybe every night) or would increasing the Java heap size be an option? Or should I upgrade to a newer version that maybe does not leak memory? Also, this seems like an awful lot of memory for it to be using. I don't believe we are doing anything unusual or strenuous with this instance and there should only be around 100 users, most of whom are not accessing the Confluence Wiki most of the time.  

I am a new admin for Confluence so any advice is appreciated.  

<br />

2017-10-13 top output:  
----------------------  

KiB Mem: 4045208 total, 3922252 used, 122956 free, 8 buffers  
KiB Swap: 2103292 total, 1093008 used, 1010284 free. 626232 cached Mem  

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND   
1815 mysql 20 0 5859980 314296 0 S 0.332 7.770 136:18.75 mysqld   
7584 root 20 0 9.808g 2.755g 7600 S 0.332 71.41 24:23.52 java   
1 root 20 0 123888 7480 2020 S 0.000 0.185 2:47.21 systemd   
2 root 20 0 0 0 0 S 0.000 0.000 0:00.73 kthreadd   

<br />

2017-10-16 top output:  
----------------------  

KiB Mem: 4045208 total, 3923984 used, 121224 free, 8 buffers  
KiB Swap: 2103292 total, 963492 used, 1139800 free. 694372 cached Mem  

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND   
1767 root 20 0 9.981g 2.721g 11596 S 1.329 70.54 20:31.58 java   
1815 mysql 20 0 5859980 305032 0 S 0.332 7.541 144:17.28 mysqld   

<br />

When this crash happens I see the following in /var/log/messages:  

2017-10-12T00:50:37.358126-07:00 \<machine name\> kernel: \[4185923.786905\] Out of memory  
: Kill process 9122 (java) score 741 or sacrifice child  
2017-10-12T00:50:37.358126-07:00 \<machine name\> kernel: \[4185923.787010\] Killed proces  
s 9122 (java) total-vm:10343332kB, anon-rss:3436684kB, file-rss:0kB, shmem-rss:1  
2kB  
2017-10-12T00:50:37.711803-07:00 \<machine name\> systemd\[1\]: confluence.service: Main p  
rocess exited, code=killed, status=9/KILL  
2017-10-12T00:50:37.712208-07:00 \<machine name\> stop-confluence.sh\[4448\]: executing as  
current user  

<br />

<br />

Thank you,  

Bryan  

-----  
Bryan Stephenson  
bryan.stephenson@suse.com  


Answers:
author: AnnWorley
body: Hi Bryan,

Confluence is up to version 6.4.2 so it would be a good idea to upgrade to a supported version.

In the meantime, it sounds like the VM may be killing the Confluence process. Please check out this article and see if it fits your issue: [Confluence process dies unexpectedly due to Linux OOM-Killer](https://confluence.atlassian.com/confkb/confluence-process-dies-unexpectedly-due-to-linux-oom-killer-321258129.html)

Thanks,

Ann


author: Peter DeWitt
body: [@Bryan Stephenson](/t5/user/viewprofilepage/user-id/1156866), Confluence seems to have this problem from time to time. Upgrading is the way to go. It usually only takes the next point update but since you are so far behind upgrade to current like [@AnnWorley](/t5/user/viewprofilepage/user-id/502040) suggested.

comments: author: Bryan Stephenson
body: Thank you. I will update to 6.4.2.


Tags: confluence-administration