Change:
resolved "https://registry.yarnpkg.com/wrappy/-/wrappy-1.0.2.tgz#b5243d8f3ec1aa35f1364605bc0d1036e30ab69f" integrity sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8= <del> xterm-addon-search@0.10.0-beta.2: version "0.10.0-beta.2" resolved "https://registry.yarnpkg.com/xterm-addon-search/-/xterm-addon-search-0.10.0-beta.2.tgz#a937d1e9a70fde8eeb7d1df485039b2d5fc1d707" integrity sha512-ybafAbX9V4sfkzmUsWmtfEYExG8jj73bTF9pEa/Lhd5q4bviW4LcFaw/n3lKHn/1tSgSVgzoD13u1ZaZR78SfQ== </del> <ins> xterm-addon-search@0.10.0-beta.3: version "0.10.0-beta.3" resolved "https://registry.yarnpkg.com/xterm-addon-search/-/xterm-addon-search-0.10.0-beta.3.tgz#5194434d86105637c71f6f20139a9d0b5c1a956a" integrity sha512-UeGm/ymnp7HUYJJtsP0D+bljOWbdk3MctcLJ+0jv8AmU6YlAzJFtouvYSQrD5SAMyht5CRsvjzFgqic9X02JYg== </ins> <del> xterm-addon-serialize@0.8.0-beta.2: version "0.8.0-beta.2" resolved "https://registry.yarnpkg.com/xterm-addon-serialize/-/xterm-addon-serialize-0.8.0-beta.2.tgz#f30656d4ff1570ac105bacffe443385666654598" integrity sha512-IDaRxO1zwjF9fDJp6u27Lv8852kEZ0HlbB0wLZbcIGZxDuPDLfvw8s/BV7f6MFB+mZq19CjyHGH4oPzZkc0rLQ== </del> <ins> xterm-addon-serialize@0.8.0-beta.3: version "0.8.0-beta.3" resolved "https://registry.yarnpkg.com/xterm-addon-serialize/-/xterm-addon-serialize-0.8.0-beta.3.tgz#47ade3fedacbb75bd26e63cfe0120586623e0e4f" integrity sha512-gvfempZCYuAhLqN4O6fA2TuoavPjOxFKlh8hLcOzPackiLUhwKr1jQpDXcnq8VgqUiGgb+XNZpPEbI0Q7EhTgA== </ins> xterm-addon-unicode11@0.4.0-beta.3: version "0.4.0-beta.3" resolved "https://registry.yarnpkg.com/xterm-addon-unicode11/-/xterm-addon-unicode11-0.4.0-beta.3.tgz#f350184155fafd5ad0d6fbf31d13e6ca7dea1efa" integrity sha512-FryZAVwbUjKTmwXnm1trch/2XO60F5JsDvOkZhzobV1hm10sFLVuZpFyHXiUx7TFeeFsvNP+S77LAtWoeT5z+Q== <del> xterm-addon-webgl@0.13.0-beta.7: version "0.13.0-beta.7" resolved "https://registry.yarnpkg.com/xterm-addon-webgl/-/xterm-addon-webgl-0.13.0-beta.7.tgz#9b514fa9792ced755c8321ddd06529f9912db2a1" integrity sha512-U3sKzkziZRwb13MyNp0eMwt5BWyM938epAv0ZOnI5Vjq06S2naS37GgO/FY+0eu5wuBKkZhT9lNDv7n8rMqd+w== </del> <ins> xterm-addon-webgl@0.13.0-beta.9: version "0.13.0-beta.9" resolved "https://registry.yarnpkg.com/xterm-addon-webgl/-/xterm-addon-webgl-0.13.0-beta.9.tgz#66a9ac142ae347d0548abbf4e66bb2f35f415adb" integrity sha512-x1o1tpCqIsICvhcRsZs+BLcwUIdizYS2G4TIH0KBnUDiSN+oSqpVBQNG8qKg56xbK8WtpdbQ9dLB7JR2W5cX0g== </ins> <del> xterm-headless@4.20.0-beta.13: version "4.20.0-beta.13" resolved "https://registry.yarnpkg.com/xterm-headless/-/xterm-headless-4.20.0-beta.13.tgz#a7d9d8837e3f78e106006cc94cf63ec13a9fd991" integrity sha512-y4YI+Ogv2R2I++tsyvx5Q7csAaN7mG2yMMMBb/u4dXnrFmSGYs/R8ZFkeHgAW4Ju4uI3Rizb+ZdwtN1uG043Rw== </del> <ins> xterm-headless@4.20.0-beta.20: version "4.20.0-beta.20" resolved "https://registry.yarnpkg.com/xterm-headless/-/xterm-headless-4.20.0-beta.20.tgz#da2d8131b02d6f1e37f47cc17e578f2c2980fbb6" integrity sha512-JK4jUIiUH7TdzvMrpfDnbGxTuC4s7byjqnMHR8+gIpY8qCFjz0xcMFSbp+ZshxGwVyziI4jtJqTHZjFToT2/kw== </ins> <del> xterm@4.20.0-beta.13: version "4.20.0-beta.13" resolved "https://registry.yarnpkg.com/xterm/-/xterm-4.20.0-beta.13.tgz#7526e19310daa56a11c26eb81a2706593c1378ec" integrity sha512-ovXGhvM/qDRNGlGO653WY1pxIZrnI4+ayVM7pCnxO/tRwUIym6LGS3NurYrhGYrXOjoLJZxQ4EjToAfHvAg2Gg== </del> <ins> xterm@4.20.0-beta.20: version "4.20.0-beta.20" resolved "https://registry.yarnpkg.com/xterm/-/xterm-4.20.0-beta.20.tgz#2979a31839f7b8ee3ffe4f063b40c02facdb0fed" integrity sha512-ltDtTquH+33tXQPFSDqenbgz6LkvIob6l6Rac85L4aX5Ve7P3ubVLrq+lTFJGQn3iiwGqNmnE1t1EUuGhxsXcQ== </ins> yallist@^4.0.0: version "4.0.0"

Discussion:
Follow up on Thanks for the fix, but I can still reproduce the issue in the insiders build I just downloaded: ! I've opened lots of terminals which I have now closed again (only welcome screen opened now). If vscode is visible in the background: ! If vscode is either minimized or completely behind other windows: !
I feel like this is just Chromium perhaps batching the webgl calls or something, the GPU memory according to devtools jumps around a bit but always ends around the same value even after creating many terminals: ! The terminal is unique in VS Code wrt GPU as it's currently the only part that uses webgl, so that explains why GPU memory is affected primarily by the terminal and no other component. I'm not sure there is anything we can do here since we are now disposing of all the webgl objects that need to be manually freed in
This is very interesting, I get the same results as you with the frame rate performance overlay, one more clue I found but I guess that doesn't help too much is that this also is reflected in the gpu process memory usage. This is when many terminals has been opened and closed again: ! Same instance minimized: ! I can make GPU process usage go beyond 4GB without any sign of stopping by just opening ~10 tabs, closing them again and doing it over again many times, so I guess it's something more than just Chrome doing batching or keeping memory for performance reasons etc.
One more clue, after opening and closing terminals again (now with no terminals open) I take a javascript heap profile in devtools (after pressing "Collect garbage" a few times), top usage is from many instances of arrays in belonging to ! Number and size of these seems to scale well with how many terminals that have been opened historically, so I think each instance of still leave resources behind.
With ram, terminals take a fairly constant amount as it's pee allocated when you create the terminal. This is based on your scrollback setting so if you have that as a large number your memory usage per terminal will go up, even when the scrollback isn't full.