Memory measurement

Methods and limitations

We use various tools to measure memory usage in JortJams. Top-level measurements are usually obtained using Massif, which measures heap usage over time. JortJams also has an allocator that logs a message every time an allocation occurs.

The Massif measurements are run on test harnesses and on the main programs. Massif on the main programs is run as follows:

  1. The fetcher is run under massif by itself, starting from a database containing no data except 16 sources. The fetcher is allowed to run for a few minutes.
  2. The GUI is run under massif, and typical actions are done: songs played, lists scrolled, etc.

The benchmark setup is a laptop with a Skylake-era Intel Core i3, an Intel HD Graphics 520 GPU, and 16 GB of RAM running 64-bit Ubuntu 18.04. The graphical environment is KDE; the window system is Xorg.

(More consistent benchmarking procedures would be a good thing, and is something that we would like to tackle in the future.)

The Massif output is loaded into massif-visualizer for analysis.

The measurement tools in JortJams do not consider the size of shared libraries, which can be quite large, especially for the AppImage distribution of JortJams. If you use (e.g.) htop or top to view JortJams GUI memory use, you may find that the resident size of the GUI is around 90 MB, whereas here it’s reported as ~60 MB. The resident size of the fetch process is much smaller because it can reuse memory allocated for shared libraries.

The solution for this problem is to use fewer libraries (unfortunately difficult due to the decision to use Qt) or link against copies of libraries already on the system, which have a higher chance of being loaded in memory before JortJams starts up. That may be a possibility as more Linux distributions adopt newer Qt versions.

Fetcher memory graph

Massif-visualizer graph for the fetcher

The fetcher’s memory usage hovers around 15 MB, though it is not bounded: it is possible for very large incoming statuses to cause the fetcher to request large amounts of memory. JortJams needs to be modified to reject very large responses from servers; before we do that, we need to understand what “very large” means.

GUI memory graph

Massif-visualizer graph for the GUI

The GUI uses several times more memory than the fetcher, and the memory graph only shows part of the story. Large amounts of the GUI memory are allocated by i965_dri.so, which is part of Intel’s GPU driver. (The benchmarks are run on a system using an Intel HD Graphics 520 GPU.) So there’s some work to be done here to understand these GPU driver allocations and what can be done to reduce them.