Memory and CPU Profiling
Mantis allows you to directly connect profiling tools to workers in order to do performance or memory profiling. This can be helpful for debugging OOMs. Below are the steps to start profiling your running Mantis job:
- Download a profiling tool. We recommend Zulu Mission Control (ZMC).
- Untar ZMC
tar -xzvf <zulu_mission_control_package>.tar.gz
On Mac the
Zulu Mission Controlapp is actually a directory even though it has an icon. The
zmcexecutable is inside and needs to be called directly.
<your dir>/<zulu_mission_control_package>/Zulu\ Mission\ Control.app/Contents/MacOS/zmc1. Find a worker you want to debug in your job and click on it to get its ip address and debug port In this example these are
Setup an SSH tunnel with local portforwarding for your worker and port. The tunnel command for the example worker in the previous step would be:
ssh -L 7156:127.0.0.1:7156 100.82.159.148
Tunneling is only required if you can't access the Mantis worker's ip directly.
After setting up local port forwarding. Add the worker JVM to ZMC by clicking the plug and star icon in the top right corner.
Input your mantis worker information
Start the flight recorder
Wait for the Flight recording to complete
Look at the collected profiling info. Below is an example of the memory profiling results. You can see the percentage of the heap occuppied by different objects, and some stack profiling information about where the allocations happen.