YARN-9565 RMAppImpl#ranNodes not cleared on FinalTransition

This is a reproduction and graph generation for YARN-9565 bug, a minor memory leak.

Code modification such that it is easily reproducible

Note line 1503 in RMAppImpl.java, there is a System.getenv("APP_RANNODES_CLEAR") if statement. Therefore, setting this environmental variable to "true" would be applying the patch. Setting it to any other thing would mean not applying the patch.

How to measure the effect of patch (follow the markdown sections in the notebook)

Repetitively run wordcount mapreduce task, and take java heap dump afterward. Try to find the memory usage of CustomConcurrentSkipListSet class which is an inherited class of the original ConcurrentSkipListSet. This allows easier searching in the java heap dump.

18 2 - 1 May. 14, 2024, 4:02 AM


Launch on Chameleon

Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.

Download Archive

Download an archive containing the files of this artifact.

Download with git

Clone the git repository for this artifact, and checkout the version's commit

git clone https://github.com/jiajunmao/hadoop-repro
# cd into the created directory
git checkout 577017b416aa8757ecdd68aba3a85a6e50745762

Submit feedback through GitHub issues

Version Stats

18 2 -