HDFS-14547 Bug reproduction

HDFS-14547 DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set

Reproduction Step:

a. set up hadoop-3.1.0 and hadoop-3.1.4 in two nodes

b. create ~5000 empty directories in hdfs (on namenode)

c. set storageSpaceQuota to all these directories

d. moniter # of EnumCounters, EnumCounters heap size , Namenode HDFS serive heap size, during the creation of empty dirs and setting storageSpaceQuota

e. compare, ideally we could observe the memory taken by EnumCounters in 3.1.4 is smaller than that of in 3.1.0 (about 3/4)

10 - 1 4 Mar. 9, 2023, 8:51 PM

Authors

Launch on Chameleon

Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.

Download Archive

Download an archive containing the files of this artifact.

Version Stats

10 - 1