HDFS-13821

While testing Router-based Federation (RBF), a performance issue was encountered. High ProxyAvgTime values were observed, indicating slow performance. Upon further investigation using jstack, it was found that many threads were blocked on the LocalCache, leading to performance degradation.

To address the performance degradation issue, a new configuration property, dfs.federation.router.mount-table.cache.enable, was introduced. This configuration allows users to disable the LocalCache feature in RBF setups. By disabling the cache, users can mitigate the lock contention problem and improve performance, especially in scenarios with a high number of concurrent threads accessing the LocalCache.

6 2 - 1 Apr. 4, 2024, 2:05 PM

Authors

Launch on Chameleon

Launching this artifact will open it within Chameleon’s shared Jupyter experiment environment, which is accessible to all Chameleon users with an active allocation.

Download Archive

Download an archive containing the files of this artifact.

Version Stats

6 2 -