Tuesday, May 6, 2014

An Insight into Hadoop YARN NodeManager

Conceptually, the NodeManager is more of generic and efficient version of TaskTracker of Hadoop1 architecture which is more flexible and efficient than TaskTracker. In contrast to fixed number of slots for map and reduce tasks in MRV1,the NodeManager of MRV2 has a number of dynamically created resource containers. There is no hardcoded split into Map and Reduce slots as in MRV1.The container refers to collection of resources such as memory,CPU,disk and network IO.The number of containers on a node is the product of configuration parameter and the total amount of node resources. The NodeManager is the per-machine/per-node framework agent who is responsible for containers, monitoring their resource usage and reporting the same to the ResourceManager. Overseeing container’s life cycle management, NodeManager also tracks the health of the node on which it is running, controls auxiliary services which different YARN applications may exploit at any point of time. NodeManager can execute any computations that make sense to ApplicationMaster  just by creating the container for each task.
Source : Hortonworks

The above architecture diagram, gives a detailed view of the NodeManager components.



  1. Hadoop is the most understood language.Initially hadoop is made for immense measure of data sets in OLAP environment. With presentation of Hbase on top of hadoop, can be used for OLAP Processing in addition. Hadoop is a structure with all the subcomponents like aide reduce,hdfs,hbase,pig.Thanks for taking time to share this post.
    Hadoop Training in Chennai | Hadoop course in Chennai | Hadoop Training institutes in Chennai

  2. The expansion of internet and other business intelligence leads to large volume of data. Industries are looking for talented professionals to maintain and process huge volume of data with latest tools available in the market. Taking Hadoop Training in Chennai | Big Data Training in Chennai will ensure better career prospects for talented professionals.

  3. Thanks for your informative article on Hadoop technology.
    Cloud Computing Courses|Cloud computing course in Chennai