Tuesday, October 2, 2012

Optimize Map Reduce Job Performance

Optimize Hadoop Performance. To improve Hadoop performance, you need to change various configuration parameter in core-site.xml, hdfs-site.xml, mapred-site.xml. The configuration / optimization of parameter to improve performance depends on the type of processing, it depends on case to case, there is no hard and fast rule.

To install Hadoop on ubuntu cluster you can refer this post

We can change block size, number of mappers and reducers, sort factor, jvm reuse, memory for java process, enable compression, map output compression, use combiner, etc.
I found a very nice description given by Cloudera



3 comments:

  1. quite nice post, i certainly enjoy this website, keep it up Hadoop Online Training .

    ReplyDelete
  2. Nice keep blogging and hadoop is the online training course in hyderabad
    hadoop online training

    ReplyDelete
  3. Good job and keep efforts to post the informative data like informatica online training course in hyderabad for more details refer at
    informatica online training

    ReplyDelete