Hadoop Troubleshooting-002

How to resolve java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FsShell error.


bash-3.00$ hadoop dfs -ls
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FsShell

Solution:
Run following command and check whether jar hadoop-common-2.0.2-alpha-gphd-2.0.1.0.jar is in classpath.
$hadoop classpath


if given jar is not in the classpath then add following entry in hadoop-env.sh
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:YOUR-PATH/hadoop-2.0.2-alpha-gphd-2.0.1.0/share/hadoop/common/hadoop-common-2.0.2-alpha-gphd-2.0.1.0.jar

Hadoop Troubleshooting-001

Hadoop getting following error (java.lang.UnsupportedClassVersionError) when run Hadoop / HDFS Command

bash-3.00$ hadoop  dfs -ls /
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

Solution:
This is typical java error, which is caused when you run a class that is compiled with newer java version then you have currently in your system. If you are running Hadoop with older version (older than 1.6) you will get this error. Please check the version of java (java -version) and point Hadoop to correct java version.

Big Data Use-cases Across Industries, It's Becoming Bigger and Bigger

Big Data will become bigger and bigger, According to certain market forecasts it will increase by 1211%.

Big Data is increase at very fast speed, as the volume of data is growing like any thing, according to Gartner we create 2.5 Quintillion bytes of data per day (90% of the data in the world has been created in last two years alone).

An interesting infographic from Wikibon.org

Image source: wikibon.org/