Hadoop/HDFS Commands

Sunday, April 15, 2018

Hadoop/HDFS Commands

Hadoop version
Command : hadoop version

List the contents of the root directory in HDFS
Command: hadoop fs -ls /

Report the amount of space used and available on currently mounted file system
Command: hadoop fs -df hdfs:/
Command: hadoop fs -df -h hdfs:/

Create a new directory named “hadoop”
Command: hadoop fs -mkdir /user/entechlog/hadoop

Add a sample text file from the local directory to HDFS
Command: hadoop fs -put /data/home/entechlog/hadoop/input/HadoopFile1.txt /user/entechlog/hadoop/input/

Command: hadoop fs -copyFromLocal /data/home/entechlog/hadoop/input/HadoopFile1.txt /user/entechlog/hadoop/input/

List the contents of a directory in HDFS
Command: hadoop fs -ls /user/entechlog/hadoop/input/

Add the entire local directory to HDFS

Command: hadoop fs -put /data/home/entechlog/hadoop/input/copydir/ /user/entechlog/hadoop/input/

Space used by a directory in HDFS
Command: hadoop fs -du -s -h /user/entechlog/hadoop/input/copydir/

Delete a file
Command: hadoop fs -rm /user/entechlog/hadoop/input/copydir/HadoopFile1.txt

Ensure this file is no longer in HDFS
Command: hadoop fs -ls /user/entechlog/hadoop/input/copydir/HadoopFile1.txt

Delete all files from the folder
Command: hadoop fs -rm /user/entechlog/hadoop/input/copydir/*

Empty the trash
Command: hadoop fs –expunge

Remove all files from a directory and the directory itself
Command: hadoop fs -rm -r /user/entechlog/hadoop/input/copydir/

Copy files between HDFS
Command: hadoop fs -cp /user/entechlog/hadoop/input/HadoopFile3.txt /user/entechlog/hadoop/input/HadoopFile4.txt

Copy file from HDFS to Local
Command: hadoop fs -copyToLocal /user/entechlog/hadoop/input/HadoopFile4.txt /data/home/entechlog/hadoop/input/HadoopFile4.txt
Command: hadoop fs -get /user/entechlog/hadoop/input/HadoopFile4.txt /data/home/entechlog/hadoop/input/HadoopFile5.txt

Display last kilobyte of the file
Command: hadoop fs -tail /user/entechlog/hadoop/input/HadoopFile4.txt
changing permission of a file
hadoop fs -chmod 600 /user/entechlog/hadoop/input/HadoopFile4.txt

Changing the owner 
Command: hadoop fs -chown newUser:newGroup /user/entechlog/hadoop/input/HadoopFile4.txt

Changing the group
Command: hadoop fs -chgrp newGroup /user/entechlog/hadoop/input/HadoopFile4.txt

Move a directory from one location to other
Command: hadoop fs -mv source target

List all the hadoop file system commands
Command: hadoop fs

hadoop file system commands help
Command: hadoop fs –help

To know the system is running under safemode or not
Command: hadoop dfsadmin -safemode get

To make safemode on when it is off
Command: hadoop  dfsadmin  -safemode enter

To make safemode off when it is on

Command: hadoop  dfsadmin  -safemode leave

No comments

Post a Comment

Error 404

The page you were looking for, could not be found. You may have typed the address incorrectly or you may have used an outdated link.

Go to Homepage