Hadoop/HDFS Commands

  • Home
  • /
  • Hadoop/HDFS Commands
Hadoop/HDFS Commands

Hadoop/HDFS Commands

Hadoop 15 Apr 2018

In this article we will go over some frequently used HDFS commands

Hadoop version

Command : hadoop version

List the contents of the root directory in HDFS

Command: hadoop fs -ls

Report the amount of space used and available on currently mounted file system

Command: hadoop fs -df hdfs
Command: hadoop fs -df -h hdfs

Create a new directory named “hadoop”

Command: hadoop fs -mkdir /user/entechlog/hadoop

Add a sample text file from the local directory to HDFS

Command:hadoop fs -put /data/home/entechlog/hadoop/input/HadoopFile1.txt /user/entechlog/hadoop/input
Command: hadoop fs -copyFromLocal /data/home/entechlog/hadoop/input/HadoopFile1.txt /user/entechlog/hadoop/input/

List the contents of a directory in HDFS

Command: hadoop fs -ls /user/entechlog/hadoop/input/

Add the entire local directory to HDFS

Command:hadoop fs -put /data/home/entechlog/hadoop/input/copydir/ /user/entechlog/hadoop/input/

Space used by a directory in HDFS

Command: hadoop fs -du -s -h /user/entechlog/hadoop/input/copydir/

Delete a file

Command: hadoop fs -rm /user/entechlog/hadoop/input/copydir/HadoopFile1.txt

Ensure this file is no longer in HDFS

Command: hadoop fs -ls /user/entechlog/hadoop/input/copydir/HadoopFile1.txt

Delete all files from the folder

Command: hadoop fs -rm /user/entechlog/hadoop/input/copydir/\*

Empty the trash

Command: hadoop fs –expunge

Remove all files from a directory and the directory itself

Command: hadoop fs -rm -r /user/entechlog/hadoop/input/copydir/

Copy files between HDFS

Command: hadoop fs -cp /user/entechlog/hadoop/input/HadoopFile3.txt /user/entechlog/hadoop/input/HadoopFile4.txt

Copy file from HDFS to Local

Command: hadoop fs -copyToLocal /user/entechlog/hadoop/input/HadoopFile4.txt /data/home/entechlog/hadoop/input/HadoopFile4.txt
Command: hadoop fs -get /user/entechlog/hadoop/input/HadoopFile4.txt /data/home/entechlog/hadoop/input/HadoopFile5.txt

Display last kilobyte of the file

Command: hadoop fs -tail /user/entechlog/hadoop/input/HadoopFile4.txt

Changing permission of a file

Command: hadoop fs -chmod 600 /user/entechlog/hadoop/input/HadoopFile4.txt

Changing the owner

Command: hadoop fs -chown newUser:newGroup /user/entechlog/hadoop/input/HadoopFile4.txt

Changing the group

Command: hadoop fs -chgrp newGroup /user/entechlog/hadoop/input/HadoopFile4.txt

Move a directory from one location to other

Command: hadoop fs -mv source target

List all the hadoop file system commands

Command: hadoop fs

Hadoop file system commands help

Command: hadoop fs –help

To know the system is running under safemode or not

Command: hadoop dfsadmin -safemode get

Turn on safemode on when it is off

Command: hadoop  dfsadmin  -safemode enter

Turn off safemode off when it is on

Command: hadoop  dfsadmin  -safemode leave

Top

Share: