These are the most helpful commands for the Hadoop HDFS command line shell.
I run these commands on Hadoop 2.7.1 (OS: CentOS 6.7)
list HDFS directory contents
hdfs dfs -ls <hdfs/path>
create a folder in the HDFS
hdfs dfs -mkdir
put the file from the given path to the remote HDFS location
hdfs dfs -put
view contain of file store to HDFS
hdfs dfs -cat
copy files from HDFS to the local file system
hdfs dfs -get
copy files from source to destination
hdfs dfs -cp
move files from source to destination
hdfs dfs -mv
change the permissions of files, the -R makes the change recursively through the directory structure. More information about the permission, here.
hdfs dfs -chmod [-R]
delete files specified as args. To delete a folder use rmdir . The -R option deletes the directory and any content under it recursively.
hdfs dfs -rm [-R]
returns the checksum information of a file
hdfs dfs -checksum
count the number of directories, files and bytes under the paths that match the specified file pattern
The parameter allows us to view the information in human readable way
hdfs dfs -count [-q] [-v] [-h]
displays sizes of files and directories contained in the given directory or the length of a file in case its just a file.
The parameter -h will format file sizes in a “human-readable” fashion , the -s parameter will result in an aggregate summary of file lengths
hdfs dfs -du [-h] [-s]
Aggregated size visualization
displays free space. The -h option will format file sizes in a “human-readable” fashion
hdfs dfs -df [-h]
empty the trash. Here more information about the trash architecture.
hdfs dfs -expunge
changes the replication factor of a file. If path is a directory then the command recursively changes the replication factor of all files under the directory tree rooted at path
hdfs dfs -setrep [-R]
shows the number of blocks that their location among the cluster nodes and the status of the HDFS for the given path
hdfs fsck [-files] [-blocks] [-locations] [-racks]
find out the total usage of the cluster
hdfs dfsadmin -report
shows the version of the hadoop cluster
For the list of all the commands, check the official documentation: Hadoop FileSystem Shell