fsck

Sergey Ostapov

Runs the HDFS checking utility. This command allows to detect problems with various files, for example, missing file blocks or under-replicated blocks.

Unlike a traditional fsck utility for native file systems, this command does not fix the errors it detects. Normally NameNode automatically corrects most of the recoverable failures. By default, fsck ignores open files but provides an option to select all files during the analysis.

The usage is as follows:

$ hdfs fsck <path>
  [-list-corruptfileblocks |
  [-move | -delete | -openforwrite]
  [-files [-blocks [-locations | -racks | -replicaDetails | -upgradedomains]]]
  [-includeSnapshots][-storagepolicies] [-maintenance]
  [-blockId <blk_Id>] [-replicate]

Arguments
path	Starts checking from this path
-delete	Deletes corrupted files
-files	Prints out files being checked
-files -blocks	Prints out the block report
-files -blocks -locations	Prints out locations for every block
-files -blocks -racks	Prints out network topology for DataNode locations
-files -blocks -replicaDetails	Prints out each replica details
-files -blocks -upgradedomains	Prints out upgrade domains for every block
-move	Moves corrupted files to /lost+found
-openforwrite	Prints out files opened for write
-storagepolicies	Prints out storage policy summary for the blocks
-storagepolicies	Prints out storage policy summary for the blocks
-maintenance	Prints out maintenance state node details
-blockId	Prints out information about the block
-replicate	Initiates replication work to make mis-replicated blocks satisfy block placement policy

Example:

$ hdfs fsck myDir/file_test.csv -files -blocks -locations

Found a mistake? Seleсt text and press Ctrl+Enter to report it