count

Sergey Ostapov

Counts the number of directories, files and bytes under the paths that match the specified file pattern. Gets the quota and the usage.

The output columns with -count are: DIR_COUNT, FILE_COUNT, CONTENT_SIZE, PATHNAME.

Returns 0 on success and -1 on error.

The usage is as follows:

$ hadoop fs -count [-q] [-h] [-v] [-x] [-t [<storage type>]] [-u] [-e] <paths>
Arguments

-u

Controls what columns the output contains: QUOTA, REMAINING_QUOTA, SPACE_QUOTA, REMAINING_SPACE_QUOTA, PATHNAME

-q

Controls what columns the output contains: QUOTA, REMAINING_QUOTA, SPACE_QUOTA, REMAINING_SPACE_QUOTA, DIR_COUNT, FILE_COUNT, CONTENT_SIZE, PATHNAME

-t

The list of possible parameters that can be used: all, ram_disk, ssd, disk, archive

-h

Shows sizes in human-readable format

-v

Displays a header line

-x

Excludes snapshots from the result calculation. Without the -x option (default), the result is always calculated from all INodes, including all snapshots under the given path. The -x option is ignored if -u or -q option is given

-e

Shows the erasure coding policy for each file

-e

Controls what columns the output contains: DIR_COUNT, FILE_COUNT, CONTENT_SIZE,
ERASURECODING_POLICY, PATHNAME

-ERASURECODING_POLICY

Name of the policy for the file.
If an erasure coding policy is set on that file, it will return name of the policy. If no erasure coding policy is set, it will return Replicated which means it uses replication storage strategy

Examples:

$ hadoop fs -count hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2
$ hadoop fs -count -q hdfs://nn1.example.com/file1
$ hadoop fs -count -q -h hdfs://nn1.example.com/file1
$ hadoop fs -count -q -h -v hdfs://nn1.example.com/file1
$ hadoop fs -count -u hdfs://nn1.example.com/file1
$ hadoop fs -count -u -h hdfs://nn1.example.com/file1
$ hadoop fs -count -u -h -v hdfs://nn1.example.com/file1
$ hadoop fs -count -e hdfs://nn1.example.com/file1
Found a mistake? Seleсt text and press Ctrl+Enter to report it