Measure Data Flow Between Pipes

Measure data flow between commands during pipeline execution.

Terminal

<command1> | pv -s <expected_amount_of_data_for_eta> | <command2>

<command1> | pv -s <expected_amount_of_data_for_eta> | <command2>

#!/bin/bash

# Measure Data Flow Between Pipes
{{command1}} | pv {{[-s|--size]}} {{expected_amount_of_data_for_eta}} | {{command2}}

When To Use

When optimizing resource usage in complex command sequences.

Pro Tip

Use the -s flag to predict completion times based on expected data size.

Command Builder

Tune the command before you copy it

Generated Command

<command1> | pv -s <expected_amount_of_data_for_eta> | <command2>

Anatomy of Output

Data flowing: 45.5MiB/s Throughput Rate

Indicates how fast data is passing through.

ETA: 00:01:30 Estimated Time of Arrival (ETA)

Forecasted completion time.

Data processed: 500MiB Total Data

Indicates total amount of data processed.

Power User Variants

command1 | pv -s 500M | command2

Measure data flow with an expected size of 500MB.

Troubleshooting

Invalid option: '-s'

Solution: Ensure the flag is supported by the version of pv being used.

Command not found: command1 or command2

Solution: Validate that both commands are correctly installed and accessible.

Expected data size not specified

Solution: Provide a valid data size for ETA calculation.

Command Breakdown

<command1>: Base Command; The executable that performs this operation. Here it runs Command1 before the shell applies any redirect operators.
<command1>: command1; The value supplied for command1.
-s: s| size; The value supplied for s| size.
<expected_amount_of_data_for_eta>: expected amount of data for eta; The value supplied for expected amount of data for eta.
<command2>: command2; The value supplied for command2.
-s: Command Option; Tool-specific option used by this command invocation.

Alternative Approaches

Alternative data processing tools for the same job.