Constructing a Bash command with arguments containing spaces, pipes and process substitution

Posted on

Problem :

I am writing a small utility script that can be parametrized to additionally time its execution and also write its stdout to another file.

Let’s assume the basic script is simply:

ls "$@"

Then the following would be a naïve implementation, here using cat as a dummy for more complex handling of the output:

if [[ "$TIMEIT" -eq 1 ]]; then
  if [[ "$LOGIT" -eq 1 ]]; then
    time ls "$@" | tee >(cat)
  else
    time ls "$@"
  fi
else
  if [[ "$LOGIT" -eq 1 ]]; then
    ls "$@" | tee >(cat)
  else
    ls "$@"
  fi
fi

You can see what the probiem is with that, particularly if new parameters should be added.

A rewrite that would be more flexible is:


cmd="ls $@"

if [[ "$TIMEIT" -eq 1 ]]; then
  cmd="time $cmd"
fi

if [[ "$LOGIT" -eq 1 ]]; then
  cmd="$cmd | tee >(cat)"
fi

eval $cmd 

This works, but it breaks once you call it as:

./script.sh "file 1" file2

Because the file names will not be handled properly.

When I apply this solution, I can properly tokenize the arguments when prepending the command with time:

function token_quote {
  local quoted=()
  for token; do
    quoted+=( "$(printf '%q' "$token")" )
  done
  printf '%sn' "${quoted[*]}"
}

cmd=(ls "$@")

if [[ "$TIMEIT" -eq 1 ]]; then
  cmd=(time "${cmd[@]}")
fi

eval "$(token_quote "${cmd[@]}")"

But the same method for handling the pipe does not work. When I adapt it using this solution:

function eval_args {
  local quoted=''
  while (( $# )); do
    if [[ $1 = '|' ]]; then
      quoted+="| "
    else
      printf -v quoted '%s%q ' "$quoted" "$1"
    fi
    shift
  done
  eval "$quoted"
}

cmd=(ls "$@")

# prepend to array
if [[ "$TIMEIT" -eq 1 ]]; then
  cmd=(time "${cmd[@]}")
fi

# append to array?
if [[ "$LOGIT" -eq 1 ]]; then
  cmd=("${cmd[@]}" "|" "tee" ">(cat)")
fi

eval_args "${cmd[@]}"

It creates a file >(cat) … so I’m going down the rabbit hole here, and it seems like this is not the way to go.

What can I do to construct the command and handle spaces/special chararacters in filenames, including pipes/process substitution?

Solution :

Eval

Your last method might work, as long as you skip it for your custom additions and only use it for generating the original command. That is, change eval_args to only build the command string:

build_args() {
    local arg quoted=""
    for arg; do
        printf -v quoted '%s%q ' "$quoted" "$1"
    done
    echo "$quoted"
}

cmd=$(build_args ls "$@")
if (( TIMEIT )); then
    cmd="time $cmd"
fi
if (( LOGIT )); then
    cmd="$cmd | tee >(foo)"
fi

eval "$cmd"

Actually the loop is unnecessary, as bash’s printf will automatically repeat the same format string with extra arguments:

argv=(ls -la "$@")
cmd=$(printf '%q ' "${argv[@]}")
eval "time $cmd | tee >(foo)"

Bash 5.1 has a ${foo@Q} modifier which is a more convenient alternative:

argv=(ls -la "$@")
cmd=${argv[*]@Q}
eval "time $cmd | tee >(foo)"

Alternative to eval

Let’s say your script has its main code in a function:

run() { ls "$@"; }

You can conditionally define a wrapper function for it:

if (( LOGIT )); then
    run_logged() { run "$@" | tee >(cat); }
else
    run_logged() { run "$@"; }
fi

if (( TIMEIT )); then
    run_timed() { time run_logged "$@"; }
else
    run_timed() { run_logged "$@"; }
fi

run_timed "$@"

(The function definition itself could be generated into a string and eval’d.)

Reexec

In some cases, it might make sense to re-execute the entire script under the new environment:

if (( UID > 0 )); then
    exec sudo "$0" "$@" || exit
fi

if (( ! _LOGIT_ACTIVE )); then
    exec _LOGIT_ACTIVE=1 "$0" "$@" | tee >(foo) || exit
fi

ls "$@"

Misc

As for redirects specifically, you could hardcode the use of tee and only swap the filename (not very efficient as it still causes tee to run and duplicate writes, but… it does the job):

if (( LOGIT )); then
    out=/var/log/foo
else
    out=/dev/null
fi

ls "$@" | tee $out

Same for straightforward redirections to a file:

if (( LOGIT )); then
    exec {fd}>/var/log/foo
else
    fd=1     # stdout
fi

ls "$@" >&$fd

if (( fd != 1 )); then
    exec {fd}>&-    # close
fi

Alternatively you could pipe into a condition, or a function that wraps the condition (also not 100% efficient due to mandatory cat):

ls "$@" | if (( LOGIT )); then tee >(whatever); else cat; fi

Leave a Reply

Your email address will not be published. Required fields are marked *