I have a large amount of log files that I need to remove sensitive data from. The sensitive data is provided to me in a text file and is prone to change.
I had hoped to do the equivalent of this:
#!/usr/bin/bash pattern="" for val in 'sed -e 's/.*=//' Client_clean.txt do pattern=$pattern$val"|" done #egrep -e $pattern $1 sed -i 's/$pattern/CLIENT/g' $1 exit 0
The commented out egrep works fine, the sed doesn’t.
Am I right to use sed for this? Or is there a more apt route to take?
Any help appreciated.
To be able to expand a shell variable in the
sed command, you need to use double quotes.
You either need to use the
-r option to
sed or precede the pipe characters with backslashes.
You also need to avoid adding a pipe character at the end. If you have it there, empty strings will be matched.
#!/usr/bin/bash pattern="" delim="" for val in 'sed -e 's/.*=//' Client_clean.txt do pattern=$pattern$delim$val # setting delim after its first use and using it before the new value # results in "foo|bar|baz" instead of "foo|bar|baz|" delim='|' done sed -ri "s/$pattern/CLIENT/g" "$1"
Yes, sed is the way to go.
What I see wrong in your sed command are the quotes. the $pattern won’t be replaced with it’s contents if you use ‘ for quoting. Instead of using ‘ you should use ” :
sed -ri "s/$pattern/CLIENT/g" $1
You also need “-r” for extended regular expressions.
And pattern=$pattern$val”|” should probably be