Delete empty lines using sed
LinuxUnixSedLinux Problem Overview
I am trying to delete empty lines using sed:
sed '/^$/d'
but I have no luck with it.
For example, I have these lines:
xxxxxx
yyyyyy
zzzzzz
and I want it to be like:
xxxxxx
yyyyyy
zzzzzz
What should be the code for this?
Linux Solutions
Solution 1 - Linux
You may have spaces or tabs in your "empty" line. Use POSIX classes with sed
to remove all lines containing only whitespace:
sed '/^[[:space:]]*$/d'
A shorter version that uses ERE, for example with gnu sed:
sed -r '/^\s*$/d'
(Note that sed does NOT support PCRE.)
Solution 2 - Linux
I am missing the awk
solution:
awk 'NF' file
Which would return:
xxxxxx
yyyyyy
zzzzzz
How does this work? Since NF
stands for "number of fields", those lines being empty have 0 fiedls, so that awk evaluates 0 to False and no line is printed; however, if there is at least one field, the evaluation is True and makes awk
perform its default action: print the current line.
Solution 3 - Linux
sed '/^$/d'
should be fine, are you expecting to modify the file in place? If so you should use the -i
flag.
Maybe those lines are not empty, so if that's the case, look at this question https://stackoverflow.com/questions/8562782/remove-empty-lines-from-txtfiles-remove-spaces-from-start-and-end-of-line I believe that's what you're trying to achieve.
Solution 4 - Linux
sed
grep
awk
Solution 5 - Linux
I believe this is the easiest and fastest one:
cat file.txt | grep .
If you need to ignore all white-space lines as well then try this:
cat file.txt | grep '\S'
Example:
s="\
\
a\
b\
\
Below is TAB:\
\
Below is space:\
\
c\
\
"; echo "$s" | grep . | wc -l; echo "$s" | grep '\S' | wc -l
outputs
7
5
Solution 6 - Linux
With help from the accepted answer https://stackoverflow.com/questions/8562782/remove-empty-lines-from-txtfiles-remove-spaces-from-start-and-end-of-line">here</a> and the accepted answer above, I have used:
$ sed 's/^ *//; s/ *$//; /^$/d; /^\s*$/d' file.txt > output.txt
`s/^ *//` => left trim
`s/ *$//` => right trim
`/^$/d` => remove empty line
`/^\s*$/d` => delete lines which may contain white space
This covers all the bases and works perfectly for my needs. Kudos to the original posters @Kent and @kev
Solution 7 - Linux
Another option without sed
, awk
, perl
, etc
strings $file > $output
strings - print the strings of printable characters in files.
Solution 8 - Linux
You can say:
sed -n '/ / p' filename #there is a space between '//'
Solution 9 - Linux
You are most likely seeing the unexpected behavior because your text file was created on Windows, so the end of line sequence is \r\n
. You can use dos2unix to convert it to a UNIX style text file before running sed or use
sed -r "/^\r?$/d"
to remove blank lines whether or not the carriage return is there.
Solution 10 - Linux
The command you are trying is correct, just use -E flag with it.
sed -E '/^$/d'
-E flag makes sed catch extended regular expressions. More info here
Solution 11 - Linux
You can do something like that using "grep", too:
egrep -v "^$" file.txt
Solution 12 - Linux
This works in awk as well.
awk '!/^$/' file
xxxxxx
yyyyyy
zzzzzz
Solution 13 - Linux
My bash
-specific answer is to recommend using perl
substitution operator with the global pattern g
flag for this, as follows:
$ perl -pe s'/^\n|^[\ ]*\n//g' $file
xxxxxx
yyyyyy
zzzzzz
This answer illustrates accounting for whether or not the empty lines have spaces in them ([\ ]*
), as well as using |
to separate multiple search terms/fields. Tested on macOS High Sierra and CentOS 6/7.
FYI, the OP's original code sed '/^$/d' $file
works just fine in bash
Terminal on macOS High Sierra and CentOS 6/7 Linux at a high-performance supercomputing cluster.
Solution 14 - Linux
If you want to use modern Rust tools, you can consider:
- ripgrep:
cat datafile | rg '.'
line with spaces is considered non emptycat datafile | rg '\S'
line with spaces is considered emptyrg '\S' datafile
line with spaces is considered empty (-N
can be added to remove line numbers for on screen display)
- sd
cat datafile | sd '^\n' ''
line with spaces is considered non emptycat datafile | sd '^\s*\n' ''
line with spaces is considered emptysd '^\s*\n' '' datafile
inplace edit
Solution 15 - Linux
Using vim editor to remove empty lines
:%s/^$\n//g
Solution 16 - Linux
NF is the command of awk you can use to delete empty lines in a file
awk NF filename
and by using sed
sed -r "/^\r?$/d"
Solution 17 - Linux
For me with FreeBSD 10.1 with sed worked only this solution:
sed -e '/^[ ]*$/d' "testfile"
inside []
there are space and tab symbols.
test file contains:
fffffff next 1 tabline ffffffffffff
ffffffff next 1 Space line ffffffffffff
ffffffff empty 1 lines ffffffffffff
============ EOF =============