Removing duplicate lines from a file
Wednesday, January 14th, 2009Case-insensitive:
{ rm “$file” && awk ‘!x[tolower($1)]++’ > “$file”; } < "$file"
Case-sensitive:
{ rm "$file" && awk '!x[$1]++' > “$file”; } < “$file”
Here’s a shell script you could use, as it stands, only works for one file at a time:
#!/bin/bash [...]