You can combine the sort and uniq command line programs to remove duplicate lines from a text file. In the example below, it will produce a new file without the dups while preserving the original.
1 | sort original.txt | uniq > dups_removed.txt |
You can combine the sort and uniq command line programs to remove duplicate lines from a text file. In the example below, it will produce a new file without the dups while preserving the original.
1 | sort original.txt | uniq > dups_removed.txt |