How to Remove Duplicate Lines From a Text File In Linux

You can combine the sort and uniq command line programs to remove duplicate lines from a text file. In the example below, it will produce a new file without the dups while preserving the original.

sort original.txt | uniq > dups_removed.txt

Leave a Reply

Your email address will not be published. Required fields are marked *