How to Remove Duplicate Lines From a Text File In Linux

You can combine the sort and uniq command line programs to remove duplicate lines from a text file. In the example below, it will produce a new file without the dups while preserving the original.

sort original.txt | uniq > dups_removed.txt