How to use the Remove Duplicate Lines
- Paste your list, one item per line, or drop a text or CSV file.
- Choose whether to keep the first or the last copy of each repeated line, and how blank lines are handled.
- Decide whether case and extra spaces matter when lines are matched.
- Check the counts and the preview of removed lines, then copy or download the unique lines.
What does this tool do?
Matching and output are separate: the options under “when matching” only decide which lines count as the same, while the kept lines stay exactly as you pasted them — indentation and spacing included — unless you choose to trim them. So “Apple” and “apple ” can be treated as one item while the line you keep is unchanged.
Keeping the last copy is useful for logs and exports where later entries are more up to date. The result can stay in its original order or be sorted A–Z, Z–A or naturally (item2 before item10). The tool reports how many lines were read, how many unique values remain, which values repeated most, and the line numbers of every removed line.
Why use it?
- Keep the first or last occurrence.
- Case- and space-insensitive matching without changing your text.
- Shows the most repeated values and every removed line.
- Optional sorting; works on lists with hundreds of thousands of lines.
Example
Input:
apple
banana
Apple
cherry
bananaOutput with the default settings (case-sensitive, keep first):
apple
banana
Apple
cherryWith “Ignore upper/lower case” on, Apple is also removed: apple, banana, cherry.
Privacy
Everything you type is processed in your browser. Nothing you enter is sent to our servers or stored by us. There's no account to create and nothing to install.
Frequently asked questions
Does it change the order of my list?
No. The unique lines stay in their original order unless you choose a sort order.
What happens to blank lines?
By default they're removed. You can keep them all, or keep one and treat further blank lines as duplicates.
How do I compare two lists instead?
Use Compare Two Lists to see which items are in one list but not the other, or in both.
Can it treat “Apple” and “apple” as duplicates?
Yes. Turn on “Ignore upper/lower case when matching”. “Ignore extra spaces when matching” (on by default) also treats lines that differ only in spacing as duplicates. Either way, the line you keep is left exactly as you typed it.
Last reviewed by the M2Toolkit team.