Remove Duplicate Lines
Remove repeated lines from a list, keep the first or last copy, or show only the lines that appear once. Runs in your browser, nothing uploaded.
Related tools
All Text tools →This tool runs entirely in your browser. Nothing you enter is sent to our servers, so there is nothing for us to store or see.
About the Remove Duplicate Lines
Paste a list and get it back with the repeats gone. You choose which copy survives — the first occurrence, the last, or neither, which leaves only the lines that appeared exactly once.
What counts as a duplicate is yours to decide too. By default leading and trailing whitespace is ignored and case is not significant, so Hello and hello collapse into one — but the line that survives keeps its original spelling and spacing. Turn on case sensitivity when the difference matters, as it does for identifiers, hashes and passwords.
Original order is preserved unless you ask for the result to be sorted, which matters when the list is a log, a changelog or anything else where sequence carries meaning. The counter below the boxes reports how many lines went in, how many came out and how many distinct values were repeated, so you can see at a glance whether the options did what you meant.
How to use the Remove Duplicate Lines
-
Paste your list
One item per line in the left box, or drop a .txt or .csv file onto it.
-
Decide what counts as a duplicate
Case sensitivity and whitespace trimming are the two settings that change the answer most. Leave them at their defaults for ordinary lists of names, emails or URLs.
-
Choose which copy to keep
Keep first preserves the earliest occurrence, keep last the most recent, and keep neither leaves only lines that were never repeated.
-
Copy or download the result
The count below the boxes tells you how many lines were removed. Copy the result to your clipboard or download it as a text file.
Frequently asked questions
Does it change the order of my list?
No, not unless you ask it to. Lines stay in the order you pasted them and only the repeats are dropped, which matters for logs, changelogs and anything where sequence carries meaning. Turn on the sort option if you would rather have the survivors in alphabetical order.
Are "Hello" and "hello" treated as the same line?
By default yes, and the copy that survives keeps its original capitalisation. Turn on Case sensitive when the difference is significant — for API keys, hashes, passwords or case-sensitive identifiers, two lines differing only in case are genuinely two different values.
What does "keep neither copy" do?
It removes every line that appears more than once, leaving only the lines that occurred exactly once in the whole list. That answers a different question from de-duplication: not "what is in this list" but "what is unique to it", which is how you find one-off entries in a set of records.
Can I use it on a CSV file?
Yes, if you want to remove duplicate rows — each line is compared whole, so identical rows collapse. It does not understand columns, so it cannot de-duplicate on a single field, and a quoted value containing a newline will be treated as two lines.
Is my list uploaded anywhere?
No. Everything runs in your browser and nothing is transmitted or stored, which is why this is safe for customer lists, email addresses and anything else you would not paste into a server-side tool.