Remove Duplicate Lines
Deduplicate any list: repeated lines are removed, the first occurrence and the original order are kept.
Private by design — every conversion runs locally in your browser. Your text is never uploaded, stored or tracked.
Duplicate lines creep into every kind of list — email addresses exported twice, keywords merged from two files, log entries, inventory, survey answers. Reading through and deleting repeats by hand is slow and easy to get wrong; this tool does it in one paste.
The rule is simple and predictable: the first time a line appears it is kept, every later repeat is removed, and the surviving lines stay in their original order. Options let you treat lines that differ only by letter case (“Apple” vs “apple”) or by surrounding spaces as duplicates too. The tool reports exactly how many lines were removed, so you can sanity-check the result.
When to use it
- Cleaning email and contact lists merged from several sources
- Deduplicating keyword, tag and category lists for SEO and ads
- Tidying logs, code snippets and configuration entries
- Merging shopping, inventory or vocabulary lists without repeats
- Preparing unique values before importing into a spreadsheet or database
How it works
- The text is split into lines, and the lines are scanned once, top to bottom.
- The first occurrence of each line is kept, in its original position.
- Every later line that matches an already-kept line is dropped, and the removals are counted and shown above the result.
- With the options on, matching ignores letter case and/or surrounding spaces — and with “trim spaces”, the kept lines are output trimmed too.
Examples
2 duplicate lines removed; order preserved.
With “ignore case” off, only exact repeats are removed — here “Paris” and “london” repeats are exact, so both go.
Good to know
Deduplication sounds trivial until the definition of “same line” matters. Two lines can differ by a trailing space, a capital letter, or an invisible character copied from a web page — and each difference makes them unique to an exact matcher while looking identical to you. That is why this tool exposes its two judgement calls as visible options instead of hiding them: ignore case when the list is words or names, trim spaces when the list was pasted from a document or spreadsheet, and leave both off when the data is case-sensitive, like codes or passwords.
Order preservation is the other deliberate choice. Many command-line deduplication recipes sort the list first, which destroys information — the order of a keyword list, a set list or a merged contact file usually means something. This tool never sorts: the output is your list with only the repeats removed, so it can be pasted straight back where the original lived. If you do need sorting as well, sort first in a spreadsheet, then deduplicate here.
Frequently asked questions
Does the tool keep the first or the last occurrence?
The first. Lines are scanned top to bottom; the first occurrence is kept in place and later repeats are dropped, so the output reads like your original list, just without the repeats.
What counts as a duplicate line?
By default, only an exact match — same characters, same case, same spacing. Turn on “Ignore case” to treat “Apple” and “apple” as the same line, and “Trim spaces” to ignore leading and trailing whitespace when comparing (and in the output).
Can I remove duplicate lines in Excel or Google Sheets instead?
Yes: Excel has Data → Remove Duplicates, and Sheets has Data → Data cleanup → Remove duplicates. Those work on spreadsheet columns. This tool is for everything else — text files, pasted lists, code, logs — and it keeps your lines as plain text you can copy anywhere.
Is my text uploaded or stored?
No. The deduplication runs entirely in your browser. Nothing you paste is sent to a server, stored or logged — safe for email lists and other private data.
What happens to empty lines?
An empty line is a line like any other: the first one is kept and later empty lines are treated as duplicates and removed. If your text has many blank lines, the result keeps exactly one — paste sections separately if you need paragraph gaps preserved.
Can it remove duplicate words instead of lines?
Not directly — this tool works line by line. A common workaround: put each word on its own line (most editors can replace spaces with line breaks), deduplicate here, then join the lines back with spaces using the remove line breaks tool on this site.