Text & Regex
Duplicate Line Remover
Paste your text and remove duplicate lines. Match with or without case and whitespace, keep the first or last occurrence, and strip excess blank space — including removing all blank lines. Copy or download the clean result. Your text never leaves the page.
Options
liveInput
originalOne entry per line. Line endings are preserved in the output.
Output
deduplicatedRead-only result. Copy or download it with the buttons above.
How it works
Each line is compared using the options above. “Ignore all whitespace” strips every space and tab before comparing, while “Ignore leading & trailing” only trims the edges. Keeping the first or last occurrence preserves the order of the retained lines. Blank line cleanup treats empty lines and whitespace-only lines the same way: “Collapse repeats to one” keeps at most one blank line per run, and “Remove all blank lines” drops every blank line so the result contains only lines with text, one entry per line. Everything runs locally in your browser — no text is uploaded.
Practical uses
Remove repeated lines from pasted text while keeping the order of the lines that remain.
Common ways to use it
- De-duplicate a pasted list of emails, IDs or hostnames before importing it.
- Strip repeated log lines so the distinct events are easier to scan.
- Remove blank lines and extra spacing from text copied out of a PDF or spreadsheet.
Remove repeated lines with the default options
Input
apple banana apple
Output
3 lines in · 2 unique · 1 duplicate removed apple banana
The default comparison trims leading and trailing whitespace and is case-sensitive, so "apple" and " apple " count as duplicates but "Apple" does not. The surviving line keeps its original text unless "Trim output lines" is enabled.
Options that matter
- Whitespace — compare lines exactly, ignore only the edges, or ignore all spaces and tabs.
- Keep — retain the first or the last occurrence of each duplicate.
- Blank line cleanup — keep, collapse runs, or remove every blank line.
- Ignore case — treat Apple and apple as the same line.
Things to keep in mind
- A duplicate is matched as a whole line after the chosen rules; there is no column or key-based matching.
- The output keeps the original line text unless "Trim output lines" is on, so comparison rules and displayed text can differ.
- Processing happens in the browser, so extremely large inputs are limited by the memory available on your device.