Remove Duplicate Lines

Delete repeated lines from a list and keep the first copy of each, with options to ignore case and trim spaces. Free and private.

Runs in your browserNo sign-upWorks offline
Ctrl+Enter to run
Input131 characters
Output
✓ ReadySample content shown

What the Remove Duplicate Lines does

Remove Duplicate Lines cleans up lists that have the same entry more than once, such as email lists, product codes, keywords or log lines. It keeps the first copy of each line in its original position and drops the repeats, then tells you how many lines were removed and how many unique lines are left.

How to use it

  1. Paste your list into the Input box, one item per line.
  2. Choose the options: ignore upper and lower case, trim spaces at the ends of lines, and remove empty lines.
  3. Read the cleaned list in the Output box and the count in the status line.
  4. Copy the result or download it as a text file.

Example

A spreadsheet of newsletter sign-ups has 1,200 rows, but people signed up more than once and some typed their address in capitals. Paste the email column, keep “Ignore upper and lower case” and “Trim spaces” on, and the tool removes 137 repeats, leaving 1,063 unique addresses in the order they first appeared.

How lines are compared

With “Ignore upper and lower case” on, “Apple” and “apple” count as the same line. With “Trim spaces” on, spaces before and after the text are ignored, which catches copies that only differ by a stray space. The first copy is kept exactly as you typed it.

The order of your list never changes. To sort the lines as well, use the Text Sorter, which can also remove duplicates while sorting.

Questions people ask

Which copy is kept?

The first one in the list. Later copies are removed, and everything else stays in its original order.

Can it find duplicates that are almost the same?

Only differences in case and in leading or trailing spaces. Lines with different spelling or extra words are treated as different.

How long can my list be?

Lists with hundreds of thousands of lines work, limited only by your device’s memory.

Is my list private?

Yes. It is processed in your browser and never uploaded.

Related tools