How to remove repeated elements from ArrayList in Java

There are many methods available in java to remove repeated (or) duplicate elements from ArrayList. Here let me explain some of the efficient ways to remove duplicates.

In this article, we are going to see 3 easiest way to remove duplicate elements from ArrayList.

1. Using Set (Recommended Method).
2. By Using Contains method of List.
3. Using Stream - Java 8.

Remove Duplicates from ArrayList by using Set

By default, Set doesn't allow duplicate elements in Java but the list allows. So we can transfer the list elements to Set then it automatically remove duplicate occurrence.

The only thing Set doesn't maintain the insertion order whereas LinkedHashSet maintains it. So if you want to remove duplicates as well as maintain the order then please go with LinkedHashSet.

The program which is using LinkedHashSet below.

package com.tipstocode.packages;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.LinkedHashSet;
import java.util.List;
public class RemoveDuplicatesUsingSet {

public static void main(String[] args) {			
List<String> originalList = new ArrayList<String>();
originalList.add("3");
originalList.add("2");
originalList.add("1");
originalList.add("4");
originalList.add("2");
originalList.add("6");
originalList.add("3");
		
List<String> modifiedList = new ArrayList<String>();		
HashSet<String> modifiedSet = new LinkedHashSet<String>();
		
modifiedSet.addAll(originalList);
modifiedList.addAll(modifiedSet);
		
System.out.println("Original List - "+originalList);		
System.out.println("List without duplicates using LinkedHashSet - "+modifiedList);

}
}

The output for the above program is

Original List - [3, 2, 1, 4, 2, 6, 3]
List without duplicates using LinkedHashSet - [3, 2, 1, 4, 6]

Remove duplicates from ArrayList by using Contains Method of List

We can remove duplicates with the help of the existing contains method of list. We have to transfer the elements from one list to another list if the transferable list does not contain the same element.

Contains method of List checks whether the element is already there or not in the list. if the element already there then we don't need to add anything else we have to add it into the new list.

The below program uses contains method.

package com.tipstocode.packages;
import java.util.ArrayList;
import java.util.List;
public class RemoveDuplicatesUsingContains {
public static void main(String[] args) {	
List<String> originalList = new ArrayList<String>();
originalList.add("3");
originalList.add("2");
originalList.add("1");
originalList.add("4");
originalList.add("2");
originalList.add("6");
originalList.add("3");
		
List<String> modifiedList = new ArrayList<String>();		
if(originalList!=null&&!originalList.isEmpty()){			
for(String element:originalList){
if(!modifiedList.contains(element))
modifiedList.add(element);
}			
}		
System.out.println("Original List - "+originalList);		
System.out.println("List without duplicates using contains method - "+modifiedList);
}
}

The output for the above program is

Original List - [3, 2, 1, 4, 2, 6, 3]
List without duplicates using contains method - [3, 2, 1, 4, 6]

Remove duplicates from ArrayList by using Stream from Java 8

Stream method from Java 8 can easily remove the duplicates from ArrayList. The method distinct().collect(Collectors.toList()) is used to collect the distinct elements from ArrayList and make the new list without duplicates.

The below program uses Stream.

package com.tipstocode.packages;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
public class RemoveDuplicatesUsingStream {
public static void main(String[] args) {		
List<String> originalList = new ArrayList<String>();
originalList.add("3");
originalList.add("2");
originalList.add("1");
originalList.add("4");
originalList.add("2");
originalList.add("6");
originalList.add("3");		
List<String> modifiedList = new ArrayList<String>();		
if(originalList!=null&&!originalList.isEmpty())			
modifiedList = originalList.stream().distinct().collect(Collectors.toList());		
System.out.println("Original List - "+originalList);		
System.out.println("List without duplicates using Stream - "+modifiedList);
}
}

The output for the above program is

Original List - [3, 2, 1, 4, 2, 6, 3]
List without duplicates using Stream - [3, 2, 1, 4, 6]

Remove duplicates from ArrayList of all data types

The above three examples work only for the data type String. it removes the duplicates from ArrayList of string elements. If you want to remove duplicates of Integer elements or double elements from ArrayList then please take a look at the below program.

The below program uses generic <T> data type to support all kinds of data types and remove duplicates from ArrayList.

package com.tipstocode.packages;
import java.util.ArrayList;
import java.util.List;
public class RemoveDuplicatesFromArrayList {	
	public static <T> List<T> removeDuplicatesAllDataTypes(List<T> originalList){			
		List<T> modifiedList = new ArrayList<T>();		
		if(originalList!=null&&!originalList.isEmpty()){			
			for(T element:originalList){
				if(!modifiedList.contains(element))
					modifiedList.add(element);
			}
		}
		return modifiedList;		
	}	
	public static void main(String[] args) {		
		List<String> stringList = new ArrayList<String>();
		stringList.add("3");
		stringList.add("2");
		stringList.add("1");
		stringList.add("4");
		stringList.add("2");
		stringList.add("6");
		stringList.add("3");
		
		List<Integer> integerList = new ArrayList<Integer>();
		integerList.add(3);
		integerList.add(2);
		integerList.add(1);
		integerList.add(4);
		integerList.add(2);
		integerList.add(6);
		integerList.add(3);
		
		List<Double> doubleList = new ArrayList<Double>();
		doubleList.add(3.0);
		doubleList.add(2.0);
		doubleList.add(1.0);
		doubleList.add(4.0);
		doubleList.add(2.0);
		doubleList.add(6.0);
		doubleList.add(3.0);		
		System.out.println("Original List - "+stringList);		
		List<String> modifiedStringList = RemoveDuplicatesFromArrayList.removeDuplicatesAllDataTypes(stringList);		
		List<Integer> modifiedIntegerList = RemoveDuplicatesFromArrayList.removeDuplicatesAllDataTypes(integerList);		
		List<Double> modifiedDoubleList = RemoveDuplicatesFromArrayList.removeDuplicatesAllDataTypes(doubleList);		
		System.out.println("List without duplicates of String elements - "+modifiedStringList);		
		System.out.println("Without duplicates of Integer elements - "+modifiedIntegerList);		
		System.out.println("Removed duplicates of Double elements - "+modifiedDoubleList);
	}
}

The output for the above program is

Original List - [3, 2, 1, 4, 2, 6, 3]
List without duplicates of String elements - [3, 2, 1, 4, 6]
Without duplicates of Integer elements - [3, 2, 1, 4, 6]
Removed duplicates of Double elements - [3.0, 2.0, 1.0, 4.0, 6.0]

The one and only function in the above program works for all the data types of ArrayList.

You may have doubt whether the above example program will work for ArrayList which is having duplicates of an empty string and null values. Yes, Obviously it works.

Using contains() method of list and Stream from Java 8 removes the duplicates even if the ArrayList contains the empty string or null value. contains() method considers empty string and null both as element and check if already exist or not.

stream().distinct().collect(Collectors.toList()) method also consider both empty string and null value as elements and make the list as unique values with one empty string and one null value.

The below example of ArrayList having duplicates of an empty string and null values

package com.tipstocode.packages;
import java.util.ArrayList;
import java.util.List;
public class RemoveDuplicatesUsingContains {
	public static void main(String[] args) {		
		List<String> originalList = new ArrayList<String>();
		originalList.add("3");
		originalList.add("2");
		originalList.add("1");
		originalList.add("4");
		originalList.add("2");
		originalList.add("6");
		originalList.add("3");
		originalList.add("");
		originalList.add("");
		originalList.add(null);
		originalList.add(null);
		
		List<String> modifiedList = new ArrayList<String>();
		
		if(originalList!=null&&!originalList.isEmpty()){			
			for(String element:originalList){
				if(!modifiedList.contains(element))
					modifiedList.add(element);
			}			
		}		
		System.out.println("Orginal List - "+originalList);		
		System.out.println("List without duplicates using contains method - "+modifiedList);
	}
}

The output of the above program is

Orginal List - [3, 2, 1, 4, 2, 6, 3, , , null, null]
List without duplicates using contains method - [3, 2, 1, 4, 6, , null]

The example we have seen by using the Set method also works for removing duplicates of ArrayList of having empty string and null values. Because List allows many empty string values and null values but Set allows at most only one empty string and null value.

If you want the resultant list does not contain null and empty strings then you have to check manually before pushing into a new ArrayList like below.

if(element!=null&&element!="") (or) if(!StringUtils.isBlank(element))

StringUtils.isBlank() methods belongs to apache commons library. It checks if the string is empty or null.