Duplicates are probably the most common cause of waste of storage space, folder clutter, and slow file management operations. Duplicates typically occur when you copy files from one drive to another, download the same files several times, sync cloud storages, or make backup copies of your files. While discovering duplicates throughout your entire computer can be helpful at times, there are numerous cases when you need to compare two folders only. That is why it is essential to learn how to find duplicate files using Reference folder.
The Reference folder is the folder that contains the original versions of all the files and needs to stay untouched. You do not compare every file on your computer but rather use other folders as a basis for comparison with the Reference one in order to discover duplicates. It is very helpful for photographers, developers, designers, businesspeople, and anyone working with huge numbers of files. This guide will show you several ways of comparing folders and detecting duplicates.
What Does “Find Duplicate Files Using Reference Folder” Mean?
In case of duplicate detection by Reference folder, there is only one folder that is used as the Reference folder. This means that each file contained within this folder will be taken as the original one, while other folders containing copies of these files will be analyzed next.
Example:
- Folder A has your original photos.
- Folder B has copied photos.
- Folder C has backup copies.
With the help of Folder A as the Reference folder, duplicates will be found within B and C without touching the original files. Using this approach, you can manage and prevent the accidental deletion of essential files, thereby simplifying duplicate file management process.
Why Use a Reference Folder?
There are a number of reasons to use a Reference folder to find duplicate files instead of scanning a whole drive:
Greater Precision
No need to compare each file with each other file; only the selected source folder is compared.
Protection of Original Files
The Reference folder stays intact as duplicate files are found in other folders.
Time Savings
Fewer folders means faster scanning.
Clean-up
One can delete duplicates found in secondary folders without deleting any important original files.
Ideal For Backup
One can check backup folders with their original folder to see duplicates that aren’t necessary.
Common Situations Where Reference Folder Comparison Helps
There are many advantages for users of this comparison approach in their everyday activities.
Basic examples are:
- Comparing backup folders
- Distinguishing USB drives with your computer
- Comparing old and new hard disks
- Eliminating duplicate photographs
- Managing music collections
- Sorting downloaded files
- Comparing cloud sync folders
- Distinguishing archived project folders
#1: Compare Folders Manually Using Windows File Explorer
Even though Windows lacks its built-in duplicate file finder utility to compare and delete duplicate files, basic comparison operations can be done manually.
Step 1: Opening Both Folders
- Open your Reference folder and the folder you want to compare.
It is better to keep them in front of each other for easier comparison.
Step 2: Sorting Files
Sort files in both folders according to:
- Name
- Size
- Date modified
- Type of file
Step 3: Searching by Filename
Search for particular filenames using the search box in the File Explorer.
If there are identical filenames in both folders, then compare:
- File size
- Date of creation
- Date of modification
Step 4: Verify the File Before Deleting
One should never delete any file relying on just its name since some files have the same name but differ in contents.
- Check file
- Size
- Preview
- Properties
The above manual process is efficient for smaller folders, but if you have to compare thousands of files, it will be very tough.
#2: Compare Files Using PowerShell
File comparison may be done using hashes in PowerShell.
Hash is a unique digital fingerprint that is created from the file content.
Step 1: Launch PowerShell
Press:
Windows + X
Select:
Windows PowerShell
Step 2: Create File Hashes
Enter the following command:
Get-FileHash "C:\Reference\File.jpg"
Repeat for another file from another folder.
Step 3: Compare File Hashes
If the SHA-256 hashes are identical, then the files are exactly the same.
This technique is highly accurate but very time-consuming for hundreds/thousands of files.
#3: Compare Folders Using Command Prompt
FC (file compare) is available in Windows.
Step 1: Open Command Prompt
Type:
cmd
Run as Administrator.
Step 2: Comparing Files
For example:
fc file1.txt file2.txt
In case of comparing binary files:
fc /b file1.iso file2.iso
This command is useful for single files comparison but not for large folder comparison.
#4: Use Dedicated Advanced Software
The easiest to use and most accurate method is using the Professional Duplicate File Finder Software that offers specialized toolkit to identify duplicate files with a Reference folder.
This software automatically performs comparisons of the files with the help of complex algorithms.
One such good software is SysTools Duplicate File Finder Software, where one can select a Reference folder, compare various folders, evaluate duplicates by various comparison techniques, see previews, and remove duplicate files.
Steps to Find Duplicate Files Using Reference Folder
To find duplicates using Reference folder effectively, it would be easier if you have a dedicated duplicate finder tool that supports the use of Reference folder. The duplicate finder tool will compare the files from the scanning location with the specified Reference folder and not compare all the selected folders.
Step 1: Start the Duplicate Finder Tool and Scan Location Selection
Start your duplicate finder program and select the storage locations for scanning. As per your preference, you may select the whole hard drive, partition, USB drive or even individual folders. In case of nested folders, the Recursive Scan feature needs to be enabled so that all subfolders are scanned along with the main folder.
Step 2: Set Up Duplicate Files Detection Method
Prior to starting the scan, set up the duplicate detection method. You may choose one of the following to detect duplicate files:
- File Content
- File Name
- File Extension
In terms of content-based comparison, choose the desired comparison algorithm. The best compromise between speed and accuracy is the MD5 Hash, SHA-256 is better for uniqueness while Bit-by-Bit Comparison provides the most accurate analysis since it compares bytes of the file.
Step 3: Activate the Reference Folder Option
The third step is to activate the Reference Folder option and specify the path to the folder that contains your original/master files. Then the program starts to detect duplicates based on the specified folder of original files.
Instead of comparing all scanned folders against each other, the program compares files in the specified folders against the files from the Reference folder. Such an approach allows you to protect your original files while detecting duplicates in other folders. If there is no Reference folder set, the program conducts a regular duplicate scan in all selected folders.
Step 4: Apply Optional Scan Filters
In order to make your scan more effective, you have the ability to apply some filters prior to the scan launch. These filters will:
- Set minimum or maximum file size
- Specify a certain date range when the files were created or modified
- Exclude system folders, hidden files, temporary files, restricted files, or any other unnecessary folders
This will make your scan faster since you will only be scanning relevant files.
Step 5: Initiate Duplicate File Scan
Once you have checked all scan settings, press “Start Scan” and the program will analyze all selected folders comparing the files with files from the Reference folder based on the chosen comparison type.
Step 6: Checking the Duplicate Results
Once the process of detection has been completed, go through all detected duplicate files. The results will normally provide full information about all the files, which includes file name, size, path, and the list of duplicates.
However, it is quite important to go through the results before you take the decision to delete the important files, while only the redundant copies are selected.
Step 7: Deleting, Moving or Creating Hard Links
Once you have made sure about the duplicate files, you can take the following actions depending on your needs:
- Delete the duplicate files from your disk.
- Move the duplicate files to another folder.
- Replace duplicate files with Hard Links (if possible), where different file copies refer to the same physical file and do not occupy much disk space.
Once the selected process is completed, you can keep your storage in order while keeping all original files in the Reference folder.
Benefits of Reference Folder Comparison
Selection of a Reference folder offers a number of benefits.
Saving Disk Space
Extra copies unnecessarily occupy disk space.
Efficient Organization
No need to look through all the files; only extra copies appear.
Reduced Backups
Elimination of extra copies prior to creation of backups requires less space.
Increased Efficiency
It becomes easier to find needed documents.
Conclusion
The knowledge on how to find duplicate files using Reference folder is one method of organizing your data in a safe manner without touching the original files. Instead of searching your whole storage space, it is better to focus on selected folders to compare them to a trusted source.
By either using the built-in Windows applications for comparison in case of smaller folders or dedicated applications for larger folders, selecting a Reference folder can be helpful in managing duplicate files. Through the above process, you will be able to minimize the amount of wasted storage space.