Skip to content
3 min read

Duplicate Candidates and Repeat Outreach in Your ATS

Duplicate candidate records cause repeat outreach, which annoys candidates and wastes recruiter time. Duplicates form when the same person applies with a new email, when sourcing tools import profiles again, and when resumes are parsed into new records. One vendor case study found about one in five records in a database of 34,000 candidates were probable duplicates. Exact matching catches the easy ones, and fuzzy matching with human review handles the rest.

Key takeaways

  • Duplicates create repeat outreach, which is the problem candidates notice first.
  • Match exact keys such as email and phone first, then review fuzzy matches by hand.
  • Keep one outreach history per person, and check it before any new contact.
  • Audit a sample of records each month, because duplicates come back.

A strong engineer applies through a job board in March. A recruiter at the agency sources the same person from a professional network in April, using a different email address. In May, a second recruiter sends a message about a different role. The candidate replies with one line: they have already been contacted about this. The agency now has two records, and a reputation problem that started with a spreadsheet.

How common are duplicates?

We could not find an independent survey that gives an industry-wide rate of duplicate candidate records. The one hard figure we found is a vendor case study. A recruiting consultancy that sells deduplication reported a database of about 34,000 candidates, with roughly 6,800 flagged as probable duplicates. That is about one record in five, and it is one example, not a sample. Fewer than 2,000 of the flagged records were exact matches on name and email, so most needed fuzzy matching.

Candidate records in one vendor case study

  • Candidate records in the database
    34,000
  • Flagged as probable duplicates
    6,800
About one record in five was flagged as a probable duplicate, and most of those needed fuzzy matching to confirm.

Source: 4spot Consulting case study (see Sources). A single vendor example, not a survey.

Why duplicates form

Duplicates are usually created by normal work, not by carelessness. A candidate applies with a personal email and later uses a work address. A sourcing tool imports a profile that already exists under a slightly different name spelling. A resume parser creates a new record because the name field was read differently. Several recruiters or agencies each keep their own list, and nobody owns the master record.

Signs you have this problem

  • Candidates mention they were contacted about the same role twice.
  • Searching for a name returns two or more records for the same person.
  • Recruiters ask in a meeting whether someone is already in the system.
  • Outreach is sent from a sourcing list that was not checked against the ATS.
  • The ATS has no report of duplicate records.

How to measure your own duplicates

Export a sample of 200 candidate records. Group them by email address and by normalized phone number, and count the exact duplicates. Then review a smaller set of likely pairs with matching names and employers, and count the true duplicates. Finally, count how many candidates received more than one outreach in the last 90 days. That last number is the one candidates feel.

The tools that help, and the decisions they leave to you

OpenCATS is an open-source applicant tracking system for recruiting agencies and internal HR teams. pyresparser is a Python tool that extracts skills, experience and contact details from resumes using spaCy and NLTK. flair is an NLP framework that teams use to build entity recognition for candidate documents. Together they cover the record, the parsing and the text extraction.

Deciding that two records are the same person, which record to keep, and who may contact that person next is a policy decision. Tools can suggest matches, but a recruiter should confirm the merge.

A deduplication pass that keeps one record per person

  1. 01Normalize email addresses and phone numbers
  2. 02Merge exact matches on email or phone
  3. 03Score fuzzy matches on name and current employer
  4. 04A recruiter confirms or rejects each probable match
  5. 05Keep one outreach history and check it before any new contact
Exact keys are merged automatically. Fuzzy matches go to a recruiter, and the outreach history stays with the surviving record.

The steps, in order.

Best practice: one person, one record, one history

  • Normalize emails and phone numbers at the point of entry, not later.
  • Match exact keys first, and send fuzzy matches to a person before merging.
  • Check the outreach history before any message, and block contact for a set period after the last one.
  • Audit a sample of records each month, because duplicates come back after every import.
  • Keep candidate data handling within the privacy rules that apply to your candidates, and check with your legal adviser where the rules differ by country.

How Asauvia helps

We build the deduplication pass, the outreach check and the monthly audit for recruiting teams, and connect them to the ATS and sourcing tools you already use. A first engagement usually starts with the 200-record sample above, so you can see the duplicate rate and the repeat-outreach count before anything is merged.

Automated merging of candidate records needs clear rules and a person to confirm the uncertain cases. We agree those rules with your recruiting leads before anything runs.

  • #Recruiting
  • #ATS
  • #TalentAcquisition
  • #RecruitingOps
  • #CandidateData

Sources

  1. 4spot Consulting: Duplicate Candidates Are a Data Discipline Problem
  2. OpenCATS project (GitHub)
  3. pyresparser project (GitHub)
  4. flair NLP project (GitHub)
Before you ask

Questions worth
answering upfront.

See every question we get asked
Can fuzzy matching merge the wrong people?

Yes, which is why fuzzy matches should go to a recruiter before any merge. Exact matches on email or phone are safe to merge automatically.

How do we stop repeat outreach between recruiters?

Check the outreach history before every message, and keep one history per person. A simple rule, such as no new contact within 60 days by another recruiter, prevents most repeat messages.

Do we need to switch to a different ATS?

Usually not. Most duplicates come from how records are created and imported, so the fix is in the process and the checks around the ATS.

Does candidate data deduplication have privacy implications?

It can. Processing candidate data is subject to privacy rules that differ by country. Have your legal adviser confirm the process, especially for candidates in the EU or UK.

Keep reading

All posts

Let's solve
something together.

Tell us what you're building, improving, or automating. We'll come back with a plan, not a pitch.

NDA before any scoping conversation

This blog uses Google Analytics to count visits and see which articles are read. If we show ads, it also uses Google advertising cookies. Nothing is set until you choose. You can change your mind at any time from Cookie settings in the footer. Read our cookie policy.