Home → Cataloging Bibliographic Records → Deduplication and Clean Up Projects → "Needs Humans" Manual Deduplication Process

11.4. "Needs Humans" Manual Deduplication Process

Last Updated 03/09/2026

What are "Needs Humans" Lists?

On This Page

The automantic de-duplication process may flag potential records that can't be verified as matches. These are added to a list to be reviewed by catalogers. Each library recieves their list in a bucket. Catalogers should review the list, compare potential matching bibs, and merge any confirmed matches. The process for comparing and merging directly from the shared bucket is outlined below.

As described in the Deduplicating Matching Records Knowledge Book page, "Needs Humans" lists are lists of MARC records that may be duplicate records but are different enough that they did not automatically merge in the standard de-deuplication process.  These potential matches are added to a "Needs Humans" list for catalogers to review and decide whether or not the matched records should be merged. 

In an effort to simplify the process of reviewing these potential duplicates, flagged bibliographic records are added to a bucket. Each library with items attached to a potential duplicate record will recieve their own bucket.  This allows catalogers to take responsibility for records most relevant to them.  It also makes it possible for catalogers reivewing the recordss to easily access their own physical copy if they need it for reference.

Additionally, merge/overlay profiles simplify the merge process by copying necessary fields from the sub bib to the lead bib.  Detailed instructions on how these profiles work and how to use them follow below.

Merge/Overlay Profiles

The following merge/overlay profiles have been created specifically for this process, and their respective fields that they copy over are listed with them:

• Cardinal Book Dedup
  • =020
  • =035
• Cardinal LP Book Dedup
  • =020
  • =035
  • =250 (important for edition statement like "Large print edition")
• Cardinal Audiobook Dedup
  • =020
  • =024 (audiobooks often have a UPC)
  • =028 (audiobooks often have a publisher/distributor number)
  • =035
  • =250 (important for edition statements like "Abridged" versus "Unabridged")
• Cardinal Video Dedup
  • =020
  • =024 (videos often have a UPC)
  • =028 (videos often have a publisher/distributor number)
  • =035
  • =250 (important for edition statements like "Widescreen" versus "Full screen") 

Each merge/overlay profile has been created to bring over relevant fields from the sub bib to the lead bib upon merging.  There is no way to only bring over non-duplicate fields, so you may end up with multiple fields with identical information on the lead bib.  In such instances, you must delete these duplicate fields.  

Accessing Your Record Bucket

NC Cardinal staff will email a record bucket ID to all library staff with the necessary cataloging permissions.  

To access your shared record bucket:

1. Select "Record Buckets" from the "Cataloging" dropdown menu in the staff OPAC:
   
2. In the Bucket interface, click Shared with Me:
   
3. If you have a significant number of buckets shared with you, use the Bucket ID search to locate the correct one using the number supplied to you by NC Cardinal:
   
4. Click on the hyperlinked bucket name to open the shared bucket.

Organizing the Record Bucket

Records are often included in buckets for multiple libraries because any library with attached copies is eligible to review the record. If a cataloger at another library reviews and merges a record, the record will be deleted, but it will remain in your bucket. Before reviewing the records in the bucket, remove any that have already been reviewed and merged by other catalogers.

1. Ensure that both the Is Deleted? and the Fingerprint columns are visible in your bucket by clicking on the gear icon at the top of the grid. Select the check boxes for Is Deleted? and Fingerprint.
   
2. Click on the "Is Deleted?" column twice to sort the records. The second click will bring any flagged as "Yes" to the top of the list:
   
   
   
3. Check the box in the far left of the grid for any titles flagged as deleted.
   
   
   
4. Select "Remove Selected Records from Bucket" from the Actions menu:
   
5. Once you have removed the deleted records from your bucket, click the Fingerprint column to sort records by a combine title/author/part identifier. Note: This column may appear empty. It should populate after sorting. This will organize duplicate records so that they appear in the list one after the other:
   
   
   

Now you are ready to begin comparing and merging records.

Note:There may be some outliers that have been flagged as potential duplicates, but don't have matching fingerprints, in which case you may still find other matching records elsewhere in the bucket.

Comparing and Merging Records

The Cataloging Knowledge Book provides instructions for comparing MARC records and how to merge them. Catalogers should follow these guidelines when merging records fromteh Needs Humans list.

To compare records in your record bucket:

1. Select the matching records and select "Merge Selected Records" from the Action Menu:
   
   
   
2. This will bring up the "Merge records?" window:
   
   
   
3. Select the appropriate merge/overlay profile from the "Choose merge profile" dropdown list.  Do this first before selecting a lead bib, because once you select a lead bib, the MARC fields from the sub bib will automatically be added to the lead bib.  You do not want to have the incorrect merge profile selected when you select your lead bib.  In the example below, two large print book records are being compared, so the "Cardinal LP Book Dedup" merge/overlay profile is selected:
   
   
   
4. Compare the records using the criteria and guidelines linked above at the beginning of this section and, if they are a good match, select a lead bib by clicking the "Use as lead record" button:
   
   
   
5. After selecting the lead bib, the relevant fields from the sub bib will be added to the lead bib:
   
   
   
6. Remove duplicates, such as the "=250 \\$aLarge print edition." field in the example above, by clicking the "Edit" button above the record:
   
   
   
7. This will open the flat text editor:
   
   
   
8. Delete the duplicate field and click the blue "Save" button.
9. Merge the records by clicking the blue "Merge" button at the bottom of the window:
   
   
   

Note:  If you edit the lead bib and save, but do not merge the records for whatever reason, the edits you made to the lead bib will still be saved and will now be reflected in the bib record in the catalog, just as if you had edited it normally.  Before you make and save any edits to the lead bib, make sure you are going to merge the records after.

Bucket Clean-Up

Once the records have been merged, you may remove them from the bucket:

1. Refresh the bucket to cahnge the deleted flag from No to Yes for the sub record.
2. Select both the lead record and the (now deleted) sub record.
3. Choose "Remove Selected Records from Bucket" from the Actions menu. 
   
   
   

Note: If you have determined the records are not a good match and should not be merged, you should still remove them from the bucket.  That way, if multiple people are working on the same bucket, they will know that those records do not need to be reviewed again.