Critical Data Cleanup Tasks for Vanguard Campuses

Created by Catherine Busselen
Last updated: Jul 09, 2020
7 min read

While the following data cleanup tasks have been identified as critical, the ILSDC group recognizes that the vanguard is only the first step in seeing how our data migrates and how it will work together in this new environment. The vanguard is an opportunity to identify and prioritize data cleanup tasks based on the impact of how data migrates to Alma. 

It would be unrealistic to expect that any campus has flawless data or that we ever will. Furthermore, for some campuses it is not feasible to do any significant data cleanup within such a short timeframe. Campuses must make decisions based on their own needs and resources available at this time. More significant cleanup can be done in preparation for our full test and beyond and we may find that some of the cleanup will best be handled post-migration. 

We will be tracking what campuses are able to accomplish prior to the vanguard extracts so that we know what to test in the vanguard and to better understand what we are seeing once the five campuses have migrated their data.


**It is recommended that any large-scale data cleanup needs be communicated to systems administrators since timing may need to be coordinated.  Your systems administrator may also be able to advise on whether the data cleanup should be done on the system records or within the extracted records that will be sent to ExLibris.

  • Correct Record Matching

  • Prevention of Duplicate Records

  • Prevention of Data Loss

  • Correct P2E Processing

Critical data cleanup recommendations that are pending policy or harmonization decisions are listed at the end of the post.

CORRECT RECORD MATCHING

Millennium Libraries: Migration of OCLC Numbers

  • Action: Confirm records with OCLC number in 001 also have an 003 field with $a(OCoLC) or $aOCoLC

  • Purpose: Ensure accurate creation of Alma 035 field

  • Ramifications if not done: Absence of 003 $a(OCoLC) or 003 $aOCoLC will cause problems creating the Alma 035 OCLC number and result in no match with NZ record

Millennium Libraries: Migration of OCLC Numbers

  • Action: If records contain an OCLC number in both the 001 and the 035 fields, ensure the numeric data matches

  • Purpose: Ensure correct record matching in the NZ

  • Ramifications if not done: IZ record possibly reported as a multi-match and not linked to NZ

Migration/Matching of Vendor System Numbers

For vendor records lacking OCLC numbers that will be included in the NZ:

  • Action for Millennium Libraries: Confirm records contain vendor system number in 001 field and the standard vendor prefix in 003

  • Action for other libraries: Ensure use of the standard vendor number prefix

  • Purpose: Ensure accurate creation of Alma 035 field

  • Ramifications if not done: Absence of 003 $a[vendor prefix] will cause problems creating the Alma 035 system number and result in matching issues in the NZ record

PREVENTION OF DUPLICATE RECORDS

  • Action:  Identify SCP records in local catalog

    • If holdings are SCP-only, remove record (do not send with data extract)

    • If holdings are SCP and local, remove all data related to SCP (599, 793, 856, 9xx fields; 035 SFXObjID for SCP in bib record and remove related holdings record, if applicable)

  • Purpose:  Prevent duplicate records

  • Ramifications if not done:  Duplicate holdings for CDL-licensed resources will show in the Alma NZ

  • Action: For local e-resources that are activated in a link resolver, mark the ILS record to identify and exclude from the P2E data extract

  • Purpose: Prevents duplicate records 

  • Ramifications if not done: There will be duplicate records for e-resource titles migrated from both the link resolver and the ILS

PREVENTION OF DATA LOSS

MARC Tagging

  • Action: Verify that bib records to be included in migration have a MARC 245 field

  • Purpose: Correct migration of bib  records

  • Ramifications if not done: Inclusion of a bib record lacking a 245 field causes the migration to reject records in that file

Empty Data Fields

  • Action: Identify records lacking data in MARC Bibliographic 245 $a and provide a title

  • Purpose: Include record in migration

  • Ramifications if not done:  Bib records lacking 245 $a data will not migrate

Remove Unneeded Institution-Specific Data in 9xx Fields 

Campus-specific data that has been stored in 9xx fields pre-migration will be limited to the 950-999 field range, and some tags in that range will be reserved for consortia use.  This task will help prevent data loss by freeing up 9xx fields that can be used for local data to be stored in fields 950-999.

  • Action:  Since there will be limited 9xx fields for campus-specific data, identify and delete any 9xx data that will not be needed in Alma. Consider data that has been used to create “work arounds” in your current system and whether this data will be needed in Alma.

  • Purpose:  Ensures we do not migrate unnecessary/obsolete data and that we have more flexibility in defining use of the 950-999 fields

  • Ramifications if not done: Unnecessary/obsolete data will be migrated to Alma, and we will have fewer 950-999 fields to work with for storing needed local data.

Other Local Data in Bibliographic Records

  • Action: Identify local data to be retained and:

    • Move the data to one of the following Ex Libris-defined local MARC tags: 

      • 09x

      • 59x

      • 69x

      • 77x/78x

    • Add “$9 LOCAL” to the end of each of those fields (no punctuation or other characters)

    • Note:  Fields 950-999 are also defined by Ex Libris as local fields, but campuses should wait for a pending harmonization decision by the Resource Management Functional Group before moving any data to fields in the 950-999 MARC tag range.

    • Note: 938 is WorldCat data. These can stay and not be marked with $9 LOCAL

  • Purpose: Ensures that local data is not lost in migration and will be retained in the IZ

  • Ramifications if not done: Local data not placed in one of the Ex Libris’ “local MARC tags” with $9 LOCAL will be lost in migration, unless the record is the first into the NZ

Millennium Libraries with Holdings records  852 field $a data

  • Action: Move any 852 $a data to a separate 852 subfield prior to data extract.

  • Purpose: Migration removes 852 $a data

  • Ramifications if not done: Data loss

Millennium Libraries: Multiple Check-in Records with Single Location

  • Action: Collapse/delete/merge check-in records that have the same location so there is only one check-in record per location

  • Purpose: Holdings data for multiple check-in records with identical locations will be migrated to a single holdings record

  • Ramifications if not done: Only holdings from the first check-in record will migrate and the holdings on subsequent records will be lost

Millennium Libraries: Order records with Improper Frequency or Format Coding 

  • Action: Review active order format codes (order type, status, format): ensure order codes for e-resource purchases are accurately reflected in codes, and ensure order codes for print purchases are accurately reflected in codes. 

  • Action: Review active order frequency codes (order type, status, format): ensure order codes for recurring/continual payments are for continual orders (serials, database, packages, or other continuations), and ensure order codes for one-time purchases (serial one-time, mono one-time, multi-volume one time) are for single or one-time orders. 

  • Purpose: Ensure order records are migrated with appropriate PO-Line Type. 

  • Ramifications if not done: Orders, once made in Alma, cannot have their frequency or format changed for the PO-Line Type. Orders would have to be closed, and new orders created post-migration, in order for inventory and orders to fully function in Alma Receipt Bench or E-Activation Task List workflows. Old order data would be split between multiple order records, and difficult to retrieve or collate in Alma/Analytics. Order data would have to be re-sent to vendors for invoicing (specifically for EDI invoicing). 

CORRECT P2E PROCESSING

P2E Process

When identifying local e-resource records to provide in a separate data extract file for P2E processing, consider the following:

  • The P2E process will create a portfolio for every 856 field, so limit the file to records for actual electronic resources. 

  • Some records with 856 fields (e.g., records for finding aids, links to tables of contents, etc.) do not represent ERs and should not be included with the P2E file. 

  • Ideally, the bibliographic record represents the e-resource, not the print equivalent

  • Data cleanup may be needed, depending on how Ex Libris requires your specific ILS to identify electronic resources for the P2E file. 

  • Action: Identify/mark each record with one of the following Alma e-resource types: portfolio, package, database

    • Portfolio: an individual title/resource. The majority of titles on the P2E list will be portfolios. 

    • Package: a collection of resources, usually a collection of portfolios.  

      • Also known as an e-collection in Alma, these are not usually published to Primo. 

      • Packages are “behind-the-scenes” records that allow for the management of individual records for e-resources in a collection to be managed at the package level.  

      • Post-migration, you can associate stand-alone portfolios created by the P2E process with a package in Alma to manage those resources at the package level. Information changes made at the package level (i.e., platform change) will apply to all portfolios associated with that package.

      • The P2E process cannot associate portfolios for various e-resources with a package, so all packages created by the P2E process will be empty.

    • Database:  online databases

      • These migrate as an electronic bib record with an active URL link in Primo VE

    • For more information on portfolio, package, database, and the P2E process see Physical to Electronic (P2E) Processing

  • Purpose: correct migration and creation of e-resources in Alma

  • Ramifications if not done: post-migration cleanup will be needed

PREVENTION OF DUPLICATE RECORDS Pending policy decision on scope of NZ records

Duplicate ER records  

  • Action:  Identify intentional electronic resource duplicates (bibs with same OCLC #) and merge inventory (856 fields) onto one bibliographic record.

  • Alternative action: Maintain intentional duplicate ER records in the IZ with the removal of the OCLC #

  • Purpose:  Ensures all local e-inventory will show in the Alma NZ and in Discovery

  • Ramifications if not done: only inventory from the first matched IZ record will be shown in the Alma NZ and in Discovery

PREVENTION OF DATA LOSS Pending harmonization decisions by Resource Management FG

Move institution-specific data in 900-949 fields to 950-999 fields defined for campus use

Fields 900-949 are reserved by Ex Libris for Network Zone use. Institution-specific 9xx data will be limited to a subset of the 950-999 field range

  • Actions:  

    • Move institution-specific local data in fields 900-949 to fields in the 950-999 range defined for institution use.  

    • Add “$9 LOCAL” to the end of each 950-999 field defined for institution use.

  • Purpose: Ensures that local data is retained in the IZ

  • Ramifications if not done: Local data in fields 900-949 will not be migrated

 

The SILS mission is to transform library services and operations through innovation and collaboration. The future is shared!

Question? Contact AskSILS-L@ucop.edu