Statistics: Dates for item counts

See Best Practices for Decision Pages and Tags for groups

Legend: not started IN PROGRESS STALLED decided

Recommendations

1. All materials: account for Alma date variable quirks. Adjust queries to account for inconsistent behavior for dates due to Alma’s method of manipulating and storing date variables, as well as migration artifacts. For example, many Network Zone portfolios migrated with a null value for Activation Date.

2. Physical materials: use “Physical Items”.”Item Creation Date”

3. Electronic materials: use “Activation Date”

Using the data that currently exists in these fields provides the best balance of costs and benefits: it avoids significant, immediate local cleanup projects while providing counts that are within an acceptable margin of error.

Impact

Reasoning

Background

The AASA-PT Harmonization group reviewed all the UCOP statistics data that can be retrieved via Alma Analytics. Factors considered during this review were the existing UCOP, ACRL and ARL requirements.

• UCOP statistical reporting are collected for risk management purposes.

• ARL reporting guidelines would like institutions to be consistent with their reporting.

• ACRL and ARL reporting requirements focus on catalogued material.

• CEAL reporting requirements ask about both catalogued and un-catalogued material

UCOP and other requirements statistics want counts for a snapshot in time: usually, it’s the end of the most recent fiscal year. Therefore, campuses have in the past used various approaches to exclude any material added after the end of the fiscal year when running reports after that date. For example, a report run on August 1 would be set up to exclude material added after July 1. Alma Analytics has a variety of date fields that could be used to accomplish this. However, these fields each have a variety of drawbacks for several reasons, e.g., campus procedures, migration artifacts, and other data issues. The following variables were reviewed :

• Physical

  • Lifecycle

  • Item Creation Date v. Creation Date

  • Item Receiving Date v. Receiving Date v. Receiving Date (Calendar)

  • Item Modification Date v. Modification Date

• Electronic

  • Lifecycle

  • Portfolio Modification Date v. Modification Date

  • Portfolio Creation Date

  • Portfolio Activation Date

Observations about the current data:

For counts of items added or withdrawn by fiscal year, options include:

• For Added :

  • Physical

    • Use “Physical Items“.”Physical Item Details”.”Lifecycle” = Active AND

    • Use “Physical Items”.”Item Creation Date” OR

    • Use “Physical Items”.”Item Receiving Date” and research significant impacts at San Diego, Davis, Irvine, Los Angeles, Riverside, San Francisco, and Santa Barbara when dates are added into the data extract in Alma Analytics.

  • Electronic

    • Use “E-Inventory”.”Portfolio”.”Lifecycle” = Active AND

    • Use “E-Inventory“.”Portfolio Activation Date”.”Portfolio Activation Date”

• For Withdrawn:

  • Physical

    • Use “Physical Items”.”Item Modification Date”.”Item Modification Date” AND

    • Use “Physical Items”.”Physical Item Details”.”Lifecycle” = Deleted and/or None or NULL AND/OR

    • Find out reasons behind the usage of the Deleted value and harmonize on a field and values for deletion notes

  • Electronic

    • Use “E-Inventory”.”Portfolio”.”Lifecycle” = Deleted and/or None or NULL AND

    • Use “E-Inventory“.”Portfolio Modification Date”.”Portfolio Modification Date”

Note: ExLibris is reviewing the differences in counts when dates are used in Analytics reports as of May, 2023.

Physical

• Lifecycle

  • Three values are available for “Physical Items“.”Physical Item Details”.”Lifecycle”:

    • Active items are active and discoverable in Primo.

    • Deleted items are not discoverable in Primo.

    • None items are not discoverable in Primo. These records are also associated with records without creation dates and possibly other pertinent metadata; consultation with Ex Libris is ongoing.

  • For withdrawn counts, filtering based on lifecycle = “Deleted” or “None” causes:

    • Exclusion of items that were deleted from the repository after the reporting deadline but before the report run date. For example, if a weeding project happens in August 2023 and hundreds of print items and their associated records are removed, a report looking for the items from before July 2023 that runs in October 2023 will exclude those items, even though they were still in the collection before July 2023.

    • Inclusion of items deleted because they were added by mistake, but that do not actually represent items that were removed from the collection.

• Modification Date

  • There are two modification date options:

    • “Physical Items”.”Item Modification Date”, which cleans up and harmonizes the value available in “Physical Items”.”Physical Items Details”.”Modification Date”. ExLibris documentation says: “The Item Modification date is the date of the last change to the item. Therefore, if on January 19, 2016 the item was in process type Acquisition, on January 20, 2016 the item was in process type Request, and on January 21, 2016 the item was in process type Loan (and no additional change to the item was made after January 21), the Item Modification Date is January 21, 2016.“

    • “Physical Items”.”Physical Items Details”.”Modification Date”, which allows us to know when the item record is modified. ExLibris documentation says it “Holds the date the physical item was modified“.

  • Alma Analytics data-based findings:

    • When crossed with Lifecycle = Delete, modification dates could be used to count items withdrawn from the collection (see Prototype Version 1 at https://docs.google.com/spreadsheets/d/1IBV9tKyvO3xq-UZeuLoeEViSmBwgp2YO/edit#gid=552748451 ).

      Another option after a full fiscal year in Alma is to use [count last year] + [count of items added this FY] - [current count] = Withdrawn.

  • Based on the reported knowledge and practices, “Physical Items”.”Item Modification Date” is the best variable to use to determine when the items' records were last modified, especially when the record of the item was deleted.

• Creation Date

  • Automatic date and time stamps for when records are created are a norm in relational databases. In Alma physical items, there are two creation dates:

    • “Physical Items”.”Item Creation Date”, which cleans up and harmonizes the value available in “Physical Items”.”Physical Items Details”.”Creation Date.” ExLibris documentation says this “Stores the item creation date in a date format such as 2/29/2014” and that “All date dimensions include dates up to and including 30 years back and 20 years forward. So, for example, if today is March 17, 2021: The earliest loan date that would appear is March 17, 1991. The latest due date that would appear is March 17, 2041.“).

    • “Physical Items”.”Physical Items Details”.”Creation Date”, which allows us to know when the record on the item was created. ExLibris documentation says that this “Holds the date the physical item was created” and that “This date is assigned by Alma when the physical item is created.“

  • Note: ExLibris documentation does not completely align with findings.

• Alma Analytics data findings:

  • Initial findings for creation date are at https://docs.google.com/document/d/125dx-__jZOHNRH0ysuRLtEr1j33HjV7e/;

    • 14% (4.9 million records of 34+ million total records in Alma Analytics) show no creation date. Therefore, using Creation Date = NULL as filter does not provide accurate counts of records created in each specific, non-NULL fiscal year or date range. (See also https://docs.google.com/spreadsheets/d/12B7ICBG4DLiViDerm4EwN1Y5BtKByngn/).

  • Recent findings from comparing inclusion and exclusion of creation date in report design criteria show that significant records are dropped when dates such as Creation Date is included. Compare https://docs.google.com/spreadsheets/d/1IBV9tKyvO3xq-UZeuLoeEViSmBwgp2YO/edit#gid=552748451 where Creation Date is included against https://docs.google.com/spreadsheets/d/1eoN1qFGTYA4JQG_SY89UvhznCvgGZj6w/ where Creation Date is excluded. See Table 3.

  • Based on the following and our knowledge of the campuses' current practices, “Physical Items”.”Item Creation Date” would be the best variable to use, if reporting counts of items added into the catalog by fiscal year and/or a particular date range is needed:

    • “Physical Items”.”Physical Items Details”.”Creation Date” is the raw data transferred from Alma to Alma Analytics.

    • “Physical Items”.”Item Creation Date” is the processed variable based on the Physical Items.Physical Items Details.Creation Date.

• Receiving Date

  • There are three available receiving date options:

    • “Physical Items”.”Item Receiving Date”

    • “Physical Items”.”Physical Items Details”.”Receiving Date (calendar)”

    • “Physical Items”.”Physical Items Details”.”Receiving Date”

  • Alma :

    • Screenshot of receive new material screen:

      Open

      

      The Receiving Date variable in Alma Analytics comes from a “Received Date” field in Alma that is currently validated. End users can use a calendar drop down; and they can also type dates into the field—the values must translate to a calendar date for the record to save.

  • Alma Analytics:

    • Future date values are found in receiving dates. (see ).

    • Blank and null values are found in receiving dates. This is true for records added after migration. See https://docs.google.com/document/d/125dx-__jZOHNRH0ysuRLtEr1j33HjV7e/

    • Data review for migrated data shows that some date values are read as strings rather than dates, and Alma Analytics cannot filter these correctly. See https://docs.google.com/spreadsheets/d/12B7ICBG4DLiViDerm4EwN1Y5BtKByngn/).

  • Institution-Level Consultation findings:

    • Some campuses report that they do not use this date.

  • Based on the following and our knowledge of the campuses' current practices, “Physical Items”.”Item Receiving Date” would be the best variable to use, if reporting counts of items added into the collection by fiscal year and/or a particular date range is needed:

    • “Physical Items”.”Physical Items Details”.”Receiving Date” is the raw data transferred from the corresponding text-free field in Alma to Alma Analytics.

    • “Physical Items”.”Physical Items Details”.”Receiving Date (Calendar)” is the raw data transferred from the corresponding calendar dropdown menu in Alma to Alma Analytics.

    • “Physical Items”.”Item Receiving Date” is the harmonization/data cleanup processed variable based on “Physical Items”.”Physical Items Details”.”Receiving Date” and “Physical Items”.”Physical Items Details”.”Receiving Date (Calendar)”.

Options Considered

Electronic Items

• Lifecycle

  • Three values are available for “E-Inventory“.”Portfolio”.”Lifecycle”:

    • Active items are active and discoverable in Primo.

    • Deleted items are not discoverable in Primo.

    • None items are not discoverable in Primo.

  • Records where Lifecycle = Deleted or None may not be reliable for the following reasons:

    • Portfolios that were deleted from the repository after the reporting deadline but before the report run date may not be accounted for.

    • Campuses noted that deletions due to deactivations may not be relevant as campuses delete, deactivate and reactivate portfolios regularly as part of normal electronic resource management processes.

    • Items available via “E-Inventory” Subject Area are not available for manual backup counting processes.

    • No thorough historical record modification data available for review.

• Modification Date

  • Two options available:

    • “E-Inventory”.”Portfolio”.”Modification Date”, which is the raw and automatically generated date of when the record on the portfolio was last modified. ExLibris notes “The date information was updated in the portfolio“.

    • “E-Inventory”.”Portfolio Modification Date”.”Portfolio Modification Date”, which cleans up and harmonizes “E-Inventory”.”Portfolio”.”Modification Date”. Per ExLibris: “The Portfolio Modification Date table is a dimension table that stores details about the Portfolio modification date. The modification date of the portfolio (either in the Alma UI or by running a job)“ denotes that the raw “E-Inventory”.”Portfolio”.”Modification Date” date value is automatically generated.

  • When crossed with Lifecycle = Delete, modification dates can be useful to determine and account for portfolios withdrawn from the collection. However, due to the problems noted in the Lifecycle section of this decision page, institutions are requesting to opt out of reporting at least withdrawal counts in statistical reporting.

• Creation Date

  • Only one option is available:

    • “E-Inventory”.”Portfolio Creation Date”.”Portfolio Creation Date”, which is the date when the record on the portfolio was created. Per ExLibris documentation: “The Portfolio Creation Date table is a dimension table that stores details about the date on which the portfolio was activated from the Community Zone and became part of the Institution Zone. Key fields are used whenever calculations are required. Description fields may be used for formatting the display of the report. Alma stores the following types of fields: Calendar Fields – These are date fields as they display in the calendar. Fiscal Date Fields – These are date fields that match the institution's fiscal period. Using this dimension, the user may drill down from year to month to the specific date on which the portfolio was created in the Institution Zone. The number of portfolios is accumulated to the relevant level in the hierarchy.” Further, ExLibris notes that it “Stores the portfolio creation date in the date format 2/29/2012“, meaning that this variable was created to clean up and harmonize a certain raw date of record creation, which we were not provided with.

  • This is a risky option. It is not consistent with the design of the important date variables that have been made into a separate dimension, where the raw Alma Analytics variable the Portfolio Creation Date originated can be determined. However, it does hide up any similar flaw which we currently experience with the Physical Items Subject Area.

  • Based on the design of the Subject Area, “E-Inventory”.”Portfolio Activation Date”.”Portfolio Creation Date” would be the most reasonable variable to use for accounting the counts of portfolios available to the us to consider for activation within a certain date, date range and fiscal year.

• Activation Date

  • Only one option is available:

    • “E-Inventory”.”Portfolio Activation Date”.”Portfolio Activation Date”, which suggests the date when the Portfolio was activated. Per ExLibris documentation: “The Portfolio Activation Date table is a dimension table that stores details about the Portfolio activation date. Key fields are used whenever calculations are required. Description fields may be used for formatting the display of the report. Alma stores the following types of fields: Calendar Fields – These are date fields as they display in the calendar. Fiscal Date Fields – These are date fields that match the institution's fiscal period. Using this dimension, the user may drill down from year to month to the specific date on which the Portfolio was created. The number of portfolios is accumulated to the relevant level in the hierarchy.“ Further, ExLibris notes that it “Stores the portfolio activation date in the date format 2/29/2012“, meaning that this variable was created to clean up and harmonize a certain raw date of activation, which we were not provided with.

  • This is a risky option. It is not consistent with the design of the important date variables that have been made into a separate dimension, where the raw Alma Analytics variable the Portfolio Activation Date originated. However, it does hide up any similar flaw which we currently experience with the Physical Items Subject Area.

  • Based on the design of the Subject Area, “E-Inventory”.”Portfolio Activation Date”.”Portfolio Activation Date” would be the most reasonable variable to use for accounting the counts of portfolios added to the collection within a certain date, date range and fiscal year.

Open

References

ExLibris. (2023). “Physical Items”. Retrieved from last accessed May 26, 2023.

ExLibris. (2023). “Subject Areas”. Retrieved from last accessed May 26, 2023.

Action Log