Rendition Version Regeneration and Cleanup
This section discusses the regeneration of missing renditions and versions and the cleanup of incomplete records that must be deleted. The following new heartbeats are introduced:
-
Validation HeartBeat - This heartbeat runs every 10 minutes and validates the media items by checking the following:
- if they are invalid and then marks them as cleanup required, and
- if they have broken renditions/versions/thumbnails and then marks them as renditions/versions required.
-
Rendition or Version Heartbeat - This heartbeat runs every one hour and regenerates renditions and versions for items marked as renditions/versions required.
- CleanUp HeartBeat - This heartbeat runs once a day and deletes binary and records of all invalid media which are marked as cleanup required.
- Orphan Data and File Cleanup HeartBeat - This heartbeat runs once a day and deletes orphan directories/binaries and orphan media storage records.
All of these heartbeats run on a configured interval. Heartbeat intervals and Threshold times can be configured in values.yaml
under incubator.configuration.digitalAssetManagement
section.
configuration:
# Digital asset management configurations
digitalAssetManagement:
# Enable or disable the heartbeats (validationHeartbeat, renditionOrVersionHeartbeat, cleanUpHeartbeat, orphanDataAndFileCleanupHeartbeat)
enableCleanUpOrRenditionVersionHeartbeats: true
# Interval to run Validation heartbeat
validationHeartbeatIntervalTimeInMinutes: 10
# Interval to run Rendition or Version generation heartbeat
renditionOrVersionHeartbeatIntervalTimeInMinutes: 60
# Interval to run clean up heartbeat
cleanUpHeartbeatIntervalTimeInMinutes: 1440
# Interval to run orphan data and file cleanup heartbeat
orphanDataAndFileCleanupHeartbeatIntervalTimeInMinutes: 1440
# A time period based on creation time, after which asset can be validated
mediaCreationThresholdTimeInMinutes: 1440
# A time period based on last scan time, after which a valid asset can be re-validated
lastScanThresholdTimeInMinutes: 360
# A time period based on creation time, after which orphan directory can be cleaned up
orphanDirectoryModificationThresholdTimeInMinutes: 1440
# A time period based on creation time, after which orphan data in database can be cleaned up
orphanMediaStorageCreationThresholdTimeInMinutes: 1440
# Maximum number of items to be scanned on each validation heartbeat
maxValidationProcessingLimit: 100
Note
enableCleanUpOrRenditionVersionHeartbeats is false
by default. To enable all four heartbeats (validationHeartbeat, renditionOrVersionHeartbeat, cleanUpHeartbeat, orphanDataAndFileCleanupHeartbeat), set enableCleanUpOrRenditionVersionHeartbeats to true
.
Starting from CF213, the following variable names are changed for consistency in variable naming. If these variables are used in a custom values.yaml
file, make sure to update them accordingly.
- enableCleanUpOrRenditionVersionHeartbeats
- validationHeartbeatIntervalTimeInMinutes
- cleanUpHeartBeatIntervalTimeInMinutes
- orphanDataAndFileCleanupHeartBeatIntervalTimeInMinutes