libera_utils.aws.s3_utilities.verify_ingestion#

libera_utils.aws.s3_utilities.verify_ingestion(libera_filenames: list[L0Filename | LiberaDataProductFilename], *, boto_session: Session, timeout: float = 300.0, poll_interval: float = 10.0) → None#

Block until every staged file is confirmed fully ingested, or raise on timeout.

For each file, up to three read-only checks are polled until they pass:

  1. The file exists in its expected archive bucket (at its filename-derived archive prefix).

  2. A Data Availability record exists for the data product/version/applicable-date (skipped for L0 PDS/CR files, which the SDC does not write availability records for).

  3. A File Metadata record exists for the file basename.

All required AWS resources are resolved once up front; finding zero or more than one of any resource raises immediately (it indicates a mismatch between Libera Utils and the deployed SDC). Checks are polled every poll_interval seconds and each check stops being polled as soon as it passes. A per-file summary is always logged; if any check is still unsatisfied at timeout, a TimeoutError is raised.

Parameters:

  • libera_filenames (list of L0Filename or LiberaDataProductFilename) – The validated filenames staged for ingest (as returned by manual_ingest_data_products).

  • boto_session (boto3.Session) – Boto3 session used for all (read-only) AWS interactions.

  • timeout (float, optional) – Maximum number of seconds to wait for full ingestion. Default 300 (5 minutes).

  • poll_interval (float, optional) – Number of seconds between polling passes. Default 10.

Raises:

TimeoutError – If any expected record/object is still missing when the timeout elapses.