libera_utils.obsids

ICIE software ObsID catalog for NOM-HK trimming and calibration pipelines.

Radiometer and camera ObsID numeric values are not globally unique: the same integer can mean different events depending on whether it appears in ICIE__SW_OBSID_RAD or ICIE__SW_OBSID_WFOV. Registry keys are therefore (NomHkObsidSource, obsid).

Several ObsIDs share one TRIMMED product. A TRIMMED product names a calibration dependency family — NOM-HK-SWC-FAMILY-TRIMMED covers all six shortwave LED ObsIDs — and one processing step is deployed per family, not per ObsID. The trimmed file carries the source NOM-HK ObsID field, so a consumer recovers the exact ObsID from the data rather than from the ProductID. Each ObsID keeps its own CAL product, which is what a family step dispatches on.

The catalog itself lives in OBSID_REGISTRY_CSV (libera_utils/data) rather than in this module. Product columns hold DataProductIdentifier member names (e.g. cal_gain), which are resolved and validated when this module is imported. An unknown member name, a product named at the wrong data level, a kind that disagrees with source, a duplicate (source, obsid) key, a CAL product claimed by more than one ObsID, or a TRIMMED family spanning more than one NOM-HK ObsID field all raise ValueError at import time.

A companion file, TRIM_FAMILY_INPUTS_CSV, records the L1A products each family’s cal step consumes besides its own TRIMMED product, exposed as FAMILY_INPUTS / get_family_inputs(). The two files must cover exactly the same set of families, which is checked at import; an empty required_inputs cell means the dependency set is still undecided. A cal step is expected to take its family’s NOM-HK already trimmed as the family product, so required_inputs normally names only the other APIDs. Trimming is not applied to those inputs here: a cal container reads its run’s time range off the TRIMMED NOM-HK filename it was handed and subsets the full daily L1A products itself.

The list of ObsIDs in this repo is meant for practical purposes of science data processing and is a subset of the instrument level source of truth of all ObsIDs, which is owned by the engineering team and available in internal team documentation. That documentation is the authority: when calibration ObsIDs are added, removed, or renamed upstream, this registry has to be reconciled against it.

Module Attributes

OBSID_REGISTRY	Sole source of truth for ObsID → CAL / TRIMMED ProductIDs and catalog metadata.
TRIM_FAMILIES	Inverse view of the TRIMMED column.
FAMILY_INPUTS	L1A products each calibration dependency family's cal step consumes besides its own TRIMMED product, loaded from TRIM_FAMILY_INPUTS_CSV.

Functions

get_family_inputs(trimmed_product)	Return the L1A products a calibration dependency family's cal step consumes.
get_family_specs(trimmed_product)	Return every ObsID belonging to a TRIMMED calibration dependency family.
get_obsid_spec(source, obsid)	Return the registry entry for (source, obsid).
iter_trim_eligible([source])	Yield registry entries that produce TRIMMED NOM-HK products.

Classes

NomHkObsidSource(*values)	Which NOM-HK ObsID variable to use for run detection / trimming.
ObsIdKind(*values)	Category of a known ObsID entry.
ObsIdSpec(obsid, source, kind, description, ...)	One known ICIE software ObsID and its product / telemetry binding.