libera_utils.footprint_matching.product.aggregate_external_variables#

libera_utils.footprint_matching.product.aggregate_external_variables(mode: OperationalMode, footprints: Sequence[RadiometerFootprint | PseudoFootprint], sources: dict[str, AncillaryReader] | None = None, *, source_file_paths: dict[str, Sequence[Path]] | None = None, weigher: PixelWeigher | None = None, return_coverage: bool = False, n_workers: int | None = None) → dict[str, np.ndarray] | tuple[dict[str, np.ndarray], np.ndarray]#

Aggregate every active reader’s native samples to one value per footprint.

For the given operational mode this selects the active readers via ReaderRegistry.get_readers_for_mode(mode), and for each footprint asks each active reader (samples_in()) for the native samples inside the footprint’s bounding box matched to its time, weights each sample with the PSF weigher at the sample’s own location, and applies each variable’s aggregation strategy (weighted mean / mode / log-mean / std / ranked scene) to collapse the samples to a single value per footprint. No sample is averaged, binned or interpolated before the weigher sees it. The active reader set - and therefore the keys of the returned dict - grows with the mode’s latency (e.g. CAM has ERA5_Surface, IGBP, NISE, VIIRS_BRDF, VIIRS_CLDPROP_D3; IMAGER additionally has ERA5_Pressure, VIIRS_AOD, and the RBSP FLASHFlux and CLDPIX fields). Per-spec gating also applies (see below).

Every output variable is named <READER_KEY>_<spec_name> (e.g. ERA5_Surface_Wind_U10, IGBP_Surface_Type, CLDPIX_Cloud_Mask), the name the product definition declares. The reader’s INSTRUMENT is recorded in each variable’s long_name rather than in the name. A spec is aggregated only when mode is in its modes (e.g. the extended FLASHFlux cloud/aerosol/albedo fields are aggregated for FMATCH-IMAGER only).

Processing is done one footprint at a time, so only the current footprint’s samples are held beyond what each reader holds of its source. With n_workers above one the footprints are split into contiguous along-track chunks and the chunks run in a process pool; the result is identical to the single-process run (every footprint is still processed in time order within its chunk, and a stepped source reloads its step at most once per chunk boundary). Where the platform offers fork (Linux, macOS) the pool is forked after the sources are loaded, so the workers share one copy of every source; otherwise (Windows) the workers are spawned and each builds its own sources from source_file_paths.

The PSF weigher is swappable via weigher. It defaults to the angular-frame PSF (AngularPSFWeigher), oriented by the L1B ephemeris (subsatellite point + satellite radius) and each footprint’s cone-angle rate, for both timescales. A caller whose footprints carry no cone-angle rate passes RadialWeigher explicitly. Both weighers size the PSF from the ephemeris, so camera pseudo-footprints, which carry none yet, raise ValueError.

Coverage-based acceptance (the CERES 75%/95% discard/flag rule) is not applied here – this function computes each footprint’s coverage and, when return_coverage is set, returns it alongside the values, but turning coverage into QA flags (or discarding a footprint) is the caller’s job (see _merge_coverage_qa). Per-footprint coverage is the fraction of the PSF’s in-contour energy backed by a usable sample, measured by the weigher on its reference grid (see WeightField), taken as the minimum across the active sources so a footprint is scored no better than its worst-covered input.

Parameters:

  • mode (OperationalMode) – Operational mode selecting the active readers and product variables.

  • footprints (Sequence[libera_utils.footprint_matching.types.RadiometerFootprint | libera_utils.footprint_matching.types.PseudoFootprint]) – Footprints to aggregate over, in time order: a stepped source reloads its grid whenever the nearest step changes. Each on-Earth footprint must carry the L1B ephemeris the weigher sizes its PSF from, so a PseudoFootprint raises (see _footprint_geometry).

  • sources (dict[str, AncillaryReader], optional) – Reader registry key to an instantiated reader, as build_sources() returns them: exactly the readers active in mode. Pass this or source_file_paths, not both.

  • source_file_paths (dict[str, Sequence[Path]], optional) – Reader-key -> staged granule(s) map the sources are built from. Pass this or sources, not both; a pool that must spawn its workers needs it.

  • weigher (PixelWeigher, optional) – The per-sample PSF weight provider. Defaults to a fresh AngularPSFWeigher, which needs each on-Earth footprint’s cone-angle rate.

  • return_coverage (bool, optional) – When True, also return a per-footprint coverage array. Defaults to False (return only the values dict).

  • n_workers (int, optional) – Worker processes to run the footprints on. None (the default) reads the FMATCH_WORKERS configuration key (config.json, overridable through the environment variable of the same name); 1 runs in the calling process. Fewer than _MIN_FOOTPRINTS_PER_WORKER footprints per worker also runs in the calling process, since the pool would cost more than it saves.

Returns:

With return_coverage=False (default): the mapping of aggregated-variable name to a 1-D array indexed by footprint, carrying NaN where a footprint had no usable data. Float variables keep their reader dtype; categorical (integer) variables are returned as float64 so they can carry NaN too, and the product’s integer fill (see _fill_value_for) replaces it when the column is written. With return_coverage=True: a (values, coverage) tuple, where coverage is a 1-D float array in [0, 1] indexed by footprint (the minimum source coverage; 0 where the footprint had no covered energy).

Return type:

dict[str, np.ndarray] or tuple[dict[str, np.ndarray], np.ndarray]

Raises:

  • ValueError – If neither or both of sources and source_file_paths are supplied; if an on-Earth footprint is a PseudoFootprint, which carries no L1B ephemeris; if a footprint has no finite viewing zenith angle; if an on-Earth footprint has no usable cone-angle rate and weigher is the default AngularPSFWeigher; if a footprint’s time is NaT and a source selects by time; if n_workers, or the FMATCH_WORKERS configuration key it defaults to, is below one or not an integer; or if the pool must be spawned (no fork on this platform) and source_file_paths was not given, so a worker cannot build its sources.

  • KeyError – If the keys of sources or source_file_paths are not exactly the readers active in mode.

  • TypeError – If a footprint is neither a RadiometerFootprint nor a PseudoFootprint.