libera_utils.scene_identification.scene_id.FootprintData#

class libera_utils.scene_identification.scene_id.FootprintData(data: Dataset)#

Bases: object

Container for footprint data with scene identification capabilities.

Manages satellite footprint data through the complete scene identification workflow, including data extraction, preprocessing, derived field calculation, and scene classification.

Parameters:: data (xr.Dataset) – Input dataset containing required footprint variables
Variables:: _data (xr.Dataset) – Internal dataset of footprint data. During scene identification, scene IDs are added as variables to this dataset.

process_ssf_and_camera(ssf_path, scene_definitions)#: Process SSF and camera data to identify scenes

process_cldpx_viirs_geos_cam_groundscene()#: Process alternative data format (not implemented)

process_clouds_groundscene()#: Process cloud/ground scene data (not implemented)

Notes

This class handles the complete pipeline from raw satellite data to scene identification, including: 1. Data extraction from NetCDF files 2. Missing value handling 3. Derived field calculation (cloud fraction, optical depth, etc.) 4. Scene ID matching based on classification rules

__init__(data: Dataset)#

Methods

`from_ceres_ssf`(ssf_path)	Process SSF (Single Scanner Footprint) and camera data to identify scenes.
`from_fmatch_cam`(fmatch_path)	Read a FMATCH-CAM product into a FootprintData (radiometer timescale).
`from_fmatch_cam_camtime`(fmatch_path)	Read a FMATCH-CAM-CAMTIME product into a FootprintData (camera timescale).
`identify_scenes`(scene_definitions, ...)	Identify and assign scene IDs to all footprints based on scene definitions.
`to_radiometer_time_product`()	Return the footprint data ready to write on the Libera `RADIOMETER_TIME` axis.
`to_time_product`([time_variable])	Return the footprint data ready to write on its record grid.

_calculate_required_fields(result_fields: list[str])#

Calculate necessary derived fields on data from input FootprintVariables.

Computes derived atmospheric variables needed for scene identification, handling dependencies between calculated fields automatically.

Parameters:: result_fields (list of str) – List of field names to calculate (e.g., ‘Cloud_Fraction’, ‘optical_depth’)
Raises:: ValueError – If an unknown field is requested or if circular dependencies exist

Notes

This method modifies self._data in place to conserve memory. It automatically resolves dependencies between calculated fields (e.g., optical depth depends on cloud fraction being calculated first).

The calculation order is determined by dependency analysis and may require multiple passes. A maximum of 30 iterations is allowed to prevent infinite loops from circular dependencies.

Available calculated fields are defined in _CALCULATED_VARIABLE_MAP.

_calculate_single_field_from_spec(spec: CalculationSpec, calculated: set[str])#

Calculate a single field from input FootprintVariables.

Applies the calculation function specified in the CalculationSpec to the input variables, creating a new variable in the dataset.

Parameters:

spec (CalculationSpec) – Specification defining the calculation to perform
calculated (set of str) – Set of variable names already available in the dataset

Raises:

ValueError – If required input variables are not available in the dataset

_convert_missing_values(input_missing_value: float)#

Convert input missing values in footprint data to output missing values.

This method standardizes missing value representations by converting from the input dataset’s missing value convention to the output convention used in FootprintData processing (np.nan).

Parameters:: input_missing_value (float) – Missing value indicator used in input data (e.g., -999.0, 9.96921e+36)

Notes

Handles two cases: - If input_missing_value is NaN: Uses np.isnan() for comparison - If input_missing_value is numeric: Uses direct equality comparison

Modifies self._data in place, replacing all occurrences of input_missing_value with np.nan.

Examples

>>> footprint._data = xr.Dataset({'temp': [20.0, -999.0, 25.0]})
>>> footprint._convert_missing_values(-999.0)
>>> print(footprint._data['temp'].values)
array([20., nan, 25.])

static _extract_data_from_CeresSSFNOAA20FM6Ed1C(dataset: Dataset) → Dataset#

Extract data from CERES SSF file (using numpy arrays).

Parameters:: dataset (netCDF4.Dataset) – Open NetCDF4 dataset in CeresSSFNOAA20FM6Ed1C format
Returns:: Dataset containing extracted footprint variables as numpy arrays
Return type:: xr.Dataset

_fill_column_above_max_value(column_name: str, threshold: float, fill_value=nan)#

Replace values above threshold with fill value for specified column.

Parameters:

column_name (str) – Name of the column/variable to process
threshold (float) – Maximum allowed value - values above this will be replaced
fill_value (float, optional) – Value to use as replacement for out-of-range data. Default is NaN.

Raises:

ValueError – If the specified column is not found in the dataset

Examples

>>> footprint._data = xr.Dataset({'Cloud_Fraction': [50, 120, 80]})
>>> footprint._fill_column_above_max_value('Cloud_Fraction', 100.0)
>>> print(footprint._data['Cloud_Fraction'].values)
array([50., nan, 80.])

classmethod from_ceres_ssf(ssf_path: str | Path | S3Path)#

Process SSF (Single Scanner Footprint) and camera data to identify scenes.

Reads CERES SSF data, extracts relevant variables, calculates derived fields, and identifies scene classifications for each footprint.

Parameters:: ssf_path (str | pathlib.Path | cloudpathlib.S3Path) – Path (local or S3) to the SSF NetCDF file (CeresSSFNOAA20FM6Ed1C format).
Returns:: Processed footprint data object containing the extracted footprint variables. Scene IDs are added later by identify_scenes().
Return type:: FootprintData
Raises:: FileNotFoundError – If the SSF file cannot be found or opened

Notes

Processing steps: 1. Extract variables from SSF NetCDF groups 2. Apply maximum value thresholds to cloud properties 3. Calculate derived fields (cloud fraction, optical depth, wind speed, etc.) 4. Match footprints to scene IDs using provided scene definitions

Maximum value thresholds applied: - Cloud fraction: 100% - Cloud phase: 2 (ice) - Optical depth: 500

Examples

>>> footprint_data = FootprintData.from_ceres_ssf(Path("CERES_SSF_NOAA20_2024001.nc"))
>>> footprint_data.identify_scenes()

classmethod from_fmatch_cam(fmatch_path: Path) → FootprintData#

Read a FMATCH-CAM product into a FootprintData (radiometer timescale).

FMATCH-CAM is the operational input to SCENE-ID-CAM: one footprint per RADIOMETER_TIME, carrying the Libera-camera-derived cloud fraction and the scene properties needed for classification. This reader will replace the placeholder CERES SSF path (from_ceres_ssf()) once the FMATCH step is producing files.

Parameters:: fmatch_path (pathlib.Path) – Path to a Libera FMATCH-CAM NetCDF product file.
Returns:: Footprint data on the RADIOMETER_TIME dimension, ready for identify_scenes().
Return type:: FootprintData
Raises:: NotImplementedError – Always, until the FMATCH-CAM product format is finalized and this reader is implemented.

classmethod from_fmatch_cam_camtime(fmatch_path: Path) → FootprintData#

Read a FMATCH-CAM-CAMTIME product into a FootprintData (camera timescale).

FMATCH-CAM-CAMTIME is the operational input to SCENE-ID-CAM-CAMTIME. It is defined on a 2-D (CAMERA_TIME, PSEUDOFOOTPRINT) grid: one CAMERA_TIME entry per 2048x2048 image and one PSEUDOFOOTPRINT entry per image subsection (each a contiguous, possibly overlapping block of L1B WFOV camera pixels). It carries both the scene properties and the footprint identifier variables (the inclusive pixel-block bounds Camera_Pixel_X_Min/Camera_Pixel_X_Max/Camera_Pixel_Y_Min/Camera_Pixel_Y_Max, PSF bounding box, boresight geolocation) that the camera-timescale product passes straight through.

When implemented, the reader must also merge the CF-CAM-CAMTIME cloud-fraction input, which is defined on the same (CAMERA_TIME, PSEUDOFOOTPRINT) grid so it aligns 1:1 with FMATCH; the two inputs must share identical CAMERA_TIME and PSEUDOFOOTPRINT extents.

Parameters:: fmatch_path (pathlib.Path) – Path to a Libera FMATCH-CAM-CAMTIME NetCDF product file.
Returns:: Footprint data on the (CAMERA_TIME, PSEUDOFOOTPRINT) grid (one entry per image subsection), ready for identify_scenes().
Return type:: FootprintData
Raises:: NotImplementedError – Always, until the FMATCH-CAM-CAMTIME product format is finalized and this reader is implemented.

identify_scenes(scene_definitions: list[SceneDefinition] | None = <object object>, additional_scene_definitions_files: list[Path] | None = None, report_bin_bounds: bool = True)#

Identify and assign scene IDs to all footprints based on scene definitions.

Applies scene classification rules from one or more SceneDefinition objects to assign scene IDs to each footprint in the dataset.

Parameters:

scene_definitions (list[SceneDefinition] or None, optional) – List of SceneDefinition objects to apply. If omitted, the default standard definitions are used (TRMM, ERBE, and unfiltering; see default_scene_definitions()). Passing an explicit None or an empty list is treated as a caller error and raises ValueError.
additional_scene_definitions_files (list of pathlib.Path or None) – List of scene definition files containing classification rules for custom analysis.
report_bin_bounds (bool, optional) – If True (default), also report, for each classification variable, the (min, max) bounds of the property bin that each footprint’s matched scene occupies, as Scene_Bin_{type}_{variable}_Min / _Max variables. Set to False to emit only the scene ID columns.

Notes

This method modifies self._data in place by adding scene IDs for each row of footprint data.

For each SceneDefinition provided: 1. Validates that all required variables exist in the footprint data 2. Matches each footprint to a scene based on variable ranges 3. Adds a new variable to the dataset with the scene IDs

Footprints that don’t match any scene are assigned a scene ID of 0.

Examples

>>> footprint_data = FootprintData(dataset)
>>> footprint_data.identify_scenes()

to_radiometer_time_product() → Dataset#

Return the footprint data ready to write on the Libera RADIOMETER_TIME axis.

Backward-compatible wrapper around to_time_product() pinned to the RADIOMETER_TIME variable.

Returns:: A copy of the internal dataset with the RADIOMETER_TIME variable promoted to a coordinate.
Return type:: xr.Dataset

to_time_product(time_variable: str = 'RADIOMETER_TIME') → Dataset#

Return the footprint data ready to write on its record grid.

The radiometer-timescale scene-ID products (CAM/IMAGER) contain exactly one footprint per observation time and are written on the same 1-D time dimension as their upstream product, so downstream consumers can align scene IDs to the upstream records positionally. The camera-timescale product (CAM-CAMTIME) is instead written on a 2-D (CAMERA_TIME, PSEUDOFOOTPRINT) grid: one entry per image subsection. FootprintData already carries data on the correct dimensions, so this method only promotes the time variable to a coordinate so the result is ready to hand to libera_utils.io.netcdf.write_libera_data_product() with the matching time_variable.

Parameters:: time_variable (str, optional) – Name of the time variable to promote to a coordinate. Defaults to RADIOMETER_TIME (the radiometer-timescale products); the camera-timescale product passes CAMERA_TIME.
Returns:: A copy of the internal dataset with the time variable promoted to a coordinate and a placeholder Quality_Flag added over the record grid.
Return type:: xr.Dataset
Raises:: ValueError – If time_variable is not present (e.g. the instance was built directly from a synthetic dataset that did not include observation times).