libera_utils.scene_id.FootprintData#

class libera_utils.scene_id.FootprintData(data: Dataset)#

Bases: object

Container for footprint data with scene identification capabilities.

Manages satellite footprint data through the complete scene identification workflow, including data extraction, preprocessing, derived field calculation, and scene classification.

Parameters:

data (xr.Dataset) – Input dataset containing required footprint variables

_data#

Internal dataset of footprint data. During scene identification, scene IDs are added as variables to this dataset.

Type:

xr.Dataset

process_ssf_and_camera(ssf_path, scene_definitions)#

Process SSF and camera data to identify scenes

process_cldpx_viirs_geos_cam_groundscene()#

Process alternative data format (not implemented)

process_clouds_groundscene()#

Process cloud/ground scene data (not implemented)

Notes

This class handles the complete pipeline from raw satellite data to scene identification, including: 1. Data extraction from NetCDF files 2. Missing value handling 3. Derived field calculation (cloud fraction, optical depth, etc.) 4. Scene ID matching based on classification rules

Methods

from_ceres_ssf(ssf_path, scene_definitions)

Process SSF (Single Scanner Footprint) and camera data to identify scenes.

from_cldpx_viirs_geos_cam_groundscene()

Process cloud pixel/VIIRS/GEOS/camera/ground scene data format.

from_clouds_groundscene()

Process clouds/ground scene data format.

identify_scenes([additional_scene_definitions])

Calculate scene IDs for all data points.
__init__(data: Dataset)#

Methods

from_ceres_ssf(ssf_path, scene_definitions)

Process SSF (Single Scanner Footprint) and camera data to identify scenes.

from_cldpx_viirs_geos_cam_groundscene()

Process cloud pixel/VIIRS/GEOS/camera/ground scene data format.

from_clouds_groundscene()

Process clouds/ground scene data format.

identify_scenes([additional_scene_definitions])

Calculate scene IDs for all data points.
_calculate_required_fields(result_fields: list[str])#

Calculate necessary derived fields on data from input FootprintVariables.

Computes derived atmospheric variables needed for scene identification, handling dependencies between calculated fields automatically.

Parameters:

result_fields (list of str) – List of field names to calculate (e.g., ‘cloud_fraction’, ‘optical_depth’)

Raises:

ValueError – If an unknown field is requested or if circular dependencies exist

Notes

This method modifies self._data in place to conserve memory. It automatically resolves dependencies between calculated fields (e.g., optical depth depends on cloud fraction being calculated first).

The calculation order is determined by dependency analysis and may require multiple passes. A maximum of 30 iterations is allowed to prevent infinite loops from circular dependencies.

Available calculated fields are defined in _CALCULATED_VARIABLE_MAP.

_calculate_single_field_from_spec(spec: CalculationSpec, calculated: list[str])#

Calculate a single field from input FootprintVariables.

Applies the calculation function specified in the CalculationSpec to the input variables, creating a new variable in the dataset.

Parameters:

  • spec (CalculationSpec) – Specification defining the calculation to perform

  • calculated (list of str) – List of variable names already available in the dataset

Raises:

ValueError – If required input variables are not available in the dataset

_convert_missing_values(input_missing_value: float)#

Convert input missing values in footprint data to output missing values.

This method standardizes missing value representations by converting from the input dataset’s missing value convention to the output convention used in FootprintData processing (np.NaN).

Parameters:

input_missing_value (float) – Missing value indicator used in input data (e.g., -999.0, 9.96921e+36)

Notes

Handles two cases: - If input_missing_value is NaN: Uses np.isnan() for comparison - If input_missing_value is numeric: Uses direct equality comparison

Modifies self._data in place, replacing all occurrences of input_missing_value with np.NaN.

Examples

>>> footprint._data = xr.Dataset({'temp': [20.0, -999.0, 25.0]})
>>> footprint._convert_missing_values(-999.0)
>>> print(footprint._data['temp'].values)
array([20., nan, 25.])
static _extract_data_from_CeresSSFNOAA20FM6Ed1C(dataset: Dataset) → Dataset#

Extract data from CERES SSF NOAA-20 FM6 Edition 1C NetCDF file.

Reads specific variables from the hierarchical group structure of CERES SSF (Single Scanner Footprint) files and organizes them into a flat xarray Dataset with standardized variable names.

Parameters:

dataset (xr.Dataset) – Open NetCDF dataset in CeresSSFNOAA20FM6Ed1C format

Notes

Data is extracted from NetCDF groups: - Surface_Map: Surface type information - Cloudy_Footprint_Area: Cloud properties (fraction, phase, optical depth) - Full_Footprint_Area: Wind vectors - Clear_Footprint_Area: Clear sky coverage

Array indexing: - surface_igbp_type[:,0]: First surface type estimate - layers_coverages[:,1] and [:,2]: Lower and upper cloud layers - cloud_*[:,0] and [:,1]: Lower and upper cloud layers

_fill_column_above_max_value(column_name: str, threshold: float, fill_value=nan)#

Replace values above threshold with fill value for specified column.

Parameters:

  • column_name (str) – Name of the column/variable to process

  • threshold (float) – Maximum allowed value - values above this will be replaced

  • fill_value (float, optional) – Value to use as replacement for out-of-range data. Default is NaN.

Raises:

ValueError – If the specified column is not found in the dataset

Examples

>>> footprint._data = xr.Dataset({'cloud_fraction': [50, 120, 80]})
>>> footprint._fill_column_above_max_value('cloud_fraction', 100.0)
>>> print(footprint._data['cloud_fraction'].values)
array([50., nan, 80.])
classmethod from_ceres_ssf(ssf_path: Path, scene_definitions: list[SceneDefinition])#

Process SSF (Single Scanner Footprint) and camera data to identify scenes.

Reads CERES SSF data, extracts relevant variables, calculates derived fields, and identifies scene classifications for each footprint.

Parameters:

  • ssf_path (pathlib.Path) – Path to the SSF NetCDF file (CeresSSFNOAA20FM6Ed1C format)

  • scene_definitions (list of SceneDefinition) – List of scene definition objects to apply for classification

Returns:

Processed dataset containing original variables and calculated fields ready for scene identification.

Return type:

FootprintData

Raises:

FileNotFoundError – If the SSF file cannot be found or opened

Notes

Processing steps: 1. Extract variables from SSF NetCDF groups 2. Apply maximum value thresholds to cloud properties 3. Calculate derived fields (cloud fraction, optical depth, wind speed, etc.) 4. Match footprints to scene IDs

Maximum value thresholds applied: - Cloud fraction: 100% - Cloud phase: 2 (ice) - Optical depth: 500

Examples

>>> footprint = FootprintData()
>>> scene_defs = [SceneDefinition(Path("trmm.csv"))]
>>> data = footprint.process_ssf_and_camera(
...     Path("CERES_SSF_NOAA20_2024001.nc"),
...     scene_defs
... )
classmethod from_cldpx_viirs_geos_cam_groundscene()#

Process cloud pixel/VIIRS/GEOS/camera/ground scene data format.

Raises:

NotImplementedError – This data format is not yet supported

Notes

TODO: LIBSDC-672 Implement processing for alternative data formats including: - Cloud pixel data - VIIRS observations - GEOS model data - Camera data - Ground scene classifications

classmethod from_clouds_groundscene()#

Process clouds/ground scene data format.

Raises:

NotImplementedError – This data format is not yet supported

Notes

TODO: LIBSDC-673 Implement processing for cloud and ground scene data formats.

identify_scenes(additional_scene_definitions: list[Path] | None = None)#

Calculate scene IDs for all data points.

This method performs the actual scene identification algorithm on the processed footprint data. Currently a placeholder implementation that should be updated with the actual scene classification logic.

additional_scene_definitionslist of pathlib.Path or None, optional

Additional scene definition CSV files to apply beyond the default TRMM and ERBE definitions. Default is None.

Notes

Default scene definitions used: - TRMM: Tropical Rainfall Measuring Mission scenes - ERBE: Earth Radiation Budget Experiment scenes TODO: LIBSDC-674 Add unfiltering scene ID algorithm

TODO: LIBSDC-589 Implement the scene ID matching algorithm. Scene identification

The implementation should: 1. Assign scene IDs to footprint based on variable ranges in scene definitions (default and custom) 2. Add scene ID variables as columns to self._data 3. Handle unmatched footprints appropriately