libera_utils.l1a.packets.parse_packets_to_l1a_dataset#

libera_utils.l1a.packets.parse_packets_to_l1a_dataset(packet_files: list[PathLike | str], apid: int, *, ground_data: bool = False, verbose: bool = False, skip_header_bytes: int | None = None, max_workers: int = 1) → Dataset#

Parse packets to L1A dataset with configurable sample expansion.

This function parses binary packet files and expands multi-sample fields according to the a configuration identified by APID. It creates proper xarray Datasets with time coordinates as dimensions.

Parameters:

  • packet_files (list[PathLike | str]) – List of filepaths to packet files, in any order. They are parsed in file name order, which is time order for ground CCSDS names and creation order for flight PDS names.

  • apid (int) – The APID (Application Process Identifier) value for the packet type. Used to select the appropriate configuration for generating the L1A Dataset structure.

  • ground_data (bool, optional) – If True, packets that repeat a SRC_SEQ_CTR with differing contents, or samples that repeat a sample time with differing data, produce a warning instead of a ValueError. This is useful for ground test data where such repeats may be expected. Default is False.

  • verbose (bool, optional) – If True and ground_data is True, a warning will be issued for each such repeat. Default is False.

  • skip_header_bytes (int | None, optional) – Bytes to skip before each CCSDS primary header. When None, uses SKIP_PACKET_HEADER_BYTES from config (default 0, correct for flight PDS and demuxed ground CCSDS; raw ground captures that still carry a per-packet record header need 8).

  • max_workers (int, optional) – Files to parse concurrently, in worker processes. Default 1 (in-process); must be at least 1. See parse_packets_to_dataset; peak memory rises by one in-flight parse per worker, and workers start with fork, which Windows does not provide.

Returns:

xarray Dataset with: - Main packet data array with packet timestamp dimension - Separate arrays for each sample group with optional multi-field expansion - All time coordinates properly set as dimensions

Return type:

xr.Dataset

Raises:

  • ValueError – If the decoded packets have no SRC_SEQ_CTR; if packets adjacent in an input file with consecutive SRC_SEQ_CTR would be placed a counter wrap apart, other than around a single packet whose packet time is wrong, which is dropped with a UserWarning; if ground_data is False and packets with the same SRC_SEQ_CTR differ; as _drop_duplicates raises for samples; or as parse_packets_to_dataset raises, including for max_workers below 1 or a platform without fork.

  • OSError – If a packet file cannot be opened or read.

  • RuntimeError – If a worker process exits without reporting a result (max_workers above 1).

  • Exception – Any exception raised while parsing a file in a worker process is re-raised in the caller, with the worker’s traceback attached as a note.

Notes

The PACKET axis is in SRC_SEQ_CTR order, with the counter’s wraps resolved by packet time (see libera_utils.l1a.packet_ordering), whatever the order of the input files. A packet received twice is kept once. Packet time is not monotonic along the axis; check_packet_order() warns where it steps backward. Input files are parsed in file name order, which input_files lists.

For APID 1040 (ICIE WFOV SCI), the packet dataset is post-processed by enhance_wfov_l1a_dataset: complete SOP→EOP images are stitched onto a CAMERA_TIME dimension, compressed payloads and decoded header metadata are attached, ICIE__WFOV_DATA is dropped, and PACKET_IMAGE_ID traces packets back to stitched images.