libera_utils.io.smart_open.smart_open

libera_utils.io.smart_open.smart_open(path: str, mode: str = 'rb', enable_gzip: bool = True)

Open function that can handle local files or files in an S3 bucket. It also correctly handles gzip files determined by a *.gz extension.

Parameters:

• path (str or pathlib.Path or cloudpathlib.s3.s3path.S3Path) – Path to the file to be opened. Files residing in an s3 bucket must begin with “s3://”.

• mode (str, Optional) – Optional string specifying the mode in which the file is opened. Defaults to ‘rb’.

• enable_gzip (bool, Optional) – Flag to specify that *.gz files should be opened as a GzipFile object. Setting this to False is useful when creating the md5sum of a *.gz file. Defaults to True.

Return type:

IO or gzip.GzipFile