data_handle

data_handle.py - data handling functions for DAS data

This module provides various functions to handle DAS data, including loading, downloading, and conditioning. It aims at having specific functions for each interrogator type.

Authors: Léa Bouffaut, Quentin Goestchel, Erfan Horeh Date: 2023-2024-2025

das4whales.data_handle.calc_dist_to_xidx(x, selected_channels_m, selected_channels, dx)[source]

Calculate the index of the channel closest to the given distance.

Parameters:

  • x (float) – The distance along the cable.

  • selected_channels_m (List[float]) – The selected channels in meters.

  • selected_channels (List[int]) – The selected channels.

  • dx (float) – The distance between two channels.

Returns:

The index of the channel closest to the given distance.

Return type:

int

das4whales.data_handle.dl_file(url)[source]

Download the file at the given url

Parameters:

url (str) – url location of the file

Returns:

filepath – local path destination of the file

Return type:

Tuple[str, str]

das4whales.data_handle.extract_timestamp(filename)[source]

Extract timestamp from filename in format YYYY-MM-DDTHHMMSSZ.

Parameters:

filename (str)

Return type:

Optional[datetime]

das4whales.data_handle.generate_file_list(base_url, start_file, duration)[source]

Generate a list of file URLs that correspond to a given time range, starting from a known file.

Parameters:

  • base_url (str) – The base URL where the files are hosted.

  • start_file (str) – Filename of the first file to use as reference.

  • duration (int) – Duration in seconds.

Returns:

List of full file URLs covering the time range.

Return type:

List[str]

das4whales.data_handle.get_acquisition_parameters(filepath, interrogator='optasense')[source]

Retrieve acquisition parameters based on the specified interrogator.

Parameters:

  • filepath (str) – The file path to the data file.

  • interrogator (str) – The interrogator type, one of {‘optasense’, ‘silixa’, ‘mars’, ‘alcatel’, ‘onyx’, }. Defaults to ‘optasense’.

Returns:

metadata – Metadata related to the acquisition parameters. Returns None if no matching interrogator is found.

Return type:

Optional[Dict[str, Any]]

Raises:

ValueError – If the interrogator name is not in the predefined list.

das4whales.data_handle.get_cable_lat_lon_depth(file, selected_channels)[source]

Extract latitude, longitude, and depth information from a CSV or TXT file for selected cable channels.

Parameters:

  • file (str) – The file path to the CSV or TXT file containing latitude, longitude, and depth information.

  • selected_channels (Tuple[int, int, int]) – A tuple containing three integers: (start, stop, step) used to slice and select cable channels.

Returns:

position – A dictionary containing the extracted information for the selected channels: - ‘chan_idx_oj’: list of floats, representing the selected cable channel indices. - ‘lat’: list of floats, representing the selected latitude values. - ‘lon’: list of floats, representing the selected longitude values. - ‘depth’: list of floats, representing the selected depth values in absolute magnitude.

Return type:

Dict[str, List[float]]

das4whales.data_handle.get_metadata_asn(filepath)[source]

Gets DAS acquisition parameters for the ASN interrogator e.g., Svalbard data

Parameters:

filepath (str) – a string containing the full path to the data to load

Returns:

metadata – Dictionary filled with metadata, key’s breakdown:

fs: the sampling frequency (Hz)

dx: interval between two virtual sensing points also called channel spacing (m)

nx: the number of spatial samples also called channels

ns: the number of time samples

GL: the gauge length (m)

scale_factor: the value to convert DAS data to strain (strain rate?)

Return type:

Dict[str, Any]

das4whales.data_handle.get_metadata_fosina_dxs(filepath)[source]

Gets DAS acquisition parameters for the Fosina DxS interrogator

Parameters:

filepath (string) – a string containing the full path to the data to load

Returns:

metadata – dictionary filled with metadata, key’s breakdown:

fs: the sampling frequency (Hz)

dx: interval between two virtual sensing points also called channel spacing (m)

nx: the number of spatial samples also called channels

ns: the number of time samples

n: refractive index of the fiber

GL: the gauge length (m)

scale_factor: the value to convert DAS data from strain rate to strain

Return type:

dict

das4whales.data_handle.get_metadata_onyx(filepath)[source]

Gets DAS acquisition parameters for the onyx interrogator

Parameters:

filepath (str) – a string containing the full path to the data to load

Returns:

metadata – dictionary filled with metadata, key’s breakdown:

fs: the sampling frequency (Hz)

dx: interval between two virtual sensing points also called channel spacing (m)

nx: the number of spatial samples also called channels

ns: the number of time samples

n: refractive index of the fiber

GL: the gauge length (m)

scale_factor: the value to convert DAS data from unwrapped optical phase (rad) to strain

Return type:

Dict[str, Any]

das4whales.data_handle.get_metadata_optasense(filepath)[source]

Gets DAS acquisition parameters for the optasense interrogator e.g., OOI South C1 data

Parameters:

filepath (str) – a string containing the full path to the data to load

Returns:

metadata – dictionary filled with metadata, key’s breakdown:

fs: the sampling frequency (Hz)

dx: interval between two virtual sensing points also called channel spacing (m)

nx: the number of spatial samples also called channels

ns: the number of time samples

n: refractive index of the fiber

GL: the gauge length (m)

scale_factor: the value to convert DAS data from unwrapped optical phase (rad) to strain

Return type:

Dict[str, Any]

das4whales.data_handle.get_metadata_silixa(filepath)[source]

Gets DAS acquisition parameters for the silixa interrogator

Parameters:

filepath (str) – a string containing the full path to the data to load

Returns:

metadata – dictionary filled with metadata, key’s breakdown:

fs: the sampling frequency (Hz)

dx: interval between two virtual sensing points also called channel spacing (m)

nx: the number of spatial samples also called channels

ns: the number of time samples

n: refractive index of the fiber

GL: the gauge length (m)

scale_factor: the value to convert DAS data from unwrapped optical phase (rad) to strain

Return type:

Dict[str, Any]

das4whales.data_handle.get_selected_channels(selected_channels_m, dx)[source]

Get the selected channels in channel numbers.

Parameters:

  • selected_channels_m (List[float]) – The selected channels in meters. [ChannelStart_m, ChannelStop_m, ChannelStep_m]

  • dx (float) – The distance between two channels.

Returns:

The selected channels in channel numbers. [ChannelStart, ChannelStop, ChannelStep]

Return type:

List[int]

das4whales.data_handle.hello_world_das_package()[source]

Print a hello world message for the package.

Return type:

None

das4whales.data_handle.load_annotation_csv(filepath)[source]

Load the annotation data from a CSV file. The file must include the following columns: ‘file_name’, ‘apex’, ‘offset’, ‘start_time’, ‘whale_side’, as output from the DAS Source Locator annotation application (https://github.com/leabouffaut/DASSourceLocator).

Parameters:

filepath (str) – The file path to the annotation CSV file to load.

Returns:

annotations – A DataFrame containing the following columns: - ‘file_name’: str, names of files - ‘apex’: int, apex values - ‘offset’: int, offset values - ‘start_time’: float, start times - ‘whale_side’: str, indicating the side of the whale (e.g., ‘left’ or ‘right’).

Return type:

DataFrame

das4whales.data_handle.load_cable_coordinates(filepath, dx)[source]

Load the cable coordinates from a text file.

Parameters:

  • filepath (str) – The file path to the cable coordinates file.

  • dx (float) – The distance between two channels.

Returns:

df – The cable coordinates dataframe.

Return type:

DataFrame

das4whales.data_handle.load_das_data(filename, selected_channels, metadata, interrogator='optasense')[source]

Load the DAS data corresponding to the input file name as strain according to the selected channels.

Parameters:

  • filename (str) – A string containing the full path to the data to load.

  • selected_channels (List[int]) – A list containing the selected channels.

  • metadata (Dict[str, Any]) – A dictionary filled with metadata (sampling frequency, channel spacing, scale factor…).

  • interrogator (str)

Return type:

Tuple[ndarray, ndarray, ndarray, datetime]

Returns:

  • trace (np.ndarray) – A [channel x sample] nparray containing the strain data.

  • tx (np.ndarray) – The corresponding time axis (s).

  • dist (np.ndarray) – The corresponding distance along the FO cable axis (m).

  • file_begin_time_utc (datetime.datetime) – The beginning time of the file, can be printed using file_begin_time_utc.strftime(“%Y-%m-%d %H:%M:%S”).

das4whales.data_handle.load_mtpl_das_data(filepaths, selected_channels, metadata, timestamp, time_window)[source]

Load the DAS data corresponding to the input file names as strain according to the selected channels. Takes multiple files as input and concatenates them along the time axis starting from the input timestamp for the input time window.

Parameters:

  • filepaths (List[str]) – A list containing the full pathes to the data to load.

  • selected_channels (List[int]) – A list containing the selected channels.

  • metadata (Dict[str, Any]) – A dictionary filled with metadata (sampling frequency, channel spacing, scale factor…).

  • timestamp (str) – The timestamp to extract the data from.

  • time_window (float) – The time window duration to extract the data from.

Return type:

Tuple[ndarray, ndarray, ndarray, datetime]

Returns:

  • trace (np.ndarray) – A [channel x sample] nparray containing the strain data.

  • tx (np.ndarray) – The corresponding time axis (s).

  • dist (np.ndarray) – The corresponding distance along the FO cable axis (m).

  • file_begin_time_utc (datetime.datetime) – The beginning time of the file, can be printed using file_begin_time_utc.strftime(“%Y-%m-%d %H:%M:%S”).

das4whales.data_handle.raw2strain(trace, metadata)[source]

Transform the amplitude of raw das data from unwrapped optical phase (rad) to strain according to scale factor

Parameters:

  • trace (ndarray) – a [channel x time sample] nparray containing the raw data in the spatio-temporal domain

  • metadata (Dict[str, Any]) – dictionary filled with metadata (fs, dx, nx, ns, n, GL, scale_factor)

Returns:

trace – a [channel x time sample] nparray containing the strain data in the spatio-temporal domain

Return type:

ndarray