Source data and HDF5 schema#

This page documents the analysis-ready source-data interface used by laMEG. It describes the public API, the HDF5 file format, and the distinction between reconstructed cortical-depth layers and BigBrain-informed laminar estimates.

Overview#

Sliding-window source reconstructions can be exported to a self-describing HDF5 file with lameg.source.export_source_time_series_hdf5().

The resulting file can be accessed with lameg.source.LaminarSourceData without loading the complete source array into memory.

The main source dataset has one of two axis orders:

layer, column, time

or:

layer, column, time, trial

Here, layer denotes one surface in the reconstructed multilayer cortical geometry and column denotes corresponding vertices across those surfaces.

Exporting source data#

The exporter derives the number of reconstructed layers, number of cortical columns, and normalized cortical-depth coordinates from a lameg.surf.LayerSurfaceSet.

For example:

from lameg.source import export_source_time_series_hdf5
from lameg.surf import LayerSurfaceSet

surf_set = LayerSurfaceSet("sub-001", 11)

source_fname = export_source_time_series_hdf5(
    data_fname="spm_data.mat",
    out_fname="sub-001_source.h5",
    surf_set=surf_set,
)

The exporter verifies that the source geometry in the inverse solution is consistent with the supplied LayerSurfaceSet.

The cortical-depth convention used throughout the source format is:

0 = pial boundary
1 = grey/white boundary

Consequently, layer_depth increases from the pial surface towards white matter.

Reading source data#

LaminarSourceData should normally be used as a context manager so that the underlying HDF5 file is closed automatically:

from lameg.source import LaminarSourceData

with LaminarSourceData("sub-001_source.h5") as source:
    print(source.shape)
    print(source.axis_order)
    print(source.layer_depth)

    # All reconstructed layers for one cortical column.
    column_data = source.layer(column=6620)

    # One reconstructed layer over a selected time interval.
    layer_data = source.layer(
        layer=4,
        time=(-75, -25),
    )

Selections are read directly from the HDF5 file. Integer selectors remove their corresponding axis, whereas slices and None preserve it.

For files containing an explicit trial axis, a trial can be selected with the trial argument.

Layers and laminae#

layer and lamina have deliberately different meanings in the laMEG source API.

layer

A surface in the reconstructed multilayer cortical geometry. Layers are depth samples on which the electromagnetic inverse solution is defined.

lamina

A BigBrain-informed histological compartment obtained by transforming reconstructed cortical-depth activity using column-specific laminar weights.

A BigBrain mapping can be added to an existing source file without repeating the source reconstruction:

from lameg.source import add_bigbrain_mapping
from lameg.surf import LayerSurfaceSet

surf_set = LayerSurfaceSet("sub-001", 11)

add_bigbrain_mapping(
    "sub-001_source.h5",
    surf_set,
)

Mapped activity can then be accessed on demand:

from lameg.source import LaminarSourceData

with LaminarSourceData("sub-001_source.h5") as source:
    print(source.laminae)

    valid_columns = source.bigbrain_valid_columns

    lamina_v = source.lamina(
        lamina="V",
        column=6620,
    )

Warning

LaminarSourceData.lamina() does not represent a direct measurement of cytoarchitectonic cortical layers. It applies a BigBrain-informed transformation to reconstructed cortical-depth activity. The interpretability of lamina-specific estimates therefore depends on the spatial information available in the MEG or OPM-MEG data, the forward model, cortical geometry, and the assumptions of the source reconstruction.

HDF5 schema#

The current source-data schema version is 1.0.

The schema version is stored independently of the laMEG package version in the root attribute:

lameg_schema_version

LaminarSourceData validates the schema when a file is opened and rejects unsupported schema versions rather than silently interpreting them.

Required root attributes#

lameg_schema_version

Version of the laMEG source-data schema.

lameg_version

Version of laMEG that created the file.

axis_order

Either layer,column,time or layer,column,time,trial.

source_order

Ordering of sources in the original inverse representation. Schema v1 requires layer-major.

n_layers

Number of reconstructed cortical-depth surfaces.

n_columns

Number of corresponding cortical columns.

Files produced by export_source_time_series_hdf5 also contain provenance information including subject_id, orientation_method, fixed_orientation, source_geometry, n_windows, inversion_idx, overlap_combination, and export_strategy.

Core datasets#

source_ts

Shape (layer, column, time) or (layer, column, time, trial). Contains the reconstructed source time series.

time_ms

Shape (time,). Strictly increasing sample times in milliseconds.

layer_depth

Shape (layer,). Normalized reconstructed cortical depths using the convention 0=pial,1=white.

source_vertices

Shape (layer, column, 3), when present. Source coordinates associated with the SPM forward model.

woi_ms

Shape (window, 2), when present. Sliding inversion windows in milliseconds.

window_count

Shape (time,), when present. Number of inversion windows contributing to each reconstructed time sample.

BigBrain mapping#

When a BigBrain mapping has been added, the file contains a bigbrain group.

bigbrain/edges

Shape (column, 7). Column-specific normalized boundaries defining laminae I–VI.

bigbrain/weights

Shape (column, 6, layer). Linear transformation weights from reconstructed cortical-depth samples to BigBrain-informed laminar means.

bigbrain/labels

The labels I, II, III, IV, V, and VI.

bigbrain/valid_columns

Shape (column,). Boolean mask identifying cortical columns for which a valid BigBrain mapping is available.

Columns for which the anatomical mapping cannot be computed are explicitly marked invalid rather than being assigned a neighbouring or population-average mapping. Analyses using BigBrain-informed laminar estimates should therefore inspect bigbrain_valid_columns.

API reference#

class lameg.source.LaminarSourceData(fname)#

Bases: object

Reader for analysis-ready laminar source HDF5 files.

Files are expected to contain source_ts with axis order layer, column, time or layer, column, time, trial and a corresponding time_ms dataset.

Parameters:

fname (str or path-like) – Path to an HDF5 file created by export_source_time_series_hdf5.

Notes

layer refers to the reconstructed layered surfaces. lamina refers to a BigBrain-defined histological compartment. Files without an added BigBrain mapping can still be read normally, but lamina() is available only after add_bigbrain_mapping has been run.

__init__(fname)#
property axis_order#

Stored source axis-order description.

property bigbrain_edges#

BigBrain laminar boundaries as column x boundary, if stored.

property bigbrain_valid_columns#

Boolean mask of cortical columns with valid BigBrain mappings.

property bigbrain_weights#

Full BigBrain mapping weights as column x lamina x layer, if stored.

close()#

Close the underlying HDF5 file.

property closed#

Whether the underlying HDF5 file is closed.

property dtype#

On-disk datatype of source_ts.

property has_bigbrain_mapping#

Whether a complete BigBrain mapping is stored in the source file.

property has_trials#

Whether source_ts has an explicit trial axis.

lamina(lamina=None, column=None, time=None, trial=None)#

Read BigBrain-mapped laminar source activity on demand.

Parameters:

  • lamina ({"I", "II", "III", "IV", "V", "VI"}, int, slice, or None, optional) – BigBrain lamina to return. String labels are matched case-insensitively. Integer indices are zero-based. An integer or string removes the lamina axis from the returned array; a slice or None preserves it.

  • column (int, slice, or None, optional) – Zero-based cortical-column index. An integer removes the column axis from the returned array. A slice or None preserves it.

  • time ((float, float) or None, optional) – Inclusive time interval (start_ms, stop_ms). If None, all time samples are returned.

  • trial (int, slice, or None, optional) – Zero-based trial index for files with an explicit trial axis. An integer removes the trial axis. If the file has no trial axis, trial must be None.

Returns:

BigBrain-mapped source activity. With no scalar selections, the axis order is lamina x column x time or lamina x column x time x trial.

Return type:

numpy.ndarray

Notes

Laminar activity is not stored redundantly. The requested layer-space source data and column-specific BigBrain weights are read from HDF5 and combined on demand using surface_to_laminae.

Examples

src.lamina(column=6620, time=(-300, 100))

Returns lamina x selected_time for one cortical column.

src.lamina(lamina="V", time=(-75, -25))

Returns column x selected_time.

src.lamina(lamina="V", column=6620)

Returns one laminar time series.

property laminae#

Stored BigBrain lamina labels, or None if no mapping is present.

layer(layer=None, column=None, time=None, trial=None)#

Read layer-space source data without loading the full HDF5 array.

Parameters:

  • layer (int, slice, or None, optional) – Zero-based reconstructed layer index. An integer removes the layer axis from the returned array. A slice or None preserves it.

  • column (int, slice, or None, optional) – Zero-based cortical-column index. An integer removes the column axis from the returned array. A slice or None preserves it.

  • time ((float, float) or None, optional) – Inclusive time interval (start_ms, stop_ms). If None, all time samples are returned. Time selection is converted to a contiguous HDF5 slice using numpy.searchsorted.

  • trial (int, slice, or None, optional) – Zero-based trial index for files with an explicit trial axis. An integer removes the trial axis. If the file has no trial axis, trial must be None.

Returns:

Requested source data. Integer selectors remove their corresponding axes, while unselected/sliced axes are retained.

Return type:

numpy.ndarray

Examples

src.layer(column=6620)

Returns layer x time for a file without a trial axis.

src.layer(layer=4, time=(-75, -25))

Returns column x selected_time.

src.layer(layer=4, column=6620)

Returns one time series.

property layer_depth#

Reconstructed depths, with 0=pial and 1=white, if stored.

property n_columns#

Number of cortical columns.

property n_laminae#

Number of stored BigBrain laminae, or 0 if no mapping is present.

property n_layers#

Number of reconstructed layered surfaces.

property n_times#

Number of time samples.

property n_trials#

Number of trials, or 1 if no explicit trial axis exists.

property shape#

Shape of source_ts.

property time#

Stored time vector in milliseconds.

time_values(time=None)#

Return the stored time samples within a millisecond interval.

property vertices#

Source vertex coordinates as layer x column x xyz, if stored.

property window_count#

Number of inversion windows contributing to each time sample.

property woi#

Sliding inversion windows in milliseconds, if stored.

lameg.source.export_source_time_series_hdf5(data_fname, out_fname, surf_set, inv_fname=None, inversion_idx=0, orientation='link_vector', fixed=True, block_size=512, dtype='float32', compression='lzf', compression_opts=4, overwrite=False, max_memory_gb=16.0)#

Reconstruct a sliding-window inverse to an analysis-ready HDF5 file.

The output source dataset is stored as layer x cortical_column x time (or layer x cortical_column x time x trial for multi-trial data).

For typical single-trial/averaged EBBlayer datasets this function uses a fast window-major reconstruction: each window-specific M matrix is opened exactly once, source activity is reconstructed for that window, and the overlapping estimates are accumulated in RAM. This is substantially faster than repeatedly reopening every M file for small cortical blocks. If the full accumulator would exceed max_memory_gb, the function automatically falls back to the lower-memory blockwise strategy.

Parameters:

  • data_fname (str) – SPM-compatible M/EEG dataset containing the sensor data.

  • out_fname (str) – HDF5 file to create.

  • surf_set (LayerSurfaceSet) – Laminar surface hierarchy used to construct the forward model. The number of layers, number of cortical columns, and normalized cortical-depth coordinates are derived from this object.

  • inv_fname (str, optional) – File containing the inverse. Defaults to data_fname.

  • inversion_idx (int, optional) – Zero-based inversion index.

  • orientation (str, optional) – Orientation variant of the downsampled multilayer surface. Default is "link_vector".

  • fixed (bool, optional) – Whether the forward model used fixed source orientations. Default is True.

  • block_size (int, optional) – Cortical columns per block for the low-memory fallback path.

  • dtype (str or numpy dtype, optional) – On-disk source datatype. Accumulation is performed in float64.

  • compression ({"lzf", "gzip", None}, optional) – HDF5 compression filter. lzf is the default because it is much faster than gzip for floating-point source data. Use None for the fastest export and largest file.

  • compression_opts (int, optional) – Gzip compression level. Ignored for LZF/None.

  • overwrite (bool, optional) – Replace an existing output file when True.

  • max_memory_gb (float, optional) – Maximum RAM allocated to the full source accumulator before falling back to blockwise export. Default is 16 GB.

Returns:

Absolute path to the created HDF5 file.

Return type:

str

Notes

Source ordering is assumed to be layer-major, matching the multilayer SPM meshes used by laMEG: all columns from layer 0, followed by all columns from layer 1, and so on.

For each inversion window the reconstruction uses associativity to avoid constructing the large M @ U matrix explicitly:

(M @ U) @ Y == M @ (U @ Y)

This is mathematically equivalent (up to floating-point rounding) and is much cheaper because the window contains far fewer time samples than there are sensors.

lameg.source.add_bigbrain_mapping(source_fname, surf_set, overwrite=False, compression='lzf')#

Add a BigBrain laminar mapping to an exported source HDF5 file.

The mapping is stored independently of the source time series so that an existing source reconstruction does not need to be regenerated when the anatomical laminar mapping is added or updated.

The resulting HDF5 layout is:

/layer_depth
    (layer,)

/bigbrain/edges
    (column, 7)

/bigbrain/weights
    (column, 6, layer)

/bigbrain/labels
    ["I", "II", "III", "IV", "V", "VI"]

/bigbrain/valid_columns
    (column,)

weights[column] is the linear transformation from reconstructed layer activity to mean activity in BigBrain laminae I-VI for that cortical column.

Parameters:

  • source_fname (str or path-like) – HDF5 source file created by export_source_time_series_hdf5.

  • surf_set (LayerSurfaceSet) – Surface set corresponding to the source reconstruction.

  • overwrite (bool, optional) – Replace an existing layer_depth/bigbrain mapping when True. Default is False.

  • compression ({"lzf", "gzip", None}, optional) – HDF5 compression used for edges and weights. Default is "lzf".

Returns:

Absolute path to the modified source HDF5 file.

Return type:

str

Notes

The depth convention used throughout the stored mapping is:

0 = pial boundary
1 = grey/white boundary

Reconstructed layer depths are stored in the same coordinate system as the BigBrain cumulative laminar boundaries.

lameg.source.load_source_time_series(data_fname, mu_matrix=None, inv_fname=None, vertices=None, inversion_idx=0)#

Load or compute source-space time series from MEG data.

This function reconstructs source-level activity either from precomputed inverse solutions (stored in the MEG data file or an external inversion file) or directly from a provided forward/inverse matrix (mu_matrix). Optionally, the reconstruction can be restricted to a subset of vertices.