4. Build a dataset from a YAML config¶

alp_data uses Pydantic for config management and validation. Our DatasetConfig classes contains information about which keys need to be included in a yaml config for a dataset to be instantiated

In [1]:

from alp_data import DatasetConfig

from alp_data import DatasetConfig

In [ ]:

from alp_data.io import read_text

# `read_text` works on local paths and gs:// / r2:// URIs alike (see notebook 02).
print(read_text("configs/beans_basic.yaml"))

from alp_data.io import read_text # `read_text` works on local paths and gs:// / r2:// URIs alike (see notebook 02). print(read_text("configs/beans_basic.yaml"))

Build from yaml¶

dataset_from_config reads a YAML file and returns (dataset, transform_metadata). The top-level key tells it what to build:

• dataset: → single dataset
• concat: → ConcatenatedDataset
• chain: → ChainedDataset

In [2]:

from alp_data import dataset_from_config

dataset, transform_metadata = dataset_from_config("configs/beans_basic.yaml")
print(f"name:   {dataset.info.name}")
print(f"len:    {len(dataset)}")
print(f"meta:   {transform_metadata}")

from alp_data import dataset_from_config dataset, transform_metadata = dataset_from_config("configs/beans_basic.yaml") print(f"name: {dataset.info.name}") print(f"len: {len(dataset)}") print(f"meta: {transform_metadata}")

name:   beans
len:    139
meta:   {}

In [ ]:

sample = dataset[0]
print("keys:", list(sample.keys()))
print("audio shape:", sample["audio"].shape)

sample = dataset[0] print("keys:", list(sample.keys())) print("audio shape:", sample["audio"].shape)