Configuration Guide

AccelHydra uses Hydra for configuration management, which provides a powerful and flexible way to organize training configurations. This guide covers the essential Hydra patterns used in AccelHydra.

Basic Concepts

`_target_` Pattern

The _target_ key specifies the fully qualified path to a class or function that should be instantiated. Hydra’s hydra.utils.instantiate() uses this to create objects from the configuration.

Example:

loss_fn:
  _target_: torch.nn.CrossEntropyLoss
  ignore_index: -100

Loading this and instantiate it by:

loss_fn = hydra.utils.instantiate(config["loss_fn"], _convert_="all")

is equivalent to:

import torch.nn

loss_fn = torch.nn.CrossEntropyLoss(ignore_index=-100)

Parameter Passing with `_convert_`

When instantiating objects, use _convert_="all" to ensure OmegaConf types are properly converted to Python native types:

model = hydra.utils.instantiate(config["model"], _convert_="all")

Configuration Structure

Single File Configuration

You can put everything in a single train.yaml:

seed: 42
exp_name: my_experiment
epochs: 10

model:
  _target_: model.MyModel
  hidden_dim: 512

optimizer:
  _target_: torch.optim.AdamW
  lr: 0.001

trainer:
  _target_: trainer.MyTrainer
  epochs: ${epochs}
  project_dir: experiments/${exp_name}

Modular Configuration with `defaults`

For better organization, you can split configurations into multiple files using Hydra’s defaults list:

# configs/train.yaml
defaults:
  - basic
  - override hydra/hydra_logging: disabled
  - override hydra/job_logging: disabled
  - _self_

seed: 42
exp_name: my_experiment
epochs: 10

model:
  _target_: model.MyModel

optimizer:
  _target_: torch.optim.AdamW
  lr: 0.001

trainer:
  _target_: trainer.MyTrainer
  epochs: ${epochs}

# configs/basic.yaml
defaults:
  - _self_
  - override hydra/hydra_logging: disabled
  - override hydra/job_logging: disabled

data_root: /path/to/data

hydra:
  output_subdir: null
  run:
    dir: .

The defaults list:

basic: Loads configs/basic.yaml and merges it into the current config
override hydra/hydra_logging: disabled: Disables Hydra’s default logging
_self_: Includes the current file’s configuration

Variable Interpolation

Use ${variable_name} to reference other configuration values:

exp_name: mnist
exp_dir: experiments/${exp_name}  # Results in: experiments/mnist

trainer:
  epochs: ${epochs}  # References the epochs value defined above

Command Line Overrides

Hydra allows you to override any configuration value from the command line:

Basic Overrides

python train.py epochs=20
python train.py optimizer.lr=0.0005
python train.py model.hidden_dim=1024

Multiple Overrides

python train.py epochs=20 optimizer.lr=0.0005 trainer.save_every_n_steps=1000

Adding New Fields

python train.py +new_param=value

Removing Fields

python train.py ~val_dataloader  # Remove val_dataloader

Changing Configuration Groups

python train.py model=resnet50  # Switch to a different model config

OmegaConf Resolvers

AccelHydra registers custom OmegaConf resolvers that can be used in YAML files:

`len()` Resolver

vocab_size: ${len:${vocab_list}}  # Get length of a list

`multiply()` Resolver

total_params: ${multiply:${num_layers},${hidden_dim}}  # Multiply values

These are registered via register_omegaconf_resolvers() which is typically called at the start of train.py.

Advanced Patterns

Conditional Configuration

You can use variable interpolation to create conditional logic:

epochs: 10
use_warmup: true
warmup_epochs: ${epochs:2}  # Uses epochs value (10) if exists, otherwise defaults to 2

warmup_params:
  warmup_epochs: ${warmup_epochs}  # Only used if configured

Config Composition

You can compose multiple config files:

# configs/model/resnet18.yaml
_target_: model.ResNet18

# configs/model/resnet50.yaml
_target_: model.ResNet50

# configs/train.yaml
defaults:
  - model: resnet18
  - basic
  - _self_

Then switch models:

python train.py model=resnet50

Limitations

AccelHydra does not use all of Hydra’s features. Specifically:

Config groups: Limited support, basic usage works but complex config groups may not
Multi-run (sweeps): Not directly supported, use external tools or Hydra’s sweep CLI
Advanced plugins: May require additional setup

For complete Hydra functionality, refer to the official Hydra documentation.

Examples

See the examples for complete configuration examples.