params-proto
  1. Guides
  2. API Reference

Hyperparameter Sweeps

Hyperparameter sweeps are essential for finding optimal configurations in machine learning and experimentation. This guide shows you how to perform systematic parameter searches using params-proto. Simply wrap the configuration class or the cli entrypoint function with the Sweep class:

Looking for a lighter alternative? Check out piter for a dictionary-based approach with the clean piter @ {...} syntax and operator composition (*, %, **).

python
from params_proto import proto, Sweep


@proto
class Params:
  """Training configuration."""
  lr: float = 0.001  # Learning rate
  batch_size: int = 32  # Batch size

@proto.cli
def train(seed):
  print(seed);

# here we define the sweep:
sweep = Sweep(Params, train)

with sweep.zip as Params, train:
  train.seed = [100, 100, 100]
  Params.batch_size = [32, 64, 128]
  Params.lr = [0.001, 0.01, 0.1]

# This produces 3 configurations (zipped together)
assert tuple(sweep) == (
  {"seed": 100, "params.batch_size": 32, "params.lr": 0.001},
  {"seed": 100, "params.batch_size": 64, "params.lr": 0.01},
  {"seed": 100, "params.batch_size": 128, "params.lr": 0.1},
)

Sweep Operators Reference

The Sweep class provides several operators for combining parameter configurations:

OperatorBehaviorExampleResult
sweep.productCartesian product of all parameter listslr=[0.001, 0.01]
batch=[32, 64]		4 configs:
(0.001, 32), (0.001, 64)
(0.01, 32), (0.01, 64)
sweep.zipZip parameter lists element-wiselr=[0.001, 0.01]
batch=[32, 64]		2 configs:
(0.001, 32), (0.01, 64)
sweep.chainConcatenate multiple sweep configurationsTwo sweeps with 15 configs each30 configs total (concatenated)
sweep.setSet fixed values for parametersseed=42 in outer scopeAll configs inherit seed=42
sweep.each(fn)Apply function to each config (for derived params)fn computes postfix from seedDynamic parameter computation

Combining Operators

Operators can be nested to create complex sweep patterns:

python
with Sweep(Params) as sweep:
    with sweep.set:
        Params.seed = 42  # Fixed for all

        with sweep.product:
            Params.lr = [0.001, 0.01, 0.1]

            with sweep.zip:
                Params.env = ["small", "large"]
                Params.batch = [32, 128]

# Produces 3 × 2 = 6 configs:
# lr=0.001, env="small", batch=32, seed=42
# lr=0.001, env="large", batch=128, seed=42
# lr=0.01, env="small", batch=32, seed=42
# lr=0.01, env="large", batch=128, seed=42
# lr=0.1, env="small", batch=32, seed=42
# lr=0.1, env="large", batch=128, seed=42

Advanced: Dynamic Parameters with .each()

Use .each() to compute parameters that depend on other sweep values:

python
@proto.prefix
class Config:
    seed: int = 10
    experiment_name: str = "default"

with Sweep(Config).product as sweep:
    Config.seed = [10, 20, 30]

@sweep.each
def compute_name(Config):
    Config.experiment_name = f"run-seed-{Config.seed}"

# Produces:
# {"Config.seed": 10, "Config.experiment_name": "run-seed-10"}
# {"Config.seed": 20, "Config.experiment_name": "run-seed-20"}
# {"Config.seed": 30, "Config.experiment_name": "run-seed-30"}

Utility Methods

The Sweep class also provides utility methods for managing and persisting sweep configurations:

MethodDescriptionExample Usage
sweep.save(filename, overwrite=True)Save sweep configs to JSONL filesweep.save("experiments.jsonl")
Sweep.read(filename)Read JSONL file into list of dictsconfigs = Sweep.read("experiments.jsonl")
sweep.load(file, strict=True)Load sweep from JSONL file or listsweep.load("experiments.jsonl")
Sweep.log(deps, filename)Append single config to JSONL fileSweep.log(config, "results.jsonl")
sweep.listConvert sweep to list of config dictsall_configs = sweep.list
sweep.dataframeConvert sweep to pandas DataFramedf = sweep.dataframe
sweep[index]Index or slice into sweep configssweep[0], sweep[:10], sweep[::2]

Example: Save and Load Sweeps

python
from params_proto import proto, Sweep

@proto.prefix
class Params:
    lr: float = 0.001
    batch_size: int = 32

# Create and save a sweep
with Sweep(Params).product as sweep:
    Params.lr = [0.001, 0.01, 0.1]
    Params.batch_size = [32, 64]

sweep.save("my_sweep.jsonl")

# Later, load the sweep
sweep = Sweep(Params).load("my_sweep.jsonl")
for config in sweep:
    print(config)

Example: Using DataFrame for Analysis

python
# Convert sweep to pandas DataFrame for analysis
df = sweep.dataframe
print(df.describe())

# Filter configurations
high_lr = df[df["Params.lr"] > 0.005]

Related