API Reference

Complete API documentation for all optimizers and their methods.

---

Overview

All 23 optimizers in Gradient-Free-Optimizers share a common interface, making it easy to switch between algorithms without changing your code.

# All optimizers follow this pattern
from gradient_free_optimizers import SomeOptimizer

opt = SomeOptimizer(
    search_space,          # Required: dict of parameter arrays
    initialize=None,       # Optional: initialization strategy
    constraints=[],        # Optional: constraint functions
    random_state=None,     # Optional: random seed for reproducibility
    rand_rest_p=0.0,       # Optional: random restart probability
)

opt.search(
    objective_function,    # Your function to maximize
    n_iter=100,           # Number of iterations
    max_time=None,        # Optional: max seconds
    max_score=None,       # Optional: stop when score >= target
    early_stopping=None,  # Optional: stop when progress stalls
    memory=True,          # Cache evaluations
    memory_warm_start=None,  # Continue from previous run
    verbosity=["progress_bar", "print_results"],
    optimum="maximum",    # "maximum" or "minimum"
)

# Results available after search
best_params = opt.best_para
best_score = opt.best_score
all_evaluations = opt.search_data

---

Optimizer Categories

Local Search

• HillClimbingOptimizer

• StochasticHillClimbingOptimizer

• RepulsingHillClimbingOptimizer

• SimulatedAnnealingOptimizer

• DownhillSimplexOptimizer

Global Search

• RandomSearchOptimizer

• GridSearchOptimizer

• RandomRestartHillClimbingOptimizer

• RandomAnnealingOptimizer

• PatternSearch

• PowellsMethod

• LipschitzOptimizer

• DirectAlgorithm

Population-Based

• ParticleSwarmOptimizer

• SpiralOptimization

• ParallelTemperingOptimizer

• GeneticAlgorithmOptimizer

• EvolutionStrategyOptimizer

• DifferentialEvolutionOptimizer

• CMAESOptimizer

Sequential Model-Based

• BayesianOptimizer

• TreeStructuredParzenEstimators

• ForestOptimizer

---

Optimizers

gradient_free_optimizers.HillClimbingOptimizer(...)	Local search optimizer that iteratively moves towards better neighboring solutions.
gradient_free_optimizers.StochasticHillClimbingOptimizer(...)	Hill climbing variant that accepts worse solutions to escape local optima.
gradient_free_optimizers.RepulsingHillClimbingOptimizer(...)	Hill climbing variant that increases step size when stuck to escape local optima.
gradient_free_optimizers.SimulatedAnnealingOptimizer(...)	Probabilistic optimizer inspired by the annealing process in metallurgy.
gradient_free_optimizers.DownhillSimplexOptimizer(...)	Derivative-free optimizer using geometric simplex transformations.
gradient_free_optimizers.RandomSearchOptimizer(...)	Simple optimizer that samples random positions from the search space.
gradient_free_optimizers.GridSearchOptimizer(...)	Exhaustive search optimizer that systematically evaluates a grid of points.
gradient_free_optimizers.RandomRestartHillClimbingOptimizer(...)	Hill climbing variant that periodically restarts from random positions.
gradient_free_optimizers.RandomAnnealingOptimizer(...)	Annealing optimizer that uses temperature to control the search radius.
gradient_free_optimizers.PatternSearch(...)	Direct search optimizer using geometric patterns around the current best point.
gradient_free_optimizers.PowellsMethod(...)	Derivative-free optimizer using sequential line searches along conjugate directions.
gradient_free_optimizers.LipschitzOptimizer(...)	Global optimizer using Lipschitz continuity bounds for deterministic search.
gradient_free_optimizers.DirectAlgorithm(...)	Deterministic global optimizer using adaptive hyperrectangle subdivision.
gradient_free_optimizers.ParticleSwarmOptimizer(...)	Swarm intelligence optimizer inspired by collective behavior of bird flocks.
gradient_free_optimizers.SpiralOptimization(...)	Population-based optimizer using spiral movement patterns toward the best solution.
gradient_free_optimizers.ParallelTemperingOptimizer(...)	Ensemble of simulated annealers at different temperatures with periodic swapping.
gradient_free_optimizers.GeneticAlgorithmOptimizer(...)	Evolutionary optimizer inspired by natural selection and genetics.
gradient_free_optimizers.EvolutionStrategyOptimizer(...)	Evolutionary optimizer focused on self-adaptive mutation for continuous domains.
gradient_free_optimizers.DifferentialEvolutionOptimizer(...)	Evolutionary optimizer using vector differences for mutation.
gradient_free_optimizers.CMAESOptimizer(...)	Evolutionary optimizer using covariance matrix adaptation.
gradient_free_optimizers.BayesianOptimizer(...)	Sequential model-based optimizer using Gaussian Process surrogate models.
gradient_free_optimizers.TreeStructuredParzenEstimators(...)	Sequential model-based optimizer using kernel density estimation.
gradient_free_optimizers.ForestOptimizer(...)	Sequential model-based optimizer using tree ensemble surrogate models.

Common Interface

All optimizers provide these methods and attributes:

Constructor

optimizer = OptimizerClass(
    search_space,           # dict: parameter name -> numpy array of values
    initialize=None,        # dict: initialization strategy
    constraints=[],         # list: constraint functions
    random_state=None,      # int: random seed
    rand_rest_p=0.0,        # float: random restart probability
    nth_process=0,          # int: process ID for parallel runs
)

search() Method

optimizer.search(
    objective_function,     # callable: params dict -> score
    n_iter,                 # int: number of iterations
    max_time=None,          # float: max seconds
    max_score=None,         # float: stop if score >= this
    early_stopping=None,    # dict: early stopping config
    memory=True,            # bool: cache evaluations
    memory_warm_start=None, # DataFrame: previous evaluations
    verbosity=["progress_bar", "print_results"],
    optimum="maximum",      # str: "maximum" or "minimum"
)

Result Attributes

optimizer.best_para    # dict: best parameters found
optimizer.best_score   # float: best score achieved
optimizer.search_data  # DataFrame: all evaluations

Initialization Options

The initialize parameter controls how the optimizer starts:

initialize = {
    "grid": 4,           # N positions on a grid
    "random": 2,         # N random positions
    "vertices": 4,       # N corner/edge positions
    "warm_start": [      # Specific starting positions
        {"x": 0.5, "y": 1.0},
    ],
}

Stopping Conditions

Multiple stopping conditions can be combined:

optimizer.search(
    objective,
    n_iter=1000,              # Max iterations
    max_time=3600,            # Max seconds
    max_score=0.99,           # Target score
    early_stopping={
        "n_iter_no_change": 50,  # Stop if no improvement
    },
)

Verbosity Options

Control output during search:

verbosity = []                    # Silent
verbosity = ["progress_bar"]      # Progress bar only
verbosity = ["print_results"]     # Print final results
verbosity = ["progress_bar", "print_results"]  # Both (default)