User Guide
Master gradient-free optimization with GFO. This guide covers core concepts, algorithm selection, and advanced features for efficient black-box optimization.
Tip
New to GFO? Follow this path through the guide:
Search Spaces – learn how to define parameter ranges
Objective Functions – write functions to optimize
Optimizer Selection – choose the right algorithm for your problem
How GFO Works
Gradient-Free-Optimizers provides a simple pattern: define your search space, choose an optimizer, and run. All algorithms share the same interface, making it easy to experiment with different approaches.
# The GFO optimization pattern
from gradient_free_optimizers import BayesianOptimizer
import numpy as np
# 1. Define where to search
search_space = {
"x": np.linspace(-10, 10, 100),
"y": np.linspace(-10, 10, 100),
}
# 2. Choose how to optimize
opt = BayesianOptimizer(search_space)
# 3. Run optimization
opt.search(objective_function, n_iter=100)
# 4. Get results
print(opt.best_para) # Best parameters
print(opt.best_score) # Best score achieved
Core Concepts
Every optimization in GFO involves three components:
Where to search
Parameter ranges defined as NumPy arrays. Supports continuous, discrete, and categorical parameters.
How to optimize
The algorithm that explores your search space. Choose from 22 algorithms across local search, global search, population methods, and SMBO.
What to optimize
Your function that takes parameters and returns a score to maximize.
Guide Sections
Define parameter ranges with NumPy arrays. Start here for the fundamentals.
Writing functions to optimize and handling expensive evaluations.
Choosing the right algorithm for your problem type.
Restrict the search space with constraint functions.
Control how the search starts: grid, random, or warm-start.
Cache evaluations and warm-start from previous runs.
Stop based on iterations, time, or early stopping.
Using the simple .search()
method.
Algorithms
GFO provides 22 optimization algorithms organized into four categories:
5 algorithms for exploiting promising regions
Hill Climbing variants, Simulated Annealing, and Downhill Simplex.
8 algorithms for broad exploration
Random Search, Grid Search, Pattern Search, DIRECT, and more.
6 algorithms using collective intelligence
Particle Swarm, Genetic Algorithm, Evolution Strategy, Differential Evolution.
3 algorithms that learn from evaluations
Bayesian Optimization, TPE, and Forest Optimizer.