Open-source software · Journal of Open Source Software, 2026
RegularizedOptimization.jl: our methods, ready to use
An open-source Julia package that makes our algorithms for nonsmooth optimization available to students, researchers and engineers, behind a single interface.
An algorithm only becomes useful when people can run it. RegularizedOptimization.jl gathers, in a single Julia package, the trust-region and quadratic-regularization methods developed in our group for problems of the form \(f + h\). It is part of the JuliaSmoothOptimizers ecosystem: a problem is defined once and can then be solved with any of the methods, through the same interface.
What problems it solves
The package targets problems of the form
\[ \min_x \; f(x) + h(x) \quad \text{subject to} \quad c(x) = 0, \]where \(f\) and \(c\) are continuously differentiable and \(h\) is lower semicontinuous and possibly nonconvex. The term \(h\) can be a regularizer (sparsity, low rank), the indicator of a constraint set, or a combination of both.
What is inside
- Trust-region methods: TR and TRDH.
- Quadratic-regularization methods: R2, R2DH and R2N (see the R2N project).
- Levenberg–Marquardt methods for nonlinear least squares: LM and LMTR.
- An augmented Lagrangian method, AL, for problems with constraints.
These model-based methods typically need fewer evaluations of \(f\) and its gradient than first-order line-search methods, which matters when each evaluation is expensive.
How it fits in the ecosystem
- NLPModels.jl and ADNLPModels.jl define the smooth part (with automatic differentiation).
- ProximalOperators.jl and ShiftedProximalOperators.jl provide the nonsmooth part and its proximal operators.
- LinearOperators.jl provides quasi-Newton approximations such as LBFGS and LSR1.
- RegularizedProblems.jl provides ready-made test problems.
A short example
Training a support vector machine with a nonconvex ℓ1/2 penalty, using R2N with an LSR1 quasi-Newton model (example from the JOSS paper):
using LinearAlgebra, Random, ProximalOperators
using NLPModels, RegularizedProblems, RegularizedOptimization
model, nls_model, _ = RegularizedProblems.svm_train_model()
f = LSR1Model(model) # quasi-Newton model of the smooth part
h = RootNormLhalf(1.0) # nonconvex l_{1/2} regularizer
reg_nlp = RegularizedNLPModel(f, h)
solver = R2NSolver(reg_nlp)
stats = RegularizedExecutionStats(reg_nlp)
solve!(solver, reg_nlp, stats; atol = 1e-4, rtol = 1e-4)To install the package, type ] in the Julia REPL, then add RegularizedOptimization.
My role
I contribute actively to the package and to its companion libraries RegularizedProblems.jl, ShiftedProximalOperators.jl, OptimizationProblems.jl and LinearOperators.jl.