These Scilab files are implementations of the algorithms from the book "Iterative Methods for Optimization", to be published by SIAM, by C. T. Kelley. The book, which describes the algorithms, is available from SIAM (firstname.lastname@example.org).
This toolbox provides the following algorithms:
- optkelley_bfgswopt: Steepest descent/bfgs with polynomial line search.
- optkelley_cgtrust: Steihaug Newton-CG-Trust region algorithm.
- optkelley_diffhess: Compute a forward difference Hessian.
- optkelley_dirdero: Finite difference directional derivative.
- optkelley_gaussn: Damped Gauss-Newton with Armijo rule.
- optkelley_gradproj: Gradient projection with Armijo rule, simple linesearch
- optkelley_hooke: Hooke-Jeeves optimization.
- optkelley_imfil: Unconstrained implicit filtering.
- optkelley_levmar: Levenberg-Marquardt.
- optkelley_mds: Multidirectional search.
- optkelley_nelder: Nelder-Mead optimizer, No tie-breaking rule other than Scilab's gsort.
- optkelley_ntrust: Dogleg trust region.
- optkelley_polyline: Polynomial line search.
- optkelley_polymod: Cubic/quadratic polynomial linesearch.
- optkelley_projbfgs: Projected BFGS with Armijo rule, simple linesearch
- optkelley_simpgrad: Simplex gradient.
- optkelley_steep: Steepest descent with Armijo rule.
The optimization codes have the calling convention [f,g] = objective(x) returns both the objective function value f and the gradient vector g. I expect g to be a column vector. The Nelder-Mead, Hooke-Jeeves, Implicit Filtering, and MDS codes do not ask for a gradient.
- C. T. Kelley, 1999
- Yann Collette, 2008
- Michael Baudin, DIGITEO, 2010
These files can be modified for non-commercial purposes provided that the authors are acknowledged and clear comment lines are inserted that the code has been changed. The authors assume no responsibility for any errors that may exist in these routines.
These files have been ported into Scilab by Yann Collette in 2008.
In 2010, the update of the module to Scilab 5.2 has been performed by Michael Baudin, with formatting of the help pages and but fixes. The bug fixes include the optkelley_nelder function, with replacement of the call to sort with call to mtlb_sort. It also includes the replacement of sum with mtlb_sum. The functions are now consistently named. Several unit tests have been added, as well as demos. The error message use now mprintf, with the name of the routine first.