ITK
6.0.0
Insight Toolkit
|
#include <itkLBFGSOptimizer.h>
Wrap of the vnl_lbfgs algorithm for use in ITKv4 registration framework. The vnl_lbfgs is a wrapper for the NETLIB fortran code by Nocedal [1].
LBFGS is a quasi-Newton method. Quasi-Newton methods use an approximate estimate of the inverse Hessian \( (\nabla^2 f(x) )^{-1} \) to scale the gradient step:
\[ x_{n+1} = x_n - s (\nabla^2 f(x_n) )^{-1} \nabla f(x) \]
with \( s \) the step size.
The inverse Hessian is approximated from the gradients of previous iteration and thus only the gradient of the objective function is required.
The step size \( s \) is determined through line search with the approach by More and Thuente [4]. This line search approach finds a step size such that
\[ \lVert \nabla f(x + s (\nabla^2 f(x_n) )^{-1} \nabla f(x) ) \rVert \le \nu \lVert \nabla f(x) \rVert \]
The parameter \( \nu \) is set through SetLineSearchAccuracy() (default 0.9) The default step length, i.e. starting step length for the line search, is set through SetDefaultStepLength() (default 1.0).
The optimization stops when either the gradient satisfies the condition
\[ \lVert \nabla f(x) \rVert \le \epsilon \max(1, \lVert X \rVert) \]
or a maximum number of function evaluations has been reached. The tolerance \(\epsilon\) is set through SetGradientConvergenceTolerance() (default 1e-5) and the maximum number of function evaluations is set through SetMaximumNumberOfFunctionEvaluations() (default 2000).
Note: The scales set through SetScales should be set or left at one. Otherwise the Hessian approximation will be disturbed and the optimizer is unlikely to find a minima.
References:
[1] NETLIB lbfgs
[2] Jorge Nocedal. Updating Quasi-Newton Matrices with Limited Storage. Mathematics of Computation, Vol. 35, No. 151, pp. 773-782, 1980.
[3] Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming B, Vol. 45, No. 3, pp. 503-528, 1989.
[4] More, J. J. and D. J. Thuente. Line Search Algorithms with Guaranteed Sufficient Decrease. ACM Transactions on Mathematical Software 20, no. 3 (1994): 286-307.
Definition at line 86 of file itkLBFGSOptimizer.h.
Public Member Functions | |
virtual double | GetDefaultStepLength () |
virtual double | GetGradientConvergenceTolerance () |
virtual double | GetLineSearchAccuracy () |
virtual unsigned int | GetMaximumNumberOfFunctionEvaluations () |
const char * | GetNameOfClass () const override |
vnl_lbfgs * | GetOptimizer () |
std::string | GetStopConditionDescription () const override |
virtual bool | GetTrace () |
MeasureType | GetValue () const |
void | SetCostFunction (SingleValuedCostFunction *costFunction) override |
virtual void | SetDefaultStepLength (double f) |
virtual void | SetGradientConvergenceTolerance (double f) |
virtual void | SetLineSearchAccuracy (double f) |
virtual void | SetMaximumNumberOfFunctionEvaluations (unsigned int n) |
virtual void | SetTrace (bool flag) |
void | StartOptimization () override |
virtual void | TraceOn () |
Public Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizer | |
virtual bool | CanUseScales () const |
const char * | GetNameOfClass () const override |
void | SetCostFunction (SingleValuedCostFunction *costFunction) override=0 |
virtual const bool & | GetMaximize () const |
virtual void | SetMaximize (bool _arg) |
virtual void | MaximizeOn () |
bool | GetMinimize () const |
void | SetMinimize (bool v) |
void | MinimizeOn () |
void | MinimizeOff () |
virtual const MeasureType & | GetCachedValue () const |
virtual const DerivativeType & | GetCachedDerivative () const |
virtual const ParametersType & | GetCachedCurrentPosition () const |
Public Member Functions inherited from itk::SingleValuedNonLinearOptimizer | |
virtual CostFunctionType * | GetModifiableCostFunction () |
const char * | GetNameOfClass () const override |
MeasureType | GetValue (const ParametersType ¶meters) const |
virtual void | SetCostFunction (CostFunctionType *costFunction) |
const char * | GetNameOfClass () const override |
Public Member Functions inherited from itk::Optimizer | |
virtual const ParametersType & | GetCurrentPosition () const |
virtual const ParametersType & | GetInitialPosition () const |
const char * | GetNameOfClass () const override |
virtual std::string | GetStopConditionDescription () const |
virtual void | SetInitialPosition (const ParametersType ¶m) |
void | SetScales (const ScalesType &scales) |
virtual void | StartOptimization () |
virtual const ScalesType & | GetScales () const |
virtual const ScalesType & | GetInverseScales () const |
Public Member Functions inherited from itk::Object | |
unsigned long | AddObserver (const EventObject &event, Command *cmd) const |
unsigned long | AddObserver (const EventObject &event, std::function< void(const EventObject &)> function) const |
LightObject::Pointer | CreateAnother () const override |
virtual void | DebugOff () const |
virtual void | DebugOn () const |
Command * | GetCommand (unsigned long tag) |
bool | GetDebug () const |
MetaDataDictionary & | GetMetaDataDictionary () |
const MetaDataDictionary & | GetMetaDataDictionary () const |
virtual ModifiedTimeType | GetMTime () const |
const char * | GetNameOfClass () const override |
virtual const TimeStamp & | GetTimeStamp () const |
bool | HasObserver (const EventObject &event) const |
void | InvokeEvent (const EventObject &) |
void | InvokeEvent (const EventObject &) const |
virtual void | Modified () const |
void | Register () const override |
void | RemoveAllObservers () |
void | RemoveObserver (unsigned long tag) const |
void | SetDebug (bool debugFlag) const |
void | SetReferenceCount (int) override |
void | UnRegister () const noexcept override |
void | SetMetaDataDictionary (const MetaDataDictionary &rhs) |
void | SetMetaDataDictionary (MetaDataDictionary &&rrhs) |
virtual void | SetObjectName (std::string _arg) |
virtual const std::string & | GetObjectName () const |
Public Member Functions inherited from itk::LightObject | |
Pointer | Clone () const |
virtual Pointer | CreateAnother () const |
virtual void | Delete () |
virtual const char * | GetNameOfClass () const |
virtual int | GetReferenceCount () const |
void | Print (std::ostream &os, Indent indent=0) const |
virtual void | Register () const |
virtual void | SetReferenceCount (int) |
virtual void | UnRegister () const noexcept |
Static Public Member Functions | |
static Pointer | New () |
Static Public Member Functions inherited from itk::SingleValuedNonLinearOptimizer | |
static Pointer | New () |
Static Public Member Functions inherited from itk::NonLinearOptimizer | |
static Pointer | New () |
Static Public Member Functions inherited from itk::Optimizer | |
static Pointer | New () |
Static Public Member Functions inherited from itk::Object | |
static bool | GetGlobalWarningDisplay () |
static void | GlobalWarningDisplayOff () |
static void | GlobalWarningDisplayOn () |
static Pointer | New () |
static void | SetGlobalWarningDisplay (bool val) |
Static Public Member Functions inherited from itk::LightObject | |
static void | BreakOnError () |
static Pointer | New () |
Protected Types | |
using | CostFunctionAdaptorType = Superclass::CostFunctionAdaptorType |
Protected Types inherited from itk::SingleValuedNonLinearVnlOptimizer | |
using | CostFunctionAdaptorType = SingleValuedVnlCostFunctionAdaptor |
Protected Member Functions | |
LBFGSOptimizer () | |
void | PrintSelf (std::ostream &os, Indent indent) const override |
~LBFGSOptimizer () override | |
Protected Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizer | |
CostFunctionAdaptorType * | GetCostFunctionAdaptor () |
const CostFunctionAdaptorType * | GetCostFunctionAdaptor () const |
CostFunctionAdaptorType * | GetNonConstCostFunctionAdaptor () const |
void | PrintSelf (std::ostream &os, Indent indent) const override |
void | SetCostFunctionAdaptor (CostFunctionAdaptorType *adaptor) |
SingleValuedNonLinearVnlOptimizer () | |
~SingleValuedNonLinearVnlOptimizer () override | |
Protected Member Functions inherited from itk::SingleValuedNonLinearOptimizer | |
void | PrintSelf (std::ostream &os, Indent indent) const override |
SingleValuedNonLinearOptimizer () | |
~SingleValuedNonLinearOptimizer () override=default | |
Protected Member Functions inherited from itk::NonLinearOptimizer | |
NonLinearOptimizer ()=default | |
~NonLinearOptimizer () override | |
Protected Member Functions inherited from itk::Optimizer | |
Optimizer () | |
void | PrintSelf (std::ostream &os, Indent indent) const override |
virtual void | SetCurrentPosition (const ParametersType ¶m) |
~Optimizer () override=default | |
Protected Member Functions inherited from itk::Object | |
Object () | |
bool | PrintObservers (std::ostream &os, Indent indent) const |
void | PrintSelf (std::ostream &os, Indent indent) const override |
virtual void | SetTimeStamp (const TimeStamp &timeStamp) |
~Object () override | |
Protected Member Functions inherited from itk::LightObject | |
virtual LightObject::Pointer | InternalClone () const |
LightObject () | |
virtual void | PrintHeader (std::ostream &os, Indent indent) const |
virtual void | PrintSelf (std::ostream &os, Indent indent) const |
virtual void | PrintTrailer (std::ostream &os, Indent indent) const |
virtual | ~LightObject () |
Private Attributes | |
double | m_DefaultStepLength {} |
double | m_GradientConvergenceTolerance {} |
double | m_LineSearchAccuracy {} |
unsigned int | m_MaximumNumberOfFunctionEvaluations {} |
bool | m_OptimizerInitialized {} |
std::ostringstream | m_StopConditionDescription {} |
bool | m_Trace {} |
std::unique_ptr< InternalOptimizerType > | m_VnlOptimizer |
Additional Inherited Members | |
Protected Attributes inherited from itk::SingleValuedNonLinearOptimizer | |
CostFunctionPointer | m_CostFunction {} |
Protected Attributes inherited from itk::Optimizer | |
ParametersType | m_CurrentPosition {} |
bool | m_ScalesInitialized { false } |
Protected Attributes inherited from itk::LightObject | |
std::atomic< int > | m_ReferenceCount {} |
using itk::LBFGSOptimizer::ConstPointer = SmartPointer<const Self> |
Definition at line 95 of file itkLBFGSOptimizer.h.
|
protected |
Definition at line 180 of file itkLBFGSOptimizer.h.
using itk::LBFGSOptimizer::InternalOptimizerType = vnl_lbfgs |
Internal optimizer type.
Definition at line 107 of file itkLBFGSOptimizer.h.
using itk::LBFGSOptimizer::InternalParametersType = vnl_vector<double> |
InternalParameters type alias.
Definition at line 104 of file itkLBFGSOptimizer.h.
using itk::LBFGSOptimizer::Pointer = SmartPointer<Self> |
Definition at line 94 of file itkLBFGSOptimizer.h.
Standard "Self" type alias.
Definition at line 92 of file itkLBFGSOptimizer.h.
Definition at line 93 of file itkLBFGSOptimizer.h.
|
protected |
|
overrideprotected |
|
virtual |
|
virtual |
|
virtual |
|
virtual |
|
overridevirtual |
Reimplemented from itk::Object.
vnl_lbfgs * itk::LBFGSOptimizer::GetOptimizer | ( | ) |
Method for getting access to the internal optimizer.
|
overridevirtual |
Get the reason for termination
Reimplemented from itk::Optimizer.
|
virtual |
MeasureType itk::LBFGSOptimizer::GetValue | ( | ) | const |
Return Current Value
|
static |
Method for creation through the object factory.
|
overrideprotectedvirtual |
Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.
Reimplemented from itk::Object.
|
overridevirtual |
Plug in a Cost Function into the optimizer
Implements itk::SingleValuedNonLinearVnlOptimizer.
|
virtual |
Set/Get the default step size. This is a positive real number with a default value of 1.0 which determines the step size in the line search.
|
virtual |
Set/Get the gradient convergence tolerance. This is a positive real number that determines the accuracy with which the solution is to be found. The optimization terminates when: ||G|| < gtol max(1,||X||) where ||.|| denotes the Euclidean norm.
|
virtual |
Set/Get the line search accuracy. This is a positive real number with a default value of 0.9, which controls the accuracy of the line search. If the function and gradient evaluations are inexpensive with respect to the cost of the iterations it may be advantageous to set the value to a small value (say 0.1).
|
virtual |
Set/Get the maximum number of function evaluations allowed.
|
virtual |
Set/Get the optimizer trace flag. If set to true, the optimizer prints out information every iteration.
|
overridevirtual |
Start optimization with an initial value.
Reimplemented from itk::Optimizer.
|
virtual |
|
private |
Definition at line 191 of file itkLBFGSOptimizer.h.
|
private |
Definition at line 189 of file itkLBFGSOptimizer.h.
|
private |
Definition at line 190 of file itkLBFGSOptimizer.h.
|
private |
Definition at line 188 of file itkLBFGSOptimizer.h.
|
private |
Definition at line 183 of file itkLBFGSOptimizer.h.
|
mutableprivate |
Definition at line 185 of file itkLBFGSOptimizer.h.
|
private |
Definition at line 187 of file itkLBFGSOptimizer.h.
|
private |
Definition at line 184 of file itkLBFGSOptimizer.h.