#include <itkLBFGSOptimizerv4.h>

Detailed Description

Wrap of the vnl_lbfgs algorithm for use in ITKv4 registration framework. The vnl_lbfgs is a wrapper for the NETLIB fortran code by Nocedal [1].

LBFGS is a quasi-Newton method. Quasi-Newton methods use an approximate estimate of the inverse Hessian \( (\nabla^2 f(x) )^{-1} \) to scale the gradient step:

\[ x_{n+1} = x_n - s (\nabla^2 f(x_n) )^{-1} \nabla f(x) \]

with \( s \) the step size.

The inverse Hessian is approximated from the gradients of previous iteration and thus only the gradient of the objective function is required.

The step size \( s \) is determined through line search with the approach by More and Thuente [4]. This line search approach finds a step size such that

\[ \lVert \nabla f(x + s (\nabla^2 f(x_n) )^{-1} \nabla f(x) ) \rVert \le \nu \lVert \nabla f(x) \rVert \]

The parameter \( \nu \) is set through SetLineSearchAccuracy() (default 0.9) The default step length, i.e. starting step length for the line search, is set through SetDefaultStepLength() (default 1.0).

The optimization stops when either the gradient satisfies the condition

\[ \lVert \nabla f(x) \rVert \le \epsilon \max(1, \lVert X \rVert) \]

or a maximum number of function evaluations has been reached. The tolerance \(\epsilon\) is set through SetGradientConvergenceTolerance() (default 1e-5) and the maximum number of function evaluations is set through SetMaximumNumberOfFunctionEvaluations() (default 2000).

Note: The scaling of the optimization parameters, set through SetScales(), should be set or left at one. Otherwise the Hessian approximation as well as the line search will be disturbed and the optimizer is unlikely to find a minima.

References:

[1] NETLIB lbfgs

[2] Jorge Nocedal. Updating Quasi-Newton Matrices with Limited Storage. Mathematics of Computation, Vol. 35, No. 151, pp. 773-782, 1980.

[3] Dong C. Liu and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming B, Vol. 45, No. 3, pp. 503-528, 1989.

[4] More, J. J. and D. J. Thuente. Line Search Algorithms with Guaranteed Sufficient Decrease. ACM Transactions on Mathematical Software 20, no. 3 (1994): 286-307.

Definition at line 87 of file itkLBFGSOptimizerv4.h.

Inheritance diagram for itk::LBFGSOptimizerv4:

Collaboration diagram for itk::LBFGSOptimizerv4:

Public Types
using	ConstPointer = SmartPointer< const Self >

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer< Self >

using	ScalesType = Superclass::ScalesType

using	Self = LBFGSOptimizerv4

using	Superclass = LBFGSOptimizerBasev4< vnl_lbfgs >

Public Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using	ConstPointer = SmartPointer< const Self >

using	InternalOptimizerType = LBFGSOptimizerBaseHelperv4< vnl_lbfgs >

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer< Self >

using	ScalesType = Superclass::ScalesType

using	Self = LBFGSOptimizerBasev4

using	StopConditionDescriptionType = Superclass::StopConditionDescriptionType

using	StopConditionReturnStringType = Superclass::StopConditionReturnStringType

using	Superclass = SingleValuedNonLinearVnlOptimizerv4

Public Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using	CommandType = ReceptorMemberCommand< Self >

using	ConstPointer = SmartPointer< const Self >

using	DerivativeType = Superclass::DerivativeType

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer< Self >

using	ScalesType = Superclass::ScalesType

using	Self = SingleValuedNonLinearVnlOptimizerv4

using	StopConditionDescriptionType = Superclass::StopConditionDescriptionType

using	StopConditionReturnStringType = Superclass::StopConditionReturnStringType

using	Superclass = ObjectToObjectOptimizerBase

Public Types inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
using	ConstPointer = SmartPointer< const Self >

using	DerivativeType = typename MetricType::DerivativeType

using	MeasureType = typename MetricType::MeasureType

using	MetricType = ObjectToObjectMetricBaseTemplate< double >

using	MetricTypePointer = typename MetricType::Pointer

using	NumberOfParametersType = typename MetricType::NumberOfParametersType

using	ParametersType = OptimizerParameters< double >

using	Pointer = SmartPointer< Self >

using	ScalesEstimatorType = OptimizerParameterScalesEstimatorTemplate< double >

using	ScalesType = OptimizerParameters< double >

using	Self = ObjectToObjectOptimizerBaseTemplate

using	StopConditionDescriptionType = std::ostringstream

using	StopConditionReturnStringType = std::string

using	Superclass = Object

Public Types inherited from itk::Object
using	ConstPointer = SmartPointer< const Self >

using	Pointer = SmartPointer< Self >

using	Self = Object

using	Superclass = LightObject

Public Types inherited from itk::LightObject
using	ConstPointer = SmartPointer< const Self >

using	Pointer = SmartPointer< Self >

using	Self = LightObject

Public Member Functions
virtual double	GetDefaultStepLength () const

virtual double	GetLineSearchAccuracy () const

const char *	GetNameOfClass () const override

void	SetDefaultStepLength (double f)

void	SetLineSearchAccuracy (double f)

void	SetMetric (MetricType *metric) override

void	StartOptimization (bool doOnlyInitialization=false) override

void	VerboseOff ()

void	VerboseOn ()

Public Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
virtual double	GetGradientConvergenceTolerance () const

virtual unsigned int	GetMaximumNumberOfFunctionEvaluations () const

const char *	GetNameOfClass () const override

InternalOptimizerType *	GetOptimizer ()

StopConditionReturnStringType	GetStopConditionDescription () const override

virtual bool	GetTrace () const

virtual void	SetGradientConvergenceTolerance (double f)

virtual void	SetMaximumNumberOfFunctionEvaluations (unsigned int n)

void	SetMetric (MetricType *metric) override

virtual void	SetTrace (bool flag)

void	StartOptimization (bool doOnlyInitialization=false) override

virtual void	TraceOn ()

Public Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
const char *	GetNameOfClass () const override

StopConditionReturnStringType	GetStopConditionDescription () const override=0

void	SetMetric (MetricType *metric) override=0

void	StartOptimization (bool doOnlyInitialization=false) override

virtual const DerivativeType &	GetCachedDerivative () const

virtual const ParametersType &	GetCachedCurrentPosition () const

Public Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
virtual bool	CanUseScales () const

virtual SizeValueType	GetCurrentIteration () const

virtual const MeasureType &	GetCurrentMetricValue () const

virtual const ParametersType &	GetCurrentPosition () const

const char *	GetNameOfClass () const override

virtual SizeValueType	GetNumberOfIterations () const

virtual const ThreadIdType &	GetNumberOfWorkUnits () const

virtual const ScalesType &	GetScales () const

virtual const bool &	GetScalesAreIdentity () const

bool	GetScalesInitialized () const

virtual StopConditionReturnStringType	GetStopConditionDescription () const=0

virtual const MeasureType &	GetValue () const

virtual const ScalesType &	GetWeights () const

virtual const bool &	GetWeightsAreIdentity () const

virtual void	SetNumberOfIterations (SizeValueType _arg)

virtual void	SetNumberOfWorkUnits (ThreadIdType number)

virtual void	SetScalesEstimator (ScalesEstimatorType *_arg)

virtual void	SetWeights (ScalesType _arg)

virtual void	StartOptimization (bool doOnlyInitialization=false)

virtual void	SetMetric (MetricType *_arg)

virtual MetricType *	GetModifiableMetric ()

virtual void	SetScales (const ScalesType &scales)

virtual void	SetDoEstimateScales (bool _arg)

virtual const bool &	GetDoEstimateScales () const

virtual void	DoEstimateScalesOn ()

Public Member Functions inherited from itk::Object
unsigned long	AddObserver (const EventObject &event, Command *cmd) const

unsigned long	AddObserver (const EventObject &event, std::function< void(const EventObject &)> function) const

LightObject::Pointer	CreateAnother () const override

virtual void	DebugOff () const

virtual void	DebugOn () const

Command *	GetCommand (unsigned long tag)

bool	GetDebug () const

MetaDataDictionary &	GetMetaDataDictionary ()

const MetaDataDictionary &	GetMetaDataDictionary () const

virtual ModifiedTimeType	GetMTime () const

const char *	GetNameOfClass () const override

virtual const TimeStamp &	GetTimeStamp () const

bool	HasObserver (const EventObject &event) const

void	InvokeEvent (const EventObject &)

void	InvokeEvent (const EventObject &) const

virtual void	Modified () const

void	Register () const override

void	RemoveAllObservers ()

void	RemoveObserver (unsigned long tag) const

void	SetDebug (bool debugFlag) const

void	SetReferenceCount (int) override

void	UnRegister () const noexcept override

void	SetMetaDataDictionary (const MetaDataDictionary &rhs)

void	SetMetaDataDictionary (MetaDataDictionary &&rrhs)

virtual void	SetObjectName (std::string _arg)

virtual const std::string &	GetObjectName () const

Public Member Functions inherited from itk::LightObject
Pointer	Clone () const

virtual Pointer	CreateAnother () const

virtual void	Delete ()

virtual const char *	GetNameOfClass () const

virtual int	GetReferenceCount () const

void	Print (std::ostream &os, Indent indent=0) const

virtual void	Register () const

virtual void	SetReferenceCount (int)

virtual void	UnRegister () const noexcept

Static Public Member Functions
static Pointer	New ()

Static Public Member Functions inherited from itk::Object
static bool	GetGlobalWarningDisplay ()

static void	GlobalWarningDisplayOff ()

static void	GlobalWarningDisplayOn ()

static Pointer	New ()

static void	SetGlobalWarningDisplay (bool val)

Static Public Member Functions inherited from itk::LightObject
static void	BreakOnError ()

static Pointer	New ()

Protected Types
using	InternalOptimizerType = vnl_lbfgs

using	InternalParametersType = vnl_vector< double >

Protected Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using	CostFunctionAdaptorType = Superclass::CostFunctionAdaptorType

using	InternalOptimizerAutoPointer = std::unique_ptr< InternalOptimizerType >

Protected Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using	CostFunctionAdaptorType = SingleValuedVnlCostFunctionAdaptorv4

Protected Member Functions
	LBFGSOptimizerv4 ()

void	PrintSelf (std::ostream &os, Indent indent) const override

	~LBFGSOptimizerv4 () override

Protected Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
	LBFGSOptimizerBasev4 ()

void	PrintSelf (std::ostream &os, Indent indent) const override

	~LBFGSOptimizerBasev4 () override=default

Protected Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
CostFunctionAdaptorType *	GetCostFunctionAdaptor ()

const CostFunctionAdaptorType *	GetCostFunctionAdaptor () const

CostFunctionAdaptorType *	GetNonConstCostFunctionAdaptor () const

void	PrintSelf (std::ostream &os, Indent indent) const override

void	SetCostFunctionAdaptor (CostFunctionAdaptorType *adaptor)

	SingleValuedNonLinearVnlOptimizerv4 ()

	~SingleValuedNonLinearVnlOptimizerv4 () override

Protected Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
void	PrintSelf (std::ostream &os, Indent indent) const override

	ObjectToObjectOptimizerBaseTemplate ()

	~ObjectToObjectOptimizerBaseTemplate () override

Protected Member Functions inherited from itk::Object
	Object ()

bool	PrintObservers (std::ostream &os, Indent indent) const

void	PrintSelf (std::ostream &os, Indent indent) const override

virtual void	SetTimeStamp (const TimeStamp &timeStamp)

	~Object () override

Protected Member Functions inherited from itk::LightObject
virtual LightObject::Pointer	InternalClone () const

	LightObject ()

virtual void	PrintHeader (std::ostream &os, Indent indent) const

virtual void	PrintSelf (std::ostream &os, Indent indent) const

virtual void	PrintTrailer (std::ostream &os, Indent indent) const

virtual	~LightObject ()

Private Attributes
double	m_DefaultStepLength { 1.0 }

double	m_LineSearchAccuracy { 0.9 }

bool	m_Verbose { false }

Additional Inherited Members
Protected Attributes inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
double	m_CostFunctionConvergenceFactor

double	m_GradientConvergenceTolerance

double	m_InfinityNormOfProjectedGradient

unsigned int	m_MaximumNumberOfFunctionEvaluations

bool	m_OptimizerInitialized

std::ostringstream	m_StopConditionDescription

bool	m_Trace

InternalOptimizerAutoPointer	m_VnlOptimizer

Protected Attributes inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
SizeValueType	m_CurrentIteration

MeasureType	m_CurrentMetricValue

bool	m_DoEstimateScales

MetricTypePointer	m_Metric

SizeValueType	m_NumberOfIterations

ThreadIdType	m_NumberOfWorkUnits

ScalesType	m_Scales

bool	m_ScalesAreIdentity

ScalesEstimatorType::Pointer	m_ScalesEstimator

ScalesType	m_Weights

bool	m_WeightsAreIdentity

Protected Attributes inherited from itk::LightObject
std::atomic< int >	m_ReferenceCount {}

Member Typedef Documentation

◆ ConstPointer

using itk::LBFGSOptimizerv4::ConstPointer = SmartPointer<const Self>

Definition at line 96 of file itkLBFGSOptimizerv4.h.

◆ InternalOptimizerType

using itk::LBFGSOptimizerv4::InternalOptimizerType = vnl_lbfgs

protected

Internal optimizer type.

Definition at line 151 of file itkLBFGSOptimizerv4.h.

◆ InternalParametersType

using itk::LBFGSOptimizerv4::InternalParametersType = vnl_vector<double>

protected

InternalParameters type alias.

Definition at line 148 of file itkLBFGSOptimizerv4.h.

◆ MetricType

using itk::LBFGSOptimizerv4::MetricType = Superclass::MetricType

Definition at line 98 of file itkLBFGSOptimizerv4.h.

◆ ParametersType

using itk::LBFGSOptimizerv4::ParametersType = Superclass::ParametersType

Definition at line 99 of file itkLBFGSOptimizerv4.h.

◆ Pointer

using itk::LBFGSOptimizerv4::Pointer = SmartPointer<Self>

Definition at line 95 of file itkLBFGSOptimizerv4.h.

◆ ScalesType

using itk::LBFGSOptimizerv4::ScalesType = Superclass::ScalesType

Definition at line 100 of file itkLBFGSOptimizerv4.h.

◆ Self

using itk::LBFGSOptimizerv4::Self = LBFGSOptimizerv4

Standard "Self" type alias.

Definition at line 93 of file itkLBFGSOptimizerv4.h.

◆ Superclass

using itk::LBFGSOptimizerv4::Superclass = LBFGSOptimizerBasev4<vnl_lbfgs>

Definition at line 94 of file itkLBFGSOptimizerv4.h.

Constructor & Destructor Documentation

◆ LBFGSOptimizerv4()

itk::LBFGSOptimizerv4::LBFGSOptimizerv4 ( )

protected

◆ ~LBFGSOptimizerv4()

itk::LBFGSOptimizerv4::~LBFGSOptimizerv4 ( )

overrideprotected

Member Function Documentation

◆ GetDefaultStepLength()

virtual double itk::LBFGSOptimizerv4::GetDefaultStepLength ( ) const

virtual

◆ GetLineSearchAccuracy()

virtual double itk::LBFGSOptimizerv4::GetLineSearchAccuracy ( ) const

virtual

◆ GetNameOfClass()

const char * itk::LBFGSOptimizerv4::GetNameOfClass ( ) const

overridevirtual

See also: LightObject::GetNameOfClass()

Reimplemented from itk::Object.

◆ New()

static Pointer itk::LBFGSOptimizerv4::New ( )

static

Method for creation through the object factory.

◆ PrintSelf()

void itk::LBFGSOptimizerv4::PrintSelf	(	std::ostream &	os,
		Indent	indent
	)		const

overrideprotectedvirtual

Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.

Reimplemented from itk::Object.

◆ SetDefaultStepLength()

void itk::LBFGSOptimizerv4::SetDefaultStepLength ( double f )

Set/Get the default step size. This is a positive real number with a default value of 1.0 which determines the step size in the line search.

◆ SetLineSearchAccuracy()

void itk::LBFGSOptimizerv4::SetLineSearchAccuracy ( double f )

Set/Get the line search accuracy. This is a positive real number with a default value of 0.9, which controls the accuracy of the line search. If the function and gradient evaluations are inexpensive with respect to the cost of the iterations it may be advantageous to set the value to a small value (say 0.1).

◆ SetMetric()

void itk::LBFGSOptimizerv4::SetMetric ( MetricType * metric )

overridevirtual

Plug in a Cost Function into the optimizer

Implements itk::SingleValuedNonLinearVnlOptimizerv4.

◆ StartOptimization()

void itk::LBFGSOptimizerv4::StartOptimization ( bool doOnlyInitialization = false )

overridevirtual

Start optimization with an initial value.

Reimplemented from itk::ObjectToObjectOptimizerBaseTemplate< double >.

◆ VerboseOff()

void itk::LBFGSOptimizerv4::VerboseOff ( )

◆ VerboseOn()

void itk::LBFGSOptimizerv4::VerboseOn ( )

Member Data Documentation

◆ m_DefaultStepLength

double itk::LBFGSOptimizerv4::m_DefaultStepLength { 1.0 }

private

Definition at line 156 of file itkLBFGSOptimizerv4.h.

◆ m_LineSearchAccuracy

double itk::LBFGSOptimizerv4::m_LineSearchAccuracy { 0.9 }

private

Definition at line 155 of file itkLBFGSOptimizerv4.h.

◆ m_Verbose

bool itk::LBFGSOptimizerv4::m_Verbose { false }

private

Definition at line 154 of file itkLBFGSOptimizerv4.h.

The documentation for this class was generated from the following file:

itkLBFGSOptimizerv4.h

Detailed Description

Public Types

Public Member Functions

Static Public Member Functions

Protected Types

Protected Member Functions

Private Attributes

Additional Inherited Members

Member Typedef Documentation

◆ ConstPointer

◆ InternalOptimizerType

◆ InternalParametersType

◆ MetricType

◆ ParametersType

◆ Pointer

◆ ScalesType

◆ Self

◆ Superclass

Constructor & Destructor Documentation

◆ LBFGSOptimizerv4()

◆ ~LBFGSOptimizerv4()

Member Function Documentation

◆ GetDefaultStepLength()

◆ GetLineSearchAccuracy()

◆ GetNameOfClass()

◆ New()

◆ PrintSelf()

◆ SetDefaultStepLength()

◆ SetLineSearchAccuracy()

◆ SetMetric()

◆ StartOptimization()

◆ VerboseOff()

◆ VerboseOn()

Member Data Documentation

◆ m_DefaultStepLength

◆ m_LineSearchAccuracy

◆ m_Verbose