#include <itkLBFGSOptimizerv4.h>

Detailed Description

Wrap of the vnl_lbfgs algorithm for use in ITKv4 registration framework. The vnl_lbfgs is a wrapper for the NETLIB fortran code by Nocedal [1].

LBFGS is a quasi-Newton method. Quasi-Newton methods use an approximate estimate of the inverse Hessian \( (\nabla^2 f(x) )^{-1} \) to scale the gradient step:

\[x_{n+1} = x_n - s (\nabla^2 f(x_n) )^{-1} \nabla f(x) \]

with \( s \) the step size.

The inverse Hessian is approximated from the gradients of previous iteration and thus only the gradient of the objective function is required.

The step size \( s \) is determined through line search with the approach by More and Thuente [4]. This line search approach finds a step size such that

\[\| \nabla f(x + s (\nabla^2 f(x_n) )^{-1} \nabla f(x) ) \| \le \nu \| \nabla f(x) \| \]

The parameter \( \nu \) is set through SetLineSearchAccuracy() (default 0.9) The default step length, i.e. starting step length for the line search, is set through SetDefaultStepLength() (default 1.0).

The optimization stops when either the gradient satisfies the condition

\[\| \nabla f(x) \| \le \epsilon \max(1, \| X \|) \]

or a maximum number of function evaluations has been reached. The tolerance \(\epsilon\) is set through SetGradientConvergenceTolerance() (default 1e-5) and the maximum number of function evaluations is set through SetMaximumNumberOfFunctionEvaluations() (default 2000).

Note: The scaling of the optimization parameters, set through SetScales(), should be set or left at one. Otherwise the Hessian approximation as well as the line search will be disturbed and the optimizer is unlikely to find a minima.

For algorithmic details see NETLIB lbfgs, [82], [68] and [79].

Examples: Examples/RegistrationITKv4/DeformableRegistration4.cxx.

Definition at line 73 of file itkLBFGSOptimizerv4.h.

Inheritance diagram for itk::LBFGSOptimizerv4:

Collaboration diagram for itk::LBFGSOptimizerv4:

Public Types
using	ConstPointer = SmartPointer<const Self>

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer<Self>

using	ScalesType = Superclass::ScalesType

using	Self = LBFGSOptimizerv4

using	Superclass = LBFGSOptimizerBasev4<vnl_lbfgs>

Public Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using	ConstPointer

using	InternalOptimizerType

using	MetricType

using	ParametersType

using	Pointer

using	ScalesType

using	Self

using	StopConditionDescriptionType

using	StopConditionReturnStringType

using	Superclass

Public Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using	CommandType = ReceptorMemberCommand<Self>

using	ConstPointer = SmartPointer<const Self>

using	DerivativeType = Superclass::DerivativeType

using	MetricType = Superclass::MetricType

using	ParametersType = Superclass::ParametersType

using	Pointer = SmartPointer<Self>

using	ScalesType = Superclass::ScalesType

using	Self = SingleValuedNonLinearVnlOptimizerv4

using	StopConditionDescriptionType = Superclass::StopConditionDescriptionType

using	StopConditionReturnStringType = Superclass::StopConditionReturnStringType

using	Superclass = ObjectToObjectOptimizerBase

Public Types inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
using	ConstPointer

using	DerivativeType

using	MeasureType

using	MetricType

using	MetricTypePointer

using	NumberOfParametersType

using	ParametersType

using	Pointer

using	ScalesEstimatorType

using	ScalesType

using	Self

using	StopConditionDescriptionType

using	StopConditionReturnStringType

using	Superclass

Public Types inherited from itk::Object
using	ConstPointer = SmartPointer<const Self>

using	Pointer = SmartPointer<Self>

using	Self = Object

using	Superclass = LightObject

Public Types inherited from itk::LightObject
using	ConstPointer = SmartPointer<const Self>

using	Pointer = SmartPointer<Self>

using	Self = LightObject

Public Member Functions
virtual::itk::LightObject::Pointer	CreateAnother () const

virtual double	GetDefaultStepLength () const

virtual double	GetLineSearchAccuracy () const

const char *	GetNameOfClass () const override

void	SetDefaultStepLength (double f)

void	SetLineSearchAccuracy (double f)

void	SetMetric (MetricType *metric) override

void	StartOptimization (bool doOnlyInitialization=false) override

void	VerboseOff ()

void	VerboseOn ()

Public Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
virtual double	GetGradientConvergenceTolerance () const

virtual double	GetGradientConvergenceTolerance () const

virtual unsigned int	GetMaximumNumberOfFunctionEvaluations () const

virtual unsigned int	GetMaximumNumberOfFunctionEvaluations () const

InternalOptimizerType *	GetOptimizer ()

InternalOptimizerType *	GetOptimizer ()

StopConditionReturnStringType	GetStopConditionDescription () const override

StopConditionReturnStringType	GetStopConditionDescription () const override

virtual bool	GetTrace () const

virtual bool	GetTrace () const

virtual void	SetGradientConvergenceTolerance (double f)

virtual void	SetGradientConvergenceTolerance (double f)

virtual void	SetMaximumNumberOfFunctionEvaluations (unsigned int n)

virtual void	SetMaximumNumberOfFunctionEvaluations (unsigned int n)

virtual void	SetTrace (bool flag)

virtual void	SetTrace (bool flag)

virtual void	TraceOff ()

virtual void	TraceOff ()

virtual void	TraceOn ()

virtual void	TraceOn ()

Public Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
const char *	GetNameOfClass () const override

void	StartOptimization (bool doOnlyInitialization=false) override

virtual const DerivativeType &	GetCachedDerivative () const

virtual const ParametersType &	GetCachedCurrentPosition () const

Public Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
virtual bool	CanUseScales () const

virtual bool	CanUseScales () const

virtual SizeValueType	GetCurrentIteration () const

virtual SizeValueType	GetCurrentIteration () const

virtual const MeasureType &	GetCurrentMetricValue () const

virtual const MeasureType &	GetCurrentMetricValue () const

virtual const ParametersType &	GetCurrentPosition () const

virtual const ParametersType &	GetCurrentPosition () const

virtual SizeValueType	GetNumberOfIterations () const

virtual SizeValueType	GetNumberOfIterations () const

virtual const ThreadIdType &	GetNumberOfWorkUnits () const

virtual const ThreadIdType &	GetNumberOfWorkUnits () const

virtual const ScalesType &	GetScales () const

virtual const ScalesType &	GetScales () const

virtual const bool &	GetScalesAreIdentity () const

virtual const bool &	GetScalesAreIdentity () const

bool	GetScalesInitialized () const

bool	GetScalesInitialized () const

virtual const MeasureType &	GetValue () const

virtual const MeasureType &	GetValue () const

virtual const ScalesType &	GetWeights () const

virtual const ScalesType &	GetWeights () const

virtual const bool &	GetWeightsAreIdentity () const

virtual const bool &	GetWeightsAreIdentity () const

virtual void	SetNumberOfIterations (SizeValueType _arg)

virtual void	SetNumberOfIterations (SizeValueType _arg)

virtual void	SetNumberOfWorkUnits (ThreadIdType number)

virtual void	SetNumberOfWorkUnits (ThreadIdType number)

virtual void	SetScalesEstimator (ScalesEstimatorType *_arg)

virtual void	SetScalesEstimator (ScalesEstimatorType *_arg)

virtual void	SetWeights (ScalesType _arg)

virtual void	SetWeights (ScalesType _arg)

virtual MetricType *	GetModifiableMetric ()

virtual const MetricType *	GetMetric () const

virtual MetricType *	GetModifiableMetric ()

virtual const MetricType *	GetMetric () const

virtual void	SetScales (const ScalesType &scales)

virtual void	SetScales (const ScalesType &scales)

virtual void	SetDoEstimateScales (bool _arg)

virtual const bool &	GetDoEstimateScales () const

virtual void	DoEstimateScalesOn ()

virtual void	DoEstimateScalesOff ()

virtual void	SetDoEstimateScales (bool _arg)

virtual const bool &	GetDoEstimateScales () const

virtual void	DoEstimateScalesOn ()

virtual void	DoEstimateScalesOff ()

Public Member Functions inherited from itk::Object
unsigned long	AddObserver (const EventObject &event, Command *cmd) const

unsigned long	AddObserver (const EventObject &event, std::function< void(const EventObject &)> function) const

LightObject::Pointer	CreateAnother () const override

virtual void	DebugOff () const

virtual void	DebugOn () const

Command *	GetCommand (unsigned long tag)

bool	GetDebug () const

MetaDataDictionary &	GetMetaDataDictionary ()

const MetaDataDictionary &	GetMetaDataDictionary () const

virtual ModifiedTimeType	GetMTime () const

virtual const TimeStamp &	GetTimeStamp () const

bool	HasObserver (const EventObject &event) const

void	InvokeEvent (const EventObject &)

void	InvokeEvent (const EventObject &) const

virtual void	Modified () const

void	Register () const override

void	RemoveAllObservers ()

void	RemoveObserver (unsigned long tag) const

void	SetDebug (bool debugFlag) const

void	SetReferenceCount (int) override

void	UnRegister () const noexcept override

void	SetMetaDataDictionary (const MetaDataDictionary &rhs)

void	SetMetaDataDictionary (MetaDataDictionary &&rrhs)

virtual void	SetObjectName (std::string _arg)

virtual const std::string &	GetObjectName () const

Public Member Functions inherited from itk::LightObject
Pointer	Clone () const

virtual void	Delete ()

virtual int	GetReferenceCount () const

void	Print (std::ostream &os, Indent indent=0) const

Static Public Member Functions
static Pointer	New ()

Static Public Member Functions inherited from itk::Object
static bool	GetGlobalWarningDisplay ()

static void	GlobalWarningDisplayOff ()

static void	GlobalWarningDisplayOn ()

static Pointer	New ()

static void	SetGlobalWarningDisplay (bool val)

Static Public Member Functions inherited from itk::LightObject
static void	BreakOnError ()

static Pointer	New ()

Protected Types
using	InternalOptimizerType = vnl_lbfgs

using	InternalParametersType = vnl_vector<double>

Protected Types inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
using	CostFunctionAdaptorType

using	InternalOptimizerAutoPointer

Protected Types inherited from itk::SingleValuedNonLinearVnlOptimizerv4
using	CostFunctionAdaptorType = SingleValuedVnlCostFunctionAdaptorv4

Protected Member Functions
	LBFGSOptimizerv4 ()

void	PrintSelf (std::ostream &os, Indent indent) const override

	~LBFGSOptimizerv4 () override

Protected Member Functions inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
	LBFGSOptimizerBasev4 ()

	LBFGSOptimizerBasev4 ()

	~LBFGSOptimizerBasev4 () override=default

	~LBFGSOptimizerBasev4 () override=default

Protected Member Functions inherited from itk::SingleValuedNonLinearVnlOptimizerv4
CostFunctionAdaptorType *	GetCostFunctionAdaptor ()

const CostFunctionAdaptorType *	GetCostFunctionAdaptor () const

CostFunctionAdaptorType *	GetNonConstCostFunctionAdaptor () const

void	PrintSelf (std::ostream &os, Indent indent) const override

void	SetCostFunctionAdaptor (CostFunctionAdaptorType *adaptor)

	SingleValuedNonLinearVnlOptimizerv4 ()

	~SingleValuedNonLinearVnlOptimizerv4 () override

Protected Member Functions inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
	ObjectToObjectOptimizerBaseTemplate ()

	ObjectToObjectOptimizerBaseTemplate ()

	~ObjectToObjectOptimizerBaseTemplate () override

	~ObjectToObjectOptimizerBaseTemplate () override

Protected Member Functions inherited from itk::Object
	Object ()

bool	PrintObservers (std::ostream &os, Indent indent) const

virtual void	SetTimeStamp (const TimeStamp &timeStamp)

	~Object () override

Protected Member Functions inherited from itk::LightObject
virtual LightObject::Pointer	InternalClone () const

	LightObject ()

virtual void	PrintHeader (std::ostream &os, Indent indent) const

virtual void	PrintTrailer (std::ostream &os, Indent indent) const

virtual	~LightObject ()

Private Attributes
double	m_DefaultStepLength { 1.0 }

double	m_LineSearchAccuracy { 0.9 }

bool	m_Verbose { false }

Additional Inherited Members
Protected Attributes inherited from itk::LBFGSOptimizerBasev4< vnl_lbfgs >
double	m_CostFunctionConvergenceFactor

double	m_CostFunctionConvergenceFactor

double	m_GradientConvergenceTolerance

double	m_GradientConvergenceTolerance

double	m_InfinityNormOfProjectedGradient

double	m_InfinityNormOfProjectedGradient

unsigned int	m_MaximumNumberOfFunctionEvaluations

unsigned int	m_MaximumNumberOfFunctionEvaluations

bool	m_OptimizerInitialized

bool	m_OptimizerInitialized

std::ostringstream	m_StopConditionDescription

std::ostringstream	m_StopConditionDescription

bool	m_Trace

bool	m_Trace

InternalOptimizerAutoPointer	m_VnlOptimizer

InternalOptimizerAutoPointer	m_VnlOptimizer

Protected Attributes inherited from itk::ObjectToObjectOptimizerBaseTemplate< double >
SizeValueType	m_CurrentIteration

SizeValueType	m_CurrentIteration

MeasureType	m_CurrentMetricValue

MeasureType	m_CurrentMetricValue

bool	m_DoEstimateScales

bool	m_DoEstimateScales

MetricTypePointer	m_Metric

MetricTypePointer	m_Metric

SizeValueType	m_NumberOfIterations

SizeValueType	m_NumberOfIterations

ThreadIdType	m_NumberOfWorkUnits

ThreadIdType	m_NumberOfWorkUnits

ScalesType	m_Scales

ScalesType	m_Scales

bool	m_ScalesAreIdentity

bool	m_ScalesAreIdentity

ScalesEstimatorType::Pointer	m_ScalesEstimator

ScalesEstimatorType::Pointer	m_ScalesEstimator

ScalesType	m_Weights

ScalesType	m_Weights

bool	m_WeightsAreIdentity

bool	m_WeightsAreIdentity

Protected Attributes inherited from itk::LightObject
std::atomic< int >	m_ReferenceCount {}

Member Typedef Documentation

◆ ConstPointer

using itk::LBFGSOptimizerv4::ConstPointer = SmartPointer<const Self>

Definition at line 82 of file itkLBFGSOptimizerv4.h.

◆ InternalOptimizerType

using itk::LBFGSOptimizerv4::InternalOptimizerType = vnl_lbfgs

protected

Internal optimizer type.

Definition at line 137 of file itkLBFGSOptimizerv4.h.

◆ InternalParametersType

using itk::LBFGSOptimizerv4::InternalParametersType = vnl_vector<double>

protected

InternalParameters type alias.

Definition at line 134 of file itkLBFGSOptimizerv4.h.

◆ MetricType

using itk::LBFGSOptimizerv4::MetricType = Superclass::MetricType

Definition at line 84 of file itkLBFGSOptimizerv4.h.

◆ ParametersType

using itk::LBFGSOptimizerv4::ParametersType = Superclass::ParametersType

Definition at line 85 of file itkLBFGSOptimizerv4.h.

◆ Pointer

using itk::LBFGSOptimizerv4::Pointer = SmartPointer<Self>

Definition at line 81 of file itkLBFGSOptimizerv4.h.

◆ ScalesType

using itk::LBFGSOptimizerv4::ScalesType = Superclass::ScalesType

Definition at line 86 of file itkLBFGSOptimizerv4.h.

◆ Self

using itk::LBFGSOptimizerv4::Self = LBFGSOptimizerv4

Standard "Self" type alias.

Definition at line 79 of file itkLBFGSOptimizerv4.h.

◆ Superclass

using itk::LBFGSOptimizerv4::Superclass = LBFGSOptimizerBasev4<vnl_lbfgs>

Definition at line 80 of file itkLBFGSOptimizerv4.h.

Constructor & Destructor Documentation

◆ LBFGSOptimizerv4()

itk::LBFGSOptimizerv4::LBFGSOptimizerv4 ( )

protected

Referenced by GetNameOfClass().

◆ ~LBFGSOptimizerv4()

itk::LBFGSOptimizerv4::~LBFGSOptimizerv4 ( )

overrideprotected

Member Function Documentation

◆ CreateAnother()

virtual::itk::LightObject::Pointer itk::LBFGSOptimizerv4::CreateAnother ( ) const

virtual

Create an object from an instance, potentially deferring to a factory. This method allows you to create an instance of an object that is exactly the same type as the referring object. This is useful in cases where an object has been cast back to a base class.

Reimplemented from itk::LightObject.

◆ GetDefaultStepLength()

virtual double itk::LBFGSOptimizerv4::GetDefaultStepLength ( ) const

virtual

◆ GetLineSearchAccuracy()

virtual double itk::LBFGSOptimizerv4::GetLineSearchAccuracy ( ) const

virtual

◆ GetNameOfClass()

const char * itk::LBFGSOptimizerv4::GetNameOfClass ( ) const

overridevirtual

See also: LightObject::GetNameOfClass()

Reimplemented from itk::LBFGSOptimizerBasev4< vnl_lbfgs >.

References LBFGSOptimizerv4().

◆ New()

static Pointer itk::LBFGSOptimizerv4::New ( )

static

Method for creation through the object factory.

◆ PrintSelf()

void itk::LBFGSOptimizerv4::PrintSelf	(	std::ostream &	os,
		Indent	indent ) const

overrideprotectedvirtual

Methods invoked by Print() to print information about the object including superclasses. Typically not called by the user (use Print() instead) but used in the hierarchical print process to combine the output of several classes.

Reimplemented from itk::LBFGSOptimizerBasev4< vnl_lbfgs >.

◆ SetDefaultStepLength()

void itk::LBFGSOptimizerv4::SetDefaultStepLength ( double f )

Set/Get the default step size. This is a positive real number with a default value of 1.0 which determines the step size in the line search.

◆ SetLineSearchAccuracy()

void itk::LBFGSOptimizerv4::SetLineSearchAccuracy ( double f )

Set/Get the line search accuracy. This is a positive real number with a default value of 0.9, which controls the accuracy of the line search. If the function and gradient evaluations are inexpensive with respect to the cost of the iterations it may be advantageous to set the value to a small value (say 0.1).

◆ SetMetric()

void itk::LBFGSOptimizerv4::SetMetric ( MetricType * metric )

overridevirtual

Plug in a Cost Function into the optimizer

Reimplemented from itk::LBFGSOptimizerBasev4< vnl_lbfgs >.

◆ StartOptimization()

void itk::LBFGSOptimizerv4::StartOptimization ( bool doOnlyInitialization = false )

overridevirtual

Start optimization with an initial value.

Reimplemented from itk::LBFGSOptimizerBasev4< vnl_lbfgs >.

◆ VerboseOff()

void itk::LBFGSOptimizerv4::VerboseOff ( )

◆ VerboseOn()

void itk::LBFGSOptimizerv4::VerboseOn ( )

Member Data Documentation

◆ m_DefaultStepLength

double itk::LBFGSOptimizerv4::m_DefaultStepLength { 1.0 }

private

Definition at line 142 of file itkLBFGSOptimizerv4.h.

◆ m_LineSearchAccuracy

double itk::LBFGSOptimizerv4::m_LineSearchAccuracy { 0.9 }

private

Definition at line 141 of file itkLBFGSOptimizerv4.h.

◆ m_Verbose

bool itk::LBFGSOptimizerv4::m_Verbose { false }

private

Definition at line 140 of file itkLBFGSOptimizerv4.h.

The documentation for this class was generated from the following file:

itkLBFGSOptimizerv4.h

Detailed Description

Public Types

Public Member Functions

Static Public Member Functions

Protected Types

Protected Member Functions

Private Attributes

Additional Inherited Members

Member Typedef Documentation

◆ ConstPointer

◆ InternalOptimizerType

◆ InternalParametersType

◆ MetricType

◆ ParametersType

◆ Pointer

◆ ScalesType

◆ Self

◆ Superclass

Constructor & Destructor Documentation

◆ LBFGSOptimizerv4()

◆ ~LBFGSOptimizerv4()

Member Function Documentation

◆ CreateAnother()

◆ GetDefaultStepLength()

◆ GetLineSearchAccuracy()

◆ GetNameOfClass()

◆ New()

◆ PrintSelf()

◆ SetDefaultStepLength()

◆ SetLineSearchAccuracy()

◆ SetMetric()

◆ StartOptimization()

◆ VerboseOff()

◆ VerboseOn()

Member Data Documentation

◆ m_DefaultStepLength

◆ m_LineSearchAccuracy

◆ m_Verbose