template<typename TInternalComputationValueType>
class itk::LBFGS2Optimizerv4Template< TInternalComputationValueType >
Wrap of the libLBFGS algorithm for use in ITKv4 registration framework. LibLBFGS is a translation of LBFGS code by Nocedal NETLIB lbfgs and adds the orthantwise limited-memory Quasi-Newton method [3] for optimization with L1-norm on the parameters.
LBFGS is a quasi-Newton method uses an approximate estimate of the inverse Hessian \( (\nabla^2 f(x) )^-1 \) to scale the gradient step:
\[x_{n+1} = x_n - s (\nabla^2 f(x_n) )^-1 \nabla f(x)
\]
with \( s \) the step size.
The inverse Hessian is approximated from the gradients of previous iteration and thus only the gradient of the objective function is required.
The step size \( s \) is determined through line search which defaults to the approach by More and Thuente [85]. This line search approach finds a step size such that
\[\| \nabla f(x + s (\nabla^2 f(x_n) )^{-1} \nabla f(x) ) \|
\le
\nu \| \nabla f(x) \|
\]
The parameter \(\nu\) is set through SetLineSearchAccuracy() (default 0.9) and SetGradientLineSearchAccuracy()
Instead of the More-Tunete method, backtracking with three different conditions [33] are available and can be set through SetLineSearch():
- LINESEARCH_BACKTRACKING_ARMIJO
- LINESEARCH_BACKTRACKING_WOLFE
- LINESEARCH_BACKTRACKING_STRONG_WOLFE
The optimization stops when either the gradient satisfies the condition
\[\| \nabla f(x) \| \le \epsilon \max(1, \| X \|)
\]
or a maximum number of function evaluations has been reached. The tolerance \(\epsilon\) is set through SetSolutionAccuracy() (default 1e-5) and the maximum number of function evaluations is set through SetMaximumIterations() (default 0 = no maximum).
For algorithmic details see libLBFGS, NETLIB lbfgs, [3], [91], [72], [85], and [33].
Definition at line 144 of file itkLBFGS2Optimizerv4.h.
|
| virtual::itk::LightObject::Pointer | CreateAnother () const |
| |
| virtual PrecisionType | GetCurrentGradientNorm () const |
| |
| virtual PrecisionType | GetCurrentNumberOfEvaluations () const |
| |
| virtual PrecisionType | GetCurrentParameterNorm () const |
| |
| virtual PrecisionType | GetCurrentStepSize () const |
| |
| virtual const char * | GetNameOfClass () const |
| |
| StopConditionReturnStringType | GetStopConditionDescription () const override |
| |
| void | ResumeOptimization () override |
| |
| void | StartOptimization (bool doOnlyInitialization=false) override |
| |
|
| void | SetHessianApproximationAccuracy (int m) |
| |
| int | GetHessianApproximationAccuracy () const |
| |
|
| void | SetSolutionAccuracy (PrecisionType epsilon) |
| |
| PrecisionType | GetSolutionAccuracy () const |
| |
|
| void | SetDeltaConvergenceDistance (int nPast) |
| |
| int | GetDeltaConvergenceDistance () const |
| |
|
| void | SetDeltaConvergenceTolerance (PrecisionType tol) |
| |
| PrecisionType | GetDeltaConvergenceTolerance () const |
| |
|
| void | SetMaximumIterations (int maxIterations) |
| |
| int | GetMaximumIterations () const |
| |
|
| SizeValueType | GetNumberOfIterations () const override |
| |
| void | SetNumberOfIterations (const SizeValueType _arg) override |
| |
|
| void | SetLineSearch (const LineSearchMethodEnum &linesearch) |
| |
| LineSearchMethodEnum | GetLineSearch () const |
| |
|
| void | SetMaximumLineSearchEvaluations (int n) |
| |
| int | GetMaximumLineSearchEvaluations () const |
| |
|
| void | SetMinimumLineSearchStep (PrecisionType step) |
| |
| PrecisionType | GetMinimumLineSearchStep () const |
| |
|
| void | SetMaximumLineSearchStep (PrecisionType step) |
| |
| PrecisionType | GetMaximumLineSearchStep () const |
| |
|
| void | SetLineSearchAccuracy (PrecisionType ftol) |
| |
| PrecisionType | GetLineSearchAccuracy () const |
| |
|
| void | SetWolfeCoefficient (PrecisionType wc) |
| |
| PrecisionType | GetWolfeCoefficient () const |
| |
|
| void | SetLineSearchGradientAccuracy (PrecisionType gtol) |
| |
| PrecisionType | GetLineSearchGradientAccuracy () const |
| |
|
| void | SetMachinePrecisionTolerance (PrecisionType xtol) |
| |
| PrecisionType | GetMachinePrecisionTolerance () const |
| |
|
| void | SetOrthantwiseCoefficient (PrecisionType orthant_c) |
| |
| PrecisionType | GetOrthantwiseCoefficient () const |
| |
|
| void | SetOrthantwiseStart (int start) |
| |
| int | GetOrthantwiseStart () const |
| |
|
| void | SetOrthantwiseEnd (int end) |
| |
| int | GetOrthantwiseEnd () const |
| |
|
| virtual void | SetEstimateScalesAtEachIteration (bool _arg) |
| |
| virtual const bool & | GetEstimateScalesAtEachIteration () const |
| |
| virtual void | EstimateScalesAtEachIterationOn () |
| |
| virtual void | EstimateScalesAtEachIterationOff () |
| |
| virtual void | EstimateLearningRate () |
| |
| | itkVirtualGetConstReferenceMacro (ConvergenceValue, TInternalComputationValueType) |
| |
| | itkVirtualSetMacro (ConvergenceWindowSize, SizeValueType) |
| |
| | itkVirtualSetMacro (MinimumConvergenceValue, TInternalComputationValueType) |
| |
| void | StopOptimization () override |
| |
| virtual void | SetLearningRate (TInternalComputationValueType _arg) |
| |
| virtual const TInternalComputationValueType & | GetLearningRate () const |
| |
| virtual void | SetMaximumStepSizeInPhysicalUnits (TInternalComputationValueType _arg) |
| |
| virtual const TInternalComputationValueType & | GetMaximumStepSizeInPhysicalUnits () const |
| |
| virtual void | SetDoEstimateLearningRateAtEachIteration (bool _arg) |
| |
| virtual const bool & | GetDoEstimateLearningRateAtEachIteration () const |
| |
| virtual void | DoEstimateLearningRateAtEachIterationOn () |
| |
| virtual void | DoEstimateLearningRateAtEachIterationOff () |
| |
| virtual void | SetDoEstimateLearningRateOnce (bool _arg) |
| |
| virtual const bool & | GetDoEstimateLearningRateOnce () const |
| |
| virtual void | DoEstimateLearningRateOnceOn () |
| |
| virtual void | DoEstimateLearningRateOnceOff () |
| |
| virtual void | SetReturnBestParametersAndValue (bool _arg) |
| |
| virtual const bool & | GetReturnBestParametersAndValue () const |
| |
| virtual void | ReturnBestParametersAndValueOn () |
| |
| virtual void | ReturnBestParametersAndValueOff () |
| |
| virtual const DerivativeType & | GetGradient () const |
| |
| | itkVirtualGetConstReferenceMacro (StopCondition, StopConditionObjectToObjectOptimizerEnum) |
| |
| virtual void | ModifyGradientByScales () |
| |
| virtual void | ModifyGradientByLearningRate () |
| |
| virtual bool | CanUseScales () const |
| |
| virtual bool | CanUseScales () const |
| |
| virtual SizeValueType | GetCurrentIteration () const |
| |
| virtual SizeValueType | GetCurrentIteration () const |
| |
| virtual const MeasureType & | GetCurrentMetricValue () const |
| |
| virtual const MeasureType & | GetCurrentMetricValue () const |
| |
| virtual const ParametersType & | GetCurrentPosition () const |
| |
| virtual const ParametersType & | GetCurrentPosition () const |
| |
| virtual const ThreadIdType & | GetNumberOfWorkUnits () const |
| |
| virtual const ThreadIdType & | GetNumberOfWorkUnits () const |
| |
| virtual const ScalesType & | GetScales () const |
| |
| virtual const ScalesType & | GetScales () const |
| |
| virtual const bool & | GetScalesAreIdentity () const |
| |
| virtual const bool & | GetScalesAreIdentity () const |
| |
| bool | GetScalesInitialized () const |
| |
| bool | GetScalesInitialized () const |
| |
| virtual const MeasureType & | GetValue () const |
| |
| virtual const MeasureType & | GetValue () const |
| |
| virtual const ScalesType & | GetWeights () const |
| |
| virtual const ScalesType & | GetWeights () const |
| |
| virtual const bool & | GetWeightsAreIdentity () const |
| |
| virtual const bool & | GetWeightsAreIdentity () const |
| |
| | itkVirtualGetConstMacro (NumberOfIterations, SizeValueType) |
| |
| | itkVirtualGetConstMacro (NumberOfIterations, SizeValueType) |
| |
| | itkVirtualSetMacro (NumberOfIterations, SizeValueType) |
| |
| | itkVirtualSetMacro (NumberOfIterations, SizeValueType) |
| |
| virtual void | SetNumberOfWorkUnits (ThreadIdType number) |
| |
| virtual void | SetNumberOfWorkUnits (ThreadIdType number) |
| |
| virtual void | SetScales (const ScalesType &scales) |
| |
| virtual void | SetScales (const ScalesType &scales) |
| |
| virtual void | SetScalesEstimator (ScalesEstimatorType *_arg) |
| |
| virtual void | SetScalesEstimator (ScalesEstimatorType *_arg) |
| |
| virtual void | SetWeights (ScalesType _arg) |
| |
| virtual void | SetWeights (ScalesType _arg) |
| |
| | itkVirtualSetObjectMacro (Metric, MetricType) |
| |
| virtual MetricType * | GetModifiableMetric () |
| |
| virtual const MetricType * | GetMetric () const |
| |
| | itkVirtualSetObjectMacro (Metric, MetricType) |
| |
| virtual MetricType * | GetModifiableMetric () |
| |
| virtual const MetricType * | GetMetric () const |
| |
| virtual void | SetDoEstimateScales (bool _arg) |
| |
| virtual const bool & | GetDoEstimateScales () const |
| |
| virtual void | DoEstimateScalesOn () |
| |
| virtual void | DoEstimateScalesOff () |
| |
| virtual void | SetDoEstimateScales (bool _arg) |
| |
| virtual const bool & | GetDoEstimateScales () const |
| |
| virtual void | DoEstimateScalesOn () |
| |
| virtual void | DoEstimateScalesOff () |
| |
| unsigned long | AddObserver (const EventObject &event, Command *cmd) const |
| |
| unsigned long | AddObserver (const EventObject &event, std::function< void(const EventObject &)> function) const |
| |
| LightObject::Pointer | CreateAnother () const override |
| |
| virtual void | DebugOff () const |
| |
| virtual void | DebugOn () const |
| |
| Command * | GetCommand (unsigned long tag) |
| |
| bool | GetDebug () const |
| |
| MetaDataDictionary & | GetMetaDataDictionary () |
| |
| const MetaDataDictionary & | GetMetaDataDictionary () const |
| |
| virtual ModifiedTimeType | GetMTime () const |
| |
| virtual const TimeStamp & | GetTimeStamp () const |
| |
| bool | HasObserver (const EventObject &event) const |
| |
| void | InvokeEvent (const EventObject &) |
| |
| void | InvokeEvent (const EventObject &) const |
| |
| virtual void | Modified () const |
| |
| void | Register () const override |
| |
| void | RemoveAllObservers () |
| |
| void | RemoveObserver (unsigned long tag) const |
| |
| void | SetDebug (bool debugFlag) const |
| |
| void | SetReferenceCount (int) override |
| |
| void | UnRegister () const noexcept override |
| |
| void | SetMetaDataDictionary (const MetaDataDictionary &rhs) |
| |
| void | SetMetaDataDictionary (MetaDataDictionary &&rrhs) |
| |
| virtual void | SetObjectName (std::string _arg) |
| |
| virtual const std::string & | GetObjectName () const |
| |
| Pointer | Clone () const |
| |
| virtual void | Delete () |
| |
| virtual int | GetReferenceCount () const |
| |
| void | Print (std::ostream &os, Indent indent=0) const |
| |
|
| PrecisionType | EvaluateCost (const PrecisionType *x, PrecisionType *g, const int n, const PrecisionType step) |
| |
| | LBFGS2Optimizerv4Template () |
| |
| void | PrintSelf (std::ostream &os, Indent indent) const override |
| |
| int | UpdateProgress (const PrecisionType *x, const PrecisionType *g, const PrecisionType fx, const PrecisionType xnorm, const PrecisionType gnorm, const PrecisionType step, int n, int k, int ls) |
| |
| | ~LBFGS2Optimizerv4Template () override |
| |
| | GradientDescentOptimizerv4Template () |
| |
| void | ModifyGradientByLearningRateOverSubRange (const IndexRangeType &subrange) override |
| |
| void | ModifyGradientByScalesOverSubRange (const IndexRangeType &subrange) override |
| |
| | ~GradientDescentOptimizerv4Template () override=default |
| |
| | GradientDescentOptimizerBasev4Template () |
| |
| | ~GradientDescentOptimizerBasev4Template () override=default |
| |
| | ObjectToObjectOptimizerBaseTemplate () |
| |
| | ObjectToObjectOptimizerBaseTemplate () |
| |
| void | PrintSelf (std::ostream &os, Indent indent) const override |
| |
| void | PrintSelf (std::ostream &os, Indent indent) const override |
| |
| | ~ObjectToObjectOptimizerBaseTemplate () override |
| |
| | ~ObjectToObjectOptimizerBaseTemplate () override |
| |
| | Object () |
| |
| bool | PrintObservers (std::ostream &os, Indent indent) const |
| |
| virtual void | SetTimeStamp (const TimeStamp &timeStamp) |
| |
| | ~Object () override |
| |
| virtual LightObject::Pointer | InternalClone () const |
| |
| | LightObject () |
| |
| virtual void | PrintHeader (std::ostream &os, Indent indent) const |
| |
| virtual void | PrintTrailer (std::ostream &os, Indent indent) const |
| |
| virtual | ~LightObject () |
| |
template<typename TInternalComputationValueType>
Start index for computing L1 norm of the variables. This parameter is valid only for OWL-QN method (i.e., \( orthantwise_c != 0 \)). This parameter b (0 <= b < N) specifies the index number from which the library computes the L1 norm of the variables x,
\[ |x| := |x_{b}| + |x_{b+1}| + ... + |x_{N}| .
\]
In other words, variables \(x_1, ..., x_{b-1}\) are not used for computing the L1 norm. Setting b, (0 < b < N), one can protect variables, \(x_1, ..., x_{b-1}\) (e.g., a bias term of logistic regression) from being regularized. The default value is zero.
template<typename TInternalComputationValueType>
Start index for computing L1 norm of the variables. This parameter is valid only for OWL-QN method (i.e., \( orthantwise_c != 0 \)). This parameter b (0 <= b < N) specifies the index number from which the library computes the L1 norm of the variables x,
\[ |x| := |x_{b}| + |x_{b+1}| + ... + |x_{N}| .
\]
In other words, variables \(x_1, ..., x_{b-1}\) are not used for computing the L1 norm. Setting b, (0 < b < N), one can protect variables, \(x_1, ..., x_{b-1}\) (e.g., a bias term of logistic regression) from being regularized. The default value is zero.
Inheritance diagram for itk::LBFGS2Optimizerv4Template< TInternalComputationValueType >:
1.13.2