Progress Reporting and Command Observers

SimpleITK Filters and other classes derived from ProcessObjects have the ability for user code to be executed when certain events occur. This is known as the Command and Observer design patters to implement user callbacks. This allows for the monitoring and abortion of processes as they are being executed.

Consider the following image source which takes a few seconds to execute. It would be nice to quickly know how long your going to need to wait, to know if you can go get a cup of coffee.

In [1]:
from __future__ import print_function
%matplotlib inline
import matplotlib.pyplot as plt
import SimpleITK as sitk
print(sitk.Version())

import sys
import os
import threading

from myshow import myshow
from myshow import myshow3d

SimpleITK Version: 1.2.4 (ITK 4.13)
Compiled: Nov 12 2019 23:05:07
In [2]:
size=256 # if this is too fast increase the size
img = sitk.GaborSource(sitk.sitkFloat32, size=[size]*3, sigma=[size*.2]*3, mean=[size*0.5]*3, frequency=.1)
myshow3d(img,zslices=[int(size/2)],dpi=40);
In [3]:
myshow(img);

We need to add a command to display the progress reported by the ProcessObject::GetProgress method during the sitkProgressEvent. This involves three components:

  1. Events
  2. ProcessObject's methods

  3. Commands

    We'll look at some examples after a brief explanation of these components.

Events

The avaiable events to observed are defined in a namespace enumeration.

sitkAnyEventOccurs for all event types.
sitkAbortEventOccurs after the process has been aborted, but before exiting the Execute method.
sitkDeleteEventOccurs when the underlying itk::ProcessObject is deleted.
sitkEndEventOccurs at then end of normal processing.
sitkIterationEventOccurs with some algorithms that run for a fixed or undetermined number of iterations.
sitkProgressEventOccurs when the progress changes in most process objects.
sitkStartEventOccurs when then itk::ProcessObject is starting.
sitkUserEventOther events may fall into this enumeration.

The convention of pre-fixing enums with "sitk" is continued, although it's getting a little crowded.

C++ is more strongly typed than Python it allows for implicit conversion from an enum type to an int, but not from an int to an enum type. Care needs to be made to ensure the correct enum value is passed in Python.

ProcessObject's methods

To be able to interface with the ProcessObject during execution, the object-oriented interface must be used to access the method of the ProcessObject. While any constant member function can be called during a command call-back there are two common methods:

  1. ProcessObject::GetProgress()
  2. ProcessObject::Abort()

The methods are only valid during the Command while a process is being executed, or when the process is not in the Execute method.

Additionally it should be noted that follow methods can not be called during a command or from another thread during execution Execute and RemoveAllCommands. In general the ProcessObject should not be modified during execution.

Commands

The command design pattern is used to allow user code to be executed when an event occurs. It is implemented in the Command class. The Command class provides an Execute method to be overridden in derived classes.

There are three ways to define a command with SimpleITK in Python.

  1. Derive from the Command class.
  2. Use the PyCommand class' SetCallbackPyCallable method.
  3. Use an inline lambda function in ProcessOject::AddCommand.
In [4]:
help(sitk.Command)

Help on class Command in module SimpleITK.SimpleITK:

class Command(builtins.object)
 |  An implementation of the Command design pattern for callback.
 |  
 |  
 |  This class provides a callback mechanism for event that occur from the ProcessObject. These commands can be utilized to observe these events.
 |  
 |  The Command can be created on the stack, and will automatically unregistered it's
 |  self when destroyed.
 |  
 |  For more information see the page Commands and Events for SimpleITK.
 |  
 |  C++ includes: sitkCommand.h
 |  
 |  Methods defined here:
 |  
 |  Execute(self)
 |      Execute(Command self)
 |      
 |      
 |      
 |      The method that defines action to be taken by the command
 |  
 |  GetName(self)
 |      GetName(Command self) -> std::string
 |      
 |      
 |      
 |      Set/Get Command Name
 |  
 |  SetName(self, name)
 |      SetName(Command self, std::string const & name)
 |  
 |  __del__ lambda self
 |  
 |  __disown__(self)
 |  
 |  __getattr__ lambda self, name
 |  
 |  __init__(self)
 |      __init__(itk::simple::Command self) -> Command
 |      
 |      
 |      
 |      Default Constructor.
 |  
 |  __repr__ = _swig_repr(self)
 |  
 |  __setattr__ lambda self, name, value
 |  
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  
 |  __swig_destroy__ = delete_Command(...)
 |      delete_Command(Command self)
 |      
 |      
 |      
 |      Destructor.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  __swig_getmethods__ = {}
 |  
 |  __swig_setmethods__ = {}
In [5]:
class MyCommand(sitk.Command):
    def __init__(self):
        # required
        super(MyCommand,self).__init__()

    def Execute(self):
        print("MyCommand::Execute Called")
        
cmd = MyCommand()
cmd.Execute()

MyCommand::Execute Called
In [6]:
help(sitk.PyCommand)

Help on class PyCommand in module SimpleITK.SimpleITK:

class PyCommand(Command)
 |  Proxy of C++ itk::simple::PyCommand class.
 |  
 |  Method resolution order:
 |      PyCommand
 |      Command
 |      builtins.object
 |  
 |  Methods defined here:
 |  
 |  Execute(self)
 |      Execute(PyCommand self)
 |      
 |      
 |      
 |      The method that defines action to be taken by the command
 |  
 |  GetCallbackPyCallable(self)
 |      GetCallbackPyCallable(PyCommand self) -> PyObject *
 |  
 |  SetCallbackPyCallable(self, obj)
 |      SetCallbackPyCallable(PyCommand self, PyObject * obj)
 |  
 |  __del__ lambda self
 |  
 |  __getattr__ lambda self, name
 |  
 |  __init__(self)
 |      __init__(itk::simple::PyCommand self) -> PyCommand
 |  
 |  __repr__ = _swig_repr(self)
 |  
 |  __setattr__ lambda self, name, value
 |  
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  
 |  __swig_destroy__ = delete_PyCommand(...)
 |      delete_PyCommand(PyCommand self)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  __swig_getmethods__ = {}
 |  
 |  __swig_setmethods__ = {}
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from Command:
 |  
 |  GetName(self)
 |      GetName(Command self) -> std::string
 |      
 |      
 |      
 |      Set/Get Command Name
 |  
 |  SetName(self, name)
 |      SetName(Command self, std::string const & name)
 |  
 |  __disown__(self)
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from Command:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
In [7]:
cmd = sitk.PyCommand()
cmd.SetCallbackPyCallable( lambda: print("PyCommand Called") )
cmd.Execute()

PyCommand Called

Back to watching the progress of out Gabor image source. First lets create the filter as an object

In [8]:
size=256
filter = sitk.GaborImageSource()
filter.SetOutputPixelType(sitk.sitkFloat32)
filter.SetSize([size]*3)
filter.SetSigma([size*.2]*3)
filter.SetMean([size*0.5]*3)
filter.SetFrequency(.1)
img = filter.Execute()
myshow3d(img,zslices=[int(size/2)],dpi=40);

The ProcessObject interface for the Invoker or Subject

SimpleITK doesn't have a large heirachy of inheritance. It has been kept to a minimal, so there is no common Object or LightObject base class as ITK has. As most of the goals for the events have to do with observing processes, the "Subject" interface of the Observer patter or the "Invoker" part of the Command design pattern, has been added to a ProcessObject base class for filters.

The ProcessObject base class has the following methods of handling commands: AddCommand, RemoveAllCommands, and HasCommand.

Adding these functionalities are not available in the procedural interface available for SimpleITK. They are only available through the Object Oriented interface, and break the method chaining interface.

In [9]:
help(sitk.ProcessObject)

Help on class ProcessObject in module SimpleITK.SimpleITK:

class ProcessObject(builtins.object)
 |  ProcessObject(*args, **kwargs)
 |  
 |  Base class for SimpleITK classes based on ProcessObject.
 |  
 |  C++ includes: sitkProcessObject.h
 |  
 |  Methods defined here:
 |  
 |  Abort(self)
 |      Abort(ProcessObject self)
 |      
 |      
 |      
 |      Sets an abort flag on the active process.
 |      
 |      Requests the current active process to abort. Additional, progress or
 |      iteration event may occur. If aborted then, an AbortEvent should
 |      occur. The Progress should be set to 1.0 after aborting.
 |      
 |      The expected behavior is that not exception should be throw out of
 |      this processes Execute method. Additionally, the results returned are
 |      valid but undefined content. The content may be only partially
 |      updated, uninitialized or the a of size zero.
 |      
 |      If there is no active process the method has no effect.
 |  
 |  AddCommand(self, *args)
 |      AddCommand(ProcessObject self, itk::simple::EventEnum event, Command cmd) -> int
 |      AddCommand(ProcessObject self, itk::simple::EventEnum e, PyObject * obj) -> int
 |      
 |      
 |      
 |      Add a Command Object to observer the event.
 |      
 |      
 |      The Command object's Execute method will be invoked when the internal ITK Object has the event. These events only occur during this ProcessObject's Execute method when the ITK filter is running. The command occurs
 |      in the same thread as this objects Execute methods was called in.
 |      
 |      An internal reference is made between the Command and this ProcessObject which enable automatic removal of the command when deleted. This
 |      enables both object to exist as stack based object and be
 |      automatically cleaned up.
 |      
 |      Unless specified otherwise, it's safe to get any value during
 |      execution. "Measurements" will have valid values only after the
 |      Execute method has returned. "Active Measurements" will have valid
 |      values during events, and access the underlying ITK object.
 |      
 |      Deleting a command this object has during a command call-back will
 |      produce undefined behavior.
 |      
 |      For more information see the page Commands and Events for SimpleITK.
 |      
 |      
 |      The return value is reserved for latter usage.
 |  
 |  DebugOff(self)
 |      DebugOff(ProcessObject self)
 |  
 |  DebugOn(self)
 |      DebugOn(ProcessObject self)
 |  
 |  GetDebug(self)
 |      GetDebug(ProcessObject self) -> bool
 |  
 |  GetName(self)
 |      GetName(ProcessObject self) -> std::string
 |      
 |      
 |      
 |      return user readable name for the filter
 |  
 |  GetNumberOfThreads(self)
 |      GetNumberOfThreads(ProcessObject self) -> unsigned int
 |  
 |  GetProgress(self)
 |      GetProgress(ProcessObject self) -> float
 |      
 |      
 |      
 |      An Active Measurement of the progress of execution.
 |      
 |      
 |      Get the execution progress of the current process object. The progress
 |      is a floating number in [0,1] with 0 meaning no progress and 1 meaning
 |      the filter has completed execution (or aborted).
 |      
 |      This is an Active Measurement so it can be accessed during Events
 |      during the execution.
 |  
 |  HasCommand(self, event)
 |      HasCommand(ProcessObject self, itk::simple::EventEnum event) -> bool
 |      
 |      
 |      
 |      Query of this object has any registered commands for event.
 |  
 |  RemoveAllCommands(self)
 |      RemoveAllCommands(ProcessObject self)
 |      
 |      
 |      
 |      Remove all registered commands.
 |      
 |      
 |      Calling when this object is invoking anther command will produce
 |      undefined behavior.
 |  
 |  SetDebug(self, debugFlag)
 |      SetDebug(ProcessObject self, bool debugFlag)
 |  
 |  SetNumberOfThreads(self, n)
 |      SetNumberOfThreads(ProcessObject self, unsigned int n)
 |  
 |  __del__ lambda self
 |  
 |  __getattr__ lambda self, name
 |  
 |  __init__(self, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  
 |  __repr__ = _swig_repr(self)
 |  
 |  __setattr__ lambda self, name, value
 |  
 |  __str__(self)
 |      __str__(ProcessObject self) -> std::string
 |  
 |  ----------------------------------------------------------------------
 |  Static methods defined here:
 |  
 |  GetGlobalDefaultCoordinateTolerance()
 |      GetGlobalDefaultCoordinateTolerance() -> double
 |  
 |  GetGlobalDefaultDebug()
 |      GetGlobalDefaultDebug() -> bool
 |  
 |  GetGlobalDefaultDirectionTolerance()
 |      GetGlobalDefaultDirectionTolerance() -> double
 |  
 |  GetGlobalDefaultNumberOfThreads()
 |      GetGlobalDefaultNumberOfThreads() -> unsigned int
 |  
 |  GetGlobalWarningDisplay()
 |      GetGlobalWarningDisplay() -> bool
 |  
 |  GlobalDefaultDebugOff()
 |      GlobalDefaultDebugOff()
 |  
 |  GlobalDefaultDebugOn()
 |      GlobalDefaultDebugOn()
 |  
 |  GlobalWarningDisplayOff()
 |      GlobalWarningDisplayOff()
 |  
 |  GlobalWarningDisplayOn()
 |      GlobalWarningDisplayOn()
 |  
 |  SetGlobalDefaultCoordinateTolerance(arg1)
 |      SetGlobalDefaultCoordinateTolerance(double arg1)
 |  
 |  SetGlobalDefaultDebug(debugFlag)
 |      SetGlobalDefaultDebug(bool debugFlag)
 |  
 |  SetGlobalDefaultDirectionTolerance(arg1)
 |      SetGlobalDefaultDirectionTolerance(double arg1)
 |  
 |  SetGlobalDefaultNumberOfThreads(n)
 |      SetGlobalDefaultNumberOfThreads(unsigned int n)
 |  
 |  SetGlobalWarningDisplay(flag)
 |      SetGlobalWarningDisplay(bool flag)
 |  
 |  __swig_destroy__ = delete_ProcessObject(...)
 |      delete_ProcessObject(ProcessObject self)
 |      
 |      
 |      
 |      Default Destructor
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  __swig_getmethods__ = {}
 |  
 |  __swig_setmethods__ = {}

Deriving from the Command class

The traditional way of using Commands in ITK involves deriving from the Command class and adding to the ProcessObject.

In [10]:
class MyCommand(sitk.Command):
    def __init__(self, msg):
        # required
        super(MyCommand,self).__init__()
        self.msg = msg
    
    def __del__(self):
        print("MyCommand begin deleted: \"{0}\"".format(self.msg))

    def Execute(self):
        print(self.msg)
In [11]:
cmd1 = MyCommand("Start")
cmd2 = MyCommand("End")
filter.RemoveAllCommands() # this line is here so we can easily re-execute this code block
filter.AddCommand(sitk.sitkStartEvent, cmd1)
filter.AddCommand(sitk.sitkEndEvent, cmd2)
filter.Execute()

Start
End
Out[11]:
<SimpleITK.SimpleITK.Image; proxy of <Swig Object of type 'std::vector< itk::simple::Image >::value_type *' at 0x10db9bbd0> >

A reference to the Command object must be maintained, or else it will be removed from the ProcessObject.

In [12]:
filter.AddCommand(sitk.sitkStartEvent, MyCommand("stack scope"))
print("Before Execution")
filter.Execute()

MyCommand begin deleted: "stack scope"
Before Execution
Start
End
Out[12]:
<SimpleITK.SimpleITK.Image; proxy of <Swig Object of type 'std::vector< itk::simple::Image >::value_type *' at 0x10db9b840> >

Using a labmda function as the Command

In Python the AddCommand has been extended to accept PyCommand objects and implicitly creates a PyCommand from a callable python argument. This is really useful.

In [13]:
filter.RemoveAllCommands() # this line is here so we can easily re-execute this code block
filter.AddCommand(sitk.sitkStartEvent, lambda: print("Starting...",end=''))
filter.AddCommand(sitk.sitkStartEvent, lambda: sys.stdout.flush())
filter.AddCommand(sitk.sitkEndEvent, lambda: print("Done"))
filter.Execute()

Starting...Done
Out[13]:
<SimpleITK.SimpleITK.Image; proxy of <Swig Object of type 'std::vector< itk::simple::Image >::value_type *' at 0x10db9bb70> >

Access to ITK data during command execution

The commands are not too useful unless you can query the filter through the SimpleITK interface. A couple status variables and methods are exposed in the SimpleITK ProcessObject through the polymorphic interface of the same ITK class.

In [14]:
filter.RemoveAllCommands()
filter.AddCommand(sitk.sitkProgressEvent, lambda: print("\rProgress: {0:03.1f}%...".format(100*filter.GetProgress()),end=''))
filter.AddCommand(sitk.sitkProgressEvent, lambda: sys.stdout.flush())
filter.AddCommand(sitk.sitkEndEvent, lambda: print("Done"))
filter.Execute()

Progress: 100.0%...Done
Out[14]:
<SimpleITK.SimpleITK.Image; proxy of <Swig Object of type 'std::vector< itk::simple::Image >::value_type *' at 0x10db9b660> >

Utilizing Jupyter Notebooks and Commands

Utilization of commands and events frequently occurs with advanced integration into graphical user interfaces. Let us now export this advanced integration into Jupyter Notebooks.

Jupyter notebooks support displaying output as HTML, and execution of javascript on demand. Together this can produce animation.

In [15]:
import uuid
from IPython.display import HTML, Javascript, display

divid = str(uuid.uuid4())

html_progress="""
<p style="margin:5px">FilterName:</p>
<div style="border: 1px solid black;padding:1px;margin:5px">
  <div id="{0}" style="background-color:blue; width:0%%">&nbsp;</div>
</div>
""".format(divid)

def command_js_progress(processObject):
    p = processObject.GetProgress()
    display(Javascript("$('div#%s').width('%i%%')" % (divid, int(p*100))))
In [16]:
filter.RemoveAllCommands()
filter.AddCommand(sitk.sitkStartEvent, lambda:  display(HTML(html_progress)))
filter.AddCommand(sitk.sitkProgressEvent, lambda: command_js_progress(filter))

filter.Execute()

FilterName:

 
Out[16]:
<SimpleITK.SimpleITK.Image; proxy of <Swig Object of type 'std::vector< itk::simple::Image >::value_type *' at 0x10db9bb40> >

Support for Bi-direction JavaScript

It's possible to get button in HTML to execute python code...

In [17]:
import uuid
from IPython.display import HTML, Javascript, display

g_Abort = False
divid = str(uuid.uuid4())

html_progress_abort="""
<div style="background-color:gainsboro; border:2px solid black;padding:15px">
<p style="margin:5px">FilterName:</p>
<div style="border: 1px solid black;padding:1px;margin:5px">
  <div id="{0}" style="background-color:blue; width:0%%">&nbsp;</div>
</div>
<button onclick="set_value()" style="margin:5px" >Abort</button>
</div>
""".format(divid)

javascript_abort = """
<script type="text/Javascript">
    function set_value(){
        var command = "g_Abort=True"
        console.log("Executing Command: " + command);
        
        var kernel = IPython.notebook.kernel;
        kernel.execute(command);
    }
</script>
"""

def command_js_progress_abort(processObject):
    p = processObject.GetProgress()
    display(Javascript("$('div#%s').width('%i%%')" % (divid, int(p*100))))
    if g_Abort:
        processObject.Abort()
        
def command_js_start_abort():        
    g_Abort=False
In [18]:
g_Abort=False
filter.RemoveAllCommands()
filter.AddCommand(sitk.sitkStartEvent, command_js_start_abort )
filter.AddCommand(sitk.sitkStartEvent, lambda:  display(HTML(html_progress_abort+javascript_abort)))
filter.AddCommand(sitk.sitkProgressEvent, lambda: command_js_progress_abort(filter))
Out[18]:
0

A caveat with this approach is that the IPython kernel must continue to execute while the filter is running. So we must place the filter in a thread.

In [19]:
import threading
threading.Thread( target=lambda:filter.Execute() ).start()

FilterName:

 

While the lambda command are convenient, the lack for having an object to hold data can still be problematic. For example in the above code the uuid, is used to uniquely identify the HTML element. So if the filter is executed multiple times then the JavaScript update will be confused on what to update.

In [20]:
#### The following shows a failure that you will want to avoid.
threading.Thread( target=lambda:filter.Execute() ).start()

FilterName:

 

A Reusable class for IPython Progress

There currently are too many caveats without support for Abort. Let us create a reusable class which will automatically generate the UUID and just display the progress.

In [21]:
import uuid
from IPython.display import HTML, Javascript, display

class HTMLProgressWatcher:
    def __init__(self, po):
        self.processObject = po
        self.abort = False
        
        po.AddCommand(sitk.sitkStartEvent, lambda: self.cmdStartEvent() )
        po.AddCommand(sitk.sitkProgressEvent, lambda: self.cmdProgressEvent() )
        po.AddCommand(sitk.sitkEndEvent, lambda: self.cmdEndEvent() )
        
    def cmdStartEvent(self):
        global sitkIPythonProgress_UUID
        self.abort=False
        self.divid = str(uuid.uuid4())
        
        try:
            sitkIPythonProgress_UUID[self.divid] = self
        except NameError:
            sitkIPythonProgress_UUID = {self.divid: self}
        
        html_progress_abort="""
<p style="margin:5px">{0}:</p>
<div style="border: 1px solid black;padding:1px;margin:5px">
  <div id="{1}" style="background-color:blue; width:0%%">&nbsp;</div>
</div>
""".format(self.processObject.GetName(), self.divid)
        
        display(HTML(html_progress_abort+javascript_abort))
    
    def cmdProgressEvent(self):
        p = self.processObject.GetProgress()
        display(Javascript("$('div#%s').width('%i%%')" % (self.divid, int(p*100))))
        if self.abort:
            self.processObject.Abort()
    
    def cmdEndEvent(self):
        global sitkIPythonProgress_UUID
        del sitkIPythonProgress_UUID[self.divid]
        del self.divid
In [22]:
filter.RemoveAllCommands()
watcher  = HTMLProgressWatcher(filter)
In [23]:
filter.Execute()

GaborImageSource:

 
Out[23]:
<SimpleITK.SimpleITK.Image; proxy of <Swig Object of type 'std::vector< itk::simple::Image >::value_type *' at 0x11dc9f3c0> >
In [24]:
threading.Thread.start?
In [ ]: