Images
| SimpleITK conventions: |
|
The unique feature of SimpleITK (derived from ITK) as a toolkit for image manipulation and analysis is that it views images as physical objects occupying a bounded region in physical space. In addition images can have different spacing between pixels along each axis, and the axes are not necessarily orthogonal. The following figure illustrates these concepts.
Pixel Types¶
The pixel type is represented as an enumerated type. The following is a table of the enumerated list.
| sitkUInt8 | Unsigned 8 bit integer |
| sitkInt8 | Signed 8 bit integer |
| sitkUInt16 | Unsigned 16 bit integer |
| sitkInt16 | Signed 16 bit integer |
| sitkUInt32 | Unsigned 32 bit integer |
| sitkInt32 | Signed 32 bit integer |
| sitkUInt64 | Unsigned 64 bit integer |
| sitkInt64 | Signed 64 bit integer |
| sitkFloat32 | 32 bit float |
| sitkFloat64 | 64 bit float |
| sitkComplexFloat32 | complex number of 32 bit float |
| sitkComplexFloat64 | complex number of 64 bit float |
| sitkVectorUInt8 | Multi-component of unsigned 8 bit integer |
| sitkVectorInt8 | Multi-component of signed 8 bit integer |
| sitkVectorUInt16 | Multi-component of unsigned 16 bit integer |
| sitkVectorInt16 | Multi-component of signed 16 bit integer |
| sitkVectorUInt32 | Multi-component of unsigned 32 bit integer |
| sitkVectorInt32 | Multi-component of signed 32 bit integer |
| sitkVectorUInt64 | Multi-component of unsigned 64 bit integer |
| sitkVectorInt64 | Multi-component of signed 64 bit integer |
| sitkVectorFloat32 | Multi-component of 32 bit float |
| sitkVectorFloat64 | Multi-component of 64 bit float |
| sitkLabelUInt8 | RLE label of unsigned 8 bit integers |
| sitkLabelUInt16 | RLE label of unsigned 16 bit integers |
| sitkLabelUInt32 | RLE label of unsigned 32 bit integers |
| sitkLabelUInt64 | RLE label of unsigned 64 bit integers |
There is also sitkUnknown, which is used for undefined or erroneous pixel ID's. It has a value of -1.
The 64-bit integer types are not available on all distributions. When not available the value is sitkUnknown.
from __future__ import print_function
import SimpleITK as sitk
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
from ipywidgets import interact, fixed
import os
OUTPUT_DIR = 'Output'
# Utility method that either downloads data from the Girder repository or
# if already downloaded returns the file name for reading from disk (cached data).
%run update_path_to_download_script
from downloaddata import fetch_data as fdata
Load your first image and display it¶
logo = sitk.ReadImage(fdata('SimpleITK.jpg'))
plt.imshow(sitk.GetArrayViewFromImage(logo))
plt.axis('off');
Image Construction¶
There are a variety of ways to create an image.
The following components are required for a complete definition of an image:
- Pixel type [fixed on creation, no default]: unsigned 32 bit integer, sitkVectorUInt8, etc., see list above.
- Sizes [fixed on creation, no default]: number of pixels/voxels in each dimension. This quantity implicitly defines the image dimension.
- Origin [default is zero]: coordinates of the pixel/voxel with index (0,0,0) in physical units (i.e. mm).
- Spacing [default is one]: Distance between adjacent pixels/voxels in each dimension given in physical units.
- Direction matrix [default is identity]: mapping, rotation, between direction of the pixel/voxel axes and physical directions.
Initial pixel/voxel values are set to zero.
image_3D = sitk.Image(256, 128, 64, sitk.sitkInt16)
image_2D = sitk.Image(64, 64, sitk.sitkFloat32)
image_2D = sitk.Image([32,32], sitk.sitkUInt32)
image_RGB = sitk.Image([128,64], sitk.sitkVectorUInt8, 3)
Basic Image Attributes¶
You can change the image origin, spacing and direction. Making such changes to an image already containing data should be done cautiously.
image_3D.SetOrigin((78.0, 76.0, 77.0))
image_3D.SetSpacing([0.5,0.5,3.0])
print(image_3D.GetOrigin())
print(image_3D.GetSize())
print(image_3D.GetSpacing())
print(image_3D.GetDirection())
Image dimension queries:
print(image_3D.GetDimension())
print(image_3D.GetWidth())
print(image_3D.GetHeight())
print(image_3D.GetDepth())
What is the depth of a 2D image?
print(image_2D.GetSize())
print(image_2D.GetDepth())
Pixel/voxel type queries:
print(image_3D.GetPixelIDValue())
print(image_3D.GetPixelIDTypeAsString())
print(image_3D.GetNumberOfComponentsPerPixel())
What is the dimension and size of a Vector image and its data?
print(image_RGB.GetDimension())
print(image_RGB.GetSize())
print(image_RGB.GetNumberOfComponentsPerPixel())
Accessing Pixels and Slicing¶
The Image class's member functions GetPixel and SetPixel provide an ITK-like interface for pixel access.
help(image_3D.GetPixel)
print(image_3D.GetPixel(0, 0, 0))
image_3D.SetPixel(0, 0, 0, 1)
print(image_3D.GetPixel(0, 0, 0))
# This can also be done using Pythonic notation.
print(image_3D[0,0,1])
image_3D[0,0,1] = 2
print(image_3D[0,0,1])
Slicing of SimpleITK images returns a copy of the image data.
This is similar to slicing Python lists and differs from the "view" returned by slicing numpy arrays.
# Brute force sub-sampling
logo_subsampled = logo[::2,::2]
# Get the sub-image containing the word Simple
simple = logo[0:115,:]
# Get the sub-image containing the word Simple and flip it
simple_flipped = logo[115:0:-1,:]
n = 4
plt.subplot(n,1,1)
plt.imshow(sitk.GetArrayViewFromImage(logo))
plt.axis('off');
plt.subplot(n,1,2)
plt.imshow(sitk.GetArrayViewFromImage(logo_subsampled))
plt.axis('off');
plt.subplot(n,1,3)
plt.imshow(sitk.GetArrayViewFromImage(simple))
plt.axis('off')
plt.subplot(n,1,4)
plt.imshow(sitk.GetArrayViewFromImage(simple_flipped))
plt.axis('off');
Draw a square on top of the logo image: After running this cell, uncomment "Version 3" and see its effect.
# Version 0: get the numpy array and assign the value via broadcast - later on you will need to construct
# a new image from the array
logo_pixels = sitk.GetArrayFromImage(logo)
logo_pixels[0:10,0:10] = [0,255,0]
# Version 1: generates an error, the image slicing returns a new image and you cannot assign a value to an image
#logo[0:10,0:10] = [255,0,0]
# Version 2: image slicing returns a new image, so all assignments here will not have any effect on the original
# 'logo' image
logo_subimage = logo[0:10, 0:10]
for x in range(0,10):
for y in range(0,10):
logo_subimage[x,y] = [255,0,0]
# Version 3: modify the original image, iterate and assign a value to each pixel
#for x in range(0,10):
# for y in range(0,10):
# logo[x,y] = [255,0,0]
plt.subplot(2,1,1)
plt.imshow(sitk.GetArrayViewFromImage(logo))
plt.axis('off')
plt.subplot(2,1,2)
plt.imshow(logo_pixels)
plt.axis('off');
Conversion between numpy and SimpleITK¶
SimpleITK and numpy indexing access is in opposite order!
SimpleITK: image[x,y,z]
numpy: image_numpy_array[z,y,x]
From SimpleITK to numpy¶
We have two options for converting from SimpleITK to numpy:
- GetArrayFromImage(): returns a copy of the image data. You can then freely modify the data as it has no effect on the original SimpleITK image.
- GetArrayViewFromImage(): returns a view on the image data which is useful for display in a memory efficient manner. You cannot modify the data and the view will be invalid if the original SimpleITK image is deleted.
nda = sitk.GetArrayFromImage(image_3D)
print(image_3D.GetSize())
print(nda.shape)
nda = sitk.GetArrayFromImage(image_RGB)
print(image_RGB.GetSize())
print(nda.shape)
gabor_image = sitk.GaborSource(size=[64,64], frequency=.03)
# Getting a numpy array view on the image data doesn't copy the data
nda_view = sitk.GetArrayViewFromImage(gabor_image)
plt.imshow(nda_view, cmap=plt.cm.Greys_r)
plt.axis('off');
# Trying to assign a value to the array view will throw an exception
nda_view[0,0] = 255
From numpy to SimpleITK¶
Remember to to set the image's origin, spacing, and possibly direction cosine matrix. The default values may not match the physical dimensions of your image.
nda = np.zeros((10,20,3))
#if this is supposed to be a 3D gray scale image [x=3, y=20, z=10]
img = sitk.GetImageFromArray(nda)
print(img.GetSize())
#if this is supposed to be a 2D color image [x=20,y=10]
img = sitk.GetImageFromArray(nda, isVector=True)
print(img.GetSize())
There and back again¶
The following code cell illustrates a situation where your code is a combination of SimpleITK methods and custom Python code which works with intensity values or labels outside of SimpleITK. This is a reasonable approach when you implement an algorithm in Python and don't care about the physical spacing of things (you are actually assuming the volume is isotropic).
def my_algorithm(image_as_numpy_array):
# res is the image result of your algorithm, has the same grid size as the original image
res = image_as_numpy_array
return res
# Starting with SimpleITK
img = sitk.ReadImage(fdata('training_001_mr_T1.mha'))
# Custom Python code working on a numpy array.
npa_res = my_algorithm(sitk.GetArrayFromImage(img))
# Converting back to SimpleITK (assumes we didn't move the image in space as we copy the information from the original)
res_img = sitk.GetImageFromArray(npa_res)
res_img.CopyInformation(img)
# Continuing to work with SimpleITK images
res_img - img
Image operations¶
SimpleITK supports basic arithmetic operations between images, taking into account their physical space.
Repeatedly run this cell. Fix the error (comment out the SetDirection, then SetSpacing). Why doesn't the SetOrigin line cause a problem? How close do two physical attributes need to be in order to be considered equivalent?
img1 = sitk.Image(24,24, sitk.sitkUInt8)
img1[0,0] = 0
img2 = sitk.Image(img1.GetSize(), sitk.sitkUInt8)
img2.SetDirection([0,1,0.5,0.5])
img2.SetSpacing([0.5,0.8])
img2.SetOrigin([0.000001,0.000001])
img2[0,0] = 255
img3 = img1 + img2
print(img3[0,0])
Reading and Writing¶
SimpleITK can read and write images stored in a single file, or a set of files (e.g. DICOM series).
Images stored in the DICOM format have a meta-data dictionary associated with them, which is populated with the DICOM tags. When a DICOM series is read as a single image, the meta-data information is not available since DICOM tags are specific to each file. If you need the meta-data, you have three options:
Using the object oriented interface's ImageSeriesReader class, configure it to load the tags using the
MetaDataDictionaryArrayUpdateOnmethod and possibly theLoadPrivateTagsOnmethod if you need the private tags. Once the series is read you can access the meta-data from the series reader using theGetMetaDataKeys,HasMetaDataKey, andGetMetaData.Using the object oriented interface's ImageFileReader, set a specific slice's file name and only read it's meta-data using the
ReadImageInformationmethod which only reads the meta-data but not the bulk pixel information. Once the meta-data is read you can access it from the file reader using theGetMetaDataKeys,HasMetaDataKey, andGetMetaData.Using the object oriented interface's ImageFileReader, set a specific slice's file name and read it. Or using the procedural interface's, ReadImage function, read a specific file. You can then access the meta-data directly from the Image using the
GetMetaDataKeys,HasMetaDataKey, andGetMetaData.
In the following cell, we read an image in JPEG format, and write it as PNG and BMP. File formats are deduced from the file extension. Appropriate pixel type is also set - you can override this and force a pixel type of your choice.
img = sitk.ReadImage(fdata('SimpleITK.jpg'))
print(img.GetPixelIDTypeAsString())
# write as PNG and BMP
sitk.WriteImage(img, os.path.join(OUTPUT_DIR, 'SimpleITK.png'))
sitk.WriteImage(img, os.path.join(OUTPUT_DIR, 'SimpleITK.bmp'))
Read an image in JPEG format and cast the pixel type according to user selection.
# Several pixel types, some make sense in this case (vector types) and some are just show
# that the user's choice will force the pixel type even when it doesn't make sense
# (e.g. sitkVectorUInt16 or sitkUInt8).
pixel_types = { 'sitkUInt8': sitk.sitkUInt8,
'sitkUInt16' : sitk.sitkUInt16,
'sitkFloat64' : sitk.sitkFloat64,
'sitkVectorUInt8' : sitk.sitkVectorUInt8,
'sitkVectorUInt16' : sitk.sitkVectorUInt16,
'sitkVectorFloat64' : sitk.sitkVectorFloat64}
def pixel_type_dropdown_callback(pixel_type, pixel_types_dict):
#specify the file location and the pixel type we want
img = sitk.ReadImage(fdata('SimpleITK.jpg'), pixel_types_dict[pixel_type])
print(img.GetPixelIDTypeAsString())
print(img[0,0])
plt.imshow(sitk.GetArrayViewFromImage(img))
plt.axis('off')
interact(pixel_type_dropdown_callback, pixel_type=list(pixel_types.keys()), pixel_types_dict=fixed(pixel_types));
Read a DICOM series and write it as a single mha file
data_directory = os.path.dirname(fdata("CIRS057A_MR_CT_DICOM/readme.txt"))
series_ID = '1.2.840.113619.2.290.3.3233817346.783.1399004564.515'
# Get the list of files belonging to a specific series ID.
reader = sitk.ImageSeriesReader()
# Use the functional interface to read the image series.
original_image = sitk.ReadImage(reader.GetGDCMSeriesFileNames(data_directory, series_ID))
# Write the image.
output_file_name_3D = os.path.join(OUTPUT_DIR, '3DImage.mha')
sitk.WriteImage(original_image, output_file_name_3D)
# Read it back again.
written_image = sitk.ReadImage(output_file_name_3D)
# Check that the original and written image are the same.
statistics_image_filter = sitk.StatisticsImageFilter()
statistics_image_filter.Execute(original_image - written_image)
# Check that the original and written files are the same
print('Max, Min differences are : {0}, {1}'.format(statistics_image_filter.GetMaximum(), statistics_image_filter.GetMinimum()))
Write an image series as JPEG. The WriteImage function receives a volume and a list of images names and writes the volume according to the z axis. For a displayable result we need to rescale the image intensities (default is [0,255]) since the JPEG format requires a cast to the UInt8 pixel type.
sitk.WriteImage(sitk.Cast(sitk.RescaleIntensity(written_image), sitk.sitkUInt8),
[os.path.join(OUTPUT_DIR, 'slice{0:03d}.jpg'.format(i)) for i in range(written_image.GetSize()[2])])
Select a specific DICOM series from a directory and only then load user selection.
data_directory = os.path.dirname(fdata("CIRS057A_MR_CT_DICOM/readme.txt"))
# Global variable 'selected_series' is updated by the interact function
selected_series = ''
file_reader = sitk.ImageFileReader()
def DICOM_series_dropdown_callback(series_to_load, series_dictionary):
global selected_series
# Print some information about the series from the meta-data dictionary
# DICOM standard part 6, Data Dictionary: http://medical.nema.org/medical/dicom/current/output/pdf/part06.pdf
file_reader.SetFileName(series_dictionary[series_to_load][0])
file_reader.ReadImageInformation()
tags_to_print = {'0010|0010': 'Patient name: ',
'0008|0060' : 'Modality: ',
'0008|0021' : 'Series date: ',
'0008|0080' : 'Institution name: ',
'0008|1050' : 'Performing physician\'s name: '}
for tag in tags_to_print:
try:
print(tags_to_print[tag] + file_reader.GetMetaData(tag))
except: # Ignore if the tag isn't in the dictionary
pass
selected_series = series_to_load
# Directory contains multiple DICOM studies/series, store
# in dictionary with key being the series ID
reader = sitk.ImageSeriesReader()
series_file_names = {}
series_IDs = reader.GetGDCMSeriesIDs(data_directory)
# Check that we have at least one series
if series_IDs:
for series in series_IDs:
series_file_names[series] = reader.GetGDCMSeriesFileNames(data_directory, series)
interact(DICOM_series_dropdown_callback, series_to_load=list(series_IDs), series_dictionary=fixed(series_file_names));
else:
print('Data directory does not contain any DICOM series.')
reader.SetFileNames(series_file_names[selected_series])
img = reader.Execute()
# Display the image slice from the middle of the stack, z axis
z = int(img.GetDepth()/2)
plt.imshow(sitk.GetArrayViewFromImage(img)[z,:,:], cmap=plt.cm.Greys_r)
plt.axis('off');
Multi-channel images and color¶
Generally speaking, SimpleITK represents color images as multi-channel images independent of a color space. It is up to you to interpret the channels correctly based on additional color space knowledge prior to using them for display or any other purpose.
The following cells illustrate reading and interpretation of an interesting image in DICOM format. It is a photograph of an X-ray on a light box (yes, there are some strange things in the wild). The meta-data dictionary for this image contains the relevant information that will allow us to intelligently interpret it.
We will look at the image's modality (0008|0060), which in our case is XC, which stands for "External Camera Photography". After reading the image we see that it has three channels and try to display it. At this point we realize that something is wrong, and we check the image's Photometric Interpretation (0028|0004), color space in DICOM speak. We then modify the pixel values so that we can display them using our image viewer's expected color space.
xray_photo = sitk.ReadImage(fdata('photo.dcm'))
print('Image Modality: {0}'.format(xray_photo.GetMetaData('0008|0060')))
print('Number of channels: {0}'.format(xray_photo.GetNumberOfComponentsPerPixel()))
# Display the image using Fiji which expects the channels to be in the RGB color space
sitk.Show(xray_photo)
# In what color space are the pixels?
print('Photomertic Interpretation: {0}'.format(xray_photo.GetMetaData('0028|0004')))
# Our pixels are in the YCbCr color space, and we want them in RGB, so we need to change the representation
def full_ycbcr_to_rgb(image):
channels = [sitk.VectorIndexSelectionCast(image,i, sitk.sitkFloat32) for i in range(image.GetNumberOfComponentsPerPixel())]
chan0 = channels[0]
chan1 = channels[1] - 128
chan2 = channels[2] - 128
return sitk.Compose([sitk.Clamp(chan0 + 1.402*chan2, sitk.sitkUInt8),
sitk.Clamp(chan0 - 0.114 * 1.772 / 0.587*chan1 - 0.299 * 1.402 / 0.587*chan2, sitk.sitkUInt8),
sitk.Clamp(chan0 + 1.772*chan1, sitk.sitkUInt8)])
sitk.Show(full_ycbcr_to_rgb(xray_photo))
# x-rays are expected to be a single channel gray scale image and not a color image. To obtain a gray scale image
# corresponding to the original three channel image we only need to take the luminance (Y channel).
sitk.Show(sitk.VectorIndexSelectionCast(xray_photo,0))
Finer control¶
The ImageFileReader's interface provides finer control for reading, allowing us to require the use of a specific IO and allowing us to stream parts of an image to memory without reading the whole image (supported by a subset of the ImageIO components).
Selecting a Specific Image IO¶
SimpleITK relies on the registered ImageIOs to indicate whether they can read a file and then perform the reading. This is done automatically, going over the set of ImageIOs and inquiring whether they can read the given file. The first one that can is selected. If multiple ImageIOs can read a specific format, we do not know which one was used for the task (e.g. TIFFImageIO and LSMImageIO, which is derived from it, can both read tif files). In some cases you may want to use a specific IO, possibly one that reads the file faster, or supports a more complete feature set associated with the file format.
The next cell shows how to find out which ImageIOs are registered and specify the one we want.
file_reader = sitk.ImageFileReader()
# Get a tuple listing all registered ImageIOs
image_ios_tuple = file_reader.GetRegisteredImageIOs()
print("The supported image IOs are: " + str(image_ios_tuple))
# Optionally, just print the reader and see which ImageIOs are registered
print('\n',file_reader)
# Specify the JPEGImageIO and read file
file_reader.SetImageIO('JPEGImageIO')
file_reader.SetFileName(fdata('SimpleITK.jpg'))
logo = file_reader.Execute()
# Unfortunately, now reading a non JPEG image will fail
try:
file_reader.SetFileName(fdata('cthead1.png'))
ct_head = file_reader.Execute()
except RuntimeError:
print('Got a RuntimeError exception.')
# We can reset the file reader to its default behaviour so that it automatically
# selects the ImageIO
file_reader.SetImageIO('')
ct_head = file_reader.Execute()
Streaming Image IO¶
Some of the ImageIOs supported in SimpleITK allow you to stream in sub-regions of an image without the need to read the whole image into memory. This is very useful when you are memory constrained (either your images are large or your memory is limited).
The ImageIOs that support streaming include HDF5ImageIO, VTKImageIO, NiftiImageIO, MetaImageIO...
The next cell shows how to read in a sub/cropped image from a larger image. We read the central 1/3 portion of the image [1/3,2/3] of the original image.
file_reader = sitk.ImageFileReader()
file_reader.SetFileName(fdata('vm_head_rgb.mha'))
file_reader.ReadImageInformation()
image_size = file_reader.GetSize()
start_index, extract_size = zip(*[(int(1.0/3.0*sz), int(1.0/3.0*sz)) for sz in file_reader.GetSize()])
file_reader.SetExtractIndex(start_index)
file_reader.SetExtractSize(extract_size)
sitk.Show(file_reader.Execute())
The next cells show how to subtract two large images from each other with a smaller memory footprint than the direct approach, though the code is much more complex and slower than the direct approach:
sitk.ReadImage(image1_file_name) - sitk.ReadImage(image2_file_name)Note: The code assume that the two images occupy the same spatial region (origin, spacing, direction cosine matrix).
def streaming_subtract(image1_file_name, image2_file_name, parts):
'''
Subtract image1 from image2 using 'parts' number of sub-regions.
'''
file_reader = sitk.ImageFileReader()
file_reader.SetFileName(image1_file_name)
file_reader.ReadImageInformation()
image_size = file_reader.GetSize()
# Create the result image, initially empty
result_img = sitk.Image(file_reader.GetSize(), file_reader.GetPixelID(), file_reader.GetNumberOfComponents())
result_img.SetSpacing(file_reader.GetSpacing())
result_img.SetOrigin(file_reader.GetOrigin())
result_img.SetDirection(file_reader.GetDirection())
extract_size = list(file_reader.GetSize())
extract_size[-1] = extract_size[-1]//parts
current_index = [0]*file_reader.GetDimension()
for i in range(parts):
if i == (parts-1): # last region may be smaller than the standard extract region
extract_size[-1] = image_size[-1] - current_index[-1]
file_reader.SetFileName(image1_file_name)
file_reader.SetExtractIndex(current_index)
file_reader.SetExtractSize(extract_size)
sub_image1 = file_reader.Execute()
file_reader.SetFileName(image2_file_name)
file_reader.SetExtractIndex(current_index)
file_reader.SetExtractSize(extract_size)
sub_image2 = file_reader.Execute()
# Paste the result of subtracting the two subregions into their location in the result_img
result_img = sitk.Paste(result_img, sub_image1 - sub_image2, extract_size, [0]*file_reader.GetDimension(), current_index)
current_index[-1] += extract_size[-1]
return result_img
# If you have the patience and RAM you can try this with the vm_head_rgb.mha image.
image1_file_name = fdata('fib_sem_bacillus_subtilis.mha')
image2_file_name = fdata('fib_sem_bacillus_subtilis.mha')
A simple way of seeing your system's memory usage is to open the appropriate monitoring program: (Windows) Resource Monitor; (Linux) top; (OS X) Activity Monitor. This will give you a rough idea of the memory used by the streaming vs. non streaming approaches.
result_img = streaming_subtract(image1_file_name, image2_file_name, parts=5)
del result_img
result_img = sitk.ReadImage(image1_file_name) - sitk.ReadImage(image2_file_name)
del result_img