atlalign.ml_utils package¶
Submodules¶
atlalign.ml_utils.augmentation module¶
Augmentation related tools.
- augmenter_1(p=0.99)[source]¶
Create augmenter.
Contains no coordinate transforms.
- Parameters
p (float) – Number in [0, 1] representing the probability of a random augmentation happening.
- Returns
seq – Augmenter where each augmentation was manually inspected and makes sense.
- Return type
iaa.Augmenter
atlalign.ml_utils.callbacks module¶
Callbacks and aggregation functions.
- class MLFlowCallback(merged_path, train_original_ixs_path, val_original_ixs_path, freq=10, workers=1, return_inverse=False, starting_step=0, use_validation=True)[source]¶
Bases:
keras.callbacks.Callback
Logs metrics into ML.
Notes
Only runs inside of an mlflow context.
- Parameters
merged_path (str) – Path to the master h5 file containing all the data.
train_original_ixs_path (str) – Path to where original training indices are stored.
val_original_ixs_path (str) – Path to where original validation indices are stored.
freq (int) – Reports metrics on every freq batch.
workers (int) – Number of workers to be used for each of the evaluations.
return_inverse (bool) – If True, then generators behave differently.
starting_step (int) – Useful when we want to use a checkpointed model and log metrics as of a different step then 1.
use_validation (int) – If True, then the custom metrics are computed on the validation set. Otherwise they will be computed on the training set.
- static compute_external_metrics(model, gen)[source]¶
Compute external matrics sample by sample.
- Parameters
model – Keras model
gen (SupervisedGenerator) – Generator
- Returns
metrics – Various metrics.
- Return type
dict
- get_mlflow_artifact_path(start_char=7)[source]¶
Get path to the MLFlow artifacts of the active run.
Stupid implementation.
- Parameters
start_char (int) – Since the string will start like “file:///actual/path…” we just slice it.
atlalign.ml_utils.io module¶
Collection of functions dealing with input and output.
- class SupervisedGenerator(hdf_path, batch_size=32, shuffle=False, augmenter_ref=None, augmenter_mov=None, return_inverse=False, mlflow_log=False, indexes=None)[source]¶
Bases:
keras.utils.data_utils.Sequence
Generator streaming supervised data from a HDF5 files.
- Parameters
hdf_path (str or pathlib.Path) – Path to where the hdf5 file is.
batch_size (int) – Batch size.
shuffle (bool) – If True, then data shuffled at the end of each epoch and also in the constructor.
augmenter_ref (None or imgaug.augmenters.Sequential) – If None, no augmentation. If instance of a imgaug Sequential then its a pipeline that will be used to augment all reference images in a batch.
augmenter_mov (None or imgaug.augmenters.Sequential) – If None, no augmentation. If instance of a imgaug Sequential then its a pipeline that will be used to augment all moving images in a batch.
return_inverse (bool) – If True, then targets are [img_reg, dvf, dvf_inv] else its only [img_reg, dvf].
indexes (None or list or str/pathlib.Path) – A user defined list of indices to be used for streaming. This is used to give user the chance to only stream parts of the data. If str or pathlib.Path then read from a .npy file
- indexes¶
List of indices determining the slicing order.
- Type
list
- volume¶
Array representing the nissle stain volume of dtype float32.
- Type
np.array
- times¶
List of timedeltas representing the for each batch yeild.
- Type
list
atlalign.ml_utils.layers module¶
Custom keras layers.
- class Affine2DVF(*args, **kwargs)[source]¶
Bases:
keras.engine.base_layer.Layer
Given an affine transformation matrix (2 x 3) generate the corresponding DVF.
- call(a, mask=None)[source]¶
Turn a batch of affine matrices into the corresponding batch of displacement vector fields.
- Parameters
a (K.Tensor) – A tensor of shape (batch_size, 2, 3) representing an affine transformation for each sample.
- Returns
dvfs – A tensor of shape (batch_size, h, w, 2) representing the corresponding displacement vector fields.
- Return type
K.Tensor
- compute_output_shape(*args)[source]¶
Compute the output shape.
- Parameters
args (list) – Whatever positional arguments. We already know the expected shape from the self.shape passed into the constructor.
- Returns
batch_size (int) – Always None since we want to allow for variable batch_size.
height (int) – Height of the image.
width (int) – Width of the image.
n_channels (int) – Always equal to 2 because it represents delta_x and delta_y.
- class BilinearInterpolation(*args, **kwargs)[source]¶
Bases:
keras.engine.base_layer.Layer
Implementation using tf.contrib.
- call(tensors, mask=None)[source]¶
Perform forward pass.
- Parameters
tensors (list) –
A list of two elements - images to we warped and the displacement vector fields. Their shapes are the following
images to be warped : (batch_size, h, w, 1)
dvfs : (batch_size, h, w, 2)
mask (None) –
- Returns
output – Warped images with the provided dvfs. Shape (batch_size, h, w, 1).
- Return type
K.Tensor
- compute_output_shape(input_shapes)[source]¶
Compute output shape.
Notes
This is a mandatory method to implement for custom Keras layers.
- Parameters
input_shapes (list) – A list of the shape of images to be warped and the dvfs respectively. - images to be warped : (batch_size, h, w, 1) - dvfs : (batch_size, h, w, 2)
- Returns
None – Batch size
height (int) – Height of the image.
width (int) – Width of the image.
n_channels (int) – Number of channels, should be 1.
- class BilinearInterpolation_(*args, **kwargs)[source]¶
Bases:
keras.engine.base_layer.Layer
Perform bilinear interpolation as a Keras layer.
Notes
Currently only works with tensorflow backend since we need to use tf.meshgrid.
- call(tensors, mask=None)[source]¶
Perform forward pass.
- Parameters
tensors (list) –
A list of two elements - images to we warped and the displacement vector fields. Their shapes are the following
images to be warped : (batch_size, h, w, 1)
dvfs : (batch_size, h, w, 2)
mask (None) –
- Returns
output – Warped images with the provided dvfs. Shape (batch_size, h, w, 1).
- Return type
K.Tensor
- compute_output_shape(input_shapes)[source]¶
Compute output shape.
Notes
This is a mandatory method to implement for custom Keras layers.
- Parameters
input_shapes (list) – A list of the shape of images to be warped and the dvfs respectively. - images to be warped : (batch_size, h, w, 1) - dvfs : (batch_size, h, w, 2)
- Returns
None – Batch size
height (int) – Height of the image.
width (int) – Width of the image.
n_channels (int) – Number of channels, should be 1.
- class DVFComposition(*args, **kwargs)[source]¶
Bases:
keras.engine.base_layer.Layer
Composition of 2 displacement vector fields.
Notes
Computes the DVF of f ∘ g
- call(tensors, mask=None)[source]¶
Composition.
- Parameters
tensors (list) – A list of two elements - outer and inner displacement vector field. Their shapes are identical and equal to (batch_size, h, w, 2).
- Returns
output – A displacement vector field of the composite transformation of shape (batch_size, h, w, 2).
- Return type
K.tensor
- compute_output_shape(input_shapes)[source]¶
Compute output shape.
- Parameters
input_shapes (list) – A list of the shape of th 2 dvfs respectively. - dvfs_outer: (batch_size, h, w, 2) - dvfs_inner : (batch_size, h, w, 2)
- Returns
None – Batch size
height (int) – Height of the image.
width (int) – Width of the image.
n_dim (int) – 2
- class ExtractMoving(*args, **kwargs)[source]¶
Bases:
keras.engine.base_layer.Layer
Extract the moving image from the input.
- block_stn(images, filters=(16, 32, 64), dense_layers=(50,), block_name='STN', return_dvfs=True, return_inverse=True, include_bn_dense=False, include_bn_conv=False)[source]¶
Spatial transformer network block.
- Parameters
images (K.tensor) –
- Tensor of shape (batch_size, h, w, 2) representing the batch of images to be warped. The first channel
represents the atlas image and the second one is the moving image.
filters (tuple) – A tuple whose length represents the number of convolutional layers and each element represents the number of filters of that layer. Make sure that h % (2 ** len(filters)) == 0 and w % (2 ** len(filters)) == 0 because of max pooling.
dense_layers (tuple) – A tuple whose length represents the number of dense layers after the convolutional layers and each element represents the number of nodes. Note a final dense layer with 8 nodes (number of parameters in the affine transformation) is added automatically everytime.
block_name (str) – Name of the block.
return_dvfs (bool) – If True then also returns the dsvf tensor then can later be used for composition.
return_inverse (bool) – If True, then warping applied to the second channel (moving image), otherwise the first image (atlas).
include_bn_dense (bool) – If True, batch normalization is performed after each dense layer layer.
include_bn_conv (bool) – If True, batch normalization is performed after each convolutional layer.
- Returns
warped_images –
- Tensor of shape (batch_size, h, w, 1) representing the batch of warped images with the predicted affine
transformation. If return_inverse True then this is the moving image. Else the reference.
- dvfsK.tensor
Tensor of shape (batch_size, h, w, 2) representing the batch of delta_x and delta_y displacements. Only returned if return_dvfs is True.
- Return type
K.tensor
- get_initial_weights(previous_layer_size)[source]¶
Initialize affine matrix to identity transformation.
Notes
It makes a lot of sense to set the behavior before training to learning an identity transformation.
- Parameters
previous_layer_size (int) – Size of the previous dense layer.
- Returns
initial_weights –
- A list of 2 numpy ndarrays:
weight matrix, shape (previous_layer_size, 6)
biases, shape (6,)
- Return type
list
atlalign.ml_utils.losses module¶
keras and Tensorflow implementation of different losses and metrics.
- class DVF2IMG(img_loss_callable, weights=(0.5, 0.5))[source]¶
Bases:
object
Class turning image losses into dvf losses.
- class Grad(penalty='l1')[source]¶
Bases:
object
Class computing gradient loss with different hyperparameters.
- class NCC(win=None, eps=1e-05)[source]¶
Bases:
object
Class computing normalized cross correlations under different hyperparameters.
- class PerceptualLoss(model='net-lin', net='alex')[source]¶
Bases:
object
Class computing perceptual loss under different hyperparameters.
- cross_correlation(y_true, y_pred)[source]¶
Pearson cross correlation.
- Parameters
y_true (K.tensor) – Tensor of shape (None, h, w, 1) representing the ground truth image.
y_pred (K.tensor) – Tensor of shape (None, h, w, 1) representing the predicted image.
- Returns
cc – Scalar tensor representing the 1 - correlation.
- Return type
K.Tensor
- jacobian_distance(y_true, y_pred, norm='l2')[source]¶
Compute average per pixel distance between jacobians.
- Parameters
y_true (K.tensor) – True DVF.
y_pred (K.tensor) – Pred DVF.
norm (str, {'l1', 'l2'}) – Norm to use.
- psnr(y_true, y_pred)[source]¶
Compute the negative of PSNR.
- Parameters
y_true (K.tensor) – Tensor of shape (None, h, w, 1) representing the ground truth image.
y_pred (K.tensor) – Tensor of shape (None, h, w, 1) representing the predicted image.
- Returns
res – Scalar tensor representing the -psnr.
- Return type
K.Tensor
- ssim(y_true, y_pred, **kwargs)[source]¶
Compute the negative of structural similarity.
- Parameters
y_true (K.tensor) – Tensor of shape (None, h, w, 1) representing the ground truth image.
y_pred (K.tensor) – Tensor of shape (None, h, w, 1) representing the predicted image.
kwargs (dict) – Additional hyperparameters to be passed into the tensorflow implementation of ssim.
- Returns
res – Scalar tensor representing the -ssim.
- Return type
K.Tensor
References
[1] https://www.tensorflow.org/api_docs/python/tf/image/ssim
atlalign.ml_utils.models module¶
Module containing utilities for manipulation of keras models.
- load_model(path, compile=False)[source]¶
Load a model that uses custom atlalign layers.
The benefit of using this function as opposed to the keras equivalent is that the user does not have to care about how the model was saved (whether architecture and weights were separated). Additionally, all custom possible custom layers are provided.
- Parameters
path (str or pathlib.Path) – If path is a folder then the folder is expected to have one .h5 file (weights) and one .json (architecture). If path is a file then it needs to be an .h5 and it needs to encapsulate both the weights and the architecture.
compile (bool) – Only possible if path refers to a .h5 file.
- Returns
Model ready for inference. If compile=True then also ready for continuing the training.
- Return type
keras.Model
- merge_global_local(model_g, model_l, expose_global=False)[source]¶
Merge a global and local aligner models into a new one.
Seems to be also changing the input models.
- Parameters
model_g (keras.Model) – Model performing global alignment.
model_l (keras.Model) – Model performing local alignment.
expose_global (bool, optional) – If True, then model has 4 outputs where the last two represent image after global alignment and the corresponding dvfs. If False, then only 2 outputs : image after both global and local and the overall dvfs.
- Returns
model_gl – Model performing both the local and the global alignment.
- Return type
keras.Model
- replace_lambda_in_config(input_config, output_format='dict', verbose=False)[source]¶
Replace Lambda layers with full blown keras layers.
This function only exists because we pretrained a lot of models with 2 different Lambda layers and only after that realized that they cause issues during serialization. One can then use this function to just fix it.
Notes
To make this clear let us define the top dictionary as the one that has keys
‘backend’
‘config’
‘keras_version’
‘class_name’
The bottom dictionary is top[‘config’].
- Parameters
input_config (dict or str or pathlib.Path or keras.Model) – Config containing an architecture generated by one of the functions in atlalign.nn possibly containing Lambda layers.
output_format (str, {'dict', 'keras', 'json'}) –
What output type to use. See how below how to instantiate a model out of each of the formats
’dict’ - keras.Model.from_config
’json’ - kersa.models.model_from_json
’keras’ - already a model instance
verbose (bool) – If True, printing to standard output.
- Returns
output_config – Config containing an architecture of the input network but all Lambda layers are replaced by full blown operations.
- Return type
dict
- save_model(model, path, separate=True, overwrite=True)[source]¶
Save model.
- Parameters
model (keras.Model) – Keras model to be saved.
path (str or pathlib.Path) – Path to where to save the serialized model. If separate=True then it needs to represent a folder name. Inside of the folder 2 files are created - weights (.h5) and architecture (.json). If separate=False then an extension .h5 is added and architecture + weights are dumped into one file.
separate (bool) – If True, then architecture and weights are stored separately. Note that if False then one might encounter issues when loading in in a different Python environment (see references).
overwrite (bool) – If True, then possible existing files/folders are overwritten.
References
Module contents¶
Prevent 3 level imports for important objects.