vulcanai.models package

vulcanai.models.basenetwork module

Defines the basenetwork class.

class vulcanai.models.basenetwork.BaseNetwork(name, config, in_dim=None, save_path=None, input_networks=None, num_classes=None, activation=ReLU(), pred_activation=None, optim_spec={'lr': 0.001, 'name': 'Adam'}, lr_scheduler=None, early_stopping=None, criter_spec=CrossEntropyLoss(), device='cuda:0')

Bases: torch.nn.modules.module.Module

Defines the BaseNetwork object.

Parameters:
name : str
The name of the network. Used when saving the file.
config : dict
The configuration of the network module, as a dict.
in_dim : tuple
The input dimensions of the network. Not required to specify when the network has input_networks.
save_path : str
The name of the file to which you would like to save this network.
input_networks : list of BaseNetwork
A network object provided as input.
num_classes : int or None
The number of classes to predict.
activation : torch.nn.Module
The desired activation function for use in the network.
pred_activation : torch.nn.Module
The desired activation function for use in the prediction layer.
optim_spec : dict
A dictionary of parameters for the desired optimizer.
lr_scheduler : torch.optim.lr_scheduler
A callable torch.optim.lr_scheduler
early_stopping : str or None
So far just ‘best_validation_error’ is implemented.
criter_spec : dict
criterion specification with name and all its parameters.
device : str or torch.device
Sets the network module to the relevant device. If cuda available in the host console, “cuda:0” will be run which can be overriden.
Returns:
network : BaseNetwork
__init__(name, config, in_dim=None, save_path=None, input_networks=None, num_classes=None, activation=ReLU(), pred_activation=None, optim_spec={'lr': 0.001, 'name': 'Adam'}, lr_scheduler=None, early_stopping=None, criter_spec=CrossEntropyLoss(), device='cuda:0')

Define, initialize, and build the BaseNetwork.

criter_spec

Return the criterion specification.

Returns:
_criter_spec : dict
The criterion specification.
cross_validate(data_loader, k, epochs, average_results=True, retain_graph=None, valid_interv=4, plot=False, save_path=None)

Will conduct the test suite to determine model strength.

device

Return the relevant device associalted with network module.

Returns:
device : torch.device
Relevant device associalted with the network module.
early_stopping

Return the stopping rule.

Returns:
stopping_rule : str
The stoping rule
extra_repr()

Set the extra representation of the module.

fit(train_loader, val_loader, epochs, retain_graph=None, valid_interv=4, plot=False, save_path=None)

Train the network on the provided data.

Parameters:
train_loader : DataLoader
The DataLoader object containing the training data.
val_loader : DataLoader
The DataLoader object containing the validation data.
epochs : int
The number of epochs to train for.
retain_graph : {None, True, False}
Whether retain_graph will be true when .backwards is called.
valid_interv : int
Specifies the period of epochs before validation calculation.
plot : boolean
Whether or not to plot training metrics in real-time.
save_path : str
Path to save graphics at
Returns:
None
forward(inputs, **kwargs)

Perform a forward pass through the modules.

If the network is defined with num_classes then it contains a classification layer/network tail. The inputs will be passed through the networks and then through the classifier. If not, the input is passed through the network and returned without passing through a classification layer.

Parameters:
inputs : list(torch.Tensor)
The inputs to pass throught the network.
Returns:
output : torch.Tensor
forward_pass(data_loader, convert_to_class=False)

Allow the user to pass data through the network.

Parameters:
data_loader : DataLoader
DataLoader object to make the pass with.
convert_to_class : boolean
If true, list of class predictions instead of class probabilites.
Returns:
outputs : numpy.ndarray
Numpy matrix with the output. Same shape as network out_dim.
freeze(apply_inputs=False)

Freeze network weights so training doesn’t modify them.

Parameters:
apply_inputs : boolean
Whether to freeze all input networks recursively
Returns:
None
get_layers()

Return an ordered dict of all modules in this network (layers).

Returns:
layers : OrderedDict()
get_weights()

Return a dictionary containing a whole state of the module.

Returns:
weights : dict
A dictionary containing a whole state of the module.
is_cuda

Return boolean about whether the network is on cuda device (i.e gpu).

Returns:
is_cuda : boolean
Specifies whether the network is on gpu or not.
classmethod load_model(load_path, load_complete_model_stack=True)

Load the model from the given directory.

Parameters:
load_path : str
The load directory (not a file)
load_complete_model_stack : boolean
Whether to load all parent networks as well. Not yet implemented.
Returns:
network : BaseNetwork
A network object with all components intact.
lr_scheduler

Return the network lr_scheduler.

Returns:
lr_scheduler : torch.optim.lr_scheduler
name

Return the name.

Returns:
name : string
The name of the network.
run_test(data_loader, plot=False, save_path=None)

Will conduct the test suite to determine model strength.

save_model(save_path=None)

Save the model (and it’s input networks).

Parameters:
save_path : str
The save directory (not a file)
Returns:
save_path : str
The save path where you’ll find the model directly.
save_path

Return the save path of the network.

Returns:
save_path : string
The save path of the network.
unfreeze(apply_inputs=False)

Unfreeze network weights so training does modify them.

Parameters:
apply_inputs : boolean
Whether to unfreeze all input networks recursively
Returns:
None

vulcanai.models.cnn module

Defines the ConvNet class.

class vulcanai.models.cnn.ConvNet(name, config, in_dim=None, save_path=None, input_networks=None, num_classes=None, activation=ReLU(), pred_activation=None, optim_spec={'lr': 0.001, 'name': 'Adam'}, lr_scheduler=None, early_stopping=None, criter_spec=CrossEntropyLoss(), device='cuda:0')

Bases: vulcanai.models.basenetwork.BaseNetwork

Subclass of BaseNetwork defining a ConvNet.

Parameters:
name : str
The name of the network. Used when saving the file.
config : dict
The configuration of the network module, as a dict.
in_dim : tuple
The input dimensions of the network. Not required to specify when the network has input_networks.
save_path : str
The name of the file to which you would like to save this network.
input_networks : list of BaseNetwork
A network object provided as input.
num_classes : int or None
The number of classes to predict.
activation : torch.nn.Module
The desired activation function for use in the network.
pred_activation : torch.nn.Module
The desired activation function for use in the prediction layer.
optim_spec : dict
A dictionary of parameters for the desired optimizer.
lr_scheduler : torch.optim.lr_scheduler
A callable torch.optim.lr_scheduler
early_stopping : str or None
So far just ‘best_validation_error’ is implemented.
criter_spec : dict
criterion specification with name and all its parameters.
Returns:
network : ConvNet
A network of type BaseNetwork.
__init__(name, config, in_dim=None, save_path=None, input_networks=None, num_classes=None, activation=ReLU(), pred_activation=None, optim_spec={'lr': 0.001, 'name': 'Adam'}, lr_scheduler=None, early_stopping=None, criter_spec=CrossEntropyLoss(), device='cuda:0')

Define the ConvNet object.

class vulcanai.models.cnn.ConvNetConfig(raw_config)

Bases: object

Define the necessary configuration for a ConvNet.

__init__(raw_config)

Take in user config dict and clean it up.

Cleaned units is stored in self.units

Parameters:
raw_config : dict of dict
User specified dict

vulcanai.models.dnn module

Defines the DenseNet class.

class vulcanai.models.dnn.DenseNet(name, config, in_dim=None, save_path=None, input_networks=None, num_classes=None, activation=ReLU(), pred_activation=None, optim_spec={'lr': 0.001, 'name': 'Adam'}, lr_scheduler=None, early_stopping=None, criter_spec=CrossEntropyLoss(), device='cuda:0')

Bases: vulcanai.models.basenetwork.BaseNetwork

Subclass of BaseNetwork defining a DenseNet.

Parameters:
name : str
The name of the network. Used when saving the file.
config : dict
The configuration of the network module, as a dict.
in_dim : tuple
The input dimensions of the network. Not required to specify when the network has input_networks.
save_path : str
The name of the file to which you would like to save this network.
input_networks : list of BaseNetwork
A network object provided as input.
num_classes : int or None
The number of classes to predict.
activation : torch.nn.Module
The desired activation function for use in the network.
pred_activation : torch.nn.Module
The desired activation function for use in the prediction layer.
optim_spec : dict
A dictionary of parameters for the desired optimizer.
lr_scheduler : torch.optim.lr_scheduler
A callable torch.optim.lr_scheduler
early_stopping : str or None
So far just ‘best_validation_error’ is implemented.
criter_spec : dict
criterion specification with name and all its parameters.
Returns:
network : DenseNet
A network of type BaseNetwork.
__init__(name, config, in_dim=None, save_path=None, input_networks=None, num_classes=None, activation=ReLU(), pred_activation=None, optim_spec={'lr': 0.001, 'name': 'Adam'}, lr_scheduler=None, early_stopping=None, criter_spec=CrossEntropyLoss(), device='cuda:0')

Define the DenseNet object.

class vulcanai.models.dnn.DenseNetConfig(raw_config)

Bases: object

Defines the necessary configuration for a DenseNet.

__init__(raw_config)

Take in user config dict and clean it up.

Cleaned units is stored in self.units

Parameters:
raw_config : dict of dict
User specified dict

vulcanai.models.ensemble module

Contains all ensemble models.

class vulcanai.models.ensemble.SnapshotNet(name, template_network, n_snapshots=3)

Bases: vulcanai.models.basenetwork.BaseNetwork

Initialize snapshot ensemble given a template network.

A wrapper class for any Network inheriting from BaseNetwork to train the template network using Snapshot Ensembling.

Parameters:
name : str
String of snapshot ensemble name.
template_network : BaseNetwork
Network object which you want to ensemble.
n_snapshots : int
Number of snapshots in ensemble.
Returns:
network : SnapshotNet
__init__(name, template_network, n_snapshots=3)

Use Network to build model snapshots.

fit(train_loader, val_loader, epochs, retain_graph=None, valid_interv=4, plot=False)

Train each model for T/M epochs and controls network learning rate.

Collects each model in a class variable self.network

Parameters:
train_loader : DataLoader
Input data and targets to train against
val_loader : DataLoader
Input data and targets to validate against
epochs : int
Total number of epochs (evenly distributed between snapshots)
Returns:
None
forward(inputs, **kwargs)

Snapshot forward function.

Collect outputs of all internal networks and average outputs.

Parameters:
inputs : torch.Tensor
Input tensor to pass through self.
Returns:
output : torch.Tensor
save_model(save_path=None)

Save all ensembled network in a folder with ensemble name.

Parameters:
save_path : str
The folder path to save models in.
Returns:
None

vulcanai.models.layers module

Define the ConvUnit and DenseUnit.

class vulcanai.models.layers.BaseUnit(weight_init=None, bias_init=None, norm=None, dropout=None)

Bases: torch.nn.modules.container.Sequential

The base class for all layers.

Parameters:
weight_init : torch.nn.init
Torch initialization function for weights.
bias_init : int or float
A constant int or float to initialize biases with.
norm : str
‘batch’ for batch norm of ‘instance’ for instance norm.
dropout : float between 0-1
The probability of dropping out a feature during training.
Returns:
dense_unit : torch.nn.Sequential
A single fully connected layer.
__init__(weight_init=None, bias_init=None, norm=None, dropout=None)

Initialize a base unit.

class vulcanai.models.layers.ConvUnit(conv_dim, in_channels, out_channels, kernel_size, weight_init=None, bias_init=None, stride=1, padding=0, norm=None, activation=None, pool_size=None, dropout=None)

Bases: vulcanai.models.layers.BaseUnit

Define the ConvUnit object.

Parameters:
conv_dim : int
1, 2, or 3 representing spatial dimensional inputs.
in_channels : int
The number of incoming channels.
out_channels : int
The number of convolution kernels for this layer.
kernel_size : int or tuple
The size of the 1, 2, or 3 dimensional convolution kernel.
weight_init : torch.nn.init
Torch initialization function.
bias_init : int or float
A constant int or float to initialize biases with.
stride : int or tuple
The stride of the 1, 2, or 3 dimensional convolution kernel.
padding : int
Number of zero-padding on both sides per dimension.
norm : str
‘batch’ for batch norm of ‘instance’ for instance norm.
activation : torch.nn.Module
An activation function to apply after Linear unit.
pool_size : int
Max pooling by a factor of pool_size in each dimension.
dropout : float between 0-1
The probability of dropping out a feature during training.
Returns:
conv_unit : torch.nn.Sequential
A single convolution layer.
__init__(conv_dim, in_channels, out_channels, kernel_size, weight_init=None, bias_init=None, stride=1, padding=0, norm=None, activation=None, pool_size=None, dropout=None)

Initialize a single ConvUnit (i.e. a conv layer).

get_conv_output_size()

Calculate the size of the flattened features after conv.

class vulcanai.models.layers.DenseUnit(in_features, out_features, weight_init=None, bias_init=None, norm=None, activation=None, dropout=None)

Bases: vulcanai.models.layers.BaseUnit

Define the DenseUnit object.

Parameters:
in_features : int
The incoming feature size of a sample.
out_features : int
The number of hidden Linear units for this layer.
weight_init : torch.nn.init
Torch initialization function.
bias_init : int or float
A constant int or float to initialize biases with.
norm : str
‘batch’ for batch norm of ‘instance’ for instance norm.
activation : torch.nn.Module
An activation function to apply after Linear unit.
dropout : float between 0-1
The probability of dropping out a feature during training.
Returns:
dense_unit : torch.nn.Sequential
A single fully connected layer.
__init__(in_features, out_features, weight_init=None, bias_init=None, norm=None, activation=None, dropout=None)

Initialize a single DenseUnit (i.e. a dense layer).

class vulcanai.models.layers.FlattenUnit

Bases: vulcanai.models.layers.BaseUnit

Layer to flatten the input.

Returns:
flatten_unit : torch.Sequential
A flatten layer.
__init__()

Initialize flatten layer.

forward(x)

Maintain batch size but flatten all remaining dimensions.

class vulcanai.models.layers.InputUnit(in_channels, out_channels, bias=False)

Bases: vulcanai.models.layers.BaseUnit

InputUnit.

__init__(in_channels, out_channels, bias=False)

Initialize InputUnit.

forward(input)

Define forward for InputUnit.

vulcanai.models.metrics module

Defines the network test suite.

class vulcanai.models.metrics.Metrics

Bases: object

A class to calculate all metrics for a BaseNetwork.

Responsible for the test suite.

__init__()

Initialize the metrics class for a BaseNetwork.

static cross_validate(network, data_loader, k, epochs, average_results=True, retain_graph=None, valid_interv=4, plot=False, save_path=None)

Perform k-fold cross validation given a Network and DataLoader object.

Parameters:
network : BaseNetwork
Network descendant of BaseNetwork.
data_loader : torch.utils.data.DataLoader
The DataLoader object containing the totality of the data to use for k-fold cross validation.
k : int
The number of folds to split the training into.
epochs : int
The number of epochs to train the network per fold.
average_results : boolean
Whether or not to return results from all folds or just an average.
retain_graph : {None, boolean}
Whether retain_graph will be true when .backwards is called.
valid_interv : int
Specifies after how many epochs validation should occur.
plot : boolean
Whether or not to plot all results in prompt and charts.
save_path : str
Where to save all figures and results.
Returns:
results : dict
If average_results is on, return dict of floats. If average_results is off, return dict of float lists.
static extract_class_labels(in_matrix)

Reformat truth matrix to be the classes in a 1D array.

Parameters:
in_matrix : numpy.ndarray or torch.Tensor
One-hot matrix of shape [batch, num_classes].
Returns:
class_list : numpy.ndarray
1D class array.
static get_accuracy(targets, predictions)

Calculate the accuracy.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
Returns:
accuracy: array of np.float32
The accuracy.
static get_auc(targets, raw_predictions, num_classes, average=None)

Calculate the AUC. Note: raw_predictions and num_classes are required.

Parameters:
targets: numpy.ndarray of integers
The target values.
raw_predictions: numpy.ndarray of floats
The raw predicted values, not converted to classes.
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
f1: np.float32 or array of np.float32
The AUC.
static get_confusion_matrix_values(targets, predictions)

Calculate the tp, tn, fp, fn values given targets and predictions.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
Returns:
tp, tn, fp, fn: array of np.float32 with classes in sorted order
The true positive/negative, false positive/negative values.
static get_dice(targets, predictions, average=None)

Calculate the dice metric.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] Only None and ‘macro’ currently implemented. This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
dice: np.float32 or array of np.float32
The dice metric.
static get_f1(targets, predictions, average=None)

Calculate the f1 score.

Parameters:
targets: numpy.ndarray of integers
The target values
predictions: numpy.ndarray of integers
The predicted values
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
f1: np.float32 or array of np.float32
The f1 score.
static get_npv(targets, predictions, average=None)

Calculate the negative predictive value.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] Only None and ‘macro’ currently implemented. This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
npv: np.float32 or array of np.float32
The negative predictive value
static get_ppv(targets, predictions, average=None)

Calculate the positive predictive value.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] Only None and ‘macro’ currently implemented. This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
ppv: float32 or array of np.float32
The positive predictive value.
static get_score(targets, predictions, metrics='accuracy', average=None, class_converted=False)

Calculate the provided metrics given some targets and predictions.

Parameters:
targets: numpy.ndarray of integers
The target values
predictions: numpy.ndarray of integers or numpy.ndarray of floats
The predicted values
metrics: list of strings
The strings definition the “get_”… functions to call
average: string,
[None, ‘binary’ (default), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data.See scikit learn
class_converted: binary. default False
True: If raw_predictions have already been converted using extract_class_labels False: If raw_predictions are used
Returns:
metrics: dict
Values returned by the metric functions.
static get_sensitivity(targets, predictions, average=None)

Calculate the sensitivity.

Also referred to as recall, or the true positive rate.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
sensitivity: np.float32 or array of np.float32
The sensitivity.
static get_specificity(targets, predictions, average=None)

Calculate the specificity.

Parameters:
targets: numpy.ndarray of integers
The target values.
predictions: numpy.ndarray of integers
The predicted values.
average: string
[None, ‘binary’ (def), ‘micro’, ‘macro’, ‘samples’, ‘weighted’] This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data. See Scikit learn.
Returns:
specificity: np.float32 or array of np.float32
The specificity.
static run_test(network, data_loader, plot=False, save_path=None)

Will conduct the test suite to determine network strength.

Parameters:
data_loader : DataLoader
A DataLoader object to run the test with.
save_path : string
Folder to place images in.
plot: bool
Determine if graphs should be plotted in real time.
Returns:
results : dict

vulcanai.models.utils module

Define utilities for all networks.

vulcanai.models.utils.get_one_hot(in_matrix)

Reformat truth matrix to same size as the output of the dense network.

Parameters:
in_matrix : numpy.ndarray
The categorized 1D matrix
Returns:
one_hot : numpy.ndarray
A one-hot matrix representing the categorized matrix
vulcanai.models.utils.master_device_setter(network, device=None)

Convert network and input_networks to specified device.

Parameters:
network : BaseNetwork
network to be converted to the specified device.
device : str or torch.device
the desired device
vulcanai.models.utils.network_summary(network, input_size=None)

Returns the summary of shapes of all layers in the network :return: OrderedDict of shape of each layer in the network

vulcanai.models.utils.pad(tensor, target_shape)

Pad incoming tensor to the size of target_shape.

tensor must have same spatial dimenison as the target_shape. Useful for combining various conv dimension outputs and to implement ‘same’ padding for conv operations.

Parameters:
tensor : torch.Tensor
Tensor to be padded
target_shape : np.array
Final padded tensor shape [*spatial_dimensions]
Returns:
tensor : torch.Tensor
zero padded tensor with spatial dimension as target_shape
vulcanai.models.utils.print_model_structure(network, input_size=None)

Print the entire model structure.

vulcanai.models.utils.round_list(raw_list, decimals=4)

Return the same list with each item rounded off.

Parameters:
raw_list : float list
float list to round.
decimals : int
How many decimal points to round to.
Returns:
rounded_list : float list
The rounded list in the same shape as raw_list.
vulcanai.models.utils.selu_bias_init_(tensor, const=0.0)

SELU layer bias initialization function.

Function assigned to variable that will be called within _init_bias function to assign bias for selu.

Parameters:
tensor : torch.tensor
Bias tensor to be adjusted
const : float
Constant value to be assigned to tensor.
Returns:
torch.tensor
bias tensor with constant values.
vulcanai.models.utils.selu_weight_init_(tensor, mean=0.0)

SELU layer weight initialization function.

Function assigned to variable that will be called within _init_weights function to assign weights for selu.

Parameters:
tensor : torch.tensor
Weight tensor to be adjusted
mean : float
Mean value for the normal distribution
Returns:
torch.tensor
weight tensor with normal distribution
vulcanai.models.utils.set_tensor_device(data, device=None)

Convert list of data tensors to specified device.

Parameters:
data : torch.tensor or list
data to be converted to the specified device.
device : str or torch.device
the desired device
Returns:
data : torch.tensor or list
data converted to the specified device