Open Neural Network Exchange

The Basics

KnetOnnx reads an ONNX file and creates the corresponding Model in Knet that can be re-designed, re-trained or simply used for inference.

If you are planning to move your models from PyTorch or Tensorflow to Knet, or simply desiring to play with popular pre-trained neural networks: KnetOnnx provides that functionality.

Open Neural Network Exchange (ONNX) is a community project created by Facebook and Microsoft. It provides a definition of an extensible computation graph model, as well as definitions of built-in operators and standard data types.

Operators are implemented externally to the graph, but the set of built-in operators are portable across frameworks. Every framework supporting ONNX will provide implementations of these operators on the applicable data types.

Although not all operations are implemented yet, visit ONNX's model zoo to download pre-trained, state-of-the-art models in the ONNX format.

Once you download the ONNX file, call KnetModel() with the ONNX file's path to create the model.

Tutorial

Check out our tutorial notebooks to learn how you can transfer your models from PyTorch to Knet:

MLP - CNN - VGG16

Here is how you create the Knet model corresponding to an ONNX file and perform a forward pass:

using KnetOnnx

#provide the ONNX file's path
model = KnetModel("vgg16.onnx");

#dummy input for prediction
x = ones(224,224,3,10)

#call KnetModel object with the model input
model(x) #the output is a 1000×10 Array{Float32,2}

Supported Operations

• [x] ReLU
• [x] LeakyReLU
• [x] Conv
• [x] MaxPool
• [x] Dropout
• [x] Flatten
• [x] Gemm
• [x] Add
• [x] BatchNormalization
• [x] ImageScaler
• [x] RNN
• [x] Unsqueeze
• [x] Squeeze
• [x] Concatenate
• [x] ConstantOfShape
• [x] Shape
• [x] Constant
• [x] Mul
• [x] SoftMax
• [x] Sub
• [x] Exp
• [x] Transpose
• [x] Reshape
• [x] GlobalAveragePool

Collaboration

Here are some cool ideas if you want to collaborate:

• Export functionality. This could be done in one of 3 ways:

  • (1) By only exporting models that are impelemented as a KnetModel. These models have to use KnetLayers, so one could implement functions that convert KnetLayers into strings. These strings would then be combined to construct the model.onnx file. The structure of the model (inputs, outputs, connections) can be inferred from the KnetModel.

  • (2) By running a dummy input through the model and then collecting the Julia operations. These more primitive operations could then be turned into strings and combined according to the order of operations. This would be a completely novel project and does not depend on anything implemented by this package.

  • (3) A hybrid approach. Use (1) for KnetModels & functions that make use of KnetLayers. Use (2) if the layers/operations being used are unknown.

• Adding a new KnetLayer

• Adding a new Converter (ONNX Operator -> KnetLayer)

• Downloading & testing ONNX models from the zoo and sending bug reports.

• Trying to import your models from ONNX-complete frameworks.

• Writing tests for KnetLayers and Converters.

• Adding Type Constraints to converters. (See ONNX Operators for more info)

• ONNX RUNTIME (advanced)