To set the stage: what is Vitis AI? It’s essentially a framework for developing and deploying machine learning models using Xilinx FPGAs (Field Programmable Gate Arrays). If you’re not familiar with FPGAs, they’re basically like Lego blocks for electronics you can program them to do all sorts of cool stuff. And when it comes to AI, that means we can use Vitis AI to train and run our models on these devices, which is a huge deal because it opens up new possibilities for real-time inference at the edge (i.e., closer to where the data is being generated).
So how do you get started with Vitis AI? Well, first you’ll need to download and install the software package from Xilinx’s website. This will give you access to all of the tools and libraries that you’ll need for developing your own models using this framework. Once you have everything set up, it’s time to start playing around with some code!
Here’s a basic example: let’s say we want to train a simple image classification model using Vitis AI. First, we’ll load in our dataset (in this case, I’m going to use the CIFAR-10 dataset) and preprocess it so that each image is resized to 32×32 pixels:
# Import necessary libraries
import numpy as np
from sklearn.preprocessing import LabelEncoder
from keras.utils import normalize
# Load in our data
train_data = np.load('cifar-10-batches-py/data_batch_1') # Load training data from file
test_data = np.load('cifar-10-batches-py/test_batch') # Load test data from file
# Preprocess the images and labels
def preprocess(x):
x -= 128 # Subtract mean pixel value (which is around 127) to center the data around 0
x /= 128. # Normalize to range of -1 to +1 by dividing by 128
return np.expand_dims(x, axis=-1).astype('float32') # Expand dimensions to add a channel dimension and convert to float32 data type
# Load in our labels and convert them to one-hot encoding format
def load_labels():
with open('cifar-10-batches-py/batches.meta', 'r') as f: # Open file containing label information
lines = [line for line in f if 'label' in line] # Read lines containing label information
label_indices = np.array([int(x.split()[0]) for x in lines], dtype=np.uint8) # Extract label indices from lines and convert to unsigned integer data type
encoder = LabelEncoder() # Create label encoder object
labels = encoder.fit_transform(label_indices) # Fit label encoder to label indices and transform them to one-hot encoding format
return labels # Return encoded labels
# Preprocess our data and convert it to a format that Vitis AI can use
train_data = preprocess(train_data) # Preprocess training data
test_data = preprocess(test_data) # Preprocess test data
labels = load_labels() # Load and preprocess labels
Once we have our data prepped, we’ll create a new model using the Keras framework (which is included in Vitis AI):
# Import necessary modules from Keras
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
# Define our model architecture
model = Sequential() # Create a sequential model, which is a linear stack of layers
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3))) # Add a convolutional layer with 32 filters, each with a 3x3 kernel size and ReLU activation function. Specify input shape as 32x32x3 for RGB images.
model.add(MaxPooling2D((2, 2))) # Add a max pooling layer with a 2x2 pool size
model.add(Flatten()) # Flatten the output from the previous layer into a 1D vector
model.add(Dense(64, activation='relu')) # Add a fully connected layer with 64 neurons and ReLU activation function
model.add(Dense(10, activation='softmax')) # Add a fully connected layer with 10 neurons and softmax activation function for multiclass classification
And then we’ll compile and train our model using the Vitis AI framework:
# Import necessary libraries
from vitis_ai import ModelBuilder # Import ModelBuilder from vitis_ai library
import numpy as np # Import numpy library and alias it as np for easier use
# Load in our data
train_data = np.load('cifar-10-batches-py/data_batch_1') # Load training data from file
test_data = np.load('cifar-10-batches-py/test_batch') # Load test data from file
labels = load_labels() # Load labels for data
# Preprocess our data and convert it to a format that Vitis AI can use
train_data = preprocess(train_data) # Preprocess training data
test_data = preprocess(test_data) # Preprocess test data
# Define our model architecture
model = Sequential() # Create a sequential model
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3))) # Add a convolutional layer with 32 filters, 3x3 kernel size, relu activation function, and input shape of 32x32x3
model.add(MaxPooling2D((2, 2))) # Add a max pooling layer with 2x2 pool size
model.add(Flatten()) # Flatten the output from the previous layer
model.add(Dense(64, activation='relu')) # Add a dense layer with 64 units and relu activation function
model.add(Dense(10, activation='softmax')) # Add a dense layer with 10 units and softmax activation function
# Compile and train our model using Vitis AI
mb = ModelBuilder() # Create a ModelBuilder object
mb.compile_from_keras_model(model) # Compile the model using the ModelBuilder object
mb.train(x=np.array([train_data]), y=labels[0:5000], epochs=10, batch_size=32) # Train the model using the ModelBuilder object, passing in the training data, labels, number of epochs, and batch size
And that’s it! We just trained our first image classification model using Vitis AI. Of course, this is a very basic example there are all sorts of more advanced techniques and tools available for developing and deploying machine learning models using Xilinx FPGAs. But hopefully this gives you a taste of what’s possible with this exciting new technology!
So if you’re ready to dive into the world of Vitis AI, head on over to Xilinx’s website and start exploring all of the tools and resources that are available for developing your own models using this framework. And remember when it comes to artificial intelligence, the sky’s the limit!