Chapter 12

AI Library

The core strength of Nevaarize for AI engineering

The stdlib ai module provides 50+ SIMD-accelerated functions for neural network development, tensor operations, and machine learning workflows.

import stdlib ai as ai

Tensor Creation

Function	Description	Example
`Zeros(n)`	Create array of zeros	`ai.Zeros(100)`
`Ones(n)`	Create array of ones	`ai.Ones(100)`
`RandN(n)`	Normal distribution (μ=0, σ=1)	`ai.RandN(100)`
`RandU(n, min, max)`	Uniform distribution	`ai.RandU(100, -1, 1)`

Example

import stdlib ai as ai

// Create tensors
zeros = ai.Zeros(10)
ones = ai.Ones(10)
normal = ai.RandN(10)
uniform = ai.RandU(10, -1, 1)

print("Zeros:", zeros)
print("Normal sample:", normal)

Element-wise Operations

Function	Description
`Add(a, b)`	Element-wise addition (SIMD)
`Sub(a, b)`	Element-wise subtraction
`Mul(a, b)`	Element-wise multiplication (SIMD)
`Div(a, b)`	Element-wise division
`Scale(a, scalar)`	Multiply by scalar

a = [1.0, 2.0, 3.0, 4.0]
b = [0.5, 0.5, 0.5, 0.5]

sum = ai.Add(a, b)      // [1.5, 2.5, 3.5, 4.5]
prod = ai.Mul(a, b)     // [0.5, 1.0, 1.5, 2.0]
scaled = ai.Scale(a, 2) // [2.0, 4.0, 6.0, 8.0]

Reduction Operations

Function	Description
`Sum(a)`	Sum of elements (SIMD)
`Mean(a)`	Average value
`Max(a)`	Maximum value
`Min(a)`	Minimum value
`Argmax(a)`	Index of maximum
`Argmin(a)`	Index of minimum

Matrix Operations

Function	Description
`Dot(a, b)`	Dot product (SIMD, 4B+ ops/sec)
`MatMul(A, B, m, k, n)`	Matrix multiply (cache-blocked SIMD)
`Transpose(M, rows, cols)`	Matrix transpose

Matrix Multiplication Example

import stdlib ai as ai

// 2x3 matrix A
A = [1, 2, 3,
     4, 5, 6]

// 3x2 matrix B
B = [7, 8,
     9, 10,
     11, 12]

// C = A * B (2x2 result)
C = ai.MatMul(A, B, 2, 3, 2)
print("Result:", C)  // [58, 64, 139, 154]

Activation Functions

All activations are applied element-wise:

Function	Formula
`ReLU(x)`	max(0, x) — SIMD accelerated
`LeakyReLU(x, α)`	x if x > 0 else αx
`Sigmoid(x)`	1 / (1 + e⁻ˣ)
`Tanh(x)`	Hyperbolic tangent
`Softmax(x)`	Numerically stable softmax
`GELU(x)`	Gaussian Error Linear Unit
`SiLU(x)`	Swish: x * sigmoid(x)

x = [-2, -1, 0, 1, 2]

relu = ai.ReLU(x)       // [0, 0, 0, 1, 2]
sigmoid = ai.Sigmoid(x) // [0.12, 0.27, 0.5, 0.73, 0.88]
softmax = ai.Softmax(x) // Probability distribution

Loss Functions

Function	Use Case
`MSELoss(pred, target)`	Regression
`L1Loss(pred, target)`	Robust regression
`BCELoss(pred, target)`	Binary classification
`CrossEntropyLoss(pred, idx)`	Multi-class classification
`HuberLoss(pred, target, δ)`	Outlier-robust regression

pred = [0.9, 0.1, 0.0]
target = [1.0, 0.0, 0.0]

mse = ai.MSELoss(pred, target)
print("MSE Loss:", mse)

// Cross-entropy for class index 0
logits = [2.0, 1.0, 0.1]
ce = ai.CrossEntropyLoss(logits, 0)
print("CrossEntropy Loss:", ce)

Neural Network Layers

Linear Layer

// y = Wx + b
input = [1.0, 2.0, 3.0]          // 3 features
weights = ai.XavierInit(3, 2)    // 3x2 weights
bias = ai.Zeros(2)               // 2 biases

output = ai.Linear(input, weights, bias, 2)
print("Linear output:", output)

Layer Normalization

x = [1.0, 2.0, 3.0, 4.0]
gamma = ai.Ones(4)
beta = ai.Zeros(4)

normalized = ai.LayerNorm(x, gamma, beta)
print("LayerNorm:", normalized)

Dropout

x = [1.0, 2.0, 3.0, 4.0, 5.0]
dropped = ai.Dropout(x, 0.5)  // 50% dropout
print("After dropout:", dropped)

Optimizers

SGD Update

weight = [1.0, 2.0, 3.0]
grad = [0.1, 0.2, 0.3]
lr = 0.01

newWeight = ai.SGDUpdate(weight, grad, lr)
print("Updated weight:", newWeight)

Adam Update

weight = [1.0, 2.0, 3.0]
grad = [0.1, 0.2, 0.3]
m = ai.Zeros(3)  // First moment
v = ai.Zeros(3)  // Second moment

result = ai.AdamUpdate(weight, grad, m, v, 0.001, 0.9, 0.999, 1)
newWeight = result[0]
newM = result[1]
newV = result[2]

Gradient Clipping

grad = [10.0, 20.0, 30.0]
clipped = ai.ClipGradNorm(grad, 1.0)
print("Clipped grad:", clipped)

Weight Initialization

Function	Use Case
`XavierInit(fanIn, fanOut)`	Tanh, Sigmoid activations
`HeInit(fanIn, fanOut)`	ReLU activations

// For ReLU network
w1 = ai.HeInit(784, 256)   // Input to hidden
w2 = ai.HeInit(256, 10)    // Hidden to output

// For Tanh network
w = ai.XavierInit(100, 50)

Utility Functions

Function	Description
`Shape(a)`	Get array length
`Flatten(a)`	Flatten nested arrays
`OneHot(idx, n)`	One-hot encoding

// One-hot encoding
label = 2
numClasses = 5
encoded = ai.OneHot(label, numClasses)
print(encoded)  // [0, 0, 1, 0, 0]

Model Building

Build neural networks using a sequential API:

Function	Description
`Sequential(layers)`	Create a sequential neural network model
`Layer(type, ...)`	Define a layer (linear, relu, softmax, etc.)

Layer Types

Type	Syntax	Description
`"linear"`	`Layer("linear", inputSize, outputSize)`	Fully connected layer
`"relu"`	`Layer("relu")`	ReLU activation
`"sigmoid"`	`Layer("sigmoid")`	Sigmoid activation
`"softmax"`	`Layer("softmax")`	Softmax output layer

import stdlib ai as ai

// Create a neural network: 4 inputs -> 8 hidden -> 3 outputs
model = ai.Sequential([
    ai.Layer("linear", 4, 8),
    ai.Layer("relu"),
    ai.Layer("linear", 8, 3),
    ai.Layer("softmax")
])

print("Model created with ID:", model)

Model Training

Function	Description
`train(model, xTrain, yTrain, config)`	Train model and return final loss
`predict(model, input)`	Run inference on input

Training Config

The config array: [epochs, learningRate, optimizer, loss]

optimizer: "sgd" or "adam"
loss: "mse" or "crossentropy"

// Training data
xTrain = [
    [0.0, 0.0, 0.0, 0.0],
    [1.0, 1.0, 1.0, 1.0]
]
yTrain = [0, 2]  // Class labels

// Train for 50 epochs with Adam, lr=0.01
config = [50, 0.01, "adam", "crossentropy"]
loss = ai.train(model, xTrain, yTrain, config)

print("Final loss:", loss)

// Run prediction
output = ai.predict(model, [0.5, 0.5, 0.5, 0.5])
predicted = ai.Argmax(output)
print("Predicted class:", predicted)

Model Persistence

Save and load trained models to .nmod files:

Function	Description
`saveModel(model, path)`	Save model to file, returns true on success
`loadModel(path)`	Load model from file, returns model ID or -1

Path Resolution

Paths are resolved relative to the source file, not the working directory. This matches the behavior of import statements.

// Save trained model (saved next to this script)
saved = ai.saveModel(model, "mymodel.nmod")
if (saved) {
    print("Model saved successfully!")
}

// Load model later
loaded = ai.loadModel("mymodel.nmod")
if (loaded >= 0) {
    output = ai.predict(loaded, [1.0, 0.0, 1.0, 0.0])
    print("Prediction:", ai.Argmax(output))
}

Complete Training Workflow

import stdlib ai as ai

// 1. Define model architecture
model = ai.Sequential([
    ai.Layer("linear", 4, 8),
    ai.Layer("relu"),
    ai.Layer("linear", 8, 3),
    ai.Layer("softmax")
])

// 2. Prepare training data
xTrain = [
    [0.0, 0.0, 0.0, 0.0],
    [0.0, 0.0, 1.0, 1.0],
    [1.0, 1.0, 0.0, 0.0],
    [1.0, 1.0, 1.0, 1.0]
]
yTrain = [0, 1, 1, 2]

// 3. Train the model
config = [50, 0.01, "adam", "crossentropy"]
loss = ai.train(model, xTrain, yTrain, config)

// 4. Test prediction
output = ai.predict(model, [1.0, 1.0, 1.0, 1.0])
print("Predicted:", ai.Argmax(output))

// 5. Save for deployment
ai.saveModel(model, "classifier.nmod")

Model CLI Commands

Nevaarize provides CLI commands for model training and inference:

Train a Model

nevaarize model train script.nva to model.nmod

This runs your training script and saves the model. The output path is resolved relative to the script location.

Run Inference

# View model info
nevaarize model run model.nmod

# Run inference with input
nevaarize model run model.nmod --input "[1.0, 0.0, 1.0, 0.0]"

Example output:

Loading model: model.nmod
Model info:
  Input size: 4
  Output size: 3
  Epochs trained: 50
  Final loss: 0.219

Running inference with input size: 4
Output: [0.05, 0.65, 0.30]
Prediction: 1

Performance

All SIMD-accelerated functions use AVX2 instructions:

Operation	Performance
Dot Product	4B+ ops/sec
Matrix Multiply	50+ GFLOPS
ReLU	2B+ ops/sec
Sum/Mean	4B+ ops/sec