Automatic Differentiation
Reverse-mode autograd engine that supports full broadcasting, tuple-axis reductions, and gradient accumulation — the backbone of every model.
Tensor with Autograd
Every Tensor tracks its computation history. Call .backward() to flow gradients back through the entire graph automatically.
Broadcasting Support
Full NumPy-compatible broadcasting in all forward and backward passes. Handles complex shapes including tuple-axis reductions needed for CNN global pooling.
no_grad and enable_grad context managers — disable autograd during inference. Also works as a decorator. The Trainer now uses no_grad automatically during validation for memory efficiency.
Layers & Activations
All layers subclass lm.Module. Mix and match in Sequential or build custom modules.
Linear
Fully-connected layer with configurable input/output features and optional bias.
Conv2d
2D convolution with im2col-based backward pass. Supports padding, stride, and multi-channel inputs.
BatchNorm1d / 2d
Normalizes activations per batch. Supports learnable gamma/beta, running stats, and train/eval modes.
MaxPool2d / AvgPool2d
Spatial pooling layers that reduce feature map dimensions. Configurable kernel and stride.
Dropout
Randomly zeroes elements during training. Automatically disabled in eval mode via model.eval().
Embedding
Learnable lookup table for integer indices. Ideal for NLP tasks and categorical features.
LSTM New v2.1
Long Short-Term Memory layer with single-cell and multi-step variants, multi-layer support, and dropout.
GRU New v2.1
Gated Recurrent Unit — lighter than LSTM with comparable performance for many sequence tasks.
Flatten
Reshapes feature maps to 1D vectors. Configurable start_dim for flexible CNN-to-FC transitions.
Activation Functions
Optimizers & LR Schedulers
Optimizers
LR Schedulers
Metrics & Monitoring
Classification Metrics
| Function | Description |
|---|---|
| accuracy() | Top-1 accuracy, 0–1 float |
| top_k_accuracy() | Top-k accuracy (e.g. top-5) |
| precision() | Macro/micro/per-class precision |
| recall() | Macro/micro/per-class recall |
| f1_score() | Macro/micro/per-class F1 |
| confusion_matrix() | (C, C) numpy array |
Regression Metrics
| Function | Description |
|---|---|
| r2_score() | R² coefficient of determination |
| mean_squared_error() | MSE between predictions and targets |
| mean_absolute_error() | MAE — outlier-robust error metric |
System Monitoring
SystemMonitor
Track CPU usage, RAM, temperature and health score on Raspberry Pi in real time. Use memory_trace context to find bottlenecks, and ModelProfiler to measure FLOPs and throughput.
Trainer & Callbacks
Skip the boilerplate. The Trainer handles training loops, validation, logging, and callbacks automatically.
lm.Trainer
Built-in Callbacks
EarlyStopping
Stop training when a metric stops improving. Configure patience, min_delta, and mode (min/max).
ModelCheckpoint
Automatically save the best model weights to disk during training. Monitors any metric.
LRSchedulerCallback
Integrate any LR scheduler into the Trainer callback system seamlessly.
History
Record all training metrics per epoch. Access via history['train_loss'] for plotting.
Pre-built Models
MicroMLP
Multi-layer perceptron for tabular and flat data. Configurable hidden layer sizes with built-in Dropout regularization.
MicroCNN
Lightweight CNN optimized for small images (32×32). Supports RGB and grayscale inputs with configurable num_classes.
TinyResNet
ResNet with residual connections for better gradient flow. Reduce base_filters to 8 for very constrained devices.
Weight Initialization
Uniform distribution scaled by fan-in and fan-out. Ideal for tanh/sigmoid layers.
Normal distribution variant of Xavier initialization.
He initialization for ReLU networks — accounts for the non-linearity.
Normal Kaiming init — works well for deep ReLU networks.
Orthogonal matrix init. Preserves gradient norms in deep networks.
Apply initialization to a whole module recursively with one call.