An encoder-decoder Transformer (English → French translation) written entirely by hand in C++17, trained on GPU through a home-made CUDA backend.
A from-scratch implementation of the Transformer architecture (“Attention Is All You Need”) for English → French translation: all the algebra, the layers, backpropagation, the Adam optimiser, the tokenizer and the GPU backend are written by hand, with no deep learning library.
At the core sits a reference CPU implementation where every layer (attention, multi-head, feed-forward, layer norm, embeddings, encoder/decoder blocks) exposes a forward and a backward pass derived and coded by hand, validated by gradient checking.
Anatomy of a block: self-attention then feed-forward, each followed by dropout, a residual connection and a LayerNorm. The decoder additionally inserts cross-attention whose keys and values come from the encoder output.
A device-resident CUDA backend reproduces exactly the same computations but keeps the model living and training entirely on the GPU (cuBLAS for matrix products, custom kernels for softmax, layer norm and embedding scatter), with every GPU layer validated by an equivalence test against its CPU counterpart. Moving to block-diagonal batching on the GPU took training throughput from roughly 30 to 900 examples per second.
Two implementations, one logic: every CPU layer (Matrix, explicit loops) has its GPU mirror (GpuMatrix, cuBLAS, CUDA kernels) built from the same weights. An equivalence test compares their outputs layer by layer, and an identical binary checkpoint format makes the two backends interchangeable.
The project also includes a byte-level BPE tokenizer, a data pipeline (parallel corpus, teacher forcing, batching, padding), binary weight checkpointing interchangeable between CPU and GPU, autoregressive generation (greedy, then beam search), and a thorough test suite: 140 cases and around 9,000 assertions.
A batch's complete journey: the corpus is tokenized and grouped into length-sorted batches on the host, then only token ids travel to the GPU. A region captured as a CUDA Graph chains encoder, decoder, loss and backpropagation there without returning to the host, before the Adam update — the mechanism behind the ×30 throughput gain.
To drive training runs and try the model without the command line, a companion application (TrainerApp) was built on top of the library: a C++17/ImGui graphical interface with Models, Configuration, Training and Inference tabs. The Training tab shows training and validation loss live, the BLEU-4 score and the translations generated at each epoch, alongside GPU memory and utilisation. It has since been extended to drive the CNN from the ConvNetwork project too, becoming a small multi-model training platform.
TrainerApp — Training tab: train/validation loss, BLEU-4 score (35 % at epoch 31) and translations generated live.TrainerApp — Inference tab: translations produced by the model, with request history.
TrainerApp drives transformer_lib in the same process for three uses: training (batches → graphed pass → Adam), evaluation (BLEU) and inference (beam search). The library relies on VRAM (DevicePool, CUDA Graphs, cuBLAS) and on disk checkpoints, and returns the metrics that feed the interface's curves.
Key points
Backpropagation derived and hand-coded for every layer, verified by gradient checking
Device-resident CUDA backend (cuBLAS, custom kernels) validated layer by layer against the CPU
BPE tokenizer, data pipeline and CPU/GPU binary checkpointing all written from scratch
Companion GUI (TrainerApp, ImGui): live training monitoring (loss, BLEU-4) and interactive inference
