In this experiment we compare MNIST training performance on the LeNet-5 model with three different training algorithms with/without 4-bit quantization. The three training algorithms are backpropagation (BP), feedback alignment (FA), and directed feedback alignment (DFA). We chose the 4-bit quantization to establish a baseline reference for the proposed work on binarization, namely quantized DFA (QDFA). The network consists of 2 convolution networks followed by 3 fully-connected layers.
Model topology: CONV6_MaxPool_CONV16_MaxPool_FC120_FC84_FC10
The quantization library used was Xilinx Brevitas (https://github.com/Xilinx/brevitas), while the feedback alignment implementation used was based from this repository on Direct Random Target Projection (https://github.com/ChFrenkel/DirectRandomTargetProjection).