Implement Deep Neuron Networks on VPipe Parallel System: A ResNet Variant Implementation
Deep Neuron Networks (DNNs) rapidly develop across numerous industries and fields. The Residual Network (ResNet) is prevalent for its high performance and novel residual network for image classification. One new architecture derived from ResNet, the ResNet-RS, is proving to have higher accuracy while having lower training difficulty. Furthermore, with the growing network depth and expanding training set size, it is vital to apply different parallel methods to the ResNet to improve the training speed. To evaluate the parallel efficiency of the new architecture, this paper made the following efforts: (1) Implemented the ResNet-RS network on VPipe with a 2 GPU environment, (2) proposed a formal procedure for DNN implementation on VPipe, and (3) Compared the top1 and top5 accuracy, loss, and epoch time of training between an orthodox ResNet and the derived ResNet-RS on VPipe parallel system.
Proceedings of SPIE - The International Society for Optical Engineering
Zhu, Xinrong, "Implement Deep Neuron Networks on VPipe Parallel System: A ResNet Variant Implementation" (2023). Kean Publications. 378.