Implement Deep Neuron Networks on VPipe Parallel System: A ResNet Variant Implementation

Document Type

Conference Proceeding

Publication Date

1-1-2023

Abstract

Deep Neuron Networks (DNNs) rapidly develop across numerous industries and fields. The Residual Network (ResNet) is prevalent for its high performance and novel residual network for image classification. One new architecture derived from ResNet, the ResNet-RS, is proving to have higher accuracy while having lower training difficulty. Furthermore, with the growing network depth and expanding training set size, it is vital to apply different parallel methods to the ResNet to improve the training speed. To evaluate the parallel efficiency of the new architecture, this paper made the following efforts: (1) Implemented the ResNet-RS network on VPipe with a 2 GPU environment, (2) proposed a formal procedure for DNN implementation on VPipe, and (3) Compared the top1 and top5 accuracy, loss, and epoch time of training between an orthodox ResNet and the derived ResNet-RS on VPipe parallel system.

Publication Title

Proceedings of SPIE - The International Society for Optical Engineering

DOI

10.1117/12.2671359

This document is currently not available here.

Share

COinS