Lottery ticket hypothesis : using deeper conv nets and on Atari games

Abstract

The lottery ticket hypothesis proposes that the over-parameterization of deep neural networks helps training by increasing the probability of a lucky subnetwork initialization being present rather than by helping the optimization process. This phenomenon suggests that initialization strategies for DNNs can be improved substantially, but the lottery ticket hypothesis has only been previously tested on MNIST and CIFAR-10 datasets with architectures- VGG19 and Resnet18. Here we evaluate whether winning ticket initializations exist in deeper convolutional neural network architectures and fully connected networks and also on reinforcement learning domain on atari games.