Imagine you’re trying to build a house, but it’s taking forever because you can only use one worker at a time. That’s the struggle researchers face when training deep neural networks like CNNs. It’s a slow and computationally intensive process that can take weeks. But fear not! A team of scientists has developed a game-changing solution: a hybrid approach that combines the power of data parallelism and model parallelism. It’s like having multiple workers with their own specialized tasks working together in harmony, making the training process much faster without sacrificing accuracy. To make things even more exciting, they’ve also come up with a brand new activation function called NNLU, which adds a crucial touch of non-linearity to the model without causing overfitting. Plus, they replaced the bulky fully connected layers with Global Average Pooling layers, boosting both accuracy and computational performance. This groundbreaking CNN model was put to the test with a challenging bio-medical image dataset and achieved an astonishing accuracy of 98.89% in just 1 second of training time. It outperformed other state-of-the-art techniques in terms of both classification accuracy and speed. If you’re ready to dive deeper into this revolutionary research, check out the full article!

With the increasing demand for deep learning in the last few years, CNNs have been widely used in many applications and have gained interest in classification, regression, and image recognition tasks. The training of these deep neural networks is compute-intensive and takes days or even weeks to train the model from scratch. The compute-intensive nature of these deep neural networks sometimes limits the practical implementation of CNNs in real-time applications. Therefore, the computational speedup in these networks is of utmost importance, which generates interest in CNN training acceleration. Much research is going on to meet the computational requirement and make it feasible for real-time applications. Because of its simplicity, data parallelism is used primarily, but it performs badly sometimes. In most cases, researchers prefer model parallelism to data parallelism, but it is not always the best choice. Therefore, in this study, we implement a hybrid of both data and model parallelism to improve the computational speed without compromising accuracy. There is only a 1.5% accuracy drop in our proposed study with an increased speed up of 3.62X. Also, a novel activation function Normalized Non-linear Activation Unit NNLU is proposed to introduce non-linearity in the model. The activation unit is non-saturated and helps avoid the model’s over-fitting. The activation unit is free from the vanishing gradient problem. Also, the fully connected layer in the proposed CNN model is replaced by the Global Average Pooling layers (GAP) to enhance the model’s accuracy and computational performance. When tested on a bio-medical image dataset, the model achieves an accuracy of 98.89% and requires a training time of only 1 s. The model categorizes medical images into different categories of glioma, meningioma, and pituitary tumor. The model is compared with existing state-of-art techniques, and it is observed that the proposed model outperforms others in classification accuracy and computational speed. Also, results are observed for different optimizers’, different learning rates, and various epoch numbers.

Read Full Article (External Site)