This research presents a two-stage computer-aided diagnosis (CAD) system that employs a convolutional neural network (CNN) with selfsupervised learning (SSL) to classify polyps as either hyperplastic polyps (HP) or tubular adenomas (TA). The proposed model integrates lookinto-object (LIO) and contrastive learning in SimCLR to focus on the comprehensive polyp region, enhancing model performance. The
LIO scheme, however, depends on pretraining a model to provide basic representations, which is improved using a warm-up scheme to optimize the loss function. Due to the scarcity of medical images for effective polyp classification training, an alternative approach utilizes natural images for the pretext task instead of polyp images. The research findings demonstrate that the proposed scheme, which leverages polyp object structure information and self-supervised learning, produces a robust model that improves classification accuracy for HP and TA in the prediction head by transferring a
backbone. The backbone model effectively employs ResNet-18 to focus on the holistic polyp using a limited number of labeled polyp images. The proposed scheme surpasses existing methods, achieving a 4% increase in accuracy and a 3% improvement in the F1-score.