BioBERT Based Efficient Clustering Framework for Biomedical Document Analysis

Abstract

The large volumes of biomedical documents have been generating exponentially in modern applications. Document clustering methods play an important role in gathering textual content documents into a few meaningful coherent groups. However, clustering unstructured and unlabeled text is challenging to extract informative representations and find the relevant articles from the rapid growth biomedical literature. Therefore, traditional text document clustering methods often represent unsatisfactory results due to general non-contextualized vector space representations, which neglect the semantic relation between bio medical texts. Pre-trained language models have been gaining attention recently in variety of natural language processing tasks. In this paper, we propose a heavily pre-trained language representation BioBERT based clustering framework for biomedical document analysis in order to improve the clustering accuracy. In experimental architecture, we provide benchmarks of the pre-trained transformer model, statistical technique and word-embedding methods while incorporating with clustering algorithms. In order to distinguish the efficiency of the models, Fowlkes mallows score (FM), silhouette coefficient (SC), adjusted rand index (ARI), Davies-Bouldin score (DB) metrics are used. The comprehensive experimental results show that the BioBERT based K-means model achieves better clustering accuracies than other models.