Bayesian Methods Research Group

There is an emerging interest in the application of deep learning models to source code processing tasks. One of the major problems in applying deep learning to software engineering is that source code often contains a lot of rare identifiers resulting in huge vocabularies. We propose a simple yet effective method based on identifier anonymization to handle out-of-vocabulary (OOV) identifiers. Our method can be treated as a preprocessing step and therefore allows an easy implementation. We show that the proposed OOV anonymization method significantly improves the performance of the Transformer in two code processing tasks: code completion and bug fixing.