Diagnosing fashion outfit compatibility with deep learning techniques

Fashion image understanding is a popular research field with many different machine learning applications. There have been many studies regarding outfit prediction and outfit composition in the field of fashion. However, there are few works that explain the prediction. This paper addresses a method of diagnosing outfit compatibility through clothing images. The proposed system not only predicts compatibility, but also diagnoses incompatible clothing items in outfits. First, a new dataset named ModAI, which has clothing images and compatibility comments from different users was created. After this, a common compatibility comment was created according to user comments for each clothing image. Lastly, image captioning techniques were used to generate compat-ibility suggestion texts from clothing images. Different segmentation techniques were also used to improve captioning capabilities. The model achieves a 0.62 BLEU-4 score. Experiments show that image captioning techniques can also be used to diagnose outfit compatibility.