This paper describes how the Big Data Research Center of Kyung Hee University and Benple Inc. developed and deployed an artificial intelligence system to automate the quality management process for Frontec, an SME company that manufactures automobile parts. Various constraints, such as response time requirements and the limited computing resources available, needed to be considered in this project. Defect finders using large-scale images are expected to classify weld nuts within 0.2 s with an accuracy rate of over 95%. Our system uses Circular Hough Transform for preprocessing as well as an adjusted VGG (Visual Geometry Group) model. Our convolutional neural network (CNN) system shows an accuracy of over 99% and a response time of about 0.14 s. To embed the CNN model into the factory, we reimplemented the preprocessing modules using LabVIEW and had the classification model server communicate with an existing vision inspector. We share our lessons from this experience by explain-ing the procedure and real-world issues developing and embedding a deep learn-ing framework in an existing manufacturing environment without implementing any hardware changes.