## 1 Introduction  
As artificial intelligence (AI) technology develops and begins to be applied to various fields, various services using AI technology are being developed. Machine learning (ML), which is the core of AI technology, has been mostly researched in the direction of improving inference accuracy. AI models with high accuracy require vast amounts of data and parameters. OpenAI's generative pre-trained transformer 3 (GPT-3) model, widely used in the field of natural language processing, has used a total of 175 billion parameters and about 500 billion learning data to develop the model [1]. AI models should be provided to customers so that they can be applied in real life. To provide AI technology at a price such that it can be released as a service or product, technologies such as TensorFlow Serving [2], TorchServe [3], the Nvidia Transition Server [4], and Intel OpenVINO [5] have been studied in recent years to efficiently provide AI services focusing on inference rather than learning. Edge computing has attracted a lot of attention because it can efficiently operate complex AI services such as object detection and mobile object tracking by offloading computing and processing data to a cloud and edge devices [6; 7; 8; 9]. By applying edge computing to object detection, performance can be improved, which can be used to reduce inference latency and energy consumption and increase inference accuracy [9; 10]. To maximize performance through edge computing, an offloading policy that determineswhich edge or cloud devices to execute the workload plays a very important role, and inference latency, communication latency, and energy consumption should be considered to derive an optimal offloading policy. However, since inference latency is affected by the characteristics of inference models, devices, and images, it is difficult to derive an offloading policy by reflecting the actual inference latency in practice. Therefore, it is required to derive an accurate offloading policy that takes into account the predicted inference latency. In this paper, we introduce a resource optimization approach in terms of the inference latency of object detection in edge computing from a system perspective for an efficient AI service by using the TensorFlow serving system that is closely related to the ML model deployment of AI models. Inference latency is a very important factor to be considered in AI service systems, and finding optimal offloading policies in terms of inference latency when edge computing is considered is a very challenging problem. Information on the inference latency is essential to find the optimal offloading policy in the scheduling phase, and in many studies [11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21], inference latency has been set to be proportional to file size. However, in object detection, there is no correlation between the file size and inference latency, and the inference latency is determined by the characteristics of the image, requiring additional processes to calculate the inference latency. We propose two inference latency prediction approaches for object detection. The first method is aimed at inference latency prediction for the client, edge, and cloud devices, and the second method is focused on inference latency prediction for the object detection algorithms by using their statistical information on inference latency. By employing the predicted inference latency, the client determines whether to perform the object detection task locally or remotely on the edge and cloud for the optimal offloading policy. For the implementation of AI service systems, the region-based fully convolutional network (RFCN) [22] and single-shot multibox detector (SSD-MobileNet) [23] algorithms are considered as the two-stage and single-stage object detectors, respectively, and the representational state transfer (REST) [24; 25] and Google remote procedure call (gRPC) [26] protocols are used as communication protocols.