Task-Oriented Multi-Bitstream Optimization for Image Compression and Transmission via Optimal Transport

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 13:30:24

Abstract: Image compression for machine vision exhibits various rate-accuracy performance across different downstream tasks and content types. 
An efficient utilization of constrained network resource for achieving an optimal overall task performance has thus recently attracted a growing attention. In this paper, we propose Tombo, a task-oriented image compression and transmission framework that efficiently identifies the optimal encoding bitrate and routing scheme for multiple image bitstreams delivered simultaneously for different downstream tasks. Specifically, we study the characteristics of image rate-accuracy performance for different machine vision tasks, and formulate the task-oriented joint bitrate and routing optimization problem for multi-bitstreams as a multi-commodity network flow problem with the time-expanded network modeling. To ensure consistency between the encoding bitrate and routing optimization, we also propose an augmented network that incorporates the encoding bitrate variables into the routing variables. To improve computational efficiency, we further convert the original optimization problem to a multi-marginal optimal transport problem, and adopt a Sinkhorn iteration-based algorithm to quickly obtain the near-optimal solution. Finally, we adapt Tombo to efficiently deal with the dynamic network scenario where link capacities may fluctuate over time. Empirical evaluations on three typical machine vision tasks and four real-world network topologies demonstrate that Tombo achieves a comparable performance to the optimal one solved by the off-the-shelf solver Gurobi, with a $5\times \sim 114\times$ speedup.

Main paper: 
 introduction: 
 To relieve contradictions between the ever-growing Internet traffic and constrained bandwidth of real-world communication networks, compressing data before delivering them for downstream tasks becomes an efficient and popular solution, which alleviates network congestion and reduces transmission delay [6, 7] . Among the Internet traffic globally, visual streams have accounted for over 65% of the total amount [27] . Thus, studies on the efficient compression and transmission for these visual data, such as images and videos, have recently attracted a lot of attention [5, 8, 24] .Traditional image-coding frameworks have evolved over decades, aiming to solve a rate-distortion optimization problem. In lossy image compression, this optimization can be achieved by a trade-off between the bitrate and distortion, where the distortion is usually measured by peak signal-to-noise ratio (PSNR) or multiscale structural similarity (MS-SSIM) [30] . However, as more and more image traffic is now used for machine vision tasks, such as classification and detection, rather than for human watching, how to achieve the optimal rate-accuracy performance for a specific machine vision task has become an emerging research direction. Along this, various end-to-end learnable compression systems have shown their superior performance for task-oriented image coding [6, 22] . For example, Cui et al. in [11] propose a rate-adjustable learned image compression framework, providing a continuously variable bitrate for image bitstream to adapt to time-varying network throughput.Unfortunately, these existing studies on variable bitstream image compression assume that the optimal encoding bitrate is known in advance or determined by the estimated link capacity, without taking into account the subsequent network transmissions. When multiple image bitstreams are requested to be delivered simultaneously over the network, transmission cost (e.g., delay, jitter or loss) may inevitably increase, leading to a degradation on the quality of service for downstream tasks [4, 24] . Note that traffic engineering (TE) techniques can find the optimal routing scheme for these bitstreams to achieve a desired transmission performance with given traffic demands [1] . However, separately employing the bitrate allocation for image compression and TE solution for routing results in a poor performance, or even negative effect [26] . To make a full utilization of network resources for the need of specific downstream tasks, it is thus imperative to identify a compressionand-transmission optimization scheme that jointly determines the optimal bitrate allocation and routing scheme for requested image bitstreams. Recently, steps [14, 26, 34] have been taken in joint optimization of the encoding rate and routing scheme for adaptive video streaming, which, however, ignore differences in the rate-accuracy performance between different machine vision tasks.In this paper, we propose Tombo, a task-oriented multi-bitstream optimization framework that determines the optimal image encoding bitrate and routing scheme for each transmission request, according to the downstream task requirements and network conditions. Specifically, we study the characteristics of image rateaccuracy performance for different machine vision tasks, with the widely used end-to-end image compression method. Then, we formulate the task-oriented multi-bitstream image compression and transmission problem as a multi-commodity flow (MCF) problem using a time-expanded network model. This problem optimizes the image bitrates and corresponding transmission paths, aiming to strike a trade-off between minimizing the overall network link utilization and maximizing the overall downstream task performance within a pre-defined transmission delay. Though off-theshelf solvers, e.g., Gurobi [15] and CPLEX [10] , can accurately solve this problem to get the optimal solution, they cannot scale to the growing number of network nodes and commodities. To make the solution feasible in practice, we further convert the MCF problem to a multi-marginal optimal transport problem, and smooth its feasible region by introducing an entropy regularization term. An extended version of the Sinkhorn algorithm is then adopted to obtain a near-optimal solution with high computational efficiency. For the dynamic network scenario with fluctuating link capacities over time, we further enhance Tombo to rapidly adapt to the network changes. Finally, we conduct experiments on three typical machine vision tasks and four real-world network topologies of different node sizes, demonstrating effectiveness of the proposed method in terms of the overall task performance and computational efficiency. Our main contributions can be summarized as follows.• We propose Tombo, a joint optimization framework that performs task-oriented rate adaptation and routing for the encoding and transmission of multiple image bitstreams.• We formulate the multi-bitstream image compression and transmission optimization problem as a multi-marginal optimal transport problem, and develop a computationally efficient algorithm based on Sinkhorn iterations to obtain a near-optimal solution.• For dynamic networks with fluctuating link capacities, we enhance Tombo to efficiently adapt to these changes by leveraging solutions obtained in the previous transmission period.• We evaluate Tombo on four real-world networks with three machine vision tasks. Compared to the state-of-the-art commercial solver Gurobi [15] , Tombo achieves a comparable overall performance and yields a 5× ∼ 114× speedup. 
problem formulation 2.1 image compression for machine vision: 
 Rate-distortion optimization is the core of lossy image compression. Classical image compression framework is usually built upon optimizing the objective function: L = 𝑅 + 𝜆𝐷, where 𝑅 is the bitrate of the compressed image, 𝐷 measures the distortion of the reconstructed image, and 𝜆 controls the trade-off between the rate and distortion. In an end-to-end learned compression framework for machine vision tasks, 𝐷 is defined as the task-oriented distortion between the reconstructed features and the ground truth, which is measured by specific quality metrics, such as the accuracy for classification tasks, or the mean average precision (mAP) for detection and instance segmentation [7, 29, 33] . For example, the rate-performance curves of lossy image compression in Fig. 1 indicate that the trade-off between the rate and accuracy/mAP varies across different downstream tasks, which also differs between various image categories. Fig. 2 (a) shows a common scenario where a sequence of compressed image signals are requested to be transmitted from a source (e.g., a monitor) to several targets to fulfill different tasks (e.g., classification or object detection). The complex network structure in the real world (e.g., existence of relays) offers multiple feasible paths for delivering these image bitstreams. In pursuit of the optimal task performance at targets, it is desirable to transmit as many bits as possible, since higher bitrates often yield better performance. However, simultaneously transmitting these image bitstreams over bandwidth-constrained networks may lead to network congestion, resulting in detrimental effects such as package loss, retransmission, and significant transmission delays that adversely impact the downstream tasks. On the other hand, traditional routing methods strive to optimize traffic allocation for given volumes of image bitstreams across available paths, to minimize the transmission cost caused by the traffic overload [26] . Thus, it is essential to strike a trade-off between the transmission cost and overall task performance for multi-bitstream compression-and-transmission scenarios.We focus on the task-oriented optimization of encoding bitrates and routing paths for multiple image bitstreams associated with different downstream tasks and various image categories. Specifically, we consider a communication network with constrained link capacities, represented as a directed graph G = (V, E), where nodes𝑣 𝑖 , 𝑖 = 1, • • • , |V | representthe network sites and directed edges 𝑒 𝑖 𝑗 = (𝑣 𝑖 , 𝑣 𝑗 ), 𝑖 ≠ 𝑗 denote the links between these sites. Note that we will omit the subscripts of 𝑒 for notational simplicity throughout this paper, i.e., 𝑒 𝑖 𝑗 written as 𝑒 in places where there is no ambiguity.For task-oriented image transmission, a source wants to send one or multiple images to a target to perform a machine vision task, which is referred to as traffic demand of this source-target pair. Here, we assume that all traffic demands transmitted via the network must be completed within a given duration of time Γ, i.e., the maximum transmission delay. Thus, the optimization variables consist of both the amount of bits that needs to be sent and the routing paths to complete these traffic demands. Subsequent transmission of new traffic demands can initiate only upon completion of previous ones.As shown in Fig. 2 (a), we consider a common scenario with some network sites serving as source nodes, which send different categories of images to some target nodes that receive these image bitstreams for implementing various machine vision tasks. We represent each image bitstream as a commodity with index 𝑙 = {1, • • • , 𝐿}, and denote the task index by 𝜃 = {1, • • • , Θ}. Under this setting, the joint bitrate allocation and routing optimization for multi-bitstream compression-and-transmission can be modeled as a multi-commodity flow (MCF) problem, where the objective is to find an appropriate encoding/transmission bitrate and routing strategy for each commodity to strike a balance between the network transmission cost and task performance with a given maximum transmission delay Γ. We assume that the transmission begins at time 𝑡 0 . Specifically, we adopt the average overall network link utilization within the transmission period [𝑡 0 , 𝑡 0 + Γ) as the metric of transmission cost, as widely used in the TE studies [26, 32] . Similar to the setting in [26] , we use a linear-form function to calculate the instantaneous link utilization of link 𝑒 ∈ E produced by all the commodities at time 𝑡 ∈ [𝑡 0 , 𝑡 0 + Γ), which can be expressed as: Ψ 𝑒 ( ì 𝑥 𝑒,𝑡 ) = 𝐿 𝑙=1 𝑥 𝑙 𝑒,𝑡 /𝑑 𝑒 𝑡 , where ì 𝑥 𝑒,𝑡 = {𝑥 1 𝑒,𝑡 , ..., 𝑥 𝐿 𝑒,𝑡 }, 𝑥 𝑙 𝑒,𝑡 denotes the transmission bitrate of the 𝑙-th commodity on link 𝑒 at time 𝑡, and 𝑑 𝑒 𝑡 represents the instantaneous bandwidth of link 𝑒 at time 𝑡. In addition, we represent the task performance of machine vision task 𝜃 w.r.t. the 𝑙-th commodity (i.e., image bitstream) by a convex function: Φ 𝑙,𝜃 (𝑟 𝑙 ) = -𝑎 𝑙,𝜃 /(𝑟 𝑙 -𝑏 𝑙,𝜃 ) + 𝑐 𝑙,𝜃 , where 𝑟 𝑙 is the received average bitrate of the 𝑙-th commodity at the target node within the transmission period [𝑡 0 , 𝑡 0 + Γ). The value of this task performance function increases with the increase of 𝑟 𝑙 , and the parameters 𝑎 𝑙,𝜃 , 𝑏 𝑙,𝜃 , 𝑐 𝑙,𝜃 can be fitted based on any existing task-oriented image compression method. Therefore, the objective function of our multi-bitstream image compression and transmission optimization problem within the transmission period [𝑡 0 , 𝑡 0 +Γ) can be mathematically expressed as:EQUATIONwhere 𝜆 ′ is a Lagrange multiplier that controls the trade-off between the transmission cost and task performance.Notably, is intractable to optimize the variables in Eq. ( 1) with a continuous time index. Inspired by [2, 16, 21, 31] , we thus introduce a discrete time-expanded network to reformulate the objective function and address the joint optimization problem in this paper.As illustrated in Fig. 2(c ), the time-expanded network replicates the original topology at each discretized time interval, which serves as a singular acyclic directed graph integrating both the network topology and temporal aspects seamlessly [19, 28] . A feasible path in the original network can be depicted as a series of directed edges within the time-expanded network. In this context, the total transmission period Γ is divided discretely into 𝜏 ∈ N + time intervals, with the duration of each interval expressed as a constant valueΔ𝜏 = Γ/𝜏 ∈ R + .Note that a smaller value of Δ𝜏 indicates a finegrained time division in constructing the time-expanded network, which leads to a more precise optimization for the objective in Eq. ( 1), but also at the cost of a higher computational overhead. In Fig. 2 (c), we highlight the routing paths from the source node 𝐴 to the target nodes 𝐵 and 𝐶 with bold arrows for the sake of clarity. The sent and received traffic volumes of all the three commodities within the period [𝑡 0 , 𝑡 0 +Γ) can be replaced with the traffic volumes on the source edges 𝑒 + and target edges 𝑒 -, respectively. Thus, the variable 𝑟 𝑙 in Eq. ( 1) can be explicitly determined by the variable𝑥 𝑙 𝑝 .The multi-bitstream image compression and transmission optimization problem with the time-expanded network modeling can thus be formulated as: denotes the average link utilization over path 𝑝 within the transmission period [𝑡 0 , 𝑡 0 +Γ) and d𝑝 [𝑖 ] is the average transmission capacity of link 𝑝 [𝑖] within the time interval 𝑡 0 to 𝑡 0 +𝑖Δ𝜏. In addition, E task 𝑒 -= {𝜃 𝑒 -} represents the set of tasks that will be conducted in the target edge 𝑒 -. Constraints in Eqs. (2b) and (2c) state the relationship between the routing paths 𝑥 𝑙 𝑝 and the traffic demands distributed at the source and target edges, where 𝑟 𝑙 𝑒 + is the allowed maximum traffic volume originating from the source edge 𝑒 + for the 𝑙-th commodity, and 𝑟 𝑙 𝑒 -is the total received volume of bitstream at the target edge 𝑒 - for the 𝑙-th commodity. Note that the optimized encoding bitrate of the 𝑙-th commodity is determined by the value of 𝑟 𝑙 𝑒 -, which can be alternatively derived from the values of 𝑥 𝑙 𝑝 . Constraint Eq. (2d) guarantees that the total traffic volume of all image bitstreams on an edge 𝑒 does not exceed its capacity d𝑒 at each time interval.EQUATIONEQUATIONEQUATIONThe optimization problem P1 can be accurately solved using off-the-shelf solvers, such as Gurobi [15] and CPLEX [10] . However, as the values of 𝐿, Θ, and |E | increase, the number of optimization variables in P1 grows exponentially [17] , resulting in a huge overhead in identifying the optimal solution. When new images are required to be compressed and transmitted, or in the practical scenarios where the network condition varies significantly, it is necessary to re-solve problem P1. Therefore, it is crucial to find a computationally efficient approach to solving this problem. 
methodology: 
 To address the scability issue, in this section, we propose a multimarginal optimal transport-based method to efficiently find a nearoptimal solution to the problem P1. Specifically, we formulate problem P1 as a discrete form of the multi-marginal optimal transport problem, and further introduce an entropy regularization term into the objective function to make the feasible region more smooth. Then, we adopt a variant of the Sinkhorn algorithm to quickly solve the multi-marginal optimal transport problem with only a limited number of iterations. Finally, we extend our method to the dynamic network scenarios where the link capacities fluctuate over time. The computation efficiency of our proposed Tombo algorithm is further enhanced by leveraging the correlation of solutions between two consecutive transmission periods, i.e., [𝑡 0 , 𝑡 0 + Γ) and [𝑡 0 + Γ, 𝑡 0 +2Γ).Optimal transport is a mathematical framework that focuses on finding the optimal transport plan to transfer mass from an initial distribution to a final distribution while considering the associated costs and constraints [25] . To represent the transport paths for the various image bitstreams, we introduce a tensor 𝑀 ∈ R 𝐿×𝑛 (𝜏 +2) + , which denotes the transport plan over 𝜏 time intervals in the timeexpanded network as shown in Fig. 2(c Thus, the elements of 𝑀 encompass all possible combinations of directed links within the network, though it is obviously not in accordance with the actual network topology. In order to eliminate infeasible solutions, we encode the topology in the objective function by assigning an infinitely high cost to the infeasible paths, rather than explicitly defining the set of feasible paths in constraints. As a result, we denote by 𝐶 ∈ R 𝐿×𝑛 (𝜏 +2) + the cost tensor of the transport plan, with its element 𝐶 𝑙, p defined as:EQUATIONwhere 𝐶 𝑢 𝑙, ē𝑝 𝑖 denotes the transmission cost of 𝑙-th commodity on link ē𝑖 for transport path 𝑝 at time interval 𝑖, and 𝐶 * ē𝑖 , ē𝑖+1 is determined by the original network topology, i.e., 𝐶 * ē𝑖 , ē𝑖+1 = 0 if link ē𝑖 is connected to link ē𝑖+1 in the original network and equals ∞ otherwise. Here, d ē𝑖 denotes the average capacity of link ē𝑖 , which is fixed within the 𝑖-th time interval.Then, the total transmission cost of transport plan 𝑀 is calculated as the inner product of tensor 𝐶 and 𝑀, i.e., < 𝐶, 𝑀 >= 𝐿 𝑙=1 p 𝐶 𝑙, p 𝑀 𝑙, p . Keeping the task performance term in the objective the same as in problem P1, problem P1 is then reformulated as a multi-marginal optimal transport problem:EQUATIONEQUATIONwhere the initial and final distributions are imposed on the joint projections of tensor 𝑀 on the corresponding two marginals 𝑃 -1,0 (𝑀), 𝑃 -1,𝜏+1 (𝑀) ∈ R 𝐿×𝑛 . The two matrices depict the traffic distribution of each image bitstream on the augmented network at the initiation and conclusion of transmission. We denote by [𝑃 -1,𝛽 (𝑀)] 𝑙, ē𝛽 the value of the (𝑙, idx 𝛽 )-th element of the matrix 𝑃 -1,𝛽 (𝑀), where 𝛽 = 1, • • • , 𝜏 + 1, and idx 𝛽 denotes index of the link ē𝛽 in the augmented network. The elements of 𝑃 -1,0 (𝑀) and 𝑃 -1,𝜏+1 (𝑀) are calculated as:[𝑃 -1,0 (𝑀)] 𝑙, ē0 = ∑︁ p, ē𝑝 1 = ē1 𝑀 𝑙, p , [𝑃 -1,𝜏+1 (𝑀)] 𝑙, ē𝜏+1 = ∑︁ p, ē𝑝 𝜏 +1 = ē𝜏+1 𝑀 𝑙, p .And the link-capacity constraint in Eq. ( 4c) is imposed on the projection on the 𝑖-th marginal 𝑃 𝑖 (𝑀) ∈ R 𝑛 . Similarly, the values of elements of 𝑃 𝑖 (𝑀) are calculated as:[𝑃 𝑖 (𝑀)] ē𝑖 = ∑︁ 𝑙 ∑︁ p, ē𝑝 𝑖 = ē𝑖 𝑀 𝑙, p ,which denotes the sum of traffic on link ē𝑖 of the augmented network at time interval 𝑖. The vector d ∈ R 𝑛 in Eq. (4c) represents the link capacities, and 𝑅 (-1,0) ∈ R 𝐿×𝑛 in Eq. (4b) denotes the maximum allowable traffic volume for each source node to transmit, which is associated with 𝑟 𝑙 𝑒 + in Eq. (2b). For the sake of brevity, we use 𝐺 (•) to denote the function 𝜆 ′ 𝐿 𝑙=1 ē𝜏+1 𝜃 Φ 𝑙,𝜃 (•) in Eq. (4a) in the following sections. An effective approach to solve the multi-marginal optimal transport problem is to smooth the original problem by adding an entropy regularization term [3] : 𝐷 (𝑀) = 𝑙, p (𝑀 𝑙, p log𝑀 𝑙, p -𝑀 𝑙, p +1). Introducing such an entropy term modifies the feasible region boundaries of the optimization problem, resulting in smoother boundaries and facilitating the identification of the optimal solution. As its regularization parameter 𝜖 approaches zero, the obtained solution converges towards the ground truth [25] . Consequently, we add the entropy-regularized term to the objective of problem P2, and the corresponding optimization problem is expressed as: EQUATIONwhere 𝜖 > 0 is the regularization parameter, making the objective function 𝜖-strongly convex. This problem can be solved via a variant of the computationally efficient method, i.e., Sinkhorn scheme [12] .Concretely, the optimal solution to problem P3 has been proved to take the formEQUATIONwhere ⊙ denotes the Hadamard product, and the elements of 𝐾 are calculated as 𝐾 𝑙, p = exp(-𝐶 𝑙, p /𝜖). Similar to the decomposition in Eq. ( 3), 𝐾 𝑙, p is decomposed as:EQUATIONIn addition, 𝑈 ∈ R 𝐿×𝑛 (𝜏 +2) can be decomposed as:EQUATIONwhere 𝑈 (-1,0) ∈ R 𝐿×𝑛 and 𝑢 𝑖 ∈ R 𝑛 can be iteratively updated.Here, we introduce two iterative variables 𝜓 𝑖 and 𝜙 𝑖 to simplify the calculation of the projections of tensor 𝑀, i.e.,𝑃 𝑖 (𝑀), 𝑃 -1,0 (𝑀) and 𝑃 -1,𝜏+1 (𝑀). Then, the projections of tensor 𝑀 are given by:EQUATIONwhere 𝑈 (-1,0) and 𝑢 𝑖 ∈ R 𝑛 can then be iteratively computed via:EQUATION)EQUATIONwhere operator ./ denotes the element-wise division. The other component 𝑈 (-1,𝜏+1) ∈ R 𝐿×𝑛 is updated via solving the equation:EQUATIONwhere (𝐺) * denotes the Fenchel conjugate of function 𝐺. Consequently, our proposed Sinkhorn iteration-based algorithm for quickly solving problem P3 is summarized in Algorithm 1. Without loss of generality, we define a tolerance threshold 𝜂 for the iterative variable 𝑈 (-1,𝜏+1) , in order to determine when to terminate the iterations illustrated in Algorithm 1, Lines 4-18. Let Δ𝑈 (-1,𝜏+1) be the difference in 𝑈 (-1,𝜏+1) between the values of two consecutive update iterations. If Δ𝑈 (-1,𝜏+1) ≤ 𝜂, the iterations in Algorithm 1 are terminated. While implementing our method, the value of 𝜂 Algorithm 1 Tombo: the Sinkhorn iteration of solving problem P3 1: Initialize 𝑢 1 , ..., 𝑢 𝜏 , 𝑈 (-1,0) , 𝑈 (-1,𝜏+1) .⊲ Initialization 2: 𝜓 𝜏 ← 𝑈 (-1,𝜏+1) 𝐾 * 𝑇 3: for 𝑖 = 𝜏 -1 to 0 do 4:𝜓 𝑖 ← (𝜓 𝑖+1 ⊙ 𝐾 𝑢 )diag(𝑢 𝑖+1 )𝐾 * 𝑇 ⊲ Obtain initial values of 𝜓 𝑖 5: end for 6: while Δ𝑈 (-1,𝜏+1) ≥ 𝜂 do Update 𝑈 (-1,0) by Eq. (10a) 8:𝜙 1 ← 𝑈 (-1,0) 𝐾 * 9:for 𝑖 = 1 to 𝜏 do Update 𝑈 (-1,𝜏+1) by Eq. (11) 14:𝜙 𝜏 ← 𝑈 (-1,𝜏+1) 𝐾 * 𝑇 15:for 𝑖 = 𝜏 to 1 do 16:𝜙 𝑖 -1 ← (𝜙 𝑖 ⊙ 𝐾 𝑢 )diag(𝑢 𝑖 )𝐾 * 𝑇 17:end for 18: end while 19: return 𝑢 1 , ..., 𝑢 𝜏 , 𝑈 (-1,0) , 𝑈 (-1,𝜏+1) 20: /* The solution to P3 can then be derived based on Eqs. ( 6)- (8) . */ can be customized to achieve varying trade-offs between the convergence rate and computational efficiency.When there is a change in network capacity, it is common to collect samples of the new link capacities and then re-solve the optimization problem from scratch for the new network scenario. To enhance the computational efficiency of our Tombo in these dynamic network scenarios, we provide an extended version Tombo-D, which is capable of handling fluctuations in network capacities.Specifically, we assume that the link capacities change dynamically between a transmission period 𝜌 1 = [𝑡 0 , 𝑡 0 + Γ) and its subsequent transmission period 𝜌 2 = [𝑡 0 + Γ, 𝑡 0 + 2Γ). As mentioned in Eqs. ( 3) and ( 7), the matrix 𝐾 𝑢 in solving problem P3 is associated with the link capacities. We denote by 𝐾 𝑢 𝜌 1 the value of 𝐾 𝑢 within the period 𝜌 1 , and 𝐾 𝑢 𝜌 2 = 𝐾 𝑢 𝜌 1 + Δ𝐾 𝑢 the value of 𝐾 𝑢 within the period 𝜌 2 . Meanwhile, it is shown in Algorithm 1 that the initial values of 𝜓 𝑖 is of significance to the convergence rate of variables 𝑢 𝑖 , 𝑈 (-1,0) and 𝑈 (-1,𝜏+1) . Therefore, it is intuitive to leverage the prior knowledge of 𝐾 𝑢 𝜌 1 and the associated variable 𝜓 𝜌 1𝑖 to speed up the iteration process of 𝜓 𝜌 2 𝑖 , and then the update of 𝑈 (-1,0) and 𝑈 (-1,𝜏+1) , rather than attempting to update their values starting from a random initial point. Referring to the details given in Appendix A, we can replace the initialization of 𝜓 𝜌 2 𝑖 in Algorithm 1, Line 4 with the following equation:EQUATION)Here, the values of 𝜓 𝜌 1𝑖 are re-used to establish good initial values that are close to the optimal values for the updates of 𝜓 𝜌 2 𝑖 . The improved initialization of variable 𝜓 𝑖 results in a reduced number of iterations for the process illustrated in Algorithm 1, Lines 4-18. 
performance evaluation 4.1 experiment setup: 
 We implement our proposedTombo and other comparison baselines, and evaluate their performance in terms of the overall task performance and computational efficiency with three typical machine vision tasks. All the experiments are conducted on a desktop equipped with a 12-core Intel Xeon E5-2620 Processor and 32GB DDR4 DRAM. The experiment settings are outlined as follows.4.1.1 Network Topologies. We conduct the experiments over four real-world network topologies, each with varying link capacities. The number of nodes, directed edges, selected source nodes and target nodes for these test networks are listed in Table 1 , while the corresponding network topologies are shown in Fig. 3(a) .For different categories of images, we employ the end-to-end learning task-oriented image compression framework in [7] to generate the rate-accuracy curves for the classification task, and rate-mAP curves for object detection and instance segmentation tasks. The test image dataset used in our experiments is COCO2017 [23] , where the average image size is 640 × 480 pixels. We fit the parameters of the function Φ 𝑙,𝜃 (𝑟 𝑙 ) by using the task performance results in different discrete bitrate values. Examples are illustrated in Fig. 3(b) . The average coefficient determination for parameter fittings is 𝑅 2 = 0.97, which verifies that the obtained parameters can well fit the actual data.We compare our method Tombo with the following four baselines: 1) Tombo-Gurobi: a method that solves problem P2 using the commercial solver Gurobi-v10.0.3 [15] , the threads of parallel computation of Gurobi is set to 10; 2) MCF-Gurobi [13] : a method that solves the original MCF problem in problem P1. Before starting the optimization, we use the Floyd-Warshall algorithm [9] to find the top-𝑘 shortest paths for each source-target pair, and only optimize the traffic on these paths to improve computational efficiency; 3) TM-Sinkhorn: a variant of Tombo, which fixes the transmitted bitrate of each image as the same value as the average bitrate of Tombo's solution, and then optimizes the routing strategy using the Sinkhorn scheme; 4) SRD-Sinkhorn: a variant of Tombo, which replaces the task-specific performance function Φ 𝑙,𝜃 (𝑟 𝑙 ) with a task performance function that depends only on the bitrate value.We first compare the comparison baselines in terms of three metrics, including 1) the total objective value: Total Obj. in Eq. (2a); 2) the transmission cost, which is calculated as Trans. cost = 𝐿 𝑙=1 𝑝 Ψ(𝑥 𝑙 𝑝 ); and 3) the surrogate task value:Task-surr. = -𝜆 ′ 𝐿 𝑙=1 Θ 𝜃 =1 [Φ 𝑙,𝜃 (𝑟 𝑙 ) -𝑐 𝑙,𝜃 ]. Specifically, the transmission cost indicates the transmission performance of the optimized routing strategy, while the surrogate task value measures the overall task performance across various images and downstream tasks. In addition, we measure the feasibility of the solutions generated by Tombo, through measuring the constraint violation on each link, i.e., violation= 𝑖 ∥ max(0, 𝑃 𝑖 (𝑀) -d)∥ 1 . Thus, a higher violation value indicates a higher probability that the obtained solution to the optimization problem is infeasible. For the computational efficiency, we estimate the wallclock time it takes for each comparison method to meet the termination criteria as the value of running time. Update 𝑈 (0,𝜏 ) by 0 = -𝑈 (0,𝜏 ) ⊙ 𝜙𝜏 + 𝜕(𝐺) * (-𝜖log(𝑈 0,𝜏 ))𝜙𝜏 -1 ← 𝑈 (0,𝜏 ) 𝐾 𝑇 15:for 𝑡 = 𝜏 -1 to 2 do 16:𝜙𝑡 -1 ← (𝜙𝑡 ⊙ 𝐾 𝑢 )diag(𝑢𝑡 )𝐾 𝑇 17:end for 18: end while 19: return 𝑢2, ..., 𝑢𝜏 -1, 𝑈 (0,1) , 𝑈 (0,𝜏 ) 4.1.1 Network Topologies. We conduct the experiments over four real-world network topologies, each with varying link capacities. The number of nodes, directed edges, selected source nodes and target nodes for these test networks are listed in Table 1 , while the corresponding network topologies are shown in Fig. 3 . 4.1.2 Rate-Performance Function. For different categories of images, we employ the end-to-end learning task-oriented image compression framework in [6] to generate the rate-accuracy curves for 4.1.3 Baselines. We compare our method ToMO with the following four baselines: 1) ToMO-Gurobi: a method that solves the problem P2 using the commercial solver Gurobi-v10.0.3 [13] , the threads of parallel computation of Gurobi is set to 10; 2) MCF-Gurobi: a method that solves the original MCF problem in problem P1. Before starting the optimization, we use the Floyd-Warshall algorithm [8] to find the top-𝑘 shortest paths for each source-target pair, and only optimize the traffic on these paths to improve computational efficiency;3) TM-Sinkhorn: a variant of ToMO, which fixes the transmitted bitrate of each image as the same value as the average bitrate of the ToMO solution, and then optimizes the routing strategy using the Sinkhorn scheme; 4) SRD-Sinkhorn: a variant of ToMO, which replaces the task-specific performance function Φ 𝑙,𝜃 (𝑟 𝑙 ) with a task performance function that only depends on the bitrate value.To evaluate the performance of ToMO, we first compare the comparison baselines in terms of three metircs, including the total objective value: Total Obj., the transmission cost, which is calculated as Trans. cost= 𝐿 𝑙=1 𝑝 Ψ(𝑥 𝑙 𝑝 ); and the surrogate task value:Task-surr. = -𝜆 ′ 𝐿 𝑙=1 Θ 𝜃 =1 [Φ 𝑙,𝜃 (𝑟 𝑙 ) -𝑐 𝑙,𝜃 ].The transmission cost indicates the transmission performance of the optimized routing strategy, and the surrogate task value measures the overall task performance across various images and downstream tasks. In addition, we measure the feasibility of the solved solutions for the proposed method by measuring the constraint violation on each link, i.e., violation= 𝑡 ∥ max(0, 𝑃𝑡 (𝑀) -𝑑)∥1. A high violation value indicates a high probability that the obtained solution to the optimization problem is infeasible. For computational efficiency, we estimate the wallclock time it takes for each comparison method to meet the termination criteria as the value of running time.4.1.5 Other Parameters. For fairness, we use the same experiment settings for all comparison methods. We set the weight of the objective function as 𝜆 ′ = 0.2, and the number of images that are simultaneously compressed and transmitted within the time interval Γ as 𝐿 = 9, 15, 27, 27 for network B4, Janetbackbone, Carnet, and UsCarrier, respectively. The corresponding total time step is set as 𝜏 = 7. 4.1.6 Other Parameters. We set the Lagrange multiplier of the objective function as 𝜆 ′ = 0.2, and the number of images that are simultaneously compressed and transmitted as 𝐿 = {9, 15, 27, 27} for network B4, Janetbackbone, Carnet, and UsCarrier, respectively. We empirically set the total time interval Γ as {7, 7, 8, 37} milliseconds for four test networks, respectively and let Δ𝜏 always be 1 millisecond. As for the proposed Tombo, we set the entropyregularized weight in problem P2 as 𝜖 = 0.001. The termination criteria for Gurobi is set as {1%, 1%, 1%, 10%} primal-dual gap for network B4, Janetbackbone, and Carnet, and UsCarrier, respectively. We set the tolerance threshold of 𝑈 (-1,𝜏+1) as 𝜂 = 0.5, and also set the maximum iteration number of the Sinkhorn iteration as {500, 1000, 1000, 1500} for network B4, Janetbackbone, Carnet and UsCarrier, respectively. In addition, we further investigate the impact of different Δ𝜏 on the performance of Tombo in Appendix B.We compare Tombo with other baselines in terms of the objective value and running time performances in various network topologies and downstream task settings. Table 2 shows the numerical results of all comparison methods over all the four test network topologies.Transport. In the smallscale test network scenario, e.g., B4, Tombo takes only 2.69 seconds (5× speedup) on average to obtain a traffic allocation which is nearly the same as the best-performing scheme Tombo-Gurobi. Tombo also outperforms Tombo-Gurobi and MCF-Gurobi in networks Janetbackbone and Carnet in terms of the running time, with a speedup of 7× and 6×, respectively. In addition, Tombo has a remarkably competitive performance in terms of total objective values compared to Tombo-Gurobi in four test network topologies, with an average solution gap of less than 2.5%. In the UsCarrier network, which has 158 nodes and 378 links, Tombo shows a significantly faster solving speed (114× speedup) compared to other comparison methods, while maintaining competitive performance, with the total objective gap less than 1.2%. The results indicate that the speedup performance compared to Tombo-Gurobi grows as the network scale increases.Furthermore, we observe that all comparison methods achieve a solution with a tiny violation value, indicating that the link overload is negligible. Moreover, Tombo obtains comparable results for the average bitrate and link utilization as Tombo-Gurobi. Note that MCF-Gurobi requires a longer time compared to Tombo-based approaches in order to achieve the solution, even when the time used to obtain preset pathways is excluded. This demonstrates the efficacy of converting MCF problems into multi-marginal optimal transport problems as it greatly speeds up problem-solving processes. Furthermore, MCF-Gurobi simply distributes traffic along the shortest 𝑘 paths for each source-target node pair, without fully using all the available network links. Consequently, there is a decline in the performance of the total objective value.Bitrate and Routing Optimization. We then investigate the impact on the overall task performance by introducing task-specific performance metrics and the joint optimization framework. Through comparison of the task performance achieved by Tombo with the baseline methods TM-Sinkhorn and SRD-Sinkhorn, it is demonstrated that Tombo achieves the superior performance in terms of the task performance over all networks. As shown in Fig. 4 , TM-Sinkhorn and SRD-Sinkhorn present a significant degradation in the overall task performance because they do not distinguish between different downstream tasks when optimizing bitrates. On the four network topologies, Tombo consistently iterative variables from the previous solution, we provide an initial starting point for new iterations that is close to optimum solution. 4 , Tombo with the smallest 𝜖 consistently yields the closest approximation to the optimal solution. However, achieving a feasible solution with a smaller 𝜖 requires more time due to insufficient iterations, resulting in a significant violation of capacity constraints. Nevertheless, with the increase in the number of iterations, the violation of link capacity constraints gradually diminishes. Moreover, as 𝜖 increases, the time required for each iteration decreases. With a larger 𝜖, the objective value remains the same as the number of iterations increases, indicating that a higher 𝜖 fosters a swifter convergence of Tombo. This phenomenon can be attributed to the smoother boundaries of feasible regions. While Tombo with a large 𝜖 can swiftly compute a feasible solution, it deviates notably from the optimum. Notably, in comparison to B4, which comprises only 12 nodes, the disparity between the solution and the optimum is more pronounced in larger-scale networks like Janetbackbone. It is seen that Tombo-D can expedite iterative convergence and save computational time across different proportions of varying links in the two test scenarios. This phenomenon can be attributed to the effective initialization of the iteration variables, facilitating faster convergence and the discovery of optimal solutions. Nonetheless, as the value of 𝜌 vl rises, the advantage in running time diminishes. This phenomenon is reasonable since in the extreme scenario where all link capacities change, a completely new optimization problem emerges. As shown in Figs. 5(c) and 5(d), Tombo-D demonstrates a notable acceleration across all levels of link capacity fluctuations. Nevertheless, as network link capacities undergo more pronounced changes, the running time advantage diminishes. This phenomenon arises because the starting point of iterations in the feasible region becomes further away from the updated optimal solution. 
conclusion: 
 We have proposed a framework for jointly optimizing the encoding bitrates and routing scheme for multiple image bitstreams with various machine vision tasks, named Tombo. The task-oriented multi-bitstream compression and transmission problem was formulated as an MCF problem with time-expanded network modeling. To quickly solve the problem, we re-formulated the MCF problem to a multi-marginal optimal transport problem and proposed a Sinkhorn iteration-based algorithm to speed up the solving process. Then, we proposed the enhanced Tombo-D to adapt to dynamic networks where link capacities fluctuate over time. Evaluations on three typical machine vision tasks and four real-world network topologies have demonstrated the effectiveness of Tombo and Tombo-D.  
 
 Appendix:  


Decision: This paper presents a task-oriented image compression and transmission framework  that exposes encoding information at the routing layer. All the reviewers appreciate the technical contribution of the paper and find its contribution valuable. The evaluations range from weak reject to weak accept. Reviewers find the related work and the proper context for the paper to be somewhat missing. I recommend acceptance based upon the solid theoretical foundations and valuable proposal of this paper.

Reviews: review: paper_summary: paper_summary: The paper proposed a task-oriented image compression and transmission framework called Tombo, which efficiently identifies the optimal encoding bitrate and routing scheme for multiple image bitstreams delivered simultaneously for different downstream tasks.
strength_weakness: strength_weakness: One strength of this paper is the comprehensive approach taken to address the task-oriented multi-bitstream optimization problem in the context of image compression and transmission for machine vision tasks. The paper proposes the Tombo framework, which efficiently identifies the optimal encoding bitrate and routing scheme for multiple image bitstreams delivered simultaneously for different downstream tasks.
Specifically, the paper formulates the optimization problem as a multi-commodity flow problem with time-expanded network modeling, taking into account the characteristics of image rate-accuracy performance for different machine vision tasks. The use of a multi-marginal optimal transport approach and the development of a Sinkhorn iteration-based algorithm contribute to the quick and near-optimal solution finding process.
Furthermore, the paper addresses the dynamic network scenario where link capacities may fluctuate over time by enhancing Tombo to adapt rapidly to these changes. The proposed Tombo-D framework leverages solutions obtained in the previous transmission period to improve computational efficiency in dynamic network scenarios.
Overall, the paper provides a detailed and innovative solution to the task-oriented image compression and transmission problem, demonstrating effectiveness in terms of overall task performance and computational efficiency through empirical evaluations on real-world network topologies and machine vision tasks.
limitations: limitations: One limitation of this paper could be the focus on empirical evaluations on a limited set of machine vision tasks and network topologies. While the proposed Tombo framework shows effectiveness in these specific scenarios, the generalizability of the approach to a wider range of tasks and network configurations may not have been fully explored.

This paper mentions image compression in both the title and the introduction part of the main text, but in fact, the core content of the paper is the intelligent transmission of multi-task streams, which has little to do with image compression. In addition, in the experimental part, it is not clearly written what image compression and visual task network are used, and the experimental results about the bitrate and accuracy of image compression are also very few, which is not consistent with the overall joint optimization framework mentioned in the title. It is suggested to modify the description in the title and body part to weaken the image compression and focus on the transmission part.
paper_summary: paper_summary: The paper propose a method to find the optimal decision on encoding bitrate and routing for machine vision transmission tasks. In conclusion, the paper really focus on a interesting problem, but it does not provide a good and convincing solution for the problem. The innovation of this paper is to introduce multipath routing into the scenario of machine vision task transmission, but the author lacks sufficient research on state-of-the-art related work about multipath routing. What's more, the author needs to explain the challenges and difficulties of this work before starting introducing the methodology. Last, the author considers this problem from a theoretical perspective, which may be difficult to apply in practical scenarios because of the difficulty of hypothesis on fluctuating network information. It is strongly recommended that the author consider a better solution from the perspective of the actual system.
strength_weakness: strength_weakness: - The paper focuses on a very interesting issue, which has good research value for both the academic community and practical applications.

- The paper is well-written and the problem is clearly stated.
limitations: limitations: main concerns:
- lack of survey on related work. The only work discussed is a paper from ICNC 2018, which is very insufficient. However, there are currently a large number of traditional multipath routing works (such as MPTCP, MPQUIC, MPRTP), and in recent years, many works have also discussed how to achieve higher bitrate  and QoE (Raven Mobicom'18, XLink in sigcomm 21, Converge in sigcomm 23 and so on).
- the experiment baseline is too simple, just compare with optimization solver, lack comparison of the state-of-the-art routing framework

minor problems:
- "Though off-theshelf solvers, e.g., Gurobi [15] and CPLEX [10], can accurately solve this problem to get the optimal solution, they cannot scale to the growing number of network nodes and commodities" why do you need consider growing number of network nodes? In fact most multipath routing just have 2-3 available paths to use.
- I was wonder whether you can choose path between the same sender and receiver, i.e. xp11, xp33 in the example shown in figure 2, unless you are talking about sending with different NICs. Since the routing decision between the same sender and receiver is handled by ISP and Internet router, the only thing sender can do is build path on different NICs. But in the case of machine vision task, what device are using? Do they really support multiple NICs?
- there are too many notations, the authors should add a table in the text or appendix. And there are still some confusing expressions need to clarify:
    - The relation between bitstream l and task \theta. Are they the same? If they are different, why?
    - If the total bitrate of edge should be less than total bandwidth (Eqn.2d), why need the variable r? The variables of xp and r are redundent. 
- Also, there are some doubts about this optimization function：If there are many sources to send images, and each source has to send multiple images, which is pretty common in practice, how to deal with the asynchronous sending problem? After all, the service can not wait for all the sources to finish its sending and then allow the next round sending.
paper_summary: paper_summary: This well-written paper, featuring rigorous mathematical derivations, addresses the varying rate-accuracy performance of image compression for different machine vision tasks and content types. In this paper, the authors propose Tombo, a task-oriented image compression and transmission framework that identifies the optimal encoding bitrate and routing scheme for multiple image bitstreams delivered simultaneously for different downstream tasks. They study the characteristics of image rate-accuracy performance for different machine vision tasks and formulate the task-oriented joint bitrate and routing optimization problem for multi-bitstreams as a multi-commodity network flow problem with time-expanded network modeling. To ensure consistency between encoding bitrate and routing optimization, the authors propose an augmented network that incorporates encoding bitrate variables into the routing variables. To improve computational efficiency, they convert the original optimization problem to a multi-marginal optimal transport problem and adopt an existing Sinkhorn iteration-based algorithm to obtain solutions quickly. They also extended Tombo to handle dynamic network scenarios where link capacities fluctuate efficiently. Simulations with three machine vision tasks and four real-world network topologies demonstrate that Tombo achieves comparable performance to the optimal solution provided by a solver, Gurobi, with a 5× to 114× speedup.
strength_weakness: strength_weakness: + Well-written and very easy to follow. 

+ Detailed math derivations, although the major results are based on prior arts, such as:
Gabriel Peyre and Marco Cuturi. 2019. Computational Optimal Transport. Foundations and Trends in Machine Learning 11, 5-6 (2019), 355–607.

+ Detailed simulation results.
limitations: limitations: - I missed the related work section. The problem is not entirely new; for example, it has been thoroughly studied in visual-based wireless networks. How would those works differ from the current paper? The authors are suggested to cite at least a few survey paper, like:

B. A. Lungisani, C. K. Lebekwe, A. M. Zungeru and A. Yahya, "Image Compression Techniques in Wireless Sensor Networks: A Survey and Comparison," in IEEE Access, vol. 10, pp. 82511-82530, 2022, doi: 10.1109/ACCESS.2022.3195891.
keywords: {Image coding;Wireless sensor networks;Data compression;Wireless communication;Sensors;Discrete cosine transforms;Principal component analysis;Autoencoder;data compression;image compression;image compression techniques;lossy compression;wireless sensor network(s)},

Along the same line, all the baselines are only variants of the proposed solution. It will be better if some prior algorithms can be compared against.

- Is there a theoretical bound for the near-optimality?

- How practical is the proposed solution? (1) where and how often is the algorithm executed? (2) How to enforce the routing decisions? (3) What are the typical workloads (request arrival patterns)? (4) How is background traffic considered? (5) Do all nodes in the considered problem need to be synchronized?

- Evaluations only consider very few sources/targets. Does the proposed algorithms scale to bigger networks?

- "Fig. 2(a) shows a common scenario where a sequence of compressed image signals are requested to be transmitted from a source
(e.g., a monitor) to several targets to..." a monitor? Not a camera?

- Sec. 2.2: all tasks are assumed to be equally important?

- Sec. 2.2: How the Φ𝑙,𝜃 (𝑟𝑙 ) = −𝑎𝑙,𝜃 /(𝑟𝑙 − 𝑏𝑙,𝜃 ) + 𝑐𝑙,𝜃 chosen? Are there other candidates?

- Sec. 4: How were network protocol and other overheads considered?
paper_summary: paper_summary: This paper proposes a framework for jointly optimizing the encoding bitrates and routing scheme for multiple image bitstreams with various machine vision tasks. They formulated the problem as a multi-marginal optimal transport problem and proposed a Sinkhorn iteration-based algorithm to speed up the solving process.
strength_weakness: strength_weakness: 1.	This paper has a good theoretical approach.
2.	The figures in this paper are of good quality and easy to understand.
limitations: limitations: 1.	There have been many research works on image compression and transmission methods. This paper Lacks a presentation of related work and no benefits from it. I suggest increasing a new discussion to show what are the limitations of the related works.
2.	The authors mainly compared with some optimization algorithm solvers and lacked a comparison of image compression and transmission schemes. The author should compare the proposed algorithm with other recent works or provide a discussion. Otherwise, it's hard for the reader to identify the novelty and contribution of this work.
3.	This paper focuses on the joint optimization problem of image compression and transmission but lacks a detailed description of the image compression scheme.
4.	Some of the methods and experiments presented in this paper are relatively independent and lack end-to-end performance comparisons. Time is a very important performance metric in the study of image compression and transmission. It is recommended that the authors add a compression and transmission time comparison with baseline methods.
5.	This paper is mainly oriented towards different tasks, including object detection, classification, and instance segmentation. However, it does not present the specific methods or models used. 
6.	The results in Table 2 are very confusing. The criteria for bolding are inconsistent, for example, in “Trans. cost” why 47.71 is bolded instead of 45.49. The authors are advised to double-check.
7.	The results in Table 2 show the Speedup of TM-Sinkhorn is 604.53, but the method of this paper is only 5.80. It is suggested that the authors give a reasonable explanation, otherwise it is difficult to judge the contribution of this paper.
8.	There are some minor errors, such as “Fig. 1” should be “Figure 1”, and “COCO2017” should be “COCO 2017”.
9.	The references are not in a consistent format, such as [16], [20], [25].