Abstract:
In a method for managing transmission of data files to users, a first file is divided into first segments and a second file is divided into second segments. A first segment is sent to a first user and a different first segment is sent to a second user. A second segment is sent to the first user and a different second segment is sent to the second user. At least part of a segment sent to the first user is combined with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining. The combined segment is transmitted to the first user and to the second user for each user to recover a segment using the combined segment and at least part of a segment.

Description:
FIELD OF THE INVENTION 
     The present invention relates to a method and apparatus for managing transmission of data files to users, and more particularly, but not exclusively, to transmission of data files to be cache locally to a user. 
     BACKGROUND 
     A server may hold data files to be sent to a user over a network, which may be, for example, the Internet or some other network. The data files may be media content files, or of some other type, and sent to the user when the user requests them, or when the content creator or provider wants to distribute them, for example. If a user does not require immediate access to a data file, or wishes to keep it available for later access, the data file may be stored in a cache memory local to the user. For example, the cache memory may be a hard drive included in the user equipment or the cache memory may be separately provided and arranged to be readily accessible by the user, for example, by being located nearer the edge of a network than the server. When the user needs a cached file, it may be obtained from the cache memory with reduced or no network resources. 
     Data transmission may be efficiently managed by sending data files to be cached at times when network resources are at high capacity and/or low cost. For example, a data file may be sent if a WiFi network is available, or when the user is in a low-traffic area or it is a low-traffic time period. 
     BRIEF SUMMARY 
     According to a first aspect of the invention, in a method for managing transmission of data files to users, a first file is divided into a plurality of first segments and a second file is divided into a plurality of second segments. A first segment is sent to a first user and a different first segment is sent to a second user. A second segment is sent to the first user and a different second segment is sent to the second user. At least part of a segment sent to the first user is combined with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining. The combined segment is transmitted to the first user and to the second user for each user to recover a segment using the combined segment and at least part of a segment. 
     The smaller size of the combined segment compared to the total size of the at least parts of segments before combining means that the combined segment requires less space if it is to be cached in a cache, or caches, local to the users. Additionally, as the combined segment is of smaller size, it requires fewer network resources for transmission than if the segments were to be sent individually in uncombined form. This is advantageous even for arrangements in which segments are not cached at a memory cache or caches local to a user. 
     The user or users may be any node in a network. For example, in a wireless network, a user could be a router or an end user or some other network node. 
     A method in accordance with the invention may be applied to wireless, fixed line or other types of network and is not limited to one particular technology type. 
     A method in accordance with the invention may be applied where there are f files and K users, and includes partitioning each file into 2 to the power of K segments, where for each subset of users, there is a segment which is saved on all of the users in that subset. 
     In one embodiment, the first and second segments sent to the first user may be cached a first cache memory local to the first user. Similarly, the first and second segments sent to the second user may be cached at a second cache memory local to the second user. Thus, a cache memory may be arranged to store only part of the first file and part of the second file, which may be advantageous when the cache memory is of insufficient size to store the complete files. When at a later time, one or both of the users want to access the first file or the second file, the entire file need not be sent to them as they already have access to some segments locally. Thus, even though remaining segments may in some cases need to be sent at a relatively expensive time, or during high network loads, as the entire file need not be sent then, overall costs for sending the file may be reduced. Additionally, the combined segment is smaller than the total size of the contributing segments, which may provide efficient file transmission management. 
     In an embodiment, the at least part of a segment sent to the first user is combined with at least part of a segment sent to the second user by using addition in a finite field. In one embodiment, the finite field is the binary field. 
     In one method, the first and second segments sent to the first and second users are sent prior to the combined segment being sent to the first and second users. However, in another embodiment, the data is sent in a different order. 
     In one embodiment, a segment is divided into first and second parts, the first part is incorporated into the combined segment and the second part transmitted to the first and second users. The second part may be transmitted at the same time as the combined segment or as a separate transmission. 
     According to a second aspect of the invention, a data transmission manager for managing transmission of data files to users comprises: a divider for dividing a first file into a plurality of first segments and for dividing a second file into a plurality of second segments; transmitter apparatus for sending a first segment to a first user and a different first segment to a second user, and for sending a second segment to the first user and a different second segment to the second user; a combiner for combining at least part of a segment sent to the first user with at least part of a segment sent to the second user to produce a combined segment which is of smaller size than the total size of the at least parts of segments before combining; and said transmitter apparatus being operative to send the combined segment to the first user and to the second user for each of the first and second users to recover a segment using the combined segment and at least part of a segment. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Some embodiments of the present invention will now be described by way of example only, and with reference to the accompanying drawings, in which: 
         FIG. 1  schematically illustrates a flowchart of a method in accordance with the invention; 
         FIG. 2  is a schematic graph illustrating transmission; and 
         FIG. 3  schematically illustrates an apparatus for implementing the method of  FIG. 1 . 
     
    
    
     DETAILED DESCRIPTION 
     In a first embodiment, assume that there is a first file A and a second file B available from a server, each file being 1 MB in size. There are first user U 1  and second user U 2 , and each user has an associated local cache memory M 1  and M 2  respectively, M 1  and M 2  being 1 MB each. It is assumed that the history of these two users U 1  and U 2  shows that each may require file A and file B with equal probability. 
     In this caching scheme, it is necessary to broadcast on average 0.5 MB in expensive networks. Moreover, the peak rate is 0.5 MB. 
     With reference to  FIG. 1 , at  1 , the first and second files are divided into several segments. File A is divided into two equal size segments A 1  and A 2 , where each segment is 0.5 MB, such that A=(A 1 , A 2 ). 
     Similarly, file B is divided into two equal size segments B 1  and B 2 , where each segment is 0.5 MB, and B=(B 1 , B 2 ). 
     At  2 , segments A 1  and B 1  are transmitted to the first user U 1  and, at  3 , saved on the associated cache memory M 1 . Also, A 2  and B 2  are transmitted to the second user U 2  and saved on the associated cache memory M 2 . The caching strategy is summarized in Table 1 and the probability in Table 2. 
     
       
         
               
             
               
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                 Caching Table 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                   
                 User One 
                 A1, B1 
               
               
                   
                 User Two 
                 A2, B2 
               
               
                   
               
             
          
         
       
     
     
       
         
               
               
               
               
             
           
               
                 TABLE 2 
               
               
                   
               
               
                   
                 Probability Table 
                 File A 
                 File B 
               
               
                   
               
             
             
               
                   
                 User One 
                 p1A = 0.5 
                 p1B = 0.5 
               
               
                   
                 User Two 
                 p2A = 0.5 
                 p2B = 0.5 
               
               
                   
               
             
          
         
       
     
     Thus, each user has part of each file. 
     At a later time, for example, if the first user U 1  wants to access file A and the second user U 2  requires file B, neither has sufficient segments cached to provide a complete file. The users transmit their requirements to the server at  4 . 
     At  5 , the server arranges for the transmitter to broadcast a combined segment A 2 +B 1 , where + indicates summation in a finite field to generate the combined segment. In this example, combination is carried out in the binary field and therefore + is simply bit-wise XOR operation. The combined segment A 2 +B 1  has 0.5 MB size. This compares with a size of 0.5 MB for segment A 2  and 0.5 MB for segment B 1 , that is, 1 MB in total. 
     In the next stage, shown at  6 , the first user U 1  receives A 2 +B 1  and already has B 1  in its cache memory M 1 . Thus, the first user U 1  can recover A 2  by the operation (A 2 +B 1 )−B 1 . The first user U 1  also has A 1  in it cache memory M 1 . Therefore, the first user U 1  has both A 1  and A 2  and is able to reconstitute the required file A. 
     The second user U 2  also receives A 2 +B 1  broadcast by the transmitter. The second user U 2  already has A 2  in its cache memory, and therefore can recover B 1  by the operation (A 2 +B 1 )−A 2 . The second user U 2  already has B 2  in it cache memory M 2 . Therefore, both B 1  and B 2  are available to the second user U 2  to reconstitute the file B it requires. 
     In another scenario, assuming that the starting point is that shown in Table 1, assume that the first and second users both require file A. In this case, the transmitter broadcasts A 2 +A 1 , where + again indicates combination is carried out in the binary field and is a bit-wise XOR operation. The combined segment A 2 +A 1  has a size of 0.5 MB compared to a size of 0.5 MB for segment A 2  and 0.5 MB for segment A 1 , that is, 1 MB in total 
     The first user U 1  thus receives A 2 +A 1  and already has A 1  in its cache memory. Therefore it can recover A 2  by (A 2 +A 1 )−A 1 . Therefore, the first user U 1  has both segments A 1  and A 2  and can thus reconstitute file A. 
     The second user U 2  also receives the broadcast A 2 +A 1  and already has A 2  in its cache memory M 2 . Therefore it can recover A 1  by (A 2 +A 1 )−A 2 . Therefore, it will have both A 1  and A 2  and can thus reconstitute file A. 
     For other cases, the broadcasting strategy and also the recovery method are shown in Table 3 below. It can be seen that no matter which file each user requests, the transmitter needs to broadcast only 0.5 MB. Therefore, the average rate of data that the transmitter broadcasts is 0.5 MB. 
     
       
         
               
               
               
               
             
           
               
                 TABLE 3 
               
               
                   
               
               
                   
                 Modified 
                   
                   
               
               
                   
                 Requirements 
                 Transmitter 
                 Decoding 
               
               
                 Demand 
                 Due to Caching 
                 Sends 
                 Strategy 
               
               
                   
               
             
             
               
                 User One Wants A 
                 User One Wants A2 
                 A2 + B1 
                 User One: (A2 + B1) − B1 
               
               
                 User Two Wants B 
                 User Two Wants B1 
                   
                 User Two: (A2 + B1) − B2 
               
               
                 User One Wants B 
                 User One Wants B2 
                 A1 + B2 
                 User One: (A1 + B2) − A1 
               
               
                 User Two Wants A 
                 User Two Wants A1 
                   
                 User Two: (A1 + B2) − B1 
               
               
                 User One Wants A 
                 User One Wants A2 
                 A1 + A2 
                 User One: (A1 + A2) − A1 
               
               
                 User Two Wants A 
                 User Two Wants A1 
                   
                 User Two: (A1 + A2) − A2 
               
               
                 User One Wants B 
                 User One Wants B2 
                 B1 + B2 
                 User One: (B1 + B2) − B1 
               
               
                 User Two Wants B 
                 User Two Wants A1 
                   
                 User Two: (B1 + B2) − B2 
               
               
                   
               
             
          
         
       
     
     In this embodiment, the caching scheme is optimized and the average load or maximum load of the network is minimized. In the simple example given above, the caching scheme can reduce the average load up to 50% and peak load of 100%. 
       FIG. 2  illustrates the normalized average transmission rate against the normalized cache-memory size for two files, of size F, and two users, each with cache size M. In addition, the probability that each user may need one of the files is equally likely. 
     In another embodiment, it is assumed that there are first and second users User  1  and User  2  with available caching memory size of M 1  and M 2  respectively. In addition, assume that the server has two files A and B with sizes F A  and F B  respectively, i.e.
 
| A|=F   A  
 
| B|=F   B .
 
     The users may need one of the files with some probabilities listed in the following Table 4: 
     
       
         
               
               
               
               
               
               
             
           
               
                 TABLE 4 
               
               
                   
               
             
             
               
                   
                 User 1 wants 
                 A 
                 A 
                 B 
                 B 
               
               
                   
                 User 2 wants 
                 A 
                 B 
                 A 
                 B 
               
               
                   
                 Probability 
                 p AA   
                 p AB   
                 p BA   
                 p BB   
               
               
                   
               
             
          
         
       
     
     Each file is divided into 5 parts, which need not be equal, to give:
 
 A={A   0   ,A   1   ,A   2   ,A   12 }
 
 B={B   0   ,B   1   ,B   2   ,B   12 }
 
with sizes
 
| A   0   |=x   0   ,|A   1   |=x   1   ,|A   2   |=x   2   ,|A   12   |=x   12 ,
 
| B   0   |=y   0   ,|B   1   |=y   1   ,|B   2   |=y   2   ,|B   12   |=y   12 ,
 
Therefore,
 
 x   0   +x   1   +x   2   +x   12   =F   A  
 
 y   0   +y   1   +y   2   +y   12   =F   B  
 
     Parts of the files denoted by A 1  and B 1  are cached on User  1  memory. 
     Parts of the files denoted by A 2  and B 2  are cached on User  2  memory. 
     Parts of the files denoted by A 12  and B 12  are cached on both users&#39; memories. 
     Parts of the files denoted by A 0  and B 0  are cached on none of the memories, 
     Therefore,
 
 x   1   +y   1   +x   12   +y   12   &lt;=M   1  
 
 x   2   +y   2   +x   12   +y   12   &lt;=M   2  
 
     In a first scenario, User  1  asks for file A and User  2  asks for file B. Thus, 
     User  1  requires A 0  which is only available at the server 
     User  2  requires B 0  which is only available at the server 
     User  1  already has A 1  and A 12 . 
     User  2  already has B 1  and B 12    
     User  1  wants A 2  while User  2  has it on its caching memory 
     User  2  wants B 1  while User  1  has it on its caching memory 
     Therefore the server must send both A 0  and B 0  to Users  1  and  2 . 
     Assuming that |A 2 |&gt;|B 1 |, then the server partitions segment A 2  into two parts, denoted by A 2U  and A 2L , i.e. A 2 ={A 2U , A 2L }, where |A 2U |=|B 1 |. Then the server sends combined segment A 2U +B 1 , where + is addition in binary field or any other finite field, and also sends A 2L . Note that:
 
| A   2U   +B   1 |plus | A   2L   |=|A   2 |=max{| A   2   |,|B   1 |}
 
where max {|A 2 |,|B 1 |} is the maximum of the size of A 2  and the size of B 1 .
 
     Then user  1  can use A 2U +B 1 , A 2L , and B 1  to recover A 2 ={A 2U , A 2L }. Also user  2  can use A 2U +B 1  and A 2  to recover B 1 . 
     On other hand, if |A 2 |&lt;|B 1 |, then the server divides segment B 1  into two parts, denoted by B 1U  and B 1L , i.e. B 1 ={B 1U , B 1L }, where |B 1U |=|A 2 |. Then the server sends combined segment B 1U +A 2  and also sends B 1L  and again + is addition in binary field any other finite fields. Note that
 
| B   1U   +A   2 |plus | B   1L   |=|B   1 |=max{| A   2   |,|B   1 |}.
 
     Then user  2  can use B 1U +A 1 , B 1L , A 2 , to recover B 1 ={B 1U , B 1L }. Also User  1  can use B 1U +A 2 , and B 1  to recover A 2 . 
     Then each user has the necessary segments to reconstitute the file requested by it. 
     In a second scenario, User  1  and User  2  both request file A. Note: 
     Users  1  and  2  want A 0  which is only available at the server memory 
     Both Users  1  and  2  already have A 12 . 
     User  1  wants A 2  while User  2  has it on its caching memory 
     User  2  wants A 1  while User  1  has it on its caching memory 
     Therefore the server has to sends A 0  to users  1  and  2 . 
     Assume that |A 2 |&gt;|A 1 |, then the server partitions segment A 2  into two parts, denoted by A 2U  and A 2L , i.e. A 2 ={A 2U , A 2L }, where |A 2U |=|A 1 |. Then the server sends combined segment A 2U +A 1  and A 2L  where + is addition in binary field or any other finite fields. Note that
 
| A   2U   +A   1   |+|A   2L   |=|A   2 |=max{| A   2   |,|A   1 |}.
 
     Then User  1  can use A 2U +A 1 , A 2L , and A 1  to recover A 2 ={A 2U , A 2L }. Also User  2  can use A 2U +A 1  and A 2  to recover A 1 . 
     On other hand if |A 2 |&lt;|A 1 |, then the server partitions A 1  into two parts, denoted by A 1U  and A 1L , i.e. A 1 ={A 1U , A 1L }, where |A 1U |=|A 2 |. Then the server sends A 1U +A 2  and A 1L  where + is addition in binary field or any other finite fields. Note that
 
| A   1U   +A   2   |+|A   1L   |=|B   1 |=max{| A   2   |,|B   1 |}.
 
     Then User  2  uses A 1U +A 1 , A 1L , and A 2  to recover A 1 ={A 1U , A 1L }. Also User  1  can use A 1U +A 2 , and A 1  to recover A 2 . 
     Following this strategy, the rate required for different cases is illustrated in Table 5 below: 
     
       
         
               
               
               
               
               
             
           
               
                 TABLE 5 
               
               
                   
               
             
             
               
                 User 1 wants 
                 A 
                 A 
                 B 
                 B 
               
               
                 User 2 wants 
                 A 
                 B 
                 A 
                 B 
               
               
                 Probability 
                 p AA   
                 p AB   
                 p BA   
                 p BB   
               
               
                 Required 
                 x 0  + 
                 x 0  + y 0  + 
                 x 0  + y 0  + 
                 y 0  + 
               
               
                 Rate 
                 max{x 1 , x 2 } 
                 max{x 2 , y 1 } 
                 max{x 1 , y 2 } 
                 max{y 1 , y 2 } 
               
               
                   
               
             
          
         
       
     
     Therefore, the average rate is equal to:
 
 p   AA ( x   0 +max{ x   1   ,x   2 })+ p   AB ( x   0   +y   0 +max{ x   2   ,y   1 })+ p   BA ( x   0   +y   0 +max{ x   1   ,y   2 })+ p   BB ( y   0 +max{ y   1   ,y   2 })
 
     Therefore, x 0 , x 1 , x 2 , x 12 , y 0 , y 1 , y 2 , and Y 12  may be chosen to minimize the average rate:
 
Min p   AA ( x   0 +max{ x   1   ,x   2 })+ p   AB ( x   0   +y   0 +max{ x   2   ,y   1 })+ p   BA ( x   0   +y   0 +max{ x   1   ,y   2 })+ p   BB ( y   0 +max{ y   1   ,y   2 })
 
Subject to:
 
 x   0   +x   1   +x   2   +x   12   =F   A  
 
 y   0   +y   1   +y   2   +y   12   =F   B  
 
 x   1   +y   1   +x   12   +y   12   &lt;=M   1  
 
 x   2   +y   2   +x   12   +y   12   &lt;=M   2  
 
 x   0   ,x   1   ,x   2   ,x   12   ,y   0   ,y   1   ,y   2 , and  y   12 =&gt;0
 
     If the maximum rate is the main concern, the following optimization can be used:
 
MinMax[ p   AA ( x   0 +max{ x   1   ,x   2 }), p   AB ( x   0   +y   0 +max{ x   2   ,y   1 }), p   BA ( x   0   +y   0 +max{ x   1   ,y   2 }), p   BB ( y   0 +max{ y   1   ,y   2 })]
 
Subject to
 
 x   0   +x   1   +x   2   +x   12   =F   A  
 
 y   0   +y   1   +y   2   +y   12   =F   B  
 
 x   1   +y   1   +x   12   +y   12   &lt;=M   1  
 
 x   2   +y   2   +x   12   +y   12   &lt;=M   2  
 
 x   0   ,x   1   ,x   2   ,x   12   ,y   0   ,y   1   ,y   2 , and  y   12 =&gt;0
 
     Other aspects may be used to refine the method. For example, if caching has some costs, the corresponding cost may be added to the objective function of the optimization of the data rate. For example, if sending data to both users has different costs than sending data to one user, the objective function may be correspondingly modified. Also, if one user already has some parts of the files, then this may be exploited in the optimization. Also, users may have some priorities on segments of a file which should be cached on their memory. Such priorities may be taken into account in the optimization. For transmission of data to the users, users priorities may be considered. For example, files may be transmitted in a manner that allows a user to recover its file in a specific order. 
     The method can be extended to any number of files and any number of users. For example, assume that there are three users, user  1 , user  2 , and user  3 , and three files A, B, and C. 
     Each file is partitioned into 8 segments as follows:
 
 A={A   0   ,A   1   ,A   2   ,A   3   ,A   12   ,A   13   ,A   23   ,A   123 }
 
 B={B   0   ,B   1   ,B   2   ,B   3   ,B   12   ,B   13   ,B   23   ,B   123 }
 
 C={C   0   ,C   1   ,C   2   ,C   3   ,C   12   ,C   13   ,C   23   ,C   123 }
 
     Then the segments are saved as follows: 
     Segments X 0  at none of the users, for X=A, B, C 
     Segments X i  at user i, for i=1, 2, 3, and X=A, B, C 
     Segments X ij  at both users i and j for i,j=1, 2, 3, and X=A, B, C 
     Segments X 123  at all users i and X=A, B, C 
     
         
         
           
             For simplicity, it is assumed in this example that Segments X i  for i=1, 2, 3, and X=A, B, C have the same size, and also X ij  for i,j=1, 2, 3, and X=A, B, C have the same size. This assumption is just for this example, and is not general requirement. 
             Then, if for example, user  1  wants A, user wants B, and user  3  wants C, then the transmitter sends
 
 A   0  
 
 B   0  
 
 C   0  
 
 A   2   +B   1  
 
 A   3   +C   1  
 
 B   3   +C   2  
 
 A   23   +B   13   +C   12  
 
             Then each user has received enough segments and combined segments to retrieve the desired file. 
             If for example all users want A, then the transmitter sends
 
 A   0  
 
 e   1   A   1   +e   2   A   2   +e   3   A   3  
 
 g   1   A   1   +g   2   A   2   +g   3   A   3  
 
 q   1   A   12   +q   2   A   13   +q   3   A   23  
 
             where the operations are any large enough finite-field, and e k , g j , q i  are from the same field. Then each user has enough equations to solve for entire A. 
           
         
       
    
     The size of the segments can be optimized as explained above. The size of the files does not need to be the same. 
     If there are f files and K users, then each file is partitioned into 2 to the power of K segments, where for each subset of users, there is a segment which is saved on all of the users in that subset. The size of the some of these segments may be zero. 
     With reference to  FIG. 3 , a data transmission manager  7  for implementing the embodiment described with reference to  FIG. 1  includes a content store  8  which holds data files A and B. A divider  9  accesses the content store  8  to obtain the data files A and B and divides the files into segments. Some of the segments are to be transmitted to users  10  and  11  at a time when network capacity is large and/or resources required are not expensive. Those segments to be initially transmitted to the users  10  and  11  are selected by a control processor  12  which also maintains a record of which segments are transmitted. The control processor  12  instructs a server  13  as to which segment is to be transmitted to which user. The server  13  acquires the relevant segments and sends them via transmitter  14  to the users  10  and  11  over a network. The users  10  and  11  each have an associated memory cache  15  and  16  respectively in which to store the data segments sent to them from a server  11 . Each user receives segments from file A and also from file B. 
     When the users  10  and  11  wish to have a complete file A or B, they send a message to the server  13  over the network. The server  13  and control processor  12  determine what combined segment is required to fulfill the users requests. The combined segment is produced from file segments by combiner  17  and delivered via server  13  to the users  10  and  11 . The users  10  and  11  are then able to reconstitute the complete files using the previously transmitted segments and the combined segment. 
     The apparatus illustrated in  FIG. 3  may be adapted to perform more complex data file delivery to implement other methods as set out above. 
     The functions of the various elements shown in the figure, including any functional blocks labeled as “processors”, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage. Other hardware, conventional and/or custom, may also be included. 
     The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.