IP over DTN: Large-Delay Asynchronous Packet
Delivery in the Internet
Hideya Ochiai
The University of Tokyo/NICT
jo2lxq@hongo.wide.ad.jp

Kenichi Shimotada
The University of Tokyo
ken@hongo.wide.ad.jp

Abstract—The well-known DTN divides an end-to-end communication path into hop-by-hop sessions and allows asynchronous
message delivery over physically delay or disruptive network environments. They have introduced Bundling, an overlay networkfor delay tolerant networking in its architecture, which should be
globally deployed and operated with tremendous effort. In thispaper, we propose a new DTN architecture that are more practically and easily deployable in the Internet space. Our approach
is IP over DTN; DTN message delivery framework works as adata link to deliver IP packets over network boundaries. The
transport protocols and application protocols take asynchronous
communication mode in this architecture. We have carried out anexperiment with our prototype DTN framework and conﬁrmedthat applications could practically communicate well with eachother even with large-delay IP packet delivery.

I. INTRODUCTION
In delay or disruption tolerant networks (DTN), it is commonly believed that message delivery should be performedby an overlay manner: e.g., RFC4838[1]. They have introduced the Bundle layer to develop a DTN, which has its
own ID space and transportation schemes. This approachcertainly allows performance optimization at the networks oflarge-delay or intermittent connectivity, including interplanetary networks[2], village-to-village networks in developing
regions[3] and mobile ad-hoc networks[4]. However, thisoverlay approach forces us to deploy a new global network
infrastructure over the global Internet, which certainly needs
great effort. The feasibility is not clear as well.
In this paper, we propose a new DTN architecture – IPover DTN, which is more practically and easily deployable in
the Internet space. In the architecture, DTN message delivery
framework works as a data link and delivers IP packets
by encapsulating them with DTN frames. The upper layer
protocols must take asynchronous communication mode, orin other words, a postal communication mode. The widelystudied DTN related works, including PEAR[5], MaxProp[6],DTLSR[3] and Spray and Focus[7], can be implemented at
the data link layer.
The term DTN used in most part of this paper, is different
from the well-known DTN: i.e., RFC4838[1] and DTNRG[8].
We use the term well-known to distinguish them from our DTNframework. Our DTN framework has only to deliver typicalsize (i.e., about 1500 octet) IP packets, which simpliﬁes thedesign and the implementation of DTN frameworks. To senda large message, it should be divided into multiple IP packets.
In IP over DTN architecture, the transport layer or application

Hiroshi Esaki
The University of Tokyo/NICT
hiroshi@wide.ad.jp

layer should care about the delivery of application protocol
data units (APDUs) (see Section 5).
IP networks deliver packets in the best effort manner.
Basically, any latency is allowed and any packet loss is allowed(of course, not preferable). By taking a DTN framework as adata link, IP packets can be stored, physically carried, and forwarded to isolated networks over the challenged environment
even though it takes times. Then, we will witness large-delay
IP packet delivery; e.g., a receiver gets IP packets sent minutes,
hours or days before.
There are two communication mode that end-to-end applications take: synchronous and asynchronous mode [9]. In
the synchronous mode, both communication ends are online
at the same time and make interaction between them: i.e.,
hand-shaking. In the asynchronous mode, both communicationends do not always need to be online at the same time.
The asynchronous mode is also known as a postal model ofcommunication. The sender can transmit their messages evenwhen the receiver is not online. The receiver gets the messages
from the network even after the sender has left it.
The communication failure and performance reduction atlarge-delay networks [10] comes only when applications try
to take synchronous communication mode, for example, byTCP. Generally, in large-delay networks, the communication
between end-to-end applications must shift from synchronousmode to asynchronous mode. The well-known DTN avoids
end-to-end synchronization by developing an overlay anddividing an end-to-end session into hop-by-hop sessions. Our
approach also avoids end-to-end synchronization by taking
and enhancing the asynchronous delivery capability that IPnetworks originally have. Therefore, IP over DTN architecture only uses asynchronous communication protocols. As IP
multicast practically works by taking asynchronous transport
protocols like UDP instead of TCP, we believe that, in IP
over DTN, applications can be practically developed over thenetwork even with this constraint.
This paper is organized as follows. In section 2, we discuss
related works. In section 3, we propose IP over DTN architecture. Then, we present our prototype implementation andexperiment in section 4. In section 5, we provide a discussion.
Finally, we conclude this paper in section 6.
II. RELATED WORKThe concept of IP over DTN itself is simple; however, wecould not ﬁnd this approach in literature. A DTN architecture

Fig. 1.

IP networking over a DTN framework

was proposed by Burleigh et. al. [2] and Kevin Fall[11], andthe internet engineering task force(IETF) has published thearchitecture by RFC4838[1]. The architecture has introduced
the Bundle layer, which deploys DTN framework as an overlay
network and delivers messages by APDU. This approach hascome from the study of TCP performance reduction and failure
over large delay and high packet loss networks[12]. Robert et.
al.[10] has mentioned that in deep space communication, TCP
fails because of the large packet propagation delay between thetwo sites. They proposed with the architecture to divide an
end-to-end communication session into hop-by-hop sessions
and presented that it should improve the performance of endto-end throughput.
However, there can be another point of view. In large delay
or highly topology disruptive networks, the mode of communication must be shifted from synchronous to asynchronousmode. IP over DTN, our approach, takes asynchronous communication mode from the start by using DTN framework as
a data link, and enhance asynchronousness that IP networks
originally have. IP over DTN architecture uses UDP-basedor UDP-like asynchronous communication protocols in theupper layer. Burleigh et. al.[2] has discussed the possibility ofUDP-based communication in interplanetary scenarios, however, they basically suppose synchronous communication (e.g.,
data acknowledgement and re-transmissions) at the applicationlevel in their paper; although in most of the time of discussion
with regard to DTN, they suppose a postal communication
model.
IP multicast has given up using TCP-based reliable communication protocol, because synchronization of multi-point
states is a quite heavy task[13]. In this regard, our approach isthe same; i.e., IP over DTN can give up using TCP, because
synchronization of end-to-end point states is impossible. However, IP over DTN will be more deployable than IP multicast.
The difﬁculty in IP multicast dwells in management issues
over the global Internet[14], whereas in IP over DTN, a DTN

framework can be locally managed as a data link.
III. IP OVER DTN
The TCP/IP is originally designed under a wired environment, where packet loss and delivery latency is smallenough to establish synchronized communication, for example,by TCP. Most of the communication in the current Internet
is practically implemented with such synchronized manner.
However, the delay or disruptive properties of the networkthat we targeted at in this paper potentially forces the modeof communication to be asynchronous. This means that end-toend TCP-based communication is no longer a proper communication style. The well-known DTN has achieved the shift
to asynchronous by an overlay called bundling. However,we emphasize that IP itself basically provides asynchronous
packet delivery mechanism. IP packet delivery can be delayed
if the applications assume large-delay and asynchronous communication mode.
This has motivated us to use a DTN framework as a data linkin IP networking. Figure 1 shows an overview of IP over DTN
architecture. In this example, two Ethernets are connected via aDTN framework, which delivers IP packets from one gateway
to the other with large delay. Host A sends a message tohost B by an asynchronous protocol on IP (e.g., UDP-basedasynchronous application protocol). We describe here how themessage can be delivered to host B.
1) The application at host A divides the message into asequence of datagrams (i.e., a sequence of IP packets).
2) With referring to its routing table, host A forwards thepackets to node α (192.168.1.1), the gateway to the DTN
framework.
3) The gateway α, which is named ”1” in DTN framework,
looks up its IP routing table and ﬁnds that to relay thepackets to host B, it should forward them to 192.168.2.4
and to do so, the packet should be encapsulated by DTN
frames and sent to DTN(4) – node δ, the gateway to hostB. Here, IP to DTN-address mapping (i.e., functionallyequivalent to ARP table) is managed and shared in theDTN framework.
4) The DTN framework sends the frames to DTN(4) by its
own message delivery scheme.
5) Finally, the gateway δ gets IP packets from the DTN
framework, and forwards them to host B via the Ethernet.
In this way, DTN framework can be implemented into IPnetworks.
We here summarize the properties that DTN framework we
discussed above should have. They are basically the sameas the well-known DTN framework except regarding to themaximum length of the protocol data unit. This difference
simpliﬁes the design and implementation of the framework.
• The maximum length of the protocol data unit is around1500 octet – enough length to encapsulate a typical IP
packet.
• When the next hop of a DTN frame not found in thetransmission range, the node stores it in its local buffer
and try re-transmission when it encounter the next hopnode.
• RTT between any two nodes is basically much larger thantypical data links.
• The DTN framework has its own ID space for datadelivery in the network.

Fig. 2.

The detailed conﬁguration of the experimentFig. 3.

The DTN node – embedded implementation

IV. IMPLEMENTATION AND DEPLOYMENTWe have developed a prototype implementation into Linux.
The DTN framework we used in the prototype is PEAR[5].
PEAR (1) computes routing tables at the DTN level withpotential-based schemes, and (2) replicates messages in thenetwork in order to increase delivery probability. PEAR carries out these two procedures autonomously based on themovements or contact patterns of DTN nodes. In order tomanage and share IP to DTN-address mapping over the DTNframework, we have made some modiﬁcations to the PEAR
source code.
With our prototype, we carried out an experiment of integration of a DTN framework into an IP network. Figure 2
shows the detailed conﬁguration of the experiment.
We prepared ﬁve network segments; four were deployedon Ethernet and one was deployed on PEAR. Each network
was connected by IP routers as Figure 2 shows. We used
three DTN nodes named α, β and γ. These three nodes have802.11g radio interface with ad-hoc mode through which theyexchange messages in DTN manner. Node α and γ were also
networked by Ethernet and worked as IP routers. They were
deployed in separate so that they cannot directly communicate
with each other. In the experiment, they stored and forwarded
IP packets, between Ethernet and the DTN framework. We
physically carried node β, an embedded node with battery
powered (Figure 3), between the two stations around α and γ
in 10 minutes per cycle.
We setup two hosts A and B in the network as an Internetapplication. Host B provides an image repository service,and host A tries to retrieve an image from host B. The
communication protocol is implemented over UDP, and theapplication is designed assuming large-delay message delivery.
In the experiment, host A sent an image request query tohost B, and host B returned the image by ﬁve to nine UDP
segments depending on the image size. Though the overall
communication style is synchronous, the delivery of each
APDU (i.e., query and image data) was done in asynchronous
manner.
Figure 4 illustrates the IP packet delivery pattern we observed in this experiment for one request-to-response transaction.
The blue arrows show request delivery pattern, and thered arrows show image data delivery pattern. The numberalong with the arrows shows the time in second when thetransmission event had happened. The number inside thebracket shows the number of packets transmitted at the sametime. Host A submitted a query to host B at time 0[sec],
which was stored at α. At time 162[sec], β has contactedto α and got the query from it. β has arrived at γ and handed
the query to it, then the network droved the packet to hostVI. CONCLUSION

Fig. 4.

IP packet delivery pattern

B at time 447[sec]. The requested image was sent by eight
UDP segments at 447[sec] and stored at β via γ. β has recontacted to α and send seven packets at 766[sec] and thelast one packet at 776[sec]. Here, the last one packet wasalso transmitted at 766[sec] but failed to receive at α thus
re-transmitted 10 seconds later. The total time to retrieve animage over the network was 776[sec].
V. DISCUSSION
As our experiment has shown IP over DTN applications
must aware the delay of APDU delivery; the host A had to wait
776 seconds to receive the requested image data. This requires
the design of application to be DTN-aware – the delay should
be supposed at the early stage of its protocol design. However,this is not a problem because even in the well-known DTN,
application protocols should be aware of the delay[15].
In the well-known DTN, to get reliability at message
delivery, they proposed to use data acknowledgement andretransmission scheme[1], [2], [11]. This scheme can be usedin some DTN application scenarios. However, it basicallyworks effectively only at short round trip environment, because
it is a kind of synchronous communication. Round trip in DTN
is costly, thus, another scheme to get reliability should be alsoneeded.
Forward error correction (FEC), which is known as amethod to get reliability in poor data transport systems, can bealternatively used for this purpose. Large-delay asynchronouscontents delivery media (e.g., compact disks and DVDs) andhigh-rate frame-loss media (e.g., WiMAX[16]) have adopted
this kind of technology to improve reliability without dataacknowledgement. FEC detects data error at the receiver side,
and corrects them if the damage is recoverable. FEC approachfor getting reliability might practically work well in many
DTN scenarios.
In this work, we have not focused on name-to-address
resolution over large-delay environment, which we have left
as an open research issue. Sharing IP-address to DTN-address
mapping (i.e., ARP table) could be easily implemented inour experiment platform. Sharing DNS-name to IP-address
mapping should be studied in the future.

We proposed a new DTN architecture – IP over DTN, which
DTN framework works as a data link and manages to deliver
IP packets over the challenged network environment. In thisarchitecture, transport protocols and application protocols takeasynchronous communication mode.
Compared to the well-known DTN architecture proposed by[1], [2], [11], our architecture is more practically and easily
deployable in the Internet space.
We have developed a prototype system and carried out anexperiment. The experiment has shown that IP packets werecertainly delivered over isolated networks with large-delay
but the applications succeeded to exchange their messages bytaking asynchronous communication mode.
R EFERENCES[1] V. Cerf, S. Burleigh, A. Hooke, L. Torgerson, R. Durst, K. Scott, K. Fall,and H. Weiss, “RFC4838: delay tolerant networking architecture,” apr
2007.
[2] S. Burleigh, A. Hooke, L. Torgerson, K. Fall, V. Cerf, B. Durst, K. Scott,and H. Weiss, “Delay-tolerant networking: an approach to interplanetaryinternet,” IEEE Communications Magazine, vol. 41, no. 6, pp. 128–136,
jun 2003.
[3] D. Demmer and K. Fall, “DTLSR: Delay tolerant routing for developingregions,” in ACM NSDR, 2007.
[4] J. Ott, D. Kutscher, and C. Dwertmann, “Integrating DTN and MANET
routing,” in ACM SIGCOMM workshop, 2006, pp. 221–228.
[5] H. Ochiai and H. Esaki, “Mobility entropy and message routing incommunity-structured delay tolerant networks,” in ACM AINTEC, 2008,
pp. 93–102.
[6] J. Burgess, B. Gallagher, D. Jensen, and B. N. Levine, “MaxProp: Routing for vehicle-based disruption-tolerant networks,” in IEEE INFOCOM,2006.
[7] T. Spyropoulos, K. Psounis, and C. S. Raghavendra, “Efﬁcient routing
in intermittently connected mobile networks: the multiple-copy case,”
IEEE/ACM Transactions on Networking, vol. 16, no. 1, pp. 77–90, feb
2008.
[8] “DTN research group,” http://www.dtnrg.org/.
[9] P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec, “Themany faces of publish/subscribe,” ACM Computing Surveys, vol. 35,
no. 2, pp. 114–131, jun 2003.
[10] R. C. Durst, P. D. Feighery, and K. L. Scott, “Why notuse the standard internet suite for the interplanetary internet?”
http://www.ipnsig.org/reports/TCP IP.pdf.
[11] K. Fall, “A delay-tolerant network architecture for challenged internets,”in ACM SIGCOMM, 2003, pp. 27–34.
[12] T. V. Lakshman and U. Madhow, “The performance of TCP/IP for networks with high bandwidth-delay products and random loss,” IEEE/ACM
Transactions on Networking, vol. 5, no. 3, pp. 336–350, jun 1997.
[13] S. Floyd, V. Jacobson, C.-G. Liu, S. McCanne, and L. Zhang, “Areliable multicast framework for light-weight sessions and applicationlevel framing,” IEEE/ACM Transactions on Networking, vol. 5, no. 6,pp. 784–803, dec 1997.
[14] C. Diot, B. N. Levine, B. Lyles, H. Kassem, and D. Balensiefen,“Deployment issues for the IP multicast service and architecture,” IEEE
Network, vol. 14, pp. 78–88, jan 2000.
[15] J. Ott and D. Kutscher, “Bundling the Web: HTTP over DTN,” in In
Proceedings of WNEPT, 2006.
[16] “IEEE 802.16e,” http://wirelessman.org/tge/.