dedoc/paragraph_dataset huggingface dataset:
editable format based on some of the principles of PDF. We begin
by analysing the current status quo, and provide a summary of
current approaches for editing existing PDFs, other relevant doc-
ument formats, and ways to embed the document’s structure into
the PDF itself. We then ask ourselves what it really means for a for-
matted document to be editable, and discuss the related problem of
enabling WYSIWYG direct manipulation even in cases where lay-
out is usually computed or optimized using offline or batch meth-
ods (as is common with long-form documents).
After defining our goals, we propose a framework for creating
such editable portable documents and present a prototype tool that
demonstrates our initial steps and serves as a proof of concept. We
conclude by providing a roadmap for future work.
• Applied computing → Document preparation; • Human-
centered computing → Interaction techniques;
Document formats, PDF, layout optimization, interactive editing
Tamir Hassan. 2018. Towards a Universally Editable Portable Document
Format. In DocEng ’18: ACM Symposium on Document Engineering 2018,
August 28–31, 2018, Halifax, NS, Canada. ACM, New York, NY, USA, 4
pages. https://doi.org/10.1145/3209280.3229083
PDFs are unique in their ability to preserve the visual presenta-
tion of a document across different platforms, systems and view-
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distrib-
uted for profit or commercial advantage and that copies bear this notice and the full
citation on the first page. Copyrights for third-party components of this work must
be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from Permissions@acm.org.
© 2018 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM 978-1-4503-5769-2/18/08…$15.00
https://doi.org/10.1145/3209280.3229083
ing devices. This is thanks to PDF’s roots in PostScript, a printer
language on which it is based. The widespread availability of print
functions in software at the time (PDF was introduced in 1993)
meant that PDF was the “lowest common denominator” among
documents from different sources; any document that could also be
printed could be effortlessly converted to PDF.
However, this is also PDF’s main drawback, and this remains
true to this day: the resulting document is essentially a vector
graphic, and is difficult to edit, at least in such a meaningful way
that would have been possible by the authoring application, such as
a word processor. Users wishing to exchange editable documents
are therefore still forced to use their native application’s format,
and do not benefit from the robustness, openness and portability
that people have now come to expect from a modern document
This paper proposes a framework to tackle this problem, work-
ing towards a open, universal format that can be edited by a vari-
ety of applications conforming to a commonly agreed specification.
The following subsection begins by introducing the current state
of the art: previous approaches to editing PDFs, relevant develop-
ments in the PDF format, and discusses the challenges of a univer-
sally editable format with reference to current editable document
formats. We conclude the introduction by defining the problem
more precisely, stating the goals that this new format should attain.
Section 2 presents the proposed framework and the underly-
ing technologies. Section 3 presents our initial experiments using a
publishing tool developed as a proof-of-concept and Section 4 dis-
cusses further issues that need to be addressed. Finally, Section 5
provides a roadmap for the future directions of this work.
1.1 Editability of current PDFs
A born-digital PDF that has been generated directly from the source
application, such as a word processor or DTP package, is essentially
a vector graphic, lacking (sufficient) logical structure information
about its content. A number of products exist in the marketplace for
editing such PDF files, but they are usually limited to simple “touch
up” operations, such as adding or removing words and graphic
objects; some more advanced products, such as FoxIt PhantomPDF,
also support limited reflowing of the text by attempting to detect
its structure, but this does not always work predictably.
Currently, editing a previously generated PDF is generally seen
as a last resort, and is only performed if the source file or applica-
tion is unavailable. In contrast, when opening the source file in its
native application, such as a word processor, the application’s inter-
nal model stores the content in a more structured way, enabling
text to reflow when editing, so that the layout automatically adjusts
and items to move to the next page if necessary. This structure is
(mostly) lost when the document is converted to PDF.
If the document’s logical structure is known in sufficient detail,
this internal model can be rebuilt, enabling the document to be edit-
ed in a similar way. However, each application has its own internal
DocEng ’18, August 28–31, 2018, Halifax, NS, Canada Tamir Hassan
model, designed specifically for the way that it works; the way the
users interact with it and the types of document that it generates.
There are many features that are common across a wide range of
mostly-text documents from different sources, such as bulleted and
numbered lists, tables etc., and one of the main challenges in design-
ing a universal format is to determine which core features will be
present in the generic document model.
1.2 Other notable developments in PDF
Since its initial release in 1993, there have been several develop-
ments to PDF to make it more intelligent and versatile, but these
have all stopped short of the goal of making the files editable.
Tagged PDF was introduced in version 1.6 of the specification
(2004), where extra metadata tags can be added to the content
stream to denote the logical structure of the text. These tags make it
possible to perform a basic reflow for small screen devices, remove
linebreaks when text is copied out of the PDF and are a core com-
ponent of accessible PDFs for users of assistive technology such
as screenreaders. In 2017, the responsive PDF project was pub-
licly announced, which will extend the tagging concept to make it
possible to automatically derive a full HTML representation of the
content, which can be presented in a responsive way using web
PDF has always been a container format, allowing embedding of
data in other formats, which are normally ignored by most view-
ing applications. Probably the best examples of such hybrid PDFs
are those created by Open- and LibreOffice, which simply include
the document’s ODF source in one of the streams. Unfortunately,
this feature was not very well advertised (and not activated by
default); and without a suitable file extension such as, for example,
.odf.pdf, nobody would be aware that they were actually view-
ing an editable, hybrid document. Technically, the hybrid ODF/PDF
document is equivalent to having two separate files, as the PDF rep-
resentation is still totally independent from the ODF source. Thus,
if the file is opened for editing on another system, the robustness
guaranteed by PDF is lost.
1.3 Other editable formats
Applications commonly used to author complete documents
include word processors, desktop publishing packages, drawing
programs and even presentation software. They are all largely
based on a page or canvas metaphor, which allows a certain amount
of direct, explicit manipulation of the layout. In fact, the underly-
ing document model in all these programs is very similar; only the
tools that are offered to the user have a different emphasis (which
has led to the current situation where users often have to deal with
several different user interfaces for performing the same task).
Text content, on the other hand, is implicitly laid out: the
content, combined with the typesetting parameters, ultimately
determine its form and how much vertical space it will take up.
Thus, editing the text can lead to more substantial changes in lay-
out if larger objects are placed in sequence to follow the text, or are
somehow anchored to parts of the text instead of to the page.
There are also a number of non-WYSIWYG typesetting systems
for formatting content, where the content’s structure is typically
explicitly defined, and its presentation is implicitly determined by
its structure and other typesetting commands. LaTeX is probably
the best known system in academia, but there are also many for-
matting systems for specific cases that take structured content as
input, usually in XML. But the best example of this approach is
actually the Web itself; the physical presentation of websites is still
dependent on various external factors such as the size of the brows-
er window; basic HTML does not lend itself well to absolute posi-
tioning of objects. And despite continuous development by stan-
dards bodies, browser manufacturers and other stakeholders, it is
still very difficult to guarantee that a document will display identi-
cally across different browsers.
1.4 Problem definition
The previous subsections have summarized the variety of formats
for text-oriented documents and their development over the past
years. It is important that a next-generation format for editable
document exchange supports these new developments, in order to
serve its users well for the foreseeable future. We propose that such
a format should:
• be universal: an open specification enables creation and edit-
ing via a variety of applications, just like PDF does today;
• be portable: the document should be self-contained and be
engineered in such a way that missing data (e.g. fonts) do not
affect layout;
• support editing via a graphical WYSIWYG interface as well
as structured markup;
• enable layout to be defined by both explicit, direct manip-
ulation operations and implicit, batch or optimization
procedures, thus combining the benefits of both approaches
(see Section 2.2);
• store the content in a well-structured form, enabling the
markup to be easily extended by semantic annotation if
required.
It is worth noting that this paper concentrates almost exclusively
on text, as text is the most common type of content across various
document formats, and hence lends itself well to being handled in
a universal way. Other content types are discussed in Section 4.1.
2 KEY TECHNOLOGIES
2.1 Standardized layout primitives
There are two main ways that layouts are described in WYSIWYG
document authoring applications. The first method, used by word
processors, assumes a content frame taking up the whole page
(excluding margins), which is filled in sequence from top to bottom.
With this type of layout, adding or removing content always causes
the layout to be automatically updated based on simple, predictable
rules; if content is inserted, the remaining content moves down. If
the page is full, a new page is created.
In desktop publishing software, text is usually placed into
frames, which are manually defined and can be freely positioned on
the page. In contrast to word processing software, the layout usual-
ly requires manual adjustment after content is added. Such a mod-
el is more flexible and allows arbitrary layouts to be designed, but
needs to be adjusted manually when content is added or removed.
The current proof-of-concept enables the author to choose to
use either of these two models for laying out the page. In fact,
the first model is extended to allow for any number of nested X-Y
divisions. This way, multi-column layouts and even more complex
Towards a Universally Editable Portable Document Format DocEng ’18, August 28–31, 2018, Halifax, NS, Canada
pages (such as this paper) can be created, while still benefiting from
automatic layout adjustment when content is edited.
2.2 Two-step layout optimization
Non-WYSIWYG publishing systems, such as LaTeX, have remained
popular for long-form content such as scientific articles. This is due
to the system automatically taking care of the layout (at least in
theory), leaving the author to concentrate on writing the content.
At compile time, a number of optimization routines are carried out
to perform pagination, place figures in appropriate places, update
the table of contents and references, etc. In contrast to the WYSI-
WYG approaches described above, due to the global nature of the
layout optimization being used, even carrying out minor edits to the
text can often lead to unexpected, larger changes occurring, such
as a figure jumping to another page.
This makes true WYSIWYG editing very difficult, not only due
to these unexpected changes, but also due to the computation-
al power required to constantly recompute the optimum layout.
Furthermore, the lack of direct manipulation of the objects also
makes it difficult to make any adjustments to the layout after it has
been computed. These disadvantages are often the sources of much
frustration for users of such systems, yet there appears to be sur-
prisingly little research being done to solve this problem.
We propose a two-step layout optimization strategy to make our
framework amenable to long-form documents, allowing them to be
edited in a WYSIWYG fashion. When the content is updated, the
layout is adjusted according to the principles in Section 2.1. Local
optimizations, such as line breaks and balanced columns, are also
carried out, as these are unlikely to distract the author or take too
long to process. Larger optimizations, such as the repositioning of
figures, are performed at a later stage at the author’s convenience.
As soon as the relevant constraints are no longer satisfied (e.g. the
figure is no longer placed on or after its first calling position in the
text), the system should warn the user in an appropriate way (e.g.
by using a red highlight or other visual marking in the case of a
GUI) and advise that a recomputation should be carried out when
the user is ready.
As this recomputation is completely separate from the inter-
active editing process, it could also be carried out by an external
program or web service if, for example, the user has particular
requirements for auto-layouting that are not met by the editor. For
example, [Mit16, Mit17] present significant improvements to the
greedy optimization algorithms for pagination and float placement
that are included in current LaTeX distributions. Alternatively, for
some use cases, it may be preferable to use local optimization fol-
lowed by manual adjustment instead of global optimization.
Given that the underlying source of the document is still text, it
remains just as easy to edit the text (and even the layout) using a
text editor, making this format inherently suitable for collaborative
low-bandwidth, offline-first applications, allowing the use of stan-
dardized versioning systems such as Subversion and Git.
2.3 Robust micro-typography