Structure Extraction fromPresentation Slide Information
Tessai Hayama1 , Hidetsugu Nanba2 , and Susumu Kunifuji1
1

Graduate School of Knowledge Science
Japan Advanced Institute of Science and Technology
1-1, Nomishi, Ishikawa, Japan{t-hayama,kuni}@jaist.ac.jp
2
Faculty of Information Sciences
Hiroshima City University
3-4-1, Ozukahigashi, Asaminamiku, Hiroshima, Japannanba@its.hiroshima-cu.ac.jp

Abstract. Electronic presentations are used in numerous scenarios, suchas lectures and meetings. In recent years, the widespread use of electronicpresentations means that presentation slide data is increasing as one ofindustry’s most important information resources. Therefore, it is necessary to develop a practical usage method for the reutilisation of the dataon slides. An approach to achieve this is to focus on visual structure
information within a slide, because visual structure information is oneof the most valuable, easy to understand methods for humans. However,since visual structure information is not explicitly deﬁned in the slide
data itself, computers have diﬃculty comprehending structure information directly. In this paper, we propose a method of extracting structure
information from slide information. The proposed method is composedof two steps: organising objects within the slide as units, such as title,
body text, ﬁgure and table, and structuring the units as a hierarchy tree
based on a top-down approach.
Key words: Information Extraction, Presentation Slide, Visual Layout,
Web Data

1

Introduction

The widespread use of electronic presentations is increasing the number of slides
that businesses accumulate. Since used slides are often stored and reused as
e-Learning or Web content, the data stored on slides is rapidly becoming one
of industry’s most important information resources. Therefore, it is necessary
to develop a practical usage method for the reutilisation of the data stored onslides. One approach to preparing a slide data reutilisation system is to use visual
structure information within a slide. Most systems currently handling slide data
convert the slide data into simpliﬁed text data, and then users access the data byusing a linear sequence of words. Although a slide’s visual structure information,such as visual form and layout, is valuable to easily understanding the context of2

Tessai Hayama et. al.

the data, current slide data reutilisation systems always ignore such information.
If a system could handle the text data with its structural information, thenthe system would be able to facilitate the intelligent processing of slide data.
Structure information representing the relationships among the data entities is
not explicitly deﬁned in the slide however. Therefore, it is not easy to manually
add the deﬁnition to all existing slide data. Thus, we need to develop technology
to automatically extract structure information from the slide data.
Several methods for extracting structure information from documents havebeen proposed[8, 7, 1]. Rosenfeld et al.[6] and Zhai et al.[9] suggested a structure
extraction method for PDF and Web documents using probabilistic approaches,such as the machine-learning and tree-graph-matching algorithms, respectively.
These approaches need to prepare a large amount of annotated data, and themodels made from the data are dependent on the data. Although it is usefulto adapt target data containing a few types of information structure, it is difﬁcult to adapt target data containing various types of information structure,such as on slides. Nanno et al.[5] proposed a method of extracting structure
information from Web pages using the repetition of elements within the Web
page. However, the method is inapplicable for slide structure extraction because
slide data does not include an explicit regular element, such as an HTML tag.
Ishihara et al.[3] proposed an extraction method based on close distances among
the objects within a slide to analyze structure information focusing on diagrams
within a slide. Because the objects within the slide can be created freely andthen manually allocated on each slide, slides sometimes include objects in incorrectly overlapped positions. Thus, Ishihara et al.’s method cannot analyze thestructure information appropriately. Although the previous methods can eﬀectively extract structure information from a document with formal formatting,they cannot extract structure information from the information on a slide thathas a varied layout structure and incorrectly placed objects.
In this paper, we propose a method of extracting structure information fromthe information on slides. The proposed method is composed of two steps: organising primitive objects within the slide as units, such as title, body text,
ﬁgure and table, and structuring the units as a hierarchy tree based on a topdown approach. Knowledge of slide structure is useful in various applications.
For example, the knowledge of structure represents the visual structure thatthe current slide readers cannot utilise, so that it allows blind users to understand presentation documents more easily. In addition, a system that presents
slides on hand-held devices with small display screens can use the structure forsegmentation and assign a human-like layout to the segmented slide information.

2

Deﬁnition and Problems for Structure Extraction

2.1

Slide Information and Its Structure

Slide information is composed of one or more primitive objects, such as texts,
pictures, lines and basic diagrams. Each object is recognised as a functional
attribute, such as title, body text, ﬁgure, table and decoration. For example, asStructure Extraction from Presentation Slide Information

3

#
$
%
&
'

(

Fig. 1. An example of slide information and its structure. (Object(s) boxed by a dashedline represent a unit able to function as an attribute.)

shown in Fig. 1, (A), (C) and (F) are primitive text-type objects and have thefunctional attributes of title, body text and ﬁgure, respectively. In such a case,(F) is recognised with (E) as a unit of ﬁgure attribute. Thus, even if the objects
are similar in type, as in the example, they may function either as diﬀerent
attributes or as a group of two or more objects, but not as a single object.
The structure of information within a slide can be represented as a hierarchy
tree of the units in which the object(s) can function as an attribute: a set ofsimilar units. To detect the relationships between the units, cues such as theslide’s visual form and layout can be used. For example, as shown in Fig. 1,
object (A) with a title attribute can be assigned to the root node. Objects ((B)
and (C)), related by indent, and objects ((C) and (D)), itemised in same level,
can be assigned to the parent-child nodes and sibling nodes, respectively. In
addition, a text box can be represented by a partial tree composing the objects
within the box. Thus, the visual cues within a slide provide the relationships
between the units to detect the tree structure.
2.2

Extracting Structure Information from Slide Information

As described in the previous section, structure information within a slide is built
on primitive objects by the following steps: (1) Organising primitive objects
within a slide into units that can function as an attribute and (2) structuring
the units as a hierarchy tree.
In step 1, to detect the units that are able to have an attribute, the close
distance and the overlap between the objects can be used as described in [3].
However, since the objects are freely created and then manually allocated on theslide, incorrect object placement is inevitable and thus may fail to lead to thedetection of the separations between the units. To detect the separations between
the units, not only the distance relationships but also the functional relationships
between the objects can be used. For example, if text- and diagram-type objects
overlap, the text-type object can be properly identiﬁed as a body-text attributeusing a bullet point list. Therefore, inappropriate organisation of the slide canbe eliminated.

4

Tessai Hayama et. al.

In step 2, the structuring of the visual layout is often used as an approachbased on the regularity of the unit relationships as well as the matching of template layouts. Since it is diﬃcult to prepare a large collection of layout templates
as in the latter approach, we will apply the former approach to structure theunits. Although it is useful to use the visual cues, such as indents and bullet
point lists, to detect the regularity of the relationship of the units, it is inappropriate to use them alone. For instance, if a ﬁgure object is allocated in a large
area of the slide, the regularity with the visual layout might be disturbed. To
compensate the regular disturbance, the system can also use attribute information with the units. Even if the ﬁgure object disturbs the layout regularity, we
can maintain layout regularity using the regularity of units of the object.

3

Proposed Method

We propose a method of extracting structure information from the information
within a slide. The method consists of the following stages. First, organisingprimitive objects within a slide into units using function relationships among
the objects. Second, structuring the units based on a top-down approach.
3.1

Organising Information in a Slide into Units Using Functional
Relationships

To identify an uncertain attribute within an object, we assumed that the attribute could be determined by the functional relationships between that object
and the other objects on the slide with more certain attributes. Therefore, this
proposed method assigns the likely attribute within each object, and then determines the attribute of the object in order of the object with the most obvious
attribute, which also aﬀects the determination of the attribute(s) of the functionally related object(s). The organisation can be achieved with the following
procedures.
1) Assigning the likelihood of each attribute within each object: Thescore of each attribute within each object is assigned using a score sheet, asshown in Table 1. The score sheet is made based on the type, position andsize of an object with the distinction of each attribute. The points within the
sheet are scored according to the following rules: an object with the properties
that indicate the likelihood of each attribute is given points for each attribute.
If the type of the object inﬂuences the scoring for each attribute to a greater
degree than the object’s position and size, an object functionally related as thelikelihood of each attribute is given points for each attribute.
The score is calculated by the number of items that are matched with theobject. For example, as shown in Fig. 2, object (a) has attribute scores of title,
body text, ﬁgure and table as 5, 2, 0 and 0, respectively. In addition to theassignment, the functional relationship of each attribute within each object islisted if the scoring of each attribute is used as a relation to the other object(s),such as the items underlined in the score sheet.

Structure Extraction from Presentation Slide Information

5

Table 1. Score sheet of attribute based on the likelihood of the attributes
Items for title attributeWith font size > T hreshold(f ontsize1)
With position from the top> T hreshold(y axis position)
With the nearest top-positionin the slideWith the largest font size in the slideWith number of characters> T hreshold(number of characters)

Items for body-text attribute+1 With bullet symbol+1
Existing text-type object(s) with simi+1 lar format and at the same left-position. +1
Existing the other text type object+1 on the position of upper left/lower+1 right of it+1
With font size > T hreshold(f ontsize2)+1+1 With number of characters> T hreshold(number of characters)+1
Items for ﬁgure attribute
Items for table attribute
Graph/Picture object
5 With number of data more than half
Complete overlapping G/P Obj
4 of cells within table
5
Partially overlapping G/P Obj
4 With number of data less than half ofOverlapping G/P Obj indirectly
3 cells within table4
Text-type object at top/down position
Complete overlapping cell area withinamong a group overlapped G/P Obj
table
4
directly/indirectly-1 Partially overlapping cell area within
Diagram object with no text
4 table3
With number of characters
Overlapping cell area within table< T hreshold(number of characters)+1 indirectly3
T hreshold(f ontsize1) , T hreshold(f ontsize2) , T hreshold(Ya xisp osition) andT hreshold(number of characters) are represented parameters of font size, font size,
distance from the top-position and number of characters, respectively, G/P Obj andunderlined items indicate graph/picture object and the scoring using relationships
among other object(s), respectively.

(2) Identifying the attributes of the objects: By detecting an object witha maximum likelihood of an attribute, the attribute of the object is determined
and then the other object(s) functionally related to this other object is aﬀected.
The process consists of the following three steps: 2.1, 2.2 and 2.3.
– (2.1) Detecting an object with maximum likelihood of an attributeamong the objects with non-determined attributes: First, each object with a non-determined attribute is set as a candidate attribute (attri candidate), which is one with the largest scores among the four attributes.
The likelihood of the candidate attribute contains not only the likelihood
degree of the candidate attribute but also the unlikelihood degree of theother attributes. Thus, the likelihood of the candidate attribute is deﬁned
and its value (Li Attri) is given by equations (1) and (2). Here, attri,
Attri V al(attri) and M axScore(attri) indicate an attribute, its scores assigned in step 1 and the max score for each attribute3 , respectively.
3

In the score sheet of Table 1, the max score for each attribute is 5.

6

Tessai Hayama et. al.

(a)text-type object [5, 2, 0, 0](b)

text-type object [3, 5, 0, 0]
text-type object [3, 4, 0, 0](c)text-type object [2, 3, 3, 0](d)picture-type object [0, 0, 5, 0] (e)text-type object [2, 2, 3, 0]
text-type object [2, 4, 0, 0](g)

text-type object [2, 4, 0, 0]

(h)

(f)

Fig. 2. An example of a slide including attributes scores (The numbers within square
brackets indicate the attribute scores of title, body text, ﬁgure and table.){

Attri V al(attri) ( if attri candidate == attri)
M axScore(attri) − Attri V al(attri) ( otherwise )(1)

Li Attri = Ev(‘title′ ) ∗ Ev(‘body−text′ ) ∗ Ev(‘f igure′ ) ∗ Ev(‘table′ ) .

(2)

Ev(attri) =For example, as shown in Fig. 2, the likelihood attribute values of objects(b) and (g) are set by 375 and 300, respectively, so that (b) is more likely tobe identiﬁed with the body-text attribute than (g). Finally, an object withthe largest attribute value is detected and determined with the candidate
attribute.
– (2.2) Changing each attribute score with each object(s) related tothe object, which is determined in step 2.1: The rules are as follows:• If the object is identiﬁed as a title attribute, each object, including therelationship lists but excluding title, is set by each attribute score, whichis then subtracted by 1. Also, the title attributes of all objects are set
by 0.
• If the object is identiﬁed as an attribute, excluding title, each object
included in the relationships lists of the attribute is set by the attributescore, which is subtracted by 1.
– (2.3) Repeating steps 2.1 and 2.2 until the attributes of all objects
are determined: An object with a more certain attribute is preferentiallyassigned the attribute and can aﬀect the determination of an attribute within
the functionally related object(s).
(3) Organising the objects into units using close distance and object
overlap: After determining the attributes of all objects within the slide, theobjects with ﬁgure attributes are organised based on the objects’ ﬁgure relationships list.

Structure Extraction from Presentation Slide Information

7(4) Assigning a decoration attribute: After the units with an attribute aredetected, a diagram-type object including unit(s) with a body-text attribute anda non-organised arrow-shape type object are reassigned as decoration attributes.
3.2

Structuring the Units Based on a Top-Down Approach

By detecting regularity in a slide’s visual structure, the system builds the units
as a tree structure. The structuring is based on a top-down regional dividing
approach; step by step, the block region including the units is divided into more
blocks so that every additional dividing step creates a hierarchical structure bydeﬁning parent-child relationships between the units.
The procedures of the structuring are as follows:(1) Setting initial state: The initial block and a root node are set based on
the unit with the title attribute. If a unit with a title attribute is included, theunit is assigned to the root node and the initial block is created to contain theregion, including the units below the unit with the title attribute. Otherwise,
the root node is created as a blank, and the block is created by the region that
includes all the units.
(2) Dividing block(s) vertically: If block(s) have vertical blank space(s), thenthese block(s) are divided into more blocks according to each space. Otherwise,
this step is skipped.
(3) Dividing block(s) horizontally: The step consists of four stages:
In the ﬁrst stage, large horizontal blank space(s) are sought from block(s).
If horizontal blank space(s) larger than the threshold is found in the block(s),then each block is divided into more blocks according to the space. Then, thesystem proceeds to step 4.
In the second stage, the units’ attribute sequences in the block(s) are checked.
If one of the sequences in each block is matched with the following heuristic
rules, which are based on the attribute relationship between a body text and aﬁgure/table, the block is divided into two blocks according to the matched rule.
Then, the system goes to step 4.
(i) If ‘an attribute within T opObj’ == ‘body-text’ and an object with a ﬁgure
attribute to the left from T opObj is included, then the block is divided into
two blocks by the top position of the object with a ﬁgure attribute. Onlya bullet point list with the T opObj is not included in the object with theﬁgure attribute.
(ii) If ‘an attribute within T opObj’ == ‘ﬁgure/table’, then the block is divided
into two blocks by the bottom position of the object at the top of the block.
where TopObj presents an object at the top of the block.
In the third stage, the unit at the top position of each block is checked. If
the unit is identiﬁed as a body-text attribute and is included in a bullet point
list, then the block is divided into more blocks by the top position of each of thebullet points and then goes to step 4.

8Tessai Hayama et. al.

In the ﬁnal stage, the unit at the top position of each block is checked. If theunit is identiﬁed as a body-text attribute, then the block is divided into the unit
and other units within the block.
(4) Repeating steps 2 and 3 until every block includes one unit.

4

Experimental Evaluation

4.1

Method and Preparation

The experiment we conducted mainly focused on two points: (1) whether it was
eﬀective for the organising process to apply functional relationships among primitive objects within a slide, and (2) whether it was appropriate for the structuring process to apply a top-down approach, which provides attribute information
within the units. At present, no structuring methods of slide information andstandardised data sets are available for evaluation. Therefore, to evaluate this
proposed method, it is necessary to prepare a comparable method.
To compare the proposed method, we used two methods as follows. The ﬁrst
method is set as a standard method that is organised by the objects based only
on information derived from the objects’ distance relationships. For example, anobject with a ﬁgure/table type and object(s) overlapping or closely allocatedwith the object are organised into a unit with a ﬁgure/table attribute. The
second method is set as a proposal method without the functional relationship(‘F unc rel‘). For instance, an attribute within each object is assigned as an
attribute with the highest score in step 2.1 of the organisation, and steps 2.2
and 2.3 are cancelled. In addition, we examined the accuracy of the structuring
process using each unit data, which was produced in the organising process.
To compare these two methods, we used recall, precision and F-measure. These
performance measures are calculated with the following formulas.
number of detected units matched with correct
total number of correct
number of detected units matched with correct
P recision =total number of detected units
2 ∗ Recall ∗ P recisionF − measure =Recall + P recision
Recall =

(3)
(4)
(5)

We created a slide data set for the evaluation. The data set included 30 slides
randomly selected from a research paper database[4], which included papers andmore than 10,000 slides from the Internet. The average number of pages inslides was 16.4. The structure data of these slides was created manually by
manipulating a specially developed system interface. We implemented a systemthat could automatically generate the structure of information in a slide fromthe slide ﬁle. The system was developed in Microsoft Visual Studio C#, which
accepts its input data from a Microsoft PowerPoint (PPT) ﬁle and outputs an
XML ﬁle containing structure information.

Structure Extraction from Presentation Slide Information

9

Table 2. Accuracy for each attribute results in the organising process
Attribute(Number of its unit)
Proposed Method

Title Body text Figure Table Decoration(602) (2267)(430) (12)(393)
Recall 0.98
0.98
0.90
1.00
0.90
Precision 1.00
0.91
0.83
1.00
0.62
F-measure 0.99
0.94
0.86
1.00
0.73
Proposed Method
Recall 0.97
0.94
0.87
1.00
0.86
without using Precision 0.98
0.92
0.80
1.00
0.63
0.93
0.84
1.00
0.73
F unc Rel F-measure 0.98
Standard Method
Recall 0.93
0.84
0.57
1.00
0.86
Precision 0.98
0.90
0.59
1.00
0.63
F-measure 0.95
0.87
0.58
1.00
0.73

Table 3. Ratio in pages for each correct ratio of results in the structuring process
Ratio of correct link number within a page 1.00 0.99-0.80 0.79-0.60 0.59-0.00 N/A
Proposed Method0.75 0.05
0.07
0.10
0.04
Proposed Method without using F unc Rel 0.62 0.06
0.08
0.20
0.04
Standard Method0.60 0.04
0.05
0.28
0.04

4.2

Results and Discussion

The results of the organising and structuring processes are summarised in Tables
2 and 3, respectively. Table 2 shows that the proposed method can organise slide
information into units with an attribute better than other methods. In particular,
this method can also detect a unit as a ﬁgure attribute. The organisation ofunits with a ﬁgure attribute is susceptible to the eﬀect of the distance between
the objects. Thus, the proposed method providing the functional relationshipcan eliminate the incorrect placement of slide objects. Table 3 shows that the
proposed method is able to identify an attribute more correctly and also thatthe proposed method can structure the information within a slide better thanany other methods. The proposed method used attribute information within
units in the structuring process to compensate for the regularity of the visuallayout. If units and their attributes were identiﬁed more correctly, the proposed
method would be able to function more eﬀectively. Thus, attribute information
with the units is important for extracting slide structure information; therefore,
our approach is useful for extracting information.
We also checked the errors caused by the proposed method in the experiment.
One of the problems is that the relationships between the slide’s objects aredeﬁned by text content, not by the slide’s visual layout. This makes it necessary,
therefore, to apply a text analysis technique to detect the relationships among
the objects.

105Tessai Hayama et. al.

Conclusion and Future WorkIn this paper, we proposed a technique, involving organizing and structuring
processes, to extract structure information from the information within a slide.
The organising process used functional relationships between the objects, andnot only the information derived from the close distances between the objects,
to eliminate potentially inappropriate organising. In the structuring process,attribute information within the units, as well as the visual cues on the slide,
was used to detect how the regularity of the layout structure could be improved.
Although our current system still needs some modiﬁcations, our experimental result shows that the proposed method can extract structure informationfrom slide information. In our future work, we are planning to develop slide
applications using the structure data extracted by this technique and the slide
information processing technique[2] that we have developed.

References
1. Anjewierden, A.: AIDAS: Incremental Logical Structure Discovery in PDF Documents, In Procs. the 6th International Conference on Document Analysis and Recognition, pp.374–378 (2001).
2. Hayama, T., Nanba, H. and Kunifuji, S.: Alignment between a Technical Paperand Presentation Sheets Using a Hidden Markov Model, In Proc. Active Media
Technology 2005, pp.102–106 (2005).
3. Ishihara, T., Takagi, H., Itoh, T. and Asakawa, C.: Analyzing Visual Layout for a
Non-Visual Presentation-Document Interface, In Proc. the 8th International ACM
SIGACCESS Conference on Computers and Accessibility, pp.165-172 (2006).
4. Nanba, H., Abekawa, T., Okumura, M. and Saito, S.: Bilingual presri: Integrationof multiple research paper databases, In Proc. the 7th RIAO Conference: Coupling approaches, coupling media and coupling languages for. information retrieval,
pp.195-211 (2004).
5. Nanno, T., Saito, S. and Okumura, M.,; Structuring Web Pages Based on Repetition
of. Elements, In Proc. the 2nd International Workshop on Web Document Analysis,
pp.58–60 (2003).
6. Rosenfeld, B., Feldman, R. and Aumann, Y.: Structural extraction from visual layout of documents, In Procs. the 11th International Conference on Information andKnowledge Management, pp.203–210 (2002).
7. Watanabe, T., Luo, Q. and Sugie, N.: Layout Recognition of Multi-Kinds of TableForm. Documents, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.17, No.4, pp.432–445 (1995).
8. Yang, Y. and Zhang, H.: HTML Page Analysis Based on Visual Cues, In Procs. the
6th International Conference on Document Analysis and Recognition, pp.859–864,
pp.10–13 (2001).
9. Zhai, Y. and Liu, B.: Structured Data Extraction from the Web Based on Partial
Tree Alignment, IEEE Transactions on Knowledge and Data Engineering, Vol.18,
No.12, pp.1614–1628 (2006).