情報処理学会 インタラクション 2023
IPSJ Interaction 2023

2B-27
2023/3/1

A Visual Exploratory System for Data Facts in
Data-driven Articles and Its Use in Business Reports
CHEN XUEYI1,a)Ken WAKITA1,b)

概要：For lots of people working in diﬀerent ﬁelds, reports describing a variety of data facts are authored
and analyzed by lots of people every day. To get insights eﬃciently from such kind of data-rich document,
people naturally take notes by using visual marks such as highlights, underlines, and circles traditionally.
Recently, as business reporting goes electronic gradually, there is a surge of attention and interest ineﬀective support for such kind of report analysis with the power of computer science. In this paper, we
design a visual-guided method focusing on facilitating business report authoring, especially, aiming tohelp build interactive business reports to convey data facts via the non-programming interface. The proposed method aims to simplify the authoring of interactive business reports, lower the technical barrier,as well as empower people to comprehend the key information eﬀectively. A use case generated by oursystem based on real business reports is presented to show the usage and capability of the implemented
architecture.

brings technique barriers for non-technique users. Intel-

1. Introduction

ligence algorithm approaches are recently widely used toData-driven storytelling (DDS) [19] technique has at-

automate the process of both data analysis (e.g., visual-tracted a lot of attention recently, sparked numerous ex-

ization recommendation based on machine learning ap-

ploratory studies on related concepts and design princi-

proaches) and storytelling, mitigate the burden of suchples, tool implementations, discussions on its eﬀectiveness,

users [17]. However, the ideal trade-oﬀ between the au-

etc [24]. As one of the prominent forms in the design space

tomated process (e.g., text generation and data facts ex-of DDS, data-driven articles [23], have been constantly

traction) and the preservation of human involvement (e.g,discussed over the past years due to their signiﬁcance.

hand-crafted storytelling and manual insights recommen-

With the characteristic of combining visualizations withdation during data analysis based on domain knowledge

narrative text, data-driven articles provide a potentiallyand experiences of the analysts), remains to be discussed

eﬃcient way for data explanation for the general public.

[25].

Despite its potential advantages in encouraging reading

These challenges remain to be tough to solve whileat the cognitive level [3], the creation of such data-driven

adapting data-driven article authoring techniques to busi-

articles is never easy as the authoring workﬂow spans a va-

ness intelligence (BI). Besides, the demands of the BI do-riety of disciplines including data analysis, visualization,

main are still not fully taken into consideration among thehuman-computer interaction, storytelling, etc.

existing tools and solutions for interactive business report

While

works focusing on supporting the authoring of data-driven

creation [27].

articles ﬂourished and have been proven to be potentially

To ﬁll this gap, we propose a visual exploratory systemeﬃcacious, the steep learning curve keeps such tools hard

prototype to adopt designs for supporting data-driven ar-

to be applied. Programming skills and script knowledge

ticle authoring in the ﬁeld of BI. We stress the signiﬁ-

are often required to author with these systems, whichcance of human involvement including hand-crafted met-

1
a)
b)

rics without the requirement of technical knowledge, to
School of Computing, Tokyo Institute of Technology
xueyichen 980221@outlook.com
wakita@is.titech.ac.jp© 2023 Information Processing Society of Japan

ensure the quality of the generated report in an easy way,581

情報処理学会 インタラクション 2023
IPSJ Interaction 2023

2B-27
2023/3/1

aiming to support the authoring workﬂow of interactive

quality of the text. However, placing human editors inbusiness reports.

the process of automated digital text generation requires

2. Related Workfurther study. The customization of interaction between
text fragments and charts in the generated article is also

We get insights into the possible challenges and chances

limited in the workﬂow. Thus many works strike a bal-of this work by surveying existing literature from 2 per-ance between eﬃciency brought by the intelligence algo-spectives.

rithms and the quality and readability ensured by human
involvement.

2.1 Data-Driven Articles
There is a recent surge of attention and interest in Data-

2.2 Visualization in BI

driven articles that combine visualization with narrative

Traditionally, dashboards are the most popular tools

text, which are also known as ”magazine style narrativefor visualizations in BI [11, 13], which consist of visualvisualizations (MSNVs)”, ”narrative visualization” [21],component collections aggregated on a single view. Tools”visual data stories” [16], etc. Though the widely dis-for BI dashboard construction [15, 5] have been discussed

cussed beneﬁts of such synergetic association between text

exhaustively over the past decades with the support ofand visualization in promoting active reading, encourag-tools for visualization design, including libraries like D3.js

ing engagement, as well as supporting understanding, in-[7], Vega-lite [20], and commercial tools like Tableau Sto-tegrating, and remembering the information [6, 10, 3], theries[18] and Microsoft BI [12], etc. Recently, there are also

authoring of such data-driven remains to be challenging.

eﬀorts toward applying intelligence algorithms to dash-

The workﬂow acquires multiple skills, including design,board creation, such as MultiVision [26], which selects

data analysis, visualization and storytelling, therefore co-data columns and makes recommendations of charts inoperation among domain experts is often required during

dashboards based on a deep learning approach.

the process.

Nevertheless, so far, data-driven articles that combine

To ﬁll such a gap, tools focusing on improving the au-storytelling with visualization yet remain seldom dis-thoring experience of data-driven articles have emerged

cussed in BI [1]. Recently, CODAS [27] is proposed torecently. While helpful, technique barriers, including re-facilitate authoring by transforming dashboards into in-quired programming skills, are usually high and thus keep

teractive business reports.

non-technical users from using them [8, 14], especially intakes Vega-lite charts as inputs, and thus requires re-the case of creating articles supporting customized inter-lated technical knowledge like scripting. Besides, tabu-

actions between text fragments and charts [23]. Idyll Stu-lar data presentation is seldom taken considered in suchdio [9] supports WYSIWYG-style editing for interactive

tools. Though the process could be time-consuming, ac-

document authoring, to a certain extent, but the compo-

cessing and exploring the underlying data could be im-nent’s implementation requires graphics coding. Besides,

portant for authors in business analysis. As discussed by

VizFlow [23] is a tool that allows authors to build inter-

Bartram et. al [4], analysts need to get in contact withactive documents that make use of visual linking between

the underlying data throughout the analysis process as atext fragments and charts. It reports, though, that theway to deepen their perception and comprehension of thetool is targeted at more technically experienced authors.

data facts. For authors of data-driven articles, accessing

To address this problem, a recent research trend istabular data enables them to build trust in the system

the introduction of intelligence algorithms in the pipeline,

by having greater control over the content they create

aiming to automate such processes to some extent andand more self-conﬁdence in the stories they want to tellreduce the burden on non-technical authors. For exam-

during the interactive authoring process. Badam et. alple, Calliope [22] is proposed to create data-driven arti-[2] proposed Elastic Documents that enhance document

cles from spreadsheets automatically. Erato [24] provides

reading by coupling text, visualization, and tabular com-

users with key-frames to describe the story. Nevertheless,ponents, aiming to support comprehension and insights-

both tools are designed for relatively short texts. Tra-ﬁnding during the reading. But the system does not sup-ditionally, due to the limitation of NLP techniques, theport authoring and is not aiming at applying such tech-quality of the generated content is hard to ensure. It may

niques to the BI domain, thus lacks consideration of cus-

be desirable to rely on human editors’ skills to polish thetomization and structured creation of the article, and also

© 2023 Information Processing Society of Japan

While helpful, the system582

情報処理学会 インタラクション 2023
IPSJ Interaction 2023

2B-27
2023/3/1

the demands of business analysis, including re-usability.

3. Problem
Goals

Statements

andDesignis based on a literature survey and the data provided
by TEIKOKU DATA BANK, LTD (TDB).
DG2 Lower the technological barriers based on the design of easy-to-use non-programming components forThis section describes the technical issues that remain

authoring. This aims to encourage the involvement ofin the past research and the feedback gained from the dis-authors via interactive authoring approaches, ensur-

cussion with the experts in business data analysis. Froming the quality and readability of the narrative text

this observation, the design goals of the system are in-content in the created business report. Data facts

spired.

classiﬁed as References and Claims are allowed to be• For the scenario that applying techniques facilitatingdeﬁned while authoring, to support the customization

the authoring of data-driven articles to the domain

of interactions between text fragments and charts inof BI, potential challenges are still not fully deﬁned,

the presented GUI-based interface.

and the opportunities are not thoroughly investigated.

DG3 Support interactive authoring while meeting the

Traditionally, dashboards are the most popular tools

speciﬁc needs of the BI domain by providing 3 tightfor visualizations in BI and were thoroughly discussed

coupling components: Text, Visualization, and Table

and get supported by commercial tools, while data-

components allowing the visual exploratory all the way

driven articles containing large-scale narrative text

to the underlying data. Facilitate design customiza-that combine storytelling with visualization remaintion to construct a narrative structure, and decouple

seldom discussed in this speciﬁc domain.

the data and the content to enhance the reusability• Technical barriers, including the requirements of programming skills, are high and thus keep non-technicalusers unable to beneﬁt from using these tools. This
happens especially when the addition of the inter-of the created components and authored documents,
helping with dynamic data updates.

4. System Design and Implementation

active capability to the digital document is desired

We propose the system design with the guidance of de-to make more sense of the document by, for exam-

sign goals to fulﬁll the requirements driven by the problem

ple, adding visual links between text fragments andstatements (DG1). Figure 3 shows the pipeline of the sys-

charts. Meanwhile, much research work seeks an ideal

tem.

trade-oﬀ between relying on intelligence algorithmbased solutions for lowering technological barriers andthe involvement of human experts to ensure the quality of the article via hand-crafted metrics.

4.1 System Design
The prototype is a web-based editor to facilitate interactive data-driven article authoring. With the raw tabular• In particular, when considering applying data-driven

data, users are able to develop a narrative structured data

article authoring techniques to BI, i.e., in the case ofstory using the three basic components supported by theinteractive business reports authoring, the unique de-system without programming knowledge (DG2).

mands of this speciﬁc domain are not fully considered

Given the dataset (as CSV ﬁles), with the speciﬁcation

among the existing tools and solutions. While neces-ﬁle (as the JSON) which is prepared by the system forsary for business analysis, tabular data presentation isparsing the conﬁguration of the structured arrangement

seldom taken considered in such system design. Theof the article dynamically, three types of basic compo-

reusability of the generated article for data updatingnents: Visualization Component, Table Component, andand design customization is also limited.

Text Component would be generated by the system for

Three design goals are compiled based on the problem

further use (DG3). Next, users are allowed to arrange

statement and the review of the survey on related work,

these basic components into Cell(s) components in single

which guides the development of our system and is dis-or double column-layout to fulﬁll their design customiza-cussed throughout the rest of the paper to justify the us-tion demands, which would be used to further be arranged

ability and contribution.

into Section components in top-to-bottom linear order,

DG1 Give an investigation about the hidden challenges

and eventually be modularized into the ﬁnal artifact. Dur-and opportunities for adopting techniques facilitating

ing the authoring process, users are free to upload tabular

data-driven article writing into the BI domain. Thisdata in CSV forms, create new cell components and ar-© 2023 Information Processing Society of Japan583

情報処理学会 インタラクション 2023
IPSJ Interaction 2023

2B-27
2023/3/1

Fig. 1 The pipeline of our system.

range them into the editing document via the GUI-based
interface, as shown in Figure 3 (DG2, DG3).
Two types of Data facts are deﬁned by the system: Ref-

the raw tabular data from the content (DG3).

5. Use Case

erences which simply refer to the data in the table, andA use case (Figure 5) is generated with our system based

Claims which are based on statistical reasonings of theon the real case of business analysis oﬀered by TDB, sodata in the table. According to the deﬁnition, users are

as to show the applicability of the design of this work.

able to assign data facts among the created report andFigure 5(a) is a report surveying the Kinki companies

create binding relations for the same data fact presented

awareness of new coronavirus infections, which includes

in diﬀerent formats (i.e., among the diﬀerent types of com-three sections with four visualization charts and two tab-ponents) via the GUI-based interface. Given the assigned

ular presentations.

data facts embedded in text, visualization, and (or) ta-

As shown in Figure 5(a), data facts including Refer-ble, the tight-coupling among these components would beences and Claims are assigned manually by the users

represented as eﬀects like visual links and highlights trig-

and are highlighted in green and red in the text com-

gered by interaction (e.g., hover), so as to support insights

ponents. While being assigned, these data facts presented

ﬁnding and comprehension (DG2).

as text fragments would be asked to be bounded withthe elements in both visualization and table components

4.2 Implementation

in a GUI-based interface, which support further interac-A client-server architecture is followed during the de-tions among diﬀerent components (DG2). Figure 5(b)velopment of our system. The server-side is implemented

shows some of the cases of interaction relying on thewith Python Flask. Raw tabular data and the speciﬁca-

tight-coupling of text, visualization and table components.

tions of the document including layouts and textual data

Speciﬁcally, when a Reference orClaim represented in aare structured for reuse and update (DG3). Preliminary

text fragment is hovered with the mouse by the users whoanalysis is handled by the pandas library. The client-intend to dig into it, this colored text fragment wouldside is realized with HTML and JavaScript. Markdown isserve as AOI (area-of-interests) while corresponding parts

used to able manual storytelling with the support of thepresenting the same data facts assigned by the author in

GUI-based interface provided in Text components for non-visualization and the table would be highlighted imme-

technical users. Thus the quality of the text content could

diately. Visual links are displayed in the form of B ´

be ensured (DG2). D3.js and Vue.js are used for visual-ezier curve, as a further enhancement of visual aids. Be-

ization construction in Visualization components, which

sides, as shown in Figure 5(c), new cells could be added to

could be used in non-programming interaction-based ap-

the linear arrangement of the article during the process of

proaches (DG2). By creating a library of Visualization

authoring via non-programming interaction, so as to de-components using these techniques, the tool is able to en-

couple the data and facilitate the creation of a narrative

sure the reusability of the authored report, and decouple

structure in an easy way even for the non-programming© 2023 Information Processing Society of Japan584

情報処理学会 インタラクション 2023
IPSJ Interaction 2023

2B-27
2023/3/1

Fig. 2 The use case created by our system. (a) The authored business report withassigned data facts (colored in red referring to Claims and green referring toReferences) via GUI-based interactions. (b) An interaction case building visuallinks between referenced text fragments, visualization and table. (c) The supportof adding Cell components dynamically to facilitate ﬂexible style customizationand data updates.

users.

6. Conclusion and Future WorkIn this work, we present a visual exploratory systemfor data facts in data-driven articles and show its usabil-[3]

ity in business reports authoring. The system consists of
non-programming interfaces to encourage the involvementof authors. We plan to organize a user study includingsemi-structured interviews in the next step, within which
a detailed evaluation will be conducted.

[4]

7. Acknowledgment
謝辞 The authors would like to thank the experts fromTEIKOKU DATA BANK, LTD (TDB) who support this[5]

work for their time and feedback.

References
[1]

[2]

Christos Amyrotos. “Adaptive Visualizations forEnhanced Data Understanding and Interpretation”. In: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. 2021, pp. 291–297.
Sriram Karthik Badam, Zhicheng Liu, and Niklas
Elmqvist. “Elastic documents: Coupling text and© 2023 Information Processing Society of Japan

[6]

[7]

tables through contextual visualizations for enhanced document reading”. In: IEEE transactions on visualization and computer graphics 25.1
(2018), pp. 661–671.
Oswald Barral, Sébastien Lallé, and Cristina
Conati. “Understanding the eﬀectiveness of adaptive guidance for narrative visualization: a gazebased analysis”. In: Proceedings of the 25th international conference on intelligent user interfaces.
2020, pp. 1–9.
Lyn Bartram, Michael Correll, and Melanie Tory.
“Untidy data: The unreasonable eﬀectivenessof tables”. In: arXiv preprint arXiv:2106.15005(2021).
Rahul C Basole. “Visual business ecosystem intelligence: Lessons from the ﬁeld”. In: IEEE
computer graphics and applications 34.5 (2014),
pp. 26–34.
Michelle A Borkin et al. “Beyond memorability:
Visualization recognition and recall”. In: IEEE
transactions on visualization and computer graphics 22.1 (2015), pp. 519–528.
Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. “D3 data-driven documents”. In: IEEE585

情報処理学会 インタラクション 2023
IPSJ Interaction 2023

[8]

[9]

[10]

[11]

[12]
[13]

[14]

[15]

[16]

[17]

[18]

[19]
[20]

transactions on visualization and computer graphics 17.12 (2011), pp. 2301–2309.
Matthew Conlen and Jeﬀrey Heer. “Idyll: A
markup language for authoring and publishing interactive articles on the web”. In: Proceedings ofthe 31st Annual ACM Symposium on User Interface Software and Technology. 2018, pp. 977–989.
Matthew Conlen et al. “Idyll studio: A structured
editor for authoring interactive & data-driven articles”. In: The 34th Annual ACM Symposium
on User Interface Software and Technology. 2021,
pp. 1–12.
Graham Dove and Sara Jones. “Narrative visualization: Sharing insights into complex data”. In:(2012).
Micheline Elias, Marie-Aude Aufaure, and Anastasia Bezerianos. “Storytelling in visual analytics
tools for business intelligence”. In: IFIP Conference on Human-Computer Interaction. Springer.
2013, pp. 280–297.
Alberto Ferrari and Marco Russo. Introducing Microsoft Power BI. Microsoft Press, 2016.
Danyel Fisher, Steven Drucker, and Mary Czerwinski. “Business Intelligence Analytics [Guest
editors’ introduction]”. In: IEEE Computer
Graphics and Applications 34.5 (2014), pp. 22–
24.
Nam Wook Kim et al. “Datatoon: Drawing dynamic network comics with pen+ touch interaction”. In: Proceedings of the 2019 CHI Conferenceon Human Factors in Computing Systems. 2019,
pp. 1–12.
Ana Lavalle et al. “Visualization requirements
for business intelligence analytics: a goal-based,
iterative framework”. In: 2019 IEEE 27th International Requirements Engineering Conference(RE). IEEE. 2019, pp. 109–119.
Bongshin Lee et al. “More than telling a story:
Transforming data into visually shared stories”.
In: IEEE computer graphics and applications 35.5
(2015), pp. 84–90.
Dominik Moritz et al. “Formalizing visualization
design knowledge as constraints: Actionable andextensible models in draco”. In: IEEE transactions on visualization and computer graphics 25.1
(2018), pp. 438–448.
Daniel G Murray. Tableau your data!: fast andeasy visual analysis with tableau software. John
Wiley & Sons, 2013.
Nathalie Henry Riche et al. Data-driven storytelling. CRC Press, 2018.
Arvind Satyanarayan et al. “Vega-lite: A grammar
of interactive graphics”. In: IEEE transactions on© 2023 Information Processing Society of Japan

2B-27
2023/3/1

[21]

[22]

[23]

[24]

[25]

[26]

[27]

visualization and computer graphics 23.1 (2016),
pp. 341–350.
Edward Segel and Jeﬀrey Heer. “Narrative visualization: Telling stories with data”. In: IEEE
transactions on visualization and computer graphics 16.6 (2010), pp. 1139–1148.
Danqing Shi et al. “Calliope: Automatic visual
data story generation from a spreadsheet”. In:
IEEE Transactions on Visualization and Computer Graphics 27.2 (2020), pp. 453–463.
Nicole Sultanum et al. “Leveraging text-chart
links to support authoring of data-driven articles
with vizﬂow”. In: Proceedings of the 2021 CHI
Conference on Human Factors in Computing Systems. 2021, pp. 1–17.
Mengdi Sun et al. “Erato: Cooperative Data Story
Editing via Fact Interpolation”. In: IEEE Transactions on Visualization and Computer Graphics(2022).
Aoyu Wu et al. “Ai4vis: Survey on artiﬁcial intelligence approaches for data visualization”. In:
IEEE Transactions on Visualization and Computer Graphics (2021).
Aoyu Wu et al. “MultiVision: Designing Analytical Dashboards with Deep Learning Based Recommendation”. In: IEEE Transactions on Visualization and Computer Graphics 28.1 (2021),
pp. 162–172.
Zhuohao Zhang et al. “CODAS: Integrating Business Analytics and Report Authoring”. In: (2022).

586