Fraunhofer Institute for
Telecommunications

Heinrich-Hertz-Institut, HHI
Annual Report

Annual Report2019 | 2020 | 2021

﻿Contents

Contents

Preface �������������������������������������������������������������������������������������������������������������������������������� 1
Shaping the future �������������������������������������������������������������������������������������������������������������� 2
New AI department at Fraunhofer HHI ���������������������������������������������������������������������������� 2
More than 90 years of innovation for the digital society ���������������������������������������������� 3
The Fraunhofer Heinrich-Hertz-Institut ���������������������������������������������������������������������������� 4
Fraunhofer HHI facts and figures �������������������������������������������������������������������������������������� 5
Members of the board of trustees ������������������������������������������������������������������������������������ 9
Honors �������������������������������������������������������������������������������������������������������������������������������� 11

Publishing notes
Published by
Fraunhofer-Institut für Nachrichtentechnik,
Heinrich-Hertz-Institut, HHI
Corporate Communications
Einsteinufer 37, 10587 Berlinwww.hhi.fraunhofer.de
Editorial teamMartina Mueller und Ronja Schillings
DesignInfographics and layout: Juliane Grosser
Cover: Clàudia Villagrasa

Fraunhofer HHI focus topics �������������������������������������������������������������������������������������������� 13
From explainable AI to trustworthy AI������������������������������������������������������������������������������ 15
Virtual humans − modeling, interaction and streaming �������������������������������������������������� 17
Digital twins and AI in construction engineering�������������������������������������������������������������� 19
Green AI�������������������������������������������������������������������������������������������������������������������������� 21
AI-based video coding ���������������������������������������������������������������������������������������������������� 23
Quantum key distribution at Fraunhofer HHI�������������������������������������������������������������������� 25
Hybrid photonic integrated circuits for 6G wireless networks and beyond ���������������������� 27
Research for the energy transition: lithium-ion batteries ������������������������������������������������ 29
LiFi ���������������������������������������������������������������������������������������������������������������������������������� 31
THz communication for future 6G networks ������������������������������������������������������������������ 33
AI for photonics and edge ���������������������������������������������������������������������������������������������� 35
Fraunhofer HHI departments ������������������������������������������������������������������������������������������ 37
Fraunhofer HHI organizational structure ������������������������������������������������������������������������ 37
Photonic Networks and Systems ������������������������������������������������������������������������������������ 40
Photonic Components ���������������������������������������������������������������������������������������������������� 44
Fiber Optical Sensor Systems ������������������������������������������������������������������������������������������ 48
Video Communication and Applications ������������������������������������������������������������������������ 52
Vision and Imaging Technologies ������������������������������������������������������������������������������������ 56
Wireless Communications and Networks ������������������������������������������������������������������������ 60
Artificial Intelligence �������������������������������������������������������������������������������������������������������� 64

Reproduction of any material requires theeditorial team’s consent.

About us������������������������������������������������������������������������������������������������������������������������������ 68
Showrooms �������������������������������������������������������������������������������������������������������������������� 68
Collaborations and Fraunhofer groups ���������������������������������������������������������������������������� 70
Research Fab Microelectronics Germany �������������������������������������������������������������������������� 72
ITU focus groups at Fraunhofer HHI �������������������������������������������������������������������������������� 74
Career opportunities: working at Fraunhofer HHI ���������������������������������������������������������� 76
Where to meet us ���������������������������������������������������������������������������������������������������������� 78© Fraunhofer HHI, 2022

Contact details�������������������������������������������������������������������������������������������������������������������� 80

2

New AI department at Fraunhofer HHI© Fraunhofer HHI© iStockphoto.com/Traitov

PrefaceProf. Martin Schell (l.) andProf. Thomas Wiegand (r.)

PrefaceVideo conferencing was used extensively during the COVID
pandemic − and the video compression research taking place
at Fraunhofer HHI made significant contributions to this technology. Today, more than 50% of all the bits on the internet
are formatted using video codecs we have had a hand indeveloping.
We have seen an increase in the number of cyber attacks inrecent years, which is why it is vital that we keep developing
quantum cryptography, a technology that secures communication by exchanging quantum keys. As part of the QuNET
initiative, we work with partners on driving forward quantum
communications.

1

Through our work in AI explainability, we are doing our part tomake it possible to use AI reliably in security-critical areas in thenear future. This AI-related work is also brought to a worldwide
audience by means of international standardization and through
ITU focus groups involving the WHO, WMO, UNEP and FAO.
Our experts in wireless communications have begun the taskof developing the next-generation wireless standard, 6G. To
achieve this, they have managed to obtain funding for their
6G-RIC project from the German Federal Ministry of Education
and Research (BMBF) − the largest project to receive BMBF
funding. 6G will make it possible to provide universal opportunities for using digital twins and augmented reality - two areasthat our experts are developing for computer vision purposes.
Our researchers working in the area of sensors have achieved
advancements in performance and developed new industrial
applications. This is enabling them to work in a way that conserves resources and use the fiber-optic infrastructure for more
applications.
In our annual report, we showcase our research departments,
our highlights and everything else that has marked an important step for us.

Fraunhofer HHI is one of Germany’s
leaders in artificial intelligence (AI)research. With the aim of reinforcing
this position, in 2021 the instituteestablished a new research department
dedicated to this important subject.
Since then, Prof. Wojciech Samek andhis team have been working hard on
advancing their multiple award-winningbasic research into AI and developing itwith a focus on industrial applications.

Prof. Wojciech Samek

Digitalization is a comprehensive approach to developing
solutions that provide effective ways of rising to the challenges
faced by society. Digital processes that work well are essential
for creating a good quality of life − in a world where foodproduction is sustainable, healthcare coverage is universal andpersonal data are secure. Promoting digitalization is one of ourguiding principles here at Fraunhofer HHI, and it is reflected inevery area of our research. The diverse array of subject areas
we deal with puts us in a position to help society achieve progress in numerous areas.

Shaping the future
New AI department at Fraunhofer HHI
The new department arose out of theMachine Learning research group in the Video
Communication and Applications department,
immediately kicking things off with an experienced team that was able to expand on thefocus of the group using a solid foundation ofresearch work. The Explainable Artificial Intelligence, Applied Machine Learning and Efficient
Deep Learning research groups develop
solutions for the challenges that AI is currently
facing, namely explainability, energy efficiency
and expanding its fields of application.
Through the new department, Fraunhofer HHI
is advancing its expertise in Berlin as a location
for AI research and within the global AI community. The researchers are already working
on several international projects, looking at
the development of AI systems in skin cancer
detection, air quality improvement, medical
assistance systems, heat protection in urban
areas, early-warning systems and digital rail
operations.

Our aim is to completely
rethink AI by moving
away from a centrally
trained, opaque form of
AI that gobbles up dataand energy, and instead
toward something thatis always learning andhas green attributes −something that is able toexplain its decisions andis conscious of its own
inherent uncertainties.
Prof. Wojciech Samek
Head of the Artificial
Intelligence Department

2

More than 90 years of innovation for the digital society

76

More than 90 years ofinnovation for the digital society

institutes

Through its achievements in digital infrastructure research,Fraunhofer HHI has played a major role in shaping and advancing
digitalization.

Main locations
Secondary locations

The Fraunhofer
Heinrich-Hertz-Institut
Located in Berlin, the research institute wasfounded in 1928 as the Heinrich-Hertz-Institut
für Schwingungsforschung (HHI for researchon oscillations) and has been part of the
Fraunhofer-Gesellschaft since 2003. Throughits achievements in digital infrastructure
research, Fraunhofer HHI has played a majorrole in shaping and advancing digitalization.

• There is a 50% likelihoodthat a bit being transmitted overthe internet will encounter a photonic component whose core has
been developed at Fraunhofer HHI.
In many cases, these will even havebeen constructed in the in theinstitute's cleanroom.

The Fraunhofer-Gesellschaft (FhG)
With approximately 30,000 employees and
76 institutes, Germany’s Fraunhofer-Gesellschaft
is the world’s foremost applied researchorganization.

3

Over the years, many of the innovations
developed by Fraunhofer HHI researchers havegained worldwide recognition and visibility −and today, almost every second bit on theinternet is shaped by Fraunhofer HHI technology in two ways:• During data transmission, around
50% of bits on the internet areformatted using the video coding
technology which Fraunhofer HHI
played a major role in developing.

© Fraunhofer HHI

The demands being placed on broadband
capacity are rising all the time, but progressin expanding digital infrastructure is lagging
behind even in industrialized nations, making
it difficult to rise to this challenge. As partof its research, Fraunhofer HHI is working toensure that digital infrastructure is constantly
improving on its performance.

Over

50%
of bits on theinternet are basedon the work
undertaken at
Fraunhofer HHI

Fraunhofer HHI building, Berlin

Contact details
Around 570 employees deal with an extensive
range of digitalization-related subjects acrossseven research departments. Working in threedifferent locations, they focus on optical networks and systems, photonic components,
fiber optical sensor systems, wireless communications and networks, video communication
and applications, vision and imaging technologies, and artificial intelligence.

Fraunhofer Institute for
Telecommunications,Heinrich-Hertz-Institut, HHI
Einsteinufer 37, 10587 Berlin
Phone +49 30 31002-0
info@hhi.fraunhofer.de
www.hhi.fraunhofer.de4Fraunhofer HHI facts and figures

Fraunhofer HHI facts
and figures

Chairs
University

Department

Since

Technische
Universtität
BerlinOptic and Optoelectronic Integration
Prof. Dr. rer nat. Martin Schell

2012

Media Technology
Prof. Dr.-Ing. Thomas Wiegand

2008

Photonic Communication Systems
Prof. Dr. -Ing. Ronald Freund2017

Network Information Theory
Prof. Dr.-Ing. habil Slawomir Stanczak

2015

Terahertz Sensor Technology (as part of the EDCF)
Prof. Dr. rer. nat. Björn Globisch

2019

Advanced Wireless Communications
Prof. Dr. rer. nat. habil. Volker Jungnickel

2021
Previously AP

Locations
BerlinGoslarStaff** headcount 		

5

331 2017
339 2018
375 2019
361 2020
386 2021

206 2017
236 2018
240 2019
304 2020
281 2021

Permanent staff				Students

Scientificexcellence

Associate Professor
PD Dr.-Ing. Oliver Schreer

2006

Humboldt
Universität
BerlinVisual Computing
Prof. Dr.-Ing. Peter Eisert

2009

Freie
Universität
BerlinImage Signal Processing
Prof. Dr.-Ing. Heiko Schwarz

2017

Universityof Potsdam

Embedded Systems Architectures
for Signal Processing
Prof. Dr.-Ing. Benno Stabernack

2008

Clausthal
Universityof Technology

Applied Photonics
Prof. Dr. rer nat. Wolfgang Schade1998

Extraordinary Professor
Ext. Prof. Dr. Eike Hübner

2016

Total budget in euros(million)

Master’s/graduate

2017
2018
2019
2020
2021

Doctorates

33

Publications
27

28

21

31

24
39
45

Patents

537

50

514

2017

2018

2019

2020

2021

53,5

56,9

62,4

63,9

71,3

53

526

67

21

625

67

21

651

66

23,9
25,4
24,2

25,8

22,9
18,6

9,6
7,3
12,4

Industry contracts

3rd

13,5

14,0

7,7

9,4

11,0

15,2

14,1

13,1

11,1

National projects

8,7
20,1

EU projects/Other

Base funding

4,0

8,9

Investments

place** FhG’s field-specific citation rate

6

5,8
7

4,3

5,2

8

Members of the board of trustees

Members of theboard of trustees

Seniorrepresentatives
from science andindustry lend
their support toFraunhofer HHI’s
work.

Term
2019–2021

Prof. Dr.-Ing. Klaus Petermann
Chair of the board of trustees,
Institute of High-Frequency and
Semiconductor System Technologies,
Technische Universität Berlin

MinR Dr. Alexander Tettenborn
Manager of Subdivision VI B National
and European Digital Agenda,
German Federal Ministry for Economic Affairs
and Climate Action

Prof. Dr. rer. nat. Klaus-Robert Müller
Machine Learning Department,
Institute of Software Engineeringand Theoretical Computer Science,
Technische Universität Berlin

Dr.-Ing. Werner Mohr
Deputy chair of the board of trustees,
Head of Research Alliances, Nokia Solutions
and Networks GmbH & Co KG

Prof. Dr.-Ing. Josef Hausner
Division Vice President,
Apple Mobile B.V. & Co KG

Prof. Dr. Susanne Stürmer
President,
Film University Babelsberg
Konrad Wolf

Ulrich BarthHead of Mobile Network
Performance & Optimization Research,
Bell Labs Stuttgart Site Leader,
Nokia Bell Labs/Alcatel-Lucent
Deutschland AG

Thomas Caspers
Head of Technical Centresof Excellence Division,German Federal Office forInformation Security

Dr. Thomas Engel
Principal Key Expert Research Scientist
in Sensors and Optical Technologies,
Corporate Technology, Siemens AG

Dr.-Ing. Christoph Glingener
Chief Technology Officer,
Chief Operating Officer,
ADVA Optical Networking SE

9

Dr. rer. nat. Wilhelm G. Kaenders
Member of Executive Board,
TOPTICA Photonics AG
Prof. Franz Kraus
Member of Executive Board,
ARRI AG

Michael Liehr, Ph.D.
Executive Vice Presidentof Technology and Innovation,
SUNY Polytechnic InstituteBernd Lietzau, Senate Councillor
Spokesperson for Engineeringand Technology Transfer,
Senate Department for Science,
Health, Care and Equality, Berlin

Prof. Dr. Lothar H. Wieler
President,
Robert Koch Institute

Dr. Fiona Williams
Research Director,
Ericsson Eurolab Deutschland GmbH

Antje Williams
Senior Vice President,
Deutsche Telekom AG

Dr. Christian Winkler
Head of Research & Senior Principal,
Siemens AG, Corporate Technology

Dr. rer. pol. Michael Meyer
Head of Strategy & Business
Development Germany,
Head of Government Affairs Project Office,Siemens Healthineers

10© FraunhoferMarc Müller

© Iris Haidau

Honors fromrecent years

2019
1.	

Leopoldina

Between 2019 and
2021, researchers at
Fraunhofer HHI
received a range ofhonors recognizing
their research workin various areas:

Leopoldina, the German National Academy of Sciences,appointed Prof. Thomas Wiegand as a member of its
Informatics section.

2.	

Photonic Integrated Circuit Lifetime
Achievement AwardAt the PIC Awards, held at the PIC International Conference, Prof. Martin Schell was the recipient of the Lifetime
Achievement Award in recognition of his life’s work in
the area of PIC development.

3.	

Klung-Wilhelmy-Wissenschafts-Preis
The Faculty of Electrical Engineering and Computer
Science at TU Berlin honored former Fraunhofer HHI
researcher Dr. Christian Schmidt with this dissertation award given by the Dr. Wilhelmy Foundation. He
received the prize as a commendation for the dissertation he completed at Fraunhofer HHI.

11

© Fakultät IV, TU Berlin Setzpfandt

Auszeichnungen der letzten Jahre

2021

2020
4.	

Förderpreis-ARD/ZDF
Former Fraunhofer HHI researcher Dr. Jennifer Rasch
received the ARD/ZDF Women + Media Technology Award. The dissertation that she completed at
Fraunhofer HHI was selected as one of the three
best degree papers of the year.

5.	

Hugo-Geiger-Preis
Dr. Sebastian Lapuschkin, head of the Explainable
Artificial Intelligence research group at Fraunhofer
HHI, was honored with the prestigious Hugo Geiger
Prize. He received this in recognition of his doctoral
thesis, as part of which he developed the XAI process known as layer-wise relevance propagation.

7.	

Applied Photonics Award

TU Clausthal student Luise Hoffmann received
the Applied Photonics Award for young talentfollowing the completion of an outstanding
master’s thesis with the support of Fraunhofer
HHI.

8.	

Best Open Source ProjectAwards der ICMEThe Fraunhofer Versatile Video Encoder (VVenC)
developed at Fraunhofer HHI scooped first placeat the Best Open Source Project Awards held at
the IEEE International Conference on Multimedia
and Expo (ICME).

6.	

Best Trainee
Business administration trainee Charlyn Domke washonored by the Fraunhofer-Gesellschaft as one of
the best trainees of the year. She passed her training at Fraunhofer HHI with the grade “Excellent”
and was joined in the award by Martina Keil, whose
outstanding service was also recognized.

Top left: Prof. Martin Schell at the PIC Awards;
middle: Dr. Sebastian Lapuschkin receiving the
Hugo Geiger Prize; right: Dr. Christian Schmidt
receiving the dissertation award from theDr. Wilhelmy Foundation

12

Fraunhofer HHI Focus Topics

Fraunhofer HHI
Focus Topics

1

Pioneering technologies
and solutions
The highlights of Fraunhofer HHI’s research over the last three
years reflect its future strategic orientation. The institute is abyword for applied research across a broad spectrum of communication technologies. There is a constantly growing need forinterdisciplinarity, particularly as researchers become involved inincreasingly interdepartmental projects, covering topics such as
artificial intelligence, quantum communication, LiFi, augmented
reality, mixed reality and image analysis for Industry 4.0, all theway through to medical applications.

Contact details
Fraunhofer Institute for
Telecommunications,Heinrich-Hertz-Institut, HHI
Einsteinufer 37
10587 Berlin
Germany

13

13

14

© iStock.com/akinbostanci

We might consider the perceptron
as a black box.
Frank Rosenblatt, 1957
The perceptron – A perceiving and recognizing automaton

From explainable AI
to trustworthy AI
AI offers huge potential benefits in a variety of
application areas, where it is not just the performance of the AI system as measured in the lab thatis the decisive factor, but also the transparency of its
predictions. In response to this requirement, Fraunhofer HHI has established research into explainableartificial intelligence (XAI) as one of its main topics.
The institute’s Layer-wise Relevance Propagation(LRP) method, developed in partnership with theTechnical University of Berlin (TU Berlin), allows users
to understand the decision-making basis of individual
AI predictions and is one of the mostly widely used
XAI technologies all over the world.

Explainable AI beyond transparency
Over the last four years, Fraunhofer HHI has been working tosystematically enhance the LRP process and to optimize andsuccessfully deploy it for various AI problems beyond classification (e.g. clustering, anomaly detection, regression) andfor areas of application such as medicine (e.g. gait analysis,
histopathology, dermatology, neuroimaging and epidemiology). Further XAI tools have been developed that build on
LRP. These include Spectral Relevance Analysis (SpRAy) andthe Class Artifact Compensation Framework (ClArC). SpRAy
enables users to lift the lid on systematic behavior patterns and15

prediction strategies in AI models beyond individual decisions,and to explain them. Meanwhile, ClArC enables precise andefficient corrections to be made to undesired behavior patterns
that have been learned within the AI models. These new XAI
tools provide a way of systematically debugging AI models
and therefore pave the way for a transition from explainable totrustworthy AI.
Other areas of application for the XAI methods developed at
Fraunhofer HHI go far beyond the goal of increasing transparency. For instance, LRP can be used to conduct functionality
evaluations of the internal structures of AI models, allowing
non-functional elements to be compressed or removed. This
yields enormous efficiency gains in terms of model size, prediction times and energy requirements. Applying the principle of“keep the best and remove the rest” allows very large models
to be distilled down to a hundredth of their original size, without any drop in performance.
Fraunhofer HHI is currently eagerly working on new approaches to explainability (XAI 2.0), which can be used to explain
AI predictions in a way that humans can understand. These
explanations are provided in the form of generally understandable visualizations, which can also be used as an interface forinteracting with the AI model in a variety of ways (e.g. correcting the model by correcting the explanation).

Auditing and certification of AI systems
In critical areas of application such as mobility, biometrics andmedicine, transparency is not the only essential characteristicof AI systems. They also need to be secure, robust and reliable.
To ensure that AI-based systems meet these requirements,
an AI auditing process and specific AI certification procedures need to be developed. To this end, Fraunhofer HHI,
the German Technical Inspection Association (TÜV) and theGerman Federal Office for Information Security (BSI) organized
two “Auditing AI Systems” workshops, which took place in
October 2020 and October 2021. These were attended byinternational AI specialists at the Berlin CINIQ Center/Forum
Digital Technologies to lay out the groundwork for an auditable AI system. The participants published the outcomes of
their discussions in two whitepapers*.
Fraunhofer HHI is working intensively to put its expertise in
explainable AI, model debugging and verification/validationof modern AI systems towards developing future AI standards
that will ensure the responsible use of the technology in healthand other safety-critical areas, such as autonomous driving.
These standards need to include both qualitative and quantitative quality control procedures along the entire AI life cycle
in order to ensure that the AI tools are precise, robust, fair andtrustworthy as a whole.

AI standards for health and medicine:
ITU/WHO Focus Group on AI for Health
The applications in clinical and public health that could benefit
from AI-supported automation are many and varied. Before
and whilst using any AI tool in practice, it is vital to ensure thatthe method and its implementation are safe and effective andthat they can be trusted to perform the function for which
they are deployed. Here it is also important to ensure that the
AI tools − developed and tested under laboratory conditions −do actually work in clinical working environments. Ideally,
these tools should be equally deployable in different environments, clinics and German states.
In 2018, the International Telecommunication Union (ITU) andthe World Health Organization (WHO) created a focus groupon the topic of “AI for Health” (FG-AI4H) as a shared initiativefor exploring standardization options in this area. The FG-AI4H
is led by Prof. Thomas Wiegand, Executive Director of Fraunhofer HHI, and receives expert and organizational support
from the AI department. The FG-AI4H establishes processes
and the associated guidelines and issues reference documents
containing best practices. This also includes annotation tools
and an AI test platform.

* To the whitepapers

16© Fraunhofer HHI© iStock.com/TheItern /Ljupco /klikk edit by Fraunhofer© Fraunhofer HHI

Virtual humans –modelling, interaction and streaming
Building on the acclaimed volumetric video technology developed by Fraunhofer HHI researchers,
methods for modelling interactive virtual people havebeen developed, along with streaming technology
for latency-free transmission. The purpose of thesetechnologies is to make meeting and interactingwith virtual people a reality. The new methods are
opening the door to interacting with realistic virtual
people in virtual reality (VR) and augmented reality(AR) environments.

Volumetric video
The technology used to generate volumetric video records
real people with multiple cameras at the same time. This
allows highly realistic dynamic 3D models to be created, which
can then be viewed from multiple angles in a virtual scene.
Fraunhofer HHI has its own recording system consisting of 16
stereo cameras and approx. 150 light panels that can capture
3D information about the person from all directions. Using
the same principle as the human eye does to achieve threedimensional vision, this technology allows 3D geometry to be
captured, fused and transposed into a consistent, naturalisticand dynamic 3D representation of the person. It has established itself as an important step in the development of media
production around the world. Content produced by Fraunhofer

17

HHI has been presented in many VR productions. This includes
enabling visitors to the Sachsenhausen concentration camp
memorial to have a three-dimensional life-sized virtual encounter with Holocaust survivor Ernst Grube using virtual reality
glasses.

From volumetric video to virtual people
Volumetric video allows highly realistic dynamic models ofpreviously recorded actions to be reproduced and passively
observed. Now, building on these highly realistic data, Fraunhofer HHI researchers have developed methods for generatinginteractive virtual people. This involves using hybrid models
that combine semantic information from animatable computer
graphic models with highly realistic volumetric video data andthe latest methods in AI-based animation and neural rendering. The first step in this process is adapting the geometryof the computer model to the volumetric scan. Next, theconcordance between the two formats is calculated so thatthe animations and semantic information from the computer
graphics model can be transferred to the volumetric scan. This
allows a virtual person to maintain eye contact with the user,
for example. This technology has been successfully introduced
into the new MPEG standard.

Using the latest AI methods, unique personal movement
patterns or facial expressions are learned from the enrichedvolumetric video data, which then allows generative models
to synthesize new movements or facial animations, and to do
so in a way specific to the individual person. Methods for generating personalized animatable facial models for use in visual
speech synthesis have been developed in this way, amongst
other uses. These can realistically synthesize animation thatmatches spoken text or speech. Gestures and movements canalso be learned and synthesized based on the language. Thiscan then be used, for example, to provide immersive videoconferencing systems in virtual rooms. There is also scope forenormous advances in online teaching as a result. Students caninteract with highly realistic avatars of lecturers using a mixed
reality (MR) headset, allowing the interactivity of in-person
teaching to be combined with the flexibility and inclusion of
online teaching.
In addition to the technological developments, the ethical andperception theory considerations are also vital in this context.
These are being investigated in close cooperation with ethicists
and neuroscientists.

Streaming and transmission
To facilitate efficient transmission and display of interactive
volumetric videos on mobile devices, Fraunhofer HHI researchers have developed a transmission process that outsources theenergy-intensive rendering process to an edge cloud. The rendered video is transmitted as normal 2D video, decoded andintegrated into the scene on the glasses. In the other direction,
the glasses transmit line-of-sight information to the edge
cloud in order to position the render camera on the server insync with the user’s movements, and to turn the avatar’s headtowards the correct direction, for example. This process has tobe rapid, of course.
A situation where the professor is always looking at where
the student was a moment ago, i.e. always looking slightly
away from them, could hardly be described as realistic. The
latency required is under 40 milliseconds in both directions.
The system that has been developed achieves this latency by
optimizing all of the components of the WebRTC standard
recognized in online conferencing, a machine learning-based
estimation of the future movement of the user and an
extremely rapid system for adjusting the data rate to the
bandwidth available in the radio channel via direct feedback
from the network.

It also makes synthesizing entirely new movements possible.

18

The construction industry currently has major challenges to overcome in order to keep up with the
major demand for new construction, upkeep, energy-saving refurbishment, maintenance and deconstruction of buildings. Digital twins can help to simplify planning and to make processes more efficient.
It is in this context that Fraunhofer HHI is researching
AI-based methods for automatically generating andusing digital twins in construction engineering.

AI methods for digitalization and capturing of
as-built status
Most of today’s construction work is carried out on buildingsthat already exist. Whilst there are usually large amounts ofraw data available for these buildings, these data are often not
sufficiently digitized or they are inconsistent, or high-quality
building information modelling (BIM) is not available. The goalof the BIMKIT research project (the name of which is based on
the German for as-built modelling of buildings and infrastructure projects using AI to generate digital twins), supported by
the Federal Ministry for Economic Affairs and Climate Action(BMWK), is therefore to make use of AI methods to convert
multimodal raw data into detailed BIM models either automatically or using supporting tools. This involves making equal
use of, on the one hand, 2D plans in vector or raster format,
images and 3D point clouds of the current structures and,19

© Fraunhofer HHI/seele© Fraunhofer HHI© iStock.com/teekid

Digital twins and AI inconstruction engineering

Left: generating BIM models from 2D plans. middle: BIM model
as digital twin of a building. right: AR assistance supporting
complex assembly tasks.

on the other hand, documents and files on their construction, operation and maintenance. Within its modular system,
individual modules equipped with specialized neural networks
carry out partial tasks, the results of which are then merged
and validated or supplemented with expert knowledge.
Here, Fraunhofer HHI is contributing its expertise in machinelearning-based image analysis in order to extract high-quality
information from plans, images and point clouds and thengenerate detailed 3D models from that information. New
methods in geometric reconstruction allow quick live capturing of spaces to be performed via mobile devices or stereo
cameras, which can be integrated into a helmet for instance.
In addition to geometric information, AI processes can clean
up raw data or carry out semantic segmentation, i.e. allocate
the 3D points or pixels to semantic categories (e.g. door, wall,
window, fitting, etc.). Similarly, computer-aided design plandata can be analyzed and expanded into semantically enriched
models. The relationships between plan data and photos ofparts of buildings can be established using a new type of
2D-3D registration that works using semantic similarities andthe position within the plan where the image is taken. Material
classifications in the captured data enrich the model further.
These form the basis for sound simulations that can be used toevaluate the room and architectural acoustic properties within
the digital twin.

Digital twins for assembly support
The uses of digital twins are not restricted to planning processes. They can also be used to support assembly and construction of new structures. The goal of the DigitalTwin project,supported by the German Federal Ministry for Economic
Affairs and Energy (BMWi), is to use digital tools to create amore effective network between individual participants along
the entire value chain. Using extended reality (XR) glasses,
additional information can be blended into the user’s line ofsight in sync with their physical view. This allows data fromthe digital twin to be visualized within the specific context. In
the other direction, data about working processes captured
via cameras on site are then reflected in the digital model. Theinterface is an XR visualization framework developed at Fraunhofer HHI and offers extensive customization options.

track multiple objects in terms of their position in the 3D
space simultaneously and in real time, which means, forexample, that the progress of the construction can be logged.
During the assembly, arrow inscriptions in the AR glasses indicate at what point and in what orientation an object needs tobe positioned, which saves time and reduces the risk of assembly errors. As part of the DigitalTwin project, assembly support
was provided for an entire construction using a 3D-printed gridshell model. An interactive demonstration is currently being set
up in the Forum Digital Technologies in Berlin.

In one practical application case, project partner seele used thesystem to support the assembly of steel-glass facade structures. One challenge here is the geometric similarity of theindividual steel node elements that give the spherical grid shellstructures their shape. To overcome this similarity and identify
the nodes, an AI-based combined localization, classification
and angle-estimation system for 3D objects was developed
from 2D images. What makes this approach unique is that the
annotated training data for the neural network were generated entirely synthetically. This means that it can be easily andautomatically extended for new objects. The process is able to20© Kirill Savenko/Shutterstock.com

Hardware-optimized systems, efficient coding
processes and special distributed learning
methods using resource-saving devices arebeing developed in order to reduce the energy
consumption of AI processes.

Green AI

Increasing the efficiency of AI-specific algorithms
and processes is taking on a special significancein the context of worldwide efforts to reduce CO2
emissions. Artificial intelligence is currently being
deployed in many important areas and is constantly
entering new fields of application. The neural networks have become increasingly complex in recent
years and typically require significant processing
power to train and run, which means they come withmajor energy requirements. In addition, there has
been an expansion of distributed learning processes,
requiring increasing amounts of data to be transmitted regularly in the form of parameter updates fordeep neural networks.

Energy-efficient AI at Fraunhofer HHI
As part of the focus topic “Green AI,” Fraunhofer HHI is developing solutions to reduce the energy requirements of AI applications. This includes methods that make neural networks,
and therefore AI, more energy-efficient to train, apply andtransmit. These methods include hardware-optimized systems
to lower the energy requirements of running AI models, distributed learning processes and coding algorithms to increase
the efficiency of transmitting neural networks. Fraunhofer
HHI is cooperating with various partners on this point, such as
dena (Deutsche Energie-Agentur GmbH), and companies using

21

AI applications in data centers and other institutions as part ofprojects focusing on energy-efficient AI. It is also participating
in international standardization initiatives for neural network
coding.

CO2-reduced AI in data centers
Developing special hardware for running neural networks
with less energy is a major topic. These methods are speciallydesigned for data centers in order to reduce the CO2 footprint of today’s complex AI processes, such as data classification using neural networks. Currently, data centers mainly
use systems consisting of host computers or workstations,connected to a number of specialized processing accelerators such as graphics cards (GPUs), which therefore require asignificant amount of energy to carry out the processing. As
an alternative, Fraunhofer HHI is developing special processing
accelerators that are based on field-programmable gate arrays(FPGAs) and are directly connected to the network of a data
center via a network interface. This yields energy savings of up
to 70%, thanks firstly to the FGPA architecture and secondly tothe autonomous accelerator architecture (known as a “network attached accelerator” or NAA), which does not requirea host. The FPGAs are specifically configured for an individual
neural network, thereby minimizing their energy consumption
compared to a GPU.

Distributed AI

Standardized AI coding and transmission

Another major research area is the development of distributed
AI systems, focusing in particular on energy efficiency. Distributed learning systems are particularly useful in application
areas that are subject to strict data protection requirements.
This is because (unlike the centralized learning approach), theonly data exchanged are the parameters of the neural network. This means, for instance, a business unit that handles
sensitive data can train a shared neural network and use it for
further data analysis without having to give away its local data.
For example, several hospitals can train a network on theirown local computerized tomography (CT) data and create a
joint neural network that has greater diagnostic power (e.g.
in identifying tumor cells) and is shared by all the participants,
but without having to share their CT data.

An important element of energy-efficient distributed AI
processes is the efficient transmission of the neural network
parameters. Neural network updates need to be transmitted
frequently, particularly during the training processes. Consequently, Fraunhofer HHI has in recent years developed efficientcoding procedures for compressing neural networks and has
been working since 2019 on the international standardizationof these processes as part of the ISO/IEC MPEG committee.
This led to the creation of the NNC standard (Neural Network
Coding — ISO/IEC 15938-17), to which Fraunhofer HHI contributed two thirds of the underlying technologies, e.g. DeepCABAC, a particularly efficient coding method. The NNC standard can be used to compress neural networks to a fractionof their original size (e.g. <5% for a VGG16 network), without
losing any of the classification precision. A second edition of
the NNC standard is currently being developed specifically for
updates to neural networks, which will allow further bitratesavings to be achieved.

In addition, distributed learning systems can offer potential
energy savings compared to carrying out centralized training in large data centers. This is because distributed learningnormally takes place on low-resource devices, therefore doing
away with the need for energy-intensive cooling. Accordingly,
Fraunhofer HHI is focusing efforts on developing federated
learning methods, where the local learning process is specially
adapted to the client devices being used and, once trained, theparameters of the neural networks are transmitted efficiently.

Feasibility studies showed that the NNC standard also helps
to reduce energy consumption, e.g. in distributed learning
environments. Here, a net energy saving can be achieved, astransmitting heavily compressed neural networks saves significantly more energy than has to be expended for the additional
encoding and decoding required.

22© kentoh/Shutterstock.com

Data-driven approaches are being used
to design new types of video coding
processes with increasing success.
Dr.-Ing. Detlev Marpe,
Video Communication and Applications

AI-based video coding

Communicating via video conferences, video streaming
or in new formats such as virtual or augmented reality
is a permanent fixture of everyday life in modern connected societies. The scale of video data that have to betransmitted all over the world for these sorts of applications is constantly increasing. Given the limited bandwidth available within the transmission channels, there
is an urgent need for ways to improve the efficiency ofvideo-data coding.
In the Video Communication and Applications (VCA)
department, video-coding algorithms are being developed in order to improve coding efficiency comparedto the state-of-the-art video-coding standard Versatile
Video Coding (VVC). Data-driven methods are increasingly being used to design these algorithms. Three
examples are presented below in more detail to illustrate the current research activities being carried out by
the department in this area.

processes for video compression in order to reduce the bitrateof the prediction errors that need to be transmitted. In modern
video codecs, a block is predicted on the basis either of neighboring samples within the same image or of samples fromanother previously decoded image. The first option is referred
to as intra-prediction and the latter as inter-prediction. In
order to achieve noise reduction, inter-prediction is commonly carried out by averaging two individual prediction signals,
a process known as bi-prediction. The prediction method
being developed in the VCA department combines intra- andinter-prediction. To do this, the prediction signal is generatedusing a convolutional network which processes both the individual inter-prediction signals from the bi-prediction process
and the neighboring reconstructed samples of a block together. This new prediction method has been shown to be capableof significantly reducing the prediction errors in the original
inter-prediction signals. It has been integrated as a prediction
method into the framework of the VVC codec where it yields
significant coding gains.

Combined intra- and inter-production

Data-driven design of in-loop filters

Predictive coding is a key component of modern hybrid blockbased video codecs. This involves using previously reconstructed samples to generate a signal prediction for a blockwithin an image of a video sequence that needs to be coded.
The VCA department is working on more efficient prediction

When decoding a video signal, errors can occur once the prediction signal and the reconstructed residual error have been
added together. The temporal prediction applied between theimages can significantly increase these errors. For this reason,
today’s video codecs contain specific signal adjustments,

23

known as “in-loop filters,” that are applied to individual
images within a video sequence before they are displayed orused to predict further images. One of the defining characteristics of the VVC standard is that it contains a large amount ofcoding options, such as numerous combinations of different
transformations and prediction methods. It would be difficult
to describe and minimize all of the coding artifacts that may
result from these many and varied coding processes.
To deal with this issue, the VCA department took a data-driven
approach in order to develop an in-loop filter for a VVC-based
codec. Under this approach, signal adjustments are applied toan image based on a weighted sum of FIR filters. The weighting varies for each sample and is calculated from the signal
using a convolutional network. It can therefore be interpreted
as an indicator of whether a sample belongs to various classesof coding artifact. The in-loop filter developed by Fraunhofer
HHI can achieve significant coding gains against the VVC
standard. Compared to other approaches, this new method issignificantly less complex. Furthermore, splitting the process
into classification and filtering provides an interpretable architecture for convolution-based in-loop filters.

Vector quantization and motion representationin end-to-end compression
The core principle of what is known as “end-to-end” compression is using non-linear transformations to code certain typesof input signals. To transmit a signal, the transform coefficientsare quantized and the quantization indices are transmitted. It
is well established that vector quantization is the only way ofachieving optimal quantization of the transform coefficients.
However, as the transforms applied in end-to-end compression
are not orthogonal and are not even linear, more efficientquantization of the transform coefficients does not automatically lead to more efficient transmission of the original signals.
Despite this, the VCA department has successfully configured and modified the learning algorithm for determining
non-linear transforms when applying vector quantization. Thismethod yields significant coding gains compared to existing
end-to-end image compression processes.
Another key research focus in the VCA department is thedevelopment of efficient motion representations for use in
end-to-end approaches to video coding. This involves describing and transmitting the motion fields in order to predict animage based on a prior image. The first video codec based on
this concept has been successfully created.

24© Fraunhofer HHI© Fraunbofer HHI© BMBF

Quantum key distribution
at Fraunhofer HHI

Left: Federal Minister Anja Karliczek and Fraunhofer HHI Executive
Director Prof. Martin Schell during the quantum-secured videoconference; center: installation of Fraunhofer HHI’s QKD system at
the Bonn office of the German Federal Ministry of Education andResearch (BMBF); right: individual photon detectors based on InP
semiconductors and developed at Fraunhofer HHI

The challenges associated with future-proof andlong-term data security are only continuing to grow.
In response to which Fraunhofer HHI is working todevelop innovative solutions for quantum key distribution (QKD). These were successfully demonstrated in the first quantum-secured video conference
between two German Federal authorities.

and Research (BMBF) and the German Federal Office forInformation Security (BSI) in Bonn, the first practical example
of a typical application case for quantum key distribution. This
enabled these two Federal authorities to conduct their firstquantum-secured video conference.

Increased key rates for future quantum backbones
Optimizing quantum technologies for
specific applications
Fraunhofer HHI’s Photonic Networks and Systems and Photonic
Components departments are working on a large number ofprojects with partners in research, industry and authorities
on systems, components and concepts for quantum-secured
communication networks. One key area of these projects is
adapting the quantum key distribution process to the specificrequirements of different application scenarios and ensuring
it is seamlessly integrated into existing infrastructures. As oneof the four central institutes involved in the QuNET initiative,
Fraunhofer HHI has implemented a modular development
platform based on the time-phase-encoded BB84-QKD
protocol. This allows new technologies and applications to be
integrated incrementally and rapidly, and to be demonstrated
in lab and field experiments. Fraunhofer HHI’s QKD systemdelivered highly secure keys over the course of several weeks
to the offices of the German Federal Ministry of Education

25

Modern optical fiber networks allow a large number of communication channels of different wavelengths to be transmitted at the same time. Fraunhofer HHI is investigating to what
extent this approach can be used to increase transmission
rates in quantum communication. Working with university andindustry partners, up to 11 QKD channels were transmitted
simultaneously via a shared fiber. Thanks to a rapid Field Programmable Gate Array (FPGA) evaluation process developed
at Fraunhofer HHI, a key rate of over 14 Mbit/s was achieved,paving the way for highly secure quantum backbones between
metropolitan regions in future. In addition, the simultaneoustransfer of a quantum channel and traditional communication
channels via a single shared fiber was successfully demonstrated using Fraunhofer HHI’s QKD development platform.

Key technologies for quantum repeaters
As one of the 26 members of Project QR.X, Fraunhofer HHI
has also been working towards overcoming the current technical limitations on the distances over which quantum communication can operate by enhancing its own innovative key
technologies for devices called “quantum repeaters.” Fraunhofer HHI is also developing an entanglement-based quantum
communication system in order to allow quantum teleportation experiments to be conducted with partners. This will playa key role in any future quantum repeater architecture. The
institute is also permanently expanding its fiber testbed usinga hybrid connection to a fiber-coupled free-space link. This willallow the designs and modules developed within the group to
be tested.

High-performance detectors for quantum
communication

SPADs do not require extremely low temperatures to operate. This means that they are particularly suitable for use inquantum communication networks. The detector group, led
by Dr. Patrick Runge, has made significant progress in SPAD
development and, after a three-year development phase, is ina position to demonstrate high-performance components.
The SPADs developed at Fraunhofer HHI are competitive interms of characteristics such as detection efficiency, dark countrates, afterpulsing probability and jitter. As a critical component in the field of quantum key exchange, they represent afurther element of Germany’s technological independence asa quantum communication hub. The next anticipated stageof the QuNET initiative is to deploy the SPADs developed at
Fraunhofer HHI in the systems developed by the Photonic
Networks and Systems department.

As part of the QuNET initiative, funded by the German Federal
Ministry of Education and Research (BMBF), the necessary
hardware components are also being developed at Fraunhofer
HHI. A piece of technology called a “single photon avalanche
diode” (SPAD) is of relevance for exchanging quantum keys.
These InP components allow individual photons to be detected. Unlike superconducting nanowire single photon detectors,

26

Hybrid photonic integrated circuits for 6G wireless networks and beyond

Hybrid photonic integrated circuits
for 6G wireless networks and beyond
The Hybrid PICs group, in cooperation with the InP Foundry and the THz
Sensors and Systems group, has demonstrated for the first time fully integrated photonic wireless transceivers that pave the way towards the use ofphotonics in 6G and beyond wireless networks.

Using the most appropriate material systems
for the photonic functional building blocks

HHI's hybrid photonic integration platform PolyBoard enables the world’s first
fully integrated mmW/THz transmitter andreceiver PICs, demonstrating the potentialof photonics for 6G networks and beyond.

Photonic generation of mm-wave (mmW) and THz frequency
signals is key for wireless communications in 6G and beyond
due to the high bit rates that can be achieved with photonic
components. Numerous transmitter and receiver demonstrators have been reported in the literature that use separately
packaged lasers, modulators, and photonic THz transmittersand receivers, making these systems bulky and limiting theirimplementation in network devices. Moreover, to stabilize
the laser sources, e.g. to obtain a mmW/THz carrier with highstability and spectral purity, injection locking schemes basedon optical frequency combs and fiber-based optical isolatorsor circulators are typically employed, adding bulkiness andcomplexity.

© 2021 Gijs van Ouwerkerk

27

At a glance

The European project entitled Terahertz Technology for
Ultra-broadband and Ultra-wideband Operation of Backhaul
and Fronthaul Links in Systems with SDN Management of
Network and Radio Resources (TERAWAY) demonstrated fully
integrated mmW/THz transmitter and receiver PICs for wirelesscommunications based on Fraunhofer HHI's hybrid photonic
integration platform PolyBoard, addressing miniaturization
challenges by using the most appropriate material system fora photonic functional component. The PolyBoard includes, forexample, tunable lasers and optical isolators for optical signal
generation and on-chip implementation of the injection-locking scheme (see top figure at left). InP-based modulator PICs
from Fraunhofer HHI's InP foundry enable the generation of
optical frequency combs to stabilize the laser frequency andmodulate the data signal. In the emitter PIC, SiN-based opticalfilters fabricated by the Dutch partner LioniX allow for single

sideband operation and Fraunhofer HHI’s InP-based photonic
Terahertz transmitters and receivers are used for photonic
generation and detection of radio signals (see bottom figure atleft). The chiplets were partially assembled at Fraunhofer HHI
and finally packaged by the Dutch company PHIX, demonstrating for the first time fully integrated photonic wireless transmitter and receiver modules for 6G networks and beyond.

What is next?
In the coming generations of photonic wireless emitter andreceiver PICs, the number of lasers and modulators will be
increased to generate multiple wireless signals at different
frequencies. In addition, arrays of photonic transmitters andreceivers will be incorporated to allow for optical beam steering to provide the possibility of aligning the wireless links andtargeting moving nodes (e.g. drones).

Left: Hybrid-integrated photonic
mmWave/THz transmitter andreceiver PICs for communicationin wireless 6G networks

28© Fraunhofer HH© Fraunhofer HHI© Fraunhofer HHI

Research for the energy transition:
CO2-neutral real-world lab for preparing, recycling
and safety of lithium-ion batteries

Fraunhofer HHI’s CO2-neutral real-world lab is conducting thermal and electrical abuse tests of lithium-ion batteries capable of storing up to 150 kWh.
These investigations focus on a wide variety of safety
considerations for mobile and stationary applications
and the associated systems for recycling the lithium
batteries used in electric transportation (second-lifebatteries). For example, modular container-based
mini-grid systems used for the decentralized supply ofenergy in developing countries have been developed
at Fraunhofer HHI and have already been deployed in
on-site pilot studies.

this can result in fire and/or explosion. In Fraunhofer HHI’s
real-world lab at the Goslar site, thermically and electricallyinduced abuse tests are being carried out under predefinedconditions. One of the primary goals is to develop new safety
and extinguishing systems based on innovative sensors. Another is to test established safety solutions in collaboration withindustrial partners and to certify them in cooperation with the
Association for Electrical, Electronic and Information Technologies (VDE). In addition to storage modules capable of holding
up to 150 kWh, transportation and storage containers for LIBs
and extinguishing agents and equipment are being tested, andlong-term battery-cycle tests are being carried out.

Lithium-ion batteries (LIB) are currently the main energy sources in both electromobility and in stationary storage. They canplay a key role in ensuring the stability of the network or they
can supplement photovoltaic equipment on a private or industrial scale. Successful economic implementation of these storage systems places special demands on the associated safetyconcepts, including the extinguishing agents required in theevent of an accident. LIBs store large amounts of energy, butthey are also sometimes susceptible to the smallest technicalfault or inexpert handling, leading to uncontrolled dischargeof the chemically stored energy. In the worst-case scenario,

What makes Fraunhofer HHI’s real-world lab unique is that it
replicates the equipment’s entire energy supply process, including the exhaust gas treatment system via photovoltaics and alarge lithium-ion store. This storage is comprised exclusively ofwhat are referred to as “second-life photovoltaic (PV) modules” and “second-life LIBs.” This approach exemplifies how
CO2 emissions can be reduced in the development and testingof LIBs. As such, it also makes a valuable contribution to thecircular economy by reusing used PV modules and LIBs in an
ecologically and economically appropriate process that closely
replicates industrial practice. So far, little attention has been

29

Left: battery abuse test;
center: long-term batterycycle tests; right: electricalfishing boat

paid to combining second-life LIBs with second-life photovoltaics to create what are referred to as “mini-grid systems” forthe decentralized supply of energy in developing countries.
According to official estimates, approximately 86% of people
in the rural areas of Sub-Saharan Africa have no access toelectricity. The International Energy Agency (IEA) expects
that by 2040 approximately 140 million people in Sub-Saharan African could be supplied with power via mini-grids. In
addition to improving the quality of life for these populations,having access to electricity is the most important prerequisite
for local development, particularly in terms of accessing clean
drinking water. To this end, Fraunhofer HHI has been working with the start-up VoltaView GmbH to develop a modular
container-based mini-grid system for the decentralized supplyof power and treatment of drinking water.
The first of these systems was financed by the UK-based
NextEnergy Foundation and will be installed in Tanzania to
supply a school with electricity and drinking water in November 2022. Further systems are being constructed and will be
installed in western Africa (Gambia) at the end of 2022. In
addition to providing drinking water, the mini-grids can also
be used as a charging station for mobile portable LIB packs(battery-to-go with an energy capacity of 3 kWh). These canContainer-based mini-grid system with battery-charging service
and provision of drinking-water.

be used to provide households with electricity or as an energy
source for electrically powered fishing boats. A major advantage of this technology is that it does not require any elaborate
infrastructure. Working with the University of Gambia andsupported by the United Nations Industrial Development Organization, an initial pilot project has been set up on the island of
Jinack in Gambia.

30

Humans have been sending messages by light since
the dawn of history, using lighthouses or sending
Morse code via light over distances of several kilometers. The optical spectrum has been used for fiber
optic communications since 1965.
In the 21st century, visible and near-infrared lightis also being used for wireless communication inindoor spaces, achieving speeds measured in gigabits
per second. Adapted from the term wireless fidelity(WiFi) this technology is referred to as light fidelity(LiFi). LiFi provides an alternative way of achieving
wireless communication in environments where there
are challenges to using radio technologies. Light caneven be used to communicate wirelessly and within
environments affected by significant electromagnetic
interference, such as in industrial production contexts
or when using sensitive measuring devices in a medical context.
Fraunhofer HHI was involved in the earliest LiFi demonstrations and has continued to steadily develop the technology
ever since. LiFi makes it possible to use large additional
spectrums in order to, relieving the burden on the overcrowded radio spectrum. Expanding the network to include
several LiFi cells can provide access to the highest data rates
for all users. LiFi is of international interest as a new optical31

air interface solution in the context of 6G, but it can makea significant contribution to digital transformation in many
areas. Fraunhofer HHI’s constant efforts, combined with
its close partnerships worldwide, are enhancing Germany’sand Europe’s technological leadership in this area.

Supported by LiFi: digital transformation in
medicine and industry
Digital transformation is a key topic in all areas of life. This
includes medicine, where the use of digital technology is
becoming ever more important. Unlike in an office or at home,robust and secure wireless communication is essential here.
Examples include the seamless logging of a patient’s journeyfrom the site of an accident to the operating room or theprovision of high-resolution imaging content during treatment.
In these types of application, LiFi can help to improve healthprovision in smart hospitals.
As part of the LiFi-enabled 5G for Industrial and Medical
Networks (LINCNET) project, which was created in collaboration with the Carl Thiem Clinic in Cottbus and other project
partners, tailored LiFi solutions are being developed to meetthe requirements of medical and industrial settings. The goal
is to pave the way for comprehensive deployment in theseareas. In hospitals in particular, sensitive measuring devices arein constant use. These make it harder to use current or evenfuture radio-based transmission technologies. There are also
some hospital departments, such as radiology, that cannot use
radio communication because of their design and the X-ray
shielding in the walls. LiFi can offer a realistic alternative in thiscase. There are particularly extensive data-security requirements due to the sensitive nature of patients’ medical data.
LiFi networks are very easy to protect as the data remain
within the light beam and never go beyond precisely where
they are needed. They cannot be intercepted from outside. In
addition, there are plans to develop methods for simplifying
LiFi installation and to investigate new optical sources that canachieve high data rates.

© Sangikyo Corporation edit by Fraunhofer HHI

WiFi networks or Bluetooth speakers. LiFi can be retrofitted
anywhere and cannot pass through walls, which provides
additional data security. The vision is to use the future lighting
infrastructure as a distribution network for LiFi. The first
demonstrations of a powerline communication and LiFi combination have already highlighted the potential of this approach.

HHI LiFi System mit Gbit/s Datenrate

LiFi for applications in buildings
LiFi is also an attractive option for use in buildings due to theapplications that will form part of the future internet of things
and the resulting high data rates that will be required per user.
Augmented and virtual reality are becoming a more regularfeature in the home and require high-speed and stable wireless communication. LiFi hotspots with exclusive media accessand extremely high data density can be used to supplement
existing radio technologies such as WiFi. One advantage is thatthe optical spectrum is not susceptible to interference fromexisting radio-based systems, such as the many neighboring© Fraunhofer HHI© Fraunhofer HHI

LiFi

Light is becoming an increasingly important
medium for wireless communication.
LiFi systems behave like cables; they are secure
and provide high data rates, low latency andhigh reliability.

Fraunhofer HHI LiFi system with Gbit/s data rates

Top left: using LiFi for point-to-point networking of buildings,
smart cities and fixed wireless access applications; top right:
robust vehicle-to-vehicle communication using LiFi

32

© 2021 Gijs van Ouwerkerk© Fraunhofer HHI© Fraunhofer HHI

THz communication for future 6G networks

Left: laboratory setup of a THz receiver unit; center: model of a

– from optoelectronic components to field-ready systems

THz transmitter unit from the TERAWAY project

Applications like Industry 4.0, autonomous driving,
smart cities and augmented reality will require
extremely large bandwidths in the future. Thesebandwidths can be achieved with THz wirelesscommunication. As it uses frequencies above thosein 4G LTE and 5G mobile networks, this technology
can easily slot into existing communication networks.
It could potentially provide data rates a hundred
times greater than 5G.
Technologies for frequency ranges above 100 GHz for wirelessTHz communication are currently a focal point of research anddevelopment all over the world. They are considered a key
technology for fulfilling the ambitious performance targets offuture 6G networks. These networks are expected to facilitatea massive increase in wireless data rates, a significant reduction in latency and an improvement in reliability and network
coverage compared to today’s 5G networks.

Optoelectronic components for
THz communication
Fraunhofer HHI is conducting research across the field of THz
communication in various working groups. The Terahertz
Sensor Systems group within the Photonic Components
department is focusing on developing transmitter and receiver units for use in this special frequency range. The group is33

building on many years of experience in developing photonic
components for glass-fiber-based telecommunications. The
solutions developed in this area are now being further developed for use in THz communication.

THz transmitters based on
ultrasonic photodiodes
Ultrasonic photodiodes produced through Fraunhofer HHI’s
advanced InP process line are being used as optoelectronic
THz transmitters. Within these photodiodes, an optical beatsignal consisting of two slightly diverging laser lines are directly
converted into a THz beam (photomixing). This allows a single
THz transmitter to cover a very wide frequency range (0.1 THz
to 4.5 THz). In order to transmit the data, the amplitude/phaseof one of the laser lines is modulated to send the data signal at
the THz carrier frequency. With the coherent operating mode,
complex modulation formats such as quadrature amplitude
modulation (QAM) or quaternary phase-shift keying (QPSK)
can be used, and data rates of >100 Gb/s have already been
demonstrated.

dual-aperture THz antenna; right: hybrid integrated photonic

These are based on special ultrasonic semiconductor materials
that are manufactured at Fraunhofer HHI using molecularbeam epitaxy. These photomixers are also stimulated using an
optical beat signal that is mixed with the incoming THz beam.
If there is a difference between the frequency of the opticalbeat and that of the THz signal (heterodyne detection), the mix
results in an intermediate frequency, which is used to demodulate the data.

integrating THz links into the existing (fiber-optic) infrastructure. An important milestone towards early market introduction was reached as part of the TERRANOVA and Convergent
Networks (KoNET) projects in the form of the world’s first realtime data transmission through a stretch of combined fiber-optic and THz-wireless segments using a commercial 100 Gb/s
modem. In field tests at the campus of TU Berlin, error-freetransmission of a 100 Gb/s data signal was also achieved
across a stretch of 500 m.

Photonic integration of THz systems
Development of field-ready THz prototypes
Photonic integration provides significant potential for further
miniaturization of optoelectronic THz systems, because all of
the components required in this sort of system, such as lasers
and photodiodes, can be manufactured using Fraunhofer HHI’s
InP technology. Various EU projects, including Non-contact
Millimeter and Terahertz Frequency Measurement Paradigm for
Instrumentation and Sensing Applications Unlocking Metrology-grade Results (TERAmeasure) and TERAWAY are currently
working intensively on this challenge.

Wide-band THz communication

Working with the Fraunhofer Institute for Applied Solid State
Physics IAF in Freiburg, the Submarine and Core Systems groupis currently developing a field-ready prototype for point-topoint THz communication with a carrier frequency of 300 GHz
at data rates of up to 100 Gb/s across a stretch of open spaceof up to 1 km. This prototype, which has been presented atvarious exhibitions, is equipped with weatherproof housing
and a fiber-optic data interface. Initially, it is intended for use inurban fronthaul and backhaul networks to gather information
about long-term use in situations that closely replicate the realworld.

Photomixers as coherent THz receivers
Devices referred to as “photomixers” are being further developed as THz receivers for the purposes of data transmission.

The Submarine and Core Systems group within the Photonic
Networks and Systems department is looking into transmitting
very high data rates through the wireless THz channel and

34

Edge
Contributor
Node 2

Edge
Contributor
Node 1

Edge
Contributor
Node 3

Manufacturing
Site
Compute
Node
GlobalML ModelModel
Training© Fraunhofer HHI© Fraunhofer HHI

Local
ML ModelTraining Coordinator NodeAI for Photonics and Edge

Artificial intelligence (AI) is transforming almost allindustries and is considered a main technologicalenabler for 5G/6G communication networks. It is also
expected to change the status-quo and to revolutionize the functional blocks across the entire stackof optical transport networks, which span from ourrooms and offices (Fiber to the Room (FTTR)) to other
cities, countries, or continents (metro/core/submarine
networks). At Fraunhofer HHI, we develop AI-assisted
solutions for the broad ecosystem of optical networks
and innovate state-of-the-art algorithms to bring thevision of AI-assisted network automation to reality.
The realization of AI-assisted network automation
requires substantial research and developments inseveral key areas including: 1) network monitoringand programmability, 2) telemetry collection andstreaming, 3) reference dataset generation, 4) algorithm development, 5) experimental test and validation, as well as 6) explainability, certification anddeployment. These developments should consider
the foundational characteristics of the telco ecosystem, which has numerous stakeholders. On the one
hand, this requires the incorporation of a high levelof interoperability in the development process. On
the other hand, as AI-assisted network automation
relies heavily on data sharing, this adds an additional

35level of complexity due to data confidentiality andregulatory issues. These concerns motivate the incorporation of solutions that offer security, trust, andprotection of stakeholder privacy.
In the Photonic Networks & Systems department, the Data
Analytics and Signal Processing research group carries out
research in all of these key areas and beyond. We develop
innovative solutions across the entire machine learning (ML)pipeline and validate them using our large-scale test infrastructure. As one of our strategic areas, we perform research on
the application of privacy-preserving distributed AI, AI on theedge, and federated learning for telecom networks and otherdata-centric industries.

Carrier-grade machine learning pipeline
for real-time operation

Distributed intelligence forindustry automation

The realization of autonomous networks requires a reliable
and high performance ML pipeline. This enables streaming
fine-grained monitoring data across the whole infrastructure
to run ML-assisted network APPs. We developed a real-time
ML pipeline including a telemetry framework with data sovereignty features, a highly scalable data lake, and an analytic
module to run ML APPs. We contributed our implementation
to the Telemetry Work Item of the European Telecommunications Standards Institute (ETSI) Industry Specification Group(ISG) Fifth Generation Fixed Network (F5G) and performed asuccessful proof-of-concept demonstration for real-time trafficmonitoring and analysis.

Edge-Cloud is playing a significant role in the current transformation of industrial automation, where Operational Technologies (OT) and Information Technologies (IT) are converging. The
Fraunhofer Edge Cloud (FEC) offers a research platform to test
and validate industry 4.0 use cases. FEC offers a virtualization
layer based on OpenStack, GPU acceleration, and data sovereignty based on imaging development systems (IDS). We haveaccomplished several proof-of-concept demonstrations of the
FEC together with our partner German Edge Cloud. Moreover,
we have successfully demonstrated DLFi in a Kubernetes based
deployment on FEC to carry out federated learning sessions fortraining a YOLOv5 model for visual inspection.

Public datasets for network automation
The distributed learning framework
As a primary enabler for network automation, we develop andpublish reference datasets for a multitude of use cases (e.g.
quality-of-transmission estimation, predictive maintenance,
traffic forecast). We already published the first public dataset
collection for quality-of-transmission estimation in the community of optical networks. The dataset collection is currently
being extended with an experimental measurements focusedon multi-band networks covering S+C+L.

The emergence of a data economy and data-centric solutions
requires advanced techniques to protect the confidentiality,
privacy, and ownership of the data. We are developing a software solution called the Distributed Learning Framework (DLFi)
that offers state-of-the-art privacy-preserving algorithms forthe realization of AI-based products. DLFi supports different
use cases across multiple sectors including telecom, Industry
4.0, and medicine.

36Fraunhofer HHI departments

B

Administration

Wireless Communications
and Networks

Michael Klee

Quality Management
and Safety

B

Video Communication
and Applications

Dipl.-Ing. (FH) Mario Perkowski

Corporate
Communications
M. A. Martina Müller

Executive Directors

Prof. Dr.-Ing. Slawomir Stanczak
Dr.-Ing. Thomas Haustein

Division Video
Dr. Ing. Ralf Schäfer

Administration

Dipl.-Betriebsw., MBA

Dr.-Ing. Detlev Marpe
Dr.-Ing. Thomas Schierl

B

Vision and Imaging
Technologies
Prof. Dr.-Ing. Peter Eisert
Dr.-Ing. Anna Hilsmann

Thomas Wiegand
Executive Director

Artificial Intelligence
Prof. Dr. rer. nat.
Wojciech Samek

Martin Schell

Photonic
Components

Executive Director

Prof. Dr. rer. nat.
Martin Schell

37

Dr.-Ing.
Thomas Haustein

Dr.-Ing.
Martin Kasparick

Dr. phil. nat.
Renato CavalcanteMultimedia
Communications

Video Coding
Technologies

Embedded
Systems

Video Coding
Systems

Dr.-Ing.
Cornelius Hellge

Dr. rer. nat.
Jonathan Pfaff

Prof. Dr.-Ing.
Benno Stabernack

Dipl.-Ing.
Benjamin Bross

Dr.-Ing.
Thomas Schierl

Prof. Dr.-Ing.
Heiko Schwarz

Computer Vision
and Graphics

Immersive Media& Communication

Capture andDisplay Systems

Dr.-Ing.
Anna Hilsmann

Dipl.-Ing.

Dipl.-Ing.
Christian Weißig

Explainable AI

Detectors andModulators

Ingo Feldmann

B

AppliedMachine Learning

Efficient
Deep Learning

Dr. rer. nat.
Jackie Ma

Dr.-Ing.
Karsten Müller

LasersPhotonic
InP Foundry

Dr. rer. nat.
Martin Möhrle

Smart WirelessConnectivity
Dr.-Ing.
Zoran Utkovski

B – Berlin
G – Goslar

Intelligent
Network
Architecture

Prototype
Development andSystems Integration

Dr.-Ing.
Julius Schulz-Zander

Dipl. Ing. (FH)
Andreas Forck

Epitaxy

Processing

Backend

Packaging

Dr. rer. nat.
Christoph Hums

Dr. rer. nat.
Kristijan Posilovic

M. Sc.
Maximilian Herzog

B. Sc.
Felix Ganzer

mm-Wave
Dr.-Ing.
Michael Peter

Dipl.-Ing.
Adam Wieckowski

Dipl.-Phys.
Klemens Janiak

Interactive
and Cognitive
Systems
Dr.-Ing.
Sebastian Bosse

Hybrid PICs
Dipl.-Ing.
Norbert Keil

Battery Sensor
Technology

Surface
Processing

Dr. rer. nat.
Antonio
Nedjalkov

apl. Prof. Dr.
Eike G. Hübner

Submarine andCore Systems

Metro, Access andIn-house Systems

Free-Space
Optical Systems

Digital Signal
Processing

Dr. rer. nat.
Colja Schubert

Prof. Dr. rer. nat. habil.
Volker Jungnickel

Dr.-Ing.
Nicolas Perlot

Dr.-Ing.
Johannes Fischer

FiberLabDr. rer. nat.
MartinAngelmahrProf. Dr. rer. nat.
Wolfgang SchadeProf. Dr.-Ing.
Ronald Freund

Dr.-Ing.
Lars Thiele

Dr.-Ing.
Patrick RungeG

Organizational plan

Optimizationand Learning forCommunications

Dr. rer. nat.
Sebastian Lapuschkin

Fiber OpticalSensor Systems

Photonic Networks
and Systems

Signal andInformation
Processing

Prof. Dr. rer. nat.
Wojciech Samek

B

Prof. Dr. rer. nat.

Technologies
and Standards

PD Dr.-Ing.
Oliver Schreer

B

Prof. Dr.-Ing.

System Level
Innovations

Terahertz Sensors
and Systems
Dr. rer. nat.
Robert Kohlhaas

Equipment
Operations
Dipl. Ing.
Clemens Kleimann

DIN EN ISO 9001-2015 certified
Received: 2022

39

Photonic Networks and Systems

Photonic Networks
and Systems

At a glance© istockphoto/brave rabbit

The Photonic Networks and Systems department develops solutions for high-performance optical transmission systems to be
used for in-house, access, metropolitan, core and satellite communication networks. The department’s researchers focus on
improving security and energy efficiency as well as increasingtransmission capacity. The department uses the latest measuring
technology, well-equipped system laboratories and high-performance simulation tools. It is also able to perform field tests andproof-of-concept studies in realistic environments.

40

Contact details
Prof. Dr.-Ing. Ronald Freundronald.freund@hhi.fraunhofer.de41© Fraunhofer HHI© Fraunhofer IAF© Fraunhofer HHI

Photonic Networks
and Systems

Left: laser communication terminal with
optical antenna for free-space transmission;
middle: THz outdoor antenna for 100 Gbit/s
data transmission; right: LiFi system for operating multiple access points in order to coverlarger areas

Elastic optical networks with scalable capacity
and low-level latency
Network operators increasingly need to invest in their networks’ flexibility and scalability. To date, overdimensioned
transmission capacity in networks has been allocated in analmost rigid manner. Breaking away from this type of network
dimensioning in favor of needs-based allocation of bandwidth resources in the optical layer enables existing fiber-optic
infrastructure to be used more efficiently by making use ofspare capacity. The focus of the research is optical transmission
systems that can be configured with software, have flexible
data rates and are able to make each channel’s transmission capacity available with a high level of granularity in the
terabit/s range. These systems can be used in both wide-areaand metropolitan networks. Another challenge is finding anefficient way to integrate distributed computing and storage
units − known as edge clouds − so that future 6G services canbe operated with low latency. Fraunhofer HHI develops algorithms for planning and efficiently managing optical networks,
which it validates in experiments.

42

Terabit/s satellite communication and secure
quantum key distribution
Due to rising demand for bandwidth, future telecommunications satellites in geostationary orbit will be designed forterabit/s throughput rates. In light of this, Fraunhofer HHI isdeveloping optical free-space solutions that meet the strict
requirements for antenna gain (beam divergence in the
10 μrad range). The required data rates in the terabit/s range
are achieved with the help of wavelength-multiplexing technology in the atmosphere’s 1,550 nm transmission window.
Designed on behalf of the European Space Agency, theprototypes facilitate bidirectional transmission between groundstations and satellites, even in the event of atmospheric turbulence. The open-space transmission systems that are developed
are also used in the terrestrial field and for testing secure
quantum key distribution systems. Fraunhofer HHI develops
and tests systems based on both open space and fiber optics
for quantum key exchange.

Wireless terabit/s transmission withhigh carrier frequencies
In addition to wireless transmission technology that alreadyexists or is currently being researched, interest in something
known as terahertz (THz) transmission has grown over recent
years. THz transmission uses high carrier frequencies in the low
THz range (0.1–2.0 THz), enabling data to be transmitted wirelessly over short to medium distances (1–5 km). Until recently,
this transmission capacity had only been achieved by optical-fiber transmission systems. A field-ready prototype with a carrier
frequency of 0.3 THz has already been successfully demonstrated, with data rates of up to 100 Gbit/s across a point-topoint free-space distance of 1 km. This development will makeit possible to extend or bridge fiber-optic connections wirelessly. Other fields of application include backhaul and fronthaul
scenarios in next-generation cellular networks.

Data transmission using visible light
Optical wireless data communication (also known as LiFi) is
an attractive solution for fields that have special requirements
when it comes to security and electromagnetic compatibility.
Fraunhofer HHI has developed a form of transmission technology that enables conventional light-emitting diodes (LED) used
for room lighting to transmit data wirelessly. This technology
is already managing to achieve data rates in the Gbit/s range
per wavelength. Current studies are examining the use of LiFi
technology in industrial and medical environments. Researchers are also developing advanced communication protocols to
expand network functionality in order to accommodate thehigh standards these applications have in the areas of reliability, latency and jitter. They are also working to advance theirstandardization.

43

Photonic Components

Photonic
Components

At a glanceApproximately every three years, the amount of data sharedover the internet doubles. Through its research and development
work, the Photonic Components department helps the internet keep working despite this growth. Nowadays, around every
second bit on the internet comes into contact with technology
from Fraunhofer HHI.

© istockphoto/nicolas_In addition to optoelectronic semiconductor components used
for data transmission, the department develops integrated opticalcircuits. The team of Berlin-based scientists also studies related
fields, such as sensors, terahertz spectroscopy and high-performance semiconductor lasers for industrial use.

44

Contact details
Prof. Dr. rer. nat. Martin Schellmartin.schell@hhi.fraunhofer.de45© Fraunhofer HHI© Fraunhofer HHI© Fraunhofer HHI

Photonic
Components

Left: self-calibrated laser source (with integrated wavelength meter); middle: 100 GHz
balanced photodetector module; right:
PIConnect mainboard (with PIC assembly)

100 Gbaud InP-based transmitters and receivers

Terahertz sensors and communication

Quantum transmitters and receivers

Photonic integrated circuits

Approximately every three years, the volume of data on theinternet doubles − demanding increasingly faster optical transmitters and receivers with wavelengths between 1,260 nm and1,625 nm. Fraunhofer HHI is developing InP-based single-modelasers with built-in absorption modulators for short ranges
and Mach-Zehnder modulators for long ranges, along withthe accompanying photodetectors. One InP wafer holds 1,000
modulators or 30,000 lasers, making InP technology extremely
efficient.

A large number of economically important materials are transparent in the terahertz frequency range between 100 GHz and
10 THz. In contrast to ultrasonic technology, terahertz-based
non-destructive testing technology allows for contact-free
analysis. This is relevant in areas such as the automotive
industry; for example, in monitoring the thickness of individual
layers in a multi-layer paint system and their drying process.
The technology can also be used to analyze reduced-weight,
foam-like substances with a high air content.

Data security is playing an increasingly important role in oursociety. Quantum communication − in other words, hardware-secured quantum key distribution − enables physicalsecurity during data exchange. Here, we use existing fiber-optic infrastructure and replace only the optical transmitters andreceivers with equivalent quantum components.

Fraunhofer HHI offers quick and cost-effective InP-based production services for photonic integrated circuits (PICs). As such,photonics is following in the footsteps of the CMOS successmodel established in the 1960s. Multi-project wafers (MPW)
with client-designed PICs are launched on a quarterly basis.
PIConnect − a user-friendly connection to drive PICs − makes
the characterization of the PICs easier.

In addition to high data rates, the primary aim of this researchis to further reduce the amount of energy consumed per bit
transmitted. Symbol rates of 100 GBaud have already beenachieved in uncooled InP components, and researchers are
currently working on reaching 200 GBaud. At speeds this high,co-design of InP transmission and receiver chips with theirdrivers or pre-amplifiers is essential.

Fraunhofer HHI develops terahertz transmitters, receivers andspectrometers, and markets them with our industrial partners.
According to our own estimates, at least 50% of the world’s
terahertz spectrometers are based on Fraunhofer HHI technology. Another field of application is wireless communication at
carrier frequencies above 100 GHz, along with the testing andmeasurement technology for these systems. Carrier frequencies on this scale achieve significantly higher data rates. Good
compatibility with existing fiber-based telecommunications
technology is a major advantage of the optoelectronic solutions developed at Fraunhofer HHI.

46

Fraunhofer HHI is developing quantum transmitters and receivers using established InP technology as advanced versions ofcomponents used for classic communication. For instance,
Fraunhofer HHI is developing very low-noise coherent receivers and InGaAs single-photon avalanche diodes (SPAD) in thetelecommunications wavelength range. At present, Fraunhofer
HHI is Europe’s only producer of SPAD chips.
In a second phase, these components and others will be
integrated with the polymer platform to form hybrid solutions.
Besides tunable, narrowband lasers in all wavelength ranges,
this will enable waveguide integration of nonlinear crystals,
such as LiNbO3 and KTP entangled proton pair generation −a key piece of basic technology in quantum sensing andquantum computing.

Sensors or quantum technologies often require wavelengths
below 1,200 nm, making them inaccessible for InP. Fraunhofer
HHI offers a hybrid photonic integration platform PolyBoardfor this purpose. PolyBoard is transparent from near infrared
to visible and − thanks to its micro-optical bench − facilitates
the integration of isolators and circulators as well as nonlinear
optical crystals. By combining its technology with other material platforms like SiN and LNOI, Fraunhofer HHI is able to coveralmost all application fields, from medical engineering andanalytics through to innovative THz-based transceiver demonstrators for wireless networks on both sides of 6G.

47

Fiber Optical Sensor Systems

Fiber OpticalSensor Systems

At a glance© fotolia.com/xiaoliangge

Our society’s transition to electrical power as its primary sourceof energy is leading to a steady rise in demand. To meet future
demands − whether it is needed for electric mobility, green hydrogen, or heating and cooling buildings − continuous improvements
to the efficiency of the technology are needed. Alongside thedevelopment of new materials and material properties, sensor
technology plays an important role. The Fiber Optic Sensor Systems department is dedicated to this topic. Femtosecond laser
technology is used to develop integrated optical components andsystems on a glass and polymer basis, to provide novel surfacefunctionalizations for sensors, and to process highly efficient electrode materials for hydrogen generation and storage.

48

Contact details
Prof. Dr. rer. nat. Wolfgang Schadewolfgang.schade@hhi.fraunhofer.de49

Sensors and functionalized materials for theenergy transition
The interaction between femtosecond laser pulses and different materials opens up a wide field of options for adjusting
specific material properties both at a surface and interior. Femtosecond laser direct-writing methods, based on multi-photon
absorption processes in materials, are applied in fiber optics.
These ultra-thin glass foils and polymers can produce basic
elements such as optical waveguides and photonic diffractive
components. For example, this integrated optics approachhas been applied as a way of creating integrated optical
microspectrometers. When combined with fiber-optical andplanar-optical 3D shape detection, this creates novel approaches to multi-component sensors. The size of the active surfaces,
wettability as well as heat emission and catalytic properties canalso be adjusted by processing metal surfaces with lasers.

3D shape detection and optical seals
In collaboration with the University of Basel, the performanceof the fiber-optic 3D shape sensor developed at Fraunhofer
HHI has been significantly improved with the help of artificial

50© Fraunhofer HHI© Fraunhofer HHI© aapsky/i.stock.com edit by Fraunhofer HHI

Fiber OpticalSensor Systems

Left: femtosecond laser used to processmaterials for temperature regulation in spacetravel; middle: ultrathin glass for 3D shapedetection; right: optical seal

intelligence, enabling the reconstruction of the three-dimensional shape of a 30 cm long optical fiber with an accuracy of
about 2 mm compared to the reference shape. This opens up
new opportunities for controlling technical and medical endoscopes. The Inherently Flexible X-Ray Imaging Detector using
Single Photon Avalanche Photodiodes and Scintillating Fibres(FleX-RAY) project (Horizon 2020) is focusing on developing
new, flexible X-ray imaging detectors. To achieve this, Fraunhofer HHI is developing a 3D shape sensor based on functionalized, ultra-thin glass foils used to determine the detector
geometry for subsequent imaging.

Fiber-optic hydrogen sensor technology

In the Novel Active Optical Sealing for PRS Security Modules
with Thin Glass Sensors (OTP) project co-funded by the Federal
Ministry for Transport and Digital Infrastructure, the department is working with OHB Systems AG to develop a sensory
security housing for permanently monitoring physical tamper
attacks on mobile public regulated service security communication modules, which are used in the European satellite
navigation system GALILEO. To achieve this, the vulnerable
electronic components are surrounded by an optical sensory
housing made out of ultra-thin glass. This glass housing incorporates an unclonable, unitary optical signature to allow for
cryptographically secure initialization.

Material processing using ultrashort pulse laser
technology

The H2Security project, supported by ZIM (Central Innovation
Program for SMEs), is developing fiber-optic sensors with builtin Bragg gratings to selectively detect hydrogen. The physicalmeasurement principle is based on evanescent field spectroscopy. To increase their sensitivity and selectivity, the fibers arecoated with Pd nanoparticles at the measurement points. An
accuracy of rate 0.05 % is achieved in a measurement range of
0.1 % to 4 % hydrogen.

Ultrashort pulse laser technology is used to functionalize thesurfaces of a wide range of materials, enabling their properties
to be adjusted for specific purposes. The light absorption, wettability, thermal emissivity or even the size of the active surface
can be changed. Material surfaces that have been optimizedin this way are ideal for use as highly efficient electrodes forwater electrolysis with minimized overvoltage, as heat sinks
for dissipating heat in space, or as catalyst carriers for chemical
reactions. This work is integrated into the Fraunhofer Clusterof Excellence − Advanced Photon Sources and the European
Space Agency’s (ESA) Euro Material Ageing campaign.

Battery and sensor testing center
Fraunhofer HHI's battery and sensor testing center was
expanded even further in 2021, allowing it to perform thermaland electrical tests on battery systems with an energy content
of up to 150 kWh under defined conditions. In addition, newand innovative concepts for advanced battery safety are developed and reviewed. Certifications are prepared in collaboration
with the VDE. The center also tests transport containers, newextinguishing agents and extinguishing systems. Alongside
two fire test furnaces, it is equipped with one climate container and several battery facilities with electrical power sources up
to 1.2 MW. This enables battery tests in the operating range
and above, which includes the complete and safe burnup of
the battery with gas analysis.

51

Video Communication and Applications

Video Communication
and Applications

At a glance
The Video Communication and Applications department
researches areas of image and video coding, the transportationof multimedia data, embedded systems for multimedia
processing, communication protocols for cellular networks,
and the efficient representation and coding of neural networks.

Contact details
Dr.-Ing. Detlev Marpe© istockphoto.com/piranka

detlev.marpe@hhi.fraunhofer.de

52

Dr.-Ing. Thomas Schierlthomas.schierl@hhi.fraunhofer.de
Dr.-Ing. Ralf Schäfer
ralf.schaefer@hhi.fraunhofer.de

53

Nowadays, video data account for most bits in global
internet and cellular data traffic. To help overcome
the challenges it presents, the Video Communication
and Applications (VCA) department is constantly
working to develop new video compression technology. In conjunction with industry leaders, VCA incorporates its compression technology into international
standards, which facilitate interoperability when
exchanging and storing multimedia content and areapplied to billions of devices around the globe. With
the emergence of neural networks in almost every
application, the focus of compression is expanding toinclude the use of neural networks in video compression and the compression of neural networks
themselves.
The VCA department deals with all aspects relevantto the research areas of image and video coding,the transportation of multimedia data, embedded
systems for multimedia processing, communication
protocols for cellular networks, and the efficient representation and coding of neural networks.

54

Contributions to the standardizationof video codecs
The department has made a significant contribution to developing three generations of video coding standards − including
the main updates they have undergone and the relevant transport and storage formats: H.264 / Advanced Video Coding(AVC), H.265 / High Efficiency Video Coding (HEVC) and H.266/ Versatile Video Coding (VVC). Billions of devices around theworld use the AVC and HEVC video compression standards.
Half of all bits on the Internet are generated and processed bycompression technology developed by VCA.
VCA played a major role in developing the recent H.266/VVC
standard, which reduces the bit rate by 50% compared to thecurrent state of the art (the HEVC standard) while still maintaining the same image quality for viewers. The first version of
the VVC standard was finalized in July 2020 and − in addition
to advanced versions of already familiar technology − also
contains radically new compression methods developed using
techniques such as machine learning. Furthermore, the firstversion of VVC is already equipped with special coding tools
and system functions for a wide range of applications, such as
adaptive streaming with varying resolutions, 360° immersive© Fraunhofer HHI© Shutterstock.cm/ouh_desire edit by Fraunhofer HHI© Fraunhofer HHI

Video Communication
and Applications

Top middle: playback of VVC-coded video on atablet; top right: scene from a cloud rendering
for volumetric video

video, streaming with extremely low latency, game applications and screen content sharing.

Contributions to the implementationof video codecs
The open-source Fraunhofer Versatile Video Encoder (VVenC)
software developed by the VCA department was awarded
first place in the Best Open Source Project Awards during theIEEE International Conference on Multimedia and Expo (ICME)
2021. VVenC is a quick and efficient encoder implementationof the H.266/VVC standard.
VVenC makes it possible to use the significant reduction in thedata rate, as offered by the new VVC standard, in countless
multimedia applications within today’s connected society.
Examples include streaming, virtual reality and augmented
reality. With this solution, the software can also be flexibly
configured according to the various requirements for quality
and encoder runtime.

Contributions to the standardizationof multimedia systems
After two years of standardization work, the Moving Picture
Experts Group (MPEG) completed its work on scene description through an enhanced version of the Graphics Language
Transmission Format (glTF). The glTF is an industry standard
developed by the Khronos Group for describing three-dimensional models and scenes. The new update enables volumetricvideo and audio to be integrated into an immersive scene forthe first time. Fraunhofer HHI played an important role in the
research and development of the new standard.
MPEG’s upgrades to gITF include mesh linking technology,
which was developed by Fraunhofer HHI. This enables the highlevel of realism in volumetric videos to be combined with animation capabilities in computer graphics models. Photorealistic
virtual humans can now be integrated into mixed-reality scenes
to facilitate interaction with users. For example, a photorealistic avatar can now actively maintain eye contact with a user,
something that was previously only possible with computer
graphics models − not with much more realistic-looking but
also more complex volumetric video content.

55

Vision and Imaging Technologies

Vision and Imaging
Technologies

At a glance
The Vision and Imaging Technology department researches
cutting-edge technology for the entire visual data processing
chain, from the capturing and analysis of visual content and digital representation modeling through to realistic visualization andhuman-machine interaction. Its primary focus is on sophisticated
AI technology in the fields of computer vision, visual computing,
3D modeling and simulation, cognitive systems, and immersive
media. With our technology, we offer solutions for a wide range
of applications in industry, medical engineering, agriculture,
construction, safety and security as well as for virtual, augmented
and extended reality.

© istockphoto.com/Petrovich9

Contact details

56

Prof. Dr.-Ing. Peter Eisert
peter.eisert@hhi.fraunhofer.de
Dr.-Ing. Anna Hilsmannanna.hilsmann@hhi.fraunhofer.de

57

The past few years have seen a huge rise in the rolesof understanding and interpreting the visual world bycomputers, digital representations of the real world inauthentic digital models, and interaction with digitalcontent in immersive environments. The technologies
developed by researchers at Fraunhofer HHI detect,
track and analyze objects and people, and their movements and textures, and then reconstruct realistic digital
representations. Modern processes used to model andanimate digital models enable visual content to besynthesized for media applications, for simulations ofindustrial processes or for training of highly complex
AI solutions. Our researchers also develop technologies
that enable humans and machines to work together
effectively thanks to new modes of interaction, intelligent environments and visualization techniques.

58© KT Stock photos/Shutterstock.com© Zapp2Phot/Shutterstock.com© PopTika/Shutterstock.com

Vision and Imaging
Technologies

Left: intraoperative assistance systems;
middle: AR support in production and assembly; right: detection of attacks on authentication systems

Technology for industry, construction, agriculture
and medicine
State-of-the-art computer vision and AI solutions can help tomake workflows in industry, construction and agriculture more
efficient and secure. Elements such as virtual representationsof real-life objects (digital twins) are essential parts of modern
automation and engineering processes. The technologies
range from solutions for visual inspections using cutting-edge
AI solutions to image-based methods for high-precision,
reliable dimensioning, process monitoring, analysis and 3D
scanning of complex components, buildings and spaces, all
the way through to augmented reality assistance and humanmachine interaction to support workflows in production,
construction and agriculture. Even workflows in the medical
sector can be supported by cutting-edge computer vision
and AR-based technology, such as intraoperative assistancesystems. For instance, real time-capable 3D measurement
using endoscopes or operating microscopes allow implants tobe customized. Work on hyperspectral-based tissue analysis
and video-based monitoring of perfusion and vital parameters
allows additional information to be made available intraoperatively in real time. Contact-free human-machine interaction
and user-adaptive data and image visualization contribute to

the intuitive design, while still ensuring compliance withclinical hygiene standards. These methods make medical
interventions safer, simplify workflows and improve medical
working environments for the long term.

Technology for increased immersivity in XR
New media and communication formats − such as extended, virtual and augmented reality (XR, VR and AR) − call for
innovative solutions. Here, the challenge lies in making virtual
content and environments as realistic as possible in order toachieve the highest possible level of immersivity. The realistic
modeling of virtual humans who can interact with users is particularly essential for countless immersive applications. Using
volumetric video technology as a basis, Fraunhofer HHI has
developed innovative hybrid representations of virtual humans
with the help of modern AI methods. As a result, a person’s
individual movement patterns and facial expressions can belearned in relation to speech, for example, and new animations
can be accurately synthesized. Modern neural visualization
methods help to ensure a realistic appearance. Our researchers work closely with neuroscientists to research how virtualhumans are perceived. Furthermore, the department develops

new media formats for immersive audiovisual representations
in a range of applications. Its development work here focuses on immersive video and audio technology, recording andplayback processes, and low-latency transmission processes.
The goal is to achieve the highest possible level of immersion,
be it for virtual participation in life events, virtual training (e.g.,
for medical staff) or the simulation of infrastructure measuresin urban development.

Secure authentication with AI
The Vision and Imaging Technologies department develops
reliable algorithms for detecting attacks on image-based andvideo-based authentication systems using fake identities. In
particular, we study reliable AI processes for detecting what
are known as morphing attacks (manipulated reference
images), presentation attacks (the simulation of fake identities
through the presentation of images, screens or models) anddeep fakes (synthetic images or videos that use AI methods).
We have focused particularly on the reliability and interpretability of AI methods for security-related applications.

59Wireless Communications and Networks

Wireless Communications
and Networks

At a glance© iStockphoto.com/baranozdemir

The Wireless Communications and Networks department
researches the area of wireless data exchange. Working closelywith a number of companies and organizations, our researchers
contribute to theory, concept development, technical feasibility
studies and standardization in the area of wireless systems.
Scientific studies, simulations and assessments at link andsystem level, field measurements, and the development of
hardware prototypes complete the department’s rangeof services.

60

Contact details
Prof. Dr.-Ing. habil. Slawomir Stanczak
slawomir.stanczak@hhi.fraunhofer.de
Dr.-Ing. Thomas Hausteinthomas.haustein@hhi.fraunhofer.de61Research and standardization for 5G and 6G
The roll-out of fifth-generation (5G) public cellular networks
is well underway all over the world. As this is taking place,research and standardization for 5G are still progressing. Additional functionalities and protocols are being developed andstandardized by the 3rd Generation Partnership Project (3GPP)with the aim of supporting applications for vertical industries.
Within this process, the focus is shifting increasingly toward
local and customer-specific 5G cellular networks — known as
campus networks.
The creation and operation of campus networks, including
the various operator models, have yet to be fully researched.
Particular industry-specific requirements for vertical applications have to be taken into account in this context, along withthe basic conditions they need, and this presents some major
challenges. The goal is to develop bespoke solutions that areappropriate for the specific needs and basic conditions ofcampus networks. This will generate unique opportunities forGerman start-ups, the SME sector and innovations within the
5G environment. Using Open radio access network (RAN) as abasis, the Wireless Communications and Networks department
develops 5G solutions for open campus networks, helping tocreating new value chains within these networks.

62

In addition to its work on open 5G campus networks, thedepartment has started researching the sixth generation ofcellular networks (6G). The main challenges for researchers lie
in the radical rise in data rates that apply in planned 6G applications (such as mixed realities). There are also new challenges
resulting from the integration of sensor services (networkas-a-sensor solutions). Artificial intelligence will be one of
the most important sources of innovation for 6G. The aim in
applying it will be to promote the convergence of informationand communication technology. Ultimately, 6G will integrate
non-terrestrial communication and networks (satellite anddrone communication, and so on) in an effort to achieve an
extremely high degree of network coverage around the world(in rural and sparsely populated areas, for example).

Ongoing vital data transmission and processing
across 5G campus networks
Patients in the final stages of chronic heart failure often require
permanent mechanical circulatory support (in the form of left
ventricular assist devices or LVAD) and/or a heart transplant.
However, their underlying cardiological disease, the possible
complications arising from the implant and comorbidities© Falco Seliger edit by Fraunhofer HHI© Fraunhofer HHI© Fraunhofer HHI

Wireless Communications
and Networks

Left: research on virtualized 5G and 6G wireless access networks; middle: ongoing vital
data transmission and processing across 5G
campus networks; right: reliable drone communication outside the line of sight (Wingcopter during vertical take-off in Malawi)present major challenges to their post-operative long-termcare. The 5GMedCamp project’s goal is to improve telehealthcare for LVAD patients using 5G technology and methods
involving artificial intelligence, creating a technological basis
for close real-time monitoring. The aim is for the results on
the use of 5G campus networks in the medical sector to form
the foundation for enabling multi-dimensional technologicalopportunities involving connectivity, digitalization and AI to be
used in patient telehealthcare.

Reliable drone communication outside
the line of sightIn the future, drones will increasingly be used to operate outside of a controller’s line of sight. Due to their limited range,
conventional remote control systems are not suitable for thistype of application. If cellular networks are experiencing highloads or there is a lack of network coverage, simple cellular
systems are unable to guarantee reliable communication. In
order to meet the high safety and security requirements of
the aviation industry, it is essential that intervention in flightplans is possible at all times. This means that safety-critical andsecurity-critical information must be transmissible without any

interruptions. To achieve this, several types of technology are
combined to form a complete system with a high degree of
reliability. State-of-the-art 5G technology − which is likely to
be joined by 6G in future − involving multiple-input and multiple-output antenna systems (MIMO) provides the basis for this.

Research and development• Fifth-generation and sixth-generation
cellular networks (5G/6G)• Campus networks• Sub-THz communication for mobile access• Deep integrated machine learning• V2X communication• Drone communication• Network as a sensor• Intelligent reflective surfaces• PHY-layer security• Semantically supported communication• Software-defined radio (SDR)• Autonomous convergent networks

63Artificial Intelligence

Artificial
Intelligence

The AI department was named oneof the Top 20 Artificial Intelligence
Research Labs in the World in 2021¹
and has received awards such as thePattern Recognition Medal, several ESI
Highly Cited publications and the Hugo
Geiger Prize.

At a glance¹ Quelle: Analytics Insight, 2021, https://www.analyticsinsight.net/top-20-artificial-intelligence-research-labs-in-the-world-in-2021/

64© iStockphoto.com/Traitov

The Artificial Intelligence department studies the theoretical andalgorithmic foundations of machine learning, acting as a bridge
between fundamental research and practical applications. The
department is a global leader in the field of explainable artificial
intelligence (known as XAI for short). The team has also gainedinternational recognition for its contributions to distributed
machine learning, the compression of neural networks and thestandardization of AI.

Contact details
Prof. Dr. rer. nat. Wojciech Samek
wojciech.samek@hhi.fraunhofer.de65

XAI: the bridge between humans and AI

Viewing the AI lifecycle as a whole

Through its work on explainable AI, the department has created an innovative communications interface between AI andthe user. Layer-wise Relevance Propagation (LRP), a process
designed by the department in conjunction with the Technical
University of Berlin, enables users to track AI decisions at anin-depth level. LRP is one of the world’s most frequently usedmethods in the field of XAI. The method’s potential stretches
far beyond simply explaining decisions, however.

To enable AI systems to be used in safety-critical applications
such as autonomous driving or clinical diagnostic systems,
attributes such as security, robustness, explainability andreliability are of key importance. In order to safeguard these,
the entire AI lifecycle – from data acquisition and labeling to AI
model design and training, all the way through to systematic
validation and monitoring of the model – must be taken into
account. Through its work, the department contributes to allphases of the AI lifecycle and maintains a holistic overview ofthe development process. By participating in various national and international standardization initiatives in the field ofAI, such as the ITU/WHO focus group AI for Health or theGermany Institute for Standardization (DIN) roadmap, the teamis helping to establish novel testing and certification standards
for secure, robust, explainable and reliable AI.

In its work, the team has demonstrated how LRP can be used
to systematically debug and improve the AI model. Explanations also provide a wealth of opportunities for integratinga priori knowledge and carefully controlling aspects of themodel during training. Research into the potential of XAI goesmuch further than just explaining and the systematic improvement and specific adjustment of explanations to respondto practical issues are also areas of focus within the team’s
research. Various pieces of XAI technology created by Fraunhofer HHI are available to the research community as opensource software to help advance the field of explainable AI
with a leading, sustainable approach.

66© iStock.com/matejmo© iStock.com/salihkilic© iStock.com/xijian

Artificial Intelligence² Fusing Deep Learning andStatistics towards Understanding Structured Biomedical Data

AI research with a practical impact
The department develops AI methods and models that offer
practical benefits. Challenges such as limited and distributed training data, a high degree of multimodality or limited

processing resources do not impede the model training
process. Our techniques for explaining AI decisions, generating
synthetic training data or testing and debugging fully trainedmodels are used at central points in a range of cooperation
projects. These include projects in the medical sector for
analyzing epidemiological data to assess risk factors, taking
full-body images to identify abnormal changes to the skinor taking 3D CT images of the heart to identify calcification.
The technology is also used to evaluate more complex sensor
and weather data as a means of predicting air quality, and toanalyze mobility data when developing early-warning systems
for natural disasters.

Federated learning: distributed, private, efficient
Another area of focus in the department’s research is thedevelopment of improved methods for federated learning,which enables us to efficiently train AI models in distributed
data while protecting privacy. In this context, the team isparticularly interested in developing improved compression
methods for neural networks and federated learning, andin examining the interaction between learning, compression
and communication. The compression methods developed by

the AI and VCA departments lead to huge savings in neuralnetwork sizes and communication in federated learning.
They form the basis for the first international Neural Network
Coding Standard (ISO/IEC 15938-17). New approaches forindirect knowledge exchange (known as federated distillation),model personalization and verifiable privacy guarantees arealso studied by the department.

A member of the Berlin AI landscapeThe department’s team is closely integrated into Berlin’s AI
landscape in a range of ways — for example, as a member of
BIFOLD (Berlin Institute for the Foundations of Learning andData), as part of the DFG research group DeSBi² or through achair at the Technical University of Berlin. Thanks to its participation in the Konrad Zuse Schools of Excellence in AI (ELIZA)
and various graduate schools, the department is actively
involved in training talented AI specialists of the future.

67

About us

Showrooms
Politics. Science. Research.

We create a
network ofpeople fromacross the globe.
Dr.-Ing. Ralf Schäfer
Director of the Video Division, Fraunhofer HHI

Fraunhofer HHI’s showrooms are where the technologies developed at the instituteare displayed for people to experience − which in turn gives them an important rolein transferring technologies to industry. The showrooms also serve as communication
platforms for dialogue with decision-makers from the worlds of industry, politics andsociety.

CINIQ Center –Forum Digital
Technologies

Since 2019, the Forum Digital Technologies has been using the CINIQ
premises to showcase selected research projects and innovations in thefield of digital technologies. There, events and guided tours provide
visitors with an insight into the vital discourse that takes place on thesubject of digitalization, encouraging dialogue between groups ofexperts.
The Forum Digital Technologies receives financial support from theGerman Federal Ministry for Economic Affairs and Climate Action(BMWK).

TiME Lab –Tomorrow‘s
Immersive Media
Experience
Laboratory

3IT –Innovation Center
for Immersive
Imaging Technologies

68

Located in Berlin, the CINIQ Center is a location for technology
and information transfer that operates at the intersection of
innovation, science and industry.

The TiME Lab showroom was created as a research, collaboration andpresentation platform for immersive media. It is equipped with highresolution 180° video projection, plus a 3D sound system based on wave
fields. TiME Lab is home to numerous systems and fields of applicationthat have undergone systematic research and implementation over theyears. Since its foundation, it has carved out a position for itself as aplatform for realistic audiovisual presentations of city planning concepts
and measures for infrastructure building and noise protection.

The showroom encompasses a virtual network plus an exhibition spaceand events venue in the heart of Berlin. In the 3IT center, new technologies for industrial, medical and cultural applications are developed, tested
and presented to a range of target groups. Its focus is on 3D, VR/AR/XR,
volumetric video, UHD and HDR technologies.

69

About us© REDPIXEL.PL/Shutterstock.com

Collaborations andFraunhofer groups
Shaping the digital societyof the future – together

Researchers at
Fraunhofer HHI
collaborate with arange of experts fromacross the globe tocreate advancements
in technology and put
them into practice.
Fraunhofer HHI also
works with numerous
local partners with theaim of giving innovations greater visibility
and strengthening theBerlin/Brandenburg
region’s status as ascientific hub.

5G BERLIN e. V.
The 5G BERLIN cluster connects the traditional telecommunications industry with new 5G
stakeholders and the Berlin start-up scene.
It operates its own dedicated 5G test field in
the heart of Berlin and offers members in thevicinity the opportunity to test out new 5G
technologies under real-life conditions. As aninnovation cluster, its focus is on promoting
5G applications that use augmented reality (AR) and virtual reality (VR), and it also
encourages advancements in areas such as
intelligent mobility, intelligent supply networks, security applications and Industry 4.0.

Optec-Berlin-Brandenburg
e. V. (OpTecBB)
OpTecBB is an association of 115 companies
and scientific facilities that are active in thefields of optical technologies and microsystems engineering. Its focus is on photonics
and quantum technologies for communications, sensors, optical analytics, light technology, biophotonics, ophthalmic optics, laser
technology and microsystems engineering.
Prof. Martin Schell is chairman of the OptecBB
managing board.

70

European Photonics
Industry Consortium (EPIC)
EPIC is the leading industry association in
the European photonics sector, providing a
networking hub for more than 800 industrial
companies, research institutions, SMEs andstart-ups. As an advocacy group, it focuses on
the sustainable development of technologies
and on promoting a vibrant photonics ecosystem. Prof. Martin Schell was a member of the
EPIC’s Board of Directors from 2015 to 2021.

Photonics21
Technology platform Photonics21 provides
representation for the European photonics
industry and research organizations working
in the field. In collaboration with the European
Commission, it has developed a joint researchstrategy as part of a Horizon Europe public-private partnership (PPP) with the aim ofstimulating growth and employment opportunities in Europe. Since 2014, Prof. Martin
Schell has been HHI’s representative on theplatform’s Board of Stakeholders.

Berlin Institute for theFoundations of Learningand Data (BIFOLD)BIFOLD promotes fundamental research in big data
management and machine learning, plus the areasin which they intersect. Alongside its partners, theinstitute is focused on developing Berlin’s status asa standout location for AI research. To achieve this,
BIFOLD provides regular opportunities for discussion
between AI researchers from various institutions
and sets up several programs designed to support
up-and-coming talent.

Video Coding Experts
Group & Moving Picture
Experts Group (MPEG)
Fraunhofer HHI has made a significant contribution
to developing three generations of video coding
standards, the main updates they have undergone
and their associated transport and storage formats:
H.264/Advanced Video Coding (AVC), H.265/
High Efficiency Video Coding (HEVC) and H.266/
Versatile Video Coding (VVC). Fraunhofer HHI
technologies also play a key role in the standard
governing the compression of neural networks for
multimedia applications (the NNR standard) and thelatest upgrade to the Graphics Language Transmission Format (glTF).

Member of
Fraunhofer groups:• Microelectronics• ICT Group (associated institute)• Light & Surfaces (associated institute)Member of
Fraunhofer alliances:
•
•
•
•
•
•
•
•

Ambient Assisted Living AAL
Cultural Heritage ResearchBattery
Big Data and Artificial Intelligence
Digital Media
Aviation & Space
VisionDigital Networking
High-Performance Center

71

About us© Fraunhofer IAF, edited by Fraunhofer HHI

Research FabMicroelectronics Germany
Key partner to industry, science,
politics and society

Since 2017, Fraunhofer HHI has beenworking with ten other institutes in the
Fraunhofer Group for Microelectronics,
plus the FBH and IHP institutes within the
Leibniz Association, to form the cross-location Research Fab Microelectronics
Germany (or FMD for short).

2,000

researchers
in the FMD

For the first time, this collaboration isproviding an opportunity for 13 research
units and institutions from Fraunhofer
and Leibniz to pool their expertise undera shared virtual banner. In doing so, they
are reaching new heights in the standards
achieved in researching and developing
microsystems and nanosystems. Home tomore than 2,000 researchers, the FMD is
the world’s largest research and development (R&D) collaboration of its kind. The
unique diversity of specialities and infrastructure that the FMD delivers throughits institutes is helping to ensure that Germany and Europe reinforce their leading
position in research and development.
Transitioning to normal operation
At the end of 2020, the FMD was still in its
development phase. The extensive investments
in modernizing the institutes, provided by theGerman Federal Ministry of Education andResearch (BMBF), came to the end of their terms
in late 2020/early 2021 with the exception of afew delays caused by the COVID-19 pandemic.

72

At the start of 2021, the FMD began amalgamating the central offices of the FraunhoferGroup for Microelectronics and the Research
Fab Microelectronics Germany on a permanent basis, with Dr. Stephan Guttowski taking
on the role of managing director of the new
joint central office. An online conference titled“Impulsgeber FMD: Angebot & Potenzial −Köpfe & Know-how” (FMD catalysts: opportunities and potential − experts and expertise)
was held on April 22, 2021 to mark the newarrangement. The model, which representsinterdisciplinary and interorganizational
collaboration within the German researchlandscape, is already beginning to bear its first
fruits and it is hoped that it will set a precedent within Europe in the future.
Networking and collaboration fortechnological sovereignty
Since its launch, the FMD has established itself
as a prime example of a modern approachthat assembles expertise from different R&D
institutions alongside a shared strategy and a
combined range of services on offer to industry players. Through collaborative work thatcrosses the boundaries of locations, technologies and areas of expertise, the FMD is able tomaintain and develop technological sovereignty along the entire value chain.
The team at the Berlin central office represents
the FMD institutes and provides a central pointof contact for handling any questions about
microelectronics and nanoelectronics researchand development in Germany and Europe.
A range of collaboration opportunities
In addition to the services it offers its industry
customers, the FMD provides a range of collaboration opportunities for its partners working in science and education. The aim of theseis to approach research-related issues from ashared standpoint – by working together in
research projects and operating joint labs, forexample. A highlight of these collaboration
opportunities is the ability to trial concepts
and solutions stemming from fundamental
research at the facilities provided by the FMD
institutes, making it possible to gain a better
understanding of how well suited the concepts and solutions are to environments of amore applied nature.
Trustworthy, sustainable microelectronicsystems for enhanced innovative strength
A society that is built for the future will be reliant on electronic components in every important application domain, whether that involves
critical infrastructures, Industry 4.0, theautomotive sector or medical devices. People
need to be able to rely on these components
in order to use them for building trustworthy
products, systems and infrastructures.

Working in large-scale projects such as
TRAICT (TrustedResourceAware ICT) and
Velektronik, the institutes in the FMD develop the cross-technology areas of expertisethat are needed to rise to this challenge.
The TRAICT project, for example, saw eight
FMD institutes working with a further ten
Fraunhofer institutes until the end of 2021 on
developing basic conditions for ensuring that
information and communication technology
is reliable and compliant with data protection
requirements, and at the same time is able tobe used autonomously and securely.
Velektronik, a platform for trustworthy electronics, was launched in March 2021 as a wayof providing a clearer view of the value chain
as a whole and creating consistent concepts
for trustworthy electronics in Germany andEurope. The platform involves 12 partners intotal: 11 FMD institutes plus the edacentrum.
The aim is for the project to provide a venue
for developing and putting into practice standards and processes on the basis of a national
and European chip security architecture.

Central office

Research FabMicroelectronics Germany
c/o Fraunhofer Group for
Microelectronics
Anna-Louisa-Karsch-Str. 210178 Berlin73About us

ITU Focus Groups at Fraunhofer HHI

Through its collaborative work, Fraunhofer HHI
makes a major contribution to developing andstandardizing artificial intelligence (AI) in the fieldsof medicine, disaster risk reduction and agriculture.

ITU Focus Groups
at Fraunhofer HHI

Fraunhofer HHI coordinates three focus
groups within the International Telecommunication Union (ITU). Within thesegroups, the ITU brings together international experts with the aim of developing
strategies for using new communication
technologies in a range of industries andother areas of life.

ITU/WHO Focus Group on Artificial
Intelligence for Health (FG-AI4H)

540

experts work
together at
FG-AI4H

FG-AI4H focuses on establishing principles for
applying artificial intelligence globally in thehealthcare arena. With this aim in mind, its
team is developing a standardized framework
for evaluating AI-based methods to be used indecisions relating to health, diagnostics, triage
and treatment. During the COVID pandemic,
the focus group established an ad-hoc groupconcerned with using digital technologies
as a means of supporting the pandemic
response. The focus group is coordinated by
Prof Thomas Wiegand and was established in2018 in coordination with the World HealthOrganization (WHO).

The focus group is coordinated by Dr. Monique
Kuglitsch and is a partnership between the
ITU, the World Meteorological Organization(WMO) and the United Nations Environment
Programme (UNEP). It was established in 2021.

ITU/FAO Focus Group on Artificial
Intelligence (AI) and Internet of Things(IoT) for Digital Agriculture (FG-AI4A)
FG-AI4A investigates the use of artificial
intelligence and the Internet of Things inagriculture, shining a light on gaps instandardization and demonstrating options
for scaling. To achieve this, it operates aplatform that provides all of its stakeholdersopen access to digital innovations in the
agricultural realm. The focus group is coordinated by Dr. Sebastian Bosse and Dr. Ramy
Ahmed Fathy (National Telecommunications
Regulatory Authority of Egypt). It was established by the ITU and the Food and Agriculture Organization of the United Nations (FAO)
in 2021.

74© betibup33/Shutterstock.com

ITU/WMO/UNEP Focus Group on AI
for Natural Disaster Management(FG-AI4NDM)
FG-AI4NDM unites experts with the aim ofresearching how AI-based technologies canbe used to improve resilience to natural disasters. Its researchers focus on best practices
when using AI to detect, forecast, or communicate (e.g., through early warning systems)
natural disasters.

75About us

Career opportunities: working at Fraunhofer HHI

Career opportunities:
working at Fraunhofer HHI

As a research institute with aninternational standing, Fraunhofer HHI offers attractive
career opportunities.
State-of-the-art
laboratories andworkplaces in theheart of Berlin

It provides researchers from the fields ofscience, engineering, informatics and economics the chance to collaborate on exciting
projects with a national and international
scope. Furthermore, they benefit fromoutstanding working conditions plus stateof-the-art laboratory facilities.
Students, graduates and young professionals
are able to gain an early insight into the various areas in which Fraunhofer HHI operates
and then jump straight into their own careers.
They can even apply to write a bachelor’s,
master’s or doctoral thesis at the institute.

76

Employees at Fraunhofer HHI benefit froma great deal of autonomy in the work they
undertake, as well as flexible working hours
and the possibility of working from home.
Child-friendly office facilities also make iteasier to combine work and family life.
Additionally, yoga courses, massages andmuch more are on offer to help
staff stay healthy.

Quality management
For its research, development and production activities in the areas of photonics andelectronics, Fraunhofer HHI has developed andimplemented a quality management systemthat is compliant with DIN EN ISO 9001:2015.
Recertification takes place regularly. The institute most recently underwent this process atthe start of 2022 and was shown to have met
all the requirements of the standard.

© Fraunhofer HHI© Fraunhofer HHI

We're shaping
the future.

Continuing education

Promoting young talent

Our employees are continually given opportunities for professional and personal growth.
The institute offers training and education ona range of subjects, and employees are able
to choose the right options for their needs
according to their career stage and subject ofinterest.

Fraunhofer HHI is passionate about supporting
the next generation of scientists. Every year, theinstitute takes part in the annual Girls’ Day event
held across Germany, during which it invites
girls of school age to visit the institute and gives
them an insight into the world of research.
Experiments are conducted in the lab and arange of workshops are held to provide girls
with a better idea of what STEM careers involve.

Vocational training
Fraunhofer HHI offers vocational training
in the following careers:• Business administrator for officemanagement• IT specialist• Microtechnology specialist

Since 2015, Fraunhofer HHI has been working
with the Albrecht-Dürer-Gymnasium, a highschool in the Neukölln district of Berlin. This collaboration gives the school’s female students theopportunity to go on visits or do placements,
allowing them to find out more about appliedresearch and supplement the subjects they study
with hands-on experiences. As a result, they
emerge with a better idea of what career opportunities are available in science and technology.

HR and recruitment

Marny Rust

marny.rust@hhi.fraunhofer.deFraunhofer Institute for
Telecommunications,Heinrich-Hertz-Institut, HHI
Einsteinufer 37, 10587 Berlin

77About us

Where to meet us

Where tomeet us

Despite thepandemic, our experts
have managed toshowcase their latest
research findings at
trade fairs and eventsworldwide, often on a
virtual platform.

Trade fairs

Events• SENSOR+TEST• World Health Summit• SPIE Photonics West

• Girls‘ Day• GSMA Mobile World Congress• ITG-Fachkonferenz(expert conference)• Optical Fiber Communication
Conference and Exhibition(OFC)• Optics & Photonics International Exhibition (OPIE)

8+25• National Association of
Broadcasters Show (NAB
Show)

countries• LASER World of PHOTONICS• Photonics Days Berlin
Brandenburg• Auditing AI-Systems• media:net berlinbrandenburg
Talent Festival• Berlin 5G Week
• Lange Nacht derWissenschaften• Hannover Messe• FMD Innovation Days
• International Broadcasting
Convention (IBC)• European Conference on
Optical Communication(ECOC)

cities• Technology Innovation Day• ITU-Fokusgruppen Treffen• European Conference
on Networks andCommunication (EuCNC)• PIC International Conference• Consumer Electronics Show
(CES)

78

© Fraunhofer HHI• FUTURAS IN RES

79

Contact details

Contact details
Main building
Fraunhofer Institute for Telecommunications,Heinrich-Hertz-Institut, HHI
Einsteinufer 37, 10587 Berlin
Phone +49 30 31002-0
info@hhi.fraunhofer.de
www.hhi.fraunhofer.de/enVideo and AI departments
Fraunhofer Institute for Telecommunications,Heinrich-Hertz-Institut, HHI
Salzufer 15/16
10587 Berlin
3rd to 6th floor

Branch lab: Fiber-Optical Sensor Systems department
Fraunhofer Institute for Telecommunications,Heinrich-Hertz-Institut, HHI
Fiber Optical Sensor Systems
Am Stollen 19H
38640 Goslar

80