Case Studies of Hybrid Cloud Architectures
for Astronomical Observatory Data
Adopting public cloud services for scientific research will reduce the total cost ofownership, allow the use of state-of-the-art technologies (e.g., the latest GPUs),and establish BCP. However, no methodology exists for designing an optimalarchitecture to realize these advantages. We have been conducting case studiesof storing and analyzing ALMA radio telescope data in public cloud services incollaboration with the National Astronomical Observatory of Japan (NAOJ) todemonstrate the best practices and discuss the design of a suitable architectureusing cloud services.

Case Study Overview
NAOJ is considering a hybrid cloudarchitecture comprising its on-premises
system and additional public cloudservices. To establish the criteria for
optimal data allocation between thecloud and on-premises system, we
ported the ALMA data andanalysis/archiver software to public
cloud services to investigate the costeffective usage of computing resources
and storage.

Archiver
OracleWe developed a technique for dynamicallyattaching instances with the optimal number ofcores and RAM capacity according to theanalysis stage.
We established a resource estimation model
based on the metadata (e.g., observationconditions) for the dataset to be analyzed.
hour
10.00

Elapsed time (h)

$3.20

8.00
6.00
4.00

Cost (US$)$4.00
4-core32-GB
instance$1.57

1-core8-GB
instance

2.00
0.00

US$
$5.00

$1.75

$3.00

AnalysisDB$0.00

1-core 16-GB
instance:• Low cost
• Increase inelapsed time

4-core 32-GB
instance:• Decrease inelapsed time
• Increase incost

Combination of 1core 8-GB instanceand 4-core 32-GB
instance:• Optimal balance

CASANGASSINET6Archive data storeGlacier
S3
Infrequent flexible
Accessretrieval

CASA

2.
Archive data

Cloud (AWS)

On-premises
CASA: Common Astronomy Software Applications package
NGAS: New-Generation Archive System

Migrated bylifecycle management

2. Estimation of cost of storing ALMA archive datain tiered storage consisting of on-premises andcloud storage
The estimation results show the following advantages:
Investment in on-premises storage can be kept constant.
Using two cloud storage tiers (S3-IA and Glacier) inhibits
the cloud cost from increasing in proportion to the total
amount of data.
The influence of the long restore time for Glacier (200minutes) can be mitigated (limited to 10% of downloads).
Yearly cost (US$)$250,000

Glacier store cost$200,000

$2.00
$1.00

Archiver instance1.

Examples of Case Study Results
1. Selection of optimal server instances
for analyses of ALMA data

Analysis instances
NGAS

Amount of data (TiB)
2,500

S3 IA store cost

2,000

transfer cost$150,000$100,000

Instances (NGAS,DB)
Accumulated dataamount (TiB)$50,000

$0

1,500

1,000

500

year year year year year year year year year year year year year-6 -5 -4 -3 -2 -1 01234560

Acknowledgment: We would like to thank the PoC members at NAOJ for providing data and support.

National Institute of Informatics
https://ccrd.nii.ac.jp/sc23/

Amount of archive data:• 900 TiB in year 0• Increase to 250 TiB/year
Datasets are stored in:• On-premises storage for 3 years• S3 IA for 2 years• Glacier after 5 years
Downloads: 550 TiB/year• 20% from S3 IA• 10% from Glacier