content: text: Acknowledgements

content: text: We would like to extend our gratitude to the teams
at Hugging Face, particularly Clementine Four-
rier, Lewis Tunstall, Omar Sanseviero, and Philipp
Schmid. Our appreciation also extends to the teams
at AWS, notably Ritesh Vajaria, Gal Oshri, Jay
Kwon, Brandon Lee, Effie Bae, and Rahul Sharma.
We are grateful to the teams at Korea Telecom
(KT), especially Jin Hyoung Lee, Jungsuk Park,
Sungjoon Park, Hong-rae Wang, Kyeongsoo Jung,
and Sunyoong Yoon, whose significant support has
been instrumental in ensuring the broad compati-
bility of our model. Additionally, we would like to
extend our thanks to the open community for their
invaluable contributions and feedback.

content: text: Limitations

content: text: Our study on the Depth Up-Scaling (DUS) has im-
portant limitations and considerations. One key
limitation is the need for more thorough explo-
rations of hyperparameters used in the DUS ap-
proach. Namely, we removed m = 8 layers from
both ends of our base model, primarily due to hard-
ware limitations. However, we have not yet deter-
mined if this value is optimal for enhancing perfor-
mance. The extended time and cost of continued
pretraining made it challenging to conduct more
comprehensive experiments, which we aim to ad-
dress in future work through various comparative
analyses.

content: text: In terms of the model's broader implications,
there are several points to note. The model's sig-
nificant computational demands for training and
inference might limit its use, especially for those
with restricted computational resources. Addition-
ally, like all machine learning models, it is vulnera-
ble to biases in its training data, which could lead
to skewed outcomes in certain situations. Further-
more, the substantial energy consumption required
for training and operating the model raises environ-
mental concerns, which are critical in the pursuit
of sustainable AI development.

content: text: Lastly, while the fine-tuned variant of the model
shows improved performance in following instruc-
tions, it still requires task-specific fine-tuning for
optimal performance in specialized applications.
This fine-tuning process can be resource-intensive
and not always effective. Recognizing and address-
ing these limitations is essential for a comprehen-
sive understanding of the proposed Large Language
Model's capabilities and for guiding future research

content: text: and development in the field of LLMs.

content: text: Ethics Statement

content: text: We conscientiously address and emphasize the
commitment of SOLAR 10.7B in maintaining the
highest ethical standards. First, we highlight that
SOLAR 10.7B-Instruct has shown low levels of
data contamination in our evaluations, a testament
to our rigorous data handling and processing pro-
tocols. This aspect is crucial, as it underpins the
reliability and integrity of the results obtained from
SOLAR.

content: text: Furthermore, during the course of our experi-
ments, we ensured that all setups and methodolo-
gies employed steer clear of any potential ethical
pitfalls. This preemptive consideration and avoid-
ance of ethically questionable practices underscore
our dedication to conducting research that is not
only innovative but also responsible.

content: text: Additionally, we ensure that SOLAR complies
with general ethical considerations in all aspects
of its operation. This includes adherence to pri-
vacy norms, respect for intellectual property, and
ensuring the absence of bias in our algorithms. Our
commitment to these ethical principles is unwaver-
ing, and we believe it significantly contributes to
the credibility and societal acceptance of SOLAR.

content: text: In conclusion, the ethical framework within
which SOLAR operates is robust and comprehen-
sive, ensuring that our advancements in this field
are not only scientifically sound but also ethically
responsible.

content: text: References

content: text: Ian L Alberts, Lorenzo Mercolli, Thomas Pyka, George
Prenosil, Kuangyu Shi, Axel Rominger, and Ali
Afshar-Oromieh. 2023. Large language models
(llm) and chatgpt: what will the impact on nuclear
medicine be? European journal of nuclear medicine
and molecular imaging, 50(6):1549-1552.

content: text: Rohan Anil, Andrew M Dai, Orhan Firat, Melvin John-
son, Dmitry Lepikhin, Alexandre Passos, Siamak
Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng
Chen, et al. 2023. Palm 2 technical report. arXiv
preprint arXiv:2305.10403.

content: text: Aram Bahrini, Mohammadsadra Khamoshifar, Hos-
sein Abbasimehr, Robert J Riggs, Maryam Esmaeili,
Rastin Mastali Majdabadkohne, and Morteza Pase-
hvar. 2023. Chatgpt: Applications, opportunities,
and threats. In 2023 Systems and Information Engi-
neering Design Symposium (SIEDS), pages 274-279.
IEEE.