recursal/Fanatic-Fandom huggingface dataset:
Заставка

## В фильме

### Текст песни

На старт внимание взлёт!

Джет Ракета его зовут.

Джет Ракета прославится тут.

Он появился и навёл погром.

На пустыре вдруг вырос дом.

Послушай:

"Люди Земли в мире чего только нет!

Планета Бортрон 7 и меня зовут Джет!"

Джет Ракета наш герой.

Похож на нас, но совсем другой.

"Космо-турист и галактик любитель,

Приехал на летающей тарелка родителей!"

И чтобы это доказать

Решил всем космос показать.

Джет Ракета!

На старт внимание взлёт!

Джет Ракета.

На старт внимание взлёт!

Как хорошо что теперь мы вместе

Прогресс не может устоять на месте

"Человечеству привет!

Я изучил весь белый свет!"

"Джет Ракета!"

"Джет Ракета!"

"Джет Ракета!"

На старт внимание взлёт!
Зерк

Это незавершённая статья или статья-заготовка.Эта страница содержит неполную информацию. Вы можете помочь **На старт внимание взлет Вики**, дополнив её.

## Статус

### Родственники

* Цуккини и Баклажанья - родители, Джет Ракета - Двоюродный брат

## В сериале

### Место жительства

Бортрон 7

**Зерк** — второстепенный персонаж мультсериала «На старт, внимание, взлёт!» Он сын Цуккини и Баклажаньи, а также двоюродный брат Джета. Бортронианец.
На старт внимание взлёт Вики

**Добро пожаловать на На старт внимание взлет Вики!**
Добро пожаловать на **Взлётопедию** — онлайн-энциклопедию о мультсериале «На старт, внимание, взлёт!», куда любой участник может внести свой вклад. Мы рады каждому участнику! Просто нажмите кнопку «Править» в верхней части любой страницы или кнопку «Создать страницу» в меню рядом со счётчиком статей. Удачного редактирования!

## Описание

Это Вики о мультсериале «На старт, внимание, взлёт!». Здесь есть даже то, что ты мог никогда не слышать! Например, знаешь ли ты, что  Лучику 899 лет на Бортроне 7? Или что дом Джета превращается в космический корабль?

## Помогите этой Вики!

К сожалению, эта Вики пока не очень популярна. Если вы зашли сюда, то помогите ей, пожалуйста!

## Навигация

Статья о теме викиСтатья о главном персонажеPlaceholderМесто действия темы викиPlaceholderВажная статья
## Хорошая статья

**Лучик** — один из главных персонажей мультсериала «На старт, внимание, взлёт!». Он пришелец с планеты Бортрон 7 и питомец Джета.

### Служебные страницы

Все статьи вики

Категории

Свежие правки

Изображения на вики

## Сообщество

DISKORD-сервер

Доска почёта

Статусы участников

Список участников

Таблица лидеров

Уровни участников

Викимперия

### Статусные участники

ЗАЯЦ — создатель этой Вики

Sophos Troodon — модератор контента

Батарейки ― откатчик

## Категории

Персонажи

Места

Вещи

Песни

Серии

## Содружество Свободных Википроектов

### Приключения Урана и Йена Вики

Также член Содружества Свободных Википроектов.
Death's Response

## Basic

Cost - 100 mana

Effect - Summons 10 zombies. These zombies are not included in the number of minions you may command and turn to dust in 10 min.

Cd- 30min

## Experienced

Cost - 100 mana

Effect - Summons 15 zombies. These zombies are not included in the number of minions you may command and turn to dust in 20 min.

Cd- 30min
Progression

(Not yet complete, awaiting info)

**Easy Eyes**

 **Simple Eye**

 **Lost Eye**

 **Corrupted Eye**

 **Black Eye**

 **Evil Eye**

 **Exotic Eye**

 **Guardian Eye**

 **Rogue Eye**

**Medium Eyes**

 **Old Eye**

 **Witch Eye**

 **Cold Eye**

 **Cursed Eye**

 **Magical Eye**

**Hard Eyes**

 **Nether Eye**

 **Undead Eye**

 **Wither Eye**

 **...**
A Simple Eye

**Fire Charge:**

Fire Charge can be found early-game in some chests and from killing Wraiths that spawn in Plains, Forests, and Caves.

**Wraith Relation**

[1]https://www.curseforge.com/minecraft/mc-mods/dungeons-mobs

**Ender Pearl:**

Ender Pearl can be found early-game in some chests, trading with cleric villagers, or by killing Enderman.

**Glowstone Dust:**

Glowstone Dust can be found early-game in many structures, from killing witches, trading with cleric villagers, or crafted using a combination of Redstone, Ametrine Shards, and any of the following: Glow Lichen, Glow Ink Sac, Glow Berries.

**Ametrine Shard Relation**

[2]https://www.curseforge.com/minecraft/mc-mods/iter-rpg

**Witch Pupil:**

Witch Pupil can be crafted by combining Spider Eye, Glowstone, Redstone, Sugar, and Gunpowder.

**Witch Pupil Relation**

[3]https://www.curseforge.com/minecraft/mc-mods/endremastered

**Blaze Powder:**

Blaze Powder can be found early-game in some chests or by combining a Witch Pupil and a Fire Charge together.
Heart Attack

A drawing of Heart\_attack

## Overview

Heart\_attack is the Lead Director of PROJECT: Fear, also the Programmer, and a modeller, she's also a faceless VTuber that barely streams.

## Trivia

* She loves anime
* She joined GameJolt in January 1st, 2021, but started being active 10 months ago.
* First ever seen in 2019/2020

* Sometimes she can get a little mad....
TerKeno

## Overview

**TerKeno**, or **TerKenoDev**, is the Composer for PROJECT: Fear, they've made every single soundtrack and helped with some sound effects in the game.

## Trivia

* TerKeno used LMMS, FL Studio, and Audacity to compose the soundtracks and the sound effects
* TerKeno ***REALLY*** likes ULTRAKILL
* TerKeno joined the PROJECT: Fear team in 06/15/2023. [1]
* He joined GameJolt on September 3rd, 2020 because of the FNF Modding community mostly moving to GameJolt, but only started being active in 2021/2022.
* There's an inside joke that TerKeno flexes his free time to SpringFeddXD.
The Chocolate Edition Wiki

**Welcome to the The Chocolate Edition Wiki!**
This is not an edition of Vanilla Plus, Vanilla 2, or any other fancy term- it's Chocolate. It's pure, unadulterated Minecraft with the magical touch akin to Terraria developers; a Minecraft without menial tasks, one with no need to create a survival world only to abandon it when you acquire iron ore. This mod pack has taken Minecraft and made it remarkable.

## Important articles

The EyesTier 1 Locked Items

*Need help building out this community?*

* Rules of this wiki
* Getting Started
* How to Contribute
* Managing your new community
* Guides
* All Help articles

*Community Founders*: Write a good and paragraph-length description for your welcome section about your topic. Let your readers know what your topic is about and add some general information about it. Then you should visit the admin dashboard for more tips.
На старт внимание взлёт (песня)

## В фильме

### Текст песни

#### Основной вариант

Раз два, раз два три

Сядь удобней, дверь запри

Пассажиры все на месте?

Полетели в космос вместе!

На старт внимание взлёт!

#### "Возвращение на Бортрон 7"

Раз два, раз два три

Сядь удобней, дверь запри

Мусор вон, бельё сюда

Пристегнись, не зевай

Съешь обед, оглянись

Всё закрой и пригнись

Раз два, раз два три

Сядь удобней, дверь запри

Корабль наш идёт на взлёт

Отправляемся в полёт!
SpringFeddXD

## Overview

SpringfeddXD is the Animator, modeller (Enviorment) and a Tester for Project : Fear, they've helped Project : Fear to its greatest for Animations and Maps.

## SpringFeddXD

SpringFeddXD's PFP on GameJolt as of October 10th 2023

## Trivia

* SpringfeddXD uses Blender to animate cutscenes, animatronics, etc.
* SpringfeddXD also uses Blender to model the maps.
* SpringfeddXD joined in Project : Fear team around in early/mid 2022.
* Joined in Gamejolt 2 years ago for his Game development, then went quiet until he started to be active in 2023.

19/3/12 (3/19/23 for Americans)
Ubicaciones

Ubicaciones donde se ambienta *Las aventuras de Virgilio Echeverría*.

## General

Nueva Palma, oficialmente la República de Nueva Palma, es un país soberano de ficción ubicado en Centroamérica, con el Distrito Federal como capital. Es el lugar donde viven la mayoría de personajes de la franquicia, y su gentilicio es "neopalmeño" o "neopalmense".
Project Fear

The game's thumbnail on GameJolt.

## Overview

PROJECT: Fear is a in-development horror game based on FNaF: Security Breach, whose gamepage was released in April 24, 2023.

## Description

Story - Play As a girl named Charlotte Who has been dared by her friends to uncover the secrets Of A place called ~~*rEDacTED.*~~ While weird things are happening with the Animactronic mascots as time goes on. Can she survive this hellish mall?

Gameplay - Five Nights With Fear Is a 3d Freeroam game, Made in Unreal engine 5 With blueprints. Hide. Play. Uncover. *Survive*.

Replayability - Five Nights With Fear will be set in chapters, Which means coming back for more will be certain. Fears Mega Mall is big, So their will always be room for exploring.

Design - Now his is the tricky part. Because we lack funding. (And we never will have) Not everything can be made from scratch, And everyone involved isnt getting paid. Altough these setbacks are here. I'll be sure to do my best to credit models that are not ours and are/arent edited.
Eri Sendai

Eri Sendai is voice actress

## Roles

* Mermaid Melody Pichi Pichi Pitch - Hanon Hôshô
* Best Student Council - Ayumu Oume
* Happy Seven - Kuan Kitayama
* My-HiME - Kiyone Nonomiya
* xxxHOLiC - Ayaka
* Bakugan series - Runo Misaki
* Pokemon - Gardenia
* Yes Pretty Cure 5 - Milk/Kurumi Mimino/Milky Rose
* And Yet the Town Moves - Yukiko Arashiyama
* Waccha PriMagi! - Myamu
* Apparently, Disillusioned Adventurers Will Save the World - Tiana
* Beyblade X - Multi Nanairo
* Kibou no Chikara ~Otona Pretty Cure '23~ - Kurumi Mimino

## Gallery

Eri Sendai seiyus Multi Nanairo (Beyblade X)
FantasyBordanBall Wiki

**Welcome to the FantasyBordanBall Wiki!**
We're a collaborative community website about FantasyBordanBall that anyone, including you, can build and expand. Wikis like this one depend on readers getting involved and adding content. Click the "ADD NEW PAGE" or "EDIT" button at the top of any page to get started!

## Important articles

PlaceholderA page about your topicPlaceholderA Main CharacterPlaceholderThe First EpisodePlaceholderAn Important LocationPlaceholderA Key EventPlaceholderA Crucial Item

*Need help building out this community?*

* Rules of this wiki
* Getting Started
* How to Contribute
* Managing your new community
* Guides
* All Help articles

You can also be part of the larger Fandom family of communities. Visit Fandom's Community Central!

*Community Founders*: Write a good and paragraph-length description for your welcome section about your topic. Let your readers know what your topic is about and add some general information about it. Then you should visit the admin dashboard for more tips.
Ever Blue

Ever Blue is album 2003 with peformer by Eri Sendai in Mermaid Melody Pichi Pichi Pitch

## Track List

1. Ever Blue - Eri Sendai
2. Super Love Songs! (Hanon Solo Version) - Eri Sendai
3. Legend of Mermaid (Hanon Solo Version) - Eri Sendai
4. Ever Blue (Off Vocal)
5. Super Love Songs! (Off Vocal)
6. Legend of Mermaid (Off Vocal)
Wiki Las aventuras de Virgilio Echeverría

**¡Bienvenido/as a Wiki Las aventuras de Virgilio Echeverría!**
La página de Fandom oficial de la serie *fanon* **Las aventuras de Virgilio Echeverría**, estelarizada por el legendario personaje juvenil del mismo nombre y sus amigos. Sigue su página oficial en Propuestas de Fans para Fans.

Somos una comunidad web colaborativa sobre **Las aventuras de Virgilio Echeverría** en la que cualquiera, incluyéndote a ti, puede contribuir. Los wikis como este dependen de que los lectores se involucren y añadan contenido. ¡Haz clic en el botón de *"CREAR UN NUEVO ARTÍCULO"* o *"EDITAR"* al comienzo de cada página para comenzar!

**Atención: Todo el contenido relacionado a la franquicia es fanon. Esta Wiki es sobre historias y contenido escrito únicamente por fans para entretener. Por favor, salga de esta Wiki si esta buscando información legítima.**
## Artículos importantes

Conoce a los personajes de la serie.

*¿Necesitas ayuda para construir esta comunidad?*

* Normas del wiki
* Primeros pasos
* Cómo contribuir
* Dirigiendo tu comunidad
* Guías
* Todos los artículos de ayuda

También puedes ser parte de la mayor familia de comunidades de Fandom. Visita la Comunidad Central de Fandom!

*Fundadores de la comunidad*: Escribid un buen párrafo describiendo el tema de la comunidad aquí. Permitid que vuestros lectores sepan más del tema sobre el que trata este wiki y algo de información general sobre el mismo. Luego visitad el panel de administración para más consejos.
A page about your topic

To progress in this mod pack, you must craft 12 unique, special eyes that unlock items and dimensions. Almost every eye requires an ender eye to craft. An ender eye is crafted from an ender perarl (dropped from enderman that spawn at night) and blaze powder ( dropped from blazes in the nther or crafted with a fire charge and witch pupil early game to get to the nether). Look in the other tabs for how to craft specific eyes.
Oin Pod

Ion Pod is a robot built by Pual who was originally sharing the same mind as Ion Pod, but was given the ability to have free thought after he was given advanced AI. He is a strong puncher, if he ever uses a gun, it will normally be his old revolver. His AI is off of a cowboy who lived in the Western side of Woodavalia. This makes him think that he was killed and brought back to life. He spends his free time searching for the mysterious gunslinger who killed the cowboy. Back to Home
Anime World Tower Defense V2 Wiki

**Welcome to the Anime World Tower Defense V2 Wiki!**
Fan made Wiki cause erm why not.

Tierlist

Codes

## Important articles

PlaceholderA page about your topicPlaceholderA Main CharacterPlaceholderThe First EpisodePlaceholderAn Important LocationPlaceholderA Key EventPlaceholderA Crucial Item

*Need help building out this community?*

* Rules of this wiki
* Getting Started
* How to Contribute
* Managing your new community
* Guides
* All Help articles

You can also be part of the larger Fandom family of communities. Visit Fandom's Community Central!

*Community Founders*: Write a good and paragraph-length description for your welcome section about your topic. Let your readers know what your topic is about and add some general information about it. Then you should visit the admin dashboard for more tips.
The Flame Den attack

Personally made by Queen Willow during the war of stone, they found the base of the Wing'Dusts, waited until they fell asleep in there dens before setting them aflame (hence the name), after they had set the dens aflame, they killed the enemy in there sleep, if they tried to run away, they would chase them down and kill them, if they tried to fly away they would tear there wings and let them bleed out as punishment. this attack is sometimes called inhumane, as they where killing many of there young, elderly as they had no fight in the war, This attack drove the Wing'dusts into near extinction, around 58,000 died ruing the attack around 376 injured.
Eligible Draft Players

300 players are picked every year to be eligible to be drafted. Only 120 players are drafted each year, and all the teams have 300 players to choose from for there picks. Here are, in order from best to worst, all the players eligible for this year's draft.

1. Terry Fay
2. Eric Bowmens
3. Tony Lehman
4. Rod Collins
5. Bryan Hurst
6. Kyle Merill
7. Mark Crowley
8. Clay Farmer
9. Angelo Sharpp
10. Morris Sloan
11. Bruce Brock
12. Kurt Owens
13. Grant Green
14. Vince Wilnox
15. Ted Thomas
16. Roman Hailey
17. Russel Smith
18. Jhonny Hày
19. Gibbson Young
20. Yutaka Takashita
21. Scoot Parry
22. Will McCann
23. Ken Grey
24. Phillip Costa
25. Jackie Scott
26. Tyrone Malone
27. Daniel Barton
28. Jack simms
29. Gordan marsh
30. Ross Rose
31. Joey Good
32. Andrew Darling
33. Keith Connor
34. Aaron Baller
35. Lloyd Myers
36. Ash Apollo
37. Xavier Jadden
38. Sifiso Kayode
39. Cole Stone
40. Carter Jordan
41. Mark Nelson
42. Lowell Studen
Eri Sendai Wiki

**Eri Sendai Wikiへようこそ！**
このWikiは、Eri Sendaiに関するウェブサイトを、同じテーマに興味を持つインターネット上の仲間と協力しながら作り上げていくための場所です。まずは、ページの上のほうにある「新しいページをつくる」または「編集」ボタンをクリックして、サイトの構築を開始しましょう。

## ご利用の手引き

* Wikiのルール
* 使ってみよう
* 編集するには？
* コミュニティを管理しよう
* ガイド
* 全てのヘルプをチェック！

このWikiは、多くのFandomコミュニティの１つでもあります。ぜひFandom コミュニティセントラルもご活用ください！

**設立者の方へ**: このウェブサイトでどのような話題を扱うのか、解説を書きましょう。話題 (ゲーム、テレビシリーズなど…) についての基本的な情報をメインページに書き、訪問者に伝えましょう。アドミン・ダッシュボードでは、サイト構築のヒントをご案内しています。
Wing'Dusts

Silly goobers that have wings

CATS WITH WINGS

They tried there best to respect the boarders of the cosmic rays boarders but due to how huge space waS THEY HAD a bit of a struggle, eventually after war was waged against them there leader, Flower, tried her best to reason with them in exchange of peace, they refused, witch caused Flower to lunge an attack on the eggs right after that meeting, Eventually after The War of Stone, and the Flame Den attack they went into hiding, most have now lived underground inside of some planets, wich now has many hybrids due to the near extinction of there own species because of that attack, they have only left there sequence of caves every few years to spy on the (idk) kingdom.

as of Currently Quartz has been trying to avoid war with them, as they are aggressive and want revenge.

## Oc's of that species

* Maple
* Wysteria (Dead)

## Hybrids

* Mayflower
* Kelp (also dead)
딥러닝

딥러닝(deep learning) 또는 심층학습(深層學習)은 깊은 인공신경망을 이용한 기계학습 과정이다. 다른 인공지능보다 설명가능성이 부족하긴 하지만, 다양한 분야의 SOTA가 되어 많이 쓰이고 있다.

## 신경망의 종류

* 합성곱 신경망
* 순환 신경망
* 변환기
* 그래프 신경망

원리
--

딥러닝의 원리는 신경망을 임의로 넓거나 길게 만들면 모든 함수를 근사할 수 있다는 보편 근사 정리(universal approximation theorem)에서 시작된다.

## 주요 작업 분야

* 컴퓨터 비전
* 자연어 처리
* 추천 시스템
* 음성 인식
* 생성형 인공지능 : 생성 모델 등

## 같이 보기

* 설명가능 인공지능
* 파운데이션 모델
* 다층 퍼셉트론
Quartz

Im so sorry

anyways

A very gullible child that was placed into a position of power too early.

## Toby

He has no idea who Toby is, as Willow has only said that he was a dead-beat dad, wich led to some accidental hatred against him,

Once he met Toby due to him saying it was... "urgent" he was convinced he was insane for saying that he was his father (Even tho Iris, Copper and Beetle said it was true) But him saying that Willow was Alive was a bit of an insane statement, He saw the body, he saw the blood, he saw the dead eyes of his mother looking at him with fear, How could he say she was alive?

## Iris

Despite being distant with his adopted sister he feels like he has a somewhat good relationship with her, all up until the royal massacre, he doesn't despise her but just doesn't want to be around her after what she had done, leaving her as his prisoner until he convinced Copper and Beetle to help Iris get into...an atleast better mental state, once she was somewhere around better, he made her his royal advisor out of somewhat pity.

## Beetle

Best brothers, the amigos I guees, they would often hang out every once in a while, all the way until Willow had forced him to see one of the tournaments(I forgot) she was holding for Beetle, he was terrified that his own brother was killing innocent Rays for \*\*FUN\*\*, after that he had hated him and hissed at him if he ever got close, he was not afraid to show him that he hated him, After the Royal massacre he felt like he had misjudged Beetle far to quickly, Feeling guilty, he made Beetle his royal advisor for his troubles.

## Copper

Despite not knowing him much he made him his royal advisor because he was nervous and needed more new ears and voices in his Royal Advisor instead of the old Wrinkly something thousand year olds saying mean stuff about his actions, he likes Copper due to his gentleness and kindness, Quartz trusts him more out of everyone.
VoVNet

VoVNet(보브넷)은 ResNet과 DenseNet을 절충한 합성곱 신경망이다.

An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection(arXiv:1904.09730)에서 제되었다.

ResNet을 개량한 DenseNet은 한 stage에 있는 모든 레이어들이 이전 레이어에서 만들어낸 feature들을 모두 concatenate하고, 그걸 바탕으로 새로운 feature를 만들어서 다음 레이어에 전달하는 방식으로 구성된다.

이과정에서 매번 concatenation을 하느라 메모리 접근에 시간이 많이 드는 문제가 있었다.

VoVNet에서는 맨 마지막 결과를 내는게 아니면, 이전 layer의 output을 가지고만 새로운 feature를 만든 뒤, 매 레이어에서 만든 feature를 한번에 합치도록 개선하였다.

VoVNet-27-slim, VoVNet-39, VoVNet-57 구조가 있으며 실험에 사용된 VoVNet 블록은 3x3 합성곱 레이어 5개를 쌓은 뒤 그 결과를 모두 concatenate 시키는 One-Shot Aggregation을 한 뒤, 1x1 합성곱 레이어를 통과시켜 마무리 지었다. 큰 버전에선 뒷 스테이지의 경우 여러개의 VoVNet 블록으로 구성했다. 각각의 합성곱 레이어는 합성곱만으로 된게 아니라 Conv-BN-ReLU의 조합으로 구성됨에 유의해아 한다. 한 블록의 모든 layer들의 결과들이 Aggregated된 텐서는 크기가 크기 때문에 1x1 합성곱 레이어를 통과시켜 계산 시간을 줄였다.

## 관련 문서

* DenseNet
* PeleeNet
* ELAN
Willow

* Current
* Past

A long white haired Comet ray with dark navy top fur and dark cyan chest fur with twisted white horns with gold underneath it, she has a long muzzle with rabbit ears that are pierced with earrings, the middle one is a star earring while the other ones are hanging earrings. (she thinks this makes her look more majestic)

Was a soldier during The War of Stone

shes a comet ray ig

## The Creator

After she was accidentally thrown into a blazing star, she met the creator, who she judged by her sense of fashion, but then it said something that she could not understand and sent her back to her home universe, even after the creator revived Willow, she still doubts the creator, refusing to talk to her when she enters her mind,

## Toby

She first met him after almost dying to an explosion when she was a soldier fighting during the Solare War.

idc how they fell inlove honestly but they adopted Iris and Beetle right after Willow became the queen. after toby tried to runaway with Quartz (He was a egg and im to lazy to explain that stuff but watever) Willow banished Toby off of her world, while Quartz stayed with her

## Iris

Iris is probably Willows 2nd favorite child, just behind Quartz, Before Quartz was born, Iris was Originally ment to take the throne after Willow. But since Willow prefered Quartz over Iris, she was ultimately replaced by him. Willow, only made her a Royal Advisor just so she would stop pestering her about being someone important, during the royal massacre, Right before Iris killed Willow, She was thinking about how annoying Iris was.

## Beetle

This is going to be alot.

Willow didint really like Beetle he was more of a ghost, invisible. but after Toby had been banished, Willow forced Beetle into a lions diet (I am way to lazy but their diet defies what they look like, Ex: Willow ate a rabbits diet, she got rabbit ears and rabbit muzzle. but they have to be on the diet for a while for their to be noticeable changes) and made him her champion. Willow would often force Beetle to kill for her own entertainment, enjoying how he fights and how others counter it. Willow dislikes Beetle alot. once when he was a child, Willow had sent him into the dungeon for accidentally spilling some water onto a carpet. She views him as just a pawn

## Quartz

Willow's favorite child, She treats him way better than the first two, he has more freedom. Willow loved him the moment he hatched, and once he reached the age to be a heir she immediately replaced Iris for him

(More coming soon)

## Killed victims

* 32 unknown Wing'dust
* Corronet (throwing her into a blazing star, who she was also dragged into it)
* Herself (Accidental suicide)
Royalty

## Challenges

If a prince/princess chooses to challenge the throne the challenger and King/queen must abide thies rules

They both must fight until one surrenders or one kills the other.

The Challenger must be atleast 9,500 years old (Idk)

If both King and Queen are alive when a Prince/princess challenges, only ONE must fight.

If the Prince/princess wins the challenge, both King and Queen (if both of them were alive) MUST step down, or also be killed after the Royal challenge.

Cannot use any other weapons besides there teeth, horns and claws.

If cheating has been found after the Royal Challenge (Ex: Poisoning both King and Queen so they are weakened) the crown is forcefully removed by the current Queen/King(s?) and given to the oldest prince/princess or a heir.

## Kings and Queens (Keengs?)

They both share the power of the kingdom, neither has more power than the other, which often leads to if one of them dies, the other often stays widowed so they do not have to fight with the other one with decisions.

## Princess/Princes

Usually they live a normal life in the castle, no important opinion, they can challenge there king/queen, unless they are grand-children

## Heirs

If the King and Queen do not want to give there thrown to there own children, they can get heirs, they do not have to be there own blood but must have lived in the castle since childhood, they can be any age, but still have to follow the rules of Royal Challenges. If both the Queen and King die and they have a heir, the crown must go to the heir and not the princess/princes
An Empirical Study of Training Self-Supervised Vision Transformers

An Empirical Study of Training Self-Supervised Vision Transformers(arXiv:2104.02057)는 대조 학습을 통한 자기 지도 학습의 기법인MoCo(Momentum Contrast, 모멘텀 대조)의 세 번째 논문이다. MoCo를 비전 변환기에 적용하기 위해 SimCLR처럼 배치 사이즈를 크게 하고,

배치 사이즈가 큰 자기 지도 학습에 적합한 LARS optimizer를 AdamW처럼 모멘텀이 들어간 LAMB로 대체하였다. 비전 변환기에 입력되는 패치데이터를 받아 차원수를 늘려주는 random patch projection 레이어는 고정하였다.

## 같이 보기

* MoCo(Momentum Contrast, 모멘텀 대조)
* Momentum Contrast for Unsupervised Visual Representation Learning (arXiv:1911.05722): MoCo V1
* Improved Baselines with Momentum Contrastive Learning (arXiv:2003.04297): MoCo V2
* Unsupervised Learning of Visual Features by Contrasting Cluster Assignments(arXiv:2006.09882) : SwAV
* Bootstrap Your Own Latent A New Approach to Self-Supervised Learning(arXiv:2006.07733) : BYOL
* Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination(arXiv:1805.01978) : kNN monitor
* SEED: Self-supervised Distillation For Visual Representation(arXiv:2101.04731)
* SimCLR
* Large Batch Training of Convolutional Networks (arXiv:1708.03888) : LARS
* Large Batch Optimization for Deep Learning: Training BERT in 76 minutes (arXiv:1904.00962) : LAMB
Green Dude

Green Dude is an alien species known as the Grondaves. He lives on the planet Grondavion, he also has very bad brain damage. His skull was damaged when he got stuck in the middle of the war between humans and Grondaves. He snuck into Pual's apartment and was interrogated, they eventually brought him in as their own and took care of him. They didn't know his name due to his inability to talk, so they nicknamed him Green Dude. Back to Home
Iris

> "I didn't mean to! I swear! Why are you painting me as this insane massacrer?"
>
> Iris to Quartz

## Iris

### Relatives

Adopted mother: WillowAdopted Father: Toby
Brother: Beetle
Partner: Maple (Formally)

### Occupation

Currently: Royal Advisor of QuartzPast: Prisoner of Quartz
Royal advisor to Willow,
Princess of (idk)

## Physical description

### Gender

Female

Just a silly goober <3

WHO KILLED HER MOM1

idk

## Willow

At first she loved Willow, she saw Willow as her idol, and finding out she could be queen after her just fueled it. Until Quartz was born and she was replaced by him, she was angry, at Willow for replacing her after all that she has DONE for her, so she may have drank a bit too much wich caused the Royal Massacre, Killing Willow, 5 servants and a few distant cousins. She stayed in the cell for what she felt was months, suddenly Copper and yeah im finishing this later actually

## Quartz

At first, she loved Quartz, she couldn't fathom that he would ever replace her, but once Willow announced that he would be her replacement, she hated him, she sometimes wishes that Beetle had actually manage to accidentally drop him when he was an egg, Or that Toby had actually managed to steal him and take him to who knows where.

She has mixed feelings, she feels like Quartz pitys him despite her past murders.

## Toby

she doesn't have many memories of him, they where either forgotten or just her seeing him far away, She thinks that he is a coward for leaving Willow and stealing Quartz egg with him and not even trying to retrieve the egg after it was taken away from him.

she is slightly concerned and angry at Toby, one, he seemed to....rationalize her killing Willow? she is confused and angry that

## Copper

Never really talked to him until she was dragged into being A Royal Advisor for Quartz, She saw how surprised he looked when Toby started to rant about "The creator"...she seems him suspicious and thinks he knows something about The creator...

## Beetle

Being each others therapist really effected them huh? Iris would often drink with Beetle to forget about the kingdom, there jobs and other things, Iris loves Beetle because he has been with her through most of her life, but after the Royal Massacre, Iris thinks he is annoying.

## Maple

Lesbians until she was made into a Royal Advisor for Willow, wich she was forced to leave her gf :( (Im lazy deal with it)

## Killed Victims

* Willow
* 5 unamed servants
* 2 distant cousins
Beetle

## Beetle

"Are you seriously choosing \*child\* over her?" - Beetle to Willow

### Aliases

Bug (by Willow)Puffy boy (By Iris)
Lion chomper (By Maple)
disgusting (By Quartz)

### Relatives

Adopted mother: WillowAdopted father: Toby
Adopted brother: Quartz
Sister: Iris

### Occupation

Current: Quartz Royal AdvisorFormally: Queen Willow's royal champion

## Physical description

### Gender

Male

AAGDVHWYGFYUKEUYFHLGUIEHLFIU INHATEBYOI

That isint beetle Im just using it as a placeholder lol
Toby

* Current
* Past

## Description

A large Comet ray with green eyes and dark fur and mane, stained with stardust from his years staying in space and the constant contact with star dust.

He knows about The Creator as of recently.

during the hatchery massacre, he was in the hatchery hiding due to having no experience in fighting, he has told NO ONE about this due to feeling guilty and scared that someone will kill him for it. After the hatchery massacre he found two eggs who he asked Willow to raise with him before the flame den attack, Willow agreed, After a while Willow and Toby had there own egg, Quartz

He SWEARS, Willow is still alive and was not killed by Iris.

was a medic during The War of Stone

## The Creator

He hates the creator, he fears the creator for what they did to Willow, Iris, Beetle and Copper,

## Willow

He really cared for Willow, despite the fact that he was a medic and Willow was a Solider, Even after the war and after having such a long distant relationship he still loved her until he found out she had killed his mother just to become queen, He has complicated feelings for that so he ran away with Quartz out of guilt of being with the ray that killed his own mother, after he got away without Quartz, he feels anger and hatred towards her but foursome reason still cares about her

## Beetle and Iris

He loves both of them with his heart, yet he feels guilty for not taking them with him while he was running away, but after hearing about what had Iris had done in the Royal Massacre he is extremely guilty but a bit of hatred for Iris.

## Quartz

He loves him, not as much as Iris and Beetle but he loves him, he has only seen him when he was an egg until he went back to home world for Quartz coronation, he feels extremely bad that he couldn't have gotten away with Quartz, he wouldn't be king in such a young age
RandomLore Wiki

**Welcome to the RandomLore Wiki!**
Some lore of my Ocs :3
Please do not edit this page or anything on this wiki. this is for fun and i do not want any of my hard work ruined.

Important Characters:

Beetle, Toby, Willow, Quartz, Iris, The Creator

Species:

Comet'rays Wing'dusts

Important events

The War of Stone The Flame Den attack

## Important articles

PlaceholderA page about your topicPlaceholderA Main CharacterPlaceholderThe First EpisodePlaceholderAn Important LocationPlaceholderA Key EventPlaceholderA Crucial Item

*Need help building out this community?*

* Rules of this wiki
* Getting Started
* How to Contribute
* Managing your new community
* Guides
* All Help articles

You can also be part of the larger Fandom family of communities. Visit Fandom's Community Central!

*Community Founders*: Write a good and paragraph-length description for your welcome section about your topic. Let your readers know what your topic is about and add some general information about it. Then you should visit the admin dashboard for more tips.
대조 학습

대조 학습(Contrastive Learning)은 학습을 통해 서로 다른 입력은 멀게, 가까운 입력은 가깝게 변환시키는 법을 학습하는 기법이다.

논문
--

* A Simple Framework for Contrastive Learning of Visual Representations(arXiv:2002.05709): SimCLRv1
* Big Self-Supervised Models are Strong Semi-Supervised Learners(arXiv:2006.10029): SimCLRv2
* The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models(arXiv:2012.06908)
* Contrastive Representation Learning: A Framework and Review(arXiv:2010.05113)

## 같이 보기

* 유사도 학습(Similarity learning)
* 비지도 학습
* 자기 지도 학습
* 삼중항 손실
* 비대조 학습
얼굴 인식

얼굴 인식(face recognition)은 얼굴을 감지할 뿐 아니라, 그 얼굴이 누구의 얼굴인지 식별하는 것이다. 다른 이미지 분류보다 클래스의 종류가 많고 고정되지 않기 때문에 어려운 편이다.

논문
--

* FaceNet: A Unified Embedding for Face Recognition and Clustering(arXiv|1503.03832}})
* ArcFace: Additive Angular Margin Loss for Deep Face Recognition(arXiv:1801.07698)
* Sub-center ArcFace: Boosting Face Recognition by Large-scale Noisy Web Faces(Papers With Code)
* SphereFace: Deep Hypersphere Embedding for Face Recognition(arXiv:1704.08063)

## 같이 보기

* 얼굴 감지(face detection)
* 얼굴 정렬(face alignment)
대규모 언어 모델

**대규모 언어 모델**(Large language model, LLM) 또는 **대형 언어 모델**은 딥러닝 기법에 의해 수많은 지식을 습득하여 자연어 처리를 하는 언어 모델이다. 지식을 담기 위해 크기가 커진다.

종류
--

* GPT 시리즈 : OpenAI
	+ GPT
	+ GPT-1
	+ GPT-2
	+ GPT-3
	+ GPT-3.5
	+ GPT-4
	+ ChatGPT
* LLaMA : Meta AI
* LaMDA
* PaLM
* Gemini
* Inflection-1, Inflection-2 : Inflection AI

## 피드백

LLM을 학습하기 위해서는 정책을 최적화 하기 위한 피드백 과정이 도움이 된다.

* 인간 피드백을 통한 강화 학습 RLHF(Reinforcement Learning from Human Feedback)
* 직접 선호 최적화 DPO(Direct Preference Optimization)
* 인공지능 피드백을 통한 강화 학습 RLAIF(Reinforcement Learning from AI Feedback)
* Direct Preference Optimization: Your Language Model is Secretly a Reward Model(arXiv:2305.18290)
* RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback(arXiv:2309.00267)

## 성능 향상

* Chain-of-Thought Prompting Elicits Reasoning in Large Language Models(arXiv:2201.11903)

## 관련 문서

* 토큰화
* 언어 모델
* Parameter Efficient Fine-Tuning
* 멀티모달 대규모 언어 모델
* 경량 대규모 언어 모델(small Large Language Model, sLLM)은
Knowledge distillation: A good teacher is patient and consistent

Knowledge distillation: A good teacher is patient and consistent(arXiv:2106.05237)은 지식 증류에 대한 논문이다. 좋은 선생님은 인내심이 있고 한결같아야 한다고 말한다.

목적
--

이 논문에서는 지식 증류를 할때 큰 선생 모델은 로딩 문제로 선생과 학생의 출력을 다른 데이터 증대 방법을 써서 따로따로 계산하는 경우도 있다면서, 어떻게 학습을 해야 지식 증류가 잘 되는지 분석하였다.

이를 위해 선생과 학생의 데이터 증대 방법을 바꾸어 4가지의 실험을 했다.

* fixed teacher
선생은 고정된 이미지를, 학생은 random crop augmentation을 받는다.
* independent noise
선생과 학생 모두 random한 independent noise로 augmentation된다.
* consistent teaching
선생과 학생 모두 augmentation된 똑같은 이미지를 받는다.
* function matching
선생과 학생 모두 mixup 데이터 증대 기법에 의해 동일한 입력을 받는다. mixup은 두 이미지를 평균내면서, label도 평균내는 기법이다. 그렇게 되면 실제 입력 데이터셋에는 없는 종류의 이미지로, 입력 다양체 밖으로 벗어난 데이터를 입력받게 된다.

실험
--

실험은 Big Transfer (BiT): General Visual Representation Learning(arXiv:1912.11370)의 BiT을 적용한 BiT-ResNet을 가지고 했다. 선생 모델은 BiT-ResNet-152x2, 학생은 BiT-ResNet-50인데 x2는 width를 두배로 했다는 뜻이다.

손실 함수는 Distilling the knowledge in a neural network(arXiv:1503.02531)의 쿨백-라이블러 발산(Kullback-Leibler divergence, KLD)을 그대로 적용했다.

Flowers102, Pets, Food101 , SUN397 데이터셋에 대해 실험했다.

## 실험 결과

* 데이터 증대 기법
consistent teaching과 function matching이 더 우수했다.
처음에는 function matching이 낮은 정확도를 보이지만, 결국 정확도를 따라잡고 과적합으로 덜 빠지듯, 데이터 증대를 거쳐서 학생과 같은 데이터를 받으면서 오랫동안 훈련하는게 중요하다고 설명한다.
이것은 같은 위치에서 crop한 사진인데 선생과 학생 사이에 해상도가 달라, 텐서 크기가 다른 경우도 유효하다.

* optimizer 선정
Shampoo: Preconditioned Stochastic Tensor Optimization(arXiv:1802.09568)에 나온 2차 선조건자(second order preconditioner) Shampoo를 Adam 대신 사용한 경우에도 유효하고 1차 옵티마이저보다 2차 선조건자를 쓰는게 성능을 올린다고 제안했다.

* 사전학습된 가중치를 사용해야 하는가?
또한 학생 모델에 사전학습을 통해 좋은 초기화를 하는 것보다 그렇지 않은게 나을 수 있다고 실험 결과를 통해 제안했다.
* 선생과 학생 사이에 모델군이 달라도 그런 실험결과가 나오나
MobileNet v3을 학생으로 쓴 경우도 유효하다며, 선생과 학생의 모델군이 달라도 유효하다고 제안했다.
이 모델은 그룹 정규화(Group Normalization, arXiv:1803.08494)를 쓰기 때문에 배치 정규화를 쓰는 다른 네트워크와 다르다.
* "out-of-domain" 데이터의 효과
mixup으로 증대한 데이터는 일종의 "out-of-domain" 데이터이기 때문에, OOD 데이터의 효과에 대해 실험할 필요가 있다고 지적했다. "in-domain"으로 실험한게 제일 좋고 ood로만 한건 성능이 좋지 않지만 어느정도 지식을 배운다고 설명했다. 둘이 섞여있으면, 궁극적으로 in-domain으로 실험한 성능을 따라잡는다고 제안햇다.

## 관련 주제

모델 압축(model compression) 중 모델 가지치기(Model pruning)과 가중치 양자화(Weights quantization), 텐서 분해(Tensor decomposition)도 관련 주제로 소개했다.
언어 자원

언어 자원(language resource)은 언어가 가지고 있는 컨텐츠를 자원으로 분류한 개념이다. 언어 자원이 많을 수록 자연어 처리의 질이 높아진다.

## 언어 자원에 따른 언어의 분류

한국어는 아직 애석하게도 중자원 언어에 속한다. 인구 뿐만 아니라 경제력과 언어 사용자들의 언어 사용 및 정보 생산 의지에 따라 좌우된다.

* 초고자원 언어(extremely high-resource language)
	+ 영어
* 고자원 언어(high-resource language)
	+ 스페인어, 중국어, 일본어, 독일어, 프랑스어
* 중자원 언어(mid-resource language)
	+ 한국어
* 저자원 언어(low-resource language)
* 초저자원 언어(extremely low-resource language)
	+ 정확한 기준은 없으나 사용인구가 극히 적거나, 소멸위기언어인 언어를 들 수 있다. 보존 노력이 활발해서 정보가 많이 생긴다면 저자원 언어인 것으로 볼 수도 있다.
수학과학 위키

수학과학 위키는 수학과 과학에 대한 다루는 내용을 다루는 위키이나, 현재는 기계학습과 인공지능, 딥러닝, 계산수학에 초점을 맞추고 있다. 언어는 한국어를 사용한다.

**Welcome to the 수학과학 위키!**
We're a collaborative community website about 수학과학 위키 that anyone, including you, can build and expand. Wikis like this one depend on readers getting involved and adding content. Click the "ADD NEW PAGE" or "EDIT" button at the top of any page to get started! We have 340 articles.

## Important articles

PlaceholderA page about your topicPlaceholderA Main CharacterPlaceholderThe First EpisodePlaceholderAn Important LocationPlaceholderA Key EventPlaceholderA Crucial Item

*Need help building out this community?*

* Rules of this wiki
* Getting Started
* How to Contribute
* Managing your new community
* Guides
* All Help articles

You can also be part of the larger Fandom family of communities. Visit Fandom's Community Central!

*Community Founders*: Write a good and paragraph-length description for your welcome section about your topic. Let your readers know what your topic is about and add some general information about it. Then you should visit the admin dashboard for more tips.
StyleGAN

StyleGAN(스타일간)은 NVIDIA에서 만든 생성적 적대 신경망(GAN)이다.

나이나 헤어스타일, 머리색, 성별 같은 스타일을 조정할 수 있게 한 논문이다.

StyleGAN의 기반은 PGGAN이며, PGGAN은 Progressive Growing of GANs for Improved Quality, Stability, and Variation(틀:ArXic) 논문에서 발표되었다.

해상도를 높이는 Interpolation(보간법) 성능과 Disentanglement(디스인탱글먼트, 얽힘풀기) 성능을 높이는데 초점을 두었다.

생성 모델에서 얼굴 사진의 나이를 젊게 바꾸면 의도치 않게 머리카락이 길게 변하는 경우, 특징이 충분히 분리되지 않아서 Entanglement되어있다고 하고, 나이에 관계 없이 머리카락 길이를 조절할 수 있는 경우 Disentanglement 되어 있다고 한다.

논문
--

* A Style-Based Generator Architecture for Generative Adversarial Networks(arXiv:1812.04948) : StyleGAN code
* Analyzing and Improving the Image Quality of StyleGAN(arXiv:1912.04958) : StyleGAN2 code
* Training Generative Adversarial Networks with Limited Data(arXiv:2006.06676) : StyleGAN2-ADA code
지식 증류

지식 증류(knowledge distillation)는 선생 모델(teacher model)의 지식을 학생 모델(student model)로 옮기는 기술이다. 높은 온도를 이용해 지식을 추출한다는 뜻에서 증류와 비슷하여 지식 증류라는 이름이 붙었다.

설명
--

지식 증류 이전에 모델 사이즈를 줄이는 기법은 모델 압축(model compression)이 이미 있었지만 한계가 있었다.

제프리 힌턴은 2014년 NIPS에서 지식 증류 기법을 제안해  2015년에 논문으로 발표했다. Distilling the knowledge in a neural network(arXiv:1503.02531)에서 정확하지 않은 분류에 대한 확률값이더라도, 서로 다른 class 간의 예측 확률 차이는 유용한 정보인데, 기존의 방법들은 큰 모델이 학습한 class가 맞는지에 치중해, 정보의 손실이 일어났다고 제안했다.

이렇게 예측한 class가 아닌 class의 확률이 거의 없는 것을 hard target이라고 하고, 다른 class에 대해서도 확률이나 결과값이 0에서 멀게 어느정도 부드럽게 한 것을 soft target이라고 한다.

힌턴은 대부분의 분류기가 사용하는 softmax layer를 수정하여, 온도 T에 대해 조절할 수 있도록 했다. 온도가 높을 수록 정보 엔트로피가 커서 추출할 수 있는 지식이 많다고 제안했다. T가 1인 경우 일반적인 softmax에 해당된다.

q

i

=

e
x
p
(

z

i

/

T
)

∑

j

e
x
p
(

z

j

/

T
)

{\displaystyle q\_{i}={\frac {exp(z\_{i}/T)}{\sum \_{j}exp(z\_{j}/T)}}}

지식 증류에서는 T가 1보다 커서 원래의 softmax보다 확률이 작은 class의 확률이 커지는 결과값을 soft target으로 사용한다.

이를 통해 온도를 조절하면 같은 입력값에 대한 softmax 결과값이 달라지게 된다.

첫 번째 목적 함수는 온도 T로 고정해놓고, 선생 모델과 학생 모델의 온도 T에서의 soft output 사이의 교차 엔트로피(cross entropy)를 증류 로스(distillation loss), 두 번째 목적 함수는 온도 1에서의 학생 모델이 만든 결과값과 진짜 label 사이의 교차 엔트로피(cross entropy)로 학생 로스(student loss)라고 부른다. 두 로스의 합을 최소화하는 방식으로 훈련하면 기본적인 지식 증류가 끝난다.

확장
--

원래의 지식 증류는 logit을 이용한 증류라는 점에서 로짓 증류(logit distillation)라고도 부르고, 증류 로스를 계산하는 방법 등을 바꾼 변종이 나왔다.

또한 선생 모델과 학생 모델 사이에 특징(feature) 정보도 전할 수 있는 특징 증류(feature distillation) 기법으로 확장되었다.

스스로 학생이 자신의 선생이 되는 자기 지식 증류(self-knowledge distillation) 기법으로도 확장되었다.

분류
--

Knowledge Distillation: A Survey(arXiv:2006.05525) 논문과 Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation(arXiv:2306.10687) 등에서는 로짓, 특징 기반으로 나누는 2분법 대신 3가지 기준으로 지식 증류를 나눈다.

* 응답 기반 지식 증류(Response-Based Knowledge Distillation): 로짓 증류
* 특징 기반 지식 증류(Feature-Based Knowledge Distillation): 특징 증류
* 관계 기반 지식 증류(Relation-Based Knowledge Distillation)
	+ 관계 기반 샘플 간 지식 증류(Relation-based Cross-Sample Knowledge Distillation)
	+ 관계 기반 레이어 간 지식 증류(Relation-based Cross-Layer Knowledge Distillation)

관계 기반 지식 증류는 다시 여러 인스턴스의 출력의 관계를 선생과 학생 사이에서 비슷하게 만드는 것과, 같은 인스턴스 내에서 레이어간 출력의 차이를 선생과 학생 사이에서 비슷하게 만드는 것 2가지 종류로 나뉜다.

## 주요 논문

* Distilling the knowledge in a neural network(arXiv:1503.02531)
* FitNets: Hints for Thin Deep Nets(arXiv:1412.6550) - 특징 증류 기법의 시작이 되는 논문이다.
* Similarity-Preserving Knowledge Distillation(arXiv:1907.09682)
* Knowledge distillation: A good teacher is patient and consistent(arXiv:2106.05237)
* Decoupled Knowledge Distillation(arXiv:2203.08679)
* On Distillation of Guided Diffusion Models(arXiv:2210.03142): 확산 모델 용 지식 증류에 대한 논문

### 리뷰 논문

* Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks(arXiv:2004.05937)
* Knowledge Distillation: A Survey(arXiv:2006.05525

## 같이 보기

* DeiT
* 모델 압축
* 전이학습

1. ↑ Distilling Knowledge in a Neural Network, Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean, Deep Learning and Representation Learning Workshop, NIPS 2014
EfficientNet: Rethinking Model Scaling for Convolutional Neural Network

EfficientNet: Rethinking Model Scaling for Convolutional Neural Network(arXiv:1905.11946)은 EfficientNet](에피션트넷)이라는 합성곱 신경망을 발표한 논문이다. 에피션트넷은 매개변수(parameter)로도 효율적인 네트워크를 만들고, 연산자원의 크기에 따라 모델 규모를 적절한 비율로 키울 수 있도록 한 합성곱 신경망이다.

구조
--

MobileNetV2과 MnasNet에서 제안된 mobile inverted bottleneck (MBConv)으로 네트워크를 쌓고

SENet의 Squeeze and excitation 을 넣어서 네트워크를 만들었다.

### MBConv

ResNet의 bottleneck에서는

1x1 합성곱으로 채널을 줄인 뒤 3x3 합성곱을 하고 다시 1x1 합성곱 연산으로 채널을 늘이고, 3개의 합성곱을 건너뛰어 시작과 뒤를 연결하는 skip connection이 있는 Bottleneck block을 만든다

그러나 MBConv에서는 풍부한 채널에 대해 3x3 conv를 해야 성능이 잘 나온다는 철학에 따라 채널 수를 늘린뒤 3x3 conv를 하고 다시 채널수를 줄이되, 연산량을 줄이기 위해 3x3 conv는 깊이별 합성곱(depthwise convolution)을 적용한다.

### 활성화 함수

ReLU6이나 ReLU를 사용한 종래의 MBConv 기반 네트워크와 달리 SiLU (Swish-1)를 적용했다. Swish는 ReLU와 달리 입력이 항상 미분 가능하고, 입력이 음수가 되면서 감소하면 처음에는 출력도 감소하다가 최소값에 이른 뒤, 천천히 출력이 0으로 수렴한다.

### 스케일링

기존에 모델 구조를 설계할때는 한 모델 내부에서 레이어별로 채널수와 width를 어떻게 조절 및 배분할지에 관심을 기울였으나, EfficientNet은 모델 시리즈내에서 동시에 채널수와 width를 조절하는 Model Scaling에 관심을 기울였다.

모델은 깊이(depth, d), 너비(width, w), 해상도(reolution, r)이라는 3가지 지표에 의해 표시된다.

이때 큰 이미지를 사용할 수록 네트워크의 깊이와 너비를 늘려야 더 차원수가 많아진 특징을 잘 분석할 수 있다. 이를 위해서 깊이와 너비, 해상도를 동시에 조절하는 Compound Model Scaling의 중요성을 지적했다.

상수 φ를 조절해 깊이 d =αφ, 너비 w = βφ, 해상도 r = γφ을 만족하도록 하는 compound scaling method를 적용했다.

α · β2 · γ2 ≈ 2를 만족하도록 했으며 α = 1.2, β = 1.1, γ = 1.15로 잡아주었다.

최적 해상도는 논문에 직접 나오지 않지만, keras 홈페이지 페이지에 나온다.

실험을 통해 EfficientNet이 아닌 MobileNet 시리즈와 ResNet에 대해서도 스케일링이 중요함을 증명하였다.

실험
--

ImageNet에 대해 ResNet, DenseNet, Inception, Xception, Inception-resnet, PolyNet, ResNeXt, SENet, NASNet, AmoebaNet, PNASNet과 분류 성능을 비교했다.

## 같이 보기

* EfficientNetV2: Smaller Models and Faster Training(arXiv:2104.00298): EfficientNetV2라는 개량 버전을 발표한 논문.
* EfficientDet
* MobileNet
* SENet
* GPipe
* Fast and Accurate Model Scaling(arXiv:2103.06877)
* MnasNet: Platform-Aware Neural Architecture Search for Mobile(arXiv:1807.11626): MnasNet
RegNet

RegNet(regular network)은 Designing Network Design Spaces(arXiv:2003.13678)에서 제안한 합성곱 신경망이다.

개

RegNet은 ResNet의 Bottleneck을 변형하여 만든 **X 블록**(X block)을 바탕으로 하여,

매 block 마다를 기준으로는 block width가 양자화된 선형으로 증가하고, 각 스테이지 별 block width는 지수함수적으로 증가하도록 한 규칙적인 네트워크이다.

블록 갯수와 같은 길이를 가지는 증가하는 등차수열을, 같은 시작값에서 시작하는 등비수열에 속하는 값중 (log scale로 가장) 가까운 값으로 양자화하여 같은 값으로 양자화된 block끼리 한 스테이지에 배정하고, 양자화된 값을 바탕으로 block width를 결정하는 방식으로 구성된다.

이를 통해 블록 갯수 d, 수열의 초기값 w0, 등차수열의 공차 wa, 등비수열의 공비 wm, 그룹별 합성곱에 사용할 group width g, bottleneck ratio b 이렇게 6개의 자유도를 가지는 네트워크를 만들었다.

구조
--

X block은 stride에 따라 2가지 종류가 있다.

stride가 2인 X block은 각 stage의 첫부분에 배치되며, bottleneck부분의 3x3 합성곱과 skip connection에서 strider 2 연산을 적용해 사이즈를 반으로 줄인다.

stride가 1인 X block은 각 stage의 다음 부분에 배치된다. stride에 관계 없이, bottleneck부분에는 group convolution을 적용한다.

그 뒤, 블록의 총 갯수, 각 스테이지별 블록 갯수, block width등을

number of blocks d가 주어지면,
block width와 각 블록이 어떤 스테이지에 속하는지를, 등차수열을 이산화하는 방식으로 결정한다.

i번째 블록의 width wi와 i번째 블록이 어떤 스테이지에 속하는지를 결정하기 위해, 0≤j<d인 j에 대해, uj를

u
j

=

w
0

w
a

⋅
j

{\displaystyle u\_j=w\_0+w\_a\cdot j}

의 식에 따라 정한다. 이때 w0과 w a는 양수이다.

양수 매개변수 wm을 도입해

u
j

=

w
0

⋅

w
m

s
j

{\displaystyle u\_j=w\_0\cdot w\_m^{s\_j}}

가 되는 sj를 찾는다. 그 뒤 sj를 반올림한

⌊

s
j

⌉

{\displaystyle \lfloor s\_j \rceil}

를 j번째 블록의 스테이지로,

w
j

=

w
0

⋅

w
m

⌊

s
j

⌉

{\displaystyle w\_j=w\_0\cdot w\_m^{\lfloor s\_j \rceil}}

을 양자화된 block width로 삼는다.

스테이지 갯수는 4개여야 하기 때문에 그 외의 조합은 고려하지 않고, 그룹별 합성곱에 사용할 group width g와 bottleneck ratio b를 더하면 네트워크가 완성된다.

group width는 group의 갯수와 다르며, 깊이별 합성곱(depth-wise convolution)은 group width가 1이다.

만약 잘못하여 wi가 gi의 배수가 아닌경우, g가 더 큰 경우 g를 줄여서 맞추고, 그렇지 않은 경우 w를 반올림해서 g의 배수로 만들어준다.

이렇게 X block으로 이뤄진 RegNet은 RegNetX이고, SENet의 Squeeze and excitation 을 넣은 것은 RegNetY이다.
The War of Stone

The war was caused because Comet rays and Wing'Dusts couldnt respect boundaries (basically)

War was waged first by the Comet Rays by Corronett, Queen of (??), The wing'dusts tried to keep peace by saying that if they let them use the hatchery of the Comet rays they would stop trespassing, The comet rays refused wich led to the Hatchery Massacre, where the hatchery was attacked, they had crushed almost all the eggs, leaving the Comet rays without young ones for two thousand years (2,000), leading Queen Willow to lead the Flame den attack, almost extinction the once rich population of the Wing'dusts, as of currently, there are rarely any sightings of them, a few have theorized that there are around 50 or 10 left.
SKD

SKD의 뜻은 다음과 같다.

* Self-supervised Knowledge Distillation for Few-shot Learning(arXiv:2006.09785)
* Structured Knowledge Distillation for Semantic Segmentation: CVPR 19 논문
* Structured Knowledge Distillation for Dense Prediction(arXiv:1903.04197) : 위 CVPR 19 논문의 확장

SKD에 대한 명확화 문서아 문서는 같은 제목을 공유하는 문서에 대한 명확화 문서입니다.Please follow one of the disambiguation links below or search to find the page you were looking for if it is not listed. If an internal link led you here, you may wish to change the link to point directly to the intended article.
DeiT

DeiT(Data-efficient Image Transformers)는 FaceBook AI에서 내놓은 논문인 Training data-efficient image transformers & distillation through attention(arXiv:2012.12877)에서 소개된 비전 변환기이다.

코드

구조
--

* 구조는 ViT과 비슷하지만, 맨뒤에 지식 증류(knowledge distillation)를 위해 증류 토큰(distillation token)을 추가한다.
* 이를 통해 실제로 클래스를 예측함과 함께, 선생 모델이라면 어떻게 생각했을지를 예측한다.
* 지식 증류를 통해 합성곱 신경망 등 선생 모델의 귀납적 편향(inductive bias)을 전달받는다.
* 원 논문에서는 RegNet을 선생 모델로 사용했다.
CSP

* 콘텐츠 보안 정책(Content Security Policy)
* 제약충족문제(Constraint satisfaction problem)

## 같이 보기

* CSPNet

CSP에 대한 명확화 문서아 문서는 같은 제목을 공유하는 문서에 대한 명확화 문서입니다.Please follow one of the disambiguation links below or search to find the page you were looking for if it is not listed. If an internal link led you here, you may wish to change the link to point directly to the intended article.
DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification

DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification(arXiv:2203.12081)은 다중 인스턴스 학습(multiple instance
learning, MIL)에 특징 증류를 적용시켜서 만든 논문이다.

디지털 병리학에서 전체 슬라이드 이미지(whole slide image)를 분석하는 것이 어려웠고 그 해결 방법은 다중 인스턴스 학습이라고 말했다.

CAMELYON16이라는 림프절에 전이된 유방암을 진단하는 멀티 인스턴스 학습 방식의 챌린지 대회에서 좋은 성적을 거두었다.
영역 기반 합성곱 신경망

영역 기반 합성곱 신경망(Region Based Convolutional Neural Networks)은 R-CNN으로부터 시작된 2단계 검출기(Two-stage detector) 방식의 객체 탐지용 합성곱 신경망이다.

종류
--

* R-CNN
* Fast R-CNN
* Faster R-CNN
* MASK R-CNN
* Cascade R-CNN
* Libra R-CNN
* Grid R-CNN

## 주요 기술

* Selective Search for Object Recognition : 첫 버전에서 사용한 기술
* Region Proposal Network
* Feature Pyramid Network
Relational Knowledge Distillation

Relational Knowledge Distillation(arXiv:1904.05068)은 특징이나 로짓 출력값의 상대적인 관계가 선생 모델과 학생 모델에서 보존되는데 초점을 맞춘 관계 기반 지식 증류 기법에 대한 논문이다. 기존의 지식 증류 기법은 지식 증류든 특징 증류든 선생의 출력과 학생의 출력 사이의 거리를 최소화하는데 초점을 두었지만, 발상을 전환한 논문이다.

손실 함수로는 Huber loss를 사용했다.

## 같이 보기

* Similarity-Preserving Knowledge Distillation(arXiv:1907.09682) : 다른 방식으로 선생 모델과 학생 모델의 출력이 반드시 같을 필요는 없는 관계 기반 지식 증류 체계를 만든 논문.

## 외부 링크

* 논문 페이지
* github 페이지