Object Container Streaming

Object Container Streaming (OCS) is the umbrella term for all the technology in Star Engine that makes a vast seamless universe possible, by which Star Citizen (and to an extent Squadron 42) can provide an extremely large virtual world through which the players can move without seeing a loading screen.
==Related technology==
=== Server-Side Object Container Streaming ===
Server-Side Object Container Streaming (SOCS) is a streaming technology needed to expand the scale of the universe on Star Citizen servers.Star Citizen: Pillar Talk - Alpha 3.9 The basic concept is that if no player is near an object, the game can "freeze" that object's state. And instead of keeping the frozen entity in memory (incurring a cost), the game can serialize (using Serialized Variables) its state and store the serialized state in a database. While a client is moving through the virtual world, the server updates its view into the database to restore entities now in proximity as well as free and store away no-longer-needed entities.
==== Entity Streaming Manager and StarHash ====
StarHash is used to store in-game entities in a way allowing efficient searches for all entities in an area of space by utilizing a data structure called a RadixTree. The Entity-StreamingManager is then the logic-driving the StarHash-RadixTree queries to trigger loading and unloading of entities on the server, based on the positions of all connected clients.
==== Cross Session Persistence ====
Cross Session Persistence allows the game to store the entity in a frozen state in an in-process database that can be used by different server or machine. It is an efficient network access layer to allow the storing of the entity in a database on a different machine. Before Cross Session Persistence, the state of an entity is lost when the server crashes or is restarted (besides the state the game already persist). With Cross Session Persistence, object state will persist over server restarts and crashes (until the persistence database is wiped).
===== I-cache =====
I-cache is a back-end system and service that provides and manages reliable persistence of all player and entity data. First major part of I-cache was completed and tested internally in March 2019. It was introduced as a replacement of the existing pCache. It is a highly distributed and fault-tolerant storage/query engine that greatly out-performs pCache. It provides an indexing and query system that can be utilized by other services for specific and complex item queries.
==== 服务器网格化 ====
Server Meshing is a technology that allows the game instance to utilize multiple servers. Instead of having a single server manage all the views, the game will distribute the individual views over multiple servers. Doing this will reduce the load on each participating server. When those servers are placed on different machines on Amazon Web Services (AWS), the game can easily scale with the player count.
=== Network Bind Culling ===
Network Bind Culling is the process of deciding what entities to load and unload on any client. Specifically, it works on units of Entity Aggregates, using the information of the Entity Component Update Scheduler to decide what Entity Aggregates to load or unload on each client.
Network Bind Culling was first mentioned in November 2016 as entity bind culling.
==== Entity Component Update Scheduler ====
Entity Component Update Scheduler is a system designed to control the update of entity components, based on how they are spatially placed relative to the player. It allows the game to skip updates of individual components which are too far away. The same information is utilized by Network Bind Culling.
It was first mentioned in April 2018 as being refactored and nearly finished.
==== Entity Ownership Hierarchy and Entity Aggregates ====
Entity Ownership Hierarchy keeps track of entities that are related to each other. If they are related, the game treats them as one group, or Entity Aggregate. Compared to Object Containers that split the static level geometry and objects into building blocks at level design time, Entity Aggregates are consisted of dynamic entities (e.g. players, ships, etc.) that can move around in the universe.
Entity Ownership Hierarchy was first mentioned in March 2017 with the development of Entity Owner Manager.
==== Entity Spawn Batches and Entity Snapshots ====
Entity Spawn Batches represent a set of entities that should be spawned together and only made active when all spawned. Entity Snapshots are the values of the Serialized Variables belonging to the entity. They are also used for serializing entity state to persistence, or for Squadron 42 save games.
==== Serialized Variable Culling ====
Serialized Variable Culling is a system that would only send network updates to clients if that client is in proximity of the updated Entity-Aggregate. It provides a noticeable performance improvement on the clients. Additionally, it was the first real-life test of running our client game-simulation with only partial information of the whole universe.
It was originally planned for Alpha 3.0, but was delayed to Alpha 3.1.
===Groundwork technology===
====Zone System====
Zone System is a spatial partitioning system that divides all objects in a level into smaller entities. It is required for moving large, complex, and interactive groups of physicalized objects efficiently. It replaces the old CryEngine octree spatial partitioning scheme.
It was proposed in January 2015 in a tech summit with tech engineers from the CIG studios, Illfonic, Behaviour Interactive, and Wormbyte. The development began in May 2015 by the Frankfurt team. It was deployed to the Star Citizen code mainline in June 2015. It began to be integrated into various game systems by the UK, Frankfurt, and Santa Monica studios in July 2015, with Retaliator (used as the multi-crew testbed) and the Cover System were the first system being integrated. Navigation mesh and characters' paths began to be integrated in August 2015. The Frankfurt team extended the Zone System to better support multiple AI objects in October 2015. The Frankfurt team rewrote CryEngine's AreaManager (ambient sound effects and reverbs) to integrate with the Zone System in November 2015. The UK team began work on getting Object Containers to work with the Zone System in January 2016. Hierarchical culling support was added in March 2016. Support for tags (Tag System) was moved into the Zone System in May 2016. All data in Zone System are converted into AABB tree structure in June 2016. The Tag System went through a major refactoring and optimization pass in July 2016. The Area system was moved into the Zone System in November 2016.
====C++ entity logic====
C++ is a multi-thread safe scripting language which is used heavily for all kinds of entity logic. It allows resource loading in parallel to the game simulation without introducing very long wait times due to required mutual exclusion, in comparison to the old Lua code which is not multi-thread safe.
It is unknown when the conversion started, the first mention of the conversion was that the spawning logic of spaceships AI was converted from Lua to C++ in January 2016. It was mentioned again as a work in progress in April 2016. It was mentioned as being worked on by the UK team in January 2018.
====Entity component====
Entity component represents a part of specific game behaviour. With components, the behaviour of an entity is defined by the types of components it has. Without components, all kinds of different logic tends to be interleaved in one monolithic and very complex central logic block. Since they are smaller parts, it is much simpler to make them communicate efficiently with the game simulation while we load them concurrently. Additionally, they split the monolithic game logic into more manageable parts, which played a critical role in allowing a partial roll-out of concurrent entity initialization. 
Basic support of entity component was implemented in the main development branch in July 2015.
====Object Container====
Object Container is a level building block. It replaces the old game level format. Instead of developing one large level, the content creators develop a small section. The final level (or universe) is then made out of many different Object Containers. It allows the developers to split the world into many smaller building blocks when they are building a level.
The concept and first prototype of Object Container was introduced in January 2016. The UK team began to convert prefabs into containers in the same month. Basic support of loading levels with Object Container instead of as levels was implemented by the Frankfurt team in the development branch in February 2016. Support for LOD mesh merging and AI navigation meshes were added in August 2016. The old prefab system in hangars and shops were being replaced with an Object Container in February 2017.
==== Serialized Variables ====
Serialized Variables is a process to store entities state so that they can be transferred on the network and restored in the same state on a different machine. It take the parts of an entity and put it into a special wrapper object, and the wrapper object provides ways to serialize the entity state. It allows game code to be written in a uniform style, regardless of the serialized data will be transferred later. Compared to the old implementation in CryNetwork, the gameplay programmer only need to mark which variables they want to replicate to the server/clients. The underlying system can detect which have changed since the last time they were sent, and which haven't, and takes care of the rest in a nice and efficient manner.
The concept of Serialized Variables was introduced in June 2016, with the development started in the same month.
==== Mega Map ====
Mega Map is an empty map used to load and unload Object Containers. It is used to reduce the load time, and eliminate the need of load screens.Star Citizen: Around the Verse - What is Mega Map?
The concept of Mega Map was introduced to reduce load time of switching game modes in October 2016. Support for singleplayer game maps such as hangars and single player Arena Commander was implemented in Alpha 2.6.1 in February 2017. Support for multiplayer maps such as Arena Commander and Star Marine was implemented in Alpha 2.6.2 in March 2017.
==Development history==
2015-01	
*Zone System and entity streaming were proposed
2015-05	
*Zone System was being developed, and entity streaming was being investigated
2015-06	
*Deployed Zone System to Star Citizen main development branch
2015-07	
*Started implementing Zone System in the main development branch
*Added basic support of the entity component system in the main development branch
2016-01	
*Introduced the concept of Object Container
*Started converting prefabs into containers
*Started getting containers working with the Zone System
*Converted Spawning logic of spaceships AI from C++ to Lua
2016-02	
*Added basic support to load levels with Object Container instead of as levels
2016-03	
*Added Hierarchical culling support to Zone System
2016-05	
*Added support for tags (Tag System) into Zone System
*Added ability to reuse entity IDs
2016-06	
*Started work on Serialized Variables
*Converted data structure to AABB tree in Zone System
2016-07	
*Major refactor and optimization of Tag System in Zone System
2016-08	
*Added Object Container support for LOD mesh merging and AI navigation meshes
2016-10	
*Mega Map was being investigated
2016-11	
*Moved Area system was moved into the Zone System
*Started work on Network Bind Culling
2017-02	
*Implemented singleplayer maps support for Mega Map in Alpha 2.6.1
*Started work on replacing old prefab system in hangars and shops with Object Containers
2017-03	
*Implemented multiplayer maps (not including PU) support for Mega Map in Alpha 2.6.2
*Started work on Entity Owner Manager
2018-04	
* Refactored Entity Component Update Scheduler
==See also==
* by unobtanium on Spectrum
* "Developer Feature: The Road to Object-Container-Streaming". Jump Point. Vol. 7 no. 10. pp. 3–16
* Star Citizen's Next-Gen Tech In-Depth: World Generation, Galactic Scaling + More! by Digital Foundry
* Star Citizen: Pillar Talk - Alpha 3.9
* Star Citizen: Pillar Talk - Alpha 3.8
* 
* Star Citizen: Around the Verse - Performance and Optimization 
* Star Citizen: Reverse the Verse LIVE - Object Container Streaming 
* Star Citizen: Around the Verse - What is Mega Map? 
* 
* Star Citizen 3.3 | OBJECT CONTAINER STREAMING - Major Core Update by BoredGamer
==References==
Category:Development