Project: MongoDB (App Definition and Development / Database)
File: README_sessions_and_transactions.md

Some operations, such as retryable writes and transactions, require durably storing metadata in the cluster about the operation. However, it's important that this metadata does not remain in the cluster forever. Logical sessions provide a way to durably store metadata for the latest operation in a sequence of operations. The metadata is reaped if the cluster does not receive a new operation under the logical session for a reasonably long time (the default is 30 minutes). A logical session is identified by its "logical session id," or `lsid`. An `lsid` is a combination of up to four pieces of information: `id` - A globally unique id (UUID) generated by the mongo shell, driver, or the `startSession` server command `uid` (user id) - The identification information for the logged-in user (if authentication is enabled) `txnNumber` - An optional parameter set only for internal transactions spawned from retryable writes. Strictly-increasing counter set by the transaction API to match the txnNumber of the corresponding retryable write. `txnUUID` - An optional parameter set only for internal transactions spawned inside client sessions. The txnUUID is a globally unique id generated by the transaction API. A logical session with a `txnNumber` and `txnUUID` is considered a child of the session with matching `id` and `uid` values. There may be multiple child sessions per parent session, and checking out a child/parents session checks out the other and updates the `lastUsedTime` of both. Killing a parent session also kills all of its child sessions. The order of operations in the logical session that need to durably store metadata is defined by an integer counter, called the `txnNumber`. When the cluster receives a retryable write or transaction with a higher `txnNumber` than the previous known `txnNumber`, the cluster overwrites the previous metadata with the metadata for the new operation. Operations sent with an `lsid` that do not need to durably store metadata simply bump the time at which the session's metadata expires. The logical session cache is an in-memory cache of sessions that are open and in use on a certain node. Each node (router, shard, config server) has its own in-memory cache. A cache entry contains: `_id` - The sessions logical session id `user` - The sessions logged-in username (if authentication is enabled) `lastUse` - The date and time that the session was last used The in-memory cache periodically persists entries to the `config.system.sessions` collection, known as the "sessions collection." The sessions collection has different placement behavior based on whether the user is running a standalone node, a replica set, or a sharded cluster. | Cluster Type | Sessions Collection Durable Storage | | | - | | Standalone Node | Sessions collection exists on the same node as the in-memory cache. | | Replica Set | Sessions collection exists on the primary node and replicates to secondaries. | | Sharded Cluster | Sessions collection is a regular sharded collection - can exist on multiple shards and can have multiple chunks. | There is a TTL index on the `lastUse` field in the sessions collection. The TTL expiration date is thirty (30) minutes out by default, but is user-configurable. This means that if no requests come in that use a session for thirty minutes, the TTL index will remove the session from the sessions

When the logical session cache performs its periodic refresh (defined below), it will find all sessions that currently exist in the cache that no longer exist in the sessions collection. This is the set of sessions that we consider "expired". The expired sessions are then removed from the in-memory cache. When a node receives a request with attached session info, it will place that session into the logical session cache. If a request corresponds to a session that already exists in the cache, the cache will update the cache entry's `lastUse` field to the current date and time. At a regular interval of five (5) minutes (user-configurable), the logical session cache will sync with the sessions collection. Inside the class, this is known as the "refresh" function. There are four steps to this process: All sessions that have been used on this node since the last refresh will be upserted to the sessions collection. This means that sessions that already exist in the sessions collection will just have their `lastUse` fields updated. All sessions that have been ended in the cache on this node (via the endSessions command) will be removed from the sessions collection. Sessions that have expired from the sessions collection will be removed from the logical session cache on this node. All cursors registered on this node that match sessions that have been ended (step 2) or were expired (step 3) will be killed. The logical session cache class holds the periodic job to clean up the and . Inside the class, this is known as the "reap" function. Every five (5) minutes (user-configurable), the following steps will be performed: Find all sessions in the session catalog that were last checked out more than thirty minutes ago (default session expiration time). For each session gathered in step 1, if the session no longer exists in the sessions collection (i.e. the session has expired or was explicitly ended), remove the session from the session catalog. Find all entries in the transactions table that have a last-write date of more than thirty minutes ago (default session expiration time). For each entry gathered in step 3, if the session no longer exists in the sessions collection (i.e. the session has expired or was explicitly ended), remove the entry from the transactions table. | Parameter | Value Type | Default Value | Startup/Runtime | Description | | | - | -- | | -- | | | boolean | false | Startup | Disables the logical session cache's periodic "refresh" and "reap" functions on this node. Recommended for testing only. | | | integer | 300000ms (5 minutes) | Startup | Changes how often the logical session cache runs its periodic "refresh" and "reap" functions on this node. | | | integer | 30 minutes | Startup | Changes the TTL index timeout for the sessions collection. In sharded clusters, this parameter is supported only on the config server. | The logical session catalog of a mongod or mongos is an in-memory catalog that stores the runtime state for sessions with transactions or retryable writes on that node. The runtime state of each session is maintained by the session checkout mechanism, which also serves to serialize client operations on the session. This mechanism requires every operation with an `lsid` and a `txnNumber`

transaction and retryable write) to check out its session from the session catalog prior to execution, and to check the session back in upon completion. When a session is checked out, it remains unavailable until it is checked back in, forcing other operations to wait for the ongoing operation to complete or yield the session. Checking out an internal/child session additionally checks out its parent session (the session with the same `id` and `uid` value in the lsid, but without a `txnNumber` or `txnUUID` value), and vice versa. The runtime state for a session consists of the last checkout time and operation, the number of operations waiting to check out the session, and the number of kills requested. Retryable internal sessions are reaped from the logical session catalog , meaning that if a transaction session with a higher transaction number has successfully started, sessions with lower txnNumbers are removed from the session catalog and inserted into an in-memory buffer by the until a configurable threshold is met (1000 by default), after which they are deleted from the transactions table (`config.transactions`) and `config.image_collection` all at once. Eager reaping is best-effort, in that the in-memory buffer is cleared on stepdown or restart. The last checkout time is used by the to determine when a session should be reaped from the session catalog, whereas the number of operations waiting to check out a session is used to block reaping of sessions that are still in use. The last checkout operation is used to determine the operation to kill when a session is killed, whereas the number of kills requested is used to make sure that sessions are only killed on the first kill request. The runtime state in a node's in-memory session catalog is made durable in the node's `config.transactions` collection, also called its transactions table. The in-memory session catalog is if the `config.transactions` collection is dropped and whenever there is a rollback. When invalidation occurs, all active sessions are killed, and the in-memory transaction state is marked as invalid to force it to be . How and check out a session prior to executing a command. Retryable writes allow drivers to automatically retry non-idempotent write commands on network errors or failovers. They are supported in logical sessions with `retryableWrites` enabled (default), with the caveat that the writes are executed with write concern `w` greater than 0 and outside of transactions. is a complete list of retryable write commands. When a command is executed as a retryable write, it is sent from the driver with `lsid` and `txnNumber` attached. After that, all write operations inside the command are assigned a unique integer statement id `stmtId` by the mongos or mongod that executes the command. In other words, each write operation inside a batch write command is given its own `stmtId` and is individually retryable. The `lsid`, `txnNumber`, and `stmtId` constitute a unique identifier for a retryable write operation. This unique identifier enables a primary mongod to track and record its progress for a retryable write command using the `config.transactions` collection and augmented oplog entries. The oplog entry for a retryable write operation is written with a number of additional fields including `lsid`, `txnNumber`, `stmtId` and `prevOpTime`, where `prevOpTime` is the opTime of the write that precedes

In certain cases, such as time-series inserts, a single oplog entry may encode multiple client writes, and thus may contain an array value for `stmtId` rather than the more typical single value. All of this results in a chain of write history that can be used to reconstruct the result of writes that have already executed. After generating the oplog entry for a retryable write operation, a primary mongod performs an upsert into `config.transactions` to write a document containing the `lsid` (`_id`), `txnNumber`, `stmtId` and `lastWriteOpTime`, where `lastWriteOpTime` is the opTime of the newly generated oplog entry. The `config.transactions` collection is indexed by `_id` so this document is replaced every time there is a new retryable write command (or transaction) on the session. The opTimes for all committed statements for the latest retryable write command is cached in an that gets after each write oplog entry is generated, and gets cleared every time a new retryable write command starts. Prior to executing a retryable write operation, a primary mongod first checks to see if it has the commit opTime for the `stmtId` of that write. If it does, the write operation is skipped and a response is constructed immediately based on the oplog entry with that opTime. Otherwise, the write operation is performed with the additional bookkeeping as described above. This in-memory cache of opTimes for committed statements is invalidated along with the entire in-memory transaction state whenever the `config.transactions` is dropped and whenever there is rollback. The invalidated transaction state is overwritten by the on-disk transaction history at the next session checkout. To support retryability of writes across migrations, the session state for the migrated chunk is propagated from the donor shard to the recipient shard. After entering the chunk cloning step, the recipient shard repeatedly sends (also referred to as MigrateSession) commands to the donor shard until the migration reaches the commit phase to clone any oplog entries that contain session information for the migrated chunk. Upon receiving each response, the recipient shard writes the oplog entries to disk and its in-memory transaction state to restore the session state for the chunk. For most writes, persisting only the (lsid, txnId) pair alone is sufficient to reconstruct a response. For findAndModify however, we also need to respond with the document that would have originally been returned. In version 5.0 and earlier, the default behavior is to as a no-op entry. The oplog entries generated would look something like: `{ op: "d", o: {_id: 1}, ts: Timestamp(100, 2), preImageOpTime: Timestamp(100, 1), lsid: ..., txnNumber: ...}` `{ op: "n", o: {_id: 1, imageBeforeDelete: "foobar"}, ts: Timestamp(100, 1)}` There's a cost in "explicitly" replicating these images via the oplog. We've addressed this cost with 5.1 where the default is to instead with the namespace `config.image_collection`. A primary will add `needsRetryImage: <preImage/postImage>` to the oplog entry to communicate to secondaries that they must make a corollary write to `config.image_collection`. Note that this feature was backported to 4.0, 4.2, 4.4 and 5.0. Released binaries with this capability can be turned on by [setting the `storeFindAndModifyImagesInSideCollection` server parameter](https://github.com/mongodb/mongo/blob/2ac9fd6e613332f02636c6a7ec7f6cff4a8d05ab/src/mongo/db/repl/replserverparameters.idl#L506-L512). Partial cloning mechanisms such as chunk migrations, tenant migrations and resharding all support the destination picking up the responsibility for satisfying a retryable write the source had originally processed (to some degree). These cloning mechanisms naturally tail the oplog to pick up on

Because the traditional retryable findAndModify algorithm places the images into the oplog, the destination just needs to relink the timestamps for its oplog to support retryable findAndModify. For retry images saved in the image collection, the source will "downconvert" oplog entries with `needsRetryImage: true` into two oplog entries, simulating the old format. As chunk migrations use internal commands, is installed under the hood. For resharding and tenant migrations, a new aggregation stage, , was introduced to perform the identical substitution. In order for this stage to have a valid timestamp to assign to the forged no-op oplog entry as result of the "downconvert", we must always assign an extra oplog slot when writing the original retryable findAndModify oplog entry with `needsRetryImage: true`. In order to avoid certain WiredTiger constraints surrounding setting multiple timestamps in a single storage transaction, we must reserve oplog slots before entering the OpObserver, which is where we would normally create an oplog entry and assign it the next available timestamp. Here, we have a table that describes the different scenarios, along with the timestamps that are reserved and the oplog entries assigned to each of those timestamps: | Parameters | NumSlotsReserved | TS - 1 | TS | Oplog fields for entry with timestamp: TS | | | | | | | | Update, NeedsRetryImage=preImage | 2 | Reserved for forged no-op entry eventually used by tenant migrations/resharding|Update oplog entry|NeedsRetryImage: preImage | | Update, NeedsRetryImage=postImage | 2 | Reserved for forged no-op entry eventually used by tenant migrations/resharding|Update oplog entry | NeedsRetryImage: postImage | |Delete, NeedsRetryImage=preImage |2|Reserved for forged no-op entry eventually used by tenant migrations/resharding|Delete oplog entry|NeedsRetryImage: preImage| How a write operation How mongos How mongod Cross-shard transactions provide ACID guarantees for multi-statement operations that involve documents on multiple shards in a cluster. Similar to , cross-shard transactions are only supported in logical sessions. They have a configurable lifetime limit, and are automatically aborted when they are expired or when the session is killed. To run a cross-shard transaction, a client sends all statements, including the `commitTransaction` and `abortTransaction` command, to a single mongos with common `lsid` and `txnNumber` attached. The first statement is sent with `startTransaction: true` to indicate the start of a transaction. Once a transaction is started, it remains active until it is explicitly committed or aborted by the client, or unilaterally aborted by a participant shard, or overwritten by a transaction with a higher `txnNumber`. When a mongos executes a transaction, it is responsible for keeping track of all participant shards, and choosing a coordinator shard and a recovery shard for the transaction. In addition, if the transaction uses read concern `"snapshot"`, the mongos is also responsible for choosing a global read timestamp (i.e. `atClusterTime`) at the start of the transaction. The mongos will, by design, always choose the first participant shard as the coordinator shard, and the first shard that the transaction writes to as the recovery shard. Similarly, the global read timestamp will always be the logical clock time on the mongos when it receives the first statement for the transaction. If a participant shard cannot provide a snapshot at the chosen read timestamp, it will throw a snapshot error, which will trigger a client level retry of the transaction. The mongos will only keep this information in memory as it relies on the participant shards to persist their respective transaction states in their local `config.transactions`

The execution of a statement inside a cross-shard transaction works very similarly to that of a statement outside a transaction. One difference is that mongos attaches the transaction information (e.g. `lsid`, `txnNumber` and `coordinator`) in every statement it forwards to targeted shards. Additionally, the first statement to a participant shard is sent with `startTransaction: true` and `readConcern`, which contains the `atClusterTime` if the transaction uses read concern `"snapshot"`. When a participant shard receives a transaction statement with `coordinator: true` for the first time, it will infer that it has been chosen as the transaction coordinator and will set up in-memory state immediately to prepare for coordinating transaction commit. One other difference is that the response from each participant shard includes an additional `readOnly` flag which is set to true if the statement does not do a write on the shard. Mongos uses this to determine how a transaction should be committed or aborted, and to choose the recovery shard as described above. The id of the recovery shard is included in the `recoveryToken` in the response to the client. The commit procedure begins when a client sends a `commitTransaction` command to the mongos that the transaction runs on. The command is retryable as long as no new transaction has been started on the session and the session is still alive. The number of participant shards and the number of write shards determine the commit path for the transaction. If the number of participant shards is zero, the mongos skips the commit and returns immediately. If the number of participant shards is one, the mongos forwards `commitTransaction` directly to that shard. If the number of participant shards is greater than one: If the number of write shards is zero, the mongos forwards `commitTransaction` to each shard individually. Otherwise, the mongos sends `coordinateCommitTransaction` with the participant list to the coordinator shard to initiate two-phase commit. To recover the commit decision after the original mongos has become unreachable, the client can send `commitTransaction` along with the `recoveryToken` to a different mongos. This will not initiate committing the transaction, instead the mongos will send `coordinateCommitTransaction` with an empty participant list to the recovery shard to try to join the progress of the existing coordinator if any, and to retrieve the commit outcome for the transaction. The two-phase commit protocol consists of the prepare phase and the commit phase. To support recovery from failovers, a coordinator keeps a document inside the `config.transaction_coordinators` collection that contains information about the transaction it is trying commit. This document is deleted when the commit procedure finishes. Below are the steps in the two-phase commit protocol. Prepare Phase The coordinator writes the participant list to the `config.transaction_coordinators` document for the transaction, and waits for it to be majority committed. The coordinator sends to the participants, and waits for vote reponses. Each participant shard responds with a vote, marks the transaction as prepared, and updates the `config.transactions` document for the transaction. The coordinator writes the decision to the `config.transaction_coordinators` document and waits for it to be majority committed. If the `coordinateCommitTransactionReturnImmediatelyAfterPersistingDecision` server parameter is true (default), the `coordinateCommitTransaction` command returns immediately after waiting for client's write concern (i.e. let the remaining work continue in the background). Commit Phase If the decision is 'commit', the coordinator sends `commitTransaction` to the participant shards, and waits for responses. If the decision is 'abort', it sends `abortTransaction`

Each participant shard marks the transaction as committed or aborted, and updates the `config.transactions` document. The coordinator deletes the coordinator document with write concern `{w: 1}`. The prepare phase is skipped if the coordinator already has the participant list and the commit decision persisted. This can be the case if the coordinator was created as part of step-up recovery. Mongos will implicitly abort a transaction on any error except the view resolution error from a participant shard if a two phase commit has not been initiated. To explicitly abort a transaction, a client must send an `abortTransaction` command to the mongos that the transaction runs on. The command is also retryable as long as no new transaction has been started on the session and the session is still alive. In both cases, the mongos simply sends `abortTransaction` to all participant shards. In typical transactions, all transaction participants are contacted by mongos, which maintains the state of the transaction and its shard participants. However, some sub-operations and sub-pipelines executing on a shard can cause the shard to target a second shard that may or may not already be a participant in the transaction. As such, the originating participant begins acting as a "subrouter" -- a participant that can send requests to and receive responses from participants it has added to the transaction. To target another shard and add it as a participant to the transaction, the originating participant instantiates a subrouter object (`TransactionRouter`) to maintain a list of participants it has added. The originating participant attaches an internal-only field, `startOrContinueTransaction: true`, on the request it is sending to the targeted shard. The transaction-level `readConcern` argument will also be attached to the request. If the transaction's `readConcern` level is not `snapshot`, then the originating participant includes its own `placementConflictTimeForNonSnapshotReadConcern` as `placementConflictTime` in `shardVersion` on the request. Otherwise, for `snapshot` `readConcern`, there is no change: the `atClusterTime` of the transaction's snapshot is included in `shardVersion`. The targeted shard that receives this request starts a new transaction in the same way it would if receiving a request from mongos with the field `startTransaction: true`. However, if the targeted shard is already a participant of the same transaction, it performs the request as part of its current transaction. When a participant with a subrouter responds to a request, it attaches its list of participants to its response; thus mongos is made aware of participants that have been added to the transaction by the participant shard. This ensures that mongos sends abort and commit commands to participants that have been added by other participants. Because subrouters do not maintain global knowledge of the participants in a transaction, they do not initiate commit or abort of the transaction. Clients are not allowed to send the `startOrContinueTransaction` field. Internal transactions are transactions that mongos and mongod can run on behalf of a client command regardless of a client's session option configuration. These transactions are started and managed internally by mongos/mongod, thus clients are unaware of the execution of internal transactions. All internal transactions will be run within an a session started internally, which we will refer to as `internal sessions`, except for in the case where the client is already running a transaction within a session, to which we let the transaction execute as a regular client

An internal transaction started on behalf of a client command is subject to the client command's constraints such as terminating execution if the command's `$maxTimeMS` is reached, or guaranteeing retryability if the issued command was a retryable write. These constraints lead to the following concepts. If a client runs a command in a without a session or with session where retryable writes are disabled I.E. `retryWrites: false`, the server will start a non-retryable internal transaction. If a client runs a command in a session where retryable writes are enabled I.E. `retryWrites: true`, the server will start a retryable internal transaction. Note: The distinction between Retryable and Non-Retryable here is the requirement that Retryable Internal Transactions must fulfill the retryable write contract, which is described below. Both types of transactions will be . The only exception is an internal transaction that is started on behalf of a `client transaction`, which can only be retried by the client. We expect that retryable write commands that start retryable internal transactions conform to the retryable write contract which has the following stipulations: Write statements within the command are guaranteed to apply only once regardless of how many times a client retries. The response for the command is guaranteed to be reconstructable on retry. To do this, retryable write statements executed inside of a retryable internal transaction try to emulate the behavior of ordinary retryable writes. Each statement inside of a retryable write command should have a corresponding entry within a retryable internal transaction with the same `stmtId` as the original write statement. When a transaction participant for a retryable internal transaction notices a write statement with a previously seen `stmtId`, it will not execute the statement and instead generate the original response for the already executed statement using the oplog entry generated by the initial execution. The check for previously executed statements is done using the `retriedStmtIds` array, which contains the `stmtIds` of already retried statements, inside of a write command's response. In cases where a client retryable write command implicitly expects an auxiliary operation to be executed atomically with its current request, a retryable internal transaction may contain additional write statements that are not explicitly requested by a client retryable write command. An example could be that the client expects to atomically update an index when executing a write. Since these auxiliary write statements do not have a corresponding entry within the original client command, the `stmtId` field for these statements will be set to `{stmtId: kUninitializedStmtId}`. These auxiliary write statements are non-retryable, thus it is crucial that we use the `retriedStmtIds` to determine which client write statements were already successfully retried to avoid re-applying the corresponding auxilary write statements. Additionally, these statements will be excluded from the history check involving `retriedStmtIds`. To guarantee that we can reconstruct the response regardless of retries, we do a "cross sectional" write history check for retryable writes and retryable internal transactions prior to running a client retryable write/retryable internal transaction command. This ensures we do not double apply non-idempotent operations, and instead recover the response for a successful execution when appropriate. To support this, the was added as a decoration on an external session and it stores the transaction participants for all active retryable writes on the session, which we use to do our

To reconstruct responses for retryable internal transactions, we use the applyOps oplog entry, which contains an inner entry with the operation run under the `o` field that has a corresponding `stmtId`. We use the `stmtId` and `opTime` cached in the `TransactionParticipant` to lookup the operation in the applyOps oplog entry, which gives us the necessary details to reconstruct the original write response. The process for reconstructing retryable write responses works the same way. `findAndModify` additionally, requires the storage of pre/post images. Upon committing or preparing an internal transaction, we insert a document into `config.image_collection` containing the pre/post image. The operation entry for the findAndModify statement inside the applyOps oplog entry will have a `needsRetryImage` field that is set to `true` to indicate that a pre/post image should be loaded from the side collection when reconstructing the write response. We can do the lookup using a transaction's `lsid` and `txnNumber`. Currently, a retryable internal transaction can only support a single `findAndModify` statement at a time, due to the limitation that `config.image_collection` can only support storing one pre/post image entry for a given `(lsid, txnNumber)`. To be able to guarantee retryability under failover, we need to make sure that a mongod always has all the necessary transaction state loaded while executing a retryable write command. To do this, we recover the transaction state of the client and internal sessions on recovery. During checkout, we call on the current client session (if we are running in one) to refresh the TransactionParticipant for that session. We then . Due to the use of `txnUUID` in the lsid for de-duplication purposes, retries of client write statements will always spawn a different internal session/transaction than the one originally used to do the initial attempt. This has two implications for conflict resolution: If the client retries on the same mongos/mongod that the original write was run on, retries are blocked by mongos/mongod until the original attempt finishes execution. This is due to the that prevents checkout of an in-use session, which in this case would block the retry attempt from checking out the parent session. Once the original write finishes execution, the retry would either retry(if necessary) or recover the write response as described above. If the client retries on a different mongos than the original write was run on, the new mongos will not have visibility over in-progress internal transactions run by another mongos, so this retry will not be blocked and legally begin execution. When the new mongos begins execution of the retried command, it will send commands with `startTransaction` to relevant transaction participants. The transaction participants will then with the new internal transaction that is attempting to start. If so, then the transaction participant will throw `RetryableTransactionInProgress`, which will be caught and cause the new transaction to . The session history, oplog entries, and image collection entries involving the chunk being migrated are cloned from the donor shard to the recipient shard during chunk migration. Once the recipient receives the relevant oplog entries from the donor, it will . Depending on the type of operation run, the behavior will differ as such. If a non-retryable write/non-retryable internal transaction is run, then the donor shard will , which when parsed by the TransactionParticipant upon getting a retry against the recipient shard will

If a retryable write/retryable internal transaction is run, then the donor shard will send a , which when parsed by the TransactionParticipant upon getting a retry against the recipient shard will return the original write response. `Note`: "Downconverting" in this context, is the process of extracting the operation information inside an applyOps entry for an internal transaction and constructing a new retryable write oplog entry with `lsid` and `txnNumber` set to the associated client's session id and txnNumber. For resharding, the process is similar to how chunk migrations are handled. The session history, oplog entries, and image collection entries for operations run during resharding are cloned from the donor shard to the recipient shard. The only difference is that the recipient in this case will handle the "downconverting", nesting, and applying of the received oplog entries. The two cases discussed above apply to resharding as well. The is used to initiate transactions from within the server. The API starts an internal transaction on its local process, executes transaction statements specified in a callback, and completes the transaction by committing/aborting/retrying on transient errors. By default, a transaction can be retried 120 times to mirror the 2 minute timeout used by the . Additionally, the API can use router commands when running on a mongod. Each command will execute as if on a mongos, targeting remote shards and initiating a two phase commit if necessary. To enable this router behavior the defines an additional set of behaviors to rename commands to their . Transactions for non-retryable operations or operations without a session initiated through the API use sessions from the to prevent the creation and maintenance of many single-use sessions. To use the transaction API, by providing the opCtx, an executor, and resource yielder. Then, run the commands to be grouped in the same transaction session on the transaction object. Some examples of this are listed below. When a mongos or mongod executes a command that requires shard targeting, it must use routing information that matches the read concern of the command. If the command uses `"snapshot"` read concern, it must use the historical routing table at the selected read timestamp. If the command uses any other read concern, it must use the latest cached routing table. The provides an interface for obtaining the routing table at a particular timestamp and collection version, namely the `ChunkManager`. The `ChunkManager` has an optional clusterTime associated with it and a `RoutingTableHistory` that contains historical routing information for all chunks in the collection. That information is stored in an ordered map from the max key of each chunk to an entry that contains routing information for the chunk, such as chunk range, chunk version and chunk history. The chunk history contains the shard id for the shard that currently owns the chunk, and the shard id for any other shards that used to own the chunk in the past `minSnapshotHistoryWindowInSeconds` (defaults to 300 seconds). It corresponds to the chunk history in the `config.chunks` document for the chunk which gets updated whenever the chunk goes through an operation, such as merge or migration. The `ChunkManager` uses this information to determine the shards to target for a query. If the clusterTime is not provided, it will return the shards that currently own the target chunks. Otherwise, it will return the shards that owned the target chunks at that clusterTime and will throw a `StaleChunkHistory` error if it cannot find them.