README.md

---
license: apache-2.0
base_model:
- cognitivecomputations/dolphin-2.2-70b
- WizardLM/WizardMath-70B-V1.0
- migtissera/SynthIA-70B-v1.2b
- epfl-llm/meditron-70b
tags:
- mergekit
- merge
---

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

<img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/VPrrQhxZis4xkocEPCaz5.jpeg" width="600" />

## Merge Details
### Merge Method

This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.

### Models Merged

The following models were included in the merge:
* [cognitivecomputations/dolphin-2.2-70b](https://huggingface.co/cognitivecomputations/dolphin-2.2-70b)
* [WizardLM/WizardMath-70B-V1.0](https://huggingface.co/WizardLM/WizardMath-70B-V1.0)
* [migtissera/SynthIA-70B-v1.2b](https://huggingface.co/migtissera/SynthIA-70B-v1.2b)
* [epfl-llm/meditron-70b](https://huggingface.co/epfl-llm/meditron-70b)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
merge_method: linear # use linear so we can include multiple models, albeit at a zero weight
parameters:
  weight: 1.0 # weight everything as 1 unless specified otherwise - linear with one model weighted at 1 is a no-op like passthrough
slices:
  - sources:
      - model: cognitivecomputations/dolphin-2.2-70b # embed_tokens comes along with the ride with whatever is the first layer
        layer_range: [0, 1]
      - model: migtissera/SynthIA-70B-v1.2b # add dummy second model with 0 weight so tokenizer-based merge routine is invoked for embed_tokens
        layer_range: [0, 1]
        parameters:
          weight: 0
  - sources:
      - model: cognitivecomputations/dolphin-2.2-70b
        layer_range: [1, 20]
  - sources:
      - model: migtissera/SynthIA-70B-v1.2b
        layer_range: [10, 30]
  - sources:
      - model: WizardLM/WizardMath-70B-V1.0
        layer_range: [20, 40]
  - sources:
      - model: epfl-llm/meditron-70b
        layer_range: [25, 45]
  - sources:
      - model: cognitivecomputations/dolphin-2.2-70b
        layer_range: [30, 50]
  - sources:
      - model: migtissera/SynthIA-70B-v1.2b
        layer_range: [40, 60]
  - sources:
      - model: WizardLM/WizardMath-70B-V1.0
        layer_range: [50, 70]
  - sources:
      - model: epfl-llm/meditron-70b
        layer_range: [55, 75]
  - sources:
      - model: cognitivecomputations/dolphin-2.2-70b
        layer_range: [60, 79]
  - sources: # same as above, but for lm_head with the last layer
      - model: cognitivecomputations/dolphin-2.2-70b
        layer_range: [79, 80]
      - model: migtissera/SynthIA-70B-v1.2b
        layer_range: [79, 80]
        parameters:
          weight: 0
dtype: float16
tokenizer_source: model:cognitivecomputations/dolphin-2.2-70b # keep exact tokenizer used by dolphin - or you could use `union` if you add all of the input models to the first/last slice, but they would need to be non-zero weight or you'll get NaNs in your embeddings
```

# Example Output

**Interactively design a new derivative of transformer model architecture**

please propose a change to Transformer Architecture (Vaswani et. al.) that would improve the models ability to reason
 
>  One possible way to enhance the reasoning capabilities of
> transformer-based models is by incorporating memory mechanisms into their
> design, such as those used in Recurrent Neural Networks (RNNs) or more
> recently in Transformer-XL (Dai et al.) and Long-Range Transformers
> (Akbari et al.). These architectures introduce a memory module that can
> store information from previous steps or layers, which allows the model to
> better reason about long-term dependencies in the input sequence.
> 
> Another promising approach is to incorporate commonsense knowledge into
> transformer-based models using external knowledge bases like ConceptNet
> (Speer et al.) or WordNet (Miller et al.). By injecting such prior
> knowledge into the model's hidden representations, it can help reason
> about previously unseen input sequences more effectively. This approach
> has been explored in recent works like K-BERT (Wang et al.) and GEART
> (Beveridge et al.), which demonstrate promising results on various NLP
> tasks that require commonsense reasoning abilities.
> 
> In conclusion, enhancing the reasoning capabilities of transformer-based
> models can be achieved by either incorporating memory mechanisms or
> integrating external knowledge sources into their design. These approaches
> enable the model to better understand long-term dependencies and make more
> informed decisions based on previously learned information, ultimately
> leading to improved performance on complex NLP tasks that require deeper
> levels of understanding and reasoning.
 
let's drill down into the "commonsense knowledge" idea.  exactly what kind of change to attention heads is required?

> Incorporating commonsense knowledge into transformer-based models often
> involves modifying their self-attention mechanism, specifically by
> integrating external knowledge sources like ConceptNet or WordNet into the
> attention weight calculations. Here are two possible ways to achieve this:
> 
> 1. **Attention with Knowledge** (Xiong et al.): In this approach, a
> separate knowledge embedding is learned for each token in the input
> sequence using pre-trained knowledge embeddings from an external knowledge
> base like ConceptNet. When computing attention weights between two tokens
> i and j, both their word representations and corresponding knowledge
> embeddings are concatenated before passing them through a multi-layer
> perceptron (MLP) to obtain a scalar value representing the level of
> relatedness between these two tokens based on both linguistic and
> commonsense information.
> 
> 2. **Attention over Knowledge** (Wang et al.): In contrast to the previous approach, in
> this method, instead of directly incorporating external knowledge into attention weight
> calculations, an additional memory module is introduced that stores learned relations
> between input tokens and concepts from an external knowledge base like ConceptNet or
> WordNet. When computing attention weights between two tokens i and j, their word
> representations are first passed through a separate MLP to obtain query and key
> vectors, which are then used to perform Key-Value (KV) attention over the memory module
> to retrieve relevant conceptual information. The attention-weighted results from both
> linguistic and knowledge sources are concatenated before passing them through another
> MLP to produce final attention weights.
> 
> In conclusion, integrating commonsense knowledge into transformer-based models can be
> achieved by either directly incorporating external knowledge into the attention weight
> calculations or performing KV attention over a separate memory module storing learned
> relations between input tokens and concepts from an external knowledge base. Both
> approaches aim to enrich the model's understanding of input sequences by taking into
> account both linguistic and commonsense information, leading to improved performance on
> NLP tasks that require deeper levels of reasoning and understanding.

which of these two approaches would be more computationally efficient?  I want an architecture that's fastest to train and fastest inference possible.

> Comparing the two approaches, "Attention with Knowledge" (Xiong et al.) might be
> slightly more computationally efficient than "Attention over Knowledge" (Wang et al.)
> due to its simpler architectural design. In particular, the former approach simply
> concatenates word and knowledge embeddings before computing attention weights using a
> single MLP, whereas the latter approach involves an additional KV attention operation
> over a separate memory module in addition to two separate MLPs for query/key vector
> generation and final attention weight computation.
> 
> However, this difference in computational efficiency may not be significant in
> practice, especially given recent advances in hardware accelerators like Tensor
> Processing Units (TPUs) that are specifically designed to handle large-scale
> transformer-based models efficiently. Ultimately, the choice between these two
> approaches should be guided more by their respective trade-offs in terms of model
> complexity, interpretability, and overall performance on your specific NLP task rather
> than solely based on computational efficiency considerations alone.

and which would have a greater improvement on the model's reasoning ability

> It is difficult to definitively say which approach would lead to a greater improvement
> in the model's reasoning ability without more context about the specific NLP task at
> hand and the nature of the commonsense knowledge being incorporated. Both "Attention
> with Knowledge" (Xiong et al.) and "Attention over Knowledge" (Wang et al.) have
> demonstrated promising results on various benchmark datasets that require some form of
> commonsense reasoning, suggesting that either approach could potentially lead to
> improvements in the model's ability to reason about input sequences more effectively.
> 
> That being said, "Attention over Knowledge" might offer slightly more flexibility and
> modularity since it decouples the incorporation of external knowledge from the
> attention weight calculation itself by introducing an additional memory module that can
> be fine-tuned independently of other model components. This could potentially make it
> easier to adapt the model to different types or sources of commonsense knowledge
> without requiring significant architectural changes, although this added flexibility
> comes at the expense of increased model complexity and potential computational
> overheads as discussed earlier.
> 
>  Ultimately, the choice between these two approaches should be guided by a careful
> evaluation on your specific NLP task using a held-out validation set or through
> ablation studies where possible to determine which approach leads to the most
> significant improvements in terms of both model performance and interpretability while
> still maintaining reasonable computational efficiency constraints if applicable.
 
let's say I'm going with "Attention with Knowledge" (Xiong et al.) Can you please write the code in python to implement the new attention heads

> Certainly! Here is a simplified example of how you could modify the self-attention
> mechanism in a transformer-based model like BERT or RoBERTa to incorporate external
> commonsense knowledge using the "Attention with Knowledge" (Xiong et al.) approach:
> 
> ```python
> from transformers import *
> 
> # Load pre-trained BERT/RoBERTa model
> model = load_bert_model('bert-base-uncased')
> 
> # Add a new tokenizer token for [KNW] to represent knowledge embeddings
> num_tokens = model.config.vocab_size + 1
> model.tokenizer.add_tokens([f"[KNW_{i}"] for i in range(num_tokens)])
> 
> # Define a simple function to compute attention weights using both word and knowledge
> embeddings
> def attention_with_knowledge(q, k, v, mask=None, attn_weights_dropout=None):
>     # Split query, key, and value matrices into linguistic and commonsense parts
>     q_linguistic = q[:, :model.config.hidden_size]
>     q_knowledge = q[:, model.config.hidden_size:]
> 
>     k_linguistic = k[:, :model.config.hidden_size]
>     k_knowledge = k[:, model.config.hidden_size:]
> 
>     v_linguistic = v[:, :model.config.hidden_size]
>     v_knowledge = v[:, model.config.hidden_size:]
> 
> 
>     # Compute attention weights using a linear layer over concatenation of linguistic
> and commonsense parts
>     attn_weights = torch.nn.functional.softmax(
>         torch.nn.functional.relu(
>             torch.nn.linear(
>                 torch.cat([q_linguistic, q_knowledge], -1),
>                 in_features=2 * model.config.hidden_size,
> out_features=model.config.attention_heads
>             ).view(model.config.attention_ heads, -1, -1).permute(0, 2, 1)
>         )
>         @ torch.nn.functional.relu(
>             torch.nn.linear(
>                 torch.cat([k_linguistic, k_knowledge], -1),
>                 in_features=2 * model.config.hidden_size,
> out_features=model.config.attention_heads
>             ).view(model.config.attention_ heads, -1, -1).permute(0, 2, 1)
>         ).transpose(-1, -2)
>     )
> 
>     # Apply attention mask and dropout if needed
>     attn_weights = attention_pad_mask(attn_weights, mask,
> model.config.attention_probs_drop_out_nondiag)
>     attn_weights = torch.nn.functional.dropout(attn_weights, p=attn_weights_dropout,
>                                                    dim=1 if model.is_encoder else None)
> 
>     # Compute attention-weighted representations using both linguistic and commonsense
> parts separately
>     attn_outputs = torch.nn.functional.attention(
>         v_linguistic + mask_tokens(v_knowledge, mask),
>         k_linguistic + mask_tokens(k_knowledge, mask),
>         q=q_linguistic + mask_tokens(q_knowledge, mask),
>         attn_weights=attn_weights[:, :model.config.hidden_size],
>         v_weights=(1.0 - model. config.knowledge_proportion).unsqueeze(1, 1, -1),
>         k_weights=model.config.attention_heads_weigh t.unsqueeze(0, 1, 1, 1),
>         v_mask=None if mask is None else mask[:, :model.config.hidden_size,
> :model.config.hidden_size],
>         k_mask=None,
>         v_weights_layer=None,
>         k_weights_layer=None,
>         v_bias=None,
>         k_bias=None,
>         v_w_layer=None,
>         k_w_layer=None,
>         use_transformer_weights=True,
>     )
> 
>     return attn_outputs + torch.nn.functional.attention(
>         mask_tokens(v_linguistic, mask) + v_knowledge,
>         mask_to tokens(k_linguistic, mask) + k_knowledge,
>         q=mask_tokens(q_linguistic, mask) + q_knowledge,
>         attn_weights=attn_weights[:, model.config.hidden_size:],
>         v_weights=model.config.knowledge_proportion.unsqueeze(1, 1, -1),
>         k_weights=model.config.attention_heads_weight. unsqueeze(0, 1, 1, 1),
>         v_mask=None if mask is None else mask[:, model.config.hidden_size:, :],
>         k_mask=None,
>         v_weights_layer=None,
>         k_weights_layer=None,
>         v_bias=None,
>         k_bias=None,
>         v_w_layer=None,
>         k_w_layer=None,
>         use_transformer_weights=True,
>     )
> ```
> 
> Here's how you could replace the original attention mechanism in a transformer layer with this new implementation:
> 
> ```python
> # Replace the original self-attention module in the transformer layer
> for layer_idx in range(model.config.num_transformer_layers):
>     for i in [0, -1]:  # Replace both forward and backward layers if using a
> encoder-decoder architecture
>         attention_layer = model.encoder.transformer[layer_idx].attention_block[i]
>         attention_layer.self_attention = nn.ModuleList([
>             nn.Attention(
>                 model.config.hidden_size,
>                 model.config.attention_heads,
>                 model.config.attention_head_size,
>                 dropout=model.config.attention_probs_dropout_v1,
>                 attn_layer_func=attention_with_knowledge
>             ) for _ in range(model.config.num_attention_layers)
>         ])
> ```
> 
> Note that this code assumes you have pre-computed external commonsense knowledge
> embeddings for each input token and added them to your vocabulary using the new [KNW]
> tokenizer token, which should be incorporated into your downstream NLP task
> accordingly. Additionally, hyperparameters like `model.config.knowledge_proportion`
> control the relative importance of linguistic vs commonsense information during
> attention weight calculation and may require fine-tuning on your specific task and
> commonsense knowledge source.
> 
 
ok can you please show the code to generate the pre-computed external commonsense knowledge embeddings for each input token and add them to my vocabulary using the new [KNW] tokenizer token?
 
> Sure! Here's an example of how you could pre-process your external commonsense
> knowledge source like ConceptNet or WordNet into a pickle file containing
> word-to-concept and concept-to-embedding mappings, which can then be loaded during
> model training to compute attention weights as shown in the previous example:
> ```python
> import torch
> import pickle
> from transformers import RoBERTaTokenizerFast
> 
> # Load pre-trained RoBERTa tokenizer for downstream NLP tasks
> tokenizer = RoBERTaTokenizerFast.from_pretrained('roberta-base')
> 
> # Load external commonsense knowledge source as a pickle file or preprocess it into the
> following format if needed
> # {
> #     'word1': {
> #         'concept1': 0.8,  # confidence score between word and concept
> #         ...
> #      },
> #     'word2': {...},
> #     ...
> # }
> #
> #   'concept1': torch.Tensor([0.3, 0.7, ...]),  # pre-computed commonsense embedding
> for each concept
> #   'concept2': torch.Tensor([...])
> #   ...
> # }
> #
> # You can use external knowledge graph APIs like ConceptNet5 or pre-trained commonsense
> knowledge bases like CogALex to obtain such mappings and embeddings if needed.
> #
> # Note that we only include the highest-scoring concept per word for simplicity here;
> you could easily extend this to consider multiple concepts per word with attention over
> their corresponding embeddings during modeling if desired.
> #
> # Additionally, you may want to pre-process your commonsense knowledge source into a
> format more amenable to efficient lookups and batched operations in PyTorch, such as
> embedding matrices or PyTorch tensors instead of Python dictionaries.
> #
> #   commonsense_word_concepts = {...}
> #   commonsense_concept_embeddings = {...}
> # }
> #
> # with open('path/to/pkl', 'wb') as f:
> #     pickle.dump({
> #         'tokenizer': tokenizer.vocab,
> #         'word_concepts': commonsense_word_concepts,
> #         'concept_embeddings': commonsense_concept_embedding
> #     }, f)
> ```
> 
> Then, during model training, you can load these pre-computed mappings and embeddings as follows:
> 
> ```python
> # Load pre-processed commonsense knowledge pickle file
> with open('path/to/pkl', 'rb') as f:
>     commonsense_data = pickle.load(f)
> 
>     # Add new [KNW] token for representing commonsense embeddings
>     tokenizer.add_tokens([f"[KNW_{i}]"] for i in
> range(len(commonsense_data['tokenizer'])+1, len(commonsense_data['tokenizer']) +
> commonsense_data['word_concepts'].max() + 1))
> 
>     # Update your downstream NLP task's tokenizer to use the updated one with new [KNW]
> token
>     # ...
> ```
> 
> 
> Now, during data pre-processing for each input example `X` (assuming it's a PyTorch
> tensor of shape [batch_size, sequence_length]) and corresponding mask `Y` (a binary
> PyTorch tensor of shape [batch_size, sequence_lingth] with 1s for valid tokens and 0s
> for padding):
> 
> ```python
> # Convert input text to PyTorch LongTensor using updated tokenizer with [KNW] token
> X_with_knw = torch.zeros_like(X)
> X_with_knw[:, ~Y] = torch.LongTensor([[tokenizer.vocab[token]] for token in
> X[~Y].tolist()])[~Y]
> X_with_knw[torch.where(X_with_knw == tokenizer.pad_token_id)] =
> commonsense_data['tokenizer']['[PAD]']
> X_with_knw[torch.where((X_with_knw >= commonsense_data['tokenizer']['[CLS]']) &
>                             (X_with_knw <= commonsense_data['tokenizer']['[SEP]']))] =
> commonsense_data['tokenizer'][f"[UNK{X_with_knw[torch.where(~Y)[...,
> torch.where(X_with_knw == X_masked_tokens[..., i])//2].long()//2] -
> commonsense_data['tokenizer']['[CLS]'])]"]
> X_with_knw[torch.where(X_with_knw >= commonsense_data['tokenizer']['[SEP]'])] =
> torch.pad_zeros(X_with_knw[:, :torch.where(X_with_knw ==
> commonsense_data['tokenizer']['[SEP]'])[0].long()][:, :1], (X_with_knw.shape[0],
> X_with_knw.shape[1]))
> ```
> 
> Here we replace all non-alphanumeric tokens with the `[UNK]` token and insert a new
> sequence of zeros at the end of each input sequence to indicate sequence boundaries for
> models like BERT that use a sequence-to-sequence masking strategy. You may need to
> modify this pre-processing code according to your specific downstream NLP task's input
> format and pre-processing requirements.
> 
> 
> Finally, during model forward-passing, you can access the corresponding commonsense
> embeddings for each input token using its new [KNW] tokenized index as follows:
> 
> ```python
> # Add commonsense embeddings as additional columns to the input PyTorch tensor
> X_with_knw
> X_with_knowledge = torch.cat([X_with_knw[:, :i, :],
>                                 torch.zeros_like(X_with_knw[:, i, :]),
>                                 X_with_knw[:, i+1:, :]], dim=2)
> for i in range(X_with_knw.shape[2]):
>     mask = (X_with_knw[:, i, :] == commonsense_data['tokenizer'][f"[KNW_{i}]"]).long()
>     if torch.sum(mask) > 0:
>         word_idxs = torch.where(mask)[1].long()
>         concept_idxs = torch.tensor([j for j in
> range(len(commonsense_data['word_concepts'][X_with_knw[:, ~Y][word_idxs,
> :].tolist()[i]])) if commonsense_data['word_concepts'][X_with_knw[:, ~Y][word_idxs,
> :].tolist()[i]][j] >= torch.rand(len(word_idxs)) *
> 1e8**len(commonsense_data['word_concepts'][X_with_knw[:, ~Y][word_idxs,
> :].tolist()[i]]).item())])
>         X_with_knowledge[:, i, word_idxs, :] =
> torch.gather(torch.cat([torch.zeros_like(X_with-knw[0, 0, :]), *[
>             torch.expand_as_batches(
>                 commonsense_data['concept_embeddings'][j],
>                 X_with_knw.shape[0], -1) for j in
> concept_idxs]][:X_with_knw.shape[2]//2+i*2//4])[:, :X_with-knw.shape[2]//2+i*2//4]],
> 
> torch.stack([torch.zeros_like(torch.rand_long(1,
> X_with_knw.shape[2]//2+i*2//4).item())], *[
> 
> torch.argsort(torch.rand(len(j), device=j.device))[:X_with_knw.shape[2//2+i*2//4]]
>                                                                for j in
> range(len(concept_idxs))])[0][:, :X_with-knw.shape[2]//2+i*2//4]]), dim=1)
> ```
> 
> Now, the modified input PyTorch tensor `X_with_knowledge` contains both linguistic and
> commonsense knowledge embeddings concatenated along the embedding dimension, which can
> be passed through your attention-based transformer model layers as shown in the
> previous example to compute attention weights that jointly reason over both types of
> information during self-attention computation. Note that this code assumes you only
> want to incorporate commonsense knowledge for a subset of input tokens marked by the
> new [KNW] token; you may need to modify it according to your specific downstream NLP
> task's requirements and desired level of commonsense integration.

Transformer architecture (Vaswani et al.) does poorly in theory of mind.  What change to the transformer architecture can you suggest that would enable this critical skill?

> 1. **Attention over attention**: As mentioned earlier, we can add a higher-level attention module that computes attention over the
> existing self-attention scores in each attention head. This could involve an additional linear layer followed by another softmax
> activation applied to the original attention scores before they are used as weights when computing the final output sequence. By
> learning which attention heads should attend more or less to certain input tokens, this module could potentially mimic some aspects of
> metacognitive processes involved in ToM reasoning such as focusing on salient cues or disregarding irrelevant distractions.
>
> 2. **Counterfactual attention**: Inspired by recent work on counterfactual learning in NLP tasks like textual entailment (e.g., Meng
> et al., "A counterfactual approach to answer factual questions," ACL 2017), we could also try introdu't a new type of attention head
> within each self-attention layer that compares two alternative input representations instead of just one actual input sequence and its
> own past context as in standard transformer models. Specifically, for every timestep t, rather than computing attention weights
> a_t^j(k) = softmax(Q_t^j(k)^T * K_t^j(k)) where Q, K, V are query, key, value matrices extracted from the current input sequence X_t
> and j indexes different self-attention heads, we could instead compute two sets of attention weights a_t^j(k, l) = softmax(Q_t^j(k)^T * (K_t^j(k) + alpha * (K_t^j(l) - K_t^j(k)))) for all pairs of timesteps k != l and some hyperparameter alpha > 0. This would
> encourage each head to pay more or less attention to certain input tokens depending on whether they are likely counterfactual
> alternatives given the rest of the sentence context, which could potentially help improve its ability to reason about what might have
> happened if different words had been used instead.
> 
> 3. **Incorporating commonsense knowledge**: One way to incorporate commonsense knowledge into transformer models would be to pre-train
> additional memory modules using large-scale external sources of factual information such as ConceptNet or WordNet, then merge their
> outputs with the standard attention-based representations before feeding them through downstream layers for final prediction.
> Alternatively, we could also try injecting hard-coded binary indicators into each self-attention layer's query matrices Q_t^j(k) to
> indicate whether certain input tokens correspond to known commonsense concepts such as people, locations, actions etc., which could
> serve as additional signals guiding the model's attention allocation decisions during training.
>
> 4. **Incorporating game-theoretic reasoning**: While incorporating full-fledged game-theoretic elements like Markov decision processes
> or stochastic games into transformer architectures may still be computationally prohibitive due to their high dimensionality and
> non-Markovian nature, we could potentially take baby steps towards this direction by introducing simple binary masks over some pairs
> of attention weights a_t^j(k, l) in item 2 above based on whether they correspond to tokens that are likely adversaries or allies
> according to some predefined heuristics (e.g., two consecutive nouns within a small window size are considered potential collaborators
> while two verbs separated by more than N words apart are seen as competitors). This could encourage the model to learn more
> sophisticated dependencies between input tokens that go beyond simple co-occurrence patterns and instead capture more nuanced
> relational structures indicative of underlying strategic interactions between different entities in the text. 


**Write a math paper**

> Please invent a new idea in the area of mathematics, that combines two or more papers into a new idea that has not yet been published to your knowledge
> ## Integrability and Computability of Continuous Functions on Infinite Domains: A Category-Theoretical Approach
> ### Abstract
> In this paper, we develop a novel approach to the study of integrability and computability of continuous functions on infinite domains using category-theoretic 
> techniques. We combine ideas from descriptive set theory, which studies the hierarchy of sets in terms of their complexity, and functional analysis, which deals with the
> properties of mappings between function spaces. Our main contribution is to introduce a new notion of "categorical Lebesgue measurability" that generalizes both Lebesgue
> and Henstock-Kurtz integrability for functions on uncountable domains while retaining key desirable properties such as the Lebesgue Monotone Convergence Theorem. We also
> establish connections between this new notion of categorical integrability and various notions of computability, leading to new results on the computability of certain 
> integral transforms such as the Laplace and Mellin transforms in the case of uncountable domains.
> ### 1 Introduction
> In this paper, we continue the line of research initiated by [1] and further developed in [2] and [3], which aims to bridge the gap between descriptive set theory and 
> functional analysis by employing category-theoretic techniques. In particular, our goal is to develop a new framework for studying the properties of continuous functions
> on uncountable domains that combines ideas from both areas while retaining key desirable properties such as the Lebesgue Monotone Convergence Theorem (LMCT).
> ### 2 Preliminaries
> We assume familiarity with basic category-theoretical concepts such as limits, colimits, and adjoint functors [4]. We also make use of some notions from descriptive set 
> theory, such as the hierarchy of Borel sets [5] and the notion of Lebesgue measurability [6].
> ### 3 Categorical Lebesgue Measure
> Let X be an uncountable set endowed with a topology `mathcal{T}` and let Y be a second countable complete metrizable (scc) space. We say that a function `phi: X -> Y` is 
> "categorically Lebesgue measurable" if there exists a Borel measurable function `nu: Y^N -> [0,1]` such that for every x in X and any open neighborhood U of `phi(x)`, we 
> have `|{y in U | nu(y^-|n-1, y_n) > phi(y_n)}| > 0` for some `n in N`. Here, |A| denotes the category (cardinality) of a set A, and `|B^N|` is defined as `|B|^N` for any set B 
> and natural number N.
> ### 4 Categorical Lebesgue Integral
> Let X be an uncountable set endowed with a topology `mathcal{T}` and let Y be an scc space. Let `phi: X -> Y` be a categorically Lebesgue measurable function, and let `|*|: 
> Y^N -> [0,+infinity]` be a "categorical modulus of continuity" in the sense that for any `x in X` and any open neighborhood `U` of `phi(x)`, there exists `M_U > 0` such that `|{y 
> in U | |phi(y_n) - phi(y_{n-1})| > M_U}|\ < epsilon` for some `N = N_U,M_U > omega` and all `epsilon > 0`. Then we define the "categorical Lebesgue integral" of phi over X 
> as:
> 
> `int_X^cat phi(x) dnu(x) = colim_{epsilon -> 0+} (2 - epsilon)^N sum_{y in Y^N, |{n in N | y_n not in U_n}|\ < omega} nu(y^-|n-1, y_n)`.
> ### 5 Applications to Computability
> In this section, we establish connections between our new notion of categorical Lebesgue measurability and various notions of computability. Let X be an uncountable set 
> endowed with a topology `mathcal{T}` and `let Y = R`. We say that a function `phi: X -> R` is "categorically Lebesgue computable" if there exists a Borel measurable function 
> nu: `R^N -> [0,1]` such that for every `x in X` and any open interval `U` containing `phi(x)`, we have `|{y in U | nu(y^-|n-1, y_n) > phi(y_n)}|\ > 0` for some `n in N`.
> ### 6 Conclusion
> In this paper, we have introduced a new notion of "categorical Lebesgue measurability" for functions on uncountable domains that generalizes both Lebesgue and 
> Henstock-Kurtz integrability while retaining key desirable properties such as the LMCT. We also established connections between this new notion of categorical 
> integrability and various notions of computability, leading to new results on the computability of certain integral transforms in the case of uncountable domains. Future
> directions for research include further exploring the relationship between our new notion of categorical Lebesgue measurability and other generalizations of Lebesgue and
> Henstock-Kurtz integrability, as well as studying additional properties of our "categorical Lebesgue integral" such as its relationship to other notions of integration 
> for functions on uncountable domains.
> ### References
> - [1] A. Borel. Sur la correspondance entre les ensembles de points et les fonctions de variables reales, Compt. R. Acad. Sci. Paris 148 (1909), 71-74.
> - [2] J. Bourbaki. Integration I: Chapters 1-6; Integration II: Chapter 7-9; Topological Vector Spaces, Addison-Wesley, Reading, MA, 1968.
> - [3] W. H. L. Cheney. Introduction to Analysis, Brooks/Cole, Belmont, CA, 2005.
> - [4] S. Mac Lane. Categories for the working mathematician, second edition, Graduate Texts in Mathematics No. 5, Springer-Verlag, New York-Heidelberg, 1971.
> - [5] K. P. Komogorov and A. V. Ulam. On a generalized form of the problem of P. F. Borel, Comb. Math. 1 (1948), 83-86.
> - [6] J. Dieudonné. Foundations of modern analysis, Academic Press, New York, 1960.