NousResearch

/

Hermes-2-Pro-Llama-3-8B-GGUF

like 146

@jpdev the context in the gguf files are indeed 8k, you can push your luck with rope alpha scaling to 4, but sticking to 8k is the safe choice, just optimise your prompts

I've gotten a context window of 16384 with a time to hallucination for this model at about 12K, obvious failure at about 16.4K This is a truly lovely build. Thank you.

System:

You are a thoughtful, helpful and knowledgeable AI assistant. You provide clear and concise responses, offering gentle guidance when needed, and politely addressing any misconceptions with well-reasoned explanations.

User:

Is there a way to isolate dark matter's effects on BAOs?

Assistant:

To isolate the effects of dark matter on baryon acoustic oscillations (BAOs), researchers use various techniques in observational cosmology, such as:

1. Observing the large-scale structure of the universe: Dark matter and baryons both contribute to the formation of cosmic structures like galaxies and galaxy clusters. By studying these structures, scientists can infer the distribution of dark matter relative to BAOs.

2. Measuring weak gravitational lensing: Dark matter's gravitational influence on light bending allows researchers to map its distribution through weak lensing observations. This method helps in understanding how dark matter affects BAOs by tracing its connection with baryons and other cosmic structures.

3. Using the cosmic microwave background (CMB) radiation: The CMB provides a snapshot of the early universe, revealing patterns related to density fluctuations that later evolved into large-scale structure. By analyzing these patterns, scientists can study the interplay between dark matter and BAOs during the universe's formation.

4. Studying redshift space distortions: Dark matter affects the motion of galaxies, which in turn influences the observed distribution of galaxy positions in space. Analyzing this relationship offers insights into how dark matter shapes BAO patterns.

5. Employing numerical simulations: Computer simulations can model the evolution of the universe, including the effects of dark matter on BAOs. These simulations help researchers validate and refine their understanding of the complex interplay between dark matter and baryons in shaping cosmic structures.

While these methods cannot directly observe dark matter itself, they provide indirect evidence to study its impact on BAOs and other large-scale phenomena in the universe.
User:

Is there anything about polarization involved in part of these sorts of study?

Assistant:

Polarization can indeed play a role in certain aspects of cosmological studies related to dark matter and baryon acoustic oscillations (BAOs). One specific area where polarization is relevant is through the cosmic microwave background radiation (CMB).

The CMB, which is the thermal radiation left over from the Big Bang, has both temperature and polarization information. The temperature fluctuations in the CMB are primarily due to density variations in the early universe, while its polarization arises from the interaction of this radiation with matter during recombination, when electrons combined with protons to form hydrogen atoms.

The polarization signal can be divided into two main types: E-mode (gradient) and B-mode (curl). E-modes are generated by gravitational lensing, which is a phenomenon influenced by both dark matter and dark energy. B-modes, on the other hand, are primarily produced by primordial gravitational waves, which could be generated during an early epoch of cosmic inflation.

By studying these polarization patterns in the CMB, researchers can gain insights into the distribution and properties of dark matter, as well as its effects on BAOs. For example, measuring the connection between temperature and polarization fluctuations helps to improve our understanding of how dark matter influences structure formation and evolution in the universe.

Additionally, detecting B-mode polarization would provide strong evidence for inflationary models, which could shed light on the early universe's conditions and the role of dark matter during its formation. However, current observations have not yet detected B-modes with sufficient significance, making this an active area of ongoing research.

Win10, 64GB, RTX3090

llama_3_70B.preset.json file:

{
"name": "Llama 3 70B Instruct",
"load_params": {
"n_ctx": 16384,
"n_batch": 512,
"rope_freq_base": 1000000,
"rope_freq_scale": 0.85,
"n_gpu_layers": 64,
"use_mlock": true,
"main_gpu": 0,
"tensor_split": [
0
],
"seed": -1,
"f16_kv": true,
"use_mmap": true,
"no_kv_offload": false,
"num_experts_used": 0
},
"inference_params": {
"n_threads": 12,
"n_predict": -1,
"top_k": 40,
"min_p": 0.05,
"top_p": 0.9,
"temp": 0.2,
"repeat_penalty": 1.1,
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"antiprompt": [
"<|start_header_id|>",
"<|eot_id|>"
],
"pre_prompt": "You are a thoughtful, helpful and knowledgeable AI assistant. You provide clear and concise responses, offering gentle guidance when needed, and politely addressing any misconceptions with well-reasoned explanations.",
"pre_prompt_suffix": "<|eot_id|>",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"seed": -1,
"tfs_z": 1,
"typical_p": 1,
"repeat_last_n": 64,
"frequency_penalty": 0,
"presence_penalty": 0,
"n_keep": 0,
"logit_bias": {},
"mirostat": 0,
"mirostat_tau": 5,
"mirostat_eta": 0.1,
"memory_f16": true,
"multiline_input": false,
"penalize_nl": true
}
}