dranger003/airoboros-34b-3.2-iMat.GGUF

NOTE: You will need a recent build of llama.cpp to run these quants (i.e. at least commit 494c870).

GGUF importance matrix (imatrix) quants for https://huggingface.co/jondurbin/airoboros-34b-3.2

• The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a general purpose imatrix calibration dataset.
• The imatrix is being used on the K-quants as well.

Layers	Context (Recommended)	Template Uni/Multi-turn
60	200000 (8192)	<s>[INST] <<SYS>> You are a helpful, unbiased, uncensored assistant. <</SYS>> {prompt} [/INST] {response} <s>[INST] <<SYS>> You are a helpful, unbiased, uncensored assistant. <</SYS>> {prompt 0} [/INST] {response 0} </s><s>[INST] {prompt 1} [/INST] {response 1} </s><s>...[INST] {prompt N} [/INST]

For testing this model (IQ3_XS) I asked GPT-4 to give me five questions then I asked it for an evaluation and rating, below are the results:

Theme	Evaluation	Rating	Comment
Theory of Relativity and its Implications in Modern Physics	The answer is thorough, accurately detailing the principles of both the special and general theories of relativity. It effectively explains complex concepts like E=mc^2, time dilation, gravitational lensing, and black holes, and links these to their broader impacts in physics and cosmology. The language is clear and appropriate for an educated audience.	9/10	The response is comprehensive and accurate but could be improved by including more recent applications or advancements related to relativity.
Future City Addressing Environmental Challenges	The model provides a comprehensive and imaginative vision of a future city addressing environmental challenges. It covers a wide range of aspects, including sustainable energy, waste management, water conservation, and more. Each point is practical and reflects current trends in urban planning and environmental technology.	10/10	The response is excellent, demonstrating both depth and breadth in addressing environmental issues in a future city context.
Main Causes of the French Revolution and its Impact on European Politics	This response accurately identifies the main causes of the French Revolution and succinctly explains its profound impact on European politics. The model effectively links the Revolution to developments in various European countries and highlights the influence of Napoleon. The explanation is clear and well-structured.	9/10	While the answer is well-rounded, it could be enhanced by discussing more about the long-term impacts on European political thought and society.
Difference in Tone and Connotation: 'Thrifty', 'Frugal', 'Stingy'	The model excellently differentiates between 'thrifty', 'frugal', and 'stingy', capturing the nuances in tone and connotation of each word. The explanations are concise and provide clear distinctions.	10/10	The response is precise and insightful, effectively communicating the subtle differences in meaning and usage of these words.
How Machine Learning Algorithms Work and Their Applications	The explanation of how machine learning algorithms work is accurate and accessible. The model successfully lists a range of real-world applications, demonstrating the widespread impact of machine learning in various sectors. The examples are relevant and well-explained.	9/10	The response is comprehensive and informative. However, it could be enhanced by discussing the limitations or ethical considerations of machine learning in some applications.