LeroyDyer/Mixtral_AI_Minitron_2b_1.0

From Fine Tuning To merging a model into a fully working model... Right now this model is still not trained !! The mini_merge Series are For individual training Regimes Hence Being merged into a single model ; this enable for models to stay focused on a single task producing individual charactaristics ie coding question, gramer , knowledge seeking , chat, roleplay , friendly:

These models are easy enough to train on a laptop ! Hence creating a series for training start points as well as merge start points (ie construct a model with the charactersistic via merge ), Personally i beleive that a well trained model is the same as a good lora model ; hence merging is a great way to combine qualitys !

once a suitable base model starting point is arrived then previous merge iterations disappear!
Or become specialist merge points .

In general a good chat dataset is what is needed at all times. (in general they refer to the model as a friendly AI (not good for creating a personality !))

To remove such contamination you would have to retrain until its gone but ! ... these are embedded now into models hence beging from scratch with a prefered smaller model for chat and mini tasks ,,, in fact an network is a network as long as it performs well : Extra large languge models are an myth as they seem to believe that teh larger the model the more data it contains !

In fact data is not stroed in the model !!! Only weights and probabilitys of combinations of words ie conversations so repeatetive entrys can embed information in where as sparse information does not get retreived as much ... hence many simular records which have only small differences hence also doubles are also ok ! as ther are so many ways to say hello!!! so we need a dataset of at least 1000 hellos and greetings and responses ! to epoch until under 0.5 ie its in the matrix of weights as a probability! when we train other knowelge we may need to return to favrited knowlegde to re mbed this knowledge and reraise its probablity of being returned ie 0.50 is 50% Wring answer 0.45 is 45% wrong answer !!

so for encoding of data on epoch as long as the data averages around these values then it will be retrived if the values are embedded at 0.2 0.001 then it is locked in and maybe a problem as it is favorite ! hence more simular informations

Eventually it will be viable to save all collected data into a folder to be fine tuned into the llm at regular intervals there fore continuing to update its own brain ! by sending it on aresearch it can collect relevant media and on some interval fine tune its weights! (while it is loaded !!!!! and reload After tuning is complete ! or upload!)

Models Merged

The following models were included in the merge:

• Base_1
• Base_2
• LeroyDyer/MIni_Merge_Dictionary
• LeroyDyer/Mini_Merge_Dolphin
• LeroyDyer/Mixtral_AI_MiniTron_2b_Chat
• LeroyDyer/Mini_Merge_BaseAlignment
• LeroyDyer/Mini_Merge_ChatBot

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: LeroyDyer/Mini_Merge_Dolphin
    parameters:
      weight: 0.28944
  - model: LeroyDyer/MIni_Merge_Dictionary
    parameters:
      weight: 0.28944
  - model: Base_1
    parameters:
      weight: 0.68944
  - model: Base_2
    parameters:
      weight: 0.68944
  - model: LeroyDyer/Mini_Merge_ChatBot
    parameters:
      weight: 0.8853
  - model: LeroyDyer/Mini_Merge_BaseAlignment
    parameters:
      weight: 0.28944
  - model: LeroyDyer/Mixtral_AI_MiniTron_2b_Chat
    parameters:
      weight: 0.1453
merge_method: linear
dtype: float16