Giedrius FLT
said 10 months ago
on
Streaming LLM Responses
:
0
0
0
What about that context and [INST]? Why we need to add this? There wasn't enough explanation on this part. Thanks!
David Kimura
PRO
said 10 months ago
on
Streaming LLM Responses
:
1
0
0
This is considered the template or prompt format for communicating with the particular model that you're using. You can find more information and the specific templates on https://ollama.com/library/mixtral:latest. Since mistral is based on llama, it uses the [INST]prompt[/INST] template. If using gemma then the template would look something like <start_of_turn>model {{ .Response }}<end_of_turn>. It's basically formatting to yield better results from the model that is being used.
abner.figueroa
PRO
said 10 months ago
on
Streaming LLM Responses
:
0
1
0
Love this episode, awesome job David!
olimart
PRO
said 10 months ago
on
Streaming LLM Responses
:
0
1
0
Really fond of those LLM/ML-related episodes.
ahmad19
PRO
said 9 months ago
on
Streaming LLM Responses
:
0
2
0
Love this episode. It has helped me to get started with AI models. thank you