> The token_type_embedding is zero init and frozen for responses and the user prompt, but trainable for system prompt.
I think the question is, what would you then train it to do with the additional information (privileged vs unprivileged text)? Intuitively, we want it to "follow directions" in the privileged text, but not in the unprivileged text, but the problem is that LLMs are not "following directions" now. An LLM doesn't turn your English into some internal model of a command, and then execute the command.
I think the question is, what would you then train it to do with the additional information (privileged vs unprivileged text)? Intuitively, we want it to "follow directions" in the privileged text, but not in the unprivileged text, but the problem is that LLMs are not "following directions" now. An LLM doesn't turn your English into some internal model of a command, and then execute the command.