Mechanistic Interpretability Of LLM for Code Generation
LLMs inherently learn both vulnerable coding patterns along with safe coding patterns. This is based on the vast and unfiltered training data the LLM was trained on. During code generation, they may either take the safe path or the vulnerable path.
Can we look inside the model using mechanistic interpretability to see what causes the model to take the vulnerable path?
Furthermore, how can we steer or modify the model internals so that it follows the safe path instead of the vulnerable one?