Last week, the decline and recovery of US stocks by DeepSeek significantly affected the entire market. However, the full picture seems to be still in the midst of confusion and chaos regarding development methods, equipment and costs involved, performance, as well as arbitrary responses. Since the last time I wrote about CUDA, I extended it to investment actions based on its position in the AI ecosystem. This time, I will write about what I know, what I think, and my current investment actions as the key points of DeepSeek becoming a hot topic.
What I am particularly interested in due to the widespread recognition of DeepSeek this time are two techniques that DeepSeek uses for AI model development: 'Distillation' and 'PTX'.
First, in AI model development, 'Distillation' refers to the technique of transferring knowledge from a large model (teacher model) to a smaller model (student model). This is generally used to maintain performance close to the original model while reducing computational costs.
Mechanism of Distillation
1. Training of Teacher Model
...