🧠Compressing for AGI

Jack Rae

相关解读：https://zhuanlan.zhihu.com/p/619511222

原视频：https://www.youtube.com/watch?v=dO4TPJkeaaU&t=161s

Theme of talk

• Think deeply about the training objective of foundation models

• What are we doing , why dose it make sense, and what are the limitations?

Takeaways

• seek the minimum description length to solve perception

• Generative models are lossless compressors

• Large language models are state-of-the-art lossless text compressors(?!)

• Current limitation of thr apporach

Minimum Description Length

...and why it relates to perception

We want to deepest understanding of our observations

ones that generalize