๐Ÿš‚Data Engine

Notes from Andrej Karpathy talks

Data Engine HLD internally at metaforms.ai

Hypothesis:

  • Unknown unknowns: Dataset is always imperfect, all scenarios are not represented well yet and can always be more diverse

  • Capable base model/architecture: Improving dataset improves AI/product guarantees

Inspirations:

1. Become one with the data 2. Set up the end-to-end training/evaluation skeleton + get dumb baselines 3. Overfit 4. Regularize 5. Tune 6. Squeeze out the juice
from 6th to 15th minute

"The only sure certain way I have seen of making progress on any task is, you curate the dataset that is clean and varied and you grow it and you pay the labeling cost and I know that works.โ€

"Potentially nitpicky but competitive advantage in AI goes not so much to those with data but those with a data engine. And whoever can spin it fastest. Slide from Tesla to ~illustrate but concept is generalโ€

QualEval: Qualitative Evaluation for Model Improvement

Last updated

Was this helpful?