One of the things I find most difficult about Docker is how unintuitive the commands are. You have to remember different ways to install things on linux (yum, apt-get, etc.) and do things like RUN /root/.poetry/... thanks for sharing a working example :)
Nice post. I learned a new way to organise project. I have one question though- what do you suggest for model models experiments and versions?
Developing a model is not just loading data, train, evaluate, and predict. Finalising a model takes a lot of iteration of data preparation, features selection, model selection, tuning etc. With your current suggested setup, what is scope of versioning and experiment tracking?
One of the things I find most difficult about Docker is how unintuitive the commands are. You have to remember different ways to install things on linux (yum, apt-get, etc.) and do things like RUN /root/.poetry/... thanks for sharing a working example :)
Same here.
ChatGPT is very helpful in this sense.
Nice post. I learned a new way to organise project. I have one question though- what do you suggest for model models experiments and versions?
Developing a model is not just loading data, train, evaluate, and predict. Finalising a model takes a lot of iteration of data preparation, features selection, model selection, tuning etc. With your current suggested setup, what is scope of versioning and experiment tracking?
I suggest a serverless experiment tracker and model registry like comet, weights&biases