World's Best AI Learning Platform with profoundly Demanding Certification Programs
Designed by IITians, only for AI Learners.
Designed by IITians, only for AI Learners.
New to InsideAIML? Create an account
Employer? Create an account
By reducing the size of your dataset, or increasing the ram on the google colab
There are several ways to increase the speed of loading large datasets in Google Colab:
gcsfs
library to access the dataset directly, rather than downloading it to the local runtime.If your problem truly is the network speed between Collab and Drive, you should try uploading the files directly to the Google Collab instance, rather than accessing them from Drive.
Doing this will save the files directly to your Collab instance, allowing your code to access the files locally. However, I'd suspect that there might be other problems besides the network latency – perhaps your model has lots of parameters, or somehow there was a bug in the code to get CUDA going. Sometimes I would forget to change my runtime to a GPU runtime under the "Runtime" menu tab, "Change Runtime Type".
Hope this helps!