The issue was that when I go to File>Import, I wasn't able to tell which files to import and I didn't want to import all 350. I found something called HD Quick Look by Shedworx. I'm not sure if this is what solved the problem with viewing the thumbnails, but now when I go to import, I can see a thumbnail (although I cannot preview it like you normally would be able to with mp4, in my experience). So that was actually good enough for me so I at least knew which videos I was importing and could just do a small batch at a time. And yes, Premiere allows you to preview the video very easily before adding it to the composition which is great!
I was going to test it by uninstalling the HD Quick Look to see if I could still see the thumbnails in the import window, but cannot find anything on my computer other than the .pkg file for installation. Either way, Premiere was definitely the way to go for this initially step however if I do end up importing it into After Effects, I worry about having the same problem as I did initially... it being very laggy.
I will update if I have issues. Thank you so much for your help.