1/4" sized HSS blanks won't take as long to grind to shape compared to 3/8" ones. They won't be quite as strong as 3/8" cutters but I don't think an un-modified mini lathe has enough power to break one, unless you crash the tool into a rapidly spinning chuck. I've used both sizes with no mishap.
The 4 way toolholder that comes with these machines is designed for 5/16" sized cutters so that is something to consider. To use 1/4" cutters in it you will need either a spacer under the toolholder to elevate it, or spacers underneath the individual cutters. A QCTP makes it a lot easier to swap in different-sized cutters and get them on-center.