When you enable minify the system will try to create the missing .min files and save them. By default I added all the .min files so the problem is that your server can't save those files, most likely you are running php without permissions.
Since in some systems there could be issues to write the .min.file the thing will check first if it can write in the target dir so in your case there won be delays (which I insist, is not a delay is your server doing the minify work over and over).