I have this issue on my vm and if I remember correctly it was when I used host adapter. You can use bridged mode but you have to set up a virtual adapter with a separate mac for it to work properly.
I would try booting another computer from your server over lan to determine if the problem is with your vm settings or your tftp settings first. That will at least help narrow down which set up is to blame. I have a system on the bench using my server right now, when it's done I'll check it with my vm and see if I can remember how I had to set that up.