Welcome to the forum Pesi. I would think that the tip of the collet nut should touch a table top if you sat it down on it fully extended or at least very close. Maybe the problem is that you have way more shank through the collet than you need. You only need 1" of shank inserted into the collet, more doesn't help hold it. Mark an inch on your bit so you know how much is in the collet and see if that helps.
Someone I consider a master woodworker once told me that a master woodworker is not someone who never makes mistakes. He is someone who is able to cover them up so that no one can tell.