Will it affect your fate to make the mole smaller and lighter by yourself (if you don't go)?

A mole is a reflection of your fate. It is not the cause, but the result.

Compared with fate, moles are like shadows, and fate is shape. Changing this mole will not change your destiny any more than changing the shadow will change your shape.

However, once you change your destiny, this mole will also change, and your appearance will also change. Just like physical changes lead to shadow changes.

So, how to change your destiny and master it?

I suggest you read Four Disciplines of Fan and find the answer from Yuan's story.