Mice infestation in my kitchen, Should i KILL them all? and how?

Lol 1 mouse isn't an infestation, but anyway...Don't kill it. Trap it with the humane trap, then drive it out to a wooded area a few blocks/miles away from your house and let him go.