No, you do not have to quit your job to travel if you do not want to. You can find ways to incorporate travel into your life, as it currently is. The summary is, take advantage of long weekends, travel on weekends to nearby places, negotiate more vacation days during job offers, if possible. Here are some other ways.
Bonus: Is travel my full-time job? No, it isn’t. I work a 9-5 in a global technology company.