Whether you are an employee or an employer there are work-related stresses. If you are loving your work the same thing will not come as stress to you, instead you'll treat it as challenge and enjoy facing it. Then work satisfaction will in fact make you happier! That will in turn increase your efficiency and give outstanding performance as a result.
Max time of the day(wakeful part!) is spent at work. So isn't it important to be happy where you spend most of your day? Also we've not gained total dictatorship on our mind. Hence obviously previous part/incidences affect coming hours. If we could fully control our minds we would have definitely commanded 'Once I'm done with my work and office hours, no more of its thoughts'. (Yes I've seen such people too who have complete command over their different spheres of activities and switch on and off easily...but I'm sure that also needs habit-forming, not a gift by birth!).
I've heard many people say that they like the work at office but not the work environment. Ask yourself can you change the people or system around? (I'm sure answer would be NO..at the most you can bring modification in some processes if you are a highly influencial personality) If the answer is no, then why crib about something that you cannot change? But yes you can definitely take some measures to maintain your 'feel good' feeling at work like- keep your interaction with pessimistic colleagues at bare minimum level, focus on work so that you forget the environment factor.
There are already hundreds of things to care about which might also make you worry apart from work. Then why add work related worries to that basket! Don't let your work-worry rob happiness from you. Just imagine that you are not liking your work- You'll be lethargic to go to office everyday with huge amount of inertia (everyday will be a monday!).
If you do not like(rather hate) the work and you know that no job in the same field is going to make you feel good then better do what you like. I know it may not be that easy as it is to say. But yes, at least you have the option of choosing and doing what you like :) If you can neither make yourself love what you do nor do what you love then you are in a quagmire from where only you can rescue yourself!