How do I let go of the past?

My husband and I will have been married 5 yrs in August. We've dated since 1999. We have a 2 year old son. Our marriage has been very rocky. I've financially supported us during our whole marriage. He hasn't had a job for one reason or another, no health issues. He hasn't helped me clean. He cheated on me once with in the first 6 months of our marriage. about a year or two in I found pics of girls (naked) in is e~mail that he'd been corresponding with. This is when he told me that he loved me, but that I wasn't affectionate enough and that he had to go to other places to find affection. He admitted to cheating once, but said that he stuff online was just online and he'd never met any of them. We almost split up, but agreed to try harder. I had a hard time being affectionate, I felt like he'd done me wrong, plus he wasn't helping around the house or financially. Things continued to get worse. We ended up arguing all the time. eventually he ended up spending -almost all of his time with our single next door neighbor. He swore they were just friends. He said because of my nagging he had to have an outlet. I found inappropriate text between them that he just wrote off as joking. Finally we'd both had enough and agreed to separate last Nov. We stayed in the same house for our son's sake, plus I was trying to let my husband get on his feet before I left. Things went on like this for a couple of months. Then I found out that he'd slept with our neighbor and 2 other girls. (during our separation). I freaked out, I guess because that was the one thing I'd been so worried about during our marriage. During our marriage he was staying at this girls house to 3am at times. She doesn't work either. At that time, I let go completely. I guess I went numb to him and just didn't give a damn what he did after that. So I started dating. Moved out to my sisters house. I became to close to the new guy, I felt like I was in love after only 2 months. He was just so different and so sweet. Anyway, after seeing that I was happy and moving on my husband demanded that we get back together. He said that the'd only reason he'd ever wanted to separate is so that I'd see what I really have and would be more affectionate to him. He kind of went crazy at one point, stalking the other guy, making threats. calling me constantly, even at work. He kept talking about how I was tearing our family apart. I felt so guilty that I gave in and went back. Broke up with the new guy. Now my husband is being perfect. We've been back together a month, he helps me clean, starts a job next week. He's sweet and attentive, stays at home with me. But I feel nothing. I can't stop thinking about the other guy. I feel like my husband's going to end up screwing me over somehow. I know it's possible that he's really changed since seeing that I would really leave. But I can't help but me mad, he had sex with 3 different girls while i was still living with him. Still didn't want to get back together, but let me meet a decent guy and start to care about him, then he goes crazy. The worse thing is that he's still saying that he's going to beat the other guy up. It wasn't his fault! He really had feelings for me too. He's told a mutual friend of ours about it. He was really hurt when I went back to my husband. I can't help but think that I ended up hurting the wrong person. I feel like such a horrible person, my husband is doting on me, and I still don't want to give him affection.