First of all your feeling guilty because you have not forgiven him, its very necessary to forgive someone as it will open the doors of blessings for you.

 Its hard to forgive but its must.

Unforgiveness starts in seed form and grows into bitterness, which begins to affect all of our relationships.

Holding on to an offense is a plan from the enemy to ultimately isolate us from others and God.

Thats actually a punishment what your giving to yourself

When you don’t forgive You enter in a zone of isolation,  worthlessness, and negative thoughts.. and that will eventually effect your walk with God

When we choose to forgive, we free ourselves from the bondage and open up the door for God to heal us and to act on our behalf.
Scriptures tells us “But I say to you, love your enemies, bless those who curse you, do good to those who hate you, and pray for those who spitefully use you and persecute you, Matthew 5:44 NKJV”

The easiest way I have found to release someone who repeatedly offends me is to pray for them. 

Bear with each other and forgive one another if any of you has a grievance against someone. Forgive as the Lord forgave you.-Colossians 3:13

Forgive them And forget what they have done . And don’t let negativity, or revenge come in your life remember  “ ‘Do not seek revenge or bear a grudge against anyone among your people, but love your neighbor as yourself. I am the LORD.”Leviticus 19:18