Landscaping encroachment

I bought my house a few months ago. Long story short, my neighbor landscaped my side of the front yard as soon as my house went up for sale without the owners at the time permission but they were so tired of him that they didn’t say anything & expected next owner (me) deal with it. He landscaped it with decorative rocks, metal edging, laid soil and sod, and added lights which made it seem like it was all just 1 lawn extending from his house. He acted friendly and like he did us a favor by landscaping it and he would water my side all the time from standing in my driveway. I was afraid of adverse possession boundary acquiescence so I told him pls dont do that but he insisted and never listened. I got the property re-surveyed because the way he laid out the landscaping confused me as to my property line.

After being told by neighbors that he put his business sign on our shared lawn, that was the final straw for me and I politely returned his decorative rocks and edging he put on my side. I ripped out the sod he laid down and bought chop block which you see in the first photo to separate the yards and I also planted natives. Mind you, we live in an HOA and fences to the street are not allowed so this was the only solution for a visible boundary.

He seemed pissed. Hes weird. He stared at me and my landscaper as we worked. Anyway, fast forward today, he is in my side yard. He built a small retaining garden on the property line before I moved in so there’s absolutely no reason for him to be on my side to trim his trees or do any of his landscaping work. Well he did. And what you see in the 2nd photo is that he moved his “garden” more inward but did it all from my side and left all his crap on my side. He was also congregating with my neighbor to the right of me in my yard as you will see in the photo. They were talking about my plants. I was pissed because they could possibly damage my plants. I was watching on the ring camera to see if they came close to it.

So I raked all his garbage to his side and put red flags all along the property line. Then he came back out with his wife cleaned it up and they started going across the street and looking at his house from the front and side angles. I dont really know why he is doing all this but I cant build a fence and I only have 5.6 feet to the left of my house. How can I create privacy ? I cant plant evergreens because they would be too close to the foundation right? Is there some inexpensive tall trees I can line up in pots on my property line?