Am I responsible? Car insurance in NJ

I am out of town a few weeks and my boyfriend borrowed my car while I was away. I didn't remember giving him permission besides driving me to the airport and moving it when I thought my neighborhood would be flooding. Fast forward to middle of my trip my neighbors messaged me asking if I'm okay because they saw my car damaged. It looked like a side swipe with large scratches and a few dents but they said their truck which is way bigger was parked behind me on our narrow street which would have been hit first. They noticed this Tuesday morning. Boyfriend was unaware when I called and asked him about this, went to my place to help file a police report and didn't say he saw anything throughout his times he's come to visit my apartment to sleepover or feed my cats. He apparently used it three times taking it about 1/2hr to 1hr away places. I wasn't aware until I saw my Ez pass tolls and confronted him about it. All I have is some hard to see pictures from his security cameras at work on Monday afternoon when he used it last. He's not under my policy as I didn't think he'd be in the car regularly or driving long distances. Am I responsible because I asked him to take me to the airport with it and move my car during the flood? Or is it without consent because I didn't give him permission to take it to all those places? What do I tell insurance and the cops?