False insurance claim

I believe a false insurance claim is being made against me and I'm really so incredibly frustrated.

Today I arrived at a local store, parked in their parking lot, and proceeded inside with my husband and 2 friends. After shopping for 25 minutes, we were checking out when a man came to the counter and asked the manager to page the owner of my car. I spoke up and stated it was my car and he claimed we dented his door when we got out of our car. We all looked very confused at each other as we definitely would have noticed or heard if one of us hit a car so hard that it dented.

We proceed outside expecting a damaged door only to see that there was not only no dent, but no visible damage or scraped paint that any of us could notice. He insisted the door was dented and even when asking him to point out the dent was unable to see or feel any damage. I asked why he didn't mention something or stop us when it happened and he gave a vague excuse of "I couldn't leave my kids in the car" despite his wife also being in the vehicle. Another odd detail is that he claims he saw us yet when standing right next to us at checkout did not recognize us or say anything to our group despite making eye contact multiple times.

He was very accusatory, calling us all liars, but neither myself or my other passengers heard or witnessed this damage and I have no reason to believe my friend is lying. After taking many photos of both vehicles including a video proving where the impact point of the door would have been (with no damage or paint chips) we exchanged information and he drove off.

I know this sounds silly but this is the first time anything like this has even happened to me. I spoke to the store and they have no cameras in the parking lot to validate either side. I called my insurance and they said to just hang onto the photos in case an adjuster calls me. I'm so worried about my rates increasing due to this lie. I feel like I'm getting scammed. Should I be worried?