As long as you have taken the photo / filmed a video in a public place, you should be all good. When a celebrity walks out of a restaurant paparazzi / photographers rush towards them taking photos and videos. Do you think they ask for permission to do it? No – they almost never do.
Think about it: there are security cameras everywhere and cars like Tesla have cameras built in designed to record everything happening around the car.
Now, it is important to note that you might get yourself in some trouble if you are taking photos / video of people in some uncomfortable environment. For instance, you might not want to take pictures of people on the beach.