Unlike video, the only thing you really need to watch out for is anybody mentioning anybody's name. Anybody's pertinent personal information. Otherwise they were probably just wondering what it was all about what you were doing? They didn't see any camera guy. And sound in free air is free sound. It is really don't want to pick up personal conversations as you are correct, it's not nice to eavesdrop. Of course in doing so, you might be preventing a crime from happening? How about telling your football team to clobber the other football team's quarterback? LMAO and that came from a documentary crew that they knew was there. So, you can't fix stupid and they deserved what they got. Dumb floppers.You got to love that kind of crap. So, yeah, go eavesdrop in the football locker rooms with your recorder running.

The only time you actually need to have them sign a release for what you are doing is if you involve them, personally, in what you're doing. Of course I'm not a lawyer and for better peace of mind you might want to consult with an attorney? I think you'll find there should be no problems in what you're doing. Especially, if you're not doing it for any money but just personal use and enjoyment.

Did you know so-and-so was cheating on his wife?
Mx. Remy Ann David

If a person can identify himself in the recording then you've got trouble. That would require a model release. Someone looking for money through lawsuits would always be able to "identify" his own voice.

So avoid people having conversations. Try to get an 'overall' conglomeration of sound(s) at a distance.

You can sort that out on your end since you are the one on site doing the recording. Best not to even let people know you are recording, if that is at all possible.

EDIT:

How can they ID themselves? By calling the police - last thing you should do is leave before the police arrive. Then the police take the recording and it becomes evidence for both sides to listen to.

It doesn't directly address your question, but I do have this.

While I was transferring video evidence from a convenience store security system for the local DA, I asked the investigating officer why they don't use the available audio connections. He told me due to privacy laws, you can record a dozen angles of every move the customers (and the occasional crooks) make on video, but not their voices - at least that's the case here in PA.

If you're not selling the audio, I can't image anyone would ever have a beef. But to be safe I believe John T. has the right answer - try to avoid anything where specific conversations/words are discernible and capture the overall chatter. Anything more might require a release/consent form.

There's a reason you never see the faces of the people in the B-roll footage. (I wonder if anyone has ever recognized their own big bottom and sued the network news for broadcasting it with the news story on obesity.)

You could either try to be more discrete or go 180˚ the other way, and walk around with a large sandwich board, "Ambient Audio Recording In-Progress / Be Quiet, or Be Recorded... Your Choice."

While you fellas hit the important legal aspects of vibe53's questions, I think you missed his main concern -
Not looking like a weirdo and giving the impression he is a voyeur (audio version?), or some other unscrupulous fellow.

I'm guessing any decent field recording is going to require a visible microphone, so I'm at a bit of a loss.
Maybe wear a sign that says "Recording non-specific field audio" or "Recording bird calls"?

Thanks every one. You said pretty much what I had been thinking about it. I could end up recording a crime in progress or one being planned. And it's harder to identify yourself on audio than it is in an image. I've been a part time photographer for more than 30 years and come across a lot of this kind of thing. That's mainly why I was asking about it. I was on the top level of a parking structure a few years ago testing some new filters, a place I had done quite a few model shoots. A security lady came up and asked what I was doing. She told me that taking pictures up there wasn't allowed because there was a hospital across the street, in the direction I was shooting. It has to do with 911, certain buildings.
Any way, the first place I recorded yesterday was in an area with soccer fields, a kids playground and a public swimming facility that wasn't open yet. I got out of my car, walked a short distance toward the playground/swimming area and started getting out my gear. I turned around and there was a cop car sitting in the parking lot right at the edge where the park starts. I ignored them and went about my business. I put on the head phones quickly. I was carrying a camera mono-pod to mount the recorder to. My gear bag looks like a camera bag. People think as soon as the see camera-looking gear it must be camera gear. When I cam back out of the park there was a different cop car sitting in the parking lot one row behind and a couple of spaces over from my car facing away from my car. They could see me in their side view mirror. Do they like to take their brakes in that parking lot or what?
One of the main things I wanted to record yesterday was the train going by. I got it quite by accident, in the right place at the right time, I didn't know it's schedule. So when I leave the parking lot of the first park headed to the second to record bird sounds and such I might have to wait on the recording the train.
One of the main things I concerned about is recording near structures, roads and train tracks. I just got my recorder, a Tascam DR-05, and I am experimenting with the built-in mics. So you don't see an actual mic. I want to get a shotgun mic and a parabolic.
Maybe I'm just too paranoid but I got tired of being stared at when I carried my camera so now I'm sensitive to it. Maybe I should get a T-shirt that says "so and so recording company". Or "Quit Please". I could carry on of those signs they used at the golf tournaments.