Your best bet, based on what you've mentioned, would probably be a portable digital recorder... something like this: Zoom

Thanks Donny for your quick answer! the link you gave me explained me a lot actually! if I understood correctly, I could plug straight into the zoom 1 or 2 mics (could be a voice and a guitar, could be 2 voices, could be just 2 mics well positioned among the musicians), and also record sound from the 2 on board mics, with a total of 4 different tracks, right?
now would make sense to import those 4 tracks on a DAW and try to blend them together to get a nice mix?

also, regarding the vocals, if I want to plug the singer mic straight into the zoom, do I still need also an amp for him/her? otherwise how can the other musicians hear the singer? but then if there's an amp for the singer, it will be captured by the on-board mic on the zoom and I won't be able to work with it in post-production... or am I not getting it straight? pls help! :)

I'm not sure if you can use external mics and the internal mics at the same time with the Zoom... ? - I'm not saying you can't, I'm saying I don't know.
The Zoom model I have is older - I think it's the first one that they made - which has two condenser mics built it ( XY 90° and 120° ), and it does have a stereo audio input ( line level), but the one I have doesn't have any connections for external mics.

According to the description, if I'm reading it right, you can use the onboard mics as well as external mics. If this is true, then you could mic the vocalist, and connect it to the recorder, which would give you a stereo mix of everything plus one track for the vocal, which you could then import into any DAW, and tweak EQ, GR, etc., if it needs it. Your best bet is to place the recorder in a position where it sounds best to you, by your ears, and record from that position. It may take some experimenting to find the best location.

Personally, in this scenario, I think you're better off capturing all the players with the built-in stereo mic array, and use one of the other 2 tracks available for the vocalist's mic.
I wouldn't mic and record each of the four players separately, but that's just me. Others here may not agree with me on that.

FWIW

It's a difficult situation in which to record anything other than a memento of how it sounded on the day. You can certainly try a Zoom H4N or H6 using the built-in mics as a stereo pair out front and then use some of the other tracks to take close mic signals from the vocalist and any other quiet instruments. However, what you get out of the resulting recording may well be the vocal track in the foreground along with the sound of the rest of the band playing on a rooftop across the street.

The other big problem is one you touched on in the first post, and that is foldback monitoring, i.e. the muscians all being able to hear what the others are playing without their amplifiers turned up so loud that a natural acoustic balance is impossible to achieve. If you were recording a full band in a single room in a studio, this problem would be minimised by everyone wearing headphones or IEMs and having control of their own headphone mixes, while the sound level in the studio would be kept not much louder than that produced by the acoustic instruments. By way of example, '">here's a top professional studio at work for a TV programme (although the sync on that particular Youtube video is terrible). To operate in this way needs individual miking, so is unfortunately not practical with a recorder of the Zoom type.

Hi Leo,
I think we can all relate to the passion, and I think it's safe to say, most of us that have been recording for very long had very humble (low-budget) beginnings - mostly because high quality equipment was way beyond the average guy's reach until about 20 years ago. You try to figure out the best approach with whatever equipment you have available, record, experiment, learn something, adjust, try again, repeat (for as many years as you are interested in recording).

I think you just need to have realistic expectations. What is your idea of 'decent'? I believe this is one of the many instances in life where the "Cheap/Good/Fast" principle applies. Which (if you are unfamiliar) is to say, there is A) a Cheap way to do it, B) a Good way to do it, and C) a Fast way to do it. - and you can pick any two of them, but you cannot have all three. Click here, if you need something to visualize this well known service-industry principle > There may be a way to get Good results that doesn't cost a lot of money, but it will almost certainly require a lot of time, practice, and experimentation with different techniques before you find something that works. Every once in a while, all the pieces fall into place and a person might catch lightning in a bottle by sheer luck, but it's usually through planning, practice, and perseverance.

Even with a mixer and a couple dozen microphones, that wouldn't necessarily be an easy event to record. We've all heard some terrible live mixes, terrible board recordings, and terrible bootleg recordings along with some great ones. And I've heard studio projects people have worked on for months that don't sound any better than / as good as, something I could have done in a couple hours at a live show.

The video Boswell linked to is a good example of how a professional would do it (except for the noted sync issue that occurred somewhere along the way) in a nice spacious studio. It's intimate, but the (highly professional, studio-caliber) musicians are spread out a little bit. They aren't sitting on top of each other, and aren't fighting to be heard. That isn't always the case at a live show. So if by, "small performance live outdoor, let's say a rooftop of a house", you mean, you want to record a live show where the musicians need to fill a certain amount of space with sound - while setting up a short distance apart… that may require a different approach more suitable for live performance conditions.

There will be a lot of challenges in the set-up as you've described, the outdoors (wind, moisture), a live audience, an assortment of instruments with drastically different levels, some amplified, some not. With such a wide variety of instruments, and one or more vocalists - is there someone else (with a mixer and speakers) responsible for giving the audience a decent mix of all these sound sources?

I think regardless of the answer to that question, I would agree in recommending the Zoom approach. I'd walk around while the group was playing and try to listen for a place where all the instruments and vocals blend together and put the Zoom-type device at that spot, pointing the same direction (at ear-level). You should be able to get what Boswell describes as, "a memento of how it sounded on the day" and that might be all you want. I believe hoping for much more than that is setting yourself up for disappointment. But if your expectations are realistic, a Zoom with its built-in mics should be able to capture the essence of the show, if not the fine detail of the performance. Much of that will depend on the musicians. If they're disciplined, and accustomed to playing their instruments so everyone in the band can hear one another, rather than drowning someone else out, you'll have a much greater chance. If there is a sound person controlling the mix for the crowd, that's usually a good location to start. One would hope the soundman/woman (especially outdoors) has it sounding good at the mix position. But don't forget, putting the recorder out among the spectators can have a down-side too. You can expect to hear not only the band and their fans cheering, but also the audience's uncensored conversations (shouted during songs), and anything they might say between songs. So the band could be having the set of their lives, and these people mindlessly chatting could ruin an otherwise good recording.

Although YouTube is certainly full of terrible sounding point and shoot videos from audience members, some would stand a chance of sounding Fair to Good if people would just learn how to operate their cameras (or Zoom-type recorder). I think the single biggest audio mistake people make when using a video camera (or Zoom-type recorder) is leaving the Mic Level set to 'Auto'. Switch it to 'Manual' and adjust the levels so the loudest portion of the show doesn't peg the meter, it's not that difficult or time-consuming. It's much better to have a low-volume, undistorted recording, than to have a loud (but distorted) recording. You can always make it louder later. You cannot un-distort it, ever. In addition to visually keeping an eye on the levels, taking a pair of headphones can be very helpful to monitor the tone and balance of what the recorder is picking up. You might be surprised to hear some details the microphones can discern that your ears cannot, even in a moderately loud environment.

And anybody who uses their cell phone to shoot video, deserves whatever audio they get. And while we're (I'm) on the subject, remember kids... there's no excuse for vertical video.